成品模型效果展示
什么时候需要这类模型?
1希望对出图内容的某一内容进行修改而不过多的改变其他内容时,推荐使用该类模型。
pony contrast adjuster
https://civitai.com/models/536749/pony-contrast-adjuster
Illustrious lineart adjuster
https://civitai.com/models/1009332/illustrious-lineart-adjuster
Illustrious Auto Nude
https://civitai.com/models/1043351/illustrious-auto-nude
训练方法数据集准备合适数量、样式分布较广且平均(单一类型的反面)的图片,需要打标。数量根据自己的算力和修改图片的耗费决定,条件允许的情况下越多越准确。我这里以通过HakuBooru收集到的各种评级的图片为例,每种评级16张,共64张图片:
训练原始底模以这些图片作为原始图片,正常训练出一个模型作为一个原始底模,以下是在lora-scripts中的训练参数参考,将模型训练至拟合即可:
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647model_train_type = "sdxl-lora"pretrained_model_name_or_path = "D:/hmkai/ComfyUI/models/checkpoints/llustrious-XL-v0.1.safetensors"train_data_dir = "D:/data/差异/原始图片64"prior_loss_weight = 1resolution = "1024,1024"enable_bucket = truemin_bucket_reso = 256max_bucket_reso = 2048bucket_reso_steps = 32bucket_no_upscale = trueoutput_name = "IL-64-MKI"output_dir = "./output"save_model_as = "safetensors"save_precision = "fp16"save_every_n_epochs = 5max_train_epochs = 50train_batch_size = 4gradient_checkpointing = truegradient_accumulation_steps = 16network_train_unet_only = truenetwork_train_text_encoder_only = falselearning_rate = 0.0001unet_lr = 0.0001text_encoder_lr = 0.00001lr_scheduler = "cosine_with_restarts"lr_warmup_steps = 0lr_scheduler_num_cycles = 1optimizer_type = "Lion8bit"network_module = "networks.lora"network_dim = 1 # 通常dim越大对画面修改越大network_alpha = 1log_with = "tensorboard"logging_dir = "./logs"caption_extension = ".txt"shuffle_caption = falsekeep_tokens = 0max_token_length = 255seed = 2104mixed_precision = "bf16"fp8_base = falsesdpa = truelowram = falsecache_latents = truecache_latents_to_disk = falsecache_text_encoder_outputs = falsecache_text_encoder_outputs_to_disk = falsepersistent_data_loader_workers = true
制作差异图片接着我们要做差异,也就是对这些图片进行一些修改
常见的修改比如一些 线稿加深,对比度加强,面部美颜,胸部大小,星星眼等等…总之如果你想改变单一内容而且尽量保证其他内容不变时,推荐采用差异炼丹
根据你希望的“差异”,为每一张图片都制作一份差异后的图片,图片的标签不变
我这里以对比度为例(其实我很想做胸部调节和星星眼之类的局部差异丹,但是我的PS水平并不好),修改后的数据集:
训练差异模型以修改后的图片作为训练集,在lora-scripts中,对于上方原始的训练参数,你只需要且最好仅改变以下内容:
1234train_data_dir = "D:/data/差异/原始图片64" # 填写你修改后的训练集output_name = "IL-lineart-MKI" # 改一下输出模型的名称base_weights = [ "D:\\hmkai\\lora-scripts\\output\\IL-64-MKI.safetensors" ] # 填写你上面炼制的LoRA模型路径base_weights_multiplier = [ 1 ]
其中base_weights的作用是将原先炼制的LoRA融合到底模,假如你的脚本中没有这一项,请使用一些方法先融合LoRA模型至底模中,炼制差异模型时将原先的底模替换为融合后的模型
等待训练完成后即可获得一个只改变某单一内容的LoRA啦☆(<ゝω・)~Kira☆~