この記事は以下の記事と比較して読んでいただくとより分かりやすいかと思います。
拡張機能comfyui-animatediffによるノード図
まず512×512の出来上がり図はこちらになります。
インストール
Managerからインストールできるcomfyui-animatediffをGUIからインストールします。
モデルmm_sd_v15_v2.ckptはStable Diffusion WebUIのものを転用しました。
ダウンロードした後ComfyUI>custom_nodes>comfyui-animatediff>modelsに入れます。
それ以外のmm_sd_v14.ckpt、mm_sd_v15.ckptやmm-Stablized_mid.pthは私の環境では上手く動きませんでした。
基本最小構成のノード図解説
以下デフォの構成から作ることを仮定します。
1,まずKSamplerをAnimate Diff Samplerと交換して接続します。
2,次にSave ImageをAnimate Diff Combineと交換して同様に接続します。
3, Animate Diff Module Loaderの右端motion_moduleとAnimate Diff Samplerの左端MOTION_MODULEの緑点を結びます。
その時のログを参考のために載せておきます。
Starting server
To see the GUI go to: http://127.0.0.1:8188
FETCH DATA from: /Users/***/ComfyUI/custom_nodes/ComfyUI-Manager/extension-node-map.json
got prompt
[AnimateDiff] - INFO - Loading motion module mm_sd_v15_v2.ckpt
[AnimateDiff] - INFO - Converting motion module to fp16.
model_type EPS
adm 0
Using split attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using split attention in VAE
missing {'cond_stage_model.text_projection', 'cond_stage_model.logit_scale'}
left over keys: dict_keys(['cond_stage_model.transformer.text_model.embeddings.position_ids', 'model_ema.decay', 'model_ema.num_updates'])
Requested to load SD1ClipModel
Loading 1 new model
[AnimateDiff] - INFO - Injecting motion module with method default.
Requested to load BaseModel
Loading 1 new model
DDIM Sampler: 100%|█████████████████████████████| 25/25 [03:33<00:00, 8.54s/it]
/Users/***/ComfyUI/comfy/model_base.py:49: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
self.register_buffer('betas', torch.tensor(betas, dtype=torch.float32))
/Users/***/ComfyUI/comfy/model_base.py:50: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
self.register_buffer('alphas_cumprod', torch.tensor(alphas_cumprod, dtype=torch.float32))
[AnimateDiff] - INFO - Ejecting motion module with method default.
Prompt executed in 222.11 seconds
注意点
モデルはmm_sd_v15.ckptはruntimeエラーで動かず、mm_sd_v15_v2.ckptを使ったら動きました。
個人の感想として、Stable Diffusion WebUIの時は動作の仕組みが分かり辛かったのがこの図を見るとブラックボックスを開けたみたいにプログラミング動作の仕組みが少し謎が解けてきたようにも感じました。
LoRA
生成AI出力結果
使用モデルは
Checkpoint:beautifulRealistic_brav5.safetensors
LoRA:JapaneseDollLikeness_v15.safetensors
になります。
Stable Diffusion WebUIの使い回しモデルで大丈夫です。
ノード図
拡張機能のLoRAノードは動作しなかったので、デフォでついているLoad LoRAを使いました。
この結果がベストとは思いませんが可能性という具体例として共有させていただきます。
ログ
got prompt
model_type EPS
adm 0
Using split attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using split attention in VAE
missing {'cond_stage_model.text_projection', 'cond_stage_model.logit_scale'}
left over keys: dict_keys(['cond_stage_model.transformer.text_model.embeddings.position_ids', 'model_ema.input_blocks00bias', 'model_ema.input_blocks00weight', 'model_ema.input_blocks100emb_layers1bias', 'model_ema.input_blocks100emb_layers1weight', 'model_ema.input_blocks100in_layers0bias', 'model_ema.input_blocks100in_layers0weight', 'model_ema.input_blocks100in_layers2bias', 'model_ema.input_blocks100in_layers2weight', 'model_ema.input_blocks100out_layers0bias', 'model_ema.input_blocks100out_layers0weight', 'model_ema.input_blocks100out_layers3bias', 'model_ema.input_blocks100out_layers3weight', 'model_ema.input_blocks10emb_layers1bias', 'model_ema.input_blocks10emb_layers1weight', 'model_ema.input_blocks10in_layers0bias', 'model_ema.input_blocks10in_layers0weight', 'model_ema.input_blocks10in_layers2bias', 'model_ema.input_blocks10in_layers2weight', 'model_ema.input_blocks10out_layers0bias', 'model_ema.input_blocks10out_layers0weight', 'model_ema.input_blocks10out_layers3bias', 'model_ema.input_blocks10out_layers3weight', 'model_ema.input_blocks110emb_layers1bias', 'model_ema.input_blocks110emb_layers1weight', 'model_ema.input_blocks110in_layers0bias', 'model_ema.input_blocks110in_layers0weight', 'model_ema.input_blocks110in_layers2bias', 'model_ema.input_blocks110in_layers2weight', 'model_ema.input_blocks110out_layers0bias', 'model_ema.input_blocks110out_layers0weight', 'model_ema.input_blocks110out_layers3bias', 'model_ema.input_blocks110out_layers3weight', 'model_ema.input_blocks11normbias', 'model_ema.input_blocks11normweight', 'model_ema.input_blocks11proj_inbias', 'model_ema.input_blocks11proj_inweight', 'model_ema.input_blocks11proj_outbias', 'model_ema.input_blocks11proj_outweight', 'model_ema.input_blocks11transformer_blocks0attn1to_kweight', 'model_ema.input_blocks11transformer_blocks0attn1to_out0bias', 'model_ema.input_blocks11transformer_blocks0attn1to_out0weight', 'model_ema.input_blocks11transformer_blocks0attn1to_qweight', 'model_ema.input_blocks11transformer_blocks0attn1to_vweight', 'model_ema.input_blocks11transformer_blocks0attn2to_kweight', 'model_ema.input_blocks11transformer_blocks0attn2to_out0bias', 'model_ema.input_blocks11transformer_blocks0attn2to_out0weight', 'model_ema.input_blocks11transformer_blocks0attn2to_qweight', 'model_ema.input_blocks11transformer_blocks0attn2to_vweight', 'model_ema.input_blocks11transformer_blocks0ffnet0projbias', 'model_ema.input_blocks11transformer_blocks0ffnet0projweight', 'model_ema.input_blocks11transformer_blocks0ffnet2bias', 'model_ema.input_blocks11transformer_blocks0ffnet2weight', 'model_ema.input_blocks11transformer_blocks0norm1bias', 'model_ema.input_blocks11transformer_blocks0norm1weight', 'model_ema.input_blocks11transformer_blocks0norm2bias', 'model_ema.input_blocks11transformer_blocks0norm2weight', 'model_ema.input_blocks11transformer_blocks0norm3bias', 'model_ema.input_blocks11transformer_blocks0norm3weight', 'model_ema.input_blocks20emb_layers1bias', 'model_ema.input_blocks20emb_layers1weight', 'model_ema.input_blocks20in_layers0bias', 'model_ema.input_blocks20in_layers0weight', 'model_ema.input_blocks20in_layers2bias', 'model_ema.input_blocks20in_layers2weight', 'model_ema.input_blocks20out_layers0bias', 'model_ema.input_blocks20out_layers0weight', 'model_ema.input_blocks20out_layers3bias', 'model_ema.input_blocks20out_layers3weight', 'model_ema.input_blocks21normbias', 'model_ema.input_blocks21normweight', 'model_ema.input_blocks21proj_inbias', 'model_ema.input_blocks21proj_inweight', 'model_ema.input_blocks21proj_outbias', 'model_ema.input_blocks21proj_outweight', 'model_ema.input_blocks21transformer_blocks0attn1to_kweight', 'model_ema.input_blocks21transformer_blocks0attn1to_out0bias', 'model_ema.input_blocks21transformer_blocks0attn1to_out0weight', 'model_ema.input_blocks21transformer_blocks0attn1to_qweight', 'model_ema.input_blocks21transformer_blocks0attn1to_vweight', 'model_ema.input_blocks21transformer_blocks0attn2to_kweight', 'model_ema.input_blocks21transformer_blocks0attn2to_out0bias', 'model_ema.input_blocks21transformer_blocks0attn2to_out0weight', 'model_ema.input_blocks21transformer_blocks0attn2to_qweight', 'model_ema.input_blocks21transformer_blocks0attn2to_vweight', 'model_ema.input_blocks21transformer_blocks0ffnet0projbias', 'model_ema.input_blocks21transformer_blocks0ffnet0projweight', 'model_ema.input_blocks21transformer_blocks0ffnet2bias', 'model_ema.input_blocks21transformer_blocks0ffnet2weight', 'model_ema.input_blocks21transformer_blocks0norm1bias', 'model_ema.input_blocks21transformer_blocks0norm1weight', 'model_ema.input_blocks21transformer_blocks0norm2bias', 'model_ema.input_blocks21transformer_blocks0norm2weight', 'model_ema.input_blocks21transformer_blocks0norm3bias', 'model_ema.input_blocks21transformer_blocks0norm3weight', 'model_ema.input_blocks30opbias', 'model_ema.input_blocks30opweight', 'model_ema.input_blocks40emb_layers1bias', 'model_ema.input_blocks40emb_layers1weight', 'model_ema.input_blocks40in_layers0bias', 'model_ema.input_blocks40in_layers0weight', 'model_ema.input_blocks40in_layers2bias', 'model_ema.input_blocks40in_layers2weight', 'model_ema.input_blocks40out_layers0bias', 'model_ema.input_blocks40out_layers0weight', 'model_ema.input_blocks40out_layers3bias', 'model_ema.input_blocks40out_layers3weight', 'model_ema.input_blocks40skip_connectionbias', 'model_ema.input_blocks40skip_connectionweight', 'model_ema.input_blocks41normbias', 'model_ema.input_blocks41normweight', 'model_ema.input_blocks41proj_inbias', 'model_ema.input_blocks41proj_inweight', 'model_ema.input_blocks41proj_outbias', 'model_ema.input_blocks41proj_outweight', 'model_ema.input_blocks41transformer_blocks0attn1to_kweight', 'model_ema.input_blocks41transformer_blocks0attn1to_out0bias', 'model_ema.input_blocks41transformer_blocks0attn1to_out0weight', 'model_ema.input_blocks41transformer_blocks0attn1to_qweight', 'model_ema.input_blocks41transformer_blocks0attn1to_vweight', 'model_ema.input_blocks41transformer_blocks0attn2to_kweight', 'model_ema.input_blocks41transformer_blocks0attn2to_out0bias', 'model_ema.input_blocks41transformer_blocks0attn2to_out0weight', 'model_ema.input_blocks41transformer_blocks0attn2to_qweight', 'model_ema.input_blocks41transformer_blocks0attn2to_vweight', 'model_ema.input_blocks41transformer_blocks0ffnet0projbias', 'model_ema.input_blocks41transformer_blocks0ffnet0projweight', 'model_ema.input_blocks41transformer_blocks0ffnet2bias', 'model_ema.input_blocks41transformer_blocks0ffnet2weight', 'model_ema.input_blocks41transformer_blocks0norm1bias', 'model_ema.input_blocks41transformer_blocks0norm1weight', 'model_ema.input_blocks41transformer_blocks0norm2bias', 'model_ema.input_blocks41transformer_blocks0norm2weight', 'model_ema.input_blocks41transformer_blocks0norm3bias', 'model_ema.input_blocks41transformer_blocks0norm3weight', 'model_ema.input_blocks50emb_layers1bias', 'model_ema.input_blocks50emb_layers1weight', 'model_ema.input_blocks50in_layers0bias', 'model_ema.input_blocks50in_layers0weight', 'model_ema.input_blocks50in_layers2bias', 'model_ema.input_blocks50in_layers2weight', 'model_ema.input_blocks50out_layers0bias', 'model_ema.input_blocks50out_layers0weight', 'model_ema.input_blocks50out_layers3bias', 'model_ema.input_blocks50out_layers3weight', 'model_ema.input_blocks51normbias', 'model_ema.input_blocks51normweight', 'model_ema.input_blocks51proj_inbias', 'model_ema.input_blocks51proj_inweight', 'model_ema.input_blocks51proj_outbias', 'model_ema.input_blocks51proj_outweight', 'model_ema.input_blocks51transformer_blocks0attn1to_kweight', 'model_ema.input_blocks51transformer_blocks0attn1to_out0bias', 'model_ema.input_blocks51transformer_blocks0attn1to_out0weight', 'model_ema.input_blocks51transformer_blocks0attn1to_qweight', 'model_ema.input_blocks51transformer_blocks0attn1to_vweight', 'model_ema.input_blocks51transformer_blocks0attn2to_kweight', 'model_ema.input_blocks51transformer_blocks0attn2to_out0bias', 'model_ema.input_blocks51transformer_blocks0attn2to_out0weight', 'model_ema.input_blocks51transformer_blocks0attn2to_qweight', 'model_ema.input_blocks51transformer_blocks0attn2to_vweight', 'model_ema.input_blocks51transformer_blocks0ffnet0projbias', 'model_ema.input_blocks51transformer_blocks0ffnet0projweight', 'model_ema.input_blocks51transformer_blocks0ffnet2bias', 'model_ema.input_blocks51transformer_blocks0ffnet2weight', 'model_ema.input_blocks51transformer_blocks0norm1bias', 'model_ema.input_blocks51transformer_blocks0norm1weight', 'model_ema.input_blocks51transformer_blocks0norm2bias', 'model_ema.input_blocks51transformer_blocks0norm2weight', 'model_ema.input_blocks51transformer_blocks0norm3bias', 'model_ema.input_blocks51transformer_blocks0norm3weight', 'model_ema.input_blocks60opbias', 'model_ema.input_blocks60opweight', 'model_ema.input_blocks70emb_layers1bias', 'model_ema.input_blocks70emb_layers1weight', 'model_ema.input_blocks70in_layers0bias', 'model_ema.input_blocks70in_layers0weight', 'model_ema.input_blocks70in_layers2bias', 'model_ema.input_blocks70in_layers2weight', 'model_ema.input_blocks70out_layers0bias', 'model_ema.input_blocks70out_layers0weight', 'model_ema.input_blocks70out_layers3bias', 'model_ema.input_blocks70out_layers3weight', 'model_ema.input_blocks70skip_connectionbias', 'model_ema.input_blocks70skip_connectionweight', 'model_ema.input_blocks71normbias', 'model_ema.input_blocks71normweight', 'model_ema.input_blocks71proj_inbias', 'model_ema.input_blocks71proj_inweight', 'model_ema.input_blocks71proj_outbias', 'model_ema.input_blocks71proj_outweight', 'model_ema.input_blocks71transformer_blocks0attn1to_kweight', 'model_ema.input_blocks71transformer_blocks0attn1to_out0bias', 'model_ema.input_blocks71transformer_blocks0attn1to_out0weight', 'model_ema.input_blocks71transformer_blocks0attn1to_qweight', 'model_ema.input_blocks71transformer_blocks0attn1to_vweight', 'model_ema.input_blocks71transformer_blocks0attn2to_kweight', 'model_ema.input_blocks71transformer_blocks0attn2to_out0bias', 'model_ema.input_blocks71transformer_blocks0attn2to_out0weight', 'model_ema.input_blocks71transformer_blocks0attn2to_qweight', 'model_ema.input_blocks71transformer_blocks0attn2to_vweight', 'model_ema.input_blocks71transformer_blocks0ffnet0projbias', 'model_ema.input_blocks71transformer_blocks0ffnet0projweight', 'model_ema.input_blocks71transformer_blocks0ffnet2bias', 'model_ema.input_blocks71transformer_blocks0ffnet2weight', 'model_ema.input_blocks71transformer_blocks0norm1bias', 'model_ema.input_blocks71transformer_blocks0norm1weight', 'model_ema.input_blocks71transformer_blocks0norm2bias', 'model_ema.input_blocks71transformer_blocks0norm2weight', 'model_ema.input_blocks71transformer_blocks0norm3bias', 'model_ema.input_blocks71transformer_blocks0norm3weight', 'model_ema.input_blocks80emb_layers1bias', 'model_ema.input_blocks80emb_layers1weight', 'model_ema.input_blocks80in_layers0bias', 'model_ema.input_blocks80in_layers0weight', 'model_ema.input_blocks80in_layers2bias', 'model_ema.input_blocks80in_layers2weight', 'model_ema.input_blocks80out_layers0bias', 'model_ema.input_blocks80out_layers0weight', 'model_ema.input_blocks80out_layers3bias', 'model_ema.input_blocks80out_layers3weight', 'model_ema.input_blocks81normbias', 'model_ema.input_blocks81normweight', 'model_ema.input_blocks81proj_inbias', 'model_ema.input_blocks81proj_inweight', 'model_ema.input_blocks81proj_outbias', 'model_ema.input_blocks81proj_outweight', 'model_ema.input_blocks81transformer_blocks0attn1to_kweight', 'model_ema.input_blocks81transformer_blocks0attn1to_out0bias', 'model_ema.input_blocks81transformer_blocks0attn1to_out0weight', 'model_ema.input_blocks81transformer_blocks0attn1to_qweight', 'model_ema.input_blocks81transformer_blocks0attn1to_vweight', 'model_ema.input_blocks81transformer_blocks0attn2to_kweight', 'model_ema.input_blocks81transformer_blocks0attn2to_out0bias', 'model_ema.input_blocks81transformer_blocks0attn2to_out0weight', 'model_ema.input_blocks81transformer_blocks0attn2to_qweight', 'model_ema.input_blocks81transformer_blocks0attn2to_vweight', 'model_ema.input_blocks81transformer_blocks0ffnet0projbias', 'model_ema.input_blocks81transformer_blocks0ffnet0projweight', 'model_ema.input_blocks81transformer_blocks0ffnet2bias', 'model_ema.input_blocks81transformer_blocks0ffnet2weight', 'model_ema.input_blocks81transformer_blocks0norm1bias', 'model_ema.input_blocks81transformer_blocks0norm1weight', 'model_ema.input_blocks81transformer_blocks0norm2bias', 'model_ema.input_blocks81transformer_blocks0norm2weight', 'model_ema.input_blocks81transformer_blocks0norm3bias', 'model_ema.input_blocks81transformer_blocks0norm3weight', 'model_ema.input_blocks90opbias', 'model_ema.input_blocks90opweight', 'model_ema.middle_block0emb_layers1bias', 'model_ema.middle_block0emb_layers1weight', 'model_ema.middle_block0in_layers0bias', 'model_ema.middle_block0in_layers0weight', 'model_ema.middle_block0in_layers2bias', 'model_ema.middle_block0in_layers2weight', 'model_ema.middle_block0out_layers0bias', 'model_ema.middle_block0out_layers0weight', 'model_ema.middle_block0out_layers3bias', 'model_ema.middle_block0out_layers3weight', 'model_ema.middle_block1normbias', 'model_ema.middle_block1normweight', 'model_ema.middle_block1proj_inbias', 'model_ema.middle_block1proj_inweight', 'model_ema.middle_block1proj_outbias', 'model_ema.middle_block1proj_outweight', 'model_ema.middle_block1transformer_blocks0attn1to_kweight', 'model_ema.middle_block1transformer_blocks0attn1to_out0bias', 'model_ema.middle_block1transformer_blocks0attn1to_out0weight', 'model_ema.middle_block1transformer_blocks0attn1to_qweight', 'model_ema.middle_block1transformer_blocks0attn1to_vweight', 'model_ema.middle_block1transformer_blocks0attn2to_kweight', 'model_ema.middle_block1transformer_blocks0attn2to_out0bias', 'model_ema.middle_block1transformer_blocks0attn2to_out0weight', 'model_ema.middle_block1transformer_blocks0attn2to_qweight', 'model_ema.middle_block1transformer_blocks0attn2to_vweight', 'model_ema.middle_block1transformer_blocks0ffnet0projbias', 'model_ema.middle_block1transformer_blocks0ffnet0projweight', 'model_ema.middle_block1transformer_blocks0ffnet2bias', 'model_ema.middle_block1transformer_blocks0ffnet2weight', 'model_ema.middle_block1transformer_blocks0norm1bias', 'model_ema.middle_block1transformer_blocks0norm1weight', 'model_ema.middle_block1transformer_blocks0norm2bias', 'model_ema.middle_block1transformer_blocks0norm2weight', 'model_ema.middle_block1transformer_blocks0norm3bias', 'model_ema.middle_block1transformer_blocks0norm3weight', 'model_ema.middle_block2emb_layers1bias', 'model_ema.middle_block2emb_layers1weight', 'model_ema.middle_block2in_layers0bias', 'model_ema.middle_block2in_layers0weight', 'model_ema.middle_block2in_layers2bias', 'model_ema.middle_block2in_layers2weight', 'model_ema.middle_block2out_layers0bias', 'model_ema.middle_block2out_layers0weight', 'model_ema.middle_block2out_layers3bias', 'model_ema.middle_block2out_layers3weight', 'model_ema.out0bias', 'model_ema.out0weight', 'model_ema.out2bias', 'model_ema.out2weight', 'model_ema.output_blocks00emb_layers1bias', 'model_ema.output_blocks00emb_layers1weight', 'model_ema.output_blocks00in_layers0bias', 'model_ema.output_blocks00in_layers0weight', 'model_ema.output_blocks00in_layers2bias', 'model_ema.output_blocks00in_layers2weight', 'model_ema.output_blocks00out_layers0bias', 'model_ema.output_blocks00out_layers0weight', 'model_ema.output_blocks00out_layers3bias', 'model_ema.output_blocks00out_layers3weight', 'model_ema.output_blocks00skip_connectionbias', 'model_ema.output_blocks00skip_connectionweight', 'model_ema.output_blocks100emb_layers1bias', 'model_ema.output_blocks100emb_layers1weight', 'model_ema.output_blocks100in_layers0bias', 'model_ema.output_blocks100in_layers0weight', 'model_ema.output_blocks100in_layers2bias', 'model_ema.output_blocks100in_layers2weight', 'model_ema.output_blocks100out_layers0bias', 'model_ema.output_blocks100out_layers0weight', 'model_ema.output_blocks100out_layers3bias', 'model_ema.output_blocks100out_layers3weight', 'model_ema.output_blocks100skip_connectionbias', 'model_ema.output_blocks100skip_connectionweight', 'model_ema.output_blocks101normbias', 'model_ema.output_blocks101normweight', 'model_ema.output_blocks101proj_inbias', 'model_ema.output_blocks101proj_inweight', 'model_ema.output_blocks101proj_outbias', 'model_ema.output_blocks101proj_outweight', 'model_ema.output_blocks101transformer_blocks0attn1to_kweight', 'model_ema.output_blocks101transformer_blocks0attn1to_out0bias', 'model_ema.output_blocks101transformer_blocks0attn1to_out0weight', 'model_ema.output_blocks101transformer_blocks0attn1to_qweight', 'model_ema.output_blocks101transformer_blocks0attn1to_vweight', 'model_ema.output_blocks101transformer_blocks0attn2to_kweight', 'model_ema.output_blocks101transformer_blocks0attn2to_out0bias', 'model_ema.output_blocks101transformer_blocks0attn2to_out0weight', 'model_ema.output_blocks101transformer_blocks0attn2to_qweight', 'model_ema.output_blocks101transformer_blocks0attn2to_vweight', 'model_ema.output_blocks101transformer_blocks0ffnet0projbias', 'model_ema.output_blocks101transformer_blocks0ffnet0projweight', 'model_ema.output_blocks101transformer_blocks0ffnet2bias', 'model_ema.output_blocks101transformer_blocks0ffnet2weight', 'model_ema.output_blocks101transformer_blocks0norm1bias', 'model_ema.output_blocks101transformer_blocks0norm1weight', 'model_ema.output_blocks101transformer_blocks0norm2bias', 'model_ema.output_blocks101transformer_blocks0norm2weight', 'model_ema.output_blocks101transformer_blocks0norm3bias', 'model_ema.output_blocks101transformer_blocks0norm3weight', 'model_ema.output_blocks10emb_layers1bias', 'model_ema.output_blocks10emb_layers1weight', 'model_ema.output_blocks10in_layers0bias', 'model_ema.output_blocks10in_layers0weight', 'model_ema.output_blocks10in_layers2bias', 'model_ema.output_blocks10in_layers2weight', 'model_ema.output_blocks10out_layers0bias', 'model_ema.output_blocks10out_layers0weight', 'model_ema.output_blocks10out_layers3bias', 'model_ema.output_blocks10out_layers3weight', 'model_ema.output_blocks10skip_connectionbias', 'model_ema.output_blocks10skip_connectionweight', 'model_ema.output_blocks110emb_layers1bias', 'model_ema.output_blocks110emb_layers1weight', 'model_ema.output_blocks110in_layers0bias', 'model_ema.output_blocks110in_layers0weight', 'model_ema.output_blocks110in_layers2bias', 'model_ema.output_blocks110in_layers2weight', 'model_ema.output_blocks110out_layers0bias', 'model_ema.output_blocks110out_layers0weight', 'model_ema.output_blocks110out_layers3bias', 'model_ema.output_blocks110out_layers3weight', 'model_ema.output_blocks110skip_connectionbias', 'model_ema.output_blocks110skip_connectionweight', 'model_ema.output_blocks111normbias', 'model_ema.output_blocks111normweight', 'model_ema.output_blocks111proj_inbias', 'model_ema.output_blocks111proj_inweight', 'model_ema.output_blocks111proj_outbias', 'model_ema.output_blocks111proj_outweight', 'model_ema.output_blocks111transformer_blocks0attn1to_kweight', 'model_ema.output_blocks111transformer_blocks0attn1to_out0bias', 'model_ema.output_blocks111transformer_blocks0attn1to_out0weight', 'model_ema.output_blocks111transformer_blocks0attn1to_qweight', 'model_ema.output_blocks111transformer_blocks0attn1to_vweight', 'model_ema.output_blocks111transformer_blocks0attn2to_kweight', 'model_ema.output_blocks111transformer_blocks0attn2to_out0bias', 'model_ema.output_blocks111transformer_blocks0attn2to_out0weight', 'model_ema.output_blocks111transformer_blocks0attn2to_qweight', 'model_ema.output_blocks111transformer_blocks0attn2to_vweight', 'model_ema.output_blocks111transformer_blocks0ffnet0projbias', 'model_ema.output_blocks111transformer_blocks0ffnet0projweight', 'model_ema.output_blocks111transformer_blocks0ffnet2bias', 'model_ema.output_blocks111transformer_blocks0ffnet2weight', 'model_ema.output_blocks111transformer_blocks0norm1bias', 'model_ema.output_blocks111transformer_blocks0norm1weight', 'model_ema.output_blocks111transformer_blocks0norm2bias', 'model_ema.output_blocks111transformer_blocks0norm2weight', 'model_ema.output_blocks111transformer_blocks0norm3bias', 'model_ema.output_blocks111transformer_blocks0norm3weight', 'model_ema.output_blocks20emb_layers1bias', 'model_ema.output_blocks20emb_layers1weight', 'model_ema.output_blocks20in_layers0bias', 'model_ema.output_blocks20in_layers0weight', 'model_ema.output_blocks20in_layers2bias', 'model_ema.output_blocks20in_layers2weight', 'model_ema.output_blocks20out_layers0bias', 'model_ema.output_blocks20out_layers0weight', 'model_ema.output_blocks20out_layers3bias', 'model_ema.output_blocks20out_layers3weight', 'model_ema.output_blocks20skip_connectionbias', 'model_ema.output_blocks20skip_connectionweight', 'model_ema.output_blocks21convbias', 'model_ema.output_blocks21convweight', 'model_ema.output_blocks30emb_layers1bias', 'model_ema.output_blocks30emb_layers1weight', 'model_ema.output_blocks30in_layers0bias', 'model_ema.output_blocks30in_layers0weight', 'model_ema.output_blocks30in_layers2bias', 'model_ema.output_blocks30in_layers2weight', 'model_ema.output_blocks30out_layers0bias', 'model_ema.output_blocks30out_layers0weight', 'model_ema.output_blocks30out_layers3bias', 'model_ema.output_blocks30out_layers3weight', 'model_ema.output_blocks30skip_connectionbias', 'model_ema.output_blocks30skip_connectionweight', 'model_ema.output_blocks31normbias', 'model_ema.output_blocks31normweight', 'model_ema.output_blocks31proj_inbias', 'model_ema.output_blocks31proj_inweight', 'model_ema.output_blocks31proj_outbias', 'model_ema.output_blocks31proj_outweight', 'model_ema.output_blocks31transformer_blocks0attn1to_kweight', 'model_ema.output_blocks31transformer_blocks0attn1to_out0bias', 'model_ema.output_blocks31transformer_blocks0attn1to_out0weight', 'model_ema.output_blocks31transformer_blocks0attn1to_qweight', 'model_ema.output_blocks31transformer_blocks0attn1to_vweight', 'model_ema.output_blocks31transformer_blocks0attn2to_kweight', 'model_ema.output_blocks31transformer_blocks0attn2to_out0bias', 'model_ema.output_blocks31transformer_blocks0attn2to_out0weight', 'model_ema.output_blocks31transformer_blocks0attn2to_qweight', 'model_ema.output_blocks31transformer_blocks0attn2to_vweight', 'model_ema.output_blocks31transformer_blocks0ffnet0projbias', 'model_ema.output_blocks31transformer_blocks0ffnet0projweight', 'model_ema.output_blocks31transformer_blocks0ffnet2bias', 'model_ema.output_blocks31transformer_blocks0ffnet2weight', 'model_ema.output_blocks31transformer_blocks0norm1bias', 'model_ema.output_blocks31transformer_blocks0norm1weight', 'model_ema.output_blocks31transformer_blocks0norm2bias', 'model_ema.output_blocks31transformer_blocks0norm2weight', 'model_ema.output_blocks31transformer_blocks0norm3bias', 'model_ema.output_blocks31transformer_blocks0norm3weight', 'model_ema.output_blocks40emb_layers1bias', 'model_ema.output_blocks40emb_layers1weight', 'model_ema.output_blocks40in_layers0bias', 'model_ema.output_blocks40in_layers0weight', 'model_ema.output_blocks40in_layers2bias', 'model_ema.output_blocks40in_layers2weight', 'model_ema.output_blocks40out_layers0bias', 'model_ema.output_blocks40out_layers0weight', 'model_ema.output_blocks40out_layers3bias', 'model_ema.output_blocks40out_layers3weight', 'model_ema.output_blocks40skip_connectionbias', 'model_ema.output_blocks40skip_connectionweight', 'model_ema.output_blocks41normbias', 'model_ema.output_blocks41normweight', 'model_ema.output_blocks41proj_inbias', 'model_ema.output_blocks41proj_inweight', 'model_ema.output_blocks41proj_outbias', 'model_ema.output_blocks41proj_outweight', 'model_ema.output_blocks41transformer_blocks0attn1to_kweight', 'model_ema.output_blocks41transformer_blocks0attn1to_out0bias', 'model_ema.output_blocks41transformer_blocks0attn1to_out0weight', 'model_ema.output_blocks41transformer_blocks0attn1to_qweight', 'model_ema.output_blocks41transformer_blocks0attn1to_vweight', 'model_ema.output_blocks41transformer_blocks0attn2to_kweight', 'model_ema.output_blocks41transformer_blocks0attn2to_out0bias', 'model_ema.output_blocks41transformer_blocks0attn2to_out0weight', 'model_ema.output_blocks41transformer_blocks0attn2to_qweight', 'model_ema.output_blocks41transformer_blocks0attn2to_vweight', 'model_ema.output_blocks41transformer_blocks0ffnet0projbias', 'model_ema.output_blocks41transformer_blocks0ffnet0projweight', 'model_ema.output_blocks41transformer_blocks0ffnet2bias', 'model_ema.output_blocks41transformer_blocks0ffnet2weight', 'model_ema.output_blocks41transformer_blocks0norm1bias', 'model_ema.output_blocks41transformer_blocks0norm1weight', 'model_ema.output_blocks41transformer_blocks0norm2bias', 'model_ema.output_blocks41transformer_blocks0norm2weight', 'model_ema.output_blocks41transformer_blocks0norm3bias', 'model_ema.output_blocks41transformer_blocks0norm3weight', 'model_ema.output_blocks50emb_layers1bias', 'model_ema.output_blocks50emb_layers1weight', 'model_ema.output_blocks50in_layers0bias', 'model_ema.output_blocks50in_layers0weight', 'model_ema.output_blocks50in_layers2bias', 'model_ema.output_blocks50in_layers2weight', 'model_ema.output_blocks50out_layers0bias', 'model_ema.output_blocks50out_layers0weight', 'model_ema.output_blocks50out_layers3bias', 'model_ema.output_blocks50out_layers3weight', 'model_ema.output_blocks50skip_connectionbias', 'model_ema.output_blocks50skip_connectionweight', 'model_ema.output_blocks51normbias', 'model_ema.output_blocks51normweight', 'model_ema.output_blocks51proj_inbias', 'model_ema.output_blocks51proj_inweight', 'model_ema.output_blocks51proj_outbias', 'model_ema.output_blocks51proj_outweight', 'model_ema.output_blocks51transformer_blocks0attn1to_kweight', 'model_ema.output_blocks51transformer_blocks0attn1to_out0bias', 'model_ema.output_blocks51transformer_blocks0attn1to_out0weight', 'model_ema.output_blocks51transformer_blocks0attn1to_qweight', 'model_ema.output_blocks51transformer_blocks0attn1to_vweight', 'model_ema.output_blocks51transformer_blocks0attn2to_kweight', 'model_ema.output_blocks51transformer_blocks0attn2to_out0bias', 'model_ema.output_blocks51transformer_blocks0attn2to_out0weight', 'model_ema.output_blocks51transformer_blocks0attn2to_qweight', 'model_ema.output_blocks51transformer_blocks0attn2to_vweight', 'model_ema.output_blocks51transformer_blocks0ffnet0projbias', 'model_ema.output_blocks51transformer_blocks0ffnet0projweight', 'model_ema.output_blocks51transformer_blocks0ffnet2bias', 'model_ema.output_blocks51transformer_blocks0ffnet2weight', 'model_ema.output_blocks51transformer_blocks0norm1bias', 'model_ema.output_blocks51transformer_blocks0norm1weight', 'model_ema.output_blocks51transformer_blocks0norm2bias', 'model_ema.output_blocks51transformer_blocks0norm2weight', 'model_ema.output_blocks51transformer_blocks0norm3bias', 'model_ema.output_blocks51transformer_blocks0norm3weight', 'model_ema.output_blocks52convbias', 'model_ema.output_blocks52convweight', 'model_ema.output_blocks60emb_layers1bias', 'model_ema.output_blocks60emb_layers1weight', 'model_ema.output_blocks60in_layers0bias', 'model_ema.output_blocks60in_layers0weight', 'model_ema.output_blocks60in_layers2bias', 'model_ema.output_blocks60in_layers2weight', 'model_ema.output_blocks60out_layers0bias', 'model_ema.output_blocks60out_layers0weight', 'model_ema.output_blocks60out_layers3bias', 'model_ema.output_blocks60out_layers3weight', 'model_ema.output_blocks60skip_connectionbias', 'model_ema.output_blocks60skip_connectionweight', 'model_ema.output_blocks61normbias', 'model_ema.output_blocks61normweight', 'model_ema.output_blocks61proj_inbias', 'model_ema.output_blocks61proj_inweight', 'model_ema.output_blocks61proj_outbias', 'model_ema.output_blocks61proj_outweight', 'model_ema.output_blocks61transformer_blocks0attn1to_kweight', 'model_ema.output_blocks61transformer_blocks0attn1to_out0bias', 'model_ema.output_blocks61transformer_blocks0attn1to_out0weight', 'model_ema.output_blocks61transformer_blocks0attn1to_qweight', 'model_ema.output_blocks61transformer_blocks0attn1to_vweight', 'model_ema.output_blocks61transformer_blocks0attn2to_kweight', 'model_ema.output_blocks61transformer_blocks0attn2to_out0bias', 'model_ema.output_blocks61transformer_blocks0attn2to_out0weight', 'model_ema.output_blocks61transformer_blocks0attn2to_qweight', 'model_ema.output_blocks61transformer_blocks0attn2to_vweight', 'model_ema.output_blocks61transformer_blocks0ffnet0projbias', 'model_ema.output_blocks61transformer_blocks0ffnet0projweight', 'model_ema.output_blocks61transformer_blocks0ffnet2bias', 'model_ema.output_blocks61transformer_blocks0ffnet2weight', 'model_ema.output_blocks61transformer_blocks0norm1bias', 'model_ema.output_blocks61transformer_blocks0norm1weight', 'model_ema.output_blocks61transformer_blocks0norm2bias', 'model_ema.output_blocks61transformer_blocks0norm2weight', 'model_ema.output_blocks61transformer_blocks0norm3bias', 'model_ema.output_blocks61transformer_blocks0norm3weight', 'model_ema.output_blocks70emb_layers1bias', 'model_ema.output_blocks70emb_layers1weight', 'model_ema.output_blocks70in_layers0bias', 'model_ema.output_blocks70in_layers0weight', 'model_ema.output_blocks70in_layers2bias', 'model_ema.output_blocks70in_layers2weight', 'model_ema.output_blocks70out_layers0bias', 'model_ema.output_blocks70out_layers0weight', 'model_ema.output_blocks70out_layers3bias', 'model_ema.output_blocks70out_layers3weight', 'model_ema.output_blocks70skip_connectionbias', 'model_ema.output_blocks70skip_connectionweight', 'model_ema.output_blocks71normbias', 'model_ema.output_blocks71normweight', 'model_ema.output_blocks71proj_inbias', 'model_ema.output_blocks71proj_inweight', 'model_ema.output_blocks71proj_outbias', 'model_ema.output_blocks71proj_outweight', 'model_ema.output_blocks71transformer_blocks0attn1to_kweight', 'model_ema.output_blocks71transformer_blocks0attn1to_out0bias', 'model_ema.output_blocks71transformer_blocks0attn1to_out0weight', 'model_ema.output_blocks71transformer_blocks0attn1to_qweight', 'model_ema.output_blocks71transformer_blocks0attn1to_vweight', 'model_ema.output_blocks71transformer_blocks0attn2to_kweight', 'model_ema.output_blocks71transformer_blocks0attn2to_out0bias', 'model_ema.output_blocks71transformer_blocks0attn2to_out0weight', 'model_ema.output_blocks71transformer_blocks0attn2to_qweight', 'model_ema.output_blocks71transformer_blocks0attn2to_vweight', 'model_ema.output_blocks71transformer_blocks0ffnet0projbias', 'model_ema.output_blocks71transformer_blocks0ffnet0projweight', 'model_ema.output_blocks71transformer_blocks0ffnet2bias', 'model_ema.output_blocks71transformer_blocks0ffnet2weight', 'model_ema.output_blocks71transformer_blocks0norm1bias', 'model_ema.output_blocks71transformer_blocks0norm1weight', 'model_ema.output_blocks71transformer_blocks0norm2bias', 'model_ema.output_blocks71transformer_blocks0norm2weight', 'model_ema.output_blocks71transformer_blocks0norm3bias', 'model_ema.output_blocks71transformer_blocks0norm3weight', 'model_ema.output_blocks80emb_layers1bias', 'model_ema.output_blocks80emb_layers1weight', 'model_ema.output_blocks80in_layers0bias', 'model_ema.output_blocks80in_layers0weight', 'model_ema.output_blocks80in_layers2bias', 'model_ema.output_blocks80in_layers2weight', 'model_ema.output_blocks80out_layers0bias', 'model_ema.output_blocks80out_layers0weight', 'model_ema.output_blocks80out_layers3bias', 'model_ema.output_blocks80out_layers3weight', 'model_ema.output_blocks80skip_connectionbias', 'model_ema.output_blocks80skip_connectionweight', 'model_ema.output_blocks81normbias', 'model_ema.output_blocks81normweight', 'model_ema.output_blocks81proj_inbias', 'model_ema.output_blocks81proj_inweight', 'model_ema.output_blocks81proj_outbias', 'model_ema.output_blocks81proj_outweight', 'model_ema.output_blocks81transformer_blocks0attn1to_kweight', 'model_ema.output_blocks81transformer_blocks0attn1to_out0bias', 'model_ema.output_blocks81transformer_blocks0attn1to_out0weight', 'model_ema.output_blocks81transformer_blocks0attn1to_qweight', 'model_ema.output_blocks81transformer_blocks0attn1to_vweight', 'model_ema.output_blocks81transformer_blocks0attn2to_kweight', 'model_ema.output_blocks81transformer_blocks0attn2to_out0bias', 'model_ema.output_blocks81transformer_blocks0attn2to_out0weight', 'model_ema.output_blocks81transformer_blocks0attn2to_qweight', 'model_ema.output_blocks81transformer_blocks0attn2to_vweight', 'model_ema.output_blocks81transformer_blocks0ffnet0projbias', 'model_ema.output_blocks81transformer_blocks0ffnet0projweight', 'model_ema.output_blocks81transformer_blocks0ffnet2bias', 'model_ema.output_blocks81transformer_blocks0ffnet2weight', 'model_ema.output_blocks81transformer_blocks0norm1bias', 'model_ema.output_blocks81transformer_blocks0norm1weight', 'model_ema.output_blocks81transformer_blocks0norm2bias', 'model_ema.output_blocks81transformer_blocks0norm2weight', 'model_ema.output_blocks81transformer_blocks0norm3bias', 'model_ema.output_blocks81transformer_blocks0norm3weight', 'model_ema.output_blocks82convbias', 'model_ema.output_blocks82convweight', 'model_ema.output_blocks90emb_layers1bias', 'model_ema.output_blocks90emb_layers1weight', 'model_ema.output_blocks90in_layers0bias', 'model_ema.output_blocks90in_layers0weight', 'model_ema.output_blocks90in_layers2bias', 'model_ema.output_blocks90in_layers2weight', 'model_ema.output_blocks90out_layers0bias', 'model_ema.output_blocks90out_layers0weight', 'model_ema.output_blocks90out_layers3bias', 'model_ema.output_blocks90out_layers3weight', 'model_ema.output_blocks90skip_connectionbias', 'model_ema.output_blocks90skip_connectionweight', 'model_ema.output_blocks91normbias', 'model_ema.output_blocks91normweight', 'model_ema.output_blocks91proj_inbias', 'model_ema.output_blocks91proj_inweight', 'model_ema.output_blocks91proj_outbias', 'model_ema.output_blocks91proj_outweight', 'model_ema.output_blocks91transformer_blocks0attn1to_kweight', 'model_ema.output_blocks91transformer_blocks0attn1to_out0bias', 'model_ema.output_blocks91transformer_blocks0attn1to_out0weight', 'model_ema.output_blocks91transformer_blocks0attn1to_qweight', 'model_ema.output_blocks91transformer_blocks0attn1to_vweight', 'model_ema.output_blocks91transformer_blocks0attn2to_kweight', 'model_ema.output_blocks91transformer_blocks0attn2to_out0bias', 'model_ema.output_blocks91transformer_blocks0attn2to_out0weight', 'model_ema.output_blocks91transformer_blocks0attn2to_qweight', 'model_ema.output_blocks91transformer_blocks0attn2to_vweight', 'model_ema.output_blocks91transformer_blocks0ffnet0projbias', 'model_ema.output_blocks91transformer_blocks0ffnet0projweight', 'model_ema.output_blocks91transformer_blocks0ffnet2bias', 'model_ema.output_blocks91transformer_blocks0ffnet2weight', 'model_ema.output_blocks91transformer_blocks0norm1bias', 'model_ema.output_blocks91transformer_blocks0norm1weight', 'model_ema.output_blocks91transformer_blocks0norm2bias', 'model_ema.output_blocks91transformer_blocks0norm2weight', 'model_ema.output_blocks91transformer_blocks0norm3bias', 'model_ema.output_blocks91transformer_blocks0norm3weight', 'model_ema.time_embed0bias', 'model_ema.time_embed0weight', 'model_ema.time_embed2bias', 'model_ema.time_embed2weight'])
Requested to load SD1ClipModel
Loading 1 new model
[AnimateDiff] - INFO - Injecting motion module with method default.
Requested to load BaseModel
Loading 1 new model
DDIM Sampler: 100%|█████████████████████████████| 25/25 [03:33<00:00, 8.54s/it]
[AnimateDiff] - INFO - Ejecting motion module with method default.
Using split attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using split attention in VAE
Prompt executed in 222.30 seconds
LoRA+アップスケール(モデル)
生成AI出力結果
2048×2048の画像になりました。
ノード図
アップスケールモデルではRealESRGAN_x4plus.pthを使いました。
ComfyUIの場合512×512でアニメgifを一度作るとそれがストックされており読み出すようになっているようです。
LoRA+アップスケール(モデル+hires fix)
生成AI出力結果
使用モデルは
Checkpoint:beautifulRealistic_brav5.safetensors
LoRA:JapaneseDollLikeness_v15.safetensors
ポジ:c(best quality, masterpiece), fine lady looking back at me.
, full body
ネガ:(easynegative:1.0),(worst quality,low quality:1.2),(bad anatomy:1.4),(realistic:1.1),nose,lips,adult,fat,sad, (inaccurate limb:1.2),extra digit,fewer digits,six fingers,(monochrome:0.95)
になります。学習されたモデルが学習してきた写真によって得意なポーズがある程度あるような気がします。
モデル図
hires fixの場合はKSamplerを使う代わりにAnimate Diff Samplerを使います。それ以外は画像生成の場合と同じように追加すれば大丈夫です。
Controlnet
拡張機能ComfyUI-AnimateDiff-Evolvedによるノード図
現在のところMac apple siliconではWarningになって真っ黒なgif画像が生成されてしまいます。主要なコンテンツが非推奨(deprecated)と表示されるので今現在は使えない可能性が高いかもしれません。またgithubの解説記事を再現することそのものができないようです。(2023年10月現在)