u/Anzhc

It's been a while. Missed me?

I needed some control for gens, but was not satisfied with existing solutions, so i took some time to develop better approach.

https://huggingface.co/CabalResearch/Anima-Canny-Scribble-Adjustable-Control-LoRA

https://github.com/Anzhc/Anzhc-ComfyUI-Cosmos-Reference

Those lora and nodes allow for somewhat adjustable control input, unlike previous attempts.

For more linear scaling i recommend KV gating, for smoother scale effect use temporal masking. You need node pack linked above for either, as they are built into new node.

This lora was trained with Scribble, Canny and Depth. All 3 are recognized by model, but only scribble and canny are reliable, use depth only as secondary input. Model is very receptive to mix of controls.

You can find example workflow in both github and hf repos.

This was trained basically overnight(but not on my famous 4060ti), and can be much higher quality, with more inputs and better strength adjustment.

This prototype also shows that presence of lora does not necessarily need to force model to use any reference (kv gating 0 basically turns it off, while lora is present), which means that possible next approach is native control support, right in model, without lora.

But i doubt anyone would bother doing that, right...

Also i have tested Edit loras with Anima. They also work fine(for what i tested, that is). (Yes that means Anima could be a native t2i+Control+Edit model)

Do what you will with that information. :doro:

Ashen Badlands - Free Pack (Hey guys, im back)

Anima Scribble+Canny (and Depth in the corner), now with adjustable strength