u/GrapefruitOk9723

I would like to emphasize the latter requirement especially since I find that a lot of existing character Loras fail to recreate more complex facial expressions of a character. For example, when I prompt the character to smile, it is as if the Lora pastes some other person’s smile on that character’s face, which ruins the resemblance.

I know that this limitation is likely due to small dataset the Lora has been trained on, so I prepared a dataset of around 300 images of a character from a variety of angles with different facial expressions. Essentially, I am looking to train a Lora that can actually remember and recreate these expressions.

I have 3 main questions:

1. What base model should I use to train the Lora? I don’t care about VRAM or time requirements since I am planning to train online.

2. What settings should I use to get the desired result? I imagine that Lora Rank/Dim should be higher so that the Lora has enough memory to learn different facial expressions. If anyone can share their full training parameters/link to some tutorial, that would be great.

3. How important is it to have environmental variety in the dataset? To get the training images for different facial expressions, I mainly took screenshots from a video. Is it ok if 2/3 of my dataset have the same background or should I batch run these images through an image-editing workflow to get some variety in lighting/background?

I would like to emphasize the latter requirement especially since I find that a lot of existing character Loras fail to recreate more complex facial expressions of a character. For example, when I prompt the character to smile, it is as if the Lora pastes some other person’s smile on that character’s face, which ruins the resemblance.

I know that this limitation is likely due to small dataset the Lora has been trained on, so I prepared a dataset of around 300 images of a character from a variety of angles with different facial expressions. Essentially, I am looking to train a Lora that can actually remember and recreate these expressions.

I have 3 main questions:

1. What base model should I use to train the Lora? I don’t care about VRAM or time requirements since I am planning to train online.

2. What settings should I use to get the desired result? I imagine that Lora Rank/Dim should be higher so that the Lora has enough memory to learn different facial expressions. If anyone can share their full training parameters/link to some tutorial, that would be great.

3. How important is it to have environmental variety in the dataset? To get the training images for different facial expressions, I mainly took screenshots from a video. Is it ok if 2/3 of my dataset have the same background or should I batch run these images through an image-editing workflow to get some variety in lighting/background?

I would like to emphasize the latter requirement especially since I find that a lot of existing character Loras fail to recreate more complex facial expressions of a character. For example, when I prompt the character to smile, it is as if the Lora pastes some other person’s smile on that character’s face, which ruins the resemblance.

I know that this limitation is likely due to small dataset the Lora has been trained on, so I prepared a dataset of around 300 images of a character from a variety of angles with different facial expressions. Essentially, I am looking to train a Lora that can actually remember and recreate these expressions.

I have 3 main questions:

1. What base model should I use to train the Lora? I don’t care about VRAM or time requirements since I am planning to train online.

2. What settings should I use to get the desired result? I imagine that Lora Rank/Dim should be higher so that the Lora has enough memory to learn different facial expressions. If anyone can share their full training parameters/link to some tutorial, that would be great.

3. How important is it to have environmental variety in the dataset? To get the training images for different facial expressions, I mainly took screenshots from a video. Is it ok if 2/3 of my dataset have the same background or should I batch run these images through an image-editing workflow to get some variety in lighting/background?