4-character Anima LoRA kept drawing one girl until prompts matched its captions
Contents
The 4-character merged LoRA went into production after hitting a 6/6 one-shot rate with fully specified prompts (triggers + position lines + ID anchors + outfit lines). But when I requested all four characters through my self-built generation server, which bridges Japanese prompts to ComfyUI, the output would often collapse into the same character over and over. Fewer characters made it less likely, and with four it took a lot of prompt work to get all four to show up.
Since the training-time checks and production generation disagreed, I needed to figure out whether the problem was in the LoRA or in how the prompts were written. I stripped information out of the prompt in stages and measured, for every character combination, how much I could remove before someone disappeared.
Test environment
| Item | Details |
|---|---|
| Base model | Anima-Base v1.0 |
| Character LoRA | anima-4char-v1_epoch100 (rank256) |
| Generation path | Direct calls to my generation server’s API, with trigger words and prompt body set per job |
| Generation settings | Turbo 8 steps / cfg 1.0 / er_sde / simple / LoRA strength 1.0 |
| Characters | kurara, keichan, kanachan, koharu (all four in the merged LoRA) |
| Seeds | 42 / 1234 / 9999, fixed |
Four prompt levels
The design varies both the number of characters and the amount of prompt information, to see at which level each character drops out.
| Level | Contents |
|---|---|
| Lv0 | Trigger words only |
| Lv1 | + ID anchors (one short line: hair color, accessories, etc.) |
| Lv2 | + position lines (who is left / right / center) |
| Lv3 | + outfit line (same outfit for everyone) |
The targets were all 11 combinations of 2 to 4 characters (6 pairs, 4 trios, 1 quad). I first measured both ends, Lv0 and Lv3, across every combination, then ran extra tests only where things broke. The pass criterion was whether every requested character comes out looking like themselves. Height differences were treated as informational only.
Trigger words alone break most combos
Running Lv0 (trigger words only) across 11 combos × 3 seeds, 33 generations in total, not a single combination passed on all 3 seeds.
| # | Combination | Lv0 passes |
|---|---|---|
| 1 | kurara+keichan | 1/3 |
| 2 | kurara+kanachan | 0/3 |
| 3 | kurara+koharu | 1/3 |
| 4 | kanachan+keichan | 0/3 |
| 5 | koharu+keichan | 0/3 |
| 6 | koharu+kanachan | 0/3 |
| 7 | kurara+keichan+kanachan | 1/3 |
| 8 | kurara+keichan+koharu | 0/3 |
| 9 | kurara+kanachan+koharu | 0/3 |
| 10 | kanachan+keichan+koharu | 0/3 |
| 11 | all four | 0/3 |
Out of 33 generations, only 3 produced every requested character; the other 30 broke. Most pairs already failed, so the failures happened regardless of head count. The failures did skew in one direction: most of the 30 involved kurara in some way. An unrequested kurara would butt in, or another character’s hairstyle and height would drift toward kurara’s. The same patterns kept showing up.
Across the merged LoRA’s whole training set, kurara’s trigger word appears 312 times, more than keichan’s and koharu’s 282 and kanachan’s 250. The simplest reading is that the character with the most training exposure becomes the default whenever a trigger-only prompt leaves things ambiguous.
Stacking ID anchors, position and outfit lines fixes almost everything
Running Lv3 (prompts with ID anchors, position lines and the outfit line all included) on the same 33 generations, 8 of the 11 combos passed 3/3. In the remaining 3 combos everyone looked like themselves, but the height difference broke. Counting the previously informational height difference as part of the judgment, koharu+keichan and koharu+kanachan never produced a height gap on any seed (0/3), and kurara+koharu passed 2/3 with one generation where the gap shrank.
I went through the training captions next. In the koharu+keichan pair images, 19 of the 20 captions mention that koharu is shorter, and 14 of the 20 koharu+kanachan pair images carry the same kind of mention (12 of them with the identical wording “about half a head shorter”). For both pairs, my original prompts had simply left that line out.
Adding the exact phrasing from the training captions made all 3 seeds separate correctly, height difference included.
# Before (height-difference line missing)
koharu, with short dark hair and red eyes, is on the left.
keichan, a blonde girl with a blue ribbon, is on the right.
# After (phrasing from the training captions added)
koharu, a shorter girl with short dark hair and red eyes, is on the left.
keichan, a blonde girl with a blue ribbon, is on the right.
Height gaps don’t invert, they disappear
The remaining kurara+koharu 2/3 didn’t respond to the same fix that worked for koharu+keichan and koharu+kanachan. Checking this pair’s training captions (6 images), not one of them mentions a height difference. The line I had put in the prompt, koharu being about a head shorter, existed nowhere in the training data; I had written it myself.
Looking back at the pre-fix generations for koharu+keichan and koharu+kanachan, the failure mode was always the same: never an inversion where one character ends up taller than the other, but the two heights converging until the gap is gone. As far as this environment and these settings show, there is a default pull toward rendering multiple characters at similar heights, and the outcome depends on how strongly the height instruction can override it. Within what I checked, adding a phrase that appears in 70–95% of the training captions produced the height gap on all 3 seeds, while instructions that don’t exist in the training data stayed unreliable.
What to fix on the generation server
Most of the failures observed across the 66 generations (11 combos × Lv0/Lv3 × 3 seeds) went away just by writing the prompt back up to training-caption level. I didn’t compare different ranks, so this test says nothing about the effect of LoRA capacity or rank itself. The ID anchors my generation server inserts automatically on character selection are just one short line (hair color and accessories), with none of the specific identifying phrases like position lines, outfit lines, or per-pair height differences. The production feeling that one particular character kept coming out was most likely this thin auto-inserted text at work.
The training captions were supposed to be in good shape to begin with. I had double-checked them with Claude and Codex and tuned them so every phrase appeared at similar frequencies and matched the training images. Given how this test undermined that assumption, I suspect this LoRA hasn’t reached the quality of the 3-character version. Still, prompting does compensate, so my next research question is whether swapping in a larger text encoder for the prompt side, if that turns out to be possible, would widen the expressive range.