Makeup on/off in an Anima character LoRA, subtractive vs additive at cfg 1.0
Contents
Before adding a fourth character (a gal type with switchable makeup) to my three-character merged LoRA, I test-baked her as a solo LoRA. 81 images (61 with makeup + 20 bare-faced), using the recipe already proven on my keichan/kanachan solo LoRAs: rank32 / alpha32 / lr 2e-5 / repeats 4. It took three runs. v1 died to tilted training data, v2 finished 150 epochs but the makeup switch never fired at my production setting of cfg 1.0, and v3 made it work by turning the switch from subtraction into addition.
Test environment
| Item | Details |
|---|---|
| Base model | Anima-Base |
| Training tool | AnimaLoraToolkit (torch2.8.0+cu128 / ubuntu24.04) |
| Training GPU | RunPod RTX 5090 |
| Training config | rank32 / alpha32 / lr2e-5 / repeats4 / 150 epochs / save_every=1 / flip false / SDPA |
| Data | v1: 81 images (61 makeup + 20 bare-faced) → v2/v3: 91 images (bare-faced raised to 30) |
| Generation check | ComfyUI, Turbo 8-step, cfg 1.0 (the production setting shared by all my LoRAs) |
Benchmark a RunPod pod right after spinning it up (the speed lottery)
The most expensive mistake this time. Two pods sold as the same RTX 5090 can differ by half in effective throughput.
- trio v2 (310 images, 155 steps/ep, rank256): 2.45 s/step (measured back then from samples mtime)
- this pod (81 images, 81 steps/ep, rank32): 4.8 s/step. Half the work, same 16-hour runtime
Ruling things out, the following were not the cause.
| What I checked | Verdict | Evidence |
|---|---|---|
| Training image pixel count | Fine | AnimaLoraToolkit normalizes everything to 1024²±10% area via AR buckets; the trio material was actually larger |
| Software stack | Fine | Identical torch2.8.0+cu128 / ubuntu24.04 / 570-series driver |
| Thermal / HW throttling | Fine | Not Active, 58°C |
| Memory clock | Fine | 13.8/14.0 GHz, 20% bandwidth use |
| CPU starvation / IO | Mostly fine | GPU util 99% at 554W, working flat out |
What remains is either shared-host/virtualization quality or a different image under the same template name (a cudnn 9.10 path with slow attention). I planned to settle it with an idle benchmark, but this run got cut at ep103 over data problems (the head-tilt issue below), the pod went down right after, and the measurement never happened. Which one it was stays unconfirmed, but what you do about it is the same either way.
Spin up the pod, run a smoke training job, and check the measured it/s. If it comes in more than 20% under the expected value (0.35 it/s or better for this config), rebuild the pod immediately. Five minutes of screening costs cents, while keeping a dud costs half a day of extra time and fees. This time I launched the real run on “it works, good enough” and a 6-8 hour estimate became 16 hours.
A dud pod charges you twice: you pay the RTX 5090 rate ($1.00/h) for 4090-class throughput. Renting a 4090 at half price would have finished in the same time. Had I benchmarked first, I would not have kept a “higher-tier GPU must be faster” pod running for half a day.
The benchmark gate paid off in the v2 run (below). A $0.70/h pod outside secure cloud passed the idle benchmark (bf16 235 TFLOPS) before I committed, and measured 0.48 it/s. Twice the trio speed, at a lower rate.
Transfers are a lottery too
The same pod also had bad transfer speeds.
Model downloads from HuggingFace ran at 200KB/s over plain curl (5 hours for 4GB). aria2 parallel ranges get rejected with 403 because Xet storage signs URLs for a fixed byte range (ByteRange). hf download (with hf_xet) pulled 4GB in one minute, so the hf CLI was the only workable option.
Uploads from home to the pod ran at 25KB/s over single scp/rsync. Eight parallel scp streams reached ~650KB/s (the cap appears to be per stream, so it scales with parallelism), which moved the 104MB dataset in about 10 minutes. Heavy files (models) get downloaded on the pod side; home bandwidth only carries the dataset up and the checkpoints back.
Dataset design for Kurara (81 images)
The split is 61 makeup images (36 full-body, 25 bust/waist-up) and 20 bare-faced (15 face-focused, 5 full-body). Identity (hair, eyes, body type, earrings) is baked into the trigger kurara with no caption mention, the same scheme as the three existing characters.
Bare-faced images get a coined makeoff tag right after the trigger. I avoided the existing no makeup tag for three reasons: it mixes with the base model’s prior knowledge and pulls output away from the dataset; 20 images of Kurara’s face would contaminate a general-purpose tag and leak into the four-character merge; and T5-style encoders handle negation (no X) poorly.
Nine of the 20 bare-faced images are deliberately clothed (loungewear) to break the makeoff≒nude confound (the first 11 were all bare skin). The four nude full-body shots have medium-sized breasts, so medium breasts is written explicitly to keep them off the slender default (the untagged side).
Making variants without losing the makeup
QwenImageEdit and Gemini (nano banana) redraw the face region when changing expression or composition, so light gal makeup at the lashes-and-gloss level gets normalized away (and QIE turns Kurara into a different person outright, already confirmed). Two routes worked.
The first is cropping existing full-body images (no generation, so no degradation by construction). The top half of a 1024×2048-class full body yields a 1024×1065-1331 waist-up.
The second is reference-guided generation with Codex (gpt-image). Writing “light gal-style makeup … IDENTICAL to the reference — do NOT remove or weaken the makeup” into the identity part of the prompt kept makeup, earrings, and body type while varying expression and composition.
On the bare-faced side, writing “center-parted hair” made Codex sweep the bangs fully off the forehead and killed two whole batches. Putting “bangs are the top priority, no exposed forehead” at the head of the prompt and swapping the reference to an image with clear bangs fixed it.
Training settings
Settings are as in the environment table, the values proven on the keichan v2/v4 and kanachan solo runs. 81 steps/epoch, 600 exposures per image.
sample_prompts included makeup comparison pairs from the start: every token identical except makeoff, with the expression pinned to smile (a wandering expression ruins face comparison). With one prompt per epoch in rotation, consecutive epochs form a makeup↔makeoff comparison pair.
First failure, a permanent head tilt (v1, 81 images, stopped at ep103)
Generating with nothing specified always produced a slight head tilt, and the training samples tilted the same way.
Checking the data, about 18 of the 45 face-focused images were slightly tilted. Codex has a habit of tilting the head for cuteness when asked for expression variants. The 36 full-body images were straight, so the tilt traces back to the generated face shots. Tagging cannot fix this data distribution problem; as with keichan v4, the tilted images have to be replaced.
The makeoff switch was weak too. It worked on clothes (loungewear↔uniform) but not on the face (earrings, lashes). Confirmed in local tests at ep42/ep100.
All six unspecified generations under production settings (makeup ×3 seeds, makeoff ×3 seeds) tilt. Adding “head straight” to every prompt would mask it, but the bias stays in the LoRA.
I cut the run at ep103. The 19 tilted images went out, refilled with 7 crops from upright full-body shots plus Codex regenerations. The regeneration prompt starts with “head perfectly upright and level, NO head tilt” and requires a self-check before saving. Same as the bangs issue: Codex’s habits have to be countered explicitly in the prompt.
Bare-faced images also went from 20 to 30, all clothed loungewear and upright, varied across everyday actions (towel, toothbrush, hairbrush). The bare-faced tag changed from makeoff alone to “makeoff, no makeup”.
Second failure, tilt fixed but makeoff dead at production settings (v2, 91 images, 150 epochs)
The head tilt was gone in every condition. Identity was stable and face degradation slow (still minor at ep141). The data-caused problem was fixed by replacing the data.
But at the production setting (Turbo 8-step, cfg 1.0) the makeoff switch did nothing. Earrings, lashes, even clothes stayed the same. Confirmed across 3 seeds × several epochs (84/130/134/138/141).
The confusing part: the toolkit’s built-in sampler (cfg 4.0, 25 steps, no turbo) produced clean bare faces from ep130 on, earrings gone. Matching ComfyUI to the exact same settings (same weights, same prompt string, cfg 4, er_sde/simple/25 steps) still would not remove the earrings.
Left: the toolkit sampler (ep133 makeup ↔ ep134 makeoff) switches everything including earrings. Right: the ComfyUI sweep under production settings (ep130-141, right side of each row is makeoff) keeps the earrings in every image. Same weights, that much difference.
The isolation pointed to pipeline implementation differences, not setting values. The toolkit encodes on HF’s Qwen3-0.6B with its own path (add_special_tokens=False, a single trailing EOS end-of-sequence token, attention mask on), while ComfyUI’s CLIPLoader (type: stable_diffusion) does SD-style token processing. That small encoding difference preserves strong signals like identity and drops weak ones like makeoff.
Even if that gap were closed and cfg 4 were an option, another problem remains. This project generates everything at Turbo + cfg 1.0 (raising cfg breaks multi-character stability, proven on the three-character LoRA). The four-character merge is the end goal, and a switch that only works at cfg 4 is unusable there.
So the design of baking an attribute in untagged and subtracting it with a tag (makeoff) did not work at cfg 1.0. A negative-direction switch only reaches the image when cfg amplifies the difference, and this pipeline pins cfg at 1.0.
Third bake, from subtraction to addition (v3)
Since subtraction failed in v2, makeup moves from something to subtract to something tags add. The 61 makeup images get earrings, makeup written into their captions (except 3 straight-from-behind shots with no visible face), which detaches earrings and makeup from the kurara trigger onto tags.
The generation grammar changes with it. The makeup face is called with presence tags, kurara, earrings, makeup, ..., and presence tags fire reliably at cfg 1.0 (the same mechanism as everyday outfit tags). Bare-faced is kurara, makeoff, no makeup, ..., and since makeup is no longer fused to the trigger, the bare face comes out without any subtraction. Tag bleed in the four-character merge (earrings landing on another character) will be handled with name-scoped structured prompts (proven in the trio article).
The goal for the solo LoRA: the makeup on/off works at the same strength and the same generation settings (cfg 1.0) as the other characters’ identities. The solo bake is a waypoint to the four-character merge, and the merge runs at cfg 1.0, so it has to work at cfg 1.0 from the solo stage.
v3 switches both ways at cfg 1.0, ep140 adopted
150 epochs completed (a good pod, 0.47 it/s, about 9 hours). At ep79 the production-setting A/B (Turbo 8-step, cfg 1.0) passed in both directions for the first time.
Left to right: seed42 makeup/off, seed100 makeup/off, seed300 makeup/off. All three makeup images show earrings, winged lashes, and lip color; all three makeoff images lose the earrings and settle into a plain face. “Earrings come off at cfg 1.0”, which v1/v2 never achieved once, went 3/3. The only change was swapping subtraction for additive presence tags, and the same cfg 1.0 passed.
After the run, a sweep across the late band: (ep110/120/130/140/150) × makeup/bare × 3 seeds = 30 images.
- Makeup (left 3 columns): earrings and makeup fire in 15/15
- Bare-faced (right 3 columns): earrings gone in 15/15, softened lashes and colorless lips stable
- Heads visually straight in 30/30 (no v1-style obvious tilt), identity held in both states, no color fade or breakdown through ep150
Full-body switches too.
The whole band (ep110-150) is flat and clean, so I adopted ep140. Separation gets stronger deeper into the band, but I stayed one step short of the last ep150 (the same margin call as taking ep143 on trio v2).
With the adopted ep140, pinning outfit, pose, composition, and seed while changing only the makeup tags gives this. The remaining difference is the makeup.
What surprised me is that the makeup on/off reads more clearly in full-body shots than in bust-ups. The facial structure stays put while the lines get heavier. The smaller the face in frame, the more the lash and eyeliner ink collapses into overall line weight.
Zooming into the eyes shows what “the eyes look different” actually is.
With makeup, the upper lid line thickens toward the outer corner and ends in an upward black wedge (the eyeliner wing). It gets drawn as part of the eye outline, so the outer corner lifts and the eye reads slightly upturned. Bare-faced, the wing disappears, the line thins out evenly, and the eye returns to its natural slight droop. The bare faces in the training data droop the same way, so this is a faithful difference, not a defect. The tells are earrings, lips, line weight, and the angle of the outer eye corner.
Stacking tags for heavier makeup
A side effect of moving makeup onto tags: the base model’s makeup vocabulary stacks right on top.
| Level | Prompt |
|---|---|
| Normal | kurara, earrings, makeup |
| Heavy | kurara, earrings, heavy makeup, thick eyelashes, long eyelashes, eyeshadow, eyeliner, glossy lips, blush |
| Maxed | heavy tags + a natural-language push: “heavy flashy gal makeup with bold winged eyeliner…” |
The rightmost image piles on thick winged liner, pink eyeshadow, long upper and lower lashes, and glossy lips, unmistakably gaudy, while the face itself holds. Bare, normal, and maxed all switch under the same cfg 1.0 settings.
The generation grammar ended up like this.
| State | Prompt |
|---|---|
| Makeup face | kurara, earrings, makeup, ... |
| Bare-faced | kurara, makeoff, no makeup, ... |
Outfits are specified separately with the usual clothing tags in both states. cfg stays at 1.0, shared across all the LoRAs.
A slight head tilt survived in bust-up shots
Re-checking tilt with a grid overlay before publishing: the four full-body shots (makeup/bare × 2 seeds) sit within 1-2° of vertical, but bust-ups keep a few degrees of head tilt. The heavier the makeup, the stronger the tilt, with the maxed heavy-makeup prompt tilting most. The 5-15° tilts of v1 are gone; 2-5° remains in face-heavy compositions.
There is an obvious suspect. I checked the v3 Codex regenerations for uprightness at thumbnail size, so a few degrees of tilt could slip through. Two intentionally tilted images (tagged as such) and a few borderline keepers also sit in the face-focused pool, and bust-up generation leans hard on that part of the distribution, which matches how the tilt shows up.
As a solo LoRA it is within usable range, so there will be no v4 rebake. When the four-character dataset gets assembled, the Kurara face shots get re-checked at grid precision and the tilted ones replaced or dropped.