LLaMA Pro block expansion on Qwen3-0.6B learns my blog but forgets Wikipedia

Last time, in Do existing character LoRAs still work on the 40-layer block-expanded Anima-2.9B, I mentioned in passing that Anima-2.9B adds its layers the LLaMA Pro way.
The method is to copy a neighboring block and insert the copy, zero the copy’s output weights, then freeze the original layers and train only the inserted ones.
I had it mixed up with Muon at first, and I have never actually run LLaMA Pro itself on this blog.

With an image model, the only check was eyeballing whether the character looks right, which is honestly a rough call. With an LLM I can compare perplexity numbers.
So I did the same thing to Qwen3-0.6B-Base to check two things: do the inserted layers alone learn new content, and does freezing the original layers really prevent forgetting.

For training data I just reused this blog’s tech posts.
Qwen3-0.6B came out in April 2025, so it naturally knows nothing about Anima, QIE, the 2026 ComfyUI releases, or Kana, Kei, Koharu and Kurara. All of it is past the cutoff.
Same setup as Anima-2.9B, which froze its original 28 layers and taught 1.7M images to 12 new layers only, except with 0.6B and text.

Test environment

Item	Details
Machine	M1 Max 64GB MacBook Pro (macOS 26.5)
Framework	MLX 0.32.1, mlx-lm 0.31.3
Base model	Qwen3-0.6B-Base (28 layers, bf16)
Methods compared	Block expansion (new layers only), LoRA, full fine-tuning
Training data	This blog’s tech posts (Japanese)
Evaluation	Perplexity on held-out posts (learned?), on Wikipedia-ja and wikitext-2 (forgotten?)

Experiment outline

graph TD
    A[763 tech posts<br/>hold out newest 40 for eval] --> B[722 training posts]
    Q[Qwen3-0.6B-Base, 28 layers] --> R[copy and insert 7 blocks, 35 layers<br/>zero the copies' outputs]
    B --> C[Block expansion<br/>train only the 7 inserted layers]
    R --> C
    B --> D[LoRA<br/>adapters on the 28 layers]
    B --> E[Full FT<br/>all parameters]
    C --> F[Compare perplexity<br/>learned? / forgotten?]
    D --> F
    E --> F
    D --> G[Apply 28-layer LoRA to 35 layers<br/>as-is / renumbered]
    F --> H[Compare greedy generations]

On the data side, I take the Markdown body of 763 tech posts, hold out a few dozen of the newest for evaluation, and train on the rest.
On the model side, I split Qwen3-0.6B-Base’s 28 layers into groups of 4, copy the last block of each group and insert it right after, giving 35 layers.
Zero the copies’ o_proj and down_proj, and confirm with logits (the raw scores right before the next token is picked) that the untrained model’s output matches the original.

Training runs the same data and the same token count through three methods: block expansion (the 7 inserted layers only), LoRA (adapters on the original 28 layers), and full FT (all parameters).
Before and after training, measure perplexity on the training posts, the held-out posts, the English versions of the held-out posts, Wikipedia-ja, and wikitext-2.

Separately, apply the LoRA baked on the plain 28-layer model to the 35-layer model both as-is and with renumbered keys, and re-measure in perplexity what the Anima-2.9B article did.
Finally, generate continuations from the same prompts before and after training and compare what comes out.

Data

The tech posts go in as training data with just the frontmatter stripped, the title turned into a # heading, and the body kept as raw Markdown, tables and code blocks included.

The held-out set is the newest 40 posts by publishedAt, published July 26 through August 22, 2026.
The Anima-2.9B article is in this set.
The English versions of the same 40 posts are held out the same way, to check whether English-side perplexity moves when only Japanese is trained.

The forgetting check uses Japanese Wikipedia and the English wikitext-2 test split.
Wikipedia is wikimedia/wikipedia, the 20231101.ja dump: from the head of one shard (one of the dataset’s split files) I picked articles of 1500+ characters and cut long ones at 6000 characters.

Purpose	Contents	Count	Qwen3 tokens
Training	Tech posts (Japanese, Nov 2025 - Jul 25, 2026)	722	2.62M
Learned?	Tech posts (Japanese, Jul 26 - Aug 22, 2026, not trained)	40	200k
Learned? (English side)	English versions of the same 40 posts	40	167k
Forgotten?	Japanese Wikipedia	132	301k
Forgotten? (English)	wikitext-2 test	1	299k

Token counts use the Qwen3 tokenizer.

Do the inserted layers start as identity

The LLaMA Pro paper cuts the 32-layer LLaMA2-7B into 8 groups of 4, copies the last block of each group, and appends it for 40 layers.
Qwen3-0.6B-Base has 28 layers, so I cut into 7 groups of 4 and added 7 blocks for 35 layers.

The copied blocks get their self_attn.o_proj and mlp.down_proj zeroed.
A Qwen3 block is two residual stages, x + attn(norm(x)) and h + mlp(norm(h)).
These two linears are the last ones on the side added to the residual, so with both zeroed the block passes its input straight through. It does nothing.

Anima-2.9B zeroed the outputs of self_attn.output_proj, cross_attn.output_proj, mlp.layer2, and adaLN, and these two are the same spots.
A Qwen3 block has no cross attention and no adaLN, so only two need zeroing.

I built a model with num_hidden_layers set to 35 in mlx-lm’s Qwen3 implementation and loaded the original weights under renumbered keys.
The mapping from old to new block numbers looks like this.
The inserted blocks are new numbers 4, 9, 14, 19, 24, 29, 34, each a copy of the block right before it.

Old	New	Old	New
0-3	0-3	16-19	20-23
4-7	5-8	20-23	25-28
8-11	10-13	24-27	30-33
12-15	15-18

Item	Value
Parameters	596.0M → 706.2M
Total for the 7 inserted blocks	110.1M
Zeroed tensors	14 (7 blocks × 2)

The pre-training check compared logits between the original and the 35-layer model on 4 sentences in Japanese and English, and the max difference was 0.0.
Even in bf16 it is exact, since only zeros get added.
Greedy generation (always picking the single highest-probability token, so the same model always produces the same text) also produced identical 30-token continuations.
This confirms, in actual numbers, the part of the Anima-2.9B article that said “With zero outputs the copy adds nothing to the residual stream, so the untrained 2.9B draws exactly what Anima-Base draws.”

Training setup

All three methods get the same data in the same order.
The 722 posts are joined into one stream separated by end-of-text (EOS) tokens, cut into 2553 chunks of 1025 tokens, shuffled with a fixed seed, and run at batch 8.
One epoch is 319 steps, two epochs are 638 steps, roughly 5.2M tokens total.

The learning rate warms up for 50 steps, then cosine-decays to 1/10.
Optimizer is AdamW (weight decay 0), gradients clipped at norm 1.0.

The training loop is hand-written with nn.value_and_grad instead of mlx-lm’s lora.py.
Freezing uses mlx-lm’s freeze() / unfreeze(), so gradients only flow to unfrozen parameters.

Method	What trains	Trainable params	LR
Block expansion	Only the 7 inserted blocks of the 35. Original 28 layers and embeddings frozen	110.1M	1e-4
LoRA	Rank 32, scale 2.0 adapters on `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` of the original 28 layers	20.2M	1e-4
Full FT	All parameters	596.0M	2e-5

Learning rates are just the usual values for each method.
The LLaMA Pro paper runs its new blocks at 2e-4, but that is with 4M-token batches per step; mine are 8k tokens, so I halved it.
Full FT moves every parameter, so it gets the commonly used 2e-5.

Pre-training perplexity looks like this.
For the training posts, I sampled 1 in 7 posts spread across dates and measured 300k tokens’ worth.

Eval set	PPL before training
Training posts (300k-token sample)	12.76
Held-out 40 posts (Japanese)	14.69
English versions of the 40	16.06
Japanese Wikipedia	13.54
wikitext-2	14.11

Measured as the mean over independent 1024-token windows.
Context gets cut at 1024, so these run higher than the PPL you see in papers.

Training speed and memory

Method	Time for 638 steps	Speed	Peak memory	Final loss
Block expansion (1e-4)	51.7 min	1685 tok/s	32.1GB	1.80
LoRA (1e-4)	51.3 min	1698 tok/s	38.7GB	2.05
Full FT (2e-5)	59.9 min	1453 tok/s	34.5GB	1.94
Block expansion (2e-5, extra)	51.6 min	1688 tok/s	32.1GB	2.03

Block expansion’s loss started at 2.77, sat around 2.2 by the end of epoch 1, and wandered between 1.6 and 1.9 in epoch 2.

A 20-step warmup run before the real one only managed 504 tok/s.
I estimated 4+ hours for two epochs and gave up on a third, and then the real run did 1680 tok/s from the start.
Peak memory was the same 32.1GB. No idea what caused the gap, but it does not matter much, so I am leaving it.

Full FT holds gradients and Adam state for all 596M, so it dropped to 1453 tok/s, but its 34.5GB peak was smaller than LoRA’s 38.7GB.
LoRA adds intermediate tensors for the adapter paths.

What it learned, what it forgot

Eval set	Before	Block exp (1e-4)	LoRA (1e-4)	Full FT (2e-5)	Block exp (2e-5)
Training posts (sample)	12.76	5.45	6.97	6.66	7.14
Held-out 40 posts (Japanese)	14.69	10.47	10.40	10.24	10.57
English versions of the 40	16.06	17.21	14.01	13.97	16.03
Japanese Wikipedia	13.54	17.91	16.40	16.92	16.07
wikitext-2	14.11	21.00	16.52	16.74	17.87

The rightmost column is an extra experiment added later (next section).

Block expansion went from 12.76 to 5.45 on the training posts, and from 14.69 to 10.47 even on the held-out newest 40.
The Anima, ComfyUI and LoRA vocabulary, and this blog’s writing style, were learned by the 7 inserted layers alone, without touching the original 28.

In exchange, Japanese Wikipedia went from 13.54 to 17.91 and English wikitext-2 from 14.11 to 21.00.
Even with the original layers frozen, the new layers add their deltas to the residual stream on text the model already knew.
I think the predictions on already-known text got worse because the training data was my posts and nothing else.

What the paper preserves is performance on general benchmarks, and that is with roughly 80B tokens of code and math mixed in; two passes over 2.6M tokens of my posts alone was not something freezing could protect against.
The English versions of the held-out posts also rose slightly, 16.06 to 17.21, so nothing learned in Japanese carried over to the English side.

LoRA trains 20.2M parameters, less than a fifth of block expansion, yet scored 10.40 on the held-out posts, nearly the same as block expansion’s 10.47.
On the training posts it reached 6.97, not as far down as block expansion’s 5.45.

LoRA forgets less, with Japanese Wikipedia at 16.40 and wikitext-2 at 16.52.
Closer to the original than block expansion’s 17.91 and 21.00.
The English versions dropped from 16.06 to 14.01, so here the content taught in Japanese did carry over to English.

LoRA’s training loss stayed higher than block expansion’s the whole way (2.14 vs 1.93 at step 400, 2.05 vs 1.80 final).
LoRA only adds small deltas to the original weights, so even with the same data the same number of times, it never got to full memorization.

Full FT dropped the most on the held-out posts at 10.24 and is lowest on the English versions at 13.97.
Its forgetting, Japanese Wikipedia 16.92 and wikitext-2 16.74, is about the same as LoRA’s.
Smaller than block expansion’s 17.91 and 21.00.

Under these conditions, block expansion with 28 frozen layers and 7 trained ones forgets already-known text more than full FT moving every parameter.
The learning rates differ 5x, 1e-4 for block expansion vs 2e-5 for full FT, so this table alone cannot say whether that is the method or the learning rate.

Rerunning block expansion at full FT’s learning rate

I dropped the learning rate to 2e-5 and ran block expansion again for the same 638 steps.
Everything else identical.

Eval set	Before	Full FT (2e-5)	Block exp (1e-4)	Block exp (2e-5)
Training posts (sample)	12.76	6.66	5.45	7.14
Held-out 40 posts (Japanese)	14.69	10.24	10.47	10.57
English versions of the 40	16.06	13.97	17.21	16.03
Japanese Wikipedia	13.54	16.92	17.91	16.07
wikitext-2	14.11	16.74	21.00	17.87

At the same 2e-5 as full FT, Japanese Wikipedia lands at 16.07, closest to the original of the four runs.
Smaller than full FT’s 16.92.
English wikitext-2, on the other hand, is 17.87, above full FT’s 16.74.

On the learning side, the held-out posts reach 10.57, not as low as full FT’s 10.24 or 1e-4’s 10.47.
The English versions sit at 16.03, about where they started.

Comparing at the same learning rate, block expansion rises less than full FT on Japanese Wikipedia, and more on wikitext-2.
Either way, at this scale there was no gap big enough to claim “the frozen original layers prevent forgetting.”

I think Japanese Wikipedia and wikitext-2 split like this because the training data is Japanese only.
The inserted layers add changes that fit the Japanese residuals, and changes the training never touched to the English residuals.
Full FT moves every layer a little, so the English-side damage does not concentrate in any particular layer.

Applying the 28-layer LoRA to 35 layers

I took the LoRA baked above (trained on the plain 28-layer Qwen3-0.6B-Base) and applied it to the 35-layer model in two ways.
As-is loads the adapter’s model.layers.N onto the same number N on the 35-layer side.
From block 4 on, the deltas get added to the wrong blocks.

Renumbered rewrites the key names with the old-to-new mapping table before loading.
The 7 inserted blocks get no adapters.
The same two ways as the Anima-2.9B article.

First, applied to the untrained 35-layer model (inserted layers still identity).

Application	Held-out 40 posts	Japanese Wikipedia
28 layers + LoRA (ground truth)	10.40	16.40
35 layers (untrained), as-is	14.32	14.26
35 layers (untrained), renumbered	10.40	16.40

As-is, the held-out posts sit at 14.32, barely below the plain model’s 14.69.
The LoRA is not doing anything.

Renumbered gives 10.401 and 16.403, matching the 28-layer + LoRA values to three decimal places.
The inserted layers add nothing, so with the numbering right, it is exactly the same model.
In the Anima-2.9B article, the character did not appear as-is and came back renumbered. This is that, in numbers.

Next, applied to the 35-layer model after training on my posts.

Application	Held-out 40 posts	Japanese Wikipedia
35 layers (trained), alone	10.47	17.91
35 layers (trained), as-is	11.00	19.78
35 layers (trained), renumbered	16.07	31.69

Here, renumbered is worse.
The held-out posts rise to 16.07, above the plain model, and Japanese Wikipedia degrades to 31.69.
The 35-layer model’s inserted layers and the LoRA both memorized the same 722 posts, so the more correctly they line up, the more the same-direction deltas get added twice.
As-is, the deltas land on shifted layers and get diluted, so the damage is smaller too.

In the Anima-2.9B case, the new layers had learned 1.7M general illustrations, and the LoRA’s character was content the new layers did not know.
What I did here is the new layers and the LoRA memorizing the same thing, the opposite of that article’s situation.

Does halving the scale halve the PPL hit

I halved the renumbered LoRA’s scale from 2.0 to 1.0 and applied it to the trained 35-layer model again.

Application	Held-out 40 posts	Japanese Wikipedia
35 layers (trained), alone	10.47	17.91
Renumbered, scale 1.0	12.09	22.69
Renumbered, scale 2.0	16.07	31.69

Halving roughly halves the degradation, but nothing beats the model alone.
A delta that memorized the same posts makes things worse in proportion to how much of it you add.

Applying a Wikipedia-trained LoRA

As content the 35-layer model does not know, I baked a LoRA on the plain 28-layer Qwen3-0.6B-Base with Japanese Wikipedia.
I took 1484 articles, 3.0M tokens, from the tail end of the same shard, avoiding overlap with the eval articles.
Training used the same rank 32, scale 2.0, LR 1e-4 for 300 steps (2.46M tokens).
This gets applied to the 35-layer model trained on my posts.

Application	Held-out 40 posts	Japanese Wikipedia
28 layers + Wiki LoRA	14.97	12.31
35 layers (trained), alone	10.47	17.91
35 layers (trained), as-is	10.55	17.58
35 layers (trained), renumbered	10.65	15.96

As-is, Japanese Wikipedia sits at 17.58, barely moved from 17.91 alone.
The Wikipedia content the LoRA learned is not coming through.

Renumbered, it drops to 15.96, while my posts stay at 10.65, nearly unchanged from 10.47.
The post content the new layers learned stays, and the Wikipedia content joins it.
Same shape as the Anima-2.9B article, where a renumbered character LoRA on top of new layers trained on 1.7M images brought the character out.

The drop is smaller than on 28 layers, though.
A LoRA that takes 13.54 down to 12.31 on 28 layers only manages 17.91 to 15.96 on 35.
The LoRA’s deltas were baked for the 28-layer model, so after the trained new layers go in, they apparently no longer fit exactly.

Did it forget because the model is small

Did it forget because it is 0.6B, or because not a single token of Wikipedia-like text was in the training data?
The inserted blocks add to the residual the same way whether the input is my posts or Wikipedia.
If the training data has no Wikipedia-side text, adding junk deltas there never raises the loss, so there is no reason for it to get fixed.
That has nothing to do with model size, so first I mixed in Wikipedia to check, then did the whole thing again on Qwen3-1.7B.

Mixing in 20% Wikipedia

I added 330k tokens each of Japanese Wikipedia and the English wikitext-2 train split (eval uses the test split) to the training data.
The Wikipedia articles were picked to not overlap the eval set.

That makes 3.28M tokens total with the Wikipedia side at 20.2%.
Still 638 steps, still 5.2M tokens run through.
Block expansion at LR 1e-4.

Eval set	Before	Block exp (1e-4)	Block exp (1e-4, 20% Wikipedia)
Training posts (sample)	12.76	5.45	6.22
Held-out 40 posts (Japanese)	14.69	10.47	10.50
English versions of the 40	16.06	17.21	15.10
Japanese Wikipedia	13.54	17.91	13.88
wikitext-2	14.11	21.00	13.18

The held-out posts are at 10.50, unchanged from 10.47 without the mix.
The learning side does not suffer from giving 20% of the data to Wikipedia.
Japanese Wikipedia is 13.88, about the pre-training 13.54, and wikitext-2 at 13.18 is below where it started.
Same frozen-original-plus-inserted-layers structure, but just putting 20% Wikipedia in the training data nearly removed the damage to already-known text.

That said, the mixed-in text comes from the same places as the eval sets (different articles of the same Wikipedia dump, a different split of the same wikitext-2).
In that sense this is the friendliest possible condition for replay.
That is why wikitext-2 ended below its starting point.
Still, 17.91 and 21.00 without the mix becoming 13.88 and 13.18 with it seems fair to call a data problem, not a size problem.

Same thing on Qwen3-1.7B

I gave Qwen3-1.7B-Base (28 layers, hidden 2048) the same every-4-layers insertion of 7 blocks for 35 layers.
1720.6M becomes 2072.9M, with the 7 inserted blocks at 352.4M.
Pre-training logit difference is 0.0 here too.

Training data, LR 1e-4, and the 5.2M tokens run through are the same as 0.6B.
I could not tell how far memory would go, so I set batch to 4 for 1276 steps.
With 3x the parameters of 0.6B, speed only fell 2.3x from 1690 tok/s to 745 tok/s, and peak memory was 24.8GB with the halved batch.
Training took 116.9 minutes.

Eval set	1.7B before	1.7B block exp (1e-4)	1.7B full FT (2e-5)
Training posts (sample)	9.30	3.87	5.27
Held-out 40 posts (Japanese)	11.15	8.63	8.28
English versions of the 40	12.76	13.16	11.16
Japanese Wikipedia	9.47	12.52	11.33
wikitext-2	10.33	14.27	11.87

Full FT keeps Adam state for all 1720.6M, so it fell to 612 tok/s and took 142.4 minutes, peaking at 30.4GB.

All the pre-training values are lower than 0.6B’s, so the comparison uses ratios against pre-training.

Eval set	0.6B block exp	1.7B block exp	0.6B full FT	1.7B full FT
Held-out 40 posts (Japanese)	-29%	-23%	-30%	-26%
Japanese Wikipedia	+32%	+32%	+25%	+20%
wikitext-2	+49%	+38%	+19%	+15%

Block expansion’s rise on Japanese Wikipedia is the same +32% on 0.6B and 1.7B.
wikitext-2 went from +49% to +38%, but that is a different order from what mixing 20% Wikipedia did (+32% to roughly zero).
Tripling the model barely changed how much the inserted layers hurt predictions on already-known text.

Full FT forgets less than block expansion on 1.7B too (+20% and +15%), the same ordering as 0.6B.
The 5x learning rate gap is also the same, so the method-vs-learning-rate question stays open at 1.7B as well.

On the training posts, block expansion went from 9.30 to 3.87, a bigger drop than on 0.6B.
With 352M in the inserted layers, 2.6M tokens can pretty much be memorized outright.

So it was not “it forgot because the model is small.”
Tripling the size left the ratios unchanged, and mixing in 20% Wikipedia nearly removed it.

Generation comparison

I gave the same 4 openers to the pre-training model, the 1e-4 block expansion, LoRA, and full FT, and let each continue for 60 tokens with greedy decoding at temperature 0.
0.6B trained on 5M tokens is not a model that answers facts.
Still, it shows roughly how the vocabulary and writing changed.
The outputs are Japanese, since the training data is my Japanese blog; the notes after each table cover what they show.
Line breaks written as two trailing spaces (Markdown’s line-break syntax, which this blog’s text is full of) are shown as ⏎.

Opener: 「Anima-Baseのテキストエンコーダは」 (“Anima-Base’s text encoder is”)

Model	Continuation
Before	、テキストエンコーダの定義を提供します。テキストエンコーダは、テキストデータをエンコードするためのアルゴリズムです。テキストエンコーダは、テキストデータをエンコードするためのアルゴリ
Block exp	Qwen3 0.6Bだった。Qwen3 0.6BはQwen3ベースだが、Qwen3 0.6BのテキストエンコーダはQwen3ベースなので、Qwen3ベースのLoRAはベースのQwen3
LoRA	Qwen3-VL-4Bで、Anima-BaseのベースモデルはQwen3-VL-4BではなくQwen3-VL-27B。⏎Qwen3-VL-4Bは27Bモデルの一部で、27Bモデルの
Full FT	、Anima-Baseのテキストエンコーダと同等の精度で、Anima-Baseの1.5倍の速度を実現している。⏎Anima-Baseのテキストエンコーダは1024トークンの長さで、An

Opener: 「ComfyUIでキャラLoRAを当てると」 (“Applying a character LoRA in ComfyUI”)

Model	Continuation
Before	、キャラの動きが悪くなることがある。これはなぜですか？また、キャラの動きが悪くなる原因は何ですか？さらに、キャラの動きが悪くなる原因の一つとして、キャラの動きが悪くなる原因の一つとして、キャラの動きが悪くなる原因の一つとして、
Block exp	、キャラの特徴がLoRAの学習素材に近づく。⏎キャラLoRAの学習素材は、キャラの特徴を強く保ったまま、LoRAの学習素材の崩れ方を抑える。⏎LoRAの崩れ方
LoRA	、キャラLoRAの効果が顔の特徴に集中する。⏎LoRAの効果は顔の特徴に集中するが、顔の特徴はLoRAの効果に集中するわけではない。⏎顔の
Full FT	、キャラの特徴がそのまま出る。⏎LoRAの効き方は「キャラLoRAの特徴をそのまま学習データに混ぜる」だけなので、LoRAの効き方を変えると効きが変わる。⏎LoRA

Opener: 「M1 Max 64GBのMacBook Proで」 (“On an M1 Max 64GB MacBook Pro”)

Model	Continuation
Before	、Mac OS X 10.11.6のアップデートを試してみた。Mac OS X 10.11.6は、Mac OS X 10.11.5のアップデートを含む。Mac OS X 10.1
Block exp	動かすのが現実的。⏎ただし、MacBook Proは「Macで動く」だけでは足りない。⏎LLMの推論コスト、GPUのメモリリーク、バックエンドの選択肢が揃っていないと、現実的な
LoRA	動くか、という話。⏎MacBook ProはM1 Maxの物理メモリを16GBに収めると、M1 Maxの64GBをそのまま使えない。⏎M1 Maxの物理メモリを16GBに収めると
Full FT	、M1 Max 64GBのMacBook Proで、M1 Max 64GBのMacBook Proで、M1 Max 64GBのMacBook Proで、M1 Max 64GBのMacBook Proで、M1 Max 64GB

Opener: 「Qwen-Image-Editで」 (“With Qwen-Image-Edit”)

Model	Continuation
Before	画像を画像に変換する方法を教えてください。はい、Qwen-Image-Editは、画像を画像に変換するためのAIモデルです。このモデルは、画像の内容や構造を変えることで、より良い画像を作成することができます。以下
Block exp	LoRAを当てたときの差分を読むと、LoRAの効きが強すぎてプロンプトを変えると効果が落ちる、という話になる。⏎LoRAを当てたときの出力は、プロンプトを変
LoRA	画像を画像に変換する。⏎画像を入力すると、画像の内容を画像に変換する。⏎画像の内容を変えると、画像の内容も変える。⏎画像を変えると、画像の内容も変える。⏎画像を変える
Full FT	画像を画像に変換する。⏎画像を画像に変換するには、画像を入力画像として渡す必要がある。⏎入力画像は画像生成モデルの入力として扱われる。⏎入力画像のサイズは、画像生成モデルの出力サイズ

Before training, everything comes out as polite です・ます explainer text or question lists, and the M1 Max opener leads to Mac OS X 10.11.
All three trained models switch to this blog’s plain 「〜だった」/「〜という話」 register, and the two-trailing-space line breaks start showing up.

Only block expansion continued Anima-Base’s text encoder with 「Qwen3 0.6B」, which is the correct fact, while LoRA said Qwen3-VL-4B and full FT drifted into accuracy-and-speed marketing copy.
Full FT collapsed into repeating the same phrase on the M1 Max opener.
This is one greedy sample, though, so it says nothing solid about block expansion knowing facts.

I ran the same 4 openers on the extra experiments too, and the other 3 behaved like the ones above: after training, only the style converges.
Below is just the first opener, the one with a checkable fact.