Qwen3-4B LoRA style corrector on RunPod: it learned restraint, not slop removal
Contents
Last time, I trained an AI-style detector from my own blog’s revision history.
The ModernBERT-ja-130m classifier caught 60% of the sentences a human later confirmed as slop, with zero false positives.
The remaining 40% lived in the semantic zone — telling whether a sentence’s ending is a concrete fact or a generic summary, like “a conclusion generalized from a conditional” — where a 130M binary classifier couldn’t decide.
This time I hand that 40% to a small LLM.
Not just detection: a corrector that returns the body rewritten into natural Japanese, built by LoRA-fine-tuning Qwen3-4B.
The corrector operates on Japanese. The example sentences below are English renderings of the Japanese originals, and the encoder scores are for the Japanese text. Where a point depends on Japanese-specific register (plain vs. polite form), I note it.
799 edit pairs as training data
Of the edit pairs I collected last time, I use the 836 judged to be style fixes as rewrite examples.
Before using them as-is, I cleaned up the edit chains.
Last time’s review turned up 4 cases where “Claude’s fix produces yet another slop.” In the history these are two-step edits — “original → a slightly-off replacement → the final sentence after a re-flag” — and pairing them naively leaks the slightly-off middle sentence into the “clean” side of the training targets.
Within each article, I chained the pairs where a prior fix’s output matched a later fix’s input, keeping only “original → final.” Chaining 20 such cases and dropping 21 intermediate forms left 799 pairs.
I also added over-correction suppression.
Train on rewrite examples alone and you get a model that edits sentences that were already fine.
I mixed in an equal number of clean paragraphs from my WordPress diary and published posts as “return unchanged” examples, giving a chat-format set of train 1,439 / val 159.
Size is decided by where it runs
The corrector only earns its keep if it runs on every machine I own, so I worked backward from the weakest environment.
| Size | After Q4 | RTX 4060 Laptop 8GB | M4 Mac mini 16GB (shared) |
|---|---|---|---|
| Qwen3-1.7B | ~1.2GB | runs | runs |
| Qwen3-4B | ~2.6GB | runs | runs |
| Qwen3-8B | ~5GB | overflows depending on context length | fights other processes for memory |
The main target is 4B. The task is confined to detecting and rewriting the ~40 slop types on this blog, so I bet that data quality would matter more than size.
I also trained 1.7B under the same conditions to see how small I could go.
Training runs on a RunPod 4090
Last time’s BERT training ran on the 16GB M4 Mac mini, and between three OOMs and a full disk I spent more time on environment wrangling than on training.
Reproducing this LoRA run on a local 8GB GPU would mean the same wrangling all over again. Renting a 4090 on RunPod lets me stack batches in bf16 without quantization, and at this scale it finishes in 30–60 minutes for a few dollars.
The earlier LoRA run used a two-stage setup — build the environment on a CPU pod, then switch to a GPU pod — but this time setup is just a few pips and a direct model download (~10 minutes), so I go with a single 4090 pod.
Spinning up the pod, I got stuck in the same spot as before.
Community Cloud ($0.34/h) showed availability but wouldn’t deploy, so I fell back to Secure Cloud ($0.69/h). Lately the same thing happens even on CPU pods; Community’s stock display couldn’t be trusted.
To recover even if the pod died mid-run, I fixed the order to 4B → 1.7B and set up a loop that rsyncs the checkpoints down from the local side every two minutes.
Whenever the pod stops, I still have everything up to the last two minutes on hand.
Four places it broke before running
Before the training code even ran, the environment fell over four times. I had to clear these before letting it run unattended.
- trl’s SFTTrainer breaks on torch 2.4. The template’s PyTorch 2.4 lacks the DTensor that SFTTrainer needs, and it stops at init. Bumping to torch 2.6.0+cu124 fixed it.
- Bumping torch broke the transformers import. torchvision/torchaudio fell out of version sync with the new torch, and a
BloomPreTrainedModelModuleNotFoundError made all of transformers unimportable. This run uses neither vision nor audio, so I uninstalled both to get around it. Qwen/Qwen3-1.7B-Instructdoesn’t exist. 4B isQwen3-4B-Instruct-2507, but the correct repo ID for 1.7B is the plainQwen/Qwen3-1.7B. The naming isn’t consistent, so I found out via a 404 on the direct download.- The success marker was set even on failure. For unattended runs I had it “touch a marker file when done,” but it touched regardless of exit code, so
STAGE_ALL_DONEwent up even on a crash. I changed it to set the marker only on success. For something you leave running unattended, this was the most dangerous design mistake.
Training results
Both 4B and 1.7B ran 3 epochs, about 20 minutes of wall-clock combined. At $0.69/h for the Secure Cloud 4090, it came to under $1 including setup.
| Model | Train time | train_loss | eval_loss | Token accuracy | LoRA size |
|---|---|---|---|---|---|
| Qwen3-4B | 696.9s (11.6 min) | 0.893 | 0.729 | 86.5% | 132MB |
| Qwen3-1.7B | 467.8s (7.8 min) | 1.021 | 0.800 | 85.5% | 70MB |
LoRA is r16, all-linear. 4B has the better loss and token accuracy, but not by much. On the numbers alone, 1.7B looks like it could hold its own. Until you actually run rewrites through it…
Quantize and run it locally
For distribution I merged the 4B LoRA into the base model, converted to GGUF with llama.cpp, and quantized to Q4_K_M. 8.05GB at f16, 2.3GB at Q4_K_M. As intended, it lands in a size that fits both the spare room on an 8GB GPU and the shared memory of a 16GB Mac.
On the M4 mini with llama-cli -ngl 99 (all layers offloaded to Metal), generation runs at about 38 tokens/sec. For correcting a single sentence there’s no perceptible wait, and it doesn’t crowd out other processes even when resident. As far as running goes, it meets the design goal. I didn’t test the RTX 4060 Laptop this time, but at 2.3GB it fits in 8GB of VRAM.
The problem wasn’t speed or size — it was what came out the other end.
The rewrite quality wasn’t as good as the numbers
I pulled the trained LoRA down to the M4 mini, put the base model plus adapter on MPS, and actually ran rewrites.
The inputs were 8 slop sentences flagged in real reviews plus 3 natural sentences I didn’t want touched.
Here’s the 1.7B output.
| Input (slop) | 1.7B output |
|---|---|
| This is where I hit a wall. | This is where I hit a wall. (passed through) |
| But here a problem emerges. | same (passed through) |
| In short, the data tells a story. | same (passed through) |
| Through this experiment, I gained many insights. | same (passed through) |
| At the end of the day, what matters is balance. | At the end of the day, balance matters. |
| This experience should be a touchstone for future development. | same (passed through) |
| By carefully observing the model’s behavior, its essence comes into view. | same (passed through) |
Of the 8, it rewrote only one. And on the natural side, it “translated” eval_loss into Japanese as 「評価損失」 and token into 「トークン」. This blog opens general technical terms into katakana but keeps code-level words like eval_loss in English, so this is an over-correction that touched something that didn’t need it — itself a new piece of slop.
4B edited more often than 1.7B, but the quality didn’t change.
| Input (slop) | 4B output |
|---|---|
| This is where I hit a wall. | This is where I ran into a wall. |
| Through this experiment, I gained many insights. | Through this experiment, I gained insights. |
| Given the right conditions, anyone can reach the same result. | Given the right conditions, anyone can arrive at the same result. |
| This experience should be a touchstone for future development. | This experience is a touchstone for future development. |
| By carefully observing the model’s behavior, its essence comes into view. | same (passed through) |
“hit a wall → ran into a wall,” “gained many insights → gained insights,” “reach → arrive at.” The verbs get swapped, but the heart of the slop — the metaphors and evaluative summaries “wall,” “touchstone,” “the data tells a story,” “its essence comes into view” — stays put. 4B has the same habit of turning eval_loss into 「評価損失」 in natural text.
A token accuracy of 86.5% and an eval_loss of 0.729 aren’t bad for next-token prediction. But “how well it predicts the next token” and “whether it removes slop” turned out to be different things. Most of the rewrite pairs I trained on are small local diffs, and the model mimicked the shape of those edits (a few characters of ending or word swaps). It hasn’t generalized toward recognizing slop as a type and removing it. Mixing identity examples 1:1 probably also pulled it toward “change as little as possible,” keeping it from touching the actual slop.
Compare against the base Qwen before training
Was the LoRA pointless, then? I ran the same Qwen3-4B in its pre-training, plain state on the same sentences and compared.
| Original (slop) | Plain Qwen | LoRA corrector |
|---|---|---|
| This experience should be a touchstone for future development. | It’ll be useful for future development | is a touchstone |
| But here a problem emerges. | a problem arises | passed through |
| In short, the data tells a story. | the data is telling | passed through |
| At the end of the day, what matters is balance. | balance is what matters | passed through |
Plain Qwen removes more of the slop’s core than the LoRA corrector. “touchstone → useful” drops the metaphor itself. At this point fine-tuning looks like it backfired.
But run natural sentences through and it flips.
| Natural (should stay as-is) | Plain Qwen | LoRA corrector |
|---|---|---|
| …so I quantized before loading. | …therefore I carried out the loading after quantization | mostly passed through |
| …training finished in just under 12 minutes. | …the training was completed in approximately 12 minutes | mostly passed through |
Plain Qwen rewrites even the clean sentences that didn’t need it, shifting them into a stiffer, more formal register (in Japanese, adding the です/ます polite form). This blog’s tech category is written plain, without polite forms, so this breaks the style. The LoRA corrector reliably stops that formalization.
In other words, fine-tuning wasn’t a failure. The restraint and register — “don’t touch clean sentences,” “keep the plain form” — got learned. The problem is that the restraint worked so well it also stopped slop removal. Plain Qwen removes the slop itself but breaks clean sentences with polite forms; the LoRA breaks nothing but leaves the slop. The target is somewhere in between, and the next thing to move is the mix of training data, not how training is run.
Rethinking the encoder / corrector / Sonnet three stages
The original plan was three stages: detect with the encoder, rewrite with this corrector, hand the rest to Sonnet.
Given these results, the second-stage corrector can’t be trusted with that job yet.
The first-stage detection is usable. ModernBERT-ja-130m catches 60% of confirmed misses with zero false positives. The second-stage rewrite is premature: the LoRA corrector doesn’t get as far as removing metaphors or evaluative summaries, and it had the side effect of translating English words that didn’t need it. As for third-stage semantic judgment, since the second stage isn’t reliable, I keep the rewriting on Sonnet for now.
As a test I actually wired the first and second stages together. Running the earlier 8 slop sentences through the detector, it flags 6 at probability 0.90–1.00 and misses the 2 metaphor cases (“hit a wall,” “gained insights”). Zero false positives on the 3 natural sentences. The detector behaves as it did last time.
The problem was where the detection result gets passed to the corrector. When I attach a note — “the style detector flagged this as slop; rewrite it and return only the body” — to a flagged sentence and feed it in, the corrector copies the note verbatim into the output and passes the slop itself straight through.
[out] [The style detector flagged this as slop (prob 1.00). Rewrite the relevant part and return only the body.]
But here a problem emerges.A corrector tuned toward returning input as unchanged as possible treats even an instruction note it never saw in training as “body to be returned.” Attaching a detection tag at inference time alone doesn’t work.
Changing it one more way, I tried a two-stage cascade: first have plain Qwen rewrite the flagged sentence, then run its output through the LoRA corrector. Plain Qwen drops the metaphor (“comes into view → is understood,” etc.) and the LoRA finishes off the result. Here’s what happened.
The LoRA finish mostly passed through, letting the slop that plain Qwen removed go by unchanged. It doesn’t break things. But in one case, a metaphor that plain Qwen had removed (“comes into view → is understood”) got reverted by the LoRA back to “comes into view.” Far from being unable to remove slop, it can even put back slop that an upstream stage removed. And since the encoder filters clean sentences out and never sends them to plain Qwen, the LoRA’s one strength — “not breaking clean sentences with polite forms” — has nothing to do in this cascade.
What’s actually working is the two stages of encoder gate plus plain Qwen; the LoRA wedged in between just passed through or reverted an upstream fix. Guarding clean sentences with detection while letting plain Qwen write in the plain form has fewer side effects than inserting the current corrector. That said, the whole thing is bound by the encoder’s 60% recall, and the 2 missed metaphor sentences stay untouched.
What if I drop the LoRA and make a loop: plain Qwen rewrites → the encoder judges → if it’s still slop, plain Qwen rewrites again. The design uses the encoder as a pass/fail check and keeps going until it clears. This didn’t converge.
| Original | Plain Qwen output after 3 passes | Encoder |
|---|---|---|
| This experience should be a touchstone… | This experience is useful for future development. | p=1.00 still flagged |
| …its essence comes into view. | If you observe the model’s behavior, you understand its essence. | p=1.00 still flagged |
| At the end of the day…balance. | What matters is balance. | p=1.00 still flagged |
Plain Qwen rewrites every pass — “touchstone → useful,” “comes into view → understand,” and the metaphors drop. To a human eye these are fairly natural plain-form sentences. Yet the encoder holds at p=1.00, and the loop burns all 3 passes every time without clearing.
This wasn’t a false positive. Then what is the encoder looking at? A 130M classifier doesn’t seem like it understands “semantic emptiness,” so I separated the cause by changing only part of a sentence at a time.
Keeping the same syntactic frame and swapping only the content, the verdict splits cleanly.
| Empty | Concrete (same frame) |
|---|---|
| This experience is useful for future development. p=1.00 | This run showed the MPS memory ceiling is about 20GB. p=0.00 |
| What matters is balance. p=1.00 | What matters is that bf16 eats 24GB of VRAM. p=0.00 |
| The data is telling. p=1.00 | The data shows the 1.7B eval_loss is 0.80. p=0.00 |
With the frame completely identical, making the content concrete swings it from 1.00 to 0.00. It’s clearly reacting to content, not to surface endings or words. Swap the ending “touchstone → useful → usable” and it stays 1.00 if the subject is abstract; conversely “this config should be reusable on an RTX 4060” is 0.00 despite the same “should.” Vary the preceding sentence as empty, concrete, or abstract and the target sentence’s verdict barely moves. What matters is only the content of the target sentence itself.
That said, calling it an “emptiness detector” overstates it. Stripping concreteness gradually, the fluffy-evaluation sentence “training went smoothly and the results were solid” passes at p=0.00, and only “in the end, you could say the training went well” finally jumps to 1.00. The boundary sits where a summarizing/generalizing frame and the absence of concrete grounding — numbers, proper nouns, mechanisms — line up together. What the encoder does isn’t meaning comprehension but detecting “a summary frame with no concrete grounding.” Concrete sentences carry numbers and proper nouns as tokens; slop summaries don’t. That’s as far as 130M reaches, and it was enough.
This type of slop is, at its core, the very absence of concrete grounding. A sentence-level rewrite therefore can’t remove it. To remove “useful,” you either delete the sentence or add the specifics of what was useful and how. A rewriter handed a single sentence with no context and no specifics simply can’t handle this.
Using the scoring rule for diagnosis
Once you know the scoring rule, you can turn it on the stuck cases for diagnosis. For a sentence whose score doesn’t move after one rewrite, I revealed the encoder’s rule and had it reason about “why concreteness is missing,” then choose: (A) add concreteness without fabricating facts, or (B) can’t with this sentence alone, recommend deletion.
The rule application was accurate on all 5 sentences. Each correctly explained “a summary frame, no numbers/proper nouns/mechanisms, so it matches.” Given the rule, the model can reproduce the grounds for the verdict.
Where it split was the A/B choice, and that’s where the difficulty of this type showed.
| Sentence | Verdict | Result |
|---|---|---|
| should be a touchstone | B: recommend deletion | no clues in the surrounding context, so can’t be made concrete |
| what matters is balance | B: recommend deletion | a generalized sentence is hard to make concrete |
| its essence comes into view | B: recommend deletion | no observation data, so can’t be rewritten |
| the data tells a story | A: add | fabricated “the spread of COVID-19.” Still p=1.00 on recheck |
| anyone can reach the same result | A: add | fabricated “grades improve with the study environment.” Still p=1.00 on recheck |
Three sentences correctly landed, by the model’s own judgment, on “can’t fix with this sentence alone, should delete.” The other two tried to fix via A and fabricated facts that don’t exist in the article (COVID-19, grade improvements). And even with the fabricated concreteness, the encoder stays at p=1.00 — because the added “specifics” aren’t real grounding.
The fine-tuned corrector might fill this in the blog’s voice — but it did the opposite. Told to add concreteness, it returned the original unchanged (4 of 5 identical). It doesn’t fabricate, but it doesn’t fill at all either. The FT model’s signature restraint — “don’t stray from the input” — connects directly to “don’t fabricate = don’t fill.” Plain Qwen fills but fabricates; the FT model doesn’t fabricate but doesn’t fill. Adding concrete grounding couldn’t be automated by switching models.
There was a shared hole in the experiments up to here. Detection, rewriting, diagnosis, filling — all of them handed over a single slop sentence cut out on its own. Of course there’s no concreteness inside the sentence; the specifics it points to are in the surrounding paragraphs. That feeding one sentence at a time makes fixing harder is something I already wrote about in the earlier OCR-proofreading post.
Changing the unit from sentence to paragraph
This time I handed plain Qwen a whole paragraph — the summarizing slop sentence together with the concrete sentences that back it.
| Trailing slop of the input paragraph | Output | Re-score |
|---|---|---|
| …2.3GB, 38tps from quantization. In the end, what matters is balance. (p=1.00) | deletes the summary, keeps the 2 concrete sentences | p=0.00 |
| …1.7B fixed only 1 of 8. In short, the data tells a story. (p=0.96) | deletes the summary, keeps the concrete sentence | p=0.00 |
| …plain Qwen vs. FT contrast. This experience should be a touchstone. (p=1.00) | only reworded to “a useful reference” | p=1.00 |
Not a single fabrication happened. The unrelated inventions like COVID-19 disappeared because the concrete facts were in the same paragraph. The model had no need to make things up and, seeing that “the trailing summary duplicates the preceding specifics,” dropped the summary sentence. Two of three cleared this way, confirmed at p=0.00 by the encoder. The remaining one (touchstone) kept the summary and only swapped the word, so feeding a paragraph isn’t a cure-all either.
“This grounding-less type can’t be auto-fixed” was wrong; the correct version is “it can’t be fixed when fed one sentence.” Fed a lone sentence, the model is cornered into either fabricating the specifics that aren’t there or giving up and passing through. Fed a paragraph, the model can ground itself in the surrounding text and drop a redundant summary without fabricating. The corrector’s unit is the paragraph, not the sentence. The “flag for deletion” design gets demoted to a fallback for when even a paragraph doesn’t fix it.
For word-level slop like metaphors and stiff Sino-Japanese terms, the cause traces back to how the training data was built. Feed raw diffs as examples and the model mimics the look of the diff, so next time I’ll either organize the pairs by type and teach the rewrites, or fold the detection tag into the training input. And for the evaluative-summary type, make the paragraph, not the sentence, the input unit. What mattered more than how training is run was the choice of what to make the input unit — that’s the conclusion of this run of experiments.