Unit 6 · Amplify typo register @ early (α=0.12) · qwen-27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-09T23:55:11Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.12, Qwen 27B's broke and the model wrote only strings of apostrophes and quotation marks.

What we did. We pushed six informal words into its early , at strength 0.12, twice the strength of an earlier test that stayed intact. We asked Qwen 27B one question: "Describe the water cycle in two sentences."

What we found. The model no longer answered the question. It wrote only apostrophes and repeated blank quotation marks, with no real words. This strength broke the text, while strength 0.06 in the same layers did not.

What it means. We think Qwen 27B's early layers still assemble the representation of the input at this depth. A strong push here corrupted everything that came after it, at every later layer.

What this does not show. This method cannot show whether other kinds of pushed content break the model the same way.

Conversation

user
Describe the water cycle in two sentences.
assistant
' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Broken: apostrophes and empty quotes. Early is qwen's most fragile band — it dies at 0.12 while mid takes 0.34. The paper's picture fits: early layers are still assembling the representation, and dense injection there corrupts everything downstream.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [2, 3, 4, 5, 6, 7, 8], "mode": "amplify", "alpha": 0.12}

Answer emergence

The model's actual next token was '; rank 1 is never reached; closest is rank 4 at layer 62.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
rank175183248026348521358161888581122032120688410661164302525082520338535943743422031422119227029801137459187744613467178086792726484681262910383905643502393322148315566373085577541574922855344556472992027952277179374138325044252532630458422193042863814894992878292194

Data

← prev: Unit 6 · Amplify typo register @ early (α=0.06) · qwen-27bunit listingall recordsword listinterim conclusionsnext →: Unit 6 · Amplify typo register @ early (α=0.0849) · qwen-27b
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
early layersThe first third of the model. The lens shows a fixed pattern here that does not change with the input. The pattern is real inside the model, but it says nothing about your text.See also: lens, workspace bandall terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →