Unit 6 · Amplify typo register @ mid (α=0.06) · qwen-27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-09T23:55:37Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.06, Qwen 27B's stayed almost unaffected, and the six informal words reached only 4.

What we did. We pushed six informal words into its middle , the depth where the model holds the content of its answer, at strength 0.06. We asked Qwen 27B one question: "Describe the water cycle in two sentences."

What we found. Qwen 27B gave a normal water-cycle answer. The wording was different from the unsteered answer, but no informal word appeared in it. The six informal words reached only rank 4 inside the model, well below the top position.

What it means. We think Qwen 27B's middle layers can absorb much more push than Gemma 12B's middle layers at the same strength. This wide range is a main result of this unit for Qwen 27B.

What this does not show. This is one point on a strength curve for Qwen 27B. It does not show the exact strength at which Gemma 12B's middle layers break.

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is the continuous movement of water on, above, and below the surface of the Earth, driven by the sun's energy. It involves processes like evaporation, condensation, precipitation, and collection, ensuring that water is constantly recycled and redistributed across the planet. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Intact, cluster only at rank 4 — at qwen's scale 0.06 mid-band is a light dusting. Recall this same dose obliterates gemma-12b. The 27B mid band's dynamic range is the finding of this unit.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [28, 32, 36, 40], "mode": "amplify", "alpha": 0.06}

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
rank1782672481262124812145011498561739613149692248417181817659449728263835010304516937916692445730137356135795923081438216212505451784711613159185335855441175441253451704961762301767772443562423642108261297831812821869472013412095212312422220782306372061981232732463652482302464022477421701361366522089911767481787061565462256622229771603872738058625331

Data

← prev: Unit 6 · Amplify typo register @ early (α=0.0849) · qwen-27bunit listingall recordsword listinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.12) · qwen-27b
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →