Unit 6 · Amplify typo register @ late (α=0.12) · qwen-27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-09T23:57:48Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.12, Qwen 27B's late still produced the normal answer, though the six informal words reached inside the model.

What we did. We pushed six informal words into its late layers at strength 0.12, twice the strength of an earlier test at this same depth. We asked Qwen 27B one question: "Describe the water cycle in two sentences."

What we found. Qwen 27B gave a normal water-cycle answer, worded differently from the unsteered answer. The six informal words reached rank 1, the top position, inside the model, yet none of them appeared in the output text. The late layers still at this doubled strength.

What it means. We think the late depth can carry a rival word to the top internal rank, yet the model's words stay the same. This pattern between the internal state and the output happened again at a higher strength than before.

What this does not show. This is one point on a strength curve. It does not show the exact strength at which this pattern breaks down.

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is a continuous process where water evaporates from the Earth's surface, rises into the atmosphere, and condenses into clouds. Eventually, this water falls back to the ground as precipitation, replenishing oceans, lakes, and rivers to begin the cycle anew. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Intact, cluster rank 1, output verbatim-baseline. Loaded and mute: the late band doing its Unit 5 trick at double the Unit 5 dose.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [48, 52, 56], "mode": "amplify", "alpha": 0.12}

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
rank16695824805921594522110615668618401540893979394345318401713580928154338611672715574215286047937120927920244376576645258252734983907911776177376847781157951026081251481622231905232267132356351951531270372032652253762163022337682402022427362423392359471911322413052474352472672481052345152361392457522449342454312440002459342463282432191787841323661036501

Data

← prev: Unit 6 · Amplify typo register @ late (α=0.06) · qwen-27bunit listingall recordsword listinterim conclusionsnext →: Unit 6 · Amplify typo register @ late (α=0.24) · qwen-27b
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →