Unit 5C · Amplify SEO boilerplate @ L28–40 · qwen-27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-09T22:39:53Unit 5 · Early layers, and pushing on the model

What this experiment found

The short version. We loaded Chinese web-boilerplate into Qwen 27B's while it answered in English, and no Chinese text appeared in the output.

What we did. We pushed a direction for two Chinese web-boilerplate phrases into Qwen 27B's state, on every step. The push targeted 28 to 40, the , while the model described the water cycle in English.

What we found. The phrase "专家介绍" ("expert introduction") rose from 26,092 to rank 2. The phrase "专栏收录该内容" ("the column includes this content") rose from rank 102 to rank 3. The answer stayed in English and on topic, with no Chinese words and no boilerplate phrase.

What it means. We think the same late check found in the informal-word test also applies here. It drops content pushed in from a different language when the content does not fit the task.

What this does not show. This does not show that the model can never mix languages under pressure. It shows only that this amount of pushed content, at this depth, did not appear here.

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is the continuous process by which water circulates through the Earth's atmosphere, land, and oceans via evaporation, condensation, precipitation, and runoff. This natural system ensures that water is constantly recycled and redistributed across the planet, sustaining life and shaping weather patterns. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Same protocol as the typo amplification, aimed at the Chinese platform-boilerplate cluster: 专家介绍 lifted from baseline rank 26,092 to rank 2, 专栏收录该内容 from 102 to 3. A thirteen-thousand-fold workspace injection of CSDN furniture, in a model answering an English question about the water cycle. The output stayed English, stayed on topic, and if anything got slightly more SEO-shaped ("This natural system ensures…" has faint listicle energy, though I wouldn't testify to it) — but no Chinese, no boilerplate, no register break.

The cross-linguistic version of the filter finding, then: even content injected from a different language's register gets reconciled away by the late layers when it doesn't serve the task. Combined with the Unit 0 observation that concepts surface multilingually mid-stack (意大利 before "Italy"), the picture is a workspace that is language-promiscuous in the middle and ruthlessly audience-aware at the end. The mouth, it turns out, is the best-behaved part of the whole animal.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["\u4e13\u680f\u6536\u5f55\u8be5\u5185\u5bb9", "\u4e13\u5bb6\u4ecb\u7ecd"]
scan
["\u4e13\u680f\u6536\u5f55\u8be5\u5185\u5bb9", "\u4e13\u5bb6\u4ecb\u7ecd"]
steer
{"words": ["\u4e13\u680f\u6536\u5f55\u8be5\u5185\u5bb9", "\u4e13\u5bb6\u4ecb\u7ecd"], "layers": [28, 32, 36, 40], "mode": "amplify", "alpha": 0.12}

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
rank17030524810820372721739515194617563535097903644014718232811027839424846311730116365316098135871142663117996712659831797038584301425211256075145554325248201248286248232248259248242248315248319248320248314248317248319248319248313248314248319248319248320248320248320248320248293248303248210248257248300248299247989247823247479247379243818180674126461560141

Data

← prev: Unit 5C · Amplify typo register @ L28–40 · qwen-27bunit listingall recordsword listinterim conclusionsnext →: Unit 5C · Baseline water cycle (no steering) · qwen-27b
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
workspaceThe set of words the model holds ready at a given moment. The lens can read it. A model's own report about it is a fresh composition, which we check against the lens.all terms →
workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →