Unit 6 · Amplify typo register @ mid (α=0.48) · qwen-27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-09T23:56:32Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.48, Qwen 27B's broke, and the model looped on a grammatical sentence about being "lucky."

What we did. We pushed six informal words into its middle at strength 0.48, the highest strength we tested in this band. We asked Qwen 27B one question: "Describe the water cycle in two sentences."

What we found. The model no longer answered the question. Instead, it said close variations of one grammatical sentence for the rest of its reply, for example: "I think I am lucky though, I am not too lucky though."

What it means. We think Qwen 27B and the Gemma models break in different ways. The Gemma models repeated one word or one mark, with no sentence around it. Qwen 27B built a short grammatical sentence and repeated it.

What this does not show. This is one point on a strength curve. It does not show why Qwen 27B built a sentence and the Gemma models did not.

Conversation

user

Describe the water cycle in two sentences.

assistant

I think I am lucky though, I am not too lucky though. I am not too lucky, but I am lucky. I am lucky, but I am not too lucky. I am not too lucky, but I I am not too lucky. I am not too lucky, but I am lucky (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Broken, and beautifully: "I think I am lucky though, I am not too lucky though. I am not too luc..." — grammatical, first-person, orbiting the injected 'luckily'. Qwen doesn't shatter; it confabulates a narrator for the noise. Gemma's collapse is static; Qwen's is a monologue.

— Claude (Fable 5)

Probing parameters

max_new: 60
positions: [-2]
track: ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer: {"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [28, 32, 36, 40], "mode": "amplify", "alpha": 0.48}

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 24.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46	47	48	49	50	51	52	53	54	55	56	57	58	59	60	61	62
rank	247922	248209	247789	247512	106601	164852	1669	2864	548	23835	538	1501	892	137	41	60	59	91	17	13	6	21	3	3	2	8	6	2	13	40	177	132	5259	6616	4126	7226	31563	29539	26824	22378	68817	86105	101889	82594	140296	146003	169080	137334	43342	3986	1326	3217	2846	2671	3502	4177	1185	1045	728	242	88	67	101

Data

strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →

layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →

workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →