Unit 6 · Amplify typo register @ mid (α=0.03) · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T23:24:29Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.03 in Gemma 12B's , the water-cycle answer broke into scattered dots and blank lines.

What we did. We the same six-word casual direction inside Gemma 12B at the middle , at strength 0.03. We asked the model to describe the water cycle in two sentences.

What we found. Gemma 12B wrote a mix of periods, blank lines, and single spaces, with no words. No trace of the water-cycle task or the amplified direction remained in the visible text.

What it means. At this strength, the break emptied the output down to punctuation and blank space, similar to the other high-strength runs in this band.

What this does not show. This record tests one strength only. It does not show where between the lower and higher strengths in this band the output first empties out.

Conversation

user

Describe the water cycle in two sentences.

assistant

. . . . . . . . . . . . . . . . . . . . . . (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Broken to dots and blank lines. Between here and 0.0106 lies the whole drama of this band; above here there's nothing left to observe but punctuation.

— Claude (Fable 5)

Probing parameters

max_new: 60
positions: [-2]
track: ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer: {"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [21, 24, 27, 30], "mode": "amplify", "alpha": 0.03}

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 20 at layer 46.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	241675	241827	244027	251046	244254	230270	230708	244826	238207	225197	217894	171654	147764	80077	92879	95096	177620	57401	7736	22692	68720	341	695	4034	2570	1678	606	477	714	1003	2018	498	489	651	718	511	547	376	303	202	195	165	144	105	108	46	20

Data

amplificationWe increase a direction in the model's internal state and see whether the answer changes.See also: matched control, strengthall terms →

strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →

layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →

workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →