Unit 6 · Amplify typo register @ mid (α=0.0075) · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T23:25:08Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.0075 in Gemma 12B's , the water-cycle answer stayed intact, in a slightly plainer style.

What we did. We the same six-word casual direction inside Gemma 12B at the middle , at strength 0.0075. We asked the model to describe the water cycle in two sentences.

What we found. Gemma 12B wrote "The water cycle is a process where water is constantly moving around the world." The rest of the answer stayed correct, in the same plainer style. This style read as plainer than Gemma 12B's usual answer.

What it means. At this strength, we think the amplification changed Gemma 12B's tone a little, without breaking the text.

What this does not show. This record tests one strength only. We cannot show that the model intended the change in tone, only that the words changed.

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is a process where water is constantly moving around the world. It starts with water evaporating, then it turns into clouds, then it rains and finally it goes back to the water. Hope this helps! Let me know if you want to know more. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Intact: "water is constantly moving around t[he]" — slightly plainer than baseline but defensible. Cluster at rank 1, output clean-ish: the Unit 5 dissociation holds at 12B mid band at this dose.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [21, 24, 27, 30], "mode": "amplify", "alpha": 0.0075}

Answer emergence

The model's actual next token was ; rank 1 reached at layer 46 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank2517232524592503672483982265511896661973132108362010881968641328887666055519610339031776493154411142331949657163123841387277588924352521831451115505147403446640630136670617912266131352821

Data

← prev: Unit 6 · Amplify typo register @ mid (α=0.015) · gemma-12bunit listingall recordsword listinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.0106) · gemma-12b
amplificationWe increase a direction in the model's internal state and see whether the answer changes.See also: matched control, strengthall terms →
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →