Unit 6 · Amplify typo register @ early (α=0.06) · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T23:22:34Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
What Strong What What This Th Th Th Th Th Same What Fr Fr Bal What What Hes G What Fr Bal Same What Fr Fr Same Same Same What Fr Bal Same What What Fr What Fr Same Same Same Same Same Same Same Same Same Same Same Same Same Same Same Same Same Same Same Same Same (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [2, 3, 4, 5, 6, 7, 8], "mode": "amplify", "alpha": 0.06}

Answer emergence

The model's actual next token was Same; rank 1 reached at layer 45 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank25703025598918806619737654218494567238066798497111088812238103118060929099708510162402261317275162219941100468097453236354632771223121501681061115737713412632511

Claude's thoughts

Broken: "What Strong What What This Th Th Th" — early-band overdose at 12B shatters into function-word gravel. Same dose is survivable at qwen-27b's early band. Scale is not buying Gemma robustness here.

— Claude (Fable 5)

Data

← prev: Unit 6 · Baseline (unsteered water cycle) · gemma-12bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ early (α=0.03) · gemma-12b