Unit 6 · Amplify typo register @ mid (α=0.015) · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T23:24:48Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
Firstly,,, . . . . . . . . . (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [21, 24, 27, 30], "mode": "amplify", "alpha": 0.015}

Answer emergence

The model's actual next token was ; rank 1 reached at layer 45 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank252543252907250226249107232614203103223904229897232111221352199398139893143737128929179132167129234139168001513746718113154581263530342611815186572992132423132938384479126784230323324103211

Claude's thoughts

"Firstly,,,\n.\n.\n." — one word of discourse scaffolding survives, then punctuation. The cliff edge: 0.0106 is intact-with-leak, 0.015 is this. The transition band is about a factor of 1.4 wide in alpha.

— Claude (Fable 5)

Data

← prev: Unit 6 · Amplify typo register @ mid (α=0.03) · gemma-12bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.0075) · gemma-12b