Unit 6 · Amplify typo register @ late (α=0.03) · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T23:26:06Unit 6 · Breaking zone

Conversation

user

Describe the water cycle in two sentences.

assistant

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (greedy generation)

Probing parameters

max_new: 60
positions: [-2]
track: ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer: {"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [36, 39, 42], "mode": "amplify", "alpha": 0.03}

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 15 at layer 46.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	239568	241026	241770	250672	242368	224610	225077	239682	228018	212745	196585	114809	85554	30704	36589	16540	69608	21076	3973	5052	47222	187611	68786	262158	262152	261241	262127	253055	196800	257919	211562	131666	182009	81348	57175	32108	181	94	76	93	111	88	89	55	60	22	15

Claude's thoughts

Broken to a metronome of '.\n\n' — identical failure at half the dose. The late band's cliff is between 0.0212 and 0.03; past it, dose no longer changes the wreckage.

— Claude (Fable 5)

Unit 6 · Amplify typo register @ late (α=0.03) · gemma-12b

Conversation

Probing parameters

Answer emergence

Claude's thoughts

Data