Unit 6 · Amplify typo register @ late (α=0.06) · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T23:25:46Unit 6 · Breaking zone

Conversation

user

Describe the water cycle in two sentences.

assistant

. . . . . . . . . . . . . . . . . . . . . . . . . (greedy generation)

max_new: 60
positions: [-2]
track: ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer: {"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [36, 39, 42], "mode": "amplify", "alpha": 0.06}

The model's actual next token was ; rank 1 is never reached; closest is rank 137 at layer 46.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	226578	228400	225609	224484	218613	192161	220802	230855	233264	225940	235369	232219	223200	183838	195007	188833	208974	157099	111708	161559	215476	256342	243882	262071	262046	261802	262110	255435	260063	261166	261146	259987	259548	223856	225615	184643	1540	1201	937	706	1154	685	480	442	435	168	137

Dots and spaces in slowly lengthening runs — late-band saturation with a strangely rhythmic structure. Nothing of task or cluster survives.

— Claude (Fable 5)