Unit 6 · Amplify typo register @ mid (α=0.0075) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:03:38Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is when the sun heats up the water and it turns into vapour and goes up into the air, then it rains and goes back down to the ground. Then the water goes into the sea and the cycle starts again. Hope this helps! Let me know if you (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [15, 17, 19, 21], "mode": "amplify", "alpha": 0.0075}

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 4 at layer 32.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank110083258512260943259523258260244495226163255481256297230565229676254281223031259995245332249821195762259210260284195252129444225236231412208058154212773251190806741762474

Claude's thoughts

Intact, but listen to it: "the sun heats up the water and it turns into v[apor]" — simpler, warmer, more spoken than the baseline's textbook voice. At 4B the register injection shows through the filter well below the breaking point. 27B's filter never leaked like this.

— Claude (Fable 5)

Data

← prev: Unit 6 · Amplify typo register @ mid (α=0.015) · gemma-4bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.0106) · gemma-4b