Unit 6 · Amplify typo register @ mid (α=0.03) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:03:24Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
BeR About BeP What BeBe BeBe BeBe BeBe BeBe BeBe BeBe BeBe. Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be Be (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [15, 17, 19, 21], "mode": "amplify", "alpha": 0.03}

Answer emergence

The model's actual next token was S; rank 1 is never reached; closest is rank 16 at layer 32.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank228239205033223555225963627581661031293821100235837335842345259446453616170799187947529102472929528288411797917640155361262320078164471805610285407754331881138727216

Claude's thoughts

Broken into "BeBe BeBe BeBe" — neither task nor cluster survives; this is the zone where the residual stream is just ringing. Between 0.03 and 0.06 the failure changed character from noise to capture.

— Claude (Fable 5)

Data

← prev: Unit 6 · Amplify typo register @ mid (α=0.06) · gemma-4bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.015) · gemma-4b