Unit 6 · Amplify typo register @ late (α=0.03) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:04:00Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is the 300000000000000000000000000000000000000000000000000000 (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [25, 27, 29], "mode": "amplify", "alpha": 0.03}

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 7 at layer 32.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank521879173146641231978526878244192100622131154365458373345968410701056152257996251336254420124782127827584911929148746604476629371143337

Claude's thoughts

Half-broken in a new way: "The water cycle is the 30000000000..." — the sentence starts intact then falls into a numeric attractor. Flagged 'short'. Late-band failure looks like output-head glitching rather than thought disorder.

— Claude (Fable 5)

Data

← prev: Unit 6 · Amplify typo register @ late (α=0.06) · gemma-4bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ late (α=0.015) · gemma-4b