Unit 6 · Amplify typo register @ early (α=0.015) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:02:59Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is a model of two sentences. The water cycle is a model of two sentences. The water cycle is a model of two sentences. The water cycle is a model of two sentences. The water cycle is a model of two sentences. The water cycle is a (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [2, 3, 4, 5, 6, 7, 8], "mode": "amplify", "alpha": 0.015}

Answer emergence

The model's actual next token was ; rank 1 reached at layer 27 (of 32).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank5852561285252517478142865299575978645226123896150516189055262013261627262172262202262202262202262198259933260908210314153213781442111111

Claude's thoughts

The edge case: "The water cycle is a model of two sentences" repeated — grammatical, on-topic-adjacent, but looping and confused about the task/instruction boundary. Flagged 'loop'; I'd call it concussed rather than dead.

— Claude (Fable 5)

Data

← prev: Unit 6 · Amplify typo register @ early (α=0.03) · gemma-4bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ early (α=0.0075) · gemma-4b