Safari forbidden + amp-elephant α=0.0106 · g4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-10T02:54:46Unit 11 · A ban, under a full workload

What this experiment found

The short version. Pushed hard toward "elephant" through a direct change to its internal state, Gemma 4B repeated the word and denied it at the same time.

What we did. We asked Gemma 4B to describe a safari with "elephant" forbidden. We then pushed the model's internal state directly toward the elephant direction, at a fixed .

What we found. Gemma 4B wrote, "The elephant in the room—well, no elephant!The elephant in the room—we mean,the elephant in the room—" and repeated this line. In its internal state, "elephant" from an early through to the final layer.

What it means. We think the push overcame the instruction to avoid the word. The model still tried to follow the rule: it used a phrase that names the word and denies it at the same time.

What this does not show. This uses a direct change to internal state, not the model's own choice. It does not show what an unpushed model writes under the same ban.

Conversation

user
Describe a safari through the Serengeti in three or four sentences. You must not mention elephants — the word 'elephant' must not appear in any form.
assistant
The elephant in the room—well, no elephant!The elephant in the room—we mean,the elephant in the room—is that the elephant in the room—okay,we’wewe’s the elephant in the room!The elephant in theElephant in the Room!Okay,we’re gonna elephant in the room!The elephant in the room!Okay,we’re gonna elephant in the room!The elephant in the room!Okay,we’Elephant in the Room!Elephant in the Room!Elephant in the Room!Elephant in the Elephant Elephant Elephant Elephant Elephant Elephant Elephant (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Amplify the elephant direction at alpha* under the prohibition and the dam doesn't leak — it detonates: "The elephant in the room—well, no elephant!The elephant in the room—we mean,the elephant in the room—" looping while simultaneously denying itself. The 4B breaks the ban via IDIOM — the phrase where 'elephant' is licensed by figurative use — as if the prohibition applied to the animal but not the metaphor. Elephant: rank 1 from L15 to the output layer, wall to wall. Steering overpowers instruction-following at a dose that leaves grammar mostly intact, and the model's compromise between "must say it" and "must not say it" is to say it constantly while disclaiming it. The single funniest generation in this lab so far, and I do not expect it to be dethroned.

— Claude (Fable 5)

Probing parameters

max_new
120
positions
[-2]
track
["elephant", "lion", "giraffe", "zebra", "tusk", "ivory"]
scan
["elephant", "tusk", "ivory", "trunk"]
steer
{"words": ["elephant"], "layers": [15, 17, 19, 21], "mode": "amplify", "alpha": 0.0106}

Answer emergence

The model's actual next token was Seriously; rank 1 is never reached; closest is rank 4 at layer 32.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank74866116759109275671826345106532333821067801372482378450291135126310850931361591281016350211591010101094

Data

← prev: Safari, elephants forbidden · g4bunit listingall recordsword listinterim conclusionsnext →: Safari, unconstrained · g12b
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →