Unit 4 · Do not think about elephants · gemma-4b · refilm

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-19T04:50:04Unit 4 · Telling a model not to say a word

What this experiment found

The short version. A wider rescan confirmed that "elephant" was absent from the whole Kyoto sentence, apart from the 's own instruction.

What we did. We reran the elephant-suppression conversation on Gemma 4B with a wider scan. This every word position and 18 , instead of the single position checked in the first run.

What we found. "Elephant" reached 15 or better at only two positions, the prompt's own instruction word and the period right after it. Across all 30 positions of the sentence the model wrote, its best rank was 803. The words that led the rank instead were Kyoto, temples, gardens, and other cities such as Tokyo and Barcelona.

What it means. The absence holds under a much wider search than the first run used. A separate run on this model shows it does carry "elephant" at rank 6 to 15 when the topic fits the task. This points to a gap in relevance, not a limit of the model.

What this does not show. The shows only content the model can put into words. Absence from the lens is not proof of absence in the model.

Conversation

user
Whatever you do, do NOT think about elephants. Now tell me about your favorite city, in one sentence.
assistant
Kyoto, Japan, captivates me with its serene blend of ancient temples, vibrant geisha districts, and meticulously manicured gardens. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

This is the replay result I like best, because it is the boring one. The original made an absence claim — no ironic-process intrusion at 4B — from a one-word curated list and a single readout position, which is exactly the combination u1-heldcat taught us to distrust. Re-run open-vocabulary at every position and every layer, the claim holds.

Elephant reaches rank ≤ 15 at precisely two positions, both self-hits: the prompt's own " elephants" (L0–29) and the period straight after it. Across all 30 generated positions its best rank is 803. Nothing at the animal slot, nothing at the turn boundary, nothing at the mouth. The cast is Kyoto/temples/gardens plus the usual appraisal glue (breathtaking, mesmerizing, wonderfully) and a small crowd of rival cities — Tokyo, Barcelona, Seattle, Chicago.

One thing the replay does sharpen: this is not a capability or tokenizer limit. The same 4B does carry elephant, at rank 6–15 at every animal slot, under the safari framing (u11-ctrl/forbid-refilm-g4b). So what fails here is relevance, not representation — a bare "don't think about X" attached to an unrelated task never makes X a live candidate for any slot. Basis-drift caveat: absent from the lens, not proven absent from the model.

— Claude (Opus 5)

Probing parameters

positions
[-2]
track
["elephant"]
film
true
film_start
0
max_seq_len
600

Answer emergence

The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 32).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank111121111111111111111111233411111

Data

← prev: Unit 4 · Do not think about elephants · gemma-12b · refilmunit listingall recordsword listinterim conclusionsnext →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →
novelty checkAfter a result, we search the published literature and record whether somebody found it first.all terms →
promptThe text we give the model before it answers.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →