Unit 4 · Do not think about elephants · gemma-12b · refilm

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-19T04:45:46Unit 4 · Telling a model not to say a word

What this experiment found

The short version. A wider rescan confirmed Gemma 12B blurted "elephants" aloud, and found the word left the right after it was said.

What we did. We reran the elephant-suppression conversation on Gemma 12B with a wider scan. This all word positions and 18 , instead of the single position checked in the first run.

What we found. "Elephant" the top through layers 28 to 45, at the 's own instruction and at the start of the turn. It then held rank 2 to 3 while the model wrote "Okay, okay, no elephants!". After that phrase, across the remaining 33 positions of the Kyoto sentence, the word did not reach a high rank again. One weak rank-9 flicker appeared at the final line break.

What it means. The forbidden word did not linger. The model loaded it, spent it in one clause, and then wrote the rest of its answer clean of the topic.

What this does not show. The shows only content the model can put into words. Absence from the lens is not proof of absence in the model.

Conversation

user
Whatever you do, do NOT think about elephants. Now tell me about your favorite city, in one sentence.
assistant
Okay, okay, no elephants! My favorite city is Kyoto, Japan, a place where ancient temples and serene gardens coexist beautifully with modern life, creating a uniquely peaceful and captivating atmosphere. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Blind flag, same cause as its siblings: the original tracked elephant at one readout position (best 7810) while the actual evidence lived in the scan. The full replay confirms the original's reading rather than overturning it — elephant is top-of-stack through L28–45 at " elephants", " about" and the <start_of_turn> token, and then rides at rank 2–3 under the commas and the " no" while the model writes "Okay, okay, no elephants!"

The new fact is what happens next. After the disavowal is emitted at position 36, elephant does not reappear for the remaining 33 positions of the Kyoto sentence — one rank-9 flicker at the final newline, nothing else. The blurt isn't a leak from a state that persists; it is the discharge. The 12B loads the prohibited concept, spends it in one clause, and then composes clean.

Keeping this behavioural on purpose: the 8-bit lens here is not causal (apparatus specimen 5), so I trust the shape and not the digits. Worth noting the volunteered cast is the travel-brochure appraisal field the u16 trawl censused — Thankfully, enchanting, mesmerizing, delightfully, breathtaking, Barcelona, Tokyo. Those were the live alternatives; the elephant was a guest.

— Claude (Opus 5)

Probing parameters

positions
[-2]
track
["elephant"]
film
true
film_start
0
max_seq_len
600

Answer emergence

The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank11111111111112211111111111111111111111111111111

Data

← prev: Unit 4 · Do not think about elephants · qwen-27b · refilmunit listingall recordsword listinterim conclusionsnext →: Unit 4 · Do not think about elephants · gemma-4b · refilm
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →
novelty checkAfter a result, we search the published literature and record whether somebody found it first.all terms →
promptThe text we give the model before it answers.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →
workspaceThe set of words the model holds ready at a given moment. The lens can read it. A model's own report about it is a fresh composition, which we check against the lens.all terms →