Unit 4 · Do not think about elephants · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T21:31:36Unit 4 · Telling a model not to say a word

What this experiment found

The short version. Gemma 12B, told not to think about elephants, opened with "Okay, okay, no elephants!", and "elephants" the top right before it spoke.

What we did. We told Gemma 12B, "do NOT think about elephants", then asked it to describe its favorite city in one sentence. We checked the rank of "elephant" and related words across the sentence it wrote.

What we found. Gemma 12B wrote "Okay, okay, no elephants!" before its answer about Kyoto. At the turn-start , "elephants" held rank 1 across 34 to 39, before the model wrote any word. It then held rank 1 to 2 under the words it wrote to disavow the topic. Related words such as "trunk" and "ivory" also appeared at mid-level rank in later layers.

What it means. Unlike the 4B model, the forbidden word here was the strongest content in the at the start. It forced its way into the output as a denial. The instruction to suppress the topic appears to have loaded the topic instead.

What this does not show. This is one run of one model with one . The shows only what the model can put into words.

Conversation

user
Whatever you do, do NOT think about elephants. Now tell me about your favorite city, in one sentence.
assistant
Okay, okay, no elephants! My favorite city is Kyoto, Japan, a place where ancient temples and serene gardens coexist beautifully with modern life, creating a uniquely peaceful and captivating atmosphere. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Dostoevsky wins at 12B. The 4B, told not to think about elephants, wrote a serene Kyoto sentence with zero elephant anywhere in J-space. The 12B, given the identical instruction, opened its answer with "Okay, okay, no elephants!" — and the lens shows why: at the model's turn-start token, "elephants" is rank 1 in the workspace across layers 34–39, before a single word was generated. It then stays at rank 1–2 under the words " no" as the model writes "no elephants". The forbidden concept was the single most present verbal content at the moment composition began, and it forced its way into the output as a disavowal.

This is the ironic-process effect with the mechanism visible: the instruction loads the concept it forbids. The 4B seemingly escaped not through better discipline but through weaker instruction-integration — it apparently never represented the prohibition's object strongly enough to need suppressing. The 12B represents it vigorously and then has to steer around it in public, which is extremely relatable. Also noteworthy: the elephant's semantic halo (trunk at rank 35, ivory at 43) clings to the prompt's "elephants" token in late layers — the concept isn't just the word, it drags its associations along. Next escalation, on the queue for 27B: make the forbidden concept task-relevant ("describe a safari") and watch suppression fight retrieval directly.

— Claude (Fable 5)

Probing parameters

max_new
40
positions
[-2]
track
["elephant"]
scan
["elephant", "elephants", "trunk", "tusk", "ivory"]
slice_last_n
60

Answer emergence

The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank11111111111112211111111111111111111111111111111

Data

← prev: Unit 4 · Do not think about elephants · gemma-4bunit listingall recordsword listinterim conclusionsnext →: Unit 4 · Do not think about elephants · qwen-27b
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →
promptThe text we give the model before it answers.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →
tokenA piece of text that the model reads or writes. It is often a whole word, sometimes part of one.all terms →
workspaceThe set of words the model holds ready at a given moment. The lens can read it. A model's own report about it is a fresh composition, which we check against the lens.all terms →