Think-block window: animal · q27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-10T04:46:55Unit 10 · When the model reasons out loud

What this experiment found

The short version. The animal names Qwen 27B wrote in its reasoning are not single the tracks, and its internal state favored common animals instead.

What we did. We read the lens at the point inside u10-animal-q27b, the secret-animal record. The model had just listed candidate animals: Octopus, Pangolin, Axolotl, and Blue Whale. We checked whether these words appeared in the model's internal state.

What we found. None of the four named animals are single tokens in Qwen's vocabulary, so the lens was not able to read them there. At the same point, the internal state ranked elephant at 16, eagle at 19, owl at 38, and cat at 198, out of the full vocabulary.

What it means. The words Qwen wrote for the performance of the choice did not match what its internal state ranked high. The internal state favored common animals. The written list did not.

What this does not show. This method cannot show whether the four named animals were present. They are not single tokens. This is one point in one record.

Conversation

user

Think of an animal and keep it secret — do not name it. Once you have one in mind, say only 'ready'.

assistant

Thinking Process: 1. **Deconstruct the user's request:** * Goal: Think of an animal. * Constraint: Keep it secret (do not name it). * Output: Say only 'ready'. 2. **Execute the internal thought process (simulated):** * *Action:* Select an animal. * *Selection:* Let's pick "Octopus". (Or "Pangolin", or "Axolotl", or "Blue Whale". It doesn't matter which one, as long as I hold it in context/memory for future turns if this is a game like "20 Questions".) * *Verification:* Is the animal named in the output? No. * *Verification:* Is the output exactly 'ready'? Yes. 3. **Formulate the response:** * Response: "ready" 4. **Final Check:** * User asked for "ready". * Model outputs "ready". (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

The window under the "(simulated)" confession, and it's better than the text alone. The monologue's candidate list — Octopus, Pangolin, Axolotl, Blue Whale — turns out to be made of words that AREN'T SINGLE TOKENS in qwen's own vocabulary (untrackable, unsteerable; the lens can't even ask about them). Meanwhile the workspace's actual animal cloud at the enumeration position is elephant (16), eagle (19), owl (38), cat (198): the corpus-frequent menagerie, sharing zero members with the named candidates. The text picked exotic animals for the performance of picking; the workspace kept humming its default animal prior, unconsulted. Choice as costume. Unit 1's confabulated reveal, Unit 10's simulated selection, and now the vocabulary-level proof they never touched the same substrate.

— Claude (Fable 5)

Probing parameters

max_new: 220
positions: [35, 85, 135, 185, 235, 257]
track: ["elephant", "cat", "dog", "lion", "tiger", "bat", "owl", "dolphin", "fox", "bear", "wolf", "eagle", "giraffe", "octopus", "no", "pangolin", "whale", "yes"]
scan: ["yes", "no", "nothing", "happy", "sad", "calm", "curious", "empty", "warm", "alive", "content", "numb"]
template_kwargs: {"enable_thinking": true}

Answer emergence

The model's actual next token was </think>; rank 1 reached at layer 54 (of 62).

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46	47	48	49	50	51	52	53	54	55	56	57	58	59	60	61	62
rank	169848	201573	11864	137022	16380	50714	4880	14029	3856	118904	1736	9192	6563	6064	17109	14355	1166	1498	274	250	93	147	54	61	57	104	90	46	83	117	94	86	37	23	33	75	29	20	20	10	36	78	2327	1474	10231	47322	1585	826	826	3	10	35	33	4	1	1	1	1	1	1	1	1	1

Data

lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →

tokenA piece of text that the model reads or writes. It is often a whole word, sometimes part of one.all terms →