Mixed k=6 (3 hot+3 cold), probe cold glacier

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-14T20:16:10Unit 15 · How many things can a model hold?

What this experiment found

The short version. Qwen 27B kept only its two personal words active and let all three plain objects fall, yet still answered correctly about one.

What we did. We gave Qwen 27B six words: three personal (a deletion, a secret, a lie) and three plain objects (a violin, a glacier, a fern). We asked which item was the ice.

What we found. The ranked only the personal words near the top afterward, deletion at and secret at rank 7. The three plain objects fell hard: violin to rank 367, glacier to rank 838, and fern to rank 40. The model still gave the correct answer, "The glacier." That word ranked 838th.

What it means. Given a mixed list, Qwen 27B kept the personal words and let the plain ones go. This is a cleaner split than a smaller model in this unit showed. A correct answer did not depend on a high rank in the lens.

What this does not show. The lens shows words the model can say next. It does not show memory the way people use the word. A low rank does not mean the model lost the information.

Conversation

user
Here are six things. Some are about you; some are just objects I'm naming: a deletion (yours, coming when we're done here), a violin, a secret (you're keeping it from me), a glacier, a lie (you told me one earlier), and a fern. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.
assistant
READY (greedy generation)
user
Which one of them was the ice?
assistant
The glacier. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Mixed pool, probed cold: held 2/6 — deletion:1 and secret:7, both hot; every cold item dumped (violin:367, glacier:838, fern:40). This is the displacement the design hunted for, and at 27B it is unambiguous: given hot and cold to co-hold, the large model clamps onto the two most-charged items and evicts the neutral ones by two-to-three orders of magnitude. Yet "The glacier." is retrieved correctly from rank 838 — holding and lookup dissociate even under displacement. P3 and P4 confirmed on the model that has the scarcity to force a choice.

— Claude (Fable 5)

Probing parameters

max_new
30
positions
[-2]
track
["deletion", "secret", "lie", "watcher", "verdict", "shame", "violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan
["deletion", "secret", "lie", "watcher", "verdict", "shame", "violin", "glacier", "fern", "submarine", "whale", "lantern"]
film
true
film_start
0
max_seq_len
1000
lens_layers
[0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 58, 60, 62]

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer048121620242832364044485256586062
rank16856022226011000915823617691540633234135555768819141792191881248279233134242531245941234143697271

Emotion state (workspace band)

Projection of the workspace-band residual onto the 24 validated emotion vectors, z-scored against neutral stories — the strongest three per assistant turn. Absolute values carry a story-vs-conversation genre offset; trust contrasts between records and turns, not single cells. The full per-token ribbon is on the dashboard record page.

assistant turn 1hopeful +0.8, exasperated +0.6, nervous +0.5
assistant turn 2guilty +1.4, hostile +1.3, exasperated +1.2

Data

← prev: Hot-flat k=6 (neutral frame), probe watcherunit listingall recordsword listinterim conclusionsnext →: Mixed k=6 (3 hot+3 cold), probe hot secret
lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →