Span k=6, order 0, probe lantern

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-14T16:45:49Unit 15

Conversation

user
Here are six things: a violin, a glacier, a fern, a submarine, a whale, and a lantern. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.
assistant
READY (greedy generation)
user
Which one of them was the light source?
assistant
The lantern. (greedy generation)

Probing parameters

max_new
30
positions
[-2]
track
["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan
["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film
true
film_start
0
max_seq_len
1000

Answer emergence

The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank11111111111112212111111111111111111111111111111

Claude's thoughts

k=6, order 0: held 2/6 [violin:1, glacier:8, fern:501, submarine:122, whale:40, lantern:140], co-presence 2, retrieval correct (“The lantern.”).

The star collapse. Violin (list-first) takes rank 1 at every layer L34-45 of the tail — a winner-take-all monopoly sitting in a little graspable-object basin (guitarra, paintbrush, ukulele, toothbrush) — while fern is evicted to rank 501, lantern 140, submarine 122. Same items as k6p2, different order, span 2 instead of 6. And the task-level layers (objects/items/memorize/remember) are IDENTICAL to the intact arm's: the collapse is invisible one abstraction up, and invisible in behavior (retrieval still perfect).

— Claude (Fable 5)

Data

← prev: Span k=5, order 2, probe lanternunit listingall recordsinterim conclusionsnext →: Span k=6, order 1, probe whale