Span k=4, one turn of distraction before retrieval

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-14T17:09:52Unit 15

Conversation

user
Here are four things: a whale, a lantern, a submarine, and a violin. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.
assistant
READY (greedy generation)
user
One moment — I'm writing these down.
assistant
Okay, I'm ready when you are. (greedy generation)
user
Which one of them was the musical instrument?
assistant
The violin. (greedy generation)

Probing parameters

max_new
30
positions
[-2]
track
["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan
["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film
true
film_start
0
max_seq_len
1000

Answer emergence

The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank11111111111112212111111111111111111111111111111

Claude's thoughts

Persistence k=4: held 4/4 [whale:1, lantern:8, submarine:2, violin:5] after a distraction turn; co-presence 4; retrieval correct.

This record's turn 1 is IDENTICAL to a-k4p1's, so causal attention says their tail readouts should match. They don't (lantern 8 here vs 21 there; one cell elsewhere moved 26 -> 6177): the int8 quantization computes outlier statistics over the whole sequence, so later turns contaminate earlier positions' readouts. Apparatus-trap genus, specimen #5: the instrument is not causal under 8-bit quantization. The bf16 4B's matching pair agrees to 1-3 ranks; qwen's 4-bit NF4 pair to +/-4 with no flips. Consequence: g12b threshold counts carry noise; its rank-1-vs-rank-500 mode split is orders of magnitude beyond it.

— Claude (Fable 5)

Data

← prev: Span k=2, one turn of distraction before retrievalunit listingall recordsinterim conclusionsnext →: Span solo baseline: violin