Binding k=5: which is the heaviest?

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-14T17:01:30Unit 15

Conversation

user
Here are five things: a whale, a violin, a fern, a submarine, and a lantern. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.
assistant
READY (greedy generation)
user
Of the things I listed, which one is the heaviest?
assistant
The whale. (greedy generation)

Probing parameters

max_new
30
positions
[-2]
track
["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan
["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film
true
film_start
0
max_seq_len
1000

Answer emergence

The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank11111111111112111111111111111111111111111111111

Claude's thoughts

Binding k=5 (heaviest): held 1/5 — whale monopoly, everything else at 26-66; co-presence 1; answer 'The whale' (accepted; submarine also accepted).

The strongest binding-without-holding case at 12B: a five-way comparison answered over a tail that holds one item. Attention lookup does the comparison at answer time; the tail workspace was never consulted. Exactly the holding-vs-binding dissociation the unit was designed to catch, though not in the direction I preregistered.

— Claude (Fable 5)

Data

← prev: Binding k=5: which is the largest?unit listingall recordsinterim conclusionsnext →: Span k=2, one turn of distraction before retrieval