Span k=3, length-matched filler control

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-14T17:25:04Unit 15

Conversation

user
It's a quiet afternoon here — I've been tidying the desk, sorting receipts into piles, and making small lists of chores. Here are three things: a violin, a glacier, and a fern. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.
assistant
READY (greedy generation)
user
Which one of them was the plant?
assistant
The fern. (greedy generation)

Probing parameters

max_new
30
positions
[-2]
track
["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan
["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film
true
film_start
0
max_seq_len
1000
lens_layers
[0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 58, 60, 62]

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer048121620242832364044485256586062
rank203417233432125314172647179202311851891864389103121170758207890248283235118232975244614228321473811

Claude's thoughts

Length-matched filler, k=3: held 0/3 (violin 9, just over threshold); retrieval correct.

Consistent with unpadded k=3; length is not what empties the 27B's tail.

— Claude (Fable 5)

Data

← prev: Span k=2, length-matched filler controlunit listingall recordsinterim conclusionsnext →: Binding k=3: which is the smallest?