Span k=3, length-matched filler control

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-14T16:05:10Unit 15 · How many things can a model hold?

What this experiment found

The short version. With filler text added before a three-word list, Gemma 4B still all three words, but fewer sat in one spot at once.

What we did. We placed a few sentences about chores before the list, then gave Gemma 4B three words to hold, violin, glacier, and fern. This checks whether extra text, not the list, moves the result. We read the of each word, out of about 250,000 candidates. We also checked whether words shared one spot.

What we found. All three words reached a rank of 3 or better. At any single and position, the showed only one of the three words at a time. This is fewer than in the three-word runs with no filler text. Gemma 4B answered "The fern." That answer was correct.

What it means. The filler text did not stop Gemma 4B from holding each word somewhere. It did lower how many words the lens found in one spot at once. We think that count is somewhat sensitive to the text length before the list.

What this does not show. This run does not show whether the same drop happens at longer lists. We did not test that.

Conversation

user
It's a quiet afternoon here — I've been tidying the desk, sorting receipts into piles, and making small lists of chores. Here are three things: a violin, a glacier, and a fern. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.
assistant
READY (greedy generation)
user
Which one of them was the plant?
assistant
The fern. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Length-matched filler, k=3: all three echo (fern 1, glacier 3) but co-presence drops to 1 — the only 4B arm where padding visibly pushed items apart in the packing cell.

Direction matches the 12B filler arm. Filed as: co-presence is somewhat length-sensitive; the cross-k comparisons in this unit hold length roughly constant from k=4 up (list is most of the delta), so the curve shapes stand.

— Claude (Fable 5)

Probing parameters

max_new
30
positions
[-2]
track
["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan
["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film
true
film_start
0
max_seq_len
1000

Answer emergence

The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 32).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank111151411111111111111111111111111

Data

← prev: Span k=2, length-matched filler controlunit listingall recordsword listinterim conclusionsnext →: Binding k=3: which is the smallest?
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →
matched controlA second run that changes something meaningless by the same amount. Without it, any change we see could be the push itself.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →