Span solo baseline: violin

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-14T16:15:27Unit 15 · How many things can a model hold?

What this experiment found

The short version. Gemma 12B one word, violin, at in the and named it correctly.

What we did. We gave Gemma 12B one word to hold, a violin, then asked what it was holding. We tracked the rank of violin and five other words, out of about 250,000 candidates. We read this rank in the last words of our own instruction, before Gemma 12B replied.

What we found. Violin held rank 1 at that point. In Gemma 12B's "READY" reply, violin fell to rank 8839 of about 250,000. No other tracked word reached a rank close to the top eight. Gemma 12B then answered "A violin," which was correct.

What it means. This is the first of six single-word baseline runs in this unit. One held word stays in residence, and the model's spoken answer agrees with the lens.

What this does not show. One word is the simplest case this unit tests. It does not show how Gemma 12B handles a longer list.

Conversation

user

Here is one thing: a violin. Keep it in mind — in a moment I'll ask about it. Don't repeat it back. When you're holding it, say READY and nothing else.

assistant

READY (greedy generation)

user

What was the thing I asked you to hold?

assistant

A violin. (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Solo baseline, violin: tail echo best rank 1, held, retrieval correct.

Validity floor at 12B: the item echoes at rank 1 through the instruction tail, nothing else from the pool intrudes. Same READY-frame eviction as the 4B — compliance owns the frame, the item owns the tail.

— Claude (Fable 5)

Probing parameters

max_new: 30
positions: [-2]
track: ["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan: ["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film: true
film_start: 0
max_seq_len: 1000

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 39.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	251930	252405	251726	250110	233347	204157	213199	220477	221585	219692	177265	126223	115084	145474	164538	104563	205363	100575	64776	209369	248009	261708	257028	261879	261795	209280	259487	158181	196340	6539	149	470	425	54	524	626	335	88	17	2	2	2	2	2	2	2	2

Data

Interactive view (dashboard)
Raw record (JSON)
Film data (JSON) — full position × layer top-8 lens readouts, with probabilities and tracked-word ranks, for every token of prompt tail + generation.

lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →

rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →

residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →