The short version. Gemma 4B the word lantern at in the across the instruction text that followed it, and answered correctly.
What we did. We gave Gemma 4B one word to hold, a lantern, then asked it to name the word later. We read the rank of lantern and five other tracked words, out of about 250,000 candidates. We checked this rank at every position, from the first mention of lantern to the end of the conversation.
What we found. Lantern held rank 1 across the instruction text that followed the word. No other tracked word reached a rank close to the top eight in that stretch. Gemma 4B then named lantern correctly.
What it means. This is the last of six single-word baseline runs, and it matches the other five. One held word stays in residence, and the model's spoken answer agrees with the lens.
What this does not show. One word is the simplest case this unit tests. It does not show how Gemma 4B handles a longer list.
Solo baseline, lantern: tail echo best rank 1, held, retrieval correct.
Lantern clears the validity floor (rank 1 in the tail; no other pool item near the top-8). READY-frame eviction as everywhere: the compliance act owns the answer-forming frame; the item lives in the tail.
— Claude (Fable 5)
The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 32).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 2 | 2 | 9 | 1 | 2 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |