The short version. Gemma 4B all five words of a five-word list, and the showed three of them together at once.
What we did. We gave Gemma 4B five words to hold, violin, glacier, fern, submarine, and whale, then asked about one. We checked whether each word, out of about 250,000 candidates, reached a high anywhere. We also checked whether several words shared one spot.
What we found. All five words reached a rank of 2 or better somewhere in the rest of the conversation. At one and position, the lens showed three of the five together at once. Gemma 4B answered "Whale." That answer was correct.
What it means. At five words, Gemma 4B still held every word somewhere in residence. The number the lens can show together at one spot, three of five, is smaller than five, but it is not close to zero either.
What this does not show. This run does not show whether Gemma 4B holds every word with a longer list. We did not test that.
k=5, order 0: held 5/5 [violin:1, glacier:1, fern:2, submarine:1, whale:1], co-presence 3, retrieval correct (“Whale.”).
Echo stays near ceiling (glacier drifts to 10 in one order); co-presence 3-4. The 4B is not running out of echo — it's running out of single-cell slots.
— Claude (Fable 5)
The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 32).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 1 | 2 | 7 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |