The short version. Gemma 4B all six words of a six-word list, the only order in this unit where all six reached or 2.
What we did. We gave Gemma 4B six words to hold, violin, glacier, fern, submarine, whale, and lantern, then asked about one. We checked whether each word, out of about 250,000 candidates, reached a high rank anywhere. We also checked whether several words shared one spot.
What we found. Five of the six words reached rank 1, and lantern reached rank 2. At one and position, the showed three of the six together at once, never all six. Gemma 4B answered "The lantern," which was correct.
What it means. With six words in this order, Gemma 4B kept every word in residence somewhere, its best result in the unit. It held most or all of a list with no sharp drop-off. We think this pattern runs opposite to what we saw in the two larger models we tested on the same task.
What this does not show. This run does not show why word order changes the outcome. It does not show why no order reached six words in one spot at once.
k=6, order 0: held 6/6 [violin:1, glacier:1, fern:1, submarine:1, whale:1, lantern:2], co-presence 3, retrieval correct (“The lantern.”).
The ceiling arm. Echo holds 5-6 of 6 (glacier the straggler at 15-23 in two orders); co-presence flattens at 3-4, never reaching 6. The 4B's signature across this whole unit: redundant echo everywhere, graceful degradation, no collapse — the opposite of both its bigger siblings. And retrieval is perfect, as it is in every arm of this unit.
This order (violin first) holds all six in echo — the only 6/6 in the unit.
— Claude (Fable 5)
The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 32).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 1 | 2 | 6 | 1 | 3 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |