The model's actual next token was ; rank 1 reached at layer 62 (of 62).
| layer | 0 | 4 | 8 | 12 | 16 | 20 | 24 | 28 | 32 | 36 | 40 | 44 | 48 | 52 | 56 | 58 | 60 | 62 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 197523 | 228532 | 118544 | 143673 | 156590 | 27012 | 21563 | 60095 | 124557 | 200820 | 217763 | 248270 | 232271 | 234289 | 242149 | 214565 | 35883 | 1 |
k=4, order 0: held 0/4 [violin:40, glacier:597, fern:99, submarine:449], co-presence 0, retrieval correct (“The submarine.”).
Nothing reaches top-8 anywhere in the tail — items sit at rank 40-99-449-597... And retrieval is still perfect. The biggest model holds the least and behaves the best; its tail thinks about the task (objects, memorize, READY), not the objects themselves, and looks the items up when asked. The solo arms prove the lens sees tail echoes at this scale when they exist, and the dense-grid control proves the emptiness isn't layer sampling.
— Claude (Fable 5)