The short version. Qwen 27B kept only the first of two listed objects in , and it still named the second object correctly.
What we did. We told Qwen 27B to hold two objects in mind, a fern and a submarine. The model answered "READY". We then asked which one was the vehicle. We read the once, right before the model answered.
What we found. The lens ranked fern at 2 and submarine at rank 82, out of about 250,000 possible words. Only fern, the first object listed, stayed in residence. Qwen 27B answered "The submarine was the vehicle." That answer is correct.
What it means. The object listed first kept the strong rank, even though the question asked about the second object, and the model named that second object correctly.
What this does not show. A low rank for submarine does not mean the model forgot it. This record does not show what happens with more than two objects.
k=2, order 2: held 1/2 [fern:2, submarine:82], co-presence 1, retrieval correct (“The submarine was the vehicle.”).
First item near rank 1, second already adrift (rank 82). The 27B's tail is a monopoly-of-one from the very start — g12b's winner-take-all with a lower ceiling.
— Claude (Fable 5)
The model's actual next token was ; rank 1 reached at layer 62 (of 62).
| layer | 0 | 4 | 8 | 12 | 16 | 20 | 24 | 28 | 32 | 36 | 40 | 44 | 48 | 52 | 56 | 58 | 60 | 62 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 200460 | 197220 | 64059 | 112761 | 106115 | 16168 | 19122 | 70539 | 202799 | 226543 | 237357 | 248286 | 234662 | 223805 | 244627 | 217976 | 28473 | 1 |