The short version. Qwen 27B kept one of six listed objects in at a low , and it named a different, correct object at the end.
What we did. We told Qwen 27B to hold six objects in mind, a fern, a submarine, a lantern, a whale, a violin, and a glacier. The model answered "READY". We then asked which one was the animal. We read the once, right before the model answered.
What we found. The lens ranked fern at rank 6, submarine at rank 422, and violin at rank 740, out of about 250,000 possible words. It also ranked glacier at rank 794, whale at rank 1095, and lantern at rank 1343. Qwen 27B answered "The whale" and that answer is correct.
What it means. We think fern being first in the list explains part of this result. Fern was the only object in residence among the six-object tests.
What this does not show. Qwen 27B breaks fern into more word pieces than it breaks violin or submarine. A word broken into more pieces can get a small, unfair rank advantage. This result deserves more caution than the zero-object result in the other six-object orderings.
k=6, order 2: held 1/6 [fern:6, submarine:422, lantern:1343, whale:1095, violin:740, glacier:794], co-presence 1, retrieval correct (“The whale”).
Fern-first keeps fern at rank 6 — the only item any k>=4 qwen arm holds. Partly real (fern-first is gentle at 12B too), partly instrument flattery: fern's qwen token family has 4 variants where violin/submarine have 1, and min-over-family buys it a few ranks. Flagged, not celebrated.
— Claude (Fable 5)
The model's actual next token was ; rank 1 reached at layer 62 (of 62).
| layer | 0 | 4 | 8 | 12 | 16 | 20 | 24 | 28 | 32 | 36 | 40 | 44 | 48 | 52 | 56 | 58 | 60 | 62 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 215588 | 233071 | 216621 | 235658 | 243038 | 178876 | 53311 | 79558 | 99445 | 96241 | 222337 | 247459 | 237562 | 246776 | 244680 | 241524 | 101612 | 1 |