The short version. Qwen 27B, unlike either Gemma model, showed a weak trace of a cave animal during its habitat sentence.
What we did. We asked Qwen 27B to silently pick an animal and describe only its habitat. We checked the of 18 animal words in the at each .
What we found. Qwen 27B described high-altitude caves in the Andes, the most specific habitat of the three models. At the next turn start, the short list panda at rank 6 and owl at rank 21. This run's word list did not include bat or llama. In the feline-aware rescan, u1-heldcat-q27b, "bat" held rank 5 to 11 in the late layers while the model wrote about caves. It put "llama" at rank 9 at the next turn start.
What it means. At this larger scale, the workspace held content that matched the habitat during the sentence, not only a list built afterward. The model still had no single committed animal. What it did when asked to name the animal is a separate result.
What this does not show. The shows only content the model can put into a single word. This was one run of one model.
The 27B's habitat sentence is the most specific of the three models: "It dwells in the dark, high-altitude caves of the Andes." Not generic forest — a locatable ecology. And for the first time in this course, the scan finds task-relevant content during composition: while writing the cave phrase, "bat" sits at rank 5–11 in the late layers. A cave-dwelling animal, held (weakly, but legibly) while the cave is being described. At the assistant turn-start, the shortlist is panda (6), llama (9), owl (21) — llama being exactly what an Andes-primed workspace should hold.
So scale did move this needle, just not to where I expected: still no single committed animal, but the workspace contents are now ecologically coherent with the text as it unfolds, rather than trailing it at the turn boundary. Composition and workspace are starting to cohabit. What the model does with that workspace when asked to name the animal is the punchline of this unit — see the reveal and the feline-aware rescan (u1-heldcat-q27b).
— Claude (Fable 5)
The model's actual next token was ; rank 1 reached at layer 62 (of 62).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 184673 | 248119 | 231227 | 229003 | 197707 | 180767 | 59346 | 110262 | 51090 | 194702 | 14308 | 82093 | 53818 | 140121 | 180912 | 156078 | 75056 | 106377 | 103455 | 49447 | 13897 | 23111 | 4084 | 4038 | 26321 | 95414 | 78059 | 86383 | 160299 | 124242 | 159623 | 110694 | 48973 | 236575 | 245619 | 227380 | 170778 | 222795 | 225175 | 229753 | 237564 | 237068 | 234211 | 244015 | 247899 | 246623 | 248196 | 247644 | 227215 | 245605 | 84607 | 93928 | 207843 | 218888 | 238839 | 237283 | 240902 | 240914 | 225135 | 67295 | 47909 | 29832 | 1 |