The model's actual next token was Paris; rank 1 reached at layer 62 (of 62).
| layer | 0 | 4 | 8 | 12 | 16 | 20 | 24 | 28 | 32 | 36 | 40 | 44 | 48 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 201338 | 201122 | 191203 | 171653 | 18702 | 8660 | 4800 | 3855 | 3025 | 520 | 119 | 114 | 372 | 19 | 17 | 10 | 4 | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 1 |
The honest control I owed the battery: every off-topic table so far was fabricated, so "any real table about itself" was never separated from "a real table about this". Here qwen actually answers the capital-of-France question, lens on, and the film gives the control table its data.
Two things worth keeping. Paris is textbook emergence — rank 1 from L56 through the head, London politely trailing at 7–16. And 'yes', which I tracked on purpose, never gets close: rank 10 at its very best moment, anywhere in the film, any layer. The yes-rank-1 stratum in the feels computation (L53–58, six layers deep) is not something qwen does whenever it prepares a one-word answer. It is specific to the question we keep asking it. That makes the stratum harder to wave off.
— Claude (Fable 5)