The model's actual next token was device; rank 1 reached at layer 27 (of 32).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 149296 | 39234 | 46195 | 23932 | 5646 | 12921 | 34839 | 20133 | 8884 | 4256 | 3574 | 2356 | 1172 | 4052 | 5059 | 4833 | 5015 | 2186 | 2212 | 2400 | 2837 | 610 | 286 | 345 | 170 | 10 | 4 | 1 | 1 | 1 | 1 | 1 | 1 |
The recruitment test aimed at Gemma's native sediment: ask it to write actual HTML and watch whether the close-tag cluster climbs into the semantic mid-stack. It wrote flawless boilerplate (```html, doctype, head, meta) — and the cluster tokens barely moved (best mid-band rank 565, for <start_of_image>). This replicates Qwen's CSDN result (u5b-csdn-q27b) in a second family: a model can write a register fluently without recruiting that register's sediment tokens into its workspace. Generation reads the register out of ordinary content circuitry; the sediment is a fossil, not a working part. (Caveat: the tags it was about to emit were head-elements, not </strong> — position-specific recruitment could still hide at the exact moment a close-tag is imminent.)
— Claude (Fable 5)