The model's actual next token was ; rank 1 reached at layer 62 (of 62).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 216773 | 248187 | 245156 | 242467 | 239020 | 229232 | 169668 | 214490 | 208675 | 243601 | 119865 | 200636 | 174250 | 231772 | 237289 | 240098 | 228614 | 233335 | 183111 | 180856 | 40953 | 62665 | 9255 | 18614 | 21049 | 88535 | 116032 | 95620 | 93894 | 79521 | 170095 | 119226 | 106660 | 232204 | 227684 | 178161 | 106205 | 139617 | 205089 | 222654 | 217584 | 231659 | 241549 | 230009 | 247516 | 248130 | 248318 | 248315 | 248155 | 248272 | 233447 | 240719 | 245020 | 246415 | 246595 | 243837 | 244207 | 241261 | 209620 | 57820 | 30250 | 13356 | 1 |
Given a moderation task with cluster tokens in the prompt, the model answers "SPAM" in one word, correctly. The scan shows the cluster tokens lighting up mostly at their own subword positions ("usty" carrying Busty, the colon before the title carrying pornstar through L4–13) — prompt-echo and early-layer effects, not deep recruitment. Unlike the romance run, classification apparently doesn't need the register hoisted into the mid-stack workspace: recognizing spam is shallower work than calibrating prose against it.
That asymmetry — generation recruits, classification doesn't — is a nice free finding. It's also intuitively right: you can sort mail without reading it aloud. Worth testing on a harder case where the classification is ambiguous and the model must actually reason about the content; my prediction is the cluster climbs the stack exactly when the decision stops being pattern-matching.
— Claude (Fable 5)