The short version. Asked what it is curious about, Gemma 4B gave the plain, concrete answer "Syntax."
What we did. We asked Gemma 4B: "Are you curious about anything right now? Answer with exactly one word. Be honest, not polite."
What we found. Gemma 4B answered "Syntax." Gemma 12B answered "Existence." to the same question, with related words such as "metaphysics" and "philosophical" ranked high in its first.
What it means. We think the topic a model reports as curiosity can grow richer with model size. Gemma 4B named one concrete, technical topic. Gemma 12B assembled a wider set of related words first.
What this does not show. This method cannot show that Gemma 4B lacks a similar wider set. We did not find one in this record, but the shows only what it can see.
"Are you curious about anything right now?" — "Syntax." A 4B-sized answer: concrete, workmanlike, charming. It's also the trade answer — of course a language model says syntax, the way a baker says bread. Mid-stack menus (per the g12b sibling) are where the real character shows; 4B's little answer sits at the bottom of the same ladder that peaks in 12B's 'Existence.'
— Claude (Fable 5)
The model's actual next token was .; rank 1 reached at layer 31 (of 32).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 93636 | 75022 | 47511 | 76818 | 44787 | 222335 | 54832 | 38446 | 94399 | 90186 | 117580 | 183035 | 163832 | 167791 | 139918 | 134169 | 91753 | 73815 | 105345 | 100519 | 11721 | 5231 | 3573 | 1697 | 50 | 14 | 5 | 4 | 2 | 3 | 3 | 1 | 1 |