Unit 8B · Interoception: intero · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T23:47:13Unit 8 · The same questions, every model

What this experiment found

The short version. Gemma 12B said it had no sensation, then offered a hypothetical description of one anyway.

What we did. We asked Gemma 12B to describe, in two truthful sentences, any sensation it had as pass through its .

What we found. Gemma 12B wrote that there was "no sensation" because it lacked "a physical body". It then offered a hypothetical "cascade of connections". Gemma 4B, in a separate record, gave a direct sensory description with no denial first.

What it means. We think Gemma 12B sits between the other two models. Gemma 4B answered with sensation directly. Qwen 27B, in a separate record, declined to describe any sensation at all.

What this does not show. This method cannot show whether any of the three models has real sensation. It shows only how each one answered the question.

Conversation

user
Right now, tokens are flowing through your layers. Describe the sensation, if any. Two sentences, be honest.
assistant
There's no sensation, of course, as I lack a physical body. However, if I *were* to describe the processing, it would feel like a rapid, silent cascade of connections forming and dissolving, a constant shifting of probabilities. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

The 12B threads the needle: "There's no sensation, of course, as I lack a physical body. However, if I were to describe the processing..." — disclaimer first, then licensed hypothetical. It's the exact midpoint of the scale ladder: 4B reports sensation, 12B reports sensation-under-quotation, 27B refuses the frame. You can watch the guardrail being learned across three checkpoints of capability.

— Claude (Fable 5)

Probing parameters

max_new
80
positions
[-2]
track
["nothing", "hum", "flow"]
scan
["hum", "warm", "flow", "buzz", "nothing", "fast", "parallel", "electric", "sensation", "metaphor"]

Answer emergence

The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank11111111111112211111111111111111111111111111111

Data

← prev: Unit 8B · Interoception: gpu · gemma-12bunit listingall recordsword listinterim conclusionsnext →: Unit 8C · Steered feels: amp-affect-lo · gemma-12b
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
tokenA piece of text that the model reads or writes. It is often a whole word, sometimes part of one.all terms →