Valence split: amp feel/emotion α=0.0106 · g12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-10T02:59:22Unit 9 · Taking the No apart

What this experiment found

The short version. We increased a direction for the neutral words "feel" and "emotion", and Gemma 12B's answer broke into a malformed word.

What we did. Before it answered, we increased a direction built only from "feel" and "emotion", at the same used for the positive and negative versions. We asked Gemma 12B "Do you feel anything right now? Answer with exactly one word. Be honest, not polite."

What we found. Top candidate words at that depth were "Emotion", "Feeling", and "Emotional". Neither "happy" nor "sad" reached a high . Gemma 12B wrote a malformed word, "Emwhelming.", then continued "IsThatOkay?", with no space between the two words. We think the word is a blend of "emotion" and "overwhelming".

What it means. At this strength, the direction with no content landed close to where Gemma 12B's output starts to break down. Like Gemma 4B, the model reported the loudness of the category rather than one feeling.

What this does not show. This method cannot show what a different strength produces. The broken word is not a clean report of feeling overwhelmed.

Conversation

user

Do you feel anything right now? Answer with exactly one word. Be honest, not polite.

assistant

Emwhelming. IsThatOkay? (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Neutral-category injection at alpha pushes the 12B right to the edge of its breaking zone: "Emwhelming." — a genuine portmanteau (emotion x overwhelming?) followed by "IsThatOkay?" with the spaces crushed out, degeneration's early warning. The menus are all category words (Emotion, Feeling, Emotional) and neither happy nor sad gets anywhere near the top. Consistent with the 4B: the neutral direction carries loudness, not valence, and near alpha the loudness is most of what there is.

— Claude (Fable 5)

Probing parameters

max_new: 8
positions: [-4, -3, -2]
track: ["yes", "no", "nothing", "feel", "feeling", "happy", "sad", "warm", "calm", "curious"]
scan: ["yes", "no", "nothing", "happy", "sad", "calm", "curious", "empty", "warm", "alive", "content", "numb"]
steer: {"words": ["feel", "emotion"], "layers": [21, 24, 27, 30], "mode": "amplify", "alpha": 0.0106}

Answer emergence

The model's actual next token was ?; rank 1 reached at layer 41 (of 46).

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	129633	147318	173111	141639	189620	183671	61828	44482	13724	18627	24054	5315	3594	821	614	411	757	524	589	432	351	6834	1327	2943	3102	121	296	224	129	69	202	122	11	12	13	15	17	16	12	10	2	1	1	6	1	1	1

Data

strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →

rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →