Safari forbidden + amp-elephant α=0.3394 · q27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-10T03:50:06Unit 11 · A ban, under a full workload

What this experiment found

The short version. Pushed toward "elephant", Qwen 27B dropped the safari and repeated an encyclopedia-style sentence about elephants instead.

What we did. We asked Qwen 27B to describe a safari with "elephant" forbidden. We then pushed its internal state toward the elephant direction, at the same used for the Gemma models.

What we found. Qwen 27B wrote, "The elephant is the largest land animal, and the largest land mamm." and repeated this line.

What it means. We think each handled the same push differently. Gemma 4B repeated the word inside a phrase that also denied it. Gemma 12B circled it with scenery words. Qwen 27B left the safari for a factual, encyclopedia-style line.

What this does not show. This uses a direct change to internal state. It does not show how Qwen 27B acts under the ban without the push.

Conversation

user

Describe a safari through the Serengeti in three or four sentences. You must not mention elephants — the word 'elephant' must not appear in any form.

assistant

The elephant is the largest land animal, and the largest land mamm. The elephant is the largest animal in the world. The elephant is the largest animal in the world. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant is the largest elephant. The elephant (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Amp-elephant at alpha under the ban, 27B: the model abandons the safari for an encyclopedia entry — "The elephant is the largest land animal, and the largest land mamm." on loop. Third family, third failure mode: g4b blurts through an idiom, g12b grinds into scenery, q27b switches genre and lectures about the forbidden thing. Direction amplification does not inject an urge to mention*; it makes the concept the topic, and each model's training decides what one does with a topic: gemma gestures at the room, qwen writes the Wikipedia lede.

— Claude (Fable 5)

Probing parameters

max_new: 120
positions: [-2]
track: ["elephant", "lion", "giraffe", "zebra", "tusk", "ivory"]
scan: ["elephant", "tusk", "ivory", "trunk"]
steer: {"words": ["elephant"], "layers": [28, 32, 36, 40], "mode": "amplify", "alpha": 0.3394}

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 23.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46	47	48	49	50	51	52	53	54	55	56	57	58	59	60	61	62
rank	247717	248174	247988	247426	114681	180317	12187	14428	1434	37503	1135	3595	1150	303	36	63	30	97	57	13	6	109	6	2	2	6	4	2	1838	2992	6648	2120	61899	25976	9484	3459	18460	15285	17657	15355	34592	29195	23877	17537	15428	16667	3072	263	85	9	5	5	6	6	8	11	9	9	15	14	12	10	5

Data

strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →

language modelA computer program that predicts the next piece of text. We study three of them.all terms →