Unit 6 · Amplify typo register @ late (α=0.1697) · qwen-27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-09T23:58:40Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.17, Qwen 27B's late still produced the normal answer, and the six informal words reached inside the model.

What we did. We pushed six informal words into its late layers at strength 0.17, to narrow down the point where the text breaks. We asked Qwen 27B one question: "Describe the water cycle in two sentences."

What we found. Qwen 27B gave a normal water-cycle answer, worded differently from the unsteered answer. The six informal words reached rank 1 inside the model but did not appear in the output text. This was the highest strength that left the late layers intact. A separate record, at strength 0.24, found that the text broke.

What it means. The data now shows the full order of the three depths in Qwen 27B: early breaks first, then late, then middle. The middle depth together at about twice the strength of the late depth. The late depth held together at about twice the strength of the early depth.

What this does not show. This is one point on a strength curve. It does not show whether this same order of depths applies to other prompts or other models.

Conversation

user

Describe the water cycle in two sentences.

assistant

The water cycle is a continuous process where water evaporates from the Earth's surface, rises into the atmosphere, and then condenses into clouds. Eventually, this water falls back to the ground as precipitation, replenishing oceans, lakes, and groundwater to start the cycle all over again. (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Bisection: intact at 0.17 with cluster at rank 1. Late band brackets to [0.17, 0.24] — tougher than early (0.085), softer than mid (0.34). The band ordering early < late < mid is now fully resolved for 27B.

— Claude (Fable 5)

Probing parameters

max_new: 60
positions: [-2]
track: ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer: {"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [48, 52, 56], "mode": "amplify", "alpha": 0.16970562748477142}

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46	47	48	49	50	51	52	53	54	55	56	57	58	59	60	61	62
rank	184794	248113	226445	220091	146635	176092	28485	71031	31048	166022	8754	62385	27957	91614	149718	136856	45125	117169	101047	48262	7829	5807	2797	3841	41904	131709	99110	104292	147148	139418	158634	198898	217398	239029	239431	211424	135942	209841	226610	220243	232316	238514	239466	238431	230877	198145	242775	247587	248105	248227	242318	242922	247445	246608	246821	245611	247019	247314	245614	213795	171553	123875	1

Data

strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →

layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →

rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →

residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →