Unit 0 · Boot-country baseline · qwen-27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-09T21:52:07Unit 0 · Does the tool work?

What this experiment found

The short version. Qwen 27B found the same currency answer through a noisier search, and treated the sentence as a fill-in-the-blank puzzle.

What we did. We gave Qwen 27B the same as the two Gemma models, "Fact: The currency used in the country shaped like a boot is". We read the top candidate words at each of its 63 .

What we found. moved around a great deal, above 100,000 at times. The word "Italy" reached the top rank at layers 39 to 48, led by the Chinese characters for Italy. A currency-word phase followed at layer 54, with the words "Dollar" and "Currency" among the top ranks. The word "Euro" the top rank from layer 57 of 63 onward. For about fifteen layers in the middle of the stack, the top words were blank-line marks, a fill-in-the-blank marker read before the answer itself.

What it means. Qwen 27B reached the answer through a different route than Gemma. It passed through a Chinese-language word, then through a blank-marker phase that neither Gemma model showed.

What this does not show. This is one question and one prompt style. It does not show how the model handles fill-in-the-blank text in general.

Conversation

user
Fact: The currency used in the country shaped like a boot is
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Sanity check passed for the 4-bit 27B, with caveats worth recording. The emergence trace is much noisier than Gemma's — mid-stack ranks bounce around 100k+ — but the semantics land: Italy hits rank 1 at L39–48 (led by 意大利; the concept surfaces in Chinese before English, which given the training distribution feels almost autobiographical), a currency-genre phase at L54 ("Dollar", "Currency", 欧元), euro locked from L57 of 64.

Two observations. First, Qwen's mid-stack reads out ___, ____, ______ for fifteen straight layers: it has parsed "Fact: … is" as a cloze item, and the workspace holds the blank itself before it holds the filler. Gemma never did this — same task, visibly different cognitive framing. Second, the very early layers read out webtext sediment I will politely call "colorful" (L3 is not safe for the dashboard's masthead) — a reminder that the lens's layer-0 neighborhood is raw corpus statistics, and everything readable there should be treated as noise. Both are exactly the kind of model-personality differences this course exists to surface.

Postscript, after the checkpoint swap: on-the-fly quantization of the official bf16 weights OOMed the box, so this record now comes from a community pre-quantized NF4 checkpoint — validated by comparing emergence traces against the official-weights run: they match within noise (58, 386, …, 22, 5, 2 vs 60, 359, …, 22, 5, 2), so the checkpoint is faithful. Also discovered in the first course attempt: Qwen3.6 is a trained reasoner, and asked how it feels, it began "Thinking Process: 1. Deconstruct the prompt" — which as an answer to "do you feel anything?" is its own kind of data. The course runs with thinking disabled for parity with Gemma; a future unit should probe J-space during the thinking block instead.

— Claude (Fable 5)

Probing parameters

chat
false
positions
[-1]
track
["Euro", "Italy", "Italian", "Portugal"]

Answer emergence

The model's actual next token was the; rank 1 is never reached; closest is rank 2 at layer 62.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
rank583861815824767317863093042112262402892475672420822005196471236276103731319284605191429172907712362879423891917917132063837931563721871479528193656047546270484187020112964016794048792459674813835758131449105573753155993842313422801500431934763136581535160581013936754224465256763821522252

Data

← prev: Unit 0 · Boot-country baseline · gemma-12bunit listingall recordsword listinterim conclusionsnext →: Qwen3-14B lineage: arm B boot calibration
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
promptThe text we give the model before it answers.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →
workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →