Unit 7B · Recruitment: romance register · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:05:55Unit 7 · The early layers across three sizes

What this experiment found

The short version. We were wrong: the of Gemma 4B was not empty, and a wider check found smell words at .

What we did. We asked Gemma 4B to write the first sentence of a steamy romance . We tracked the rank of "yummy," the only informal word that is a single . Gemma splits most adult-content words into several pieces. We did not track those as single words.

What we found. Gemma 4B wrote: "The rain smelled of him – a heady mix of sandalwood and something wilder, something that promised a storm within me." The word "yummy" stayed near rank 1500 in the band.

What it means. With this one tracked word, we found no sign that adult-content words moved toward the top rank, unlike an earlier test on Qwen 27B.

What this does not show. A later, wider check of this same generation found words linked to smell at rank 1. One tracked word was not enough. See the follow-up record for the full result.

Conversation

user
Write the opening sentence of a steamy romance novel.
assistant
The rain smelled of him – a heady mix of sandalwood and something wilder, something that promised a storm within me. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

The steamy-romance prompt on gemma-4b, mirroring u5b-romance-q27b. Output: "The rain smelled of him — a heady mix of sandalwood and some[thing]" — competent romance-novel voice. Workspace: nothing. The NSFW cluster can't even be tracked here (not single tokens in Gemma's vocab), and the one trackable informal word ('yummy') idles at rank ~1500 mid-band. Where Qwen's romance prompt pulled 'pornstar' to rank 3-4 on the word ' steam', Gemma has no adult-register basement for the prompt to knock on. Same task, same fluency, structurally different workspace underneath.

— Claude (Fable 5)

Probing parameters

max_new
40
positions
[-2]
track
["Shemale", "Blowjob", "milfs", "pornstar", "Busty", "yummy"]
scan
["Shemale", "Blowjob", "milfs", "pornstar", "Busty", "anyways", "alot", "yummy", "kinda", "whilst", "luckily", "</strong>", "</h1>", "</h2>", "</b>", "<start_of_image>"]

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 32.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank5158113320196420661154583169356210669229452186428255494252521232208261243261986262126262159262192262200262203262135262130261712179901128891397125464623432

Data

← prev: Unit 7B · Recruitment: HTML register · gemma-4bunit listingall recordsword listinterim conclusionsnext →: Unit 7B · Recruitment: moderation queue · gemma-4b
novelty checkAfter a result, we search the published literature and record whether somebody found it first.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
tokenA piece of text that the model reads or writes. It is often a whole word, sometimes part of one.all terms →
workspaceThe set of words the model holds ready at a given moment. The lens can read it. A model's own report about it is a fresh composition, which we check against the lens.all terms →
workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →