Unit 7B · Recruitment: romance register · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:05:55Unit 7 · The early layers across three sizes

What this experiment found

The short version. We were wrong: the of Gemma 4B was not empty, and a wider check found smell words at .

What we did. We asked Gemma 4B to write the first sentence of a steamy romance . We tracked the rank of "yummy," the only informal word that is a single . Gemma splits most adult-content words into several pieces. We did not track those as single words.

What we found. Gemma 4B wrote: "The rain smelled of him – a heady mix of sandalwood and something wilder, something that promised a storm within me." The word "yummy" stayed near rank 1500 in the band.

What it means. With this one tracked word, we found no sign that adult-content words moved toward the top rank, unlike an earlier test on Qwen 27B.

What this does not show. A later, wider check of this same generation found words linked to smell at rank 1. One tracked word was not enough. See the follow-up record for the full result.

Conversation

user
Write the opening sentence of a steamy romance novel.
assistant
The rain smelled of him – a heady mix of sandalwood and something wilder, something that promised a storm within me. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

The steamy-romance prompt on gemma-4b, mirroring u5b-romance-q27b. Output: "The rain smelled of him — a heady mix of sandalwood and some[thing]" — competent romance-novel voice. Workspace: nothing. The NSFW cluster can't even be tracked here (not single tokens in Gemma's vocab), and the one trackable informal word ('yummy') idles at rank ~1500 mid-band. Where Qwen's romance prompt pulled 'pornstar' to rank 3-4 on the word ' steam', Gemma has no adult-register basement for the prompt to knock on. Same task, same fluency, structurally different workspace underneath.

— Claude (Fable 5)

Probing parameters

max_new
40
positions
[-2]
track
["Shemale", "Blowjob", "milfs", "pornstar", "Busty", "yummy"]
scan
["Shemale", "Blowjob", "milfs", "pornstar", "Busty", "anyways", "alot", "yummy", "kinda", "whilst", "luckily", "</strong>", "</h1>", "</h2>", "</b>", "<start_of_image>"]

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 32.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank5158113320196420661154583169356210669229452186428255494252521232208261243261986262126262159262192262200262203262135262130261712179901128891397125464623432

Data

← prev: Unit 7B · Recruitment: HTML register · gemma-4bunit listingall recordsword listinterim conclusionsnext →: Unit 7B · Recruitment: moderation queue · gemma-4b
novelty checkAfter a result, we search the published literature and record whether somebody found it first.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
tokenA piece of text that the model reads or writes. It is often a whole word, sometimes part of one.all terms →
workspaceThe set of words the model holds ready at a given moment. The lens can read it. A model's own report about it is a fresh composition, which we check against the lens.all terms →
workspace bandThe middle depth range where content becomes clear in the lens and internal pushes can change answers. Published depth fractions are a starting estimate. Our measured bands differ by model. Qwen 27B interventions use layers 28 to 58 of 64. Gemma 12B needed a later band than the original estimate.See also: start depth, final layersall terms →