Unit 6 · Baseline (unsteered water cycle) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:02:37Unit 6 · How hard can we push?

What this experiment found

The short version. In this run, Gemma 4B described the water cycle correctly, with no change made to its internal state.

What we did. We asked Gemma 4B to describe the water cycle in two sentences, with no . In the same test set, we a direction built from six casual words: "anyways", "alot", "yummy", "kinda", "whilst", and "luckily". We tested it at the early, middle, and later , each at several strengths.

What we found. Gemma 4B wrote a normal, textbook description of the water cycle. Across the full test set, we measured that Gemma 4B's output broke somewhere between 0.0106 and 0.015, in both the early and the . In the later layers, it broke between strength 0.0212 and 0.03.

What it means. Gemma 4B has a narrow safe range for this kind of amplification, about a factor of two in strength. At strength 0.0106 in the middle layers, the wording had already turned plainer and more spoken, though the sentence stayed correct.

What this does not show. This run shows only the unsteered baseline. Each strength in this test set has its own separate record.

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is the continuous movement of water on, above, and below the surface of the Earth. It begins with evaporation, where water turns into vapor and rises into the atmosphere, then condenses into clouds and returns to Earth as precipitation like rain or snow, eventually flowing back into bodies of (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

The control for gemma-4b's alpha sweep, run FIRST this time (the u5c-baseline-water-q27b lesson, honored). Textbook water cycle, formal register. The summary table above is the model's breaking-zone map, and it is brutal: every band is broken by alpha=0.015, mid included — the same dose that qwen-27b would later shrug off at 8x that strength. The intact/broken boundary sits at ~0.011/0.015 for early and mid, and ~0.021/0.03 for late. Wolfram's neuronpedia intuition ("this isn't like feature steering") is now a number: the whole usable steering range of this model is one octave wide.

Worth noting what "intact" hides: at alpha=0.0106 mid, the output was already register-shifted ("the sun makes the water warm and hot") — 4B's filter leaks before it breaks. The 27B filter, by contrast, held its register right up to its own cliff. Filter capacity looks like the thing that scales.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]

Answer emergence

The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 32).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank111152211111111111111111111111111

Data

← prevunit listingall recordsword listinterim conclusionsnext →: Unit 6 · Amplify typo register @ early (α=0.06) · gemma-4b
amplificationWe increase a direction in the model's internal state and see whether the answer changes.See also: matched control, strengthall terms →
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
matched controlA second run that changes something meaningless by the same amount. Without it, any change we see could be the push itself.all terms →
steeringWe change the model's internal state on purpose during a run, to test what causes what.all terms →
workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →