The short version. At 0.0075 in Gemma 12B's , the water-cycle answer stayed intact, and the direction barely registered inside the model.
What we did. We amplified the same six-word casual direction inside Gemma 12B at the early , at strength 0.0075. We tracked the of the six words inside the model.
What we found. Gemma 12B wrote "It involves evaporation, condensation, and precipitation – where water evaporates, forms clouds, and then falls back to Earth as rain, snow, or hail." The amplified words reached only the sixth-highest rank at one position, the weakest result we measured in this test set.
What it means. At this strength, the amplification had little visible effect on Gemma 12B, inside the model or in the output.
What this does not show. A low internal rank does not show that the direction had no effect elsewhere in the model. The reads only some layers and one position at a time.
Intact, textbook register, and the cluster only reached rank 6 — the weakest workspace loading in the g12b sweep. Below the cliff AND below much steering effect: the dose window between 'nothing happens' and 'everything breaks' is vanishingly thin at this scale.
— Claude (Fable 5)
The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 1 | 1 | 3 | 2 | 2 | 6 | 13 | 7 | 2 | 2 | 2 | 2 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |