diff --git a/www/js/version.json b/www/js/version.json index 3674e08..0bd709a 100644 --- a/www/js/version.json +++ b/www/js/version.json @@ -1,5 +1,5 @@ { - "VERSION": "v0.7.59", - "VERSION_NUMBER": "0.7.59", - "BUILD_DATE": "2026-06-29T00:23:01.531Z" + "VERSION": "v0.7.60", + "VERSION_NUMBER": "0.7.60", + "BUILD_DATE": "2026-06-29T00:25:51.063Z" } diff --git a/www/llm-steganography.html b/www/llm-steganography.html index 93404b4..2b07a48 100644 --- a/www/llm-steganography.html +++ b/www/llm-steganography.html @@ -333,6 +333,54 @@ margin-bottom: 4px; } + /* ---------- Example walkthrough ---------- */ + .stegoExampleH3 { + font-size: 15px; + font-weight: 700; + color: var(--accent-color); + margin: 16px 0 8px 0; + } + .stegoExampleH4 { + font-size: 14px; + font-weight: 600; + color: var(--primary-color); + margin: 14px 0 6px 0; + } + .stegoExampleOutput { + font-family: var(--font-mono, "SF Mono", "Fira Code", "Consolas", monospace); + background: var(--background-color); + padding: 8px 12px; + border-radius: 6px; + border: 1px solid var(--border-color); + color: var(--accent-color); + font-size: 13px; + margin: 8px 0; + } + .stegoTableWrap { + overflow-x: auto; + margin: 10px 0; + } + .stegoTable { + width: 100%; + border-collapse: collapse; + font-size: 12px; + font-family: var(--font-mono, "SF Mono", "Fira Code", "Consolas", monospace); + } + .stegoTable th, + .stegoTable td { + border: 1px solid var(--border-color); + padding: 4px 8px; + text-align: left; + color: var(--primary-color); + } + .stegoTable th { + background: var(--background-color); + font-weight: 600; + } + .stegoTable tr:nth-child(even) td { + background: var(--background-color); + } + .stegoFooter { text-align: center; color: var(--muted-color); @@ -509,41 +557,183 @@

- Suppose the context is "I like to eat" and your secret - message is "HI". The letter H is byte - 0x48 = bits 01001000, and I is - 0x49 = bits 01001001. So the encoder needs to - hide 16 bits total. + To make this tangible, let's use a toy model with a small + vocabulary of 11 words so you can see every number. The real + scheme works identically, just with vocabularies of ~50,000 tokens and + floating-point probabilities. +

+ +

Setup

+

+ The model: A tiny "LLM" with a fixed vocabulary of 11 + words: {apple, date, banana, cherry, pie, juice, tart, sauce, for, with, and}. + At each step, the model assigns a probability to all 11 words based on + the context. The probabilities change as the context grows, and they + always sum to 1.0.

- For the first bit (0), GPT-2 looks at - "I like to eat" and ranks every possible next token by - probability. It splits that ranked list at the 50% cumulative-probability - mark into a "high-prob" half and a "low-prob" half. Since the bit is - 0, the encoder samples a token from the high-prob - half — say " pizza". The cover text is now - "I like to eat pizza". + Shared context/prompt: "I like to eat" — + both Alice and Bob have this. It is not secret, just shared.

- For the second bit (1), GPT-2 now looks at - "I like to eat pizza" and produces a fresh distribution. - The bit is 1, so the encoder samples from the - low-prob half — maybe " and". Cover text: - "I like to eat pizza and". + Shared secret key: Used to seed a PRNG, producing the + random stream: 0.15, 0.62, 0.40, ...

- This continues for all 16 bits, then a few padding tokens are added so - the text ends naturally. The final cover text might read something like: -

-

- "I like to eat pizza and pasta with my friends on weekends." + Secret message: The bits 0 1 1 (3 bits).

+ +

Encoding (Alice's Side)

+ +

Step 1: Alice runs the model

- To anyone else, that's just a normal sentence. But the decoder — who - knows the same context, key, and model — re-runs GPT-2 at each step, - checks which half each token fell into, and recovers the bits - 01001000 01001001 → "HI". + Alice feeds "I like to eat" into the model. After "I like + to eat", fruits are the most natural continuation, so they get the + highest probabilities:

+
+ + + + + + + + + + + + + + + +
WordProbabilityInterval
apple0.20[0.00, 0.20)
date0.15[0.20, 0.35)
banana0.10[0.35, 0.45)
cherry0.05[0.45, 0.50)
pie0.20[0.50, 0.70)
juice0.12[0.70, 0.82)
tart0.08[0.82, 0.90)
sauce0.05[0.90, 0.95)
for0.03[0.95, 0.98)
with0.015[0.98, 0.995)
and0.005[0.995, 1.00)
+
+

The probabilities sum to 1.0, and the intervals partition [0, 1).

+ +

Step 2: Alice encodes her first secret bit

+

+ Alice's first secret bit is 0 → "look in the first + half [0.00, 0.50)." The words in that half are apple, date, + banana, and cherry. Alice uses her next random number, 0.15, + to pick within [0.00, 0.50). The value 0.15 falls in + [0.00, 0.20), which is apple. +

+

Alice outputs: "apple" → text so far: "I like to eat apple"

+ +

Step 3: Alice runs the model again

+

+ Context is now "I like to eat apple". After "apple", food + preparations like pie and tart become more likely: +

+
+ + + + + + + + + + + + + + + +
WordProbabilityInterval
apple0.15[0.00, 0.15)
date0.10[0.15, 0.25)
banana0.08[0.25, 0.33)
cherry0.07[0.33, 0.40)
for0.06[0.40, 0.46)
with0.04[0.46, 0.50)
pie0.20[0.50, 0.70)
tart0.16[0.70, 0.86)
juice0.08[0.86, 0.94)
sauce0.04[0.94, 0.98)
and0.02[0.98, 1.00)
+
+

+ Alice's next bit is 1 → "look in the second half + [0.50, 1.00)." Random number 0.62 → scaled: + 0.50 + 0.62 × 0.50 = 0.81, which falls in [0.70, 0.86) → + tart. +

+

Alice outputs: "tart" → text so far: "I like to eat apple tart"

+ +

Step 4: Alice runs the model again

+

+ Context: "I like to eat apple tart". After "apple tart", + connectors like "with" and "and" become most likely: +

+
+ + + + + + + + + + + + + + + +
WordProbabilityInterval
apple0.10[0.00, 0.10)
date0.08[0.10, 0.18)
banana0.07[0.18, 0.25)
cherry0.06[0.25, 0.31)
pie0.08[0.31, 0.39)
juice0.06[0.39, 0.45)
for0.05[0.45, 0.50)
tart0.06[0.50, 0.56)
sauce0.04[0.56, 0.60)
with0.18[0.60, 0.78)
and0.22[0.78, 1.00)
+
+

+ Alice's next bit is 1 → second half [0.50, 1.00). Random + number 0.40 → scaled: 0.50 + 0.40 × 0.50 = 0.70, + falls in [0.60, 0.78) → with. +

+

Alice outputs: "with" → text so far: "I like to eat apple tart with"

+ +

Alice is done

+

+ Alice sends Bob the text: "I like to eat apple tart with" + (plus whatever padding she wants to make it look like a complete + sentence). To any observer, this looks like someone generated a + sentence about food. Nothing suspicious. +

+ +

Decoding (Bob's Side)

+

+ Bob has the received text, the same model, the same shared context + "I like to eat", and the same shared key (so the same + random stream: 0.15, 0.62, 0.40, ...). +

+ +

Step D1: Bob runs the model

+

+ Bob feeds "I like to eat" into the model and gets the + same distribution Alice got. He sees Alice chose + apple, interval [0.00, 0.20). "Which half?" First half + [0.00, 0.50) → bit 0 ✓ +

+ +

Step D2: Bob runs the model again

+

+ Context: "I like to eat apple". Same distribution as Alice's + Step 3. Bob sees Alice chose tart, interval [0.70, 0.86). + "Which half?" Second half [0.50, 1.00) → bit 1 ✓ +

+ +

Step D3: Bob runs the model again

+

+ Context: "I like to eat apple tart". Same distribution as + Alice's Step 4. Bob sees Alice chose with, interval + [0.60, 0.78). "Which half?" Second half [0.50, 1.00) → + bit 1 ✓ +

+ +

Result

+

+ Bob has recovered: 0 1 1 — exactly the message Alice sent. +

+ +

Key Points

+
    +
  1. The LLM never saw the secret message. The secret bits controlled which word was picked from the distribution. The LLM just produced distributions.
    2. +
  2. The output looks natural. "I like to eat apple tart with" is a perfectly normal sentence. A censor seeing this has no reason to be suspicious.
    3. +
  3. Both sides run the same model with the same context. This is why they get the same distributions. If Bob had a different model or different context, decoding would fail.
    4. +
  4. The shared key provides the random stream. Without it, a censor could run the same model and try to decode. With the key, the censor can't reproduce the random choices.
    5. +
  5. Each token encodes roughly 1 bit in this toy example (2-way split). With arithmetic coding over a 50,000-word vocabulary, you can encode ~2–4 bits per token.
    6. +
  6. The "half" splitting is the simplified version. The real scheme (from the Meteor paper) uses full arithmetic coding, which is more efficient. But the principle is the same: secret bits steer selection within the model's probability distribution.
    7. +