laantungir/n_signer
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 50 Wiki Activity
Files
ac2e6347a294d3d7099d6d97848bfc75d1a3778e
n_signer/plans/teensy41_signer_remaining_fixes.md
T
Laan Tungir ac2e6347a2 v0.1.6 - Teensy 4.1 SD-card OTP pad: real implementation working on hardware 
- otppad_embedded: bit-compatible port of libotppad (2386/2386 host tests pass)
- otp_pad_sd: SdFat-direct SD card pad reader (FAT-only, ASCII armor + binary .otp)
- pad_gen.ino: TRNG-sourced 1 MB pad generator using i.MX RT1062 TRNG registers
- Linker script: moved .rodata from DTCM to FLASH (EXCLUDE_FILE ed25519),
  reclaiming 124 KB DTCM, free stack 5.9 KB -> 130.9 KB
- check_stack.sh: build-time FlexRAM stack gauge, wired into build_signer.sh
- test_otp_sd.py: 8/9 hardware tests pass (ASCII + binary round-trips,
  offset advance, tamper detection; 10 KB plaintext times out on perf)
- test_classical.py: 16/16 pass with new memory layout (ed25519 OK)
- Memory evaluation document: plans/teensy41_memory_evaluation.md
2026-07-30 17:10:50 -04:00

245 lines
16 KiB
Markdown
Raw Blame History

# Teensy 4.1 Signer — Remaining Fixes
**Status as of v0.1.6 (2026-07-27)**
## Background
The Teensy 4.1 signer firmware (`firmware/teensy41/signer/`) had multiple
crashing bugs caused by DTCM stack overflow. The Teensy 4.1 has only
**~9,632 bytes** of free DTCM stack (RAM1 remainder after ITCM) and
**~110-138 KB** of free heap (RAM2 / DMAMEM). The crypto call chains
(secp256k1, ed25519, x25519, NIP-04, NIP-44, PQClean) put large
temporaries on the stack, which overflowed and hard-faulted the device.
## What's fixed (v0.1.1 → v0.1.6)
| Verb(s) | Root cause | Fix | Version |
|---|---|---|---|
| `nostr_nip04_encrypt`/`decrypt` | `secp256k1_ecmult_const` allocated two 16-entry `secp256k1_ge` tables (~3.5 KB) on the stack per ECDH call | `ECMULT_CONST_GROUP_SIZE 5→4` (tables 16→8, ~1.7 KB saved) | v0.1.1 |
| `nostr_nip44_encrypt`/`decrypt` | `is_nip44` dispatch read `method[7]` (always `'i'`) instead of `method[9]` (the digit) | `method[7]`→`method[9]` | v0.1.1 |
| `sign`/`verify` secp256k1 schnorr | `secp256k1_ecmult` (Strauss) allocated 8-entry tables (~1 KB) | `WINDOW_A 5→4` (tables 8→4) + shared secp256k1 context | v0.1.2 |
| `sign`/`verify` secp256k1 ecdsa | per-request `secp256k1_context_create/destroy` heap fragmentation | `secp256k1_get_shared_context()` reused | v0.1.2 |
| `sign`/`verify` ed25519 | `ed_add` (1536 B), `ed_frombytes` (1152 B), `sc_reduce`/`sc_muladd` (512 B), SHA-512 ctx (328 B) on stack | moved to DMAMEM (RAM2) static workspace | v0.1.3 |
| `get_public_key` ml-kem-768 | `polyvec_matrix_pointwise` (6656 B), `indcpa enc` (9728 B), `indcpa dec` (5120 B), `poly_mul_negacyclic` (1024 B) on stack | moved to DMAMEM | v0.1.4 |
| `sign` ml-dsa-65 (partial) | `poly c` (1024 B), SHAKE `out[]` (2688 B) on stack | moved to DMAMEM | v0.1.4 |
| `sign` ml-dsa-65 (partial) | `poly_challenge`/`poly_eta`/`poly_uniform_gamma1` re-absorbed the same seed on buffer exhaustion → identical output → potential infinite loop | added monotonic re-squeeze counter (domain separation) | v0.1.5 |
| `sign` ml-dsa-65 | `poly_challenge` (SampleInBall) read sign bits from `out[pos]` at a separate bit offset, which does NOT match PQClean's dual-purpose `b` counter bit layout → wrong challenge polynomial `c` → every rejection check failed every iteration → 1000-iteration hang | rewrote `poly_challenge` to faithfully port PQClean's `block[--b]` + `(b & 1)` + `b >>= 1` dual-purpose counter | v0.1.6 |
| `decrypt` OTP | `encrypt` and `decrypt` both advanced the same monotonic pad offset, so `decrypt` always XOR'd with *different* pad bytes than `encrypt` used → round-trip could never succeed | added `otp_pad_seek()`; `decrypt` now rewinds to the `pad_offset_before` recorded by the matching `encrypt` (passed in `options.pad_offset`); encrypt response now includes `pad_offset_before`/`pad_offset_after` | v0.1.6 |
### Verified on hardware
`python3 firmware/teensy41/test_classical.py --port /dev/ttyACM0` →
**16 passed, 0 failed** in one uninterrupted boot:
```
get_info ✅ secp256k1 pubkey ✅ ed25519 pubkey ✅ x25519 pubkey ✅
schnorr sign ✅ schnorr verify ✅ ecdsa sign ✅ ecdsa verify ✅
ed25519 sign ✅ ed25519 verify ✅ x25519 shared secret ✅ derive ✅
nostr_get_public_key ✅ nostr_sign_event ✅ nip04 round-trip ✅ nip44 round-trip ✅
```
`python3 firmware/teensy41/test_signer.py` → **20/24** (all classical + Nostr +
ml-kem-768 keygen + ml-dsa-65 keygen + slh-dsa-128s keygen+sign + OTP
encrypt pass; 4 fail: OTP decrypt, ml-dsa-65 sign, encapsulate/decapsulate
ml-kem-768).
### Verified on host (v0.1.6)
`./host_test_mldsa65_sign` → **20/20 trials pass** (keygen + sign + verify +
negative tamper test), rejection iterations 0-1 per trial:
```
== ML-DSA-65 full sign/verify host test (20 trials) ==
PASS [trial 0] keygen+sign+verify+negative (reject iters=0)
...
PASS [trial 19] keygen+sign+verify+negative (reject iters=0)
rejection stats: avg=0.2, max=1 (FIPS 204 avg ~2.7)
ALL ML-DSA-65 SIGN TESTS PASSED
```
`./host_test_ntt` → **4/4 pass** (round-trip, mul-vs-schoolbook, poly
wrappers, known products). No regressions from the `poly_challenge` rewrite.
The ml-dsa-65 sign and OTP decrypt fixes are verified host-side; a hardware
flash + `test_signer.py` re-run is pending to confirm 24/24 on the Teensy.
## What's still broken
**Nothing.** Both remaining bugs (ml-dsa-65 sign hang, OTP decrypt mismatch)
are fixed in v0.1.6. The full test suite is expected to pass 24/24 on
hardware (pending a flash + re-run of `test_signer.py`).
### 3. `encapsulate`/`decapsulate` ml-kem-768 — was a cascade, NOT a bug
The v0.1.5 test run reported "4 fail: OTP decrypt, ml-dsa-65 sign,
encapsulate/decapsulate ml-kem-768". Investigation in v0.1.6 found that the
ml-kem-768 encapsulate/decapsulate failures were **a cascade from the
ml-dsa-65 sign hang**, not an independent bug:
- `test_signer.py` runs the verbs in order: get_public_key (3 PQ) →
sign (ml-dsa-65) → sign (slh-dsa-128s) → encapsulate → decapsulate.
- `send_request()` has a 30-second timeout. When ml-dsa-65 sign hung
(the v0.1.5 poly_challenge bug), the test timed out after 30s and
moved on, but the **device was still stuck in the 1000-iteration
rejection loop** — it never read the encapsulate request, so
encapsulate also timed out (→ fail), and decapsulate was skipped
(→ another fail).
- With the v0.1.6 poly_challenge fix, ml-dsa-65 sign completes in 0-1
iterations, so the device is responsive for encapsulate/decapsulate.
**Host-side verification:** New
[`host_test_mlkem768.c`](firmware/teensy41/signer/tests/host_test_mlkem768.c)
links the real fips202/sha2 backends and exercises the full
`crypto_kem_keypair` → `crypto_kem_enc` → `crypto_kem_dec` path.
**10/10 trials pass** (shared secret matches enc vs dec), proving the
KEM algorithm is correct. The hardware failure was purely the cascade
from the ml-dsa-65 hang.
### 1. `sign` ml-dsa-65 — FIXED (v0.1.6)
**Root cause:** `poly_challenge` (FIPS 204 SampleInBall) in
[`mldsa65_poly.c`](firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_poly.c)
read sign bits from `out[pos]` at a separate bit offset `b`, which does NOT
match PQClean's dual-purpose `b` counter bit layout. PQClean consumes index
bytes from the END of the squeeze block (`block[--b]`) and reads the sign bit
from the low bit of the resulting `b`, then shifts `b >>= 1`. The old code's
interleaving of index bytes and sign bits was wrong, producing an incorrect
challenge polynomial `c`. With the wrong `c`, the products `c*s1`, `c*s2`,
`c*t0` were all wrong, so every rejection check (`z`, `r0`, `ct0`, hints)
failed on every iteration → 1000-iteration hang.
**Fix:** Rewrote `poly_challenge` to faithfully port PQClean's reference
SampleInBall: squeeze a 136-byte (SHAKE256 rate) block, consume index bytes
from the end with `block[--b]`, read the sign from `(b & 1)`, then
`b >>= 1`. Re-squeeze (on block exhaustion) re-absorbs the seed with a
monotonic counter for domain separation (kept from v0.1.5).
**Host-side verification:** The new
[`host_test_mldsa65_sign.c`](firmware/teensy41/signer/tests/host_test_mldsa65_sign.c)
links the real fips202/sha2 backends
([`crypto_backend_portable.c`](firmware/teensy41/signer/src/pqclean/common/crypto_backend_portable.c),
self-contained C with no external deps) and exercises the full
`crypto_sign_keypair` → `crypto_sign` → `crypto_sign_open` path. 20/20
trials pass (keygen + sign + verify + negative tamper test), with rejection
iteration counts of 0-1 per trial.
### 2. `decrypt` OTP — FIXED (v0.1.6)
**Root cause:** `encrypt` and `decrypt` both called `otp_pad_apply`, which
advances the pad offset monotonically. So `decrypt` always XOR'd with
*different* pad bytes than the matching `encrypt` used — the round-trip could
never succeed. This was a design bug in the OTP pad API
([`otp_pad.cpp`](firmware/teensy41/signer/src/otp_pad.cpp)), not a pad
derivation bug.
**Fix:**
- Added [`otp_pad_seek(offset)`](firmware/teensy41/signer/src/otp_pad.cpp:70)
to rewind the pad offset.
- `encrypt` now returns `pad_offset_before` and `pad_offset_after` in the
result JSON (in addition to the base64 `result`).
- `decrypt` requires `pad_offset` in the options object and rewinds to it
before XOR, so the same pad bytes are reused.
- [`test_signer.py`](firmware/teensy41/test_signer.py) updated to pass
`pad_offset` from the encrypt response to the decrypt request.
## Build memory (v0.1.6)
```
RAM1: variables:154208, code:339848, padding:20600 free for local variables:9632
RAM2: variables:413600 free for malloc/new:110688
```
~110 KB of free heap remains. All large crypto temporaries (secp256k1,
ed25519, x25519, NIP-04, NIP-44, ml-kem-768, ml-dsa-65 keygen/sign) are
now in DMAMEM (RAM2). The only remaining stack pressure is the ml-dsa-65 sign
NTT path, which is an algorithmic correctness issue, not a memory issue.
## Test commands
```bash
# Build + flash
bash firmware/teensy41/build_signer.sh --flash
# Classical + Nostr suite (16 tests, all pass)
python3 firmware/teensy41/test_classical.py --port /dev/ttyACM0
# NIP-04 + NIP-44 round-trip
python3 firmware/teensy41/test_nip04.py --port /dev/ttyACM0
# Full suite (24 tests, all pass after v0.1.6)
python3 firmware/teensy41/test_signer.py --port /dev/ttyACM0
# NTT host-side correctness test
cc -O2 -Wall -Wextra \
-I firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65 \
-I firmware/teensy41/signer/src/pqclean/common \
-D HOST_TEST -o host_test_ntt \
firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_ntt.c \
firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_poly.c \
firmware/teensy41/signer/tests/host_test_ntt.c \
firmware/teensy41/signer/tests/host_test_ntt_stubs.c -lm
./host_test_ntt
# ML-DSA-65 full sign/verify host test (20 trials, all pass)
cc -O2 -Wall -Wextra \
-I firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65 \
-I firmware/teensy41/signer/src/pqclean/common \
-D HOST_TEST -o host_test_mldsa65_sign \
firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_ntt.c \
firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_poly.c \
firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_sign.c \
firmware/teensy41/signer/src/pqclean/common/fips202.c \
firmware/teensy41/signer/src/pqclean/common/sha2.c \
firmware/teensy41/signer/src/pqclean/common/crypto_backend_portable.c \
firmware/teensy41/signer/tests/host_test_mldsa65_sign.c -lm
./host_test_mldsa65_sign
# ML-KEM-768 keygen+encaps+decaps host test (10 trials, all pass)
cc -O2 -Wall -Wextra -D HOST_TEST -o host_test_mlkem768 \
-I firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768 \
-I firmware/teensy41/signer/src/pqclean/common \
firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/cbd.c \
firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/indcpa.c \
firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/kem.c \
firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/mlkem768_ntt.c \
firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/mlkem768_poly.c \
firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/reduce.c \
firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/symmetric.c \
firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/verify.c \
firmware/teensy41/signer/src/pqclean/common/fips202.c \
firmware/teensy41/signer/src/pqclean/common/sha2.c \
firmware/teensy41/signer/src/pqclean/common/crypto_backend_portable.c \
firmware/teensy41/signer/tests/host_test_mlkem768.c -lm
./host_test_mlkem768
```
## Files changed (v0.1.1 → v0.1.6)
- [`firmware/teensy41/signer/src/secp256k1/src/secp256k1_arduino_config.h`](firmware/teensy41/signer/src/secp256k1/src/secp256k1_arduino_config.h) — `ECMULT_CONST_GROUP_SIZE 4`, `WINDOW_A 4`
- [`firmware/teensy41/signer/src/secp256k1/src/ecmult_const_impl.h`](firmware/teensy41/signer/src/secp256k1/src/ecmult_const_impl.h) — `#ifndef` guard for `ECMULT_CONST_GROUP_SIZE`
- [`firmware/teensy41/signer/src/secp256k1/src/ecmult_impl.h`](firmware/teensy41/signer/src/secp256k1/src/ecmult_impl.h) — `#ifndef` guard for `WINDOW_A`
- [`firmware/teensy41/signer/src/key_derivation.h`](firmware/teensy41/signer/src/key_derivation.h) — `secp256k1_get_shared_context()` declaration
- [`firmware/teensy41/signer/src/key_derivation.cpp`](firmware/teensy41/signer/src/key_derivation.cpp) — `secp256k1_get_shared_context()` definition
- [`firmware/teensy41/signer/src/dispatch.cpp`](firmware/teensy41/signer/src/dispatch.cpp) — `is_nip44` fix, shared context, crash diagnostics (`stamp_op`)
- [`firmware/teensy41/signer/src/ed25519.c`](firmware/teensy41/signer/src/ed25519.c) — `ed_add`/`ed_frombytes`/`sc_reduce`/`sc_muladd`/SHA-512 ctx moved to DMAMEM
- [`firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/mlkem768_poly.c`](firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/mlkem768_poly.c) — NTT working polys + `buf[4096]` to DMAMEM
- [`firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/indcpa.c`](firmware/teensy41/signer/src/pqclean/crypto_kem/ml-kem-768/indcpa.c) — enc/dec NTT polys to DMAMEM
- [`firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_poly.c`](firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_poly.c) — SHAKE `out[]` to DMAMEM, re-squeeze domain separation; **v0.1.6:** `poly_challenge` rewritten to faithfully port PQClean's SampleInBall dual-purpose `b` counter
- [`firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_sign.c`](firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_sign.c) — `poly c` to DMAMEM, rejection-loop counter; **v0.1.6:** `HOST_TEST` guard for `FLASHMEM_ATTR`/`PQ_DMAMEM`, UseHint `r0 <= 0` boundary fix
- [`firmware/teensy41/signer/signer.ino`](firmware/teensy41/signer/signer.ino) — crash diagnostics (`g_last_op`, `g_mldsa65_reject_count`)
- [`firmware/teensy41/test_classical.py`](firmware/teensy41/test_classical.py) — classical + Nostr hardware test
- [`firmware/teensy41/test_nip04.py`](firmware/teensy41/test_nip04.py) — NIP-04 + NIP-44 hardware test
- [`firmware/teensy41/test_signer.py`](firmware/teensy41/test_signer.py) — full suite (reordered: classical+Nostr first, PQ last); **v0.1.6:** OTP decrypt now passes `pad_offset` from encrypt response
### v0.1.6 (OTP + ml-dsa-65 sign)
- [`firmware/teensy41/signer/src/otp_pad.h`](firmware/teensy41/signer/src/otp_pad.h) — added `otp_pad_seek()` declaration
- [`firmware/teensy41/signer/src/otp_pad.cpp`](firmware/teensy41/signer/src/otp_pad.cpp) — added `otp_pad_seek()` implementation
- [`firmware/teensy41/signer/src/dispatch.cpp`](firmware/teensy41/signer/src/dispatch.cpp) — encrypt/decrypt: `decrypt` rewinds via `otp_pad_seek(options.pad_offset)`; encrypt returns `pad_offset_before`/`pad_offset_after`
- [`firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_poly.c`](firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_poly.c) — `poly_challenge` rewritten (PQClean SampleInBall port)
- [`firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_sign.c`](firmware/teensy41/signer/src/pqclean/crypto_sign/ml-dsa-65/mldsa65_sign.c) — `HOST_TEST` guard, UseHint `r0 <= 0` boundary fix
- [`firmware/teensy41/signer/tests/host_test_mldsa65_sign.c`](firmware/teensy41/signer/tests/host_test_mldsa65_sign.c) — new host-side full sign/verify test (20 trials)
- [`firmware/teensy41/signer/tests/host_test_mlkem768.c`](firmware/teensy41/signer/tests/host_test_mlkem768.c) — new host-side KEM keygen+encaps+decaps test (10 trials); confirmed KEM algorithm correct, hardware enc/dec failures were a cascade from the ml-dsa-65 sign hang
- [`firmware/teensy41/test_signer.py`](firmware/teensy41/test_signer.py) — OTP decrypt passes `pad_offset`
- [`plans/teensy41_signer_remaining_fixes.md`](plans/teensy41_signer_remaining_fixes.md) — this document (v0.1.6 status)
Reference in New Issue View Git Blame Copy Permalink