n_signer/plans/teensy41_memory_evaluation.md

# Teensy 4.1 Signer — Memory Budget Evaluation

**Date:** 2026-07-30
**Context:** The SD-card OTP pad ([`plans/teensy41_otp_sd_pad.md`](teensy41_otp_sd_pad.md))
is blocked. Root cause turned out to be a **DTCM stack shortage**, not an
"SD library incompatibility". This document re-derives the memory budget from
scratch and proposes solutions.

---

## 1. How Teensy 4.1 memory actually works

The i.MX RT1062 has three separate RAM regions plus flash:

```
┌───────────────────────────────────────────────────────────────────────────┐
│ FLASH  8 MB  (7936 KB usable)                       @ 0x60000000          │
│   .text.code   — code + rodata routed here by the linker script           │
│   .text.itcm   — LOAD image of ITCM code (copied to ITCM at boot)         │
│   .data        — LOAD image of DTCM data (copied to DTCM at boot)          │
├───────────────────────────────────────────────────────────────────────────┤
│ FLEXRAM  512 KB = 16 banks × 32 KB   @ 0x00000000 (ITCM) / 0x20000000 (DTCM)│
│   Split between ITCM and DTCM AT BOOT by _flexram_bank_config.             │
│   ITCM = code that runs at full speed (zero wait state)                   │
│   DTCM = .data + .bss + THE STACK                                         │
├───────────────────────────────────────────────────────────────────────────┤
│ RAM2 / OCRAM  512 KB                                @ 0x20200000          │
│   .bss.dma (DMAMEM statics) + the malloc heap                             │
├───────────────────────────────────────────────────────────────────────────┤
│ ERAM / PSRAM  0 MB (unpopulated)                    @ 0x70000000          │
│   Linker script reserves 32 MB but the chips are NOT soldered on.          │
└───────────────────────────────────────────────────────────────────────────┘
```

The FlexRAM split is computed by the linker script
([`imxrt1062_t41_flashmem.ld:215`](../firmware/teensy41/signer/imxrt1062_t41_flashmem.ld:215)):

```ld
_itcm_block_count   = (SIZEOF(.text.itcm) + SIZEOF(.ARM.exidx) + 0x7FFF) >> 15;
_estack             = ORIGIN(DTCM) + ((16 - _itcm_block_count) << 15);
```

**This is the crux:** every 32 KB bank given to ITCM is taken away from DTCM.
Code size therefore directly steals stack space. And crucially:

```ld
.data : {
    *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.rodata*)))   ◄── READ-ONLY DATA IN DTCM!
    *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.data*)))
} > DTCM  AT> FLASH
```

**`.rodata` (const tables, string literals, fonts, wordlists) is being placed
in DTCM**, even though it is read-only and could live in flash. This is the
single biggest waste in the current layout.

---

## 2. Measured state of every build we tried

All numbers are bytes, measured with `arm-none-eabi-objdump -h` on the ELF.

| # | Build variant | `.text.itcm` | ITCM banks | DTCM total | `.data` | `.bss` | data+bss | **Free stack** | Result |
|---|---|---|---|---|---|---|---|---|---|
| 1 | **Baseline v0.1.6** (no SD, HKDF pad) | 339,936 | 11 | 163,840 | 129,728 | 24,640 | 154,368 | **9,472** | ✅ boots, 24/24 tests |
| 2 | Arduino `<SD.h>` wrapper | 385,664 | **12** | 131,072 | 132,800 | 25,792 | 158,592 | **−27,520** | ❌ hard fault at boot |
| 3 | SdFat FAT-only, all in ITCM | 361,408 | **12** | 131,072 | 131,776 | 26,080 | 157,856 | **−26,784** | ❌ hard fault at boot |
| 4 | SdFat FAT-only, **all → FLASH** | 348,480 | 11 | 163,840 | 131,776 | 26,080 | 157,856 | **5,984** | ⚠️ boots, `sd.begin()`/scan fails |
| 5 | SdFat FAT-only, SDIO kept in ITCM | 355,056 | 11 | 163,840 | 131,776 | 26,080 | 157,856 | **5,984** | ❓ **never tested** |

### Sanity check of the model

Build 1 arithmetic reproduces the number arduino-cli itself reports:

```
ITCM code 339,936 → ceil(339936/32768) = 11 banks = 360,448  (padding 20,512)
DTCM      = (16 − 11) × 32768 = 163,840
minus .data+.bss                = 154,368
                        free stack = 9,472  ◄── matches the documented 9,632
```

### Two corrections to earlier conclusions

1. **Builds 2 and 3 did not "crash because of the SD library."** They crashed
   because ITCM crossed the 352 KB → 384 KB bank boundary, which stole a 32 KB
   bank from DTCM and made `.data`+`.bss` (158 KB) **larger than the entire
   DTCM region** (128 KB). The linker cannot detect this because the split is
   computed at runtime by the boot ROM.

2. **I previously miscalculated build 5** as needing 12 banks. It needs 11
   (355,056 ≤ 360,448). Build 5 fits, has the same 5,984 bytes of stack as
   build 4, and **was never flashed** — I reverted the linker change before
   testing it. That test is still owed.

### Why build 4 boots but SD operations fail

Build 4 leaves **5,984 bytes of stack** — a 37% reduction from the already
marginal 9,472-byte baseline. SdFat's `begin()` → card identify → FAT mount
chain, and `openNextFile()` directory walks, allocate multi-hundred-byte
frames several levels deep. The most probable explanation for
"`sd.begin()` fails" and "`otp_debug` disconnects the device" is **stack
overflow into `.bss`**, not flash execution speed.

The 32-byte MPU guard at the end of `.bss`
([`imxrt1062_t41_flashmem.ld:177`](../firmware/teensy41/signer/imxrt1062_t41_flashmem.ld:177))
catches a hard overrun as a fault — which is exactly the "device disconnects"
symptom we saw.

---

## 3. Where the space is going

```
FLEXRAM 512 KB ── 16 banks ── current build 4 layout
┌──────────────────────────────────────────────┬────────────────────────────┐
│ ITCM  11 banks = 352 KB                      │ DTCM  5 banks = 160 KB     │
├──────────────────────────────────────────────┼────────────────────────────┤
│ ██████████████████████████████████████░░░░   │ ████████████████████████▓░ │
│ ↑ code 348,480 (99%)          ↑ pad 11,968   │ ↑ .data 131,776  ↑bss  ↑↑  │
│                                              │   (80% of DTCM!) 26,080 5,984│
└──────────────────────────────────────────────┴────────────────────────────┘
                                                                    ↑ STACK
                                                          ONLY 5.8 KB LEFT

RAM2 / OCRAM 512 KB
┌───────────────────────────────────────────────────────────────────────────┐
│ ███████████████████████████████████████████████████████████████░░░░░░░░░░ │
│ ↑ .bss.dma  413,600  (LVGL buffers, crypto workspaces)   ↑ heap 110,688   │
└───────────────────────────────────────────────────────────────────────────┘

FLASH 7936 KB
┌───────────────────────────────────────────────────────────────────────────┐
│ ██████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │
│ ↑ ~1.6 MB used                              ↑ ~6.3 MB FREE (80% unused!) │
└───────────────────────────────────────────────────────────────────────────┘
```

**The asymmetry is the whole story:** DTCM has 5.8 KB free while FLASH has
6.3 MB free. And 128 KB of DTCM — 80% of the region — is occupied by `.data`,
most of which is `.rodata` that has no business being in RAM at all.

### What is likely inside that 128 KB of `.data`/`.rodata`

Not yet measured (see Step 1 below), but the candidates, largest first:

| Suspect | Estimate | Notes |
|---|---|---|
| BIP-39 wordlist ([`mnemonic_wordlist.h`](../firmware/teensy41/signer/src/mnemonic_wordlist.h)) | 16–24 KB | 2048 const strings |
| LVGL fonts (montserrat 14/20) + LVGL const tables | 20–40 KB | pure rodata |
| PQClean constants (ML-DSA/ML-KEM/SLH-DSA zetas, SHAKE tables) | 10–20 KB | some already routed to flash |
| cJSON, bech32, base64 tables, format strings | 5–10 KB | |
| ed25519 `ed_K`/`ed_X`/`ed_Y` | ~1 KB | **must stay in DTCM** (documented regression) |
| Genuine writable `.data` | 10–30 KB | LVGL state, USB endpoint queues |

---

## 4. Evaluation

### What is genuinely working

- [`otppad_embedded.{h,c}`](../firmware/teensy41/signer/src/otppad_embedded.h) —
  bit-compatible with [`libotppad`](../libotppad/libotppad.h), **2386/2386**
  host tests pass. Zero doubt about the format layer.
- [`otp_pad_sd.{h,cpp}`](../firmware/teensy41/signer/src/otp_pad_sd.h) —
  logic complete (bind, seek/read, XOR, Padmé, armor, binary `.otp`, atomic
  offset). Never had a chance to execute.
- [`pad_gen.ino`](../firmware/teensy41/pad_gen/pad_gen.ino) — a real 1 MB
  TRNG pad exists on the card with a verified checksum.
- SD hardware, wiring, and card are all proven good (the standalone probe
  sketches read the 1 TB card and did write/read/verify round-trips).

### The actual problem, stated precisely

> The signer firmware has **5,984 bytes of stack** in the best SD-enabled
> build. SdFat needs more than that to mount a volume and walk a directory.
> There is no way around this by moving *code*; we must reclaim **DTCM**.

Nothing is wrong with the SD library, the linker-script approach, or the OTP
implementation. We are simply out of stack.

### Why this was hard to see

- The linker reports no error: the ITCM/DTCM split happens at boot, not link
  time, so an over-committed DTCM links cleanly and faults at reset.
- `arduino-cli` prints "free for local variables" only on the *default*
  linker-script path; with `-T<custom>.ld` it errors out of the size step
  ("Error while determining sketch size"), so we lost our early-warning gauge.
- Symptoms (no USB enumeration, `sd.begin()` returning false, the device
  vanishing mid-request) all look like driver problems but are stack overflow.

---

## 5. Proposed solutions

Ordered by leverage. **A is the recommended path** and is likely sufficient on
its own.

### Solution A — Move `.rodata` out of DTCM into FLASH (recommended)

**Reclaims: an estimated 40–90 KB of DTCM. Effort: low. Risk: low-moderate.**

The linker script currently puts every `.rodata*` input section into DTCM
([line 168](../firmware/teensy41/signer/imxrt1062_t41_flashmem.ld:168)).
Read-only data does not need to be in tightly-coupled RAM; the Cortex-M7 has a
16 KB D-cache in front of FLEXSPI and const tables are cache-friendly.

Change `.data` to stop absorbing rodata, and add a catch-all rodata rule to the
FLASH output section, with a **targeted exception list** for known-sensitive
tables:

```ld
.data : {
    *(.endpoint_queue)
    *ed25519.c.o(.rodata*)   /* ed_K/ed_X/ed_Y must stay in DTCM */
    *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.data*)))
    KEEP(*(.vectorsram))
} > DTCM  AT> FLASH
```

The ed25519 exception is not speculative — the linker script already documents
that moving `ed25519.c.o(.rodata*)` to flash produced an all-zeros pubkey
([lines 35–39](../firmware/teensy41/signer/imxrt1062_t41_flashmem.ld:35)).
That regression is the template for what to watch for.

**Payoff:** if `.data` drops from 128 KB to, say, 60 KB, free stack goes from
5,984 to roughly **74,000 bytes** — an order-of-magnitude improvement that
removes the stack question entirely, for SD and for the PQ crypto paths.

**Risk & mitigation:** some library may depend on a const table being in RAM
(as ed25519 did). Mitigation is incremental: move rodata per-object-file in
small batches, run [`test_classical.py`](../firmware/teensy41/test_classical.py)
(16 tests) and [`test_signer.py`](../firmware/teensy41/test_signer.py) (24
tests) after each batch, and bisect any failure to the offending `.o`.

### Solution B — Restore the build-time memory gauge

**Reclaims: nothing. Effort: very low. Value: high.**

We are flying blind. Add a post-link check to
[`build_signer.sh`](../firmware/teensy41/build_signer.sh) that computes the
same arithmetic the boot ROM will use and **fails the build** if the stack
would be under a threshold:

```
itcm_banks   = ceil((text.itcm + ARM.exidx) / 32768)
dtcm_bytes   = (16 - itcm_banks) * 32768
free_stack   = dtcm_bytes - data - bss
FAIL if free_stack < 16384
```

This converts every future "mysterious boot crash" into a build error with a
number attached. Should be done regardless of which other solution we pick.

### Solution C — Test build 5 (SDIO in ITCM, FAT layer in FLASH)

**Reclaims: nothing. Effort: trivial. Value: eliminates a hypothesis.**

Build 5 fits in 11 banks and was never flashed. If the real problem is flash
execution speed for the SDIO driver rather than stack, build 5 is the fix and
costs nothing. If it fails the same way, that confirms the stack diagnosis.
Cheap experiment; do it before or alongside A.

### Solution D — Shrink the OTP feature's own footprint

**Reclaims: a few KB. Effort: low. Value: moderate.**

- Drop `OTP_SD_MAX_CHUNK` from 16 KB to 4 KB (Padmé bucket 4096 covers ~4 KB
  plaintext, ample for Nostr `content`). Cuts the two malloc'd scratch buffers.
- Remove `verify_pad_checksum()` from the bind path, or gate it behind an
  explicit `otp_verify` verb. Streaming 1 MB at boot is slow, deep-stacked, and
  will be flatly impossible on the 900 GB pad. Verify the first and last 4 KB
  instead, or trust the filename.
- Replace the `openNextFile()` scan with a direct
  `sd.exists("/pads/<chksum>.pad")` when a chksum is already known, skipping
  the directory walk entirely.

### Solution E — Move LVGL draw buffers to the heap, shrink DMAMEM

**Reclaims: DTCM indirectly. Effort: moderate. Value: situational.**

`.bss.dma` is 413,600 of 512 KB in RAM2. The two LVGL buffers are ~46 KB of
that. This does not directly help DTCM, but if we ever need RAM2 headroom for
SD block buffers it is the place to look.

### Solution F — Reduce feature scope

**Effort: none. Value: last resort.**

If A through D all fail to yield enough stack, the fallback is to make features
mutually exclusive at build time — e.g. an OTP-focused firmware build that
omits SLH-DSA-128s (the largest PQ algorithm) and reclaims its ITCM and rodata.
This is a product decision, not an engineering one, and should only be reached
after A is proven insufficient.

### Non-solution: external PSRAM

The linker script reserves 32 MB of ERAM at `0x70000000`, and `.bss.extram`
currently has size 0. **The Teensy 4.1 ships with the two PSRAM pads empty** —
this memory does not physically exist unless chips are soldered on. Not a
software option.

---

## 6. Recommended sequence

```mermaid
flowchart TD
    B[Solution B: build-time stack gauge<br/>fail build under 16 KB] --> C[Solution C: flash build 5<br/>SDIO in ITCM, FAT in FLASH]
    C -->|works| D[Solution D: trim OTP footprint<br/>4 KB chunks, drop boot checksum]
    C -->|still fails| A[Solution A: move rodata to FLASH<br/>incremental, test each batch]
    A --> D
    D --> T[Phase 5: run test_otp_sd.py]
    T --> U[Phase 6: ui_pick_pad LVGL screen]
```

1. **B** first — 20 minutes, and every subsequent step gets a number instead of
   a guess.
2. **C** next — trivial, and it either fixes the problem or kills a hypothesis.
3. **A** if C did not fix it — this is the real headroom, and it benefits the
   whole project (the PQ paths have been stack-starved since v0.1.3).
4. **D** as cleanup once there is room to breathe.
5. Then resume Phases 5 and 6 of the OTP plan.

---

## 7. Decisions needed

1. **Is Solution A acceptable?** It touches the linker script that took six
   versions to stabilise (v0.1.1–v0.1.6 were all memory fixes). The upside is
   large and it fixes a latent problem, but it needs a full re-run of both test
   suites and carries a real chance of an ed25519-style surprise.
2. **Can we drop the boot-time pad checksum verify?** Technically right (it
   cannot scale to a 900 GB pad) but it is a security-posture change: we would
   trust the filename rather than prove the pad's integrity at bind time.
3. **Is a 4 KB max OTP chunk acceptable?** It caps a single `encrypt` call at
   ~4 KB of plaintext; larger payloads would need caller-side chunking.
4. **What stack floor do we want?** Suggest 16 KB minimum, 32 KB target. The
   historical 9.6 KB was the direct cause of six versions of crash-fixing.