n_signer/plans/teensy41_memory_evaluation.md

Teensy 4.1 Signer — Memory Budget Evaluation

Date: 2026-07-30 Context: The SD-card OTP pad (plans/teensy41_otp_sd_pad.md) is blocked. Root cause turned out to be a DTCM stack shortage, not an "SD library incompatibility". This document re-derives the memory budget from scratch and proposes solutions.

---

1. How Teensy 4.1 memory actually works

The i.MX RT1062 has three separate RAM regions plus flash:

┌───────────────────────────────────────────────────────────────────────────┐
│ FLASH  8 MB  (7936 KB usable)                       @ 0x60000000          │
│   .text.code   — code + rodata routed here by the linker script           │
│   .text.itcm   — LOAD image of ITCM code (copied to ITCM at boot)         │
│   .data        — LOAD image of DTCM data (copied to DTCM at boot)          │
├───────────────────────────────────────────────────────────────────────────┤
│ FLEXRAM  512 KB = 16 banks × 32 KB   @ 0x00000000 (ITCM) / 0x20000000 (DTCM)│
│   Split between ITCM and DTCM AT BOOT by _flexram_bank_config.             │
│   ITCM = code that runs at full speed (zero wait state)                   │
│   DTCM = .data + .bss + THE STACK                                         │
├───────────────────────────────────────────────────────────────────────────┤
│ RAM2 / OCRAM  512 KB                                @ 0x20200000          │
│   .bss.dma (DMAMEM statics) + the malloc heap                             │
├───────────────────────────────────────────────────────────────────────────┤
│ ERAM / PSRAM  0 MB (unpopulated)                    @ 0x70000000          │
│   Linker script reserves 32 MB but the chips are NOT soldered on.          │
└───────────────────────────────────────────────────────────────────────────┘

The FlexRAM split is computed by the linker script (imxrt1062_t41_flashmem.ld:215):

_itcm_block_count   = (SIZEOF(.text.itcm) + SIZEOF(.ARM.exidx) + 0x7FFF) >> 15;
_estack             = ORIGIN(DTCM) + ((16 - _itcm_block_count) << 15);

This is the crux: every 32 KB bank given to ITCM is taken away from DTCM. Code size therefore directly steals stack space. And crucially:

.data : {
    *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.rodata*)))   ◄── READ-ONLY DATA IN DTCM!
    *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.data*)))
} > DTCM  AT> FLASH

.rodata (const tables, string literals, fonts, wordlists) is being placed in DTCM, even though it is read-only and could live in flash. This is the single biggest waste in the current layout.

---

2. Measured state of every build we tried

All numbers are bytes, measured with arm-none-eabi-objdump -h on the ELF.

#	Build variant	.text.itcm	ITCM banks	DTCM total	.data	.bss	data+bss	Free stack	Result
1	Baseline v0.1.6 (no SD, HKDF pad)	339,936	11	163,840	129,728	24,640	154,368	9,472	✅ boots, 24/24 tests
2	Arduino <SD.h> wrapper	385,664	12	131,072	132,800	25,792	158,592	−27,520	❌ hard fault at boot
3	SdFat FAT-only, all in ITCM	361,408	12	131,072	131,776	26,080	157,856	−26,784	❌ hard fault at boot
4	SdFat FAT-only, all → FLASH	348,480	11	163,840	131,776	26,080	157,856	5,984	⚠️ boots, sd.begin()/scan fails
5	SdFat FAT-only, SDIO kept in ITCM	355,056	11	163,840	131,776	26,080	157,856	5,984	❓ never tested

Sanity check of the model

Build 1 arithmetic reproduces the number arduino-cli itself reports:

ITCM code 339,936 → ceil(339936/32768) = 11 banks = 360,448  (padding 20,512)
DTCM      = (16 − 11) × 32768 = 163,840
minus .data+.bss                = 154,368
                        free stack = 9,472  ◄── matches the documented 9,632

Two corrections to earlier conclusions

1. Builds 2 and 3 did not "crash because of the SD library." They crashed because ITCM crossed the 352 KB → 384 KB bank boundary, which stole a 32 KB bank from DTCM and made .data+.bss (158 KB) larger than the entire DTCM region (128 KB). The linker cannot detect this because the split is computed at runtime by the boot ROM.

2. I previously miscalculated build 5 as needing 12 banks. It needs 11 (355,056 ≤ 360,448). Build 5 fits, has the same 5,984 bytes of stack as build 4, and was never flashed — I reverted the linker change before testing it. That test is still owed.

Why build 4 boots but SD operations fail

Build 4 leaves 5,984 bytes of stack — a 37% reduction from the already marginal 9,472-byte baseline. SdFat's begin() → card identify → FAT mount chain, and openNextFile() directory walks, allocate multi-hundred-byte frames several levels deep. The most probable explanation for "sd.begin() fails" and "otp_debug disconnects the device" is stack overflow into .bss, not flash execution speed.

The 32-byte MPU guard at the end of .bss (imxrt1062_t41_flashmem.ld:177) catches a hard overrun as a fault — which is exactly the "device disconnects" symptom we saw.

---

3. Where the space is going

FLEXRAM 512 KB ── 16 banks ── current build 4 layout
┌──────────────────────────────────────────────┬────────────────────────────┐
│ ITCM  11 banks = 352 KB                      │ DTCM  5 banks = 160 KB     │
├──────────────────────────────────────────────┼────────────────────────────┤
│ ██████████████████████████████████████░░░░   │ ████████████████████████▓░ │
│ ↑ code 348,480 (99%)          ↑ pad 11,968   │ ↑ .data 131,776  ↑bss  ↑↑  │
│                                              │   (80% of DTCM!) 26,080 5,984│
└──────────────────────────────────────────────┴────────────────────────────┘
                                                                    ↑ STACK
                                                          ONLY 5.8 KB LEFT

RAM2 / OCRAM 512 KB
┌───────────────────────────────────────────────────────────────────────────┐
│ ███████████████████████████████████████████████████████████████░░░░░░░░░░ │
│ ↑ .bss.dma  413,600  (LVGL buffers, crypto workspaces)   ↑ heap 110,688   │
└───────────────────────────────────────────────────────────────────────────┘

FLASH 7936 KB
┌───────────────────────────────────────────────────────────────────────────┐
│ ██████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │
│ ↑ ~1.6 MB used                              ↑ ~6.3 MB FREE (80% unused!) │
└───────────────────────────────────────────────────────────────────────────┘

The asymmetry is the whole story: DTCM has 5.8 KB free while FLASH has 6.3 MB free. And 128 KB of DTCM — 80% of the region — is occupied by .data, most of which is .rodata that has no business being in RAM at all.

What is likely inside that 128 KB of .data/.rodata

Not yet measured (see Step 1 below), but the candidates, largest first:

Suspect	Estimate	Notes
BIP-39 wordlist (mnemonic_wordlist.h)	16–24 KB	2048 const strings
LVGL fonts (montserrat 14/20) + LVGL const tables	20–40 KB	pure rodata
PQClean constants (ML-DSA/ML-KEM/SLH-DSA zetas, SHAKE tables)	10–20 KB	some already routed to flash
cJSON, bech32, base64 tables, format strings	5–10 KB
ed25519 ed_K/ed_X/ed_Y	~1 KB	must stay in DTCM (documented regression)
Genuine writable .data	10–30 KB	LVGL state, USB endpoint queues

---

4. Evaluation

What is genuinely working

• otppad_embedded.{h,c} — bit-compatible with libotppad, 2386/2386 host tests pass. Zero doubt about the format layer.
• otp_pad_sd.{h,cpp} — logic complete (bind, seek/read, XOR, Padmé, armor, binary .otp, atomic offset). Never had a chance to execute.
• pad_gen.ino — a real 1 MB TRNG pad exists on the card with a verified checksum.
• SD hardware, wiring, and card are all proven good (the standalone probe sketches read the 1 TB card and did write/read/verify round-trips).

The actual problem, stated precisely

The signer firmware has 5,984 bytes of stack in the best SD-enabled build. SdFat needs more than that to mount a volume and walk a directory. There is no way around this by moving code; we must reclaim DTCM.

Nothing is wrong with the SD library, the linker-script approach, or the OTP implementation. We are simply out of stack.

Why this was hard to see

• The linker reports no error: the ITCM/DTCM split happens at boot, not link time, so an over-committed DTCM links cleanly and faults at reset.
• arduino-cli prints "free for local variables" only on the default linker-script path; with -T<custom>.ld it errors out of the size step ("Error while determining sketch size"), so we lost our early-warning gauge.
• Symptoms (no USB enumeration, sd.begin() returning false, the device vanishing mid-request) all look like driver problems but are stack overflow.

---

5. Proposed solutions

Ordered by leverage. A is the recommended path and is likely sufficient on its own.

Solution A — Move .rodata out of DTCM into FLASH (recommended)

Reclaims: an estimated 40–90 KB of DTCM. Effort: low. Risk: low-moderate.

The linker script currently puts every .rodata* input section into DTCM (line 168). Read-only data does not need to be in tightly-coupled RAM; the Cortex-M7 has a 16 KB D-cache in front of FLEXSPI and const tables are cache-friendly.

Change .data to stop absorbing rodata, and add a catch-all rodata rule to the FLASH output section, with a targeted exception list for known-sensitive tables:

.data : {
    *(.endpoint_queue)
    *ed25519.c.o(.rodata*)   /* ed_K/ed_X/ed_Y must stay in DTCM */
    *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.data*)))
    KEEP(*(.vectorsram))
} > DTCM  AT> FLASH

The ed25519 exception is not speculative — the linker script already documents that moving ed25519.c.o(.rodata*) to flash produced an all-zeros pubkey (lines 35–39). That regression is the template for what to watch for.

Payoff: if .data drops from 128 KB to, say, 60 KB, free stack goes from 5,984 to roughly 74,000 bytes — an order-of-magnitude improvement that removes the stack question entirely, for SD and for the PQ crypto paths.

Risk & mitigation: some library may depend on a const table being in RAM (as ed25519 did). Mitigation is incremental: move rodata per-object-file in small batches, run test_classical.py (16 tests) and test_signer.py (24 tests) after each batch, and bisect any failure to the offending .o.

Solution B — Restore the build-time memory gauge

Reclaims: nothing. Effort: very low. Value: high.

We are flying blind. Add a post-link check to build_signer.sh that computes the same arithmetic the boot ROM will use and fails the build if the stack would be under a threshold:

itcm_banks   = ceil((text.itcm + ARM.exidx) / 32768)
dtcm_bytes   = (16 - itcm_banks) * 32768
free_stack   = dtcm_bytes - data - bss
FAIL if free_stack < 16384

This converts every future "mysterious boot crash" into a build error with a number attached. Should be done regardless of which other solution we pick.

Solution C — Test build 5 (SDIO in ITCM, FAT layer in FLASH)

Reclaims: nothing. Effort: trivial. Value: eliminates a hypothesis.

Build 5 fits in 11 banks and was never flashed. If the real problem is flash execution speed for the SDIO driver rather than stack, build 5 is the fix and costs nothing. If it fails the same way, that confirms the stack diagnosis. Cheap experiment; do it before or alongside A.

Solution D — Shrink the OTP feature's own footprint

Reclaims: a few KB. Effort: low. Value: moderate.

• Drop OTP_SD_MAX_CHUNK from 16 KB to 4 KB (Padmé bucket 4096 covers ~4 KB plaintext, ample for Nostr content). Cuts the two malloc'd scratch buffers.
• Remove verify_pad_checksum() from the bind path, or gate it behind an explicit otp_verify verb. Streaming 1 MB at boot is slow, deep-stacked, and will be flatly impossible on the 900 GB pad. Verify the first and last 4 KB instead, or trust the filename.
• Replace the openNextFile() scan with a direct sd.exists("/pads/<chksum>.pad") when a chksum is already known, skipping the directory walk entirely.

Solution E — Move LVGL draw buffers to the heap, shrink DMAMEM

Reclaims: DTCM indirectly. Effort: moderate. Value: situational.

.bss.dma is 413,600 of 512 KB in RAM2. The two LVGL buffers are ~46 KB of that. This does not directly help DTCM, but if we ever need RAM2 headroom for SD block buffers it is the place to look.

Solution F — Reduce feature scope

Effort: none. Value: last resort.

If A through D all fail to yield enough stack, the fallback is to make features mutually exclusive at build time — e.g. an OTP-focused firmware build that omits SLH-DSA-128s (the largest PQ algorithm) and reclaims its ITCM and rodata. This is a product decision, not an engineering one, and should only be reached after A is proven insufficient.

Non-solution: external PSRAM

The linker script reserves 32 MB of ERAM at 0x70000000, and .bss.extram currently has size 0. The Teensy 4.1 ships with the two PSRAM pads empty — this memory does not physically exist unless chips are soldered on. Not a software option.

---

6. Recommended sequence

flowchart TD
    B[Solution B: build-time stack gauge<br/>fail build under 16 KB] --> C[Solution C: flash build 5<br/>SDIO in ITCM, FAT in FLASH]
    C -->|works| D[Solution D: trim OTP footprint<br/>4 KB chunks, drop boot checksum]
    C -->|still fails| A[Solution A: move rodata to FLASH<br/>incremental, test each batch]
    A --> D
    D --> T[Phase 5: run test_otp_sd.py]
    T --> U[Phase 6: ui_pick_pad LVGL screen]

1. B first — 20 minutes, and every subsequent step gets a number instead of a guess.
2. C next — trivial, and it either fixes the problem or kills a hypothesis.
3. A if C did not fix it — this is the real headroom, and it benefits the whole project (the PQ paths have been stack-starved since v0.1.3).
4. D as cleanup once there is room to breathe.
5. Then resume Phases 5 and 6 of the OTP plan.

---

7. Decisions needed

1. Is Solution A acceptable? It touches the linker script that took six versions to stabilise (v0.1.1–v0.1.6 were all memory fixes). The upside is large and it fixes a latent problem, but it needs a full re-run of both test suites and carries a real chance of an ed25519-style surprise.
2. Can we drop the boot-time pad checksum verify? Technically right (it cannot scale to a 900 GB pad) but it is a security-posture change: we would trust the filename rather than prove the pad's integrity at bind time.
3. Is a 4 KB max OTP chunk acceptable? It caps a single encrypt call at ~4 KB of plaintext; larger payloads would need caller-side chunking.
4. What stack floor do we want? Suggest 16 KB minimum, 32 KB target. The historical 9.6 KB was the direct cause of six versions of crash-fixing.