v0.1.7 - Teensy 4.1 OTP SD pad: complete implementation with UI pad selection

Solution D: 4 KB chunk cap, dropped boot-time checksum verify (can't scale
to 900 GB), increased transport frame + req/resp buffers to 8 KB.
test_otp_sd.py: 10/10 pass (ASCII + binary round-trips, offset advance,
2 KB large plaintext, tamper detection).

Phase 6: ui_pick_pad() LVGL pad-selection screen with chksum prefix + size
display, Skip button, 30s timeout. otp_pad_sd_list_pads() scans /pads and
returns chksums + sizes. Wired into the non-debug boot flow in signer.ino
(DEBUG_AUTO_GENERATE=0 path). DEBUG_AUTO_GENERATE=1 stays headless with
auto-bind_first.

README: added comprehensive memory management section documenting the
FlexRAM architecture, custom linker script, .rodata-to-FLASH move,
build-time stack gauge, and current memory layout.
This commit is contained in:
Laan Tungir
2026-07-30 18:20:30 -04:00
parent ac2e6347a2
commit 0ea3e60391
9 changed files with 346 additions and 29 deletions
+122 -4
View File
@@ -112,12 +112,130 @@ These constants are for the specific panel we calibrated. If you swap a
display module, re-run [`firmware/teensy41/touch_cal/touch_cal.ino`](touch_cal/touch_cal.ino)
and paste the new constants.
## Build (planned)
## Build
```bash
# Arduino CLI / Teensyduino
arduino-cli compile --fqbn teensy:avr:teensy41 firmware/teensy41
arduino-cli upload -p /dev/ttyACM0 --fqbn teensy:avr:teensy41 firmware/teensy41
# Arduino CLI / Teensyduino — uses the custom linker script
bash firmware/teensy41/build_signer.sh # compile only
bash firmware/teensy41/build_signer.sh --flash # compile + upload
bash firmware/teensy41/build_signer.sh --test # compile + upload + run tests
```
The build uses a custom linker script
([`signer/imxrt1062_t41_flashmem.ld`](signer/imxrt1062_t41_flashmem.ld)) that
routes crypto and SdFat code to FLASH (off-chip QSPI) instead of ITCM
(tightly-coupled RAM), freeing the limited FlexRAM for stack. See the memory
management section below for details.
See the port plan: [`plans/teensy41_signer_port.md`](../../plans/teensy41_signer_port.md).
## Memory management
The Teensy 4.1's NXP i.MX RT1062 has a unique FlexRAM architecture that makes
memory budgeting the central engineering challenge of this firmware. This
section documents the layout, the custom linker script, and the build-time
stack gauge that catches overflows before they become mystery crashes.
### The FlexRAM problem
The i.MX RT1062 has three RAM regions:
```
FLASH 8 MB (off-chip QSPI, @ 0x60000000) — code + rodata, slow but huge
FLEXRAM 512 KB (on-chip, split ITCM/DTCM) — fast, tiny, THE BOTTLENECK
RAM2 512 KB (OCRAM, @ 0x20200000) — DMAMEM statics + malloc heap
ERAM 0 MB (PSRAM pads empty) — not populated on the Teensy 4.1
```
The 512 KB of FlexRAM is divided into **16 banks of 32 KB** that are split
between **ITCM** (instruction tightly-coupled memory, runs code at zero wait
state) and **DTCM** (data tightly-coupled memory, holds `.data` + `.bss` +
**the stack**). The split is computed at **boot time** by the Teensy boot ROM
from a formula in the linker script:
```ld
_itcm_block_count = (SIZEOF(.text.itcm) + SIZEOF(.ARM.exidx) + 0x7FFF) >> 15;
_estack = ORIGIN(DTCM) + ((16 - _itcm_block_count) << 15);
```
**Every 32 KB bank given to code is taken away from the stack.** If ITCM code
grows past a 32 KB boundary, a whole bank is stolen from DTCM, and the stack
shrinks by 32 KB. The linker cannot detect this because the split happens at
reset, not link time — an over-committed DTCM links cleanly and then hard-faults
on boot.
### The custom linker script
[`signer/imxrt1062_t41_flashmem.ld`](signer/imxrt1062_t41_flashmem.ld) routes
specific code and data sections to FLASH to keep ITCM small and DTCM large:
1. **Crypto code → FLASH**: secp256k1, ed25519, x25519, PQClean (ML-DSA-65,
ML-KEM-768, SLH-DSA-128s), nostr_utils — all routed via per-object-file
rules (`*secp256k1.c.o(.text*)`, etc.). These run slightly slower from
FLASH but the stack headroom is the critical constraint.
2. **SdFat library → FLASH**: the FAT filesystem layer (FatFile, FatPartition,
FatVolume, etc.) is routed to FLASH. The SDIO driver (SdioCard, SdioTeensy)
stays in ITCM for fast interrupt response.
3. **`.rodata` → FLASH** (v0.1.6): read-only data (const tables, string
literals, BIP-39 wordlist, LVGL fonts, PQClean constants) is routed to
FLASH via `*(EXCLUDE_FILE(*ed25519.c.o) .rodata*)`. This reclaimed **124 KB
of DTCM** (`.data` went from 131 KB to 6.8 KB), increasing free stack from
5,984 bytes to **130,912 bytes**.
The `EXCLUDE_FILE(*ed25519.c.o)` exception is critical: the ed25519 base
point constants (`ed_K`, `ed_X`, `ed_Y`) must stay in DTCM — moving them to
FLASH produces an all-zeros pubkey (a regression documented in the linker
script comments).
### Build-time stack gauge
[`check_stack.sh`](check_stack.sh) parses the ELF's section sizes after
compilation and computes the same ITCM/DTCM split the boot ROM will perform.
It **fails the build** if free stack would be below 16 KB:
```
=== Teensy 4.1 FlexRAM stack gauge ===
.text.itcm : 355056 bytes -> 11 banks (360448 bytes)
.data (DTCM) : 6848 bytes
.bss (DTCM) : 26080 bytes
DTCM total : 163840 bytes (5 banks)
FREE STACK : 130912 bytes (threshold: 16384)
✅ OK
```
This converts every future "mysterious boot crash" into a build error with a
number. The gauge is wired into [`build_signer.sh`](build_signer.sh) and runs
automatically after every compile.
### Current memory layout (v0.1.6)
```
FLEXRAM 512 KB — 16 banks
┌──────────────────────────────────────────────┬────────────────────────────────────────┐
│ ITCM 11 banks = 352 KB │ DTCM 5 banks = 160 KB │
│ code: 355 KB (signer + SDIO driver) │ .data: 6.8 KB (writable globals) │
│ │ .bss: 25.4 KB (zero-init globals) │
│ │ FREE STACK: 130.9 KB ✅ │
└──────────────────────────────────────────────┴────────────────────────────────────────┘
RAM2 / OCRAM 512 KB
┌──────────────────────────────────────────────────────────────────────────────────────────┐
│ .bss.dma: 413.6 KB (LVGL draw buffers, crypto DMAMEM workspaces) heap: 110.7 KB free │
└──────────────────────────────────────────────────────────────────────────────────────────┘
FLASH 7936 KB
┌──────────────────────────────────────────────────────────────────────────────────────────┐
│ Used: ~1.6 MB (crypto code + SdFat + .rodata + ITCM/DTCM load images) Free: ~6.3 MB │
└──────────────────────────────────────────────────────────────────────────────────────────┘
```
### History
The memory budget was the direct cause of six versions of crash-fixing
(v0.1.1v0.1.6). The v0.1.6 `.rodata` → FLASH move was the largest single
improvement, and it also enabled the SD-card OTP pad (which requires SdFat,
adding ~7 KB of ITCM code that would have overflowed DTCM without the rodata
reclamation). See [`plans/teensy41_memory_evaluation.md`](../../plans/teensy41_memory_evaluation.md)
for the full analysis.
+32 -4
View File
@@ -496,8 +496,33 @@ void setup() {
Serial.println("otp_pad_sd_bind_first failed — no pad bound");
}
#else
// Interactive pad selection is wired in Phase 6 (ui_pick_pad).
Serial.println("Interactive pad selection not yet wired; pad unbound.");
// Interactive pad selection: scan /pads, show the list on the display,
// let the user pick one (or skip). See ui_pick_pad() in ui.cpp.
Serial.println("Scanning /pads for OTP pads...");
static char pad_chksums[4][65];
static uint64_t pad_sizes[4];
int n_pads = otp_pad_sd_list_pads(pad_chksums, pad_sizes, 4);
if (n_pads <= 0) {
Serial.println("No pads found in /pads — OTP encrypt/decrypt unavailable");
} else {
Serial.print("Found ");
Serial.print(n_pads);
Serial.println(" pad(s), showing selection screen...");
const char *chksum_ptrs[4];
for (int i = 0; i < n_pads; i++) chksum_ptrs[i] = pad_chksums[i];
char selected[65];
int rc = ui_pick_pad(chksum_ptrs, pad_sizes, n_pads,
selected, sizeof(selected));
if (rc == 0) {
Serial.print("User selected pad: ");
Serial.println(selected);
if (otp_pad_sd_bind(selected) != 0) {
Serial.println("otp_pad_sd_bind failed for selected pad");
}
} else {
Serial.println("User skipped OTP pad selection (or timeout)");
}
}
#endif
}
Serial.println("OTP pad init complete.");
@@ -508,8 +533,11 @@ void setup() {
}
// ---- Signing loop buffers (in DMAMEM/RAM2 to save RAM1) ----
DMAMEM static uint8_t req_buf[2048];
DMAMEM static char resp_buf[4096];
// Increased for OTP encrypt/decrypt: a 4 KB chunk produces ~5.5 KB of ASCII
// armor, and the base64-encoded request can be ~5.5 KB. With 110 KB of free
// RAM2 heap, 8 KB + 8 KB is comfortable.
DMAMEM static uint8_t req_buf[8192];
DMAMEM static char resp_buf[8192];
void loop() {
// 1. Keep LVGL responsive (idle screen + any approval prompts).
+34 -4
View File
@@ -269,10 +269,11 @@ int otp_pad_sd_bind(const char *chksum_or_prefix) {
}
g_sd.pad_file = f;
if (verify_pad_checksum() != 0) {
g_sd.pad_file.close();
return 8;
}
/* Skip the boot-time checksum verify — it reads the entire pad (1 MB on
* the test card, up to 900 GB on a production card) and is too slow at
* boot. The pad's integrity is already established by the filename: the
* checksum IS the filename, and pad_gen.ino verified it at generation
* time. A future otp_verify verb can do an on-demand check. */
uint64_t offset;
if (otppad_e_state_read_sd(OTP_SD_PADS_DIR, g_sd.chksum, &offset) != 0) {
@@ -366,6 +367,35 @@ int otp_pad_sd_debug_list(char *out, size_t cap) {
return count;
}
/* Scan /pads for *.pad files and fill chksums + sizes arrays. */
int otp_pad_sd_list_pads(char chksums[][65], uint64_t sizes[], int max_count) {
if (!chksums || !sizes || max_count <= 0) return 0;
File32 dir = sd.open(OTP_SD_PADS_DIR);
if (!dir) return -1;
int count = 0;
while (count < max_count) {
File32 entry = dir.openNextFile();
if (!entry) break;
if (!entry.isDir()) {
char name[128];
entry.getName(name, sizeof(name));
size_t nlen = strlen(name);
if (nlen >= 5 && strcmp(name + nlen - 4, ".pad") == 0) {
size_t base = nlen - 4;
if (base == OTPPAD_E_CHKSUM_HEX_LEN) {
memcpy(chksums[count], name, base);
chksums[count][base] = '\0';
sizes[count] = (uint64_t)entry.size();
count++;
}
}
}
entry.close();
}
dir.close();
return count;
}
/* ------------------------------------------------------------------ */
/* Encrypt / decrypt */
/* ------------------------------------------------------------------ */
+11 -5
View File
@@ -31,11 +31,11 @@
extern "C" {
#endif
/* Max chunk size we will Padmé-pad and XOR in RAM. 16 KB keeps the two DMAMEM
* scratch buffers (32 KB total) within the Teensy 4.1's tight RAM2 budget
* alongside the existing crypto workspaces. Padmé buckets up to 16 KB cover
* plaintexts up to ~15 KB, plenty for Nostr event content. */
#define OTP_SD_MAX_CHUNK 16384
/* Max chunk size we will Padmé-pad and XOR in RAM. 4 KB is ample for Nostr
* event content and keeps the malloc'd scratch buffers small. Padmé buckets
* up to 4 KB cover plaintexts up to ~3.9 KB; larger payloads need
* caller-side chunking. */
#define OTP_SD_MAX_CHUNK 4096
/* Mount the SD card via SD.begin(BUILTIN_SDCARD). Returns 0 on success,
* non-zero if no card / bad card. Must be called once at boot before bind. */
@@ -70,6 +70,12 @@ uint64_t otp_pad_sd_size(void);
* bind failures without needing serial boot output. */
int otp_pad_sd_debug_list(char *out, size_t cap);
/* Scan /pads for *.pad files and fill the caller's arrays with chksums + sizes.
* `chksums` is an array of `max_count` char* (each will point into the
* caller-provided `chksum_storage` buffer). `sizes` is an array of uint64_t.
* Returns the number of pads found (0..max_count), or -1 on error. */
int otp_pad_sd_list_pads(char chksums[][65], uint64_t sizes[], int max_count);
/* Encrypt: takes plaintext bytes, returns a malloc'd ASCII armor or binary
* blob in *out_payload (caller frees). `encoding` is "ascii" or "binary".
* On success returns 0 and sets *out_payload_len, *out_off_before,
+5 -3
View File
@@ -18,9 +18,11 @@
#include <Arduino.h>
#include <string.h>
// Maximum payload we will accept. Matches the CYD's UART_MAX_FRAME and the
// host n_signer's typical request cap. Anything larger is rejected as -1.
#define TRANSPORT_MAX_FRAME 4096
// Maximum payload we will accept. Increased from 4096 to 8192 for OTP
// encrypt/decrypt: a 4 KB Padmé chunk produces ~5.5 KB of ASCII armor, and
// the JSON-RPC response wrapper adds ~100 bytes. With 100 KB of free RAM2
// heap, 8 KB is comfortable.
#define TRANSPORT_MAX_FRAME 8192
// Accumulation buffer: 4-byte prefix + payload. One byte larger than
// TRANSPORT_MAX_FRAME so we can detect oversize frames unambiguously.
+118
View File
@@ -899,3 +899,121 @@ __attribute__((section(".flashmem"))) ui_approval_decision_t ui_approve(const ch
return s_approve_decision;
}
/* =====================================================================
* 5. ui_pick_pad — OTP pad selection list
* ===================================================================== */
static volatile int s_pad_choice = -1; /* -1 = none, 0..n = pad index, -2 = skip */
static void on_pad_select(lv_event_t *e) {
if (lv_event_get_code(e) == LV_EVENT_CLICKED) {
int idx = (int)(intptr_t)lv_event_get_user_data(e);
s_pad_choice = idx;
}
}
static void on_pad_skip(lv_event_t *e) {
if (lv_event_get_code(e) == LV_EVENT_CLICKED) {
s_pad_choice = -2;
}
}
__attribute__((section(".flashmem"))) int ui_pick_pad(
const char *pad_chksums[], const uint64_t pad_sizes[],
int count, char *out_chksum, size_t out_chksum_cap)
{
if (count <= 0) return 1; /* no pads to pick */
s_pad_choice = -1;
lv_obj_t *scr = lv_screen_active();
lv_obj_clean(scr);
lv_obj_set_style_bg_opa(scr, LV_OPA_COVER, 0);
lv_obj_set_style_bg_color(scr, lv_color_hex(UI_BG), 0);
/* Title */
lv_obj_t *title = lv_label_create(scr);
lv_label_set_text(title, "Select OTP Pad");
lv_obj_set_style_text_color(title, lv_color_hex(UI_FG), 0);
lv_obj_set_style_text_font(title, &lv_font_montserrat_20, 0);
lv_obj_align(title, LV_ALIGN_TOP_MID, 0, 10);
/* Pad buttons — up to 4 pads shown (scroll if more) */
int max_show = count < 4 ? count : 4;
for (int i = 0; i < max_show; i++) {
lv_obj_t *btn = lv_button_create(scr);
lv_obj_set_style_radius(btn, 6, 0);
lv_obj_set_style_bg_opa(btn, LV_OPA_COVER, 0);
lv_obj_set_style_bg_color(btn, lv_color_hex(UI_BG), 0);
lv_obj_set_style_border_width(btn, 2, 0);
lv_obj_set_style_border_color(btn, lv_color_hex(UI_FG), 0);
lv_obj_set_style_text_color(btn, lv_color_hex(UI_FG), 0);
lv_obj_set_style_border_color(btn, lv_color_hex(UI_ACCENT), LV_STATE_PRESSED);
lv_obj_set_size(btn, 440, 50);
lv_obj_align(btn, LV_ALIGN_TOP_MID, 0, 50 + i * 55);
lv_obj_add_event_cb(btn, on_pad_select, LV_EVENT_ALL,
(void *)(intptr_t)i);
/* Label: chksum prefix (16 chars) + size */
char label[80];
char size_str[24];
uint64_t sz = pad_sizes[i];
if (sz >= 1000000000ULL) {
snprintf(size_str, sizeof(size_str), "%llu GB",
(unsigned long long)(sz / 1000000000ULL));
} else if (sz >= 1000000ULL) {
snprintf(size_str, sizeof(size_str), "%llu MB",
(unsigned long long)(sz / 1000000ULL));
} else if (sz >= 1000ULL) {
snprintf(size_str, sizeof(size_str), "%llu KB",
(unsigned long long)(sz / 1000ULL));
} else {
snprintf(size_str, sizeof(size_str), "%llu B",
(unsigned long long)sz);
}
/* Show first 16 chars of chksum (the prefix) */
char prefix[17];
strncpy(prefix, pad_chksums[i], 16);
prefix[16] = '\0';
snprintf(label, sizeof(label), "%s... %s", prefix, size_str);
lv_obj_t *lbl = lv_label_create(btn);
lv_label_set_text(lbl, label);
lv_obj_center(lbl);
}
/* Skip button */
lv_obj_t *btn_skip = lv_button_create(scr);
lv_obj_set_style_radius(btn_skip, 6, 0);
lv_obj_set_style_bg_opa(btn_skip, LV_OPA_COVER, 0);
lv_obj_set_style_bg_color(btn_skip, lv_color_hex(UI_BG), 0);
lv_obj_set_style_border_width(btn_skip, 2, 0);
lv_obj_set_style_border_color(btn_skip, lv_color_hex(UI_MUTED), 0);
lv_obj_set_style_text_color(btn_skip, lv_color_hex(UI_MUTED), 0);
lv_obj_set_size(btn_skip, 440, 40);
lv_obj_align(btn_skip, LV_ALIGN_BOTTOM_MID, 0, -10);
lv_obj_add_event_cb(btn_skip, on_pad_skip, LV_EVENT_ALL, NULL);
lv_obj_t *lbl_skip = lv_label_create(btn_skip);
lv_label_set_text(lbl_skip, "Skip OTP (no pad)");
lv_obj_center(lbl_skip);
/* 30-second timeout */
uint32_t deadline = millis() + 30000;
while (s_pad_choice == -1 && millis() < deadline) {
lv_tick_inc(5);
lv_timer_handler();
delay(5);
}
if (s_pad_choice >= 0 && s_pad_choice < count) {
if (out_chksum && out_chksum_cap > strlen(pad_chksums[s_pad_choice])) {
strcpy(out_chksum, pad_chksums[s_pad_choice]);
return 0;
}
return 0;
} else if (s_pad_choice == -2) {
return 1; /* skip */
} else {
return 2; /* timeout */
}
}
+15
View File
@@ -23,6 +23,7 @@
#define FIRMWARE_TEENSY41_SIGNER_UI_H
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
@@ -65,6 +66,20 @@ void ui_show_idle(const char *npub, const char *version);
* Returns UI_APPROVAL_DENY, UI_APPROVAL_APPROVE, or UI_APPROVAL_TIMEOUT. */
ui_approval_decision_t ui_approve(const char *verb, const char *summary);
/* Pad selection screen for the OTP SD-card pad. Shows a list of pads found
* on the SD card, each with its chksum prefix and size, plus a "Skip OTP"
* button. Blocks (pumping LVGL) until the user picks a pad or skips.
*
* `pad_chksums` is an array of `count` NUL-terminated chksum strings (64 hex
* chars each). `pad_sizes` is an array of `count` sizes in bytes.
*
* On success: copies the selected chksum to `out_chksum` (must be >= 65
* bytes) and returns 0.
* On skip: returns 1 (out_chksum untouched).
* On timeout (30s): returns 2. */
int ui_pick_pad(const char *pad_chksums[], const uint64_t pad_sizes[],
int count, char *out_chksum, size_t out_chksum_cap);
#ifdef __cplusplus
}
#endif
+7 -7
View File
@@ -36,7 +36,7 @@ def send_request(ser, req: dict) -> dict:
ser.flush()
resp_header = b""
deadline = time.time() + 30.0
deadline = time.time() + 60.0
while len(resp_header) < 4 and time.time() < deadline:
chunk = ser.read(4 - len(resp_header))
if chunk:
@@ -280,9 +280,9 @@ def main():
print(f" ❌ Binary round-trip FAILED: expected {plaintext}, got {recovered2}")
failed += 1
# ---- Test 4: Large plaintext (10 KB) ----
print("\n=== Test 4: Large plaintext (10 KB) ===")
large_pt = bytes(range(256)) * 40 # 10240 bytes
# ---- Test 4: Large plaintext (2 KB, within 4 KB chunk cap) ----
print("\n=== Test 4: Large plaintext (2 KB) ===")
large_pt = bytes(range(256)) * 8 # 2048 bytes
large_b64 = base64.b64encode(large_pt).decode()
r6 = test_verb(ser, "encrypt", [large_b64, {"encoding": "ascii"}])
@@ -292,9 +292,9 @@ def main():
large_off_after = int(result6["pad_offset_after"])
large_consumed = large_off_after - large_off_before
print(f" 10 KB plaintext: offset {large_off_before} -> {large_off_after} (consumed {large_consumed} bytes)")
# Padme: 10 KB -> chunk doubles to 16384 bytes
if large_consumed == 16384:
print(f" ✅ Padme bucket = 16384 (correct for 10 KB)")
# Padme: 2 KB -> chunk = 4096 (minimum bucket, since 256*2^4=4096 >= 2048+1)
if large_consumed == 4096:
print(f" ✅ Padme bucket = 4096 (correct for 2 KB)")
passed += 1
else:
print(f" ⚠️ Padme bucket = {large_consumed} (expected 16384)")
+2 -2
View File
@@ -762,8 +762,8 @@ int socket_name_random(char *out, size_t out_len);
/* Version information (auto-updated by build/version tooling) */
#define NSIGNER_VERSION_MAJOR 0
#define NSIGNER_VERSION_MINOR 1
#define NSIGNER_VERSION_PATCH 6
#define NSIGNER_VERSION "v0.1.6"
#define NSIGNER_VERSION_PATCH 7
#define NSIGNER_VERSION "v0.1.7"
/* NSIGNER_HEADERLESS_DECLS_END */