laantungir/n_signer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 40 Wiki Activity

v0.0.58 - Added derive verb: HMAC-SHA256(privkey, data) for secp256k1, enabling opaque d-tag derivation via nsigner remote backend without exposing the privkey

Browse Source
This commit is contained in:
Laan Tungir
2026-07-20 19:56:51 -04:00
parent b3421c3e40
commit ca18e1e42d
30 changed files with 777 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

538
plans/derive_hmac.md Normal file
View File

@@ -0,0 +1,538 @@
# Plan: `derive` verb — HMAC-SHA256 from a derived private key
## Problem
sovereign_browser stores bookmarks as NIP-51 kind 30003 parameterized-replaceable
events, one per folder. The `d` tag must be a **deterministic function of the
folder path** so that multiple devices editing the same logical folder produce
the same `d` tag and the relay's NIP-33 replaceability keeps only the latest
event. A random `d` tag would break cross-device sync (each device would emit a
new event for the same folder, with no dedup).
The browser currently computes the `d` tag locally as:
```
hmac_key = HMAC-SHA256(privkey, "sovereign-browser/bookmarks-folder-id-v1")
d = HMAC-SHA256(hmac_key, path) → 64 hex chars
```
This only works when the privkey is in browser memory (login methods
`generate` / `import`). When the signer is the **nsigner remote backend**, the
privkey is held by n_signer and never exposed to the browser, so
[`compute_hmac_key()`](../sovereign_browser/src/bookmarks.c:194) returns -1 and
[`path_to_d_tag`](../sovereign_browser/src/bookmarks.c:221) falls back to
**plaintext** path d tags — a privacy regression that leaks folder names to
relays.
## Decision
Add a single new algorithm-based verb to n_signer:
```
derive(data) = HMAC-SHA256(privkey, data) → 64 hex chars
```
- `privkey` is the secp256k1 private key derived on demand from the mnemonic at
`(algorithm: "secp256k1", index: N)` via the existing
[`alg_key_cache_derive`](src/key_store.c:1624).
- `data` is an arbitrary caller-supplied UTF-8 string (or hex-encoded bytes).
- Returns the 32-byte HMAC digest as 64 lowercase hex chars.
This is a **single-step** scheme. The browser will switch from its two-step
scheme to single-step by concatenating the label and path into the data string
before calling `derive`:
```
d = derive("sovereign-browser/bookmarks-folder-id-v1:" + path)
= HMAC-SHA256(privkey, "sovereign-browser/bookmarks-folder-id-v1:" + path)
```
The label prefix provides domain separation (so the same path used by a
different application produces a different d tag). This is cryptographically
equivalent to the two-step scheme for the privacy properties that matter
(determinism, opacity, per-user-ness, no label/path recovery from the hash).
### Why single-step over two-step
1. **Generic primitive.** `derive(data) = HMAC(privkey, data)` is a clean
building block any caller can use for any purpose — bookmark d tags,
per-resource identifiers, per-app secrets, etc. A two-step verb bakes a
specific construction into the RPC.
2. **No state across calls.** A two-step scheme where the browser calls
`derive(LABEL)` then `derive(path)` cannot work because step 2 needs
`hmac_key` as the HMAC **key**, not `privkey` — and a privkey-keyed-only
primitive always uses `privkey` as the key. Single-step avoids this entirely.
3. **Migration cost is acceptable.** The browser already has a migration pattern
for legacy d tags (detect on load, re-publish with new d tag, kind-5 delete
old). The same pattern handles the two-step → single-step transition.
### Migration of existing two-step bookmark events
Existing bookmark events on relays have `d = HMAC-SHA256(HMAC-SHA256(privkey, LABEL), path)`.
After the switch, the browser will compute `d = HMAC-SHA256(privkey, LABEL + ":" + path)`,
which is a different hash. On next `bookmarks_init`:
1. Fetch all kind 30003 events as today.
2. For each event, decrypt content, read `path`.
3. Compute the **new** single-step d tag for that path.
4. If the event's `d` tag does **not** match the new d tag (i.e. it's an old
two-step tag), re-publish the event with the new d tag and emit a kind-5
deletion for the old event id.
5. This is the same logic the browser already uses for legacy plaintext d tags
(see [`plans/bookmarks-tree-view.md`](../sovereign_browser/plans/bookmarks-tree-view.md:61)
§"Why this is safe and nostr-ish" — "Legacy events with plaintext `d = "General"`
will be detected on load... decrypted, and re-published in the new HMAC-`d`
format; the old events get a kind 5 deletion.").
The migration is **self-healing**: each device migrates the events it sees, and
once all devices have upgraded, no old two-step d tags remain on relays.
## n_signer changes
### 1. New verb constant
[`src/enforcement.c`](src/enforcement.c:209) (and the headerless decls block in
every other src file that carries it):
```c
#define VERB_DERIVE "derive"
```
Add it to the algorithm-based verb set in
[`is_algorithm_verb()`](src/dispatcher.c:817):
```c
return (strcmp(method, VERB_SIGN) == 0 ||
strcmp(method, VERB_VERIFY) == 0 ||
strcmp(method, VERB_ENCAPSULATE) == 0 ||
strcmp(method, VERB_DECAPSULATE) == 0 ||
strcmp(method, VERB_DERIVE_SHARED) == 0 ||
strcmp(method, VERB_GET_PUBLIC_KEY) == 0 ||
strcmp(method, VERB_DERIVE) == 0);
```
### 2. Enforcement
[`enforce_verb_algorithm()`](src/enforcement.c:765): `derive` is valid for
`secp256k1` only (the privkey is a 32-byte scalar suitable as an HMAC key; PQ
private keys are not). Add:
```c
/* derive: HMAC-SHA256(privkey, data). secp256k1 only (32-byte scalar key). */
if (strcmp(verb, VERB_DERIVE) == 0) {
if (alg == CRYPTO_ALG_SECP256K1) {
return ENFORCE_OK;
}
return ENFORCE_ERR_ALGORITHM;
}
```
### 3. Dispatcher handler
[`handle_algorithm_verb()`](src/dispatcher.c:880): add a new branch after the
existing `derive_shared_secret` branch. Request shape:
```json
{"id":"...","method":"derive","params":["<data>",{"algorithm":"secp256k1","index":0}]}
```
- `params[0]` = the data string (UTF-8).
- `options.algorithm` = `"secp256k1"` (required).
- `options.index` = derivation index (**required** — no default). The dispatcher
returns error `-32602 missing_index` if `index` is absent.
Handler:
```c
/* ---- derive (secp256k1 HMAC-SHA256) ---- */
if (strcmp(method, VERB_DERIVE) == 0) {
cJSON *data_item = cJSON_GetArrayItem(params_item, 0);
cJSON *index_item = NULL;
const char *data_str;
const unsigned char *priv;
unsigned char mac[32];
char mac_hex[65];
char *result;
if (!cJSON_IsString(data_item) || data_item->valuestring == NULL) {
return make_error_response(id_str, -32602, "invalid_params");
}
data_str = data_item->valuestring;
/* index is required for derive (no default). */
if (!cJSON_IsObject(options_item)) {
return make_error_response(id_str, -32602, "missing_index");
}
index_item = cJSON_GetObjectItemCaseSensitive(options_item, "index");
if (!cJSON_IsNumber(index_item)) {
return make_error_response(id_str, -32602, "missing_index");
}
index = index_item->valueint;
/* Derive the secp256k1 key on demand. */
if (alg_key_cache_derive(ctx->alg_key_cache, ctx->mnemonic, alg, index) != 0) {
return make_error_response(id_str, -32602, "key_derivation_failed");
}
key_entry = alg_key_cache_get(ctx->alg_key_cache, alg, index);
if (key_entry == NULL || !key_entry->valid) {
return make_error_response(id_str, -32602, "key_derivation_failed");
}
priv = (const unsigned char *)key_entry->private_key.data;
/* HMAC-SHA256(privkey, data) */
if (nostr_hmac_sha256(priv, sz->priv_key_len,
(const unsigned char *)data_str, strlen(data_str),
mac) != 0) {
return make_error_response(id_str, -32602, "hmac_failed");
}
nostr_bytes_to_hex(mac, 32, mac_hex);
secure_memzero(mac, sizeof(mac));
result = build_alg_result_json(alg_name, key_entry->key_id,
"digest", mac_hex);
if (result == NULL) {
return make_error_response(id_str, -32602, "invalid_params");
}
return make_success_response(id_str, result);
}
```
`nostr_hmac_sha256` is already available via `<nostr_core/utils.h>` (linked into
n_signer through nostr_core_lib). `secure_memzero` clears the stack-local mac.
Response:
```json
{"id":"...","result":"{\"algorithm\":\"secp256k1\",\"key_id\":\"<16hex>\",\"digest\":\"<64hex>\"}"}
```
### 4. Headerless decls
The `VERB_DERIVE` define must be added to the `NSIGNER_HEADERLESS_DECLS_BEGIN`
block in every file that carries it. The same block already exists in:
- [`src/dispatcher.c`](src/dispatcher.c:209)
- [`src/enforcement.c`](src/enforcement.c:210)
- [`src/key_store.c`](src/key_store.c:208)
- [`src/main.c`](src/main.c:215)
- [`src/policy.c`](src/policy.c:209)
- [`src/selector.c`](src/selector.c:209)
- [`src/server.c`](src/server.c:215)
- [`src/role_table.c`](src/role_table.c:212)
- [`src/mnemonic.c`](src/mnemonic.c:209)
- [`src/secure_mem.c`](src/secure_mem.c:211)
- [`src/socket_name.c`](src/socket_name.c:211)
- [`src/pq_crypto.c`](src/pq_crypto.c:222)
- [`tests/test_*.c`](tests/) (each test file that carries the block)
### 5. Tests
Add a test in [`tests/test_algorithm_api.c`](tests/test_algorithm_api.c) (or a
new `tests/test_derive.c`):
1. Load a known mnemonic, derive secp256k1 key at index 0.
2. Call `derive` with `data = "test-data"`.
3. Independently compute `HMAC-SHA256(privkey, "test-data")` using
`nostr_hmac_sha256` and compare to the RPC result.
4. Assert determinism: same input → same digest.
5. Assert opaqueness: different inputs → different digests.
6. Assert rejection for non-secp256k1 algorithms (e.g. `algorithm: "ed25519"`
→ error 1010 `algorithm_not_supported_for_verb`).
7. Assert rejection when mnemonic is not loaded (error 1006).
### 6. Documentation
- [`README.md`](README.md) §4.3 verb table: add `derive` row.
- [`README.md`](README.md) §4.4 algorithms: note `derive` is secp256k1-only.
- [`api.md`](api.md): add worked example.
- [`client/README.md`](client/README.md): add `derive` to the verb table.
- [`documents/CLIENT_IMPLEMENTATION.md`](documents/CLIENT_IMPLEMENTATION.md):
add `derive` to the algorithm-based verb list with example request/response.
## nostr_core_lib client changes
[`nostr_core_lib/nostr_core/nostr_signer.h`](../sovereign_browser/nostr_core_lib/nostr_core/nostr_signer.h)
and
[`nostr_core_lib/nostr_core/nostr_signer.c`](../sovereign_browser/nostr_core_lib/nostr_core/nostr_signer.c):
Add a new high-level API:
```c
/* Compute HMAC-SHA256(privkey, data) using the signer's derived secp256k1
* private key. Returns the 32-byte digest as 64 lowercase hex chars + NUL.
* For the local backend, computes directly. For the nsigner remote backend,
* calls the "derive" verb.
* data must be a NUL-terminated UTF-8 string.
* Returns NOSTR_SUCCESS or an error code. */
int nostr_signer_derive_hmac(nostr_signer_t* signer,
const char* data,
char out_digest_hex[65]);
```
### Local backend
```c
static int signer_local_derive_hmac(nostr_signer_t* signer,
const char* data,
char out_digest_hex[65]) {
unsigned char mac[32];
if (!signer || !data || !out_digest_hex) {
return NOSTR_ERROR_INVALID_INPUT;
}
if (nostr_hmac_sha256(signer->u.local.private_key, 32,
(const unsigned char*)data, strlen(data),
mac) != 0) {
return NOSTR_ERROR_CRYPTO_FAILED;
}
nostr_bytes_to_hex(mac, 32, out_digest_hex);
memset(mac, 0, sizeof(mac));
return NOSTR_SUCCESS;
}
```
### Remote (nsigner) backend
```c
static int signer_remote_derive_hmac(nostr_signer_t* signer,
const char* data,
char out_digest_hex[65]) {
cJSON* params;
cJSON* result = NULL;
const char* result_str;
int rc;
params = cJSON_CreateArray();
if (params == NULL) return NOSTR_ERROR_MEMORY_FAILED;
cJSON_AddItemToArray(params, cJSON_CreateString(data));
params = signer_remote_params_with_selector(params,
signer->u.remote.role,
signer->u.remote.has_nostr_index,
signer->u.remote.nostr_index);
if (params == NULL) return NOSTR_ERROR_MEMORY_FAILED;
/* The remote derive verb needs algorithm:"secp256k1" in the options. */
{
cJSON* opts = cJSON_GetArrayItem(params, cJSON_GetArraySize(params) - 1);
if (opts != NULL && cJSON_IsObject(opts)) {
cJSON_AddStringToObject(opts, "algorithm", "secp256k1");
}
}
rc = nsigner_client_call(signer->u.remote.client, "derive", params, &result);
if (rc != NOSTR_SUCCESS) return rc;
/* result is a JSON string: {"algorithm":"secp256k1","key_id":"...","digest":"<64hex>"} */
if (!cJSON_IsString(result) || result->valuestring == NULL) {
cJSON_Delete(result);
return NOSTR_ERROR_NIP46_INVALID_RESPONSE;
}
{
cJSON* parsed = cJSON_Parse(result->valuestring);
cJSON* digest_item;
const char* digest_str;
cJSON_Delete(result);
if (parsed == NULL) return NOSTR_ERROR_NIP46_INVALID_RESPONSE;
digest_item = cJSON_GetObjectItemCaseSensitive(parsed, "digest");
if (!cJSON_IsString(digest_item) || digest_item->valuestring == NULL ||
strlen(digest_item->valuestring) != 64) {
cJSON_Delete(parsed);
return NOSTR_ERROR_NIP46_INVALID_RESPONSE;
}
digest_str = digest_item->valuestring;
memcpy(out_digest_hex, digest_str, 64);
out_digest_hex[64] = '\0';
cJSON_Delete(parsed);
}
return NOSTR_SUCCESS;
}
```
**Note on the options object:** the existing
`signer_remote_params_with_selector` appends a selector object as the last
params element. For algorithm-based verbs, the selector object must also carry
`algorithm` and `index`. The implementation must ensure the options object
already exists (or create one) before adding `algorithm`. If
`has_nostr_index` is false and `role` is empty, `signer_remote_params_with_selector`
does not append an options object — in that case the derive handler must append
one with just `algorithm`. This is a small extension to the helper or a local
construction in `signer_remote_derive_hmac`.
### Test
[`nostr_core_lib/tests/nsigner_client_test.c`](../sovereign_browser/nostr_core_lib/tests/nsigner_client_test.c):
add a test that calls `nostr_signer_derive_hmac` on a local signer with a known
privkey + known data, and compares against a reference HMAC-SHA256 computed
with `nostr_hmac_sha256` directly.
## sovereign_browser changes
### 1. Switch to single-step d tag
[`src/bookmarks.c`](../sovereign_browser/src/bookmarks.c:194): replace
`compute_hmac_key` + `path_to_d_tag` with a single function that calls the
signer:
```c
/* New label prefix — domain-separated, baked into the data string. */
#define BOOKMARKS_HMAC_KEY_LABEL "sovereign-browser/bookmarks-folder-id-v1"
static char *path_to_d_tag(const char *path) {
if (path == NULL) path = "";
/* Build "LABEL:" + path */
char *data = g_strdup_printf("%s:%s", BOOKMARKS_HMAC_KEY_LABEL, path);
char digest_hex[65];
int rc;
rc = nostr_signer_derive_hmac(g_signer, data, digest_hex);
g_free(data);
if (rc != NOSTR_SUCCESS) {
/* No signer or derive failed — fall back to plaintext (read-only mode). */
return g_strdup(path);
}
return g_strdup(digest_hex);
}
```
Remove `g_hmac_key`, `g_have_hmac_key`, `compute_hmac_key`, and the
`g_privkey` / `g_have_privkey` state (the privkey is no longer needed in
browser memory — the signer holds it). This is a **security improvement**: the
browser no longer carries the privkey in its own RAM when using the nsigner
backend.
### 2. Migration of existing two-step d tags
In `bookmarks_init` (or the relay-fetch load path), after decrypting an event
and reading its `path`:
1. Compute the new single-step d tag: `path_to_d_tag(path)`.
2. Compare to the event's actual `d` tag.
3. If they differ (old two-step tag), re-publish the event with the new d tag
and emit a kind-5 deletion for the old event id.
This reuses the existing legacy-plaintext-d-tag migration logic. The detection
predicate changes from "d tag is not 64 hex chars" to "d tag is 64 hex chars
but does not match `path_to_d_tag(path)`".
### 3. Remove privkey plumbing
- [`src/main.c`](../sovereign_browser/src/main.c:69): remove `privkey_hex` from
`g_state` (or stop populating it).
- [`src/main.c`](../sovereign_browser/src/main.c:652): remove the
`strncpy(g_state.privkey_hex, ...)` line.
- [`src/bookmarks.h`](../sovereign_browser/src/bookmarks.h): change
`bookmarks_init` signature to drop the `privkey_hex` parameter.
- All callers of `bookmarks_init` updated.
### 4. Test
[`tests/test_bookmarks_tree.c`](../sovereign_browser/tests/test_bookmarks_tree.c):
update `path_to_d_tag` mirror to the single-step scheme:
```c
static char *path_to_d_tag(const unsigned char *privkey, const char *path) {
char *data = g_strdup_printf("%s:%s", LABEL, path ? path : "");
unsigned char mac[32];
char *hex;
if (nostr_hmac_sha256(privkey, 32,
(const unsigned char*)data, strlen(data),
mac) != 0) {
g_free(data);
return NULL;
}
g_free(data);
hex = g_strdup_printf(/* 64 hex chars */ ...);
return hex;
}
```
All existing tests (determinism, opaqueness, 64-hex format, is_hmac_d_tag,
per-user-ness, empty path) pass unchanged — they test properties, not the
specific construction.
## Architecture diagram
```mermaid
sequenceDiagram
participant Browser as sovereign_browser
participant Signer as n_signer
participant Relay as Nostr relay
Note over Browser: User saves bookmark to "Work/Projects/Secret"
Browser->>Signer: derive("sovereign-browser/bookmarks-folder-id-v1:Work/Projects/Secret", {algorithm:"secp256k1", index:0})
Signer->>Signer: alg_key_cache_derive(secp256k1, 0)
Signer->>Signer: HMAC-SHA256(privkey, data)
Signer-->>Browser: {"digest":"<64hex>"}
Browser->>Signer: nostr_nip44_encrypt(self_pubkey, JSON{path, bookmarks})
Signer-->>Browser: ciphertext
Browser->>Relay: publish kind 30003, d=<64hex>, content=<ciphertext>
Note over Browser: Other device starts up
Browser->>Relay: fetch kind 30003 by pubkey
Relay-->>Browser: events
Browser->>Signer: nostr_nip44_decrypt(self_pubkey, content)
Signer-->>Browser: JSON{path:"Work/Projects/Secret", bookmarks:[...]}
Note over Browser: path recovered from decrypted content, not from d tag
```
## File change summary
### n_signer (this repo)
| File | Change |
|------|--------|
| [`src/enforcement.c`](src/enforcement.c) | Add `VERB_DERIVE` define + `derive` case in `enforce_verb_algorithm` (secp256k1 only) |
| [`src/dispatcher.c`](src/dispatcher.c) | Add `VERB_DERIVE` to `is_algorithm_verb` + `derive` branch in `handle_algorithm_verb` |
| [`src/key_store.c`](src/key_store.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/main.c`](src/main.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/policy.c`](src/policy.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/selector.c`](src/selector.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/server.c`](src/server.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/role_table.c`](src/role_table.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/mnemonic.c`](src/mnemonic.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/secure_mem.c`](src/secure_mem.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/socket_name.c`](src/socket_name.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/pq_crypto.c`](src/pq_crypto.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`tests/test_algorithm_api.c`](tests/test_algorithm_api.c) | Add `derive` test cases |
| [`README.md`](README.md) | Add `derive` to verb table + algorithm notes |
| [`api.md`](api.md) | Add `derive` worked example |
| [`client/README.md`](client/README.md) | Add `derive` to verb table |
| [`documents/CLIENT_IMPLEMENTATION.md`](documents/CLIENT_IMPLEMENTATION.md) | Add `derive` section |
### nostr_core_lib (sovereign_browser repo)
| File | Change |
|------|--------|
| [`nostr_core/nostr_signer.h`](../sovereign_browser/nostr_core_lib/nostr_core/nostr_signer.h) | Add `nostr_signer_derive_hmac` declaration |
| [`nostr_core/nostr_signer.c`](../sovereign_browser/nostr_core_lib/nostr_core/nostr_signer.c) | Add local + remote `derive_hmac` implementations |
| [`tests/nsigner_client_test.c`](../sovereign_browser/nostr_core_lib/tests/nsigner_client_test.c) | Add `derive_hmac` test |
### sovereign_browser
| File | Change |
|------|--------|
| [`src/bookmarks.c`](../sovereign_browser/src/bookmarks.c) | Replace two-step `compute_hmac_key` + `path_to_d_tag` with single-step `path_to_d_tag` calling `nostr_signer_derive_hmac`; add two-step→single-step migration on load; remove `g_privkey`/`g_hmac_key` state |
| [`src/bookmarks.h`](../sovereign_browser/src/bookmarks.h) | Drop `privkey_hex` param from `bookmarks_init` |
| [`src/main.c`](../sovereign_browser/src/main.c) | Remove `privkey_hex` from `g_state` + stop populating it; update `bookmarks_init` call |
| [`tests/test_bookmarks_tree.c`](../sovereign_browser/tests/test_bookmarks_tree.c) | Update `path_to_d_tag` mirror to single-step scheme |
## Open questions
1. **Should `derive` accept hex-encoded binary data, or only UTF-8 strings?**
Default: UTF-8 strings only (simpler, covers the bookmark use case). If a
caller needs binary data, they hex-encode it and we add a `data_encoding`
option later. Decision can be deferred.
2. **`index` is required.** Unlike other algorithm-based verbs that default
`index` to 0, `derive` requires the caller to specify `index` explicitly.
This forces conscious selection of which derived key to use as the HMAC key,
avoiding accidental cross-identity d-tag collisions. The dispatcher returns
`-32602 missing_index` if `index` is absent.
3. **Should `derive` be added to the `--preapprove` algorithm-based policy
syntax?**
Yes — it flows through the same `policy_check_algorithm` path as other
algorithm-based verbs. Operators can pre-approve
`caller=...,algorithm=secp256k1,index=0,verb=derive`. No code change needed
beyond the verb define; policy matching is by string.