n_signer/plans/derive_hmac.md

Plan: derive verb — HMAC-SHA256 from a derived private key

Problem

sovereign_browser stores bookmarks as NIP-51 kind 30003 parameterized-replaceable events, one per folder. The d tag must be a deterministic function of the folder path so that multiple devices editing the same logical folder produce the same d tag and the relay's NIP-33 replaceability keeps only the latest event. A random d tag would break cross-device sync (each device would emit a new event for the same folder, with no dedup).

The browser currently computes the d tag locally as:

hmac_key = HMAC-SHA256(privkey, "sovereign-browser/bookmarks-folder-id-v1")
d        = HMAC-SHA256(hmac_key, path)   → 64 hex chars

This only works when the privkey is in browser memory (login methods generate / import). When the signer is the nsigner remote backend, the privkey is held by n_signer and never exposed to the browser, so compute_hmac_key() returns -1 and path_to_d_tag falls back to plaintext path d tags — a privacy regression that leaks folder names to relays.

Decision

Add a single new algorithm-based verb to n_signer:

derive(data) = HMAC-SHA256(privkey, data)   → 64 hex chars

• privkey is the secp256k1 private key derived on demand from the mnemonic at (algorithm: "secp256k1", index: N) via the existing alg_key_cache_derive.
• data is an arbitrary caller-supplied UTF-8 string (or hex-encoded bytes).
• Returns the 32-byte HMAC digest as 64 lowercase hex chars.

This is a single-step scheme. The browser will switch from its two-step scheme to single-step by concatenating the label and path into the data string before calling derive:

d = derive("sovereign-browser/bookmarks-folder-id-v1:" + path)
  = HMAC-SHA256(privkey, "sovereign-browser/bookmarks-folder-id-v1:" + path)

The label prefix provides domain separation (so the same path used by a different application produces a different d tag). This is cryptographically equivalent to the two-step scheme for the privacy properties that matter (determinism, opacity, per-user-ness, no label/path recovery from the hash).

Why single-step over two-step

1. Generic primitive. derive(data) = HMAC(privkey, data) is a clean building block any caller can use for any purpose — bookmark d tags, per-resource identifiers, per-app secrets, etc. A two-step verb bakes a specific construction into the RPC.
2. No state across calls. A two-step scheme where the browser calls derive(LABEL) then derive(path) cannot work because step 2 needs hmac_key as the HMAC key, not privkey — and a privkey-keyed-only primitive always uses privkey as the key. Single-step avoids this entirely.
3. Migration cost is acceptable. The browser already has a migration pattern for legacy d tags (detect on load, re-publish with new d tag, kind-5 delete old). The same pattern handles the two-step → single-step transition.

Migration of existing two-step bookmark events

Existing bookmark events on relays have d = HMAC-SHA256(HMAC-SHA256(privkey, LABEL), path). After the switch, the browser will compute d = HMAC-SHA256(privkey, LABEL + ":" + path), which is a different hash. On next bookmarks_init:

1. Fetch all kind 30003 events as today.
2. For each event, decrypt content, read path.
3. Compute the new single-step d tag for that path.
4. If the event's d tag does not match the new d tag (i.e. it's an old two-step tag), re-publish the event with the new d tag and emit a kind-5 deletion for the old event id.
5. This is the same logic the browser already uses for legacy plaintext d tags (see plans/bookmarks-tree-view.md §"Why this is safe and nostr-ish" — "Legacy events with plaintext d = "General" will be detected on load... decrypted, and re-published in the new HMAC-d format; the old events get a kind 5 deletion.").

The migration is self-healing: each device migrates the events it sees, and once all devices have upgraded, no old two-step d tags remain on relays.

n_signer changes

1. New verb constant

src/enforcement.c (and the headerless decls block in every other src file that carries it):

#define VERB_DERIVE  "derive"

Add it to the algorithm-based verb set in is_algorithm_verb():

return (strcmp(method, VERB_SIGN) == 0 ||
        strcmp(method, VERB_VERIFY) == 0 ||
        strcmp(method, VERB_ENCAPSULATE) == 0 ||
        strcmp(method, VERB_DECAPSULATE) == 0 ||
        strcmp(method, VERB_DERIVE_SHARED) == 0 ||
        strcmp(method, VERB_GET_PUBLIC_KEY) == 0 ||
        strcmp(method, VERB_DERIVE) == 0);

2. Enforcement

enforce_verb_algorithm(): derive is valid for secp256k1 only (the privkey is a 32-byte scalar suitable as an HMAC key; PQ private keys are not). Add:

/* derive: HMAC-SHA256(privkey, data). secp256k1 only (32-byte scalar key). */
if (strcmp(verb, VERB_DERIVE) == 0) {
    if (alg == CRYPTO_ALG_SECP256K1) {
        return ENFORCE_OK;
    }
    return ENFORCE_ERR_ALGORITHM;
}

3. Dispatcher handler

handle_algorithm_verb(): add a new branch after the existing derive_shared_secret branch. Request shape:

{"id":"...","method":"derive","params":["<data>",{"algorithm":"secp256k1","index":0}]}

• params[0] = the data string (UTF-8).
• options.algorithm = "secp256k1" (required).
• options.index = derivation index (required — no default). The dispatcher returns error -32602 missing_index if index is absent.

Handler:

/* ---- derive (secp256k1 HMAC-SHA256) ---- */
if (strcmp(method, VERB_DERIVE) == 0) {
    cJSON *data_item = cJSON_GetArrayItem(params_item, 0);
    cJSON *index_item = NULL;
    const char *data_str;
    const unsigned char *priv;
    unsigned char mac[32];
    char mac_hex[65];
    char *result;

    if (!cJSON_IsString(data_item) || data_item->valuestring == NULL) {
        return make_error_response(id_str, -32602, "invalid_params");
    }
    data_str = data_item->valuestring;

    /* index is required for derive (no default). */
    if (!cJSON_IsObject(options_item)) {
        return make_error_response(id_str, -32602, "missing_index");
    }
    index_item = cJSON_GetObjectItemCaseSensitive(options_item, "index");
    if (!cJSON_IsNumber(index_item)) {
        return make_error_response(id_str, -32602, "missing_index");
    }
    index = index_item->valueint;

    /* Derive the secp256k1 key on demand. */
    if (alg_key_cache_derive(ctx->alg_key_cache, ctx->mnemonic, alg, index) != 0) {
        return make_error_response(id_str, -32602, "key_derivation_failed");
    }
    key_entry = alg_key_cache_get(ctx->alg_key_cache, alg, index);
    if (key_entry == NULL || !key_entry->valid) {
        return make_error_response(id_str, -32602, "key_derivation_failed");
    }
    priv = (const unsigned char *)key_entry->private_key.data;

    /* HMAC-SHA256(privkey, data) */
    if (nostr_hmac_sha256(priv, sz->priv_key_len,
                           (const unsigned char *)data_str, strlen(data_str),
                           mac) != 0) {
        return make_error_response(id_str, -32602, "hmac_failed");
    }

    nostr_bytes_to_hex(mac, 32, mac_hex);
    secure_memzero(mac, sizeof(mac));

    result = build_alg_result_json(alg_name, key_entry->key_id,
                                     "digest", mac_hex);
    if (result == NULL) {
        return make_error_response(id_str, -32602, "invalid_params");
    }
    return make_success_response(id_str, result);
}

nostr_hmac_sha256 is already available via <nostr_core/utils.h> (linked into n_signer through nostr_core_lib). secure_memzero clears the stack-local mac.

Response:

{"id":"...","result":"{\"algorithm\":\"secp256k1\",\"key_id\":\"<16hex>\",\"digest\":\"<64hex>\"}"}

4. Headerless decls

The VERB_DERIVE define must be added to the NSIGNER_HEADERLESS_DECLS_BEGIN block in every file that carries it. The same block already exists in:

• src/dispatcher.c
• src/enforcement.c
• src/key_store.c
• src/main.c
• src/policy.c
• src/selector.c
• src/server.c
• src/role_table.c
• src/mnemonic.c
• src/secure_mem.c
• src/socket_name.c
• src/pq_crypto.c
• tests/test_*.c (each test file that carries the block)

5. Tests

Add a test in tests/test_algorithm_api.c (or a new tests/test_derive.c):

1. Load a known mnemonic, derive secp256k1 key at index 0.
2. Call derive with data = "test-data".
3. Independently compute HMAC-SHA256(privkey, "test-data") using nostr_hmac_sha256 and compare to the RPC result.
4. Assert determinism: same input → same digest.
5. Assert opaqueness: different inputs → different digests.
6. Assert rejection for non-secp256k1 algorithms (e.g. algorithm: "ed25519" → error 1010 algorithm_not_supported_for_verb).
7. Assert rejection when mnemonic is not loaded (error 1006).

6. Documentation

• README.md §4.3 verb table: add derive row.
• README.md §4.4 algorithms: note derive is secp256k1-only.
• api.md: add worked example.
• client/README.md: add derive to the verb table.
• documents/CLIENT_IMPLEMENTATION.md: add derive to the algorithm-based verb list with example request/response.

nostr_core_lib client changes

nostr_core_lib/nostr_core/nostr_signer.h and nostr_core_lib/nostr_core/nostr_signer.c:

Add a new high-level API:

/* Compute HMAC-SHA256(privkey, data) using the signer's derived secp256k1
 * private key. Returns the 32-byte digest as 64 lowercase hex chars + NUL.
 * For the local backend, computes directly. For the nsigner remote backend,
 * calls the "derive" verb.
 * data must be a NUL-terminated UTF-8 string.
 * Returns NOSTR_SUCCESS or an error code. */
int nostr_signer_derive_hmac(nostr_signer_t* signer,
                             const char* data,
                             char out_digest_hex[65]);

Local backend

static int signer_local_derive_hmac(nostr_signer_t* signer,
                                    const char* data,
                                    char out_digest_hex[65]) {
    unsigned char mac[32];
    if (!signer || !data || !out_digest_hex) {
        return NOSTR_ERROR_INVALID_INPUT;
    }
    if (nostr_hmac_sha256(signer->u.local.private_key, 32,
                           (const unsigned char*)data, strlen(data),
                           mac) != 0) {
        return NOSTR_ERROR_CRYPTO_FAILED;
    }
    nostr_bytes_to_hex(mac, 32, out_digest_hex);
    memset(mac, 0, sizeof(mac));
    return NOSTR_SUCCESS;
}

Remote (nsigner) backend

static int signer_remote_derive_hmac(nostr_signer_t* signer,
                                     const char* data,
                                     char out_digest_hex[65]) {
    cJSON* params;
    cJSON* result = NULL;
    const char* result_str;
    int rc;

    params = cJSON_CreateArray();
    if (params == NULL) return NOSTR_ERROR_MEMORY_FAILED;
    cJSON_AddItemToArray(params, cJSON_CreateString(data));

    params = signer_remote_params_with_selector(params,
              signer->u.remote.role,
              signer->u.remote.has_nostr_index,
              signer->u.remote.nostr_index);
    if (params == NULL) return NOSTR_ERROR_MEMORY_FAILED;

    /* The remote derive verb needs algorithm:"secp256k1" in the options. */
    {
        cJSON* opts = cJSON_GetArrayItem(params, cJSON_GetArraySize(params) - 1);
        if (opts != NULL && cJSON_IsObject(opts)) {
            cJSON_AddStringToObject(opts, "algorithm", "secp256k1");
        }
    }

    rc = nsigner_client_call(signer->u.remote.client, "derive", params, &result);
    if (rc != NOSTR_SUCCESS) return rc;

    /* result is a JSON string: {"algorithm":"secp256k1","key_id":"...","digest":"<64hex>"} */
    if (!cJSON_IsString(result) || result->valuestring == NULL) {
        cJSON_Delete(result);
        return NOSTR_ERROR_NIP46_INVALID_RESPONSE;
    }
    {
        cJSON* parsed = cJSON_Parse(result->valuestring);
        cJSON* digest_item;
        const char* digest_str;
        cJSON_Delete(result);
        if (parsed == NULL) return NOSTR_ERROR_NIP46_INVALID_RESPONSE;
        digest_item = cJSON_GetObjectItemCaseSensitive(parsed, "digest");
        if (!cJSON_IsString(digest_item) || digest_item->valuestring == NULL ||
            strlen(digest_item->valuestring) != 64) {
            cJSON_Delete(parsed);
            return NOSTR_ERROR_NIP46_INVALID_RESPONSE;
        }
        digest_str = digest_item->valuestring;
        memcpy(out_digest_hex, digest_str, 64);
        out_digest_hex[64] = '\0';
        cJSON_Delete(parsed);
    }
    return NOSTR_SUCCESS;
}

Note on the options object: the existing signer_remote_params_with_selector appends a selector object as the last params element. For algorithm-based verbs, the selector object must also carry algorithm and index. The implementation must ensure the options object already exists (or create one) before adding algorithm. If has_nostr_index is false and role is empty, signer_remote_params_with_selector does not append an options object — in that case the derive handler must append one with just algorithm. This is a small extension to the helper or a local construction in signer_remote_derive_hmac.

Test

nostr_core_lib/tests/nsigner_client_test.c: add a test that calls nostr_signer_derive_hmac on a local signer with a known privkey + known data, and compares against a reference HMAC-SHA256 computed with nostr_hmac_sha256 directly.

sovereign_browser changes

1. Switch to single-step d tag

src/bookmarks.c: replace compute_hmac_key + path_to_d_tag with a single function that calls the signer:

/* New label prefix — domain-separated, baked into the data string. */
#define BOOKMARKS_HMAC_KEY_LABEL "sovereign-browser/bookmarks-folder-id-v1"

static char *path_to_d_tag(const char *path) {
    if (path == NULL) path = "";

    /* Build "LABEL:" + path */
    char *data = g_strdup_printf("%s:%s", BOOKMARKS_HMAC_KEY_LABEL, path);
    char digest_hex[65];
    int rc;

    rc = nostr_signer_derive_hmac(g_signer, data, digest_hex);
    g_free(data);
    if (rc != NOSTR_SUCCESS) {
        /* No signer or derive failed — fall back to plaintext (read-only mode). */
        return g_strdup(path);
    }
    return g_strdup(digest_hex);
}

Remove g_hmac_key, g_have_hmac_key, compute_hmac_key, and the g_privkey / g_have_privkey state (the privkey is no longer needed in browser memory — the signer holds it). This is a security improvement: the browser no longer carries the privkey in its own RAM when using the nsigner backend.

2. Migration of existing two-step d tags

In bookmarks_init (or the relay-fetch load path), after decrypting an event and reading its path:

1. Compute the new single-step d tag: path_to_d_tag(path).
2. Compare to the event's actual d tag.
3. If they differ (old two-step tag), re-publish the event with the new d tag and emit a kind-5 deletion for the old event id.

This reuses the existing legacy-plaintext-d-tag migration logic. The detection predicate changes from "d tag is not 64 hex chars" to "d tag is 64 hex chars but does not match path_to_d_tag(path)".

3. Remove privkey plumbing

• src/main.c: remove privkey_hex from g_state (or stop populating it).
• src/main.c: remove the strncpy(g_state.privkey_hex, ...) line.
• src/bookmarks.h: change bookmarks_init signature to drop the privkey_hex parameter.
• All callers of bookmarks_init updated.

4. Test

tests/test_bookmarks_tree.c: update path_to_d_tag mirror to the single-step scheme:

static char *path_to_d_tag(const unsigned char *privkey, const char *path) {
    char *data = g_strdup_printf("%s:%s", LABEL, path ? path : "");
    unsigned char mac[32];
    char *hex;
    if (nostr_hmac_sha256(privkey, 32,
                           (const unsigned char*)data, strlen(data),
                           mac) != 0) {
        g_free(data);
        return NULL;
    }
    g_free(data);
    hex = g_strdup_printf(/* 64 hex chars */ ...);
    return hex;
}

All existing tests (determinism, opaqueness, 64-hex format, is_hmac_d_tag, per-user-ness, empty path) pass unchanged — they test properties, not the specific construction.

Architecture diagram

sequenceDiagram
    participant Browser as sovereign_browser
    participant Signer as n_signer
    participant Relay as Nostr relay

    Note over Browser: User saves bookmark to "Work/Projects/Secret"
    Browser->>Signer: derive("sovereign-browser/bookmarks-folder-id-v1:Work/Projects/Secret", {algorithm:"secp256k1", index:0})
    Signer->>Signer: alg_key_cache_derive(secp256k1, 0)
    Signer->>Signer: HMAC-SHA256(privkey, data)
    Signer-->>Browser: {"digest":"<64hex>"}
    Browser->>Signer: nostr_nip44_encrypt(self_pubkey, JSON{path, bookmarks})
    Signer-->>Browser: ciphertext
    Browser->>Relay: publish kind 30003, d=<64hex>, content=<ciphertext>

    Note over Browser: Other device starts up
    Browser->>Relay: fetch kind 30003 by pubkey
    Relay-->>Browser: events
    Browser->>Signer: nostr_nip44_decrypt(self_pubkey, content)
    Signer-->>Browser: JSON{path:"Work/Projects/Secret", bookmarks:[...]}
    Note over Browser: path recovered from decrypted content, not from d tag

File change summary

n_signer (this repo)

File	Change
src/enforcement.c	Add VERB_DERIVE define + derive case in enforce_verb_algorithm (secp256k1 only)
src/dispatcher.c	Add VERB_DERIVE to is_algorithm_verb + derive branch in handle_algorithm_verb
src/key_store.c	Add VERB_DERIVE define (headerless decls)
src/main.c	Add VERB_DERIVE define (headerless decls)
src/policy.c	Add VERB_DERIVE define (headerless decls)
src/selector.c	Add VERB_DERIVE define (headerless decls)
src/server.c	Add VERB_DERIVE define (headerless decls)
src/role_table.c	Add VERB_DERIVE define (headerless decls)
src/mnemonic.c	Add VERB_DERIVE define (headerless decls)
src/secure_mem.c	Add VERB_DERIVE define (headerless decls)
src/socket_name.c	Add VERB_DERIVE define (headerless decls)
src/pq_crypto.c	Add VERB_DERIVE define (headerless decls)
tests/test_algorithm_api.c	Add derive test cases
README.md	Add derive to verb table + algorithm notes
api.md	Add derive worked example
client/README.md	Add derive to verb table
documents/CLIENT_IMPLEMENTATION.md	Add derive section

nostr_core_lib (sovereign_browser repo)

File	Change
nostr_core/nostr_signer.h	Add nostr_signer_derive_hmac declaration
nostr_core/nostr_signer.c	Add local + remote derive_hmac implementations
tests/nsigner_client_test.c	Add derive_hmac test

sovereign_browser

File	Change
src/bookmarks.c	Replace two-step compute_hmac_key + path_to_d_tag with single-step path_to_d_tag calling nostr_signer_derive_hmac; add two-step→single-step migration on load; remove g_privkey/g_hmac_key state
src/bookmarks.h	Drop privkey_hex param from bookmarks_init
src/main.c	Remove privkey_hex from g_state + stop populating it; update bookmarks_init call
tests/test_bookmarks_tree.c	Update path_to_d_tag mirror to single-step scheme

Open questions

1. Should derive accept hex-encoded binary data, or only UTF-8 strings? Default: UTF-8 strings only (simpler, covers the bookmark use case). If a caller needs binary data, they hex-encode it and we add a data_encoding option later. Decision can be deferred.

2. index is required. Unlike other algorithm-based verbs that default index to 0, derive requires the caller to specify index explicitly. This forces conscious selection of which derived key to use as the HMAC key, avoiding accidental cross-identity d-tag collisions. The dispatcher returns -32602 missing_index if index is absent.

3. Should derive be added to the --preapprove algorithm-based policy syntax? Yes — it flows through the same policy_check_algorithm path as other algorithm-based verbs. Operators can pre-approve caller=...,algorithm=secp256k1,index=0,verb=derive. No code change needed beyond the verb define; policy matching is by string.