n_signer/plans/derive_hmac.md

# Plan: `derive` verb — HMAC-SHA256 from a derived private key

## Problem

sovereign_browser stores bookmarks as NIP-51 kind 30003 parameterized-replaceable
events, one per folder. The `d` tag must be a **deterministic function of the
folder path** so that multiple devices editing the same logical folder produce
the same `d` tag and the relay's NIP-33 replaceability keeps only the latest
event. A random `d` tag would break cross-device sync (each device would emit a
new event for the same folder, with no dedup).

The browser currently computes the `d` tag locally as:

```
hmac_key = HMAC-SHA256(privkey, "sovereign-browser/bookmarks-folder-id-v1")
d        = HMAC-SHA256(hmac_key, path)   → 64 hex chars
```

This only works when the privkey is in browser memory (login methods
`generate` / `import`). When the signer is the **nsigner remote backend**, the
privkey is held by n_signer and never exposed to the browser, so
[`compute_hmac_key()`](../sovereign_browser/src/bookmarks.c:194) returns -1 and
[`path_to_d_tag`](../sovereign_browser/src/bookmarks.c:221) falls back to
**plaintext** path d tags — a privacy regression that leaks folder names to
relays.

## Decision

Add a single new algorithm-based verb to n_signer:

```
derive(data) = HMAC-SHA256(privkey, data)   → 64 hex chars
```

- `privkey` is the secp256k1 private key derived on demand from the mnemonic at
  `(algorithm: "secp256k1", index: N)` via the existing
  [`alg_key_cache_derive`](src/key_store.c:1624).
- `data` is an arbitrary caller-supplied UTF-8 string (or hex-encoded bytes).
- Returns the 32-byte HMAC digest as 64 lowercase hex chars.

This is a **single-step** scheme. The browser will switch from its two-step
scheme to single-step by concatenating the label and path into the data string
before calling `derive`:

```
d = derive("sovereign-browser/bookmarks-folder-id-v1:" + path)
  = HMAC-SHA256(privkey, "sovereign-browser/bookmarks-folder-id-v1:" + path)
```

The label prefix provides domain separation (so the same path used by a
different application produces a different d tag). This is cryptographically
equivalent to the two-step scheme for the privacy properties that matter
(determinism, opacity, per-user-ness, no label/path recovery from the hash).

### Why single-step over two-step

1. **Generic primitive.** `derive(data) = HMAC(privkey, data)` is a clean
   building block any caller can use for any purpose — bookmark d tags,
   per-resource identifiers, per-app secrets, etc. A two-step verb bakes a
   specific construction into the RPC.
2. **No state across calls.** A two-step scheme where the browser calls
   `derive(LABEL)` then `derive(path)` cannot work because step 2 needs
   `hmac_key` as the HMAC **key**, not `privkey` — and a privkey-keyed-only
   primitive always uses `privkey` as the key. Single-step avoids this entirely.
3. **Migration cost is acceptable.** The browser already has a migration pattern
   for legacy d tags (detect on load, re-publish with new d tag, kind-5 delete
   old). The same pattern handles the two-step → single-step transition.

### Migration of existing two-step bookmark events

Existing bookmark events on relays have `d = HMAC-SHA256(HMAC-SHA256(privkey, LABEL), path)`.
After the switch, the browser will compute `d = HMAC-SHA256(privkey, LABEL + ":" + path)`,
which is a different hash. On next `bookmarks_init`:

1. Fetch all kind 30003 events as today.
2. For each event, decrypt content, read `path`.
3. Compute the **new** single-step d tag for that path.
4. If the event's `d` tag does **not** match the new d tag (i.e. it's an old
   two-step tag), re-publish the event with the new d tag and emit a kind-5
   deletion for the old event id.
5. This is the same logic the browser already uses for legacy plaintext d tags
   (see [`plans/bookmarks-tree-view.md`](../sovereign_browser/plans/bookmarks-tree-view.md:61)
   §"Why this is safe and nostr-ish" — "Legacy events with plaintext `d = "General"`
   will be detected on load... decrypted, and re-published in the new HMAC-`d`
   format; the old events get a kind 5 deletion.").

The migration is **self-healing**: each device migrates the events it sees, and
once all devices have upgraded, no old two-step d tags remain on relays.

## n_signer changes

### 1. New verb constant

[`src/enforcement.c`](src/enforcement.c:209) (and the headerless decls block in
every other src file that carries it):

```c
#define VERB_DERIVE  "derive"
```

Add it to the algorithm-based verb set in
[`is_algorithm_verb()`](src/dispatcher.c:817):

```c
return (strcmp(method, VERB_SIGN) == 0 ||
        strcmp(method, VERB_VERIFY) == 0 ||
        strcmp(method, VERB_ENCAPSULATE) == 0 ||
        strcmp(method, VERB_DECAPSULATE) == 0 ||
        strcmp(method, VERB_DERIVE_SHARED) == 0 ||
        strcmp(method, VERB_GET_PUBLIC_KEY) == 0 ||
        strcmp(method, VERB_DERIVE) == 0);
```

### 2. Enforcement

[`enforce_verb_algorithm()`](src/enforcement.c:765): `derive` is valid for
`secp256k1` only (the privkey is a 32-byte scalar suitable as an HMAC key; PQ
private keys are not). Add:

```c
/* derive: HMAC-SHA256(privkey, data). secp256k1 only (32-byte scalar key). */
if (strcmp(verb, VERB_DERIVE) == 0) {
    if (alg == CRYPTO_ALG_SECP256K1) {
        return ENFORCE_OK;
    }
    return ENFORCE_ERR_ALGORITHM;
}
```

### 3. Dispatcher handler

[`handle_algorithm_verb()`](src/dispatcher.c:880): add a new branch after the
existing `derive_shared_secret` branch. Request shape:

```json
{"id":"...","method":"derive","params":["<data>",{"algorithm":"secp256k1","index":0}]}
```

- `params[0]` = the data string (UTF-8).
- `options.algorithm` = `"secp256k1"` (required).
- `options.index` = derivation index (**required** — no default). The dispatcher
  returns error `-32602 missing_index` if `index` is absent.

Handler:

```c
/* ---- derive (secp256k1 HMAC-SHA256) ---- */
if (strcmp(method, VERB_DERIVE) == 0) {
    cJSON *data_item = cJSON_GetArrayItem(params_item, 0);
    cJSON *index_item = NULL;
    const char *data_str;
    const unsigned char *priv;
    unsigned char mac[32];
    char mac_hex[65];
    char *result;

    if (!cJSON_IsString(data_item) || data_item->valuestring == NULL) {
        return make_error_response(id_str, -32602, "invalid_params");
    }
    data_str = data_item->valuestring;

    /* index is required for derive (no default). */
    if (!cJSON_IsObject(options_item)) {
        return make_error_response(id_str, -32602, "missing_index");
    }
    index_item = cJSON_GetObjectItemCaseSensitive(options_item, "index");
    if (!cJSON_IsNumber(index_item)) {
        return make_error_response(id_str, -32602, "missing_index");
    }
    index = index_item->valueint;

    /* Derive the secp256k1 key on demand. */
    if (alg_key_cache_derive(ctx->alg_key_cache, ctx->mnemonic, alg, index) != 0) {
        return make_error_response(id_str, -32602, "key_derivation_failed");
    }
    key_entry = alg_key_cache_get(ctx->alg_key_cache, alg, index);
    if (key_entry == NULL || !key_entry->valid) {
        return make_error_response(id_str, -32602, "key_derivation_failed");
    }
    priv = (const unsigned char *)key_entry->private_key.data;

    /* HMAC-SHA256(privkey, data) */
    if (nostr_hmac_sha256(priv, sz->priv_key_len,
                           (const unsigned char *)data_str, strlen(data_str),
                           mac) != 0) {
        return make_error_response(id_str, -32602, "hmac_failed");
    }

    nostr_bytes_to_hex(mac, 32, mac_hex);
    secure_memzero(mac, sizeof(mac));

    result = build_alg_result_json(alg_name, key_entry->key_id,
                                     "digest", mac_hex);
    if (result == NULL) {
        return make_error_response(id_str, -32602, "invalid_params");
    }
    return make_success_response(id_str, result);
}
```

`nostr_hmac_sha256` is already available via `<nostr_core/utils.h>` (linked into
n_signer through nostr_core_lib). `secure_memzero` clears the stack-local mac.

Response:

```json
{"id":"...","result":"{\"algorithm\":\"secp256k1\",\"key_id\":\"<16hex>\",\"digest\":\"<64hex>\"}"}
```

### 4. Headerless decls

The `VERB_DERIVE` define must be added to the `NSIGNER_HEADERLESS_DECLS_BEGIN`
block in every file that carries it. The same block already exists in:
- [`src/dispatcher.c`](src/dispatcher.c:209)
- [`src/enforcement.c`](src/enforcement.c:210)
- [`src/key_store.c`](src/key_store.c:208)
- [`src/main.c`](src/main.c:215)
- [`src/policy.c`](src/policy.c:209)
- [`src/selector.c`](src/selector.c:209)
- [`src/server.c`](src/server.c:215)
- [`src/role_table.c`](src/role_table.c:212)
- [`src/mnemonic.c`](src/mnemonic.c:209)
- [`src/secure_mem.c`](src/secure_mem.c:211)
- [`src/socket_name.c`](src/socket_name.c:211)
- [`src/pq_crypto.c`](src/pq_crypto.c:222)
- [`tests/test_*.c`](tests/) (each test file that carries the block)

### 5. Tests

Add a test in [`tests/test_algorithm_api.c`](tests/test_algorithm_api.c) (or a
new `tests/test_derive.c`):

1. Load a known mnemonic, derive secp256k1 key at index 0.
2. Call `derive` with `data = "test-data"`.
3. Independently compute `HMAC-SHA256(privkey, "test-data")` using
   `nostr_hmac_sha256` and compare to the RPC result.
4. Assert determinism: same input → same digest.
5. Assert opaqueness: different inputs → different digests.
6. Assert rejection for non-secp256k1 algorithms (e.g. `algorithm: "ed25519"`
   → error 1010 `algorithm_not_supported_for_verb`).
7. Assert rejection when mnemonic is not loaded (error 1006).

### 6. Documentation

- [`README.md`](README.md) §4.3 verb table: add `derive` row.
- [`README.md`](README.md) §4.4 algorithms: note `derive` is secp256k1-only.
- [`api.md`](api.md): add worked example.
- [`client/README.md`](client/README.md): add `derive` to the verb table.
- [`documents/CLIENT_IMPLEMENTATION.md`](documents/CLIENT_IMPLEMENTATION.md):
  add `derive` to the algorithm-based verb list with example request/response.

## nostr_core_lib client changes

[`nostr_core_lib/nostr_core/nostr_signer.h`](../sovereign_browser/nostr_core_lib/nostr_core/nostr_signer.h)
and
[`nostr_core_lib/nostr_core/nostr_signer.c`](../sovereign_browser/nostr_core_lib/nostr_core/nostr_signer.c):

Add a new high-level API:

```c
/* Compute HMAC-SHA256(privkey, data) using the signer's derived secp256k1
 * private key. Returns the 32-byte digest as 64 lowercase hex chars + NUL.
 * For the local backend, computes directly. For the nsigner remote backend,
 * calls the "derive" verb.
 * data must be a NUL-terminated UTF-8 string.
 * Returns NOSTR_SUCCESS or an error code. */
int nostr_signer_derive_hmac(nostr_signer_t* signer,
                             const char* data,
                             char out_digest_hex[65]);
```

### Local backend

```c
static int signer_local_derive_hmac(nostr_signer_t* signer,
                                    const char* data,
                                    char out_digest_hex[65]) {
    unsigned char mac[32];
    if (!signer || !data || !out_digest_hex) {
        return NOSTR_ERROR_INVALID_INPUT;
    }
    if (nostr_hmac_sha256(signer->u.local.private_key, 32,
                           (const unsigned char*)data, strlen(data),
                           mac) != 0) {
        return NOSTR_ERROR_CRYPTO_FAILED;
    }
    nostr_bytes_to_hex(mac, 32, out_digest_hex);
    memset(mac, 0, sizeof(mac));
    return NOSTR_SUCCESS;
}
```

### Remote (nsigner) backend

```c
static int signer_remote_derive_hmac(nostr_signer_t* signer,
                                     const char* data,
                                     char out_digest_hex[65]) {
    cJSON* params;
    cJSON* result = NULL;
    const char* result_str;
    int rc;

    params = cJSON_CreateArray();
    if (params == NULL) return NOSTR_ERROR_MEMORY_FAILED;
    cJSON_AddItemToArray(params, cJSON_CreateString(data));

    params = signer_remote_params_with_selector(params,
              signer->u.remote.role,
              signer->u.remote.has_nostr_index,
              signer->u.remote.nostr_index);
    if (params == NULL) return NOSTR_ERROR_MEMORY_FAILED;

    /* The remote derive verb needs algorithm:"secp256k1" in the options. */
    {
        cJSON* opts = cJSON_GetArrayItem(params, cJSON_GetArraySize(params) - 1);
        if (opts != NULL && cJSON_IsObject(opts)) {
            cJSON_AddStringToObject(opts, "algorithm", "secp256k1");
        }
    }

    rc = nsigner_client_call(signer->u.remote.client, "derive", params, &result);
    if (rc != NOSTR_SUCCESS) return rc;

    /* result is a JSON string: {"algorithm":"secp256k1","key_id":"...","digest":"<64hex>"} */
    if (!cJSON_IsString(result) || result->valuestring == NULL) {
        cJSON_Delete(result);
        return NOSTR_ERROR_NIP46_INVALID_RESPONSE;
    }
    {
        cJSON* parsed = cJSON_Parse(result->valuestring);
        cJSON* digest_item;
        const char* digest_str;
        cJSON_Delete(result);
        if (parsed == NULL) return NOSTR_ERROR_NIP46_INVALID_RESPONSE;
        digest_item = cJSON_GetObjectItemCaseSensitive(parsed, "digest");
        if (!cJSON_IsString(digest_item) || digest_item->valuestring == NULL ||
            strlen(digest_item->valuestring) != 64) {
            cJSON_Delete(parsed);
            return NOSTR_ERROR_NIP46_INVALID_RESPONSE;
        }
        digest_str = digest_item->valuestring;
        memcpy(out_digest_hex, digest_str, 64);
        out_digest_hex[64] = '\0';
        cJSON_Delete(parsed);
    }
    return NOSTR_SUCCESS;
}
```

**Note on the options object:** the existing
`signer_remote_params_with_selector` appends a selector object as the last
params element. For algorithm-based verbs, the selector object must also carry
`algorithm` and `index`. The implementation must ensure the options object
already exists (or create one) before adding `algorithm`. If
`has_nostr_index` is false and `role` is empty, `signer_remote_params_with_selector`
does not append an options object — in that case the derive handler must append
one with just `algorithm`. This is a small extension to the helper or a local
construction in `signer_remote_derive_hmac`.

### Test

[`nostr_core_lib/tests/nsigner_client_test.c`](../sovereign_browser/nostr_core_lib/tests/nsigner_client_test.c):
add a test that calls `nostr_signer_derive_hmac` on a local signer with a known
privkey + known data, and compares against a reference HMAC-SHA256 computed
with `nostr_hmac_sha256` directly.

## sovereign_browser changes

### 1. Switch to single-step d tag

[`src/bookmarks.c`](../sovereign_browser/src/bookmarks.c:194): replace
`compute_hmac_key` + `path_to_d_tag` with a single function that calls the
signer:

```c
/* New label prefix — domain-separated, baked into the data string. */
#define BOOKMARKS_HMAC_KEY_LABEL "sovereign-browser/bookmarks-folder-id-v1"

static char *path_to_d_tag(const char *path) {
    if (path == NULL) path = "";

    /* Build "LABEL:" + path */
    char *data = g_strdup_printf("%s:%s", BOOKMARKS_HMAC_KEY_LABEL, path);
    char digest_hex[65];
    int rc;

    rc = nostr_signer_derive_hmac(g_signer, data, digest_hex);
    g_free(data);
    if (rc != NOSTR_SUCCESS) {
        /* No signer or derive failed — fall back to plaintext (read-only mode). */
        return g_strdup(path);
    }
    return g_strdup(digest_hex);
}
```

Remove `g_hmac_key`, `g_have_hmac_key`, `compute_hmac_key`, and the
`g_privkey` / `g_have_privkey` state (the privkey is no longer needed in
browser memory — the signer holds it). This is a **security improvement**: the
browser no longer carries the privkey in its own RAM when using the nsigner
backend.

### 2. Migration of existing two-step d tags

In `bookmarks_init` (or the relay-fetch load path), after decrypting an event
and reading its `path`:

1. Compute the new single-step d tag: `path_to_d_tag(path)`.
2. Compare to the event's actual `d` tag.
3. If they differ (old two-step tag), re-publish the event with the new d tag
   and emit a kind-5 deletion for the old event id.

This reuses the existing legacy-plaintext-d-tag migration logic. The detection
predicate changes from "d tag is not 64 hex chars" to "d tag is 64 hex chars
but does not match `path_to_d_tag(path)`".

### 3. Remove privkey plumbing

- [`src/main.c`](../sovereign_browser/src/main.c:69): remove `privkey_hex` from
  `g_state` (or stop populating it).
- [`src/main.c`](../sovereign_browser/src/main.c:652): remove the
  `strncpy(g_state.privkey_hex, ...)` line.
- [`src/bookmarks.h`](../sovereign_browser/src/bookmarks.h): change
  `bookmarks_init` signature to drop the `privkey_hex` parameter.
- All callers of `bookmarks_init` updated.

### 4. Test

[`tests/test_bookmarks_tree.c`](../sovereign_browser/tests/test_bookmarks_tree.c):
update `path_to_d_tag` mirror to the single-step scheme:

```c
static char *path_to_d_tag(const unsigned char *privkey, const char *path) {
    char *data = g_strdup_printf("%s:%s", LABEL, path ? path : "");
    unsigned char mac[32];
    char *hex;
    if (nostr_hmac_sha256(privkey, 32,
                           (const unsigned char*)data, strlen(data),
                           mac) != 0) {
        g_free(data);
        return NULL;
    }
    g_free(data);
    hex = g_strdup_printf(/* 64 hex chars */ ...);
    return hex;
}
```

All existing tests (determinism, opaqueness, 64-hex format, is_hmac_d_tag,
per-user-ness, empty path) pass unchanged — they test properties, not the
specific construction.

## Architecture diagram

```mermaid
sequenceDiagram
    participant Browser as sovereign_browser
    participant Signer as n_signer
    participant Relay as Nostr relay

    Note over Browser: User saves bookmark to "Work/Projects/Secret"
    Browser->>Signer: derive("sovereign-browser/bookmarks-folder-id-v1:Work/Projects/Secret", {algorithm:"secp256k1", index:0})
    Signer->>Signer: alg_key_cache_derive(secp256k1, 0)
    Signer->>Signer: HMAC-SHA256(privkey, data)
    Signer-->>Browser: {"digest":"<64hex>"}
    Browser->>Signer: nostr_nip44_encrypt(self_pubkey, JSON{path, bookmarks})
    Signer-->>Browser: ciphertext
    Browser->>Relay: publish kind 30003, d=<64hex>, content=<ciphertext>

    Note over Browser: Other device starts up
    Browser->>Relay: fetch kind 30003 by pubkey
    Relay-->>Browser: events
    Browser->>Signer: nostr_nip44_decrypt(self_pubkey, content)
    Signer-->>Browser: JSON{path:"Work/Projects/Secret", bookmarks:[...]}
    Note over Browser: path recovered from decrypted content, not from d tag
```

## File change summary

### n_signer (this repo)

| File | Change |
|------|--------|
| [`src/enforcement.c`](src/enforcement.c) | Add `VERB_DERIVE` define + `derive` case in `enforce_verb_algorithm` (secp256k1 only) |
| [`src/dispatcher.c`](src/dispatcher.c) | Add `VERB_DERIVE` to `is_algorithm_verb` + `derive` branch in `handle_algorithm_verb` |
| [`src/key_store.c`](src/key_store.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/main.c`](src/main.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/policy.c`](src/policy.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/selector.c`](src/selector.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/server.c`](src/server.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/role_table.c`](src/role_table.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/mnemonic.c`](src/mnemonic.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/secure_mem.c`](src/secure_mem.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/socket_name.c`](src/socket_name.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`src/pq_crypto.c`](src/pq_crypto.c) | Add `VERB_DERIVE` define (headerless decls) |
| [`tests/test_algorithm_api.c`](tests/test_algorithm_api.c) | Add `derive` test cases |
| [`README.md`](README.md) | Add `derive` to verb table + algorithm notes |
| [`api.md`](api.md) | Add `derive` worked example |
| [`client/README.md`](client/README.md) | Add `derive` to verb table |
| [`documents/CLIENT_IMPLEMENTATION.md`](documents/CLIENT_IMPLEMENTATION.md) | Add `derive` section |

### nostr_core_lib (sovereign_browser repo)

| File | Change |
|------|--------|
| [`nostr_core/nostr_signer.h`](../sovereign_browser/nostr_core_lib/nostr_core/nostr_signer.h) | Add `nostr_signer_derive_hmac` declaration |
| [`nostr_core/nostr_signer.c`](../sovereign_browser/nostr_core_lib/nostr_core/nostr_signer.c) | Add local + remote `derive_hmac` implementations |
| [`tests/nsigner_client_test.c`](../sovereign_browser/nostr_core_lib/tests/nsigner_client_test.c) | Add `derive_hmac` test |

### sovereign_browser

| File | Change |
|------|--------|
| [`src/bookmarks.c`](../sovereign_browser/src/bookmarks.c) | Replace two-step `compute_hmac_key` + `path_to_d_tag` with single-step `path_to_d_tag` calling `nostr_signer_derive_hmac`; add two-step→single-step migration on load; remove `g_privkey`/`g_hmac_key` state |
| [`src/bookmarks.h`](../sovereign_browser/src/bookmarks.h) | Drop `privkey_hex` param from `bookmarks_init` |
| [`src/main.c`](../sovereign_browser/src/main.c) | Remove `privkey_hex` from `g_state` + stop populating it; update `bookmarks_init` call |
| [`tests/test_bookmarks_tree.c`](../sovereign_browser/tests/test_bookmarks_tree.c) | Update `path_to_d_tag` mirror to single-step scheme |

## Open questions

1. **Should `derive` accept hex-encoded binary data, or only UTF-8 strings?**
   Default: UTF-8 strings only (simpler, covers the bookmark use case). If a
   caller needs binary data, they hex-encode it and we add a `data_encoding`
   option later. Decision can be deferred.

2. **`index` is required.** Unlike other algorithm-based verbs that default
   `index` to 0, `derive` requires the caller to specify `index` explicitly.
   This forces conscious selection of which derived key to use as the HMAC key,
   avoiding accidental cross-identity d-tag collisions. The dispatcher returns
   `-32602 missing_index` if `index` is absent.

3. **Should `derive` be added to the `--preapprove` algorithm-based policy
   syntax?**
   Yes — it flows through the same `policy_check_algorithm` path as other
   algorithm-based verbs. Operators can pre-approve
   `caller=...,algorithm=secp256k1,index=0,verb=derive`. No code change needed
   beyond the verb define; policy matching is by string.