Files
n_signer/plans/mine_event_pow.md

431 lines
19 KiB
Markdown

# Plan: Add `mine_event` Verb (NIP-13 Proof-of-Work) to n_signer
## Executive Summary
**Recommendation: Do NOT import event_miner as a subrepo or copy its code.**
n_signer already has [`nip013.h`](resources/nostr_core_lib/nostr_core/nip013.h:1) and [`nip013.c`](resources/nostr_core_lib/nostr_core/nip013.c:1) in its `resources/nostr_core_lib/`. event_miner is a standalone CLI tool (~716 lines) whose useful logic is ~50 lines. The rest is CLI parsing, stdin/file I/O, signal handlers, and `exit()` calls that are inappropriate for a long-running server.
Instead, we add a new `mine_event` verb to n_signer's existing dispatcher that:
1. Mines PoW for a time budget (e.g., "mine for 30 seconds with 4 threads")
2. Returns the **best result found** within that time — regardless of whether the target difficulty was reached
3. Signs the event with the role's derived private key
4. Returns the mined + signed event JSON along with metadata about the achieved difficulty
## Why Not Subrepo or Copy?
| Factor | Subrepo | Copy | New verb (recommended) |
|--------|---------|------|----------------------|
| Key management | event_miner takes raw nsec CLI arg | same | Uses n_signer's role table + key_store (secure, mlock'd) |
| Architecture | CLI tool, calls exit() | same | Fits JSON-RPC dispatcher/enforcement/policy model |
| Threading | Global vars, signal handlers, exit() | same | Clean detached thread per request, no globals |
| Best-effort model | No — stops at target or max_attempts | same | Yes — returns best result within time budget |
| Code reuse | ~50 lines useful | same | Rewrite ~150 lines for server-safe best-effort model |
| Maintenance | Extra submodule to track | Drift risk | Single codebase, single nostr_core_lib |
## Design Decisions
- **Verb name**: `mine_event`
- **Params**: `[event_json, {difficulty: N, threads: N, timeout_sec: N, ...role_selector}]`
- **Dual termination model**: Both `timeout_sec` and `difficulty` are independent options. Either, both, or at least one must be specified:
- **Both set**: Mine until target reached OR timeout — return best result (with `target_reached` flag)
- **Only `timeout_sec`**: Mine for the full duration, return best result found
- **Only `difficulty`**: Mine until target reached, with a safety max timeout (e.g., 10 minutes) to prevent infinite mining
- **Neither**: Error — must specify at least one termination condition
- **Best-effort return**: The response always includes the best event found, plus metadata (`achieved_difficulty`, `target_difficulty`, `target_reached`, `elapsed_sec`, `attempts`)
- **Threading**: Spawn a detached pthread for mining. Server stays responsive. Mining thread writes result to client socket when done.
- **Key source**: Uses the role's derived private key from `key_store` (same as `sign_event`)
## Why Our Own Mining Loop (Not `nostr_add_proof_of_work()`)
The existing [`nostr_add_proof_of_work()`](resources/nostr_core_lib/nostr_core/nip013.c:121) has two problems for the best-effort model:
1. **Discards work on failure**: It returns `NOSTR_ERROR_CRYPTO_FAILED` if `max_attempts` is exceeded without reaching the target. The best nonce found is lost.
2. **No time-based termination**: It uses `max_attempts` (a count), not a time deadline. For a "mine for 30 seconds" model, we need time-based termination.
Instead, `miner.c` implements its own mining loop that:
- Iterates nonces across multiple threads
- Uses [`nostr_create_and_sign_event()`](resources/nostr_core_lib/nostr_core/nip001.h:1) from nip001 to sign each attempt
- Uses [`nostr_calculate_pow_difficulty()`](resources/nostr_core_lib/nostr_core/nip013.h:42) from nip013 to check difficulty
- Tracks the best event (highest difficulty) across all threads
- Stops when target reached OR timeout
- Returns the best event with metadata
This reuses the library's crypto/JSON primitives without modifying the shared library.
## Request/Response Format
### Request
```json
{
"id": "req-1",
"method": "mine_event",
"params": [
"{\"kind\":1,\"content\":\"Hello Nostr!\",\"tags\":[],\"created_at\":1723666800}",
{
"difficulty": 20,
"threads": 4,
"timeout_sec": 30,
"role": "main"
}
]
}
```
- `difficulty` (optional): Target leading zero bits. If reached, mining stops early. If not specified, mining runs until timeout.
- `threads` (optional, default: 1): Number of mining threads.
- `timeout_sec` (optional): How long to mine in seconds. If not specified, mining runs until target difficulty is reached (with a safety max of 10 minutes).
- **At least one of `difficulty` or `timeout_sec` must be specified.** If neither is provided, returns error `1007` (`mining_failed` / `no_termination_condition`).
- **If only `difficulty` is specified**: A safety max timeout of 600 seconds (10 min) is applied to prevent infinite mining.
- **If only `timeout_sec` is specified**: Mining runs the full duration and returns the best result found.
- **If both are specified**: Mining stops when either condition is met (target reached OR timeout elapsed).
### Success Response (best-effort, always returned if mining ran)
```json
{
"id": "req-1",
"result": {
"event": "{\"kind\":1,\"content\":\"Hello Nostr!\",\"pubkey\":\"...\",\"id\":\"00000...\",\"sig\":\"...\",\"tags\":[[\"nonce\",\"12345\",\"20\"]],\"created_at\":1755197090}",
"achieved_difficulty": 18,
"target_difficulty": 20,
"target_reached": false,
"elapsed_sec": 30,
"attempts": 4523456
}
}
```
The `result` is now a JSON **object** (not a string like other verbs) containing the signed event plus mining metadata. The `event` field within it is the signed event JSON string. This is a departure from the other verbs that return a plain string result, but the metadata is essential for the client to know whether the target was met.
### Error Responses
- `1007` - `mining_failed` — internal error (invalid event, bad params, crypto failure)
- `1008` - `mining_busy` — another mining operation is already running (optional: reject or queue)
- Existing error codes (1001-1006, -326xx) apply as usual for policy/selector/enforcement errors
Note: Timeout is NOT an error — it's the normal termination condition. The best result is always returned.
## Architecture Diagram
```mermaid
flowchart TD
Client -->|JSON-RPC mine_event| ServerHandleOne
ServerHandleOne -->|policy_check and enforce_verb_role| PolicyEnforcement
PolicyEnforcement -->|ALLOWED| SpawnDetachedThread
ServerHandleOne -->|returns immediately| ServerLoop
SpawnDetachedThread --> MinerCoordinator
MinerCoordinator -->|spawn N worker threads| Worker1
MinerCoordinator -->|spawn N worker threads| Worker2
MinerCoordinator -->|spawn N worker threads| WorkerN
Worker1 -->|nostr_create_and_sign_event| NIP001Lib
Worker2 -->|nostr_create_and_sign_event| NIP001Lib
WorkerN -->|nostr_create_and_sign_event| NIP001Lib
NIP001Lib -->|signed event with nonce| CalcDifficulty
CalcDifficulty -->|nostr_calculate_pow_difficulty| NIP013Lib
NIP013Lib -->|difficulty bits| TrackBest
TrackBest -->|mutex protected| BestEvent
MinerCoordinator -->|timeout or target reached| JoinThreads
JoinThreads -->|best event + metadata| BuildResponse
BuildResponse -->|transport_send_framed| Client
```
## Implementation Steps
### Step 1: Add nip013 to nostr_core_lib build
The Makefile's `lib` target runs:
```
cd resources/nostr_core_lib && ./build.sh --nips=1,4,6,19,44
```
Add `13` to the `--nips` flag so nip013.c is compiled into the static library.
**Files to modify:**
- [`Makefile`](Makefile:54) — change `--nips=1,4,6,19,44` to `--nips=1,4,6,13,19,44`
### Step 2: Add VERB_MINE_EVENT to enforcement.c
Add `mine_event` to the known nostr verbs so it passes enforcement.
**Files to modify:**
- [`src/enforcement.c`](src/enforcement.c:200) — add `#define VERB_MINE_EVENT "mine_event"` and add it to [`is_nostr_verb()`](src/enforcement.c:453)
- [`src/dispatcher.c`](src/dispatcher.c:200) — add the same `#define VERB_MINE_EVENT "mine_event"` (the headerless decls pattern means defines are repeated per-file)
### Step 3: Create src/miner.c — Multithreaded Best-Effort Mining Coordinator
This is the core new file. It implements a server-safe, best-effort mining loop.
**Key design:**
- No global variables (all state in context structs)
- No `exit()` calls (return result codes)
- No signal handlers (server handles signals)
- Time-based termination (not attempt-count-based)
- Tracks best event across all threads
- Returns best event + metadata regardless of whether target was reached
**Key structures:**
```c
typedef struct {
cJSON *best_event; /* best event found so far (mutex-protected) */
int best_difficulty; /* difficulty of best_event */
uint64_t total_attempts; /* total attempts across all threads */
int target_difficulty; /* 0 = no target, mine full timeout */
int target_reached; /* 1 if target was reached */
pthread_mutex_t mutex; /* protects best_event, best_difficulty, total_attempts */
volatile int stop; /* set by coordinator when target reached or timeout */
} mine_shared_state_t;
typedef struct {
mine_shared_state_t *shared;
cJSON *event_template; /* this thread's copy of the event */
unsigned char private_key[32];
int thread_id;
uint64_t nonce_start; /* starting nonce for this thread (thread_id * stride) */
uint64_t nonce_stride; /* increment per iteration to avoid overlap */
time_t deadline; /* absolute time to stop */
uint64_t attempts; /* this thread's attempt count */
} miner_worker_ctx_t;
typedef struct {
cJSON *best_event; /* caller frees */
int achieved_difficulty;
int target_difficulty;
int target_reached;
int elapsed_sec;
uint64_t total_attempts;
} mine_result_t;
```
**Main function:**
```c
/* Returns 0 on success (result populated with best event found), -1 on error */
int miner_run(cJSON *event, const unsigned char *private_key,
int target_difficulty, int thread_count, int timeout_sec,
mine_result_t *result);
```
**Mining loop (per thread):**
```c
while (!shared->stop && time(NULL) < deadline) {
/* Build event with current nonce */
cJSON *working_tags = cJSON_Duplicate(original_tags, 1);
update_nonce_tag(working_tags, nonce, target_difficulty);
cJSON *signed_event = nostr_create_and_sign_event(kind, content, working_tags,
private_key, timestamp);
cJSON_Delete(working_tags);
/* Check difficulty */
const char *id = cJSON_GetStringValue(cJSON_GetObjectItem(signed_event, "id"));
int difficulty = nostr_calculate_pow_difficulty(id);
/* Track best under mutex */
pthread_mutex_lock(&shared->mutex);
shared->total_attempts++;
if (difficulty > shared->best_difficulty) {
if (shared->best_event) cJSON_Delete(shared->best_event);
shared->best_event = cJSON_Duplicate(signed_event, 1);
shared->best_difficulty = difficulty;
}
if (target_difficulty > 0 && difficulty >= target_difficulty) {
shared->target_reached = 1;
shared->stop = 1;
}
pthread_mutex_unlock(&shared->mutex);
cJSON_Delete(signed_event);
nonce += nonce_stride;
attempts++;
}
```
**Files to create:**
- `src/miner.c` — mining coordinator (~250 lines)
### Step 4: Add crypto_mine_event() to key_store.c
Add a function that:
1. Parses the event JSON
2. Gets the role's private key from key_store
3. Calls `miner_run()` from miner.c
4. Builds the response JSON object (event string + metadata)
5. Zeroizes the private key copy
6. Returns the response JSON string (caller frees)
**Signature:**
```c
/* Returns newly-allocated JSON string with result object, or NULL on error.
* Caller frees. */
char *crypto_mine_event(const key_store_t *store, int role_index,
const char *event_json, int difficulty,
int threads, int timeout_sec);
```
**Response building:**
```c
cJSON *result_obj = cJSON_CreateObject();
cJSON *event_item = cJSON_CreateString(signed_event_json);
cJSON_AddItemToObject(result_obj, "event", event_item);
cJSON_AddNumberToObject(result_obj, "achieved_difficulty", result.achieved_difficulty);
cJSON_AddNumberToObject(result_obj, "target_difficulty", result.target_difficulty);
cJSON_AddBoolToObject(result_obj, "target_reached", result.target_reached);
cJSON_AddNumberToObject(result_obj, "elapsed_sec", result.elapsed_sec);
cJSON_AddNumberToObject(result_obj, "attempts", (double)result.total_attempts);
char *out = cJSON_PrintUnformatted(result_obj);
cJSON_Delete(result_obj);
return out;
```
**Files to modify:**
- [`src/key_store.c`](src/key_store.c:456) — add `#include <nostr_core/nip013.h>`, add `crypto_mine_event()` function, add forward declaration in the headerless decls section
### Step 5: Update Makefile
Add `miner.c` to SOURCES. pthread is already linked (`-lpthread` in LDFLAGS).
**Files to modify:**
- [`Makefile`](Makefile:13) — add `$(SRC_DIR)/miner.c \` to SOURCES list
### Step 6: Add dispatcher handler for mine_event
In [`dispatcher_handle_request()`](src/dispatcher.c:545), add a branch for `VERB_MINE_EVENT` that:
1. Extracts `event_json` from params[0]
2. Extracts `difficulty`, `threads`, `timeout_sec` from the options object (params[last])
3. Validates: at least one of `difficulty` or `timeout_sec` must be specified (else error `1007`); `threads` defaults to 1 (max 32); if only `difficulty` is set, safety timeout of 600 sec is applied
4. Calls `crypto_mine_event()`
5. Returns the result
**Files to modify:**
- [`src/dispatcher.c`](src/dispatcher.c:675) — add `else if (strcmp(method, VERB_MINE_EVENT) == 0)` branch after the `VERB_SIGN_EVENT` branch
### Step 7: Modify server.c for async mining
The current [`server_handle_one()`](src/server.c:1453) is synchronous. For `mine_event`, we need to:
1. Detect `mine_event` method after policy check passes
2. Spawn a detached thread that:
- Calls `dispatcher_handle_request()` (which calls `crypto_mine_event()`)
- Sends the response via `transport_send_framed()` on the client_fd
- Closes the client_fd
3. Return immediately from `server_handle_one()` without closing the fd (the thread owns it now)
**Thread function:**
```c
typedef struct {
int client_fd;
dispatcher_ctx_t *dispatcher;
char *request;
} mine_thread_arg_t;
static void *mine_event_thread(void *arg) {
mine_thread_arg_t *a = (mine_thread_arg_t *)arg;
char *response = dispatcher_handle_request(a->dispatcher, a->request);
if (response != NULL) {
(void)transport_send_framed(a->client_fd, response);
free(response);
}
free(a->request);
close(a->client_fd);
free(a);
return NULL;
}
```
**In server_handle_one**, after `pchk == POLICY_ALLOW` and before calling `dispatcher_handle_request()`:
```c
if (pchk == POLICY_ALLOW && strcmp(method, "mine_event") == 0) {
pthread_t tid;
mine_thread_arg_t *arg = malloc(sizeof(*arg));
if (arg == NULL) {
/* fall through to error path */
} else {
arg->client_fd = client_fd;
arg->dispatcher = ctx->dispatcher;
arg->request = request; /* transfer ownership */
request = NULL; /* prevent free in cleanup */
if (pthread_create(&tid, NULL, mine_event_thread, arg) == 0) {
pthread_detach(tid);
/* Skip synchronous response path — thread owns fd and request */
/* Log activity and return without closing fd */
verdict = "ALLOWED";
source_label = "async-mine";
/* ... log activity ... */
return 1;
}
/* pthread_create failed — fall through to error path */
free(arg->request);
free(arg);
}
}
```
**Files to modify:**
- [`src/server.c`](src/server.c:1755) — add mine_event async path before the synchronous `dispatcher_handle_request()` call
### Step 8: Add client demos
Add `mine_event` examples to the existing demo files.
**Files to modify:**
- [`client/demo_javascript.js`](client/demo_javascript.js:1) — add mine_event example
- [`client/demo_python.py`](client/demo_python.py:1) — add mine_event example
- [`client/demo_c99.c`](client/demo_c99.c:1) — add mine_event example
### Step 9: Add tests
Create a test that:
1. Starts nsigner with a test mnemonic
2. Sends a `mine_event` request with low difficulty (e.g., 2) and short timeout (e.g., 5 sec)
3. Verifies the response contains a signed event with a nonce tag
4. Verifies the `achieved_difficulty` >= 2 and `target_reached` is true
5. Sends a `mine_event` request with high difficulty (e.g., 30) and short timeout (e.g., 3 sec)
6. Verifies the response contains a signed event, `target_reached` is false, and `achieved_difficulty` < 30
**Files to create:**
- `tests/test_mine_event.c`
**Files to modify:**
- [`Makefile`](Makefile:37) add `TEST_MINE_EVENT_TARGET` and `test-mine-event` target
### Step 10: Update documentation
**Files to modify:**
- [`README.md`](README.md:1) add `mine_event` to the verbs table with params description
- [`client/README.md`](client/README.md:1) add mine_event usage example
## Security Considerations
1. **Private key never leaves key_store**: The mining thread receives a copy of the 32-byte private key, which is zeroized after use (following the pattern in [`crypto_sign_event()`](src/key_store.c:456))
2. **Policy enforcement applies**: `mine_event` goes through the same `policy_check()` and `enforce_verb_role()` as all other verbs
3. **Thread safety**:
- Each mining thread works on its own cJSON copies
- The shared state (best_event, best_difficulty, total_attempts) is protected by a mutex
- secp256k1 context is thread-safe for signing
4. **Resource limits**:
- Max threads capped at 32 to prevent resource exhaustion
- `timeout_sec` is required (must be > 0) to prevent infinite mining
- Optional: limit to 1 concurrent mining operation (reject with `mining_busy` if already running)
5. **Client fd ownership**: The detached thread owns the client_fd and is responsible for closing it. The main server loop must not close it.
## Concurrency Considerations
- Each mining thread uses a unique nonce stride (thread_id) to avoid duplicate work
- cJSON operations are NOT thread-safe across different cJSON objects, but each thread works on its own copies
- The result mutex protects only the shared best-event tracking
- `nostr_create_and_sign_event()` creates a new secp256k1 context per call (or uses a thread-local one) — need to verify this is thread-safe. If not, each thread may need its own context.
## File Summary
| File | Action | Description |
|------|--------|-------------|
| `Makefile` | Modify | Add nip013 to --nips, add miner.c to SOURCES, add test target |
| `src/enforcement.c` | Modify | Add VERB_MINE_EVENT to is_nostr_verb() |
| `src/dispatcher.c` | Modify | Add VERB_MINE_EVENT define + handler branch |
| `src/key_store.c` | Modify | Add crypto_mine_event() function + nip013 include |
| `src/miner.c` | Create | Multithreaded best-effort mining coordinator |
| `src/server.c` | Modify | Add async mine_event path with detached thread |
| `tests/test_mine_event.c` | Create | Integration test for mine_event |
| `client/demo_javascript.js` | Modify | Add mine_event example |
| `client/demo_python.py` | Modify | Add mine_event example |
| `client/demo_c99.c` | Modify | Add mine_event example |
| `README.md` | Modify | Document mine_event verb |
| `client/README.md` | Modify | Add mine_event usage |