141 lines
11 KiB
Markdown
141 lines
11 KiB
Markdown
# Multi-Threaded PoW Offloading Plan
|
|
|
|
## Problem Statement
|
|
Currently, [`nostr_mine_event`](src/dispatcher.rs:509) executes synchronously inside [`ServerContext::finish_request()`](src/server.rs:351) via [`KeyStore::mine_event()`](src/key_store.rs:202) and [`miner::miner_run()`](src/miner.rs:41). Although [`miner::miner_run()`](src/miner.rs:41) spawns worker threads internally, it blocks the calling thread (`handle.join()`) until the target difficulty is reached or the timeout (up to 600s) expires.
|
|
|
|
Because [`ServerContext`](src/server.rs:139), the TUI loop ([`src/tui.rs`](src/tui.rs:568)), the GUI loop ([`src/gui.rs`](src/gui.rs:572)), and the headless loop ([`src/main.rs`](src/main.rs:309)) run on a single thread, a 5-minute PoW mining request freezes the UI and blocks all other incoming signing requests until mining finishes.
|
|
|
|
## Design Goals
|
|
1. **Non-blocking PoW Dispatch (Option 1 — Targeted Background Offloading)**:
|
|
- Keep [`MnemonicState`](src/mnemonic.rs:1), [`RoleTable`](src/role_table.rs:1), [`KeyStore`](src/key_store.rs:24), and [`AlgorithmKeyCache`](src/alg_cache.rs:25) owned by the main thread without `Arc<Mutex<...>>` lock contention or shared secret lifetimes.
|
|
- Validate the `nostr_mine_event` request, verify any auth envelope, resolve the role selector, and ensure the role's key is derived on the main thread (microseconds).
|
|
- Extract a temporary 32-byte stack copy of the role's private key (wrapped in `zeroize::Zeroizing<[u8; 32]>`), take ownership of the client connection's [`ConnWriter`](src/server.rs:101), and spawn a background coordinator thread for the mining run.
|
|
2. **Multiple Concurrent PoW Jobs**:
|
|
- Any number of `nostr_mine_event` requests may run concurrently alongside fast signing requests (`nostr_sign_event`, `sign`, `nip44_encrypt`, `get_public_key`, etc.).
|
|
- On each poll tick, the server drains all pending incoming requests and drains any completed background mining activity messages via an `mpsc` channel.
|
|
3. **Immediate Cancellation on Stop / Seed Unload**:
|
|
- `ServerContext` holds a shared cancellation token (`Arc<AtomicBool>`) and an active job counter (`Arc<AtomicUsize>`).
|
|
- Calling [`ServerContext::stop()`](src/server.rs:204) (which is invoked on server stop, transport restart, role re-derivation/removal, or seed unload/quit in TUI, GUI, and headless modes) sets the cancellation flag to `true`, immediately signaling all in-flight [`miner::mine_worker()`](src/miner.rs:138) threads to abort, zeroize their temporary key copy, and exit cleanly.
|
|
4. **Socket Blocking Mode Fix for Background Responses**:
|
|
- Accepted sockets in [`ServerContext::poll_request()`](src/server.rs:236) are placed in `set_nonblocking(true)` for the initial probe/read. Before [`ConnWriter::send_response()`](src/server.rs:107) writes the response (especially from a background thread or for large payloads), ensure the stream is set back to blocking mode (or write timeout) so writing the response never fails silently with `WouldBlock`.
|
|
|
|
## Architecture Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant ClientA as Client A - PoW Mine
|
|
participant ClientB as Client B - Fast Sign
|
|
participant MainLoop as Main Thread - TUI / GUI / Headless
|
|
participant Server as ServerContext + Dispatcher
|
|
participant BgThread as Background Mine Coordinator
|
|
participant Workers as PoW Worker Threads 1..N
|
|
|
|
ClientA->>Server: nostr_mine_event request
|
|
MainLoop->>Server: poll_request + finish_request
|
|
Server->>Server: Verify auth, resolve role, derive key if needed
|
|
Server->>Server: Pre-validate event JSON and termination condition
|
|
Server-->>MainLoop: Return START activity immediately
|
|
Server->>BgThread: Spawn with ConnWriter, Zeroizing key, cancel_flag, activity_tx
|
|
BgThread->>Workers: Spawn N stride workers with cancel_flag
|
|
|
|
ClientB->>Server: nostr_sign_event request
|
|
MainLoop->>Server: poll_request + finish_request
|
|
Server->>ClientB: Signed event response in <1ms
|
|
Server-->>MainLoop: Return sign activity line
|
|
|
|
Workers-->>BgThread: Target reached or timeout or cancelled
|
|
BgThread->>BgThread: Sign best event and zeroize key copy
|
|
BgThread->>ClientA: Send JSON-RPC response via ConnWriter
|
|
BgThread->>Server: Send DONE / TIMEOUT / CANCELLED activity via activity_tx
|
|
MainLoop->>Server: poll_background_activities
|
|
Server-->>MainLoop: Drain completion activity lines to ActivityLog
|
|
```
|
|
|
|
## Detailed Component Changes
|
|
|
|
### 1. Cancellable Miner (`src/miner.rs`)
|
|
- Add `pub cancelled: bool` to [`MineResult`](src/miner.rs:16).
|
|
- Add [`miner_run_cancellable()`](src/miner.rs:41) (and keep [`miner_run()`](src/miner.rs:41) as a thin wrapper passing `None` for backwards compatibility with existing unit tests):
|
|
```rust
|
|
pub fn miner_run_cancellable(
|
|
event_json: &str,
|
|
private_key: &[u8; 32],
|
|
target_difficulty: i32,
|
|
thread_count: i32,
|
|
timeout_sec: u64,
|
|
cancel: Option<&Arc<AtomicBool>>,
|
|
) -> Result<MineResult, crate::SignerError>
|
|
```
|
|
- Inside [`mine_worker()`](src/miner.rs:138):
|
|
- Check the external `cancel` `AtomicBool` (with `Ordering::Relaxed`) every 1,024 iterations (alongside the shared `stop` check) so workers exit within microseconds when cancelled without lock contention on every single hash.
|
|
- Also replace the per-iteration `shared.lock()` check on line 158 with an `AtomicBool` (`stop_flag: Arc<AtomicBool>`) so worker threads do not contend on a `Mutex` on every single SHA-256 hash attempt! (This will also significantly speed up multi-threaded PoW hash rates).
|
|
- If cancelled via the external `cancel` flag, `miner_run_cancellable` sets `cancelled: true` on `MineResult` (or skips signing if shutting down).
|
|
|
|
### 2. Background Job Preparation in Dispatcher / KeyStore (`src/dispatcher.rs` & `src/key_store.rs`)
|
|
- Add a prepared mining request struct in `src/dispatcher.rs`:
|
|
```rust
|
|
pub struct PreparedMineJob {
|
|
pub rpc_id: String,
|
|
pub role_name: String,
|
|
pub event_json: String,
|
|
pub private_key: zeroize::Zeroizing<[u8; 32]>,
|
|
pub difficulty: i32,
|
|
pub threads: i32,
|
|
pub timeout_sec: u64,
|
|
}
|
|
```
|
|
- When `handle_nostr_verb` in [`src/dispatcher.rs`](src/dispatcher.rs:379) handles `VERB_NOSTR_MINE_EVENT`:
|
|
- Validate `params[0]` (must be valid unsigned `Event` JSON), `difficulty` / `timeout_sec` termination condition, role selector, purpose enforcement, and key derivation synchronously.
|
|
- If validation fails, return the immediate JSON-RPC error response as before.
|
|
- When invoked from `ServerContext` (which supports background offloading), populate `ctx.pending_mine_job: Option<PreparedMineJob>` with the `Zeroizing<[u8; 32]>` key copy and validated parameters instead of blocking on `key_store.mine_event()`.
|
|
- Keep synchronous execution in `handle_request()` when `ctx` is used directly in unit tests (or provide a flag `offload_mining: bool` on `DispatcherContext`).
|
|
|
|
### 3. Background Spawning & Activity Channel in `ServerContext` (`src/server.rs`)
|
|
- Extend [`ServerContext`](src/server.rs:139) with:
|
|
- `activity_tx: std::sync::mpsc::Sender<String>`
|
|
- `activity_rx: std::sync::mpsc::Receiver<String>`
|
|
- `cancel_flag: Arc<AtomicBool>`
|
|
- `active_jobs: Arc<AtomicUsize>`
|
|
- In [`ConnWriter::send_response()`](src/server.rs:107):
|
|
- Set the underlying `UnixStream` or `TcpStream` back to blocking (`set_nonblocking(false)`) before writing the response so the response write never fails with `WouldBlock`.
|
|
- In [`ServerContext::finish_request()`](src/server.rs:351):
|
|
- Set `dispatcher.offload_mining = true` (except in `ListenMode::Stdio | ListenMode::Qrexec` one-shot mode where the process exits after one request, or wait for active jobs before exiting).
|
|
- If `process_request` yields a `PreparedMineJob`:
|
|
- Return the `START` activity line (or `Option<String>` if already logged by two-phase caller) immediately.
|
|
- Increment `active_jobs`.
|
|
- Spawn a background thread owning `writer: ConnWriter`, `job: PreparedMineJob`, `caller_id: String`, `cancel_flag: Arc<AtomicBool>`, `active_jobs: Arc<AtomicUsize>`, and `activity_tx: Sender<String>`.
|
|
- The background thread runs `miner::miner_run_cancellable(...)`:
|
|
- Drops/zeroizes `job.private_key` immediately after `miner_run_cancellable` returns.
|
|
- If not cancelled, formats the JSON-RPC response, calls `writer.send_response(&response)`, formats the completion activity via `mine_done_activity()`, and sends it over `activity_tx`.
|
|
- If cancelled, sends a cancellation error response (best-effort) and sends a `CANCELLED` activity line over `activity_tx`.
|
|
- Decrements `active_jobs`.
|
|
- Add [`ServerContext::poll_activity(&mut self) -> Option<String>`](src/server.rs:210) (and `drain_activities(&mut self) -> Vec<String>`) to read completed background job messages from `activity_rx.try_recv()`.
|
|
- Add [`ServerContext::active_mining_count(&self) -> usize`](src/server.rs:210) so callers/tests/UI can inspect how many background jobs are currently in flight.
|
|
- Update [`ServerContext::stop(&mut self)`](src/server.rs:204):
|
|
- Set `self.cancel_flag.store(true, Ordering::SeqCst)` to abort all in-flight mining workers immediately, then reset `self.cancel_flag = Arc::new(AtomicBool::new(false))` for any future restart.
|
|
- Update [`ServerContext::handle_one()`](src/server.rs:219):
|
|
- If a pending socket request is available, handle it (for backgrounded mining, return the `START` activity line immediately).
|
|
- Otherwise, check `self.poll_activity()` and return `Ok(Some(bg_activity))` if a background mining job finished.
|
|
|
|
### 4. TUI, GUI, and Headless Loop Integration (`src/tui.rs`, `src/gui.rs`, `src/main.rs`)
|
|
- **TUI (`src/tui.rs`)**:
|
|
- In [`App::service_server()`](src/tui.rs:568):
|
|
- First drain `server.drain_activities()` and append each line to `self.activity_log`.
|
|
- Process any pending socket requests in a loop (or one per tick) without blocking on `nostr_mine_event`: when `nostr_mine_event` is dispatched in `finish_request`, it spawns the background thread and returns immediately, so `START` is logged once and `DONE`/`TIMEOUT` arrives later via `drain_activities()`.
|
|
- **GUI (`src/gui.rs`)**:
|
|
- In [`GuiApp::service_server()`](src/gui.rs:572):
|
|
- Drain `server.drain_activities()` into `self.activity_log` and request repaint when activities arrive or while `server.active_mining_count() > 0`.
|
|
- Handle incoming requests non-blockingly.
|
|
- **Headless (`src/main.rs`)**:
|
|
- In [`run_headless()`](src/main.rs:290):
|
|
- For `Tcp` / `Http` / `Unix` loop: drain and `println!` completed activities from `server.drain_activities()` on every iteration, and process incoming requests non-blockingly.
|
|
- For `Stdio` / `Qrexec` single-request mode: wait for any spawned job to finish (or run synchronously) before exiting the process.
|
|
|
|
### 5. Concurrency & Cancellation Integration Tests (`tests/integration.rs`)
|
|
- Add `test_concurrent_signing_during_pow_mining`:
|
|
- Start a `nostr_mine_event` request on Client Thread A with a high difficulty (e.g., `difficulty: 28`, `timeout_sec: 5`) that will run for several seconds.
|
|
- While Client Thread A is actively mining, send multiple `nostr_sign_event`, `nostr_get_public_key`, and `sign` (ed25519) requests from Client Thread B.
|
|
- Assert that all of Client Thread B's requests complete in `< 200ms` while Client Thread A is still mining.
|
|
- Assert that multiple concurrent `nostr_mine_event` requests can also run simultaneously.
|
|
- Assert that calling `server.stop()` cancels the in-flight mining job immediately without waiting for the full timeout.
|