signer/plans/multithreaded_signer_plan.md

Multi-Threaded PoW Offloading Plan

Problem Statement

Currently, nostr_mine_event executes synchronously inside ServerContext::finish_request() via KeyStore::mine_event() and miner::miner_run(). Although miner::miner_run() spawns worker threads internally, it blocks the calling thread (handle.join()) until the target difficulty is reached or the timeout (up to 600s) expires.

Because ServerContext, the TUI loop (src/tui.rs), the GUI loop (src/gui.rs), and the headless loop (src/main.rs) run on a single thread, a 5-minute PoW mining request freezes the UI and blocks all other incoming signing requests until mining finishes.

Design Goals

1. Non-blocking PoW Dispatch (Option 1 — Targeted Background Offloading):
   • Keep MnemonicState, RoleTable, KeyStore, and AlgorithmKeyCache owned by the main thread without Arc<Mutex<...>> lock contention or shared secret lifetimes.
   • Validate the nostr_mine_event request, verify any auth envelope, resolve the role selector, and ensure the role's key is derived on the main thread (microseconds).
   • Extract a temporary 32-byte stack copy of the role's private key (wrapped in zeroize::Zeroizing<[u8; 32]>), take ownership of the client connection's ConnWriter, and spawn a background coordinator thread for the mining run.
2. Multiple Concurrent PoW Jobs:
   • Any number of nostr_mine_event requests may run concurrently alongside fast signing requests (nostr_sign_event, sign, nip44_encrypt, get_public_key, etc.).
   • On each poll tick, the server drains all pending incoming requests and drains any completed background mining activity messages via an mpsc channel.
3. Immediate Cancellation on Stop / Seed Unload:
   • ServerContext holds a shared cancellation token (Arc<AtomicBool>) and an active job counter (Arc<AtomicUsize>).
   • Calling ServerContext::stop() (which is invoked on server stop, transport restart, role re-derivation/removal, or seed unload/quit in TUI, GUI, and headless modes) sets the cancellation flag to true, immediately signaling all in-flight miner::mine_worker() threads to abort, zeroize their temporary key copy, and exit cleanly.
4. Socket Blocking Mode Fix for Background Responses:
   • Accepted sockets in ServerContext::poll_request() are placed in set_nonblocking(true) for the initial probe/read. Before ConnWriter::send_response() writes the response (especially from a background thread or for large payloads), ensure the stream is set back to blocking mode (or write timeout) so writing the response never fails silently with WouldBlock.

Architecture Diagram

sequenceDiagram
    participant ClientA as Client A - PoW Mine
    participant ClientB as Client B - Fast Sign
    participant MainLoop as Main Thread - TUI / GUI / Headless
    participant Server as ServerContext + Dispatcher
    participant BgThread as Background Mine Coordinator
    participant Workers as PoW Worker Threads 1..N

    ClientA->>Server: nostr_mine_event request
    MainLoop->>Server: poll_request + finish_request
    Server->>Server: Verify auth, resolve role, derive key if needed
    Server->>Server: Pre-validate event JSON and termination condition
    Server-->>MainLoop: Return START activity immediately
    Server->>BgThread: Spawn with ConnWriter, Zeroizing key, cancel_flag, activity_tx
    BgThread->>Workers: Spawn N stride workers with cancel_flag

    ClientB->>Server: nostr_sign_event request
    MainLoop->>Server: poll_request + finish_request
    Server->>ClientB: Signed event response in <1ms
    Server-->>MainLoop: Return sign activity line

    Workers-->>BgThread: Target reached or timeout or cancelled
    BgThread->>BgThread: Sign best event and zeroize key copy
    BgThread->>ClientA: Send JSON-RPC response via ConnWriter
    BgThread->>Server: Send DONE / TIMEOUT / CANCELLED activity via activity_tx
    MainLoop->>Server: poll_background_activities
    Server-->>MainLoop: Drain completion activity lines to ActivityLog

Detailed Component Changes

1. Cancellable Miner (src/miner.rs)

• Add pub cancelled: bool to MineResult.
• Add miner_run_cancellable() (and keep miner_run() as a thin wrapper passing None for backwards compatibility with existing unit tests):

  pub fn miner_run_cancellable(
      event_json: &str,
      private_key: &[u8; 32],
      target_difficulty: i32,
      thread_count: i32,
      timeout_sec: u64,
      cancel: Option<&Arc<AtomicBool>>,
  ) -> Result<MineResult, crate::SignerError>
  

• Inside mine_worker():
  • Check the external cancel AtomicBool (with Ordering::Relaxed) every 1,024 iterations (alongside the shared stop check) so workers exit within microseconds when cancelled without lock contention on every single hash.
  • Also replace the per-iteration shared.lock() check on line 158 with an AtomicBool (stop_flag: Arc<AtomicBool>) so worker threads do not contend on a Mutex on every single SHA-256 hash attempt! (This will also significantly speed up multi-threaded PoW hash rates).
• If cancelled via the external cancel flag, miner_run_cancellable sets cancelled: true on MineResult (or skips signing if shutting down).

2. Background Job Preparation in Dispatcher / KeyStore (src/dispatcher.rs & src/key_store.rs)

• Add a prepared mining request struct in src/dispatcher.rs:

  pub struct PreparedMineJob {
      pub rpc_id: String,
      pub role_name: String,
      pub event_json: String,
      pub private_key: zeroize::Zeroizing<[u8; 32]>,
      pub difficulty: i32,
      pub threads: i32,
      pub timeout_sec: u64,
  }
  

• When handle_nostr_verb in src/dispatcher.rs handles VERB_NOSTR_MINE_EVENT:
  • Validate params[0] (must be valid unsigned Event JSON), difficulty / timeout_sec termination condition, role selector, purpose enforcement, and key derivation synchronously.
  • If validation fails, return the immediate JSON-RPC error response as before.
  • When invoked from ServerContext (which supports background offloading), populate ctx.pending_mine_job: Option<PreparedMineJob> with the Zeroizing<[u8; 32]> key copy and validated parameters instead of blocking on key_store.mine_event().
  • Keep synchronous execution in handle_request() when ctx is used directly in unit tests (or provide a flag offload_mining: bool on DispatcherContext).

3. Background Spawning & Activity Channel in ServerContext (src/server.rs)

• Extend ServerContext with:
  • activity_tx: std::sync::mpsc::Sender<String>
  • activity_rx: std::sync::mpsc::Receiver<String>
  • cancel_flag: Arc<AtomicBool>
  • active_jobs: Arc<AtomicUsize>
• In ConnWriter::send_response():
  • Set the underlying UnixStream or TcpStream back to blocking (set_nonblocking(false)) before writing the response so the response write never fails with WouldBlock.
• In ServerContext::finish_request():
  • Set dispatcher.offload_mining = true (except in ListenMode::Stdio | ListenMode::Qrexec one-shot mode where the process exits after one request, or wait for active jobs before exiting).
  • If process_request yields a PreparedMineJob:
    • Return the START activity line (or Option<String> if already logged by two-phase caller) immediately.
    • Increment active_jobs.
    • Spawn a background thread owning writer: ConnWriter, job: PreparedMineJob, caller_id: String, cancel_flag: Arc<AtomicBool>, active_jobs: Arc<AtomicUsize>, and activity_tx: Sender<String>.
    • The background thread runs miner::miner_run_cancellable(...):
      • Drops/zeroizes job.private_key immediately after miner_run_cancellable returns.
      • If not cancelled, formats the JSON-RPC response, calls writer.send_response(&response), formats the completion activity via mine_done_activity(), and sends it over activity_tx.
      • If cancelled, sends a cancellation error response (best-effort) and sends a CANCELLED activity line over activity_tx.
      • Decrements active_jobs.
• Add ServerContext::poll_activity(&mut self) -> Option<String> (and drain_activities(&mut self) -> Vec<String>) to read completed background job messages from activity_rx.try_recv().
• Add ServerContext::active_mining_count(&self) -> usize so callers/tests/UI can inspect how many background jobs are currently in flight.
• Update ServerContext::stop(&mut self):
  • Set self.cancel_flag.store(true, Ordering::SeqCst) to abort all in-flight mining workers immediately, then reset self.cancel_flag = Arc::new(AtomicBool::new(false)) for any future restart.
• Update ServerContext::handle_one():
  • If a pending socket request is available, handle it (for backgrounded mining, return the START activity line immediately).
  • Otherwise, check self.poll_activity() and return Ok(Some(bg_activity)) if a background mining job finished.

4. TUI, GUI, and Headless Loop Integration (src/tui.rs, src/gui.rs, src/main.rs)

• TUI (src/tui.rs):
  • In App::service_server():
    • First drain server.drain_activities() and append each line to self.activity_log.
    • Process any pending socket requests in a loop (or one per tick) without blocking on nostr_mine_event: when nostr_mine_event is dispatched in finish_request, it spawns the background thread and returns immediately, so START is logged once and DONE/TIMEOUT arrives later via drain_activities().
• GUI (src/gui.rs):
  • In GuiApp::service_server():
    • Drain server.drain_activities() into self.activity_log and request repaint when activities arrive or while server.active_mining_count() > 0.
    • Handle incoming requests non-blockingly.
• Headless (src/main.rs):
  • In run_headless():
    • For Tcp / Http / Unix loop: drain and println! completed activities from server.drain_activities() on every iteration, and process incoming requests non-blockingly.
    • For Stdio / Qrexec single-request mode: wait for any spawned job to finish (or run synchronously) before exiting the process.

5. Concurrency & Cancellation Integration Tests (tests/integration.rs)

• Add test_concurrent_signing_during_pow_mining:
  • Start a nostr_mine_event request on Client Thread A with a high difficulty (e.g., difficulty: 28, timeout_sec: 5) that will run for several seconds.
  • While Client Thread A is actively mining, send multiple nostr_sign_event, nostr_get_public_key, and sign (ed25519) requests from Client Thread B.
  • Assert that all of Client Thread B's requests complete in < 200ms while Client Thread A is still mining.
  • Assert that multiple concurrent nostr_mine_event requests can also run simultaneously.
  • Assert that calling server.stop() cancels the in-flight mining job immediately without waiting for the full timeout.