* fix(cli): wait for the old daemon to finish ongoing requests during restart
The daemon shuts down gracefully: it stops listening right after /stop,
but keeps serving ongoing requests before disposing of the wallet and
releasing wallet.pid. The restart flows only waited for the health check
to go down, then spawned a replacement daemon that failed to claim the
routstrd wallet lock and exited with code 1, surfacing a confusing
'Cannot claim the routstrd wallet lock ... PID X is still running' error.
Add waitForDaemonToExit() and use it in restart, mode, the post-update
restart, and stop:
- Phase 1: wait (10s) for the health check to stop responding.
- Phase 2: while the old process still holds the wallet lock, show
'Finishing all ongoing requests...' (heartbeat every 10s) and wait up
to 10 minutes for it to exit. A stale lock (dead PID) is not waited on;
after the timeout the error names the holding PID and how to force it.
stop now also waits for full exit instead of returning as soon as /stop
is acknowledged, and no longer auto-starts a daemon when none is running.
* fix(cli): offer 'kill -9 <PID>' to force stop a draining daemon
The drain progress messages and the drain-timeout error now suggest
'kill -9 <PID>' so the user can force the old daemon out instead of
waiting for stuck requests. A plain SIGTERM would not work: the daemon's
signal handler re-runs the same graceful shutdown (server.close()), which
keeps waiting for ongoing requests, so only SIGKILL can interrupt a stuck
drain. The wait loop already treats the resulting dead-PID lock as
released (same liveness semantics as claimPidFile), so the restart
proceeds cleanly.
The heartbeat interval is now injectable (drainHeartbeatMs, default 10s)
like the other timing knobs, which also lets the tests cover the
heartbeat message.
---------
Co-authored-by: redshift <213178690+1ftredsh@users.noreply.github.com>
`routstrd clients` only exposed list/delete/add, so operators had no way to
refresh client integrations on demand or to stop the daemon from rewriting
their client configs every 21 minutes.
- clients --manual-refresh: refresh routstr21 models from Nostr and re-run
every registered client integration now. The old `routstrd refresh` body
moves into a shared refreshModelsAndClientsAction() so both commands stay
in sync.
- clients --disable-automatic-refresh / --enable-automatic-refresh: toggle
the daemon's scheduled refresh job via a new POST /settings/auto-refresh
endpoint, so the toggle also works against a remote daemon where the config
lives on the host. Persisted as autoRefresh.enabled in config.json.
- The daemon refresh job now re-reads autoRefresh on every tick (like the NWC
auto-refill getter), so toggling takes effect without a restart. While
disabled it polls once a minute so re-enabling applies promptly.
- Startup still fetches models (the proxy needs them) but skips the client
integration pass when the job is disabled, so a restart cannot overwrite
hand-edited client configs.
Adds tests/daemon/auto-refresh.* covering the endpoint contract and the
persisted flag.
Co-authored-by: redshift <213178690+1ftredsh@users.noreply.github.com>
* pi integration
* feat: wire routstrModelsPubkey through daemon
Add RoutstrdConfig.routstrModelsPubkey and pass it to ModelManager and the HTTP handler deps, forwarding it into routeRequests so the models allowlist pubkey (kind 38423) can be set independently of the audit pubkey (kind 38425).
* chore: link @routstr/sdk as file dep; ignore findings
Switch @routstr/sdk to file:../routstr-sdk for local development, refresh bun.lock, and ignore findings artifacts.
* fix(security): write daemon config with 0600/0700 perms, atomically
The daemon config stores spend-capable credentials (operator nsec and the
NWC connection string), but saveDaemonConfig wrote it with Bun.write() and
no mode, and ensureDirs created the config dir with no mode — yielding a
0755 dir and 0644 file under the standard 022 umask, readable by any local
user. The wallet seed path already gets this right (0700/0600), so this was
an inconsistency, not a trade-off.
- saveDaemonConfig now writes via temp-file + rename with mode 0600
(mirroring saveConfig in wallet/coco-client.ts), is synchronous so write
errors propagate instead of being silently dropped, and chmods the target
so previously over-permissive files are repaired on every save.
- ensureDirs creates CONFIG_DIR/REQUESTS_DIR with mode 0700 and chmods
existing dirs, correcting older installs on every daemon start.
- loadDaemonConfig/loadDaemonConfigSync chmod the config file 0600 on read,
so even a never-saved install gets repaired.
- cli.ts routes its raw Bun.write(CONFIG_FILE) calls (init, nsec generation,
remote/local mode switch, mode set) through saveDaemonConfig and uses
ensureDirsSync for the initial directory creation.
Also fixes the crash-mid-write hazard: a torn JSON write previously made
loadDaemonConfig silently revert to DEFAULT_CONFIG, dropping nsec/NWC/
provider settings; the atomic rename prevents that.
Adds subprocess-isolated regression tests (tests/daemon/) asserting
0600/0700 on fresh installs, repair of 0644/0755 installs, synchronous
error propagation, and corrupt-JSON fallback.
---------
Co-authored-by: redshift <213178690+1ftredsh@users.noreply.github.com>
`bootstrapProviders` applies the kind-38425 lgtm review disables to the
discovery adapter, but `ensureProvidersBootstrapped` only mirrored the
discovered baseUrlsList into the SdkStore. `providers list` and the
per-model provider views read the store, so on a fresh install they
reported "0 disabled" while routing (which reads the discovery adapter)
silently excluded the review-disabled providers.
Mirror the review-disabled set returned by `syncReviewedProvidersFromNostr`
into the store, matching what `refreshProvidersAndModels` already does.
A `null` result means "unchanged" and is left alone so an empty review set
does not clobber previously disabled providers.
Adds a regression test for both the mirror and the null/unchanged paths.
Co-authored-by: redshift <213178690+1ftredsh@users.noreply.github.com>
PR #72 changed mergeHermesConfig to update an existing Routstr provider
in place (base_url/api_key/model + rename, repointing model.provider),
but the test from f90e65a still asserted the old keep-unchanged
behavior and has failed since.
Update the test to assert the in-place update and idempotency, and add
coverage for the model.provider repoint (and that unrelated provider
refs are left alone).
Co-authored-by: redshift <213178690+1ftredsh@users.noreply.github.com>
* fix(wallet): distinguish legacy cocod from routstrd locks
Treat the legacy Unix socket, rather than the shared cocod.pid file, as the authoritative cocod identity check. routstrd deliberately writes its own PID into cocod.pid as an exclusion fence, so the previous PID-only guard could falsely report an existing routstrd process as legacy cocod after an interrupted or slow startup.
Keep startup safe by relying on the atomic PID-file claim for the cocod-starting race, while continuing to fail closed for socket probe errors that do not prove cocod has stopped. Remove the now-unnecessary migration ignorePid workaround.
Harden stale-lock recovery by detecting Linux zombie processes through /proc/<pid>/stat, registering synchronous process-exit cleanup for owned PID files, and installing daemon signal handlers before migration and wallet initialization. Improve contention and startup-timeout errors with the lock identity, owning PID, and actionable recovery guidance.
Add regression tests for live shared PID owners with missing or stale cocod sockets, responsive cocod sockets, unsafe probe failures, zombie detection, and process-exit cleanup.
* Confirm expired mint quotes with their mints before failing locally
Problem
-------
0ce4c07 pruned expired pending mint quotes purely locally at startup:
any quote past its bolt11 expiry with no recorded PAID/ISSUED
observation was failed without contacting its mint. That invariant is
only forward-looking: a quote can be paid before expiry while the
daemon is down, leaving no local observation behind. Failing such a
quote strands the paid funds at the mint: failed operations are skipped
by recoverPendingMintOperations(), so the claimable proofs are never
claimed.
Concrete case: receiveBolt11 invoice created, daemon stops, user pays
within expiry, daemon restarts after expiry: the old prune failed the
op without ever asking the mint.
Change
------
Replace failExpiredMintsLocally() with settleExpiredMintQuotes(),
which adds one bounded observation round before any local fail:
1. Select expired, unobserved pending mint quotes as before
(selectCleanupOperations, minAgeMs 0).
2. For each candidate, ask its mint for the quote state via
MintOperationService.observePendingOperation() (the same check the
mint sweep uses, reached through the existing structural cast)
under a shared 15s wall-clock budget
(EXPIRED_MINT_OBSERVATION_DEADLINE_MS).
3. Act on the answer from the mint:
- UNPAID ("waiting"): the expired quote can never be issued, so
failing it locally cannot strand funds; failPendingOperation().
- PAID/ISSUED ("ready"/"completed"): leave pending; the mint
recovery sweep (or the processor, via the emitted
mint-op:quote-state-changed event) finalizes it and claims the
proofs.
- unreachable/slow mint or unknown quote: leave pending so a later
startup can still recover it. Nothing is failed without a mint
confirmation.
Why a deadline
--------------
coco-core issues mint requests via bare fetch() with no timeout, so a
hung mint could otherwise stall this phase (and with it the recovery
promise that gates value-moving operations) for minutes. The shared
budget caps the whole round at 15s; the unobserved remainder stays
pending and is handled by the normal sweep (background, per-op
contained).
Why not keep the blind local fail
---------------------------------
The mint sweep treats UNPAID as "waiting" and never fails expired
quotes itself, so some form of pruning is still required to keep
recovery quick on wallets with many dead quotes. The observation round
keeps that property: confirmed-unpaid quotes are failed before the
sweep and never contacted again, while the unsafe case (paid before
expiry, never observed) now goes through normal recovery.
Side effects
------------
- Asking the mint also closes the narrower race from the old flow
(watcher records PAID between selection and fail): quotes are now
failed only when the mint currently reports UNPAID past expiry.
- Recovery phase strings are now "Settling/Settled expired mint
quotes"; settlement counts are logged to the startup stream.
- The explicit wallet cleanup command keeps its local-only semantics:
it is user-invoked, supports dry-run, and defaults to a 7-day
minimum age, giving ample observation opportunity beforehand.
Testing
-------
- New settleExpiredMintQuotes unit tests (6): mint-confirmed unpaid is
failed locally; PAID/ISSUED is left for recovery; unreachable mint
is left pending; hung mint is bounded by the shared deadline;
unexpired/observed quotes untouched.
- bun run lint (tsc --noEmit) passes.
- bun run build passes.
- Wallet/cleanup tests pass (56/56).
- Full bun test shows one pre-existing, unrelated failure
(mergeHermesConfig) that also fails on the parent commit.
* feat(wallet): add migration conflict diagnostics and wallet doctor
When both ~/.routstrd/wallet and ~/.cocod contain different wallets,
startup now refuses with a structured, privacy-safe comparison
(mnemonic fingerprints, timestamps, proof/mint summaries) instead of
a terse one-liner, and points to the new 'routstrd wallet doctor'
command for a full report and resolution steps.
The mnemonic is never printed: only a truncated SHA-256 fingerprint,
and only for unencrypted configs. Database summaries are read-only
and degrade gracefully on malformed or corrupt files.
* fix(wallet): polish doctor verdicts and conflict error surfacing
Review follow-ups:
- Gate the mv resolution steps on an actual conflict; the doctor no
longer tells fresh installs or healthy single-wallet setups to move
directories around.
- Print WalletMigrationConflictError cleanly at 'routstrd onboard'
(message + exit 1) instead of Bun's unhandled-rejection dump with
source snippet and stack trace; startDaemon failures likewise.
- New diagnoseWallets() classifies both wallet locations the way
migration sees them (including incomplete db-only legacies) and
drives the doctor verdict, resolution gating, and exit code.
- 'routstrd wallet doctor' exits 1 when startup would refuse to
migrate, so scripts can detect the conflict state.
- Count mints from the mints registry table (falling back to mints
seen in proofs), add thousands separators to balances, say 'just
now' instead of '0s ago', and clarify the same-mnemonic verdict.
- The startup conflict message now includes the 'routstrd stop' step
via the shared renderResolutionSteps().
* fix(wallet): share migration classifier between startup and doctor
diagnoseWallets previously re-derived startup state with looser rules (presence + fingerprint), which disagreed with migrateLegacyWallet on several states:
- target init + source db-only: migration returns already-current, but the doctor claimed startup would refuse
- target db-only + source absent: migration throws, but the doctor said no migration needed
- orphaned source SQLite sidecars: migration throws, but the doctor said fresh install
- same-mnemonic wallets: doctor could not distinguish byte-identical (already-current) from same-mnemonic-different-bytes (conflict)
Extract the exact decision order into classifyWalletMigration() in a new wallet-state.ts and make both migrateLegacyWallet and diagnoseWallets consume it, so the doctor's verdict, resolution gating, and exit code can never drift from actual startup behavior again.
Add regression tests for each previously mismatched state.
* fix(wallet): never let the doctor crash on unreadable wallet files
classifyWalletMigration does raw byte reads (filesEqual) that throw on
unreadable files or delete races — fine for startup, where the same
throw is loud either way, but the doctor exists to diagnose broken
states and must render its report regardless. diagnoseWallets now
catches classification errors and falls back to a conflict verdict
(preferring the 'could not be fully read' text when the guarded
summarizers already recorded the underlying error), so the report,
resolution steps, and exit code still reach the user.
Also adds the missing trailing newlines in wallet-state.ts and
diagnostics.test.ts.
---------
Co-authored-by: redshift <213178690+1ftredsh@users.noreply.github.com>
* feat: cap proxied completion budget via configurable maxTokens
Inject a default max_tokens (chat/completions) or max_output_tokens
(responses) when a client omits one, so the SDK prices against
completion x maxTokens instead of the provider's worst-case
max_completion_cost. Default 64000; set 0 to disable.
* Silence file logger during test runs
bun test sets NODE_ENV=test, but tests import modules that pull in the
logger singleton, causing test output to be written into the real
~/.routstrd log files alongside production daemon output. Early-return
in writeLog when running under test.
* Add SECURITY.md with vulnerability reporting policy
* fix(clients): update Hermes provider in place and improve client integration error feedback
- hermes: re-running 'clients add --hermes' now updates base_url/api_key/model
in place instead of keeping a stale entry, and repoints model.provider when
the provider name changes
- clients: print a clear error on integration setup failure with a NIP-98
auth hint for rejected npubs, and exit non-zero if any integration failed
- deps: bump @routstr/sdk to 0.3.21
---------
Co-authored-by: redshift <213178690+1ftredsh@users.noreply.github.com>
redshiftandGitHub
