Backlog items 4-5 close. Both-variants-in-one-run A/B at 50k, repeated
with relay order reversed: every delta flipped with the order (the
second-running relay won queries and ingest latency in BOTH runs), so
readers=8 / mmap_size=256MiB / temp_store=MEMORY / periodic PRAGMA
optimize are all noise-level on container-class hardware. The config
plumbing stays (hardware-dependent, operators should measure their own
box); the example config now says so explicitly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Backlog items 4-5 plumbing, config-gated and off by default so quartz
library defaults stay untouched for the app-side stores:
- quartz: SQLiteEventStore/EventStore accept extraPragmas (applied on
every pooled connection AFTER the built-in configuration, so they
can override it) and expose optimize() — an analysis_limit-bounded
PRAGMA optimize for incremental planner-stats refresh.
- geode: [database] readers / mmap_size / temp_store_memory map onto
the store; optimize_interval_seconds drives a maintenance coroutine
(cancelled first in the shutdown hook, before the store closes).
The A/B verdict on whether the example config should RECOMMEND any of
these on container-class hardware follows in the next commit — the
knobs themselves are operator tools worth having either way, since
mmap/temp-store value is hardware-dependent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Found while attributing the small-REQ wire floor (backlog item 6,
latency half): geode's new WireReqFloorBenchmark measured a flat
43.7 ms per REQ round trip that survived every server-side change —
store configs, dispatchers, the pump — and then vanished when the
round's preceding CLOSE was dropped. Root cause is client-side: OkHttp
does not set TCP_NODELAY, relays never answer a CLOSE (NIP-01), so its
bytes sit unACKed for the peer's ~40 ms delayed-ACK window and Nagle
holds the next REQ behind them. CLOSE-then-REQ is a Nostr client's
hottest pattern — every feed/filter switch.
relayBench's harness client already shipped a no-delay socket factory
(which is why benchmark numbers never showed the stall) but the
production clients did not. New TcpNoDelaySocketFactory (quartz
jvmAndroid, next to BasicOkHttpWebSocket) is now used by the Android
relay pool factory, the Desktop relay client, amy's relay connections,
and geode's mirror worker. Direct connections only — SOCKS/Tor paths
are untouched.
With the factory, the benchmark puts geode's ~21-row REQ at ~1.25 ms
on the wire (matching relayBench): ~0.6 ms Ktor CIO+OkHttp loopback
floor, ~0.5 ms per-REQ server work (already investigated). Per-frame
burst cost measured negligible and the pump adds ~nothing, so the
send-path latency angle of backlog item 6 is closed as not-a-problem;
its ingest-CPU share remains a separate throughput question. Findings
recorded in quartz/plans/2026-07-04-small-req-floor.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Reverts the queryRawInline fast path (fb29d655, 1b786f31) per the
keep-only-winners rule. Three relayBench runs at 50k (baseline, cap-256
where the path never engaged, cap-512 where author-archive/by-ids/
500-limit feeds genuinely took it) showed no movement outside the
container drift band — strfry's own numbers drifted ±30% between runs
and inline-eligible scenarios moved the same as ineligible ones.
The in-process win was real but small (~17%, 0.60 -> 0.50 ms per
~21-row REQ); the wire-level p50 is 1.2-1.7 ms, so the missing ~1 ms
per REQ sits in the transport (Ktor frame send path + client round
trip) — backlog item 6 territory, not dispatch. Findings, numbers, and
the do-not-retry note live in quartz/plans/2026-07-04-small-req-floor.md;
SmallReqFloorBenchmark stays as the measurement tool.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
The first cut's rule (every filter needs limit, sum <= 256) missed the
shapes relays actually receive: relayBench's author-archive carries
limit=500 and by-ids carries only an ids list, so the fast path never
engaged in the acceptance run. Bounds now come from limit OR the ids
count (ids are unique keys, the result cannot exceed the list), summed
against a 512 cap — a 500-row inline replay measures single-digit ms,
nothing a CLOSE could meaningfully preempt. Provably unbounded filters
(no limit, no ids — e.g. a bare tag filter) keep the launched path.
Pending: relayBench verdict decides whether the fast path stays at all,
per the keep-only-winners rule.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Backlog item 2 (small-REQ dispatch floor), first cut. relayBench at 50k
shows geode ~2.5x slower than strfry when a REQ returns ~20 events
(author-archive 1.18 vs 0.48 ms EOSE p50) while WINNING the 500-event
scenarios — with tiny results the fixed per-REQ cost dominates. A new
stage benchmark (SmallReqFloorBenchmark) decomposes the floor: at ~21
rows, raw SQL is 0.18 ms and the remaining ~0.42 ms is live-subscription
machinery plus per-REQ coroutine dispatch.
A REQ whose filters all carry a limit summing to <= 256 now runs its
stored replay inline on the receive coroutine and only retains a
live-tail handle (SessionBackend.queryRawInline; LiveEventStore's
queryRaw is re-expressed on the same core) — no per-REQ launch, no Job,
no dispatcher handoffs, no awaitCancellation scaffolding. Unbounded
REQs keep the launched path so CLOSE can always interrupt a giant
replay. In-process time-to-EOSE for ~21-row REQs drops 0.60 -> 0.50 ms;
the bigger effect expected under concurrency (no per-REQ Job+dispatch
churn) is relayBench's to judge.
InlineReqFastPathTest pins wire-behavior parity: stored-then-EOSE
ordering, live tail delivery and CLOSE detachment, same-subId
replacement, launched fallback for unbounded and over-cap REQs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Same container, 50k corpus, geode --no-search, strfry built from
source: identical-set reconcile 41.4 ms (geode) vs 30.1 ms (strfry) —
down to ~1.4x from the campaign-opening full-scan-per-open; cold
reconcile 177 vs 112 ms (geode's first open pays the lazy rebuild);
ingest at parity in this container; storage 72.8 vs 106 MiB. Notes the
zero-copy IStorage follow-up that would close the remaining ~11 ms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Two relayBench runs (50k corpus, both geode variants side by side, then
order-reversed): identical-set reconcile 56/57 ms with the index vs
110/130 ms without in both orders (~2.2x, the post-write NEG-OPEN the
old cache always missed); ingest and first-ever reconcile deltas flip
with relay order, i.e. run-order noise — no regression. Also records
the run-order-bias protocol note for future A/Bs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Milestones 2-3 of quartz/plans/2026-07-03-incremental-negentropy-storage.md.
SQLiteEventStore now maintains the LiveNegentropyIndex when the strategy
opts in (geode's does by default; [negentropy].live_index = false turns
it off; app-side stores are untouched):
- Write paths collect a LiveIndexDelta and apply it after COMMIT while
still holding the writer mutex — rolled-back savepoint rows never
reach the index and updates land in exact commit order (also vs the
rebuild, which runs under the same mutex).
- Replaceable/addressable overwrites report the row their BEFORE-INSERT
trigger displaces via one indexed pre-SELECT that mirrors the trigger
predicate (including the NIP-01 lowest-id tie-break and the
d_tag-NULL case).
- Paths that can't itemize (kind-5, vanish, delete-by-filter/id,
expiration sweeps, clearDB) invalidate; the next NEG-OPEN rebuilds
from one scan on the writer connection.
- Until that first NEG-OPEN populates the index, ingest pays zero
bookkeeping — the populated check happens under the writer mutex so
it can't race the rebuild.
LiveEventStore serves a single unconstrained filter (the relay-relay
sync default; relayBench sends exactly this) from the index; everything
else keeps the scan+seal path and its single-slot cache.
Micro-benchmark at 50k events (LiveNegentropyBenchmark, in-container):
scan+seal cold path 80-100 ms; index post-write open 9-16 ms (~5-10x).
The relayBench A/B is the acceptance gate and comes next.
Correctness: LiveNegentropyIndexStoreTest asserts index content ==
snapshotIdsForNegentropy scan after every mutation pattern (overwrites,
losers, kind-5 rebuilds, filter deletes, mixed-outcome batches,
transactions); the full geode suite (NIP-77 + interop sync tests) runs
with the index on.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Milestone 1 of quartz/plans/2026-07-03-incremental-negentropy-storage.md.
Sorted array with binary-search insert (near-tail in the common case),
itemized remove for displaced rows, wholesale invalidate for delete
paths that can't itemize, and sealed snapshots memoized per mutation
generation — reconcile only reads, so one snapshot backs any number of
concurrent sessions and stays immutable under later writes. Over-cap
answers null so the caller keeps the strfry-parity NEG-ERR.
Not wired into any store yet; next milestones plumb displaced-row
deltas from the SQLite modules and serve index-total filters from it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Design for backlog item 3 of the relay performance campaign: an
always-current (created_at, id) index maintained from the store's write
path so cold NEG-OPENs stop paying the full scan + O(n log n) seal
(~340 ms at 50k events vs strfry's ~21 ms off its live tree). Covers
the snapshot/COW model, the removal-correctness split (RETURNING deltas
for replaceable overwrites, wholesale invalidation for rare delete
paths), IndexingStrategy gating so app-side stores are untouched, and
the micro + relayBench A/B measurement plan.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Measured over the real OkHttp transport (upstream killed mid-mirror and
restarted on the same port): NostrClient re-dials once on disconnect and
then falls back to its 60s keep-alive, which showed up as a 61s mirror
blackout when the immediate re-dial raced the port rebind.
- Retry pump: the worker nudges the pool every 5s. Cheap and safe —
reconnectIfNeedsTo skips connected relays and each relay's
exponential backoff (1s doubling, 5min cap) still gates real dial
attempts, so dead upstreams aren't hammered. Restart recovery drops
from 61s to ~6s in the new reconnect test.
- WebSocket pings (120s, same as the Android relay pool): without
them a half-open connection (network drop, no FIN) never fires
onDisconnected and the mirror would stall silently forever.
- MirrorWorkerReconnectTest: end-to-end over a real Ktor port — ride
through the drop, re-subscribe, drop the duplicate replay, pull the
event that only exists on the new instance.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Each [[mirror]] entry takes an optional NIP-01 filter as a JSON object
string, like strfry-router's per-stream filter. It is applied twice,
matching strfry's design (cmd_router.cpp):
- it shapes the REQ sent upstream (the mirror still owns since via
backfill_seconds and strips limit — the subscription is unbounded);
- every delivered event is re-checked against it before ingest, so an
upstream answering outside its REQ — including a trusted one whose
events skip signature verification — can only inject events inside
the operator-declared scope. Out-of-scope deliveries surface on a
'filtered' counter.
Malformed filter JSON fails the boot, not the first delivery. Several
disjoint scopes for one upstream = repeat [[mirror]] with the same url.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Backlog item 1 of the relay performance campaign: skip signature
verification for events ingested from explicitly configured trusted
upstream relays, strfry-router style.
geode now dials each [[mirror]] url from the config, subscribes to
everything newer than now - backfill_seconds, and feeds the stream
through NostrServer.ingest (same group-commit writer + live fanout as
client publishes). The NostrClient underneath owns reconnects, backoff
and REQ re-sync; duplicate replays after a reconnect are rejected by
the store's unique-id constraint and only surface as counters.
trusted = true is the per-upstream trust switch: events from that
connection skip Schnorr verify. The trusted identity is the URL this
relay dialed (TLS-authenticated for wss://), never anything an inbound
peer claims. Default is false — mirror-but-verify — and a relay with no
[[mirror]] entries behaves exactly as before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Adds a public local-ingest entry to NostrServer for events that don't
arrive over a client connection (mirror/sync workers, import jobs). It
routes through the same group-commit IngestQueue and live fanout as a
client publish, and each Submission can opt out of the parallel
Schnorr-verify hook — the relay-to-relay trust model, for events
streamed from an upstream that already verified them (verify profiles
at ~8% of busy ingest CPU).
Library defaults are unchanged: skipVerify defaults to false everywhere
and nothing in the client-publish path can set it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
The worker's writer-mutex fast path can slip past its own scope
cancellation and run one batch against a store that close() already
freed. The resulting closed-connection exception escaped scope.launch
under a SupervisorJob with no handler — harmless in production, but the
kotlinx-coroutines-test global handler attributes it to whatever runTest
starts next (seen on CI as EventSourceServerTest.countUsesSource failing
with UncaughtExceptionsBeforeTest). Catch, log, and stop the pass:
nothing depends on it — the pre-search drain covers search correctness.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
Two issues surfaced by rebasing onto main:
- SQLiteConnectionPool.close() freed the writer's native handle without
taking the writer mutex, so a block still running on another thread
(the deferred-FTS catch-up worker's current batch outlives its scope
cancellation) could call sqlite3_prepare on a freed sqlite3* — native
heap corruption, SIGSEGV in sqlite3DbMallocRawNN (reproduced twice in
:geode:test). close() now reclaims every reader from the channel and
acquires the writer mutex before closing handles, then releases so
stragglers get the managed closed-connection exception. Idempotent
via a closed flag.
- main's NIP-50 fix (7deda28d) strips search-extension tokens before
every store query, but the zero-decode queryRaw replay path added on
this branch predates it and passed raw filters through — an
extensions-only search would hit SQLite FTS as column syntax and
error. queryRaw now strips like query/count/snapshot do.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
FTS indexing ran inside every insert's transaction — a measurable slice
of write cost (relayBench: ~18% of ingest throughput) paid at publish
time for a feature only search queries read. It now runs as a watermark
catch-up:
- IndexingStrategy.deferFullTextSearchIndexing (default false; geode's
relay strategy enables it with search). Deferred inserts skip
tokenization entirely.
- FullTextSearchModule keeps a fts_catchup_state watermark (everything
<= last_row_id is indexed) and gains catchUpBatch(): scan past the
watermark, tokenize, advance — one write transaction per batch, so
publishes interleave. DATABASE_VERSION 3->4 seeds the watermark at
MAX(row_id) for existing (synchronously indexed) databases.
- NostrServer runs the catch-up worker, poked by IngestQueue's new
onBatchCommitted hook, and *yields to publish traffic*: it only
drains while the queue has no backlog (IngestQueue.hasBacklog()), so
bursts ingest at no-FTS speed and tokenization fills the gaps.
- LiveEventStore drains the backlog synchronously before serving any
filter with a search term (query, queryRaw, count) — NIP-50 results
stay exactly as fresh as the synchronous path; the deferral is
invisible to correctness. Geode's existing search tests pass
unchanged through this path.
The first implementation reused reindexBatch and collapsed ingest 8x —
its per-row 'DELETE FROM event_fts WHERE event_header_row_id = ?'
matches on a plain FTS5 column, i.e. a full FTS-table scan per row
(O(n²) overall), and the worker competed with the replay for the writer
mutex. catchUpBatch therefore inserts without the delete (rows past the
watermark are never indexed; switching a DB between deferred and
synchronous strategies requires reindexAll, same rule as a
searchable-kinds change), and the worker backs off whenever publishes
are pending.
Alternating A/B, 50k corpus, search-enabled default: 4,902/5,090/5,136
events/s synchronous vs 5,425/5,611 deferred (+8-12%), approaching the
--no-search ceiling while keeping NIP-50 advertised and fresh.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
A NIP-77 server session rebuilt its reconciliation structure from
scratch on every NEG-OPEN: full id+created_at scan, per-entry hex
decode into a fresh StorageVector, O(n log n) seal. That cost grows
with the corpus and is paid even when nothing changed — the exact
shape of a periodic mirror's heartbeat, where N peers reconcile the
same broad filter over and over. relayBench measured 342 ms per
identical-set reconcile at 50k events vs strfry's 26 ms off its
always-current LMDB tree.
Reconciliation only *reads* the sealed storage, so one instance can
back any number of concurrent sessions:
- NegentropyServerSession now accepts a pre-sealed IStorage (the
List<IdAndTime> constructor remains and delegates).
- SessionBackend.sealedNegentropyStorage() builds + seals (null when
the set exceeds maxSyncEvents); LiveEventStore overrides it with a
single-slot cache keyed by (filter set, write generation) plus a 30s
TTL. The generation bumps on every accepted ingest; the TTL bounds
staleness from delete paths the counter can't see (expiration
sweeps, admin purges) — negentropy snapshots are point-in-time sets,
so seconds of staleness only means a peer briefly re-offers ids.
- NegSessionRegistry.open consumes the shared sealed storage;
over-cap NEG-ERR behavior unchanged (strfry parity).
relayBench gains a 'heartbeat' measurement — the identical-set
reconcile repeated immediately with no writes in between. At 50k
events: geode 342 ms -> 27.8 ms vs strfry 21.1 ms (near parity; was
13x). Cold reconciles (first open after a write) are unchanged.
Also fixes the GeodeVsStrfryNegentropySyncTest fixture to write
'nofiles = 0' so the opt-in interop test can boot strfry inside
containers with a low RLIMIT_NOFILE hard cap; the interop suite passes
against strfry v1-b80cda3 with the cache in place.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
Reviewed strfry's LMDB indices (golpe.yaml) against geode's SQLite set:
the two are nearly isomorphic — time, id, kind+time, author+kind+time,
tag+time, plus conditional deletion/expiration/replaceable entries
(geode's are partial indexes, so ordinary events don't pay for them).
Nothing to drop. One real hole: strfry maintains a plain
pubkey(+created_at) index and geode had none, so an authors-only filter
(no kinds) — archive pulls, account-migration tools, 'everything by
these pubkeys' — degraded to a full walk of the time index. EXPLAIN
confirmed: SCAN query_by_created_at_id.
- quartz: IndexingStrategy.indexEventsByPubkeyAlone (default false —
clients query their supported kinds and can skip it) gates a new
query_by_pubkey_created index; DATABASE_VERSION 2→3 with an
idempotent migration that backfills it for opted-in strategies.
- geode: relayIndexingStrategy turns it on.
- relayBench: new 'author-archive' scenario — every kind by 3 *quiet*
pubkeys. Quiet is the point: prolific authors are dense in the time
index and a scan finds them quickly, which is why the suite never
caught this; sparse authors force the full walk.
Measured (50k corpus): author-archive EOSE p50 42.8 ms -> 3.6 ms (12x,
and the old path grows linearly with table size); ingest 5,337 -> 5,156
events/s (~3%, the one extra B-tree per event). strfry reference on the
same scenario: 0.52 ms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
The drain loop ran verify(batch N) and insert(batch N) strictly
back-to-back: the SQLite writer idled during every Schnorr verify and
the CPU cores idled during every commit. Split it into two stage
coroutines joined by a capacity-1 channel — a verifier that collects
and checks batch N+1 while the writer commits batch N. Same shape as
strfry's ingester/writer thread split.
Ordering (single verifier, single writer, FIFO handoff), OK-after-commit
semantics, per-row error isolation and submit() backpressure are all
unchanged; total in-flight grows by at most one batch.
Alternating A/B on the 10k corpus, 4-core host with the benchmark client
competing for the same cores: sequential 4,111/3,977/4,443 vs pipelined
4,050/5,020/5,092 events/s (~+13% mean). The overlap should widen on
dedicated relay hosts where verify has its own cores. Feature-parity
run after this change: geode --no-search 5,911 vs strfry 9,374 events/s
(1.59x, down from 2.2x at the start of the perf work).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
strfry implements no NIP-50 at all, so geode's default setup pays FTS
tokenization on every searchable event for a feature the other side of a
benchmark isn't providing. The new switch removes it cleanly:
- geode: --no-search CLI flag and [options].full_text_search TOML key
(default true). When off, the store skips the FTS index entirely,
NIP-11 stops advertising 50 (an explicit [info] nips list stays
operator-authoritative), and search filters match nothing via the
existing QueryBuilder guard. RelayIndexingStrategy becomes
relayIndexingStrategy(fullTextSearch) with the stock val kept.
- relayBench: --geode-no-search runs the geode entry with the flag (relay
named geode-nosearch in reports); README documents the apples-to-apples
rationale and a recipe for benchmarking both geode flavors side by side.
Alternating A/B on the 10k corpus: 4,767/4,792 ev/s without search vs
4,111/3,977 with — NIP-50 costs ~18% of ingest throughput at current
write-path speed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
Main's giant-REQ fix (18bd3600) replaced query()'s copy-on-add immutable
dedupe set with a spin-locked HashSet, but the rebase left queryRaw —
now the default REQ path — on the old pattern, which would have
reintroduced the O(n²) crawl for large replays. Both paths now share
the mutable-set-under-spinlock shape.
relayBench's sync driver moves to NegentropySession.fromEvents(),
following the session's new List<IdAndTime> constructor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
Three changes, each validated head-to-head against strfry with relayBench
(same 10k-event corpus a1cd3517a8296911, stock configs, sig verify on):
- geode: RelayIndexingStrategy turns on indexEventsByCreatedAtAlone for
the relay's stores (quartz's DefaultIndexingStrategy stays off for
client-side stores). A relay can't predict client filters, and the
cheapest REQ of all — {"limit":N} — was a full-table scan + top-N
sort: 40 ms and O(table) growth before; 11 ms and index-streamed
(first event 30 ms -> 1.9 ms) after.
- quartz: zero-decode REQ replay. Stored events now stream as RawEvent
(tags kept in serialized form) and are spliced directly into wire
frames — no tags parse, no EventFactory dispatch, no re-serialize per
row. Gated on the new IRelayPolicy.filtersOutgoingEvents capability:
policies that can veto per-event delivery (none today) keep the
materialized path; everyone else skips it. Live post-EOSE delivery is
unchanged (live matching needs Event objects).
- quartz: StatementCachingConnection wraps the pool's writer and reader
connections, replaying prepared statements instead of re-preparing per
event/REQ (eager reset on return keeps cursors from holding table
locks; a 256-statement cap bounds client-controlled filter-shape
variety). Ingest went 3,000 -> 4,700 events/s (+55%) — prepare
overhead was the single largest non-crypto write cost.
Net effect on the benchmark: ingest gap vs strfry narrowed from 2.2x to
1.8x, the firehose latency gap from 4x to 1.2x, and geode now wins 4 of
9 query-latency scenarios (notifications, hashtag, by-ids,
recent-window) plus most concurrent-throughput scenarios, while keeping
the smaller on-disk footprint. Result sets stayed byte-identical across
relays and NIP-77 sync still converges.
Measured but deliberately NOT taken: per-row SAVEPOINT elision (+4%,
within run noise — not worth weakening batch error isolation), FTS-off
(+25% ingest but drops NIP-50), --no-verify (+45% but unfair/unsafe).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
New :relayBench module that boots relay implementations as real external
processes under equivalent setups (persistent storage, sig verification on,
no auth) and compares them on the same corpus:
- Ingest: receipt->queryable-by-REQ latency (publish on one connection,
hammer-poll REQ{ids} on another), OK-ack latency, and pipelined corpus
replay throughput over N connections.
- Queries: client-realistic filters derived from the corpus (home feed,
thread, notifications, profiles, hashtag, by-ids, ...), time-to-EOSE
percentiles plus aggregate events/sec under concurrency; result-set
counts are cross-checked between relays and mismatches flagged.
- NIP-77 negentropy sync between every relay pair: 80%/80% slices with 60%
overlap, reconcile timing/rounds/wire-bytes per server side, delta
transfer, steady-state identical-set reconcile, convergence verified.
- Storage footprint after full ingest.
Corpora (all cached as NDJSON + manifest with a sha256 id fingerprint so
results are comparable across runs and machines):
- synthetic (default): deterministic to the byte — seeded keys, fixed
timestamps, seed-derived BIP-340 aux nonces — with a realistic social
shape (zipf authors, threads, reactions, reposts, zap request/receipt
pairs, hashtags);
- the checked-in real dump (quartz test fixture, ~31k unique 2024 events);
- external dumps (NDJSON or JSON array, gzip sniffed by magic bytes),
e.g. the 2.1M contact-list archive, with --max-event-bytes/--max-tags
raising both the corpus filter and the strfry config together;
- live download from public relays.
Every source runs through the same preparation: dedup, drop unsigned/
ephemeral/kind-5, enforce relay ingest caps, parallel Schnorr verify,
chronological sort.
relayBench/run.sh is the one-command entry point: builds geode + harness,
resolves strfry (STRFRY_BIN, PATH, or source build into .cache), runs the
suite and renders an ANSI report with per-metric bars and winners, plus
report.md and results.json under relayBench/results/<timestamp>/.
The harness client disables Nagle (TCP_NODELAY): with the JDK default, a
REQ following the previous round's CLOSE stalls a full delayed-ACK
interval and every latency floors at ~44 ms against both geode and strfry
(verified: ~0.3 ms with it off).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
Raw key:value tokens like include:spam reached SQLite FTS MATCH, where
the colon is column-filter syntax — any REQ carrying an extension token
died with CLOSED "no such column: include" instead of matching.
Adds SearchQuery.stripExtensions() plus Filter/List<Filter>
.strippingSearchExtensions() so EventStore users can drop the tokens
before querying, and applies them in LiveEventStore (query, count,
negentropy snapshots). Per NIP-50, unsupported extensions are ignored:
an extensions-only search becomes unconstrained, not match-nothing.
EventSource-backed search relays still receive the raw string since a
real search backend wants the extensions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tujoyfc2kNLZNVgiLZAR7F
Replaces the hand-rolled raw-WebSocket NIP-77 negotiate loop (single
un-windowed session — a strfry max_sync_events overflow was a hard
error) with quartz's negentropyReconcile: created_at window splitting
on overflow, keep-alive connection pinning, and streaming id batches.
Downloads and uploads now pipeline with the remaining reconcile
rounds: need-id batches feed 4 concurrent by-id drains, have-ids feed
an uploader (peak 7 subscriptions, under the common relay cap of 20).
Every downloaded event still funnels through the verify-and-store
path. Output field 'rounds' (protocol round-trips) is now 'windows'
(created_at splits).
Verified end-to-end against embedded geode relays: down-only 25/25,
up-only 5/5, and bidirectional re-runs converge to a zero diff.
Also records both adoptions in the perf plan doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Passes decoder = CachingEventDecoder() at all four NostrClient
construction sites: the Android app pool (AppModules), the Android
crawl client (buildCrawlClient — Event Sync / Cashu discovery, the
duplicate-heaviest path), the desktop RelayConnectionManager, and
amy's Context. Duplicate EVENT frames (14-57% of production traffic)
now skip the full JSON re-parse; dispatch semantics unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Same-JVM interleaved comparison against the resurrected spin-lock
implementation: single-threaded parity (the uncontended lock was ~free),
but 8 threads sharing one decoder run 3.0-4.5x faster lock-free — the
spin lock serialized the concurrent hit path just like the old global
PoolRequests lock did.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
The speedup assertion depends on AVAILABLE parallelism, which a
full-suite run (the pre-push hook) can eat — it flaked at 1.14x under
load. Now retries up to 3 measurement passes and enforces a hard 1.05x
floor that a genuine serialization regression (the per-event-async
version measured 0.94x) can never pass, warning instead of failing in
the noise band between floor and the 1.5x target.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Replaces the decoder's spin lock with lock-free concurrent maps so the
hot per-frame duplicate check never serializes across the pool's relay
consumer coroutines. New minimal ConcurrentHashCache expect/actual
(get/put/size/clear) following LargeCache's per-platform choices:
ConcurrentHashMap on JVM/Android, CacheMap on Apple, copy-on-write on
the CI-only Linux target. Counters become atomics; the generational
rotation keeps its deliberately-tolerated benign races, now documented
per failure mode (each is at worst a redundant re-parse, never a wrong
message).
CachingEventDecoderConcurrencyTest hammers one shared decoder from 8
threads with 80k duplicate-heavy frames and capacity 256 (rotations
fire constantly): zero wrong messages, exact parsed+reused accounting.
The 7 scan-safety tests and DedupDecodeBenchmark's enforced speedup
pass unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Code review confirmed each relay connection owns its own consumer
coroutine on Dispatchers.IO with no downstream funnel or shared lock
before justVerify (LargeCache is a ConcurrentSkipListMap; the
PoolRequests lock is per-subscription now), so multi-relay bursts
already verify in parallel across cores. Corrects the earlier follow-up
suggesting a CacheClientConnector integration: the accessory's scope is
single-connection bulk streams, plus a possible future dispatcher-
hygiene fix if on-device profiling shows CPU-bound verifies
oversubscribing the 64-thread IO pool.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Deliberate design decision reversing the 4096-frame receive bound:
1. The remote infrastructure isn't ours — TCP backpressure parks the
backlog in the RELAY's outbound buffers. A client should release the
relay from its duties as fast as it can send and own the buffering
itself.
2. The app holds 2000+ simultaneous relay connections; a bounded buffer
under a slow consumer blocks OkHttp reader threads, and at that
connection count blocked readers are a thread-starvation hazard far
worse than the heap growth they prevent.
The UNLIMITED channels now carry an explicit do-not-bound comment with
this rationale, and the slow-consumer risk is addressed from the other
side: keep the consumer faster than any relay's send rate
(CachingEventDecoder, ParallelEventVerifier, PoolRequests sharding).
BoundedReceiveBufferTest removed with the bound it tested; plan doc
records the decision.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Two server bugs masking each other made a single giant REQ crawl and
then wedge:
LiveEventStore's historical-replay dedup used an immutable Set under an
AtomicReference with copy-on-add — set + id copies the whole set per
streamed event, so large replays were accidentally O(n²) (100k-event
REQ: ~700 events/s, degrading as the response grew). Replaced with a
spin-lock-guarded mutable HashSet (same threads, single contains/add
per critical section).
Fixing that unmasked WebSocketSessionPump's slow-client policy: a fast
replay instantly overflowed the 8192-frame cap — which conflated 'slow
client' with 'replay outruns the socket writer', normal for bulk — and
the 'drop' only closed the internal queue, leaving the socket half-dead
(no EOSE, no close frame, tail silently missing: the likely cause of
the benchmark's 99,998/100,000). Producers are now paced against a full
backlog (bounded blocking wait, consistent with the documented ingest
fanout behavior) and only a client still behind after 30s is dropped,
by actually cancelling the socket.
GiantReqStreamTest guards the regression: 20k-event REQ pre-fix 8.4s
(~2.4k events/s), post-fix 0.7s (~27k events/s), all events + EOSE
delivered. 96 geode tests and the quartz relay/server suites pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
The per-connection-wall test prints the negotiated
Sec-WebSocket-Extensions header; against strfry OkHttp negotiates
permessage-deflate (client_no_context_takeover) out of the box, closing
the 'is compression actually on?' question from the optimization list —
it is, no change needed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
One reconcile feeds by-id download batches to N clients (one socket
each) x reqsPerClient workers; reconcile windows also round-robin
across the connections, since a single connection produced need-ids at
only ~9k/s on a 2.6M corpus and starved the downloads. Events funnel
through a bounded channel to a single consumer (exact maxEvents,
single-threaded onEvent); all stages backpressure; localEntries diffing
and have-counting match negentropyReconcile. reconcileWindows became
multi-client internally; single-client paths pass listOf(this).
Production shootout (same-run pairs, 100k cap): +18% to +64% over the
tuned single client, capped by the relay's server-side reconcile id
production rather than download parallelism (which the by-id matrix
shows scales 2.7x with connections). 4 in-process multi-client tests;
18 negentropy tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Client-side mirror of IngestQueue.parallelVerify: submit() is a cheap
bounded-channel send from the relay consumer coroutine; a drain loop
batches greedily (up to 256) and fans each batch across
Dispatchers.Default in core-sized chunks, dispatching callbacks in
submission order. preVerified short-circuits already-trusted ids; the
bounded channel backpressures the socket instead of growing heap.
Batch/chunk sizing is measurement-driven: per-event async cost ~40us of
scheduling each (swallowing the gain), and the per-batch join barrier
at 64 still cost half (64 -> 1.2x, 256 -> 2.2x, 1024 -> 3.2x on 4
cores). ParallelVerifyBenchmark (fresh signed events per pass — Event
caches derived state after first verify, so passes must not share
instances) measures 1.8x vs sequential with an enforced >=1.5x
assertion. 4 correctness tests cover valid/tampered routing, ordering,
preVerified and callback-crash resilience.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
The reader-thread-to-consumer channel in BasicOkHttpWebSocket and the
app's OkHttpWebSocket was UNLIMITED: a consumer slower than the socket
accumulated frame Strings without bound (gigabytes over a multi-million
event download). Now capped at 4096 frames — when full, OkHttp's reader
thread blocks and TCP flow control pushes back on the relay instead of
the heap. The trade-off (a blocked reader delays PING/PONG handling) is
documented on the constant; the bound is deep enough that only a
pathologically slow consumer hits it.
BoundedReceiveBufferTest forces sustained backpressure (8-frame buffer,
sleeping consumer, real socket to a local geode relay) and asserts all
events plus EOSE arrive in order with no drops or deadlock.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
BasicRelayClient's decode step becomes a pluggable MessageDecoder
(default unchanged). The opt-in CachingEventDecoder scans EVENT frames
for their id (~0.3us, JSON-escape-safe so embedded event JSON in repost
content cannot confuse it; any irregularity falls back to full parse)
and on a cache hit synthesizes the EventMessage from the already-parsed
Event with the frame's own subId — every subscription still gets its
delivery and per-relay bookkeeping is unchanged; only the redundant
parse is skipped. Production traffic measured 14-57% duplicate frames.
DedupDecodeBenchmark (60k frames, 67% dups): 10.0us/frame full parse vs
1.5us/frame cached — 6.5x, with an in-benchmark assertion so the gain
is enforced. 7 scan-safety unit tests in CachingEventDecoderTest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
The single global spin lock serialized every EVENT frame from every
relay and measured negative scaling (4 concurrent relay consumers
pushed 3.6M deliveries/s aggregate vs 11.1M for one thread alone). The
lock now lives in RequestSubscriptionState — one per subscription —
since all compound mutations are per-subId and different subs share no
wire state. decideCommandLocked takes the state instance to avoid
re-entering the non-reentrant lock; all-subs iterations lock one sub at
a time; withLock is inline to keep the hot path allocation-free.
DispatchStageBenchmark (PoolRequests-only, 1 -> 4 feeders): scaling
flips from 0.33x to 3.4-7.3x across runs. PoolRequests concurrency,
NostrClient and negentropy suites pass unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Splits the reconcile out of negentropySync so callers decide how to
load: negentropyReconcile streams needIds (relay has, local lacks —
download) and haveIds (local has, relay lacks — publish) in batchSize
chunks with back-pressure, taking local state as List<IdAndTime> and
slicing it per created_at window on overflow splits; the accumulating
negentropyReconcileIds convenience returns both lists. negentropySync
now delegates to the same window engine.
NegentropySession's primary constructor takes List<IdAndTime> (JVM
erasure forbids a List<Event> overload); the event-list form moved to
NegentropySession.fromEvents, mirroring NegentropyServerSession, with
all call sites migrated.
Adds NostrClientNegentropyReconcileTest (empty local set, partial
overlap both directions, identical sets, batch streaming, since/until
window slicing) — 49 negentropy tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Restructures the sync around the measured bottlenecks: one download
worker pool now spans the whole sync (windows no longer join before the
next reconcile starts), overflow-split windows are reconciled by a
caller-set number of concurrent NEG sessions (reconcileConcurrency,
default 1) from a shared work queue, and the reconcile-to-download
buffer depth is exposed (idBufferBatches). No NIP-11 auto-detection:
peak subscription usage (maxConcurrentReqs + reconcileConcurrency + 1)
is documented and budgeting it against the relay's max_subscriptions is
the caller's call.
All 44 negentropy tests pass unchanged. Production shootout on a 2.6M
corpus, single connection: 3.6k events/s with old-equivalent params,
4.5k/s tuned (12 reqs + 4 reconcilers) — ~82% of the measured ~5.5k/s
per-connection by-id ceiling, vs 1.6k/s for the old implementation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
The relay was backfilled from 33k to 2.6M kind-30382 events between
runs, and the picture changed: on the big corpus one connection
saturates at ~8 in-flight REQs (~5.5k events/s) and the relay tops out
at ~15k events/s around 40 total in-flight — past that, added
concurrency only inflates per-REQ latency. Axis 1 now stops at the
relay's NIP-11 max_subscriptions (20; exceeding it wedged the
connection in the first extended run), cells carry a hard 120s budget
with one retry per batch, and per-REQ latency is computed over
completed batches so partial cells report honestly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Downloads the full 33k kind-30382 corpus from nip85.nosfabrica.com per
cell over a (connections x concurrent REQs) matrix with pre-connected
sockets. One connection scales near-linearly to 16 in-flight REQs
(1,970 -> 30,735 events/s) with no wall at 4, and 1x16 matches 4x4 —
total in-flight REQs is the real variable. Every slow path measured so
far (serial page cursors ~2-3.7k/s, negentropySync 1.6k/s) is a
pipelining/serialization problem: negentropySync starves its download
workers on reconcile cadence and recurses overflow windows
sequentially. Findings and implied negentropySync improvements in the
plan doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
A raw no-parse socket and the full quartz stack page the same
production query on one connection at the same ~3.4-3.8k events/s,
proving the single-connection ceiling reported by a user is the
relay's per-connection response cadence rather than the client's
serial parse (which is ~1% busy at that rate). Findings and the
assessment of the proposed parallel-parse + ordered-dispatch pipeline
are in the plan doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Measures how to download millions of events from a single relay as fast
as possible, download+parse only: local geode ceilings (giant REQ vs
until-cursor paging vs created_at-sharded connections, quartz stack vs
raw frames), offline per-frame strategies (full parse, parallel parse,
id-scan for raw archiving), and a production case syncing kind 30382
from nip85.nosfabrica.com via NIP-77 negentropy against plain paging.
Headline results in the plan doc: paging beats giant REQs 26x (which
also dropped frames), created_at sharding stacks ~2x on top, parse is
2-5% of the budget, and negentropy is 2.4x slower than paging for a
cold download (its win is incremental re-sync).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Offline microbenchmark of NostrClient's dispatch stage — PoolRequests
state machine, listener fan-out, id dedup and handoff to a verify
stage — under 1 and 4 concurrent relay feeders. Key results: ~100ns per
message uncontended, but negative scaling under concurrency (the
PoolRequests busy-wait spin lock makes 4 feeders slower in aggregate
than 1), per-event channel handoff costs ~180ns vs a free 64-batch,
and early dedup before the locked path gives 2.7-4x aggregate
throughput at production duplicate factors. Findings appended to the
receiver-perf plan doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Gated JVM test (-PprodRelayBench=1) that connects to live relays with
realistic filters and measures per-relay queue delay, processing time,
consumer busy fraction, EOSE latency and duplicate rates, comparing the
current inline verification against a parallel verify stage, plus
offline single-thread parse/verify ceilings on captured frames.
Findings from a first run are written up in
quartz/plans/2026-07-02-nostrclient-receiver-perf.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Part A of the dispatchers/thread-caps audit — the one genuine at-scale
starvation the audit found.
UdpSocket.receive() does a blocking DatagramChannel recvfrom that parks its
thread for the ENTIRE life of the connection. It ran via
withContext(Dispatchers.IO) from a read loop already on Dispatchers.IO, so
the blocking call pinned one shared IO-pool thread per connection. Past ~64
concurrent connections that starves ALL other Dispatchers.IO work in the
process — this module's and the host app's alike.
Give each socket two dedicated daemon threads: recvDispatcher for the
perpetually-parked receive and sendDispatcher for the send (they can't share
one thread — the receive would monopolise it). QUIC's blocking socket I/O now
never touches the shared pool. connect() keeps its one-shot DNS/bind on
Dispatchers.IO (setup cost, not a lifetime parker).
close() calls shutdownNow() on both executors: interrupting the recv worker
breaks the parked recvfrom immediately (ClosedByInterruptException, caught as
ClosedChannelException -> receive() returns null), so the threads exit
promptly instead of leaking per closed connection. The closed-check is hoisted
out of withContext so a post-close call fails fast without dispatching onto a
shut-down executor.
Verified: QuicConnectionDriverLifecycleTest (100 session open/close cycles,
asserts thread growth <=16 and no FD leak) passes, confirming the two new
threads per socket are reclaimed on teardown. New UdpSocketTest covers
round-trip, the dedicated-thread isolation, thread shutdown on close, and the
after-close contract. Full :quic:jvmTest suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ANuUziXKRafSTBxbh4SMoq
Complete the "Let's be reasonable" content set:
- reports (1984) and torrents (2003/2004) — additive public events whose
reputational weight is no greater than the arbitrary kind-1 notes an app
can already publish.
- long-form articles (30023), wiki (30818), and the legacy addressable
video kinds (34235/34236) — addressable content. Re-signing with the same
d tag replaces the app's own prior version; accepted as no worse than the
arbitrary posting a kind-1 grant already permits.
Only replaceable *configuration* (profile 0, contacts 3, 10000-range lists)
stays ASK, since a bad write there can silently wipe account settings —
distinct from addressable content.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WMrHZfwN5tM4ecvdz7xigo
Add the remaining regular, additive, public, plaintext content and
engagement kinds that sit in the same risk class as notes/pictures:
relay chat (9), threads (11), public messages (24), poll votes (1018) and
polls (1068), file metadata (1063), voice messages (1222) and replies
(1244), live-stream chat (1311), and code snippets (1337).
Documents the borderline kinds left at ASK on purpose: reports (1984) and
torrents (2003/2004) carry reputational/legal weight, and
addressable/replaceable content (long-form 30023, wiki 30818) can overwrite
prior versions. Test pins several of these exclusions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WMrHZfwN5tM4ecvdz7xigo
Part B of the dispatchers/thread-caps audit.
EventDeduplicator is fed from relay subscription callbacks
(AdvancedSearchBarState.trackRelayEvent), which arrive on multiple threads
concurrently. It backed a plain mutableSet with a single KmpLock, so every
delivery from every relay thread serialized on one monitor.
Add ConcurrentSet<E> as a KMP expect/actual util:
- jvmAndroid actual: ConcurrentHashMap.newKeySet() — lock-striped writes,
lock-free reads, no single cross-thread monitor.
- iOS actual: a KmpLock-guarded set (no lock-free set in the K/N stdlib) —
same behaviour as before, no regression. The win lands on JVM/Android,
which is where the high-throughput event paths run.
Point EventDeduplicator at it. Covered by ConcurrentSetTest (commonTest,
behaviour) and ConcurrentSetConcurrencyTest (jvmTest, exactly-one-add-per-key
under 8 threads).
Scope note: the other two commonMain sites the audit flagged were left as-is
on purpose. The compose subscription managers' KmpLock is a deliberate,
documented KMP choice on a single (main-thread) writer where the lock is
effectively free; EOSECache is a bounded LRU with compound value mutation on
a per-subscription (not per-event) path. Neither is a clean fit for a
concurrent set, and converting them would fight a documented decision for
negligible gain.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ANuUziXKRafSTBxbh4SMoq
Part C of the dispatchers/thread-caps audit. Both LnurlEndpointCache and
DesktopCachedRichTextParser were bounded caches backed by a LinkedHashMap
behind a single monitor (@Synchronized / Collections.synchronizedMap with
accessOrder). An access-order map structurally mutates on get, so every
read took the lock — serializing all readers on paths that are hot
(kind-9735 zap-receipt validation; feed rich-text rendering).
Add ConcurrentLruCache<K, V> in quartz utils: storage is a
ConcurrentHashMap so get is lock-free; writes + eviction run under a small
write lock that is off the read path. Eviction is least-recently-put order
(get does not refresh recency) — exactly what LnurlEndpointCache already
did, and fine for the deterministic rich-text parse cache.
Point both caches at the shared helper. Covered by a new
ConcurrentLruCacheTest (round-trip, eviction order, re-put recency
refresh, get-does-not-refresh, clear, and a concurrent size-bound smoke
test); the existing LnurlEndpointCacheTest still passes unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ANuUziXKRafSTBxbh4SMoq
Part of the dispatchers/thread-caps audit. Two low-risk fixes that remove
thread-blocking work from paths hit on every event / every packet:
- LargeCache (iOS actual): drop the runBlocking wrapper around
createIfAbsent. The block contained only synchronous CacheMap ops (the
same get/put getOrCreate already calls without runBlocking), so it was
pure dispatcher-blocking overhead on the per-event ingest path. Now
mirrors the JVM actual's plain-function shape.
- QUIC JCA AEADs (AES-GCM + ChaCha20-Poly1305): split the single
`synchronized(this)` monitor into disjoint encryptLock / decryptLock.
seal-family touches only encryptCipher + recentEncryptNonces; open-family
touches only decryptCipher, so a connection's send loop and read loop no
longer serialize against each other through crypto on every packet. The
documented defence-in-depth against cross-coroutine Cipher corruption is
preserved per direction.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ANuUziXKRafSTBxbh4SMoq
Add video posts (kinds 21 normal, 22 short) as direct siblings of picture
posts (20) — additive, public, non-destructive content in the same risk
class as the original note set.
Also auto-approve NIP-42 relay auth (22242): an ephemeral proof-of-key
bound to a single relay + challenge (unreplayable elsewhere) that
Amethyst's own client already auto-signs for every logged-in account, so
treating it as background noise for napplets matches existing behavior.
Deliberately still ASK: NIP-98 HTTP auth (27235). Unlike relay auth it
authorizes an arbitrary HTTP request as the user — including destructive
NIP-96 blob deletes and NIP-86 relay-management admin calls — so its blast
radius is too broad to sign silently. Test pins the 42-vs-98 contrast.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WMrHZfwN5tM4ecvdz7xigo
Signing a Lightning zap request moves no money — it only fetches an
invoice. The payment itself is the separately-gated value.payInvoice
capability, which prompts on every use regardless of policy. So adding
9734 to the reasonable set drops a redundant signature prompt while the
meaningful payment prompt stays.
Nutzaps (9321) remain excluded: publishing one *is* the payment, since
the event carries the spendable ecash proofs. Test pins the contrast.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WMrHZfwN5tM4ecvdz7xigo
Add more additive, public, non-destructive event kinds to the REASONABLE
signer policy so common apps stop prompting for every action. New kinds:
16 (generic repost), 20 (picture post), 42 (public chat message),
1111 (NIP-22 comment), 9802 (highlight), and 30315 (user status) — all in
the same risk class as the original 1/6/7 set.
Kinds that can spend money, overwrite account config (profile, contacts,
relay/mute/bookmark lists), delete content, or leak private data still
prompt. Decryption also stays ASK.
Refactors reasonableDecision() to a documented REASONABLE_SIGN_KINDS set
backed by quartz KIND constants, and adds NostrSignerPermissionLedgerTest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WMrHZfwN5tM4ecvdz7xigo
printStackTrace() dumps straight to stderr, bypassing both Log.minLevel
and the consumer's Log.sink — the very thing the LogSink work exists to
control. Migrate the five production call sites:
- Lud06: drop two printStackTrace() calls that sat directly above an
existing Log.w(..., t) carrying the same throwable (pure duplication).
- ElectrumXClient: the swallowed-lookup catch said "Log but don't crash"
yet used printStackTrace(); route it through Log.w with context.
- OpenTimestamps: log the swallowed merge failure via Log.w; drop the
print-then-rethrow (the rethrown exception already carries the trace).
Socket-protocol writer.println(...) and README/KDoc println examples are
left as-is — they are wire I/O and documentation, not logging.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EK3TrDkP1EXj1d62oKJMdc
Quartz already funnels every diagnostic through the `Log` facade, but the
sink was hardcoded per platform (android.util.Log / System.err / NSLog /
println), so a consuming app couldn't route Quartz logs into its own stack
(Timber, SLF4J, Crashlytics, a file, a test buffer, or /dev/null).
Add a `LogSink` fun interface and a replaceable `Log.sink`, defaulting to
`PlatformLogSink` which reproduces the historical per-platform behavior.
All ~225 call sites and the `Log.*` signatures are unchanged; the lazy
`() -> String` overloads still short-circuit on `minLevel` before the
lambda runs, preserving the allocation-free fast path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EK3TrDkP1EXj1d62oKJMdc
Adds a Signature field to Compose Settings (global UI settings, DataStore
persisted). When opening any text-based composer — new note, reply, quote,
poll, NIP-22 comment (reply/hashtag/geohash/url), or a new long-form
article — the signature is appended to the message with a blank line,
keeping the cursor at the start so the user types above it.
Drafts, forks, and version edits are skipped since their content already
carries (or deliberately omits) a signature, and an untouched
signature-only message is treated as blank so closing the composer never
auto-saves a junk draft.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vq4JQPB9m62nJ8Vdp7xVLN
The "Good options are: …" hint in the search-relay setup dialog listed a
hardcoded, drifting subset of relays (and the zh locales had corrupted
hostnames like "reiny.nostr.band"). Make it dynamic instead:
- search_relays_not_found_examples now ends in a %1$s placeholder across
all 57 locales.
- AddSearchRelayListDialog fills it from
DefaultSearchRelayList.joinToString { " - ${it.displayUrl()}" }, so the
hint always reflects the shared AmethystDefaults search set and stays in
sync with the "reset to defaults" button right below it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Ttcqa3V78bugGraGhtehj
The embedded browser's top pull-down (TopControlSheet) and bottom pull-up
(BottomConsoleSheet) both drew with tonalElevation. Material 3 recolors a
Surface only when its color is EXACTLY colorScheme.surface, swapping in
surfaceColorAtElevation() which blends surfaceTint (= primary, the Amethyst
purple) over the surface. On the light theme that near-white + purple mix
reads as a bright pink/lilac cast instead of the plain background.
- TopControlSheet: drop tonalElevation to 0 so it renders the plain
background color; keep shadowElevation to lift it off the page.
- BottomConsoleSheet: drop both elevations. It docks flush against the
bottom navigation bar, so shadowElevation cast a shadow onto that bar (a
seam breaking the flush look); the tonalElevation had no color effect
anyway since its color isn't exactly colorScheme.surface. The grabber and
divider still separate the panel from the page above it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TJ2VbDz7C6hvHVsLpdecYf
relay.nostr.band has been decommissioned. Remove it from every runtime
relay list and route search-relay defaults through the shared
AmethystDefaults.DefaultSearchRelayList in commons:
- amy NipCommand: SEARCH_RELAYS now = DefaultSearchRelayList (drops the
hardcoded relay.nostr.band/nostr.wine pair; RelayUrlNormalizer import
no longer needed).
- desktop DesktopRelayCategories: DEFAULT_SEARCH_RELAYS now =
DefaultSearchRelayList instead of a single relay.nostr.band entry
(which would otherwise be empty after removal).
- desktop DefaultRelays and FollowPacks DISCOVERY_RELAYS: drop
relay.nostr.band.
- Update NIP-50 example hostnames in desktop comments, the search-relay
editor help text, and the localized search_relays_not_found_examples
string across all locales to nostr.wine.
Preview sample data, captured sample-event JSON, and quartz test
fixtures that mention relay.nostr.band are left untouched (no runtime
effect).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Ttcqa3V78bugGraGhtehj
relay.damus.io is being decommissioned, so remove it from every runtime
default/fallback relay set to stop the app and amy from wasting connection
slots on a dead host:
- commons Constants: remove `damus`; it dropped out of `bootstrapInbox`
(default NIP-65 inbox) and `eventFinderRelays` (default outbox/fallback),
both still carrying 6 healthy relays.
- ChessConfig: remove damus from CHESS_RELAYS / CHESS_RELAY_NAMES, leaving
the 3 relays the FETCH_TIMEOUT comment already assumes.
- desktop DefaultRelays: remove damus and the also-dead relay.snort.social.
- desktop FollowPacks DISCOVERY_RELAYS: remove damus.
- amy NipCommand SEARCH_RELAYS: swap damus for the NIP-50-capable nostr.wine.
Comments, @Preview sample data, and test fixtures that mention damus.io are
left untouched — they have no runtime effect.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Ttcqa3V78bugGraGhtehj
Fold the full-text-search on/off switch into `IndexingStrategy` as
`indexFullTextSearch` (default `true`) instead of a separate top-level
`enableFullTextSearch` constructor param on `EventStore`/`SQLiteEventStore`.
`IndexingStrategy` is already the single place that decides which indexes
the store builds — every field is a per-index toggle with a size/speed
tradeoff, and `QueryBuilder` already receives it — so FTS, being just
another index, belongs there rather than split across two config surfaces.
Behaviour is unchanged: the module's no-op path and the QueryBuilder
"search matches nothing" guards now read the flag via the strategy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjzUpY8H31c7ux669zytWg
Add an `enableFullTextSearch` flag (default `true`) to `EventStore` and
`SQLiteEventStore` so deployments that never serve NIP-50 search from
SQLite — e.g. a relay that offloads search to an external engine like
Vespa — can skip the full-text-search write cost.
When disabled:
- `FullTextSearchModule` becomes an inert no-op: the `event_fts` virtual
table and its `fts_foreign_key` delete trigger are never created,
inserts skip `indexableContent()` + tokenization, and both reindex
entry points return immediately.
- `QueryBuilder` short-circuits any query/count/delete filter carrying a
non-empty `search` term to a "matches nothing" result (an empty-string
search still imposes no constraint), so no SQL ever references the
absent `event_fts` table. In a multi-filter union the search branch
contributes nothing while the other filters resolve normally.
Everything else (replaceable/addressable handling, deletions,
expirations, right-to-vanish, negentropy) is unchanged, and the default
keeps FTS on for existing callers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjzUpY8H31c7ux669zytWg
Kotlin/Native (the iOS test target) rejects commas in backtick function
names, so test-quartz-ios failed to compile even though jvmTest — which
allows them — passed. Rename the two offending tests. No behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
The bottom-bar settings screen already offers a "Restore defaults" action, but it
saved the concrete default list to disk. That pinned the user to the default of
the version they reset on, so a future release that changes DefaultBottomBarEntries
would never reach them. The same happened to users who never touched the bar: any
unrelated settings save wrote the concrete default list.
Persist "follow the defaults" as a blank sentinel instead. On load a blank value
already resolves to the current DefaultBottomBarEntries, so whenever the default
changes in a later version, every user on the defaults is migrated automatically.
Genuinely customized bars are still stored verbatim as JSON.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011EBk58qUYuyKxz9Ju4VpEq
Replace the removed per-episode comment chip with the full NoteCompose
ReactionsRow (comment / zap / react) on every episode row in a podcast's
screen — same engagement affordance as the show header and every other note.
addPadding = false so it aligns within the row's existing padding.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Add a "NIP-11 Relay Information Document" section to the quartz-integration
skill (the guide AI consults when using Quartz in external projects),
covering the relayInformation { } DSL, serving it over
application/nostr+json, the nested limitation/fees/retention builders, and
the limitation(RelayLimits) sync overload. Also add Quick Reference rows
and extend the skill trigger description to the run-a-relay case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C64Yy2d3na7Y28u7GRrHMX
Relay operators wiring up a Quartz-based relay had to hand-write the
NIP-11 document as a large JSON string, which is error-prone and drifts
from the model. Add a DSL builder so they can describe the document in
Kotlin instead:
val info = relayInformation {
name = "sot"
description = "NIP-50 profile search ranked by Nostr web-of-trust"
software = "https://github.com/vitorpamplona/sot"
version = "0.1"
supports(1, 11, 42, 50)
}
call.respondText(info.toJson(), ContentType.parse(Nip11RelayInformation.CONTENT_TYPE))
The builder covers every field, with nested `limitation { }` / `fees { }`
DSLs, repeatable list helpers, a `retention(...)` entry adder, and a
`limitation(RelayLimits)` overload that advertises exactly the limits the
relay enforces so the two can't drift.
Also fix FlexibleIntListSerializer to emit numeric `supported_nips` as
JSON integers ([1,11,42,50]) instead of quoted strings, matching the
NIP-11 spec; non-numeric ids still fall back to strings. Geode's default
document now builds via the DSL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C64Yy2d3na7Y28u7GRrHMX
AI agents were hand-rolling their own hex/npub/nprofile/NIP-05 resolvers
(e.g. a bespoke `resolveObserver` with its own well-known fetch and JSON
parse) instead of using `resolveUserHexOrNull` in
`quartz/nip05DnsIdentifiers/`, which already handles every identifier form.
Add discoverable explainers so the canonical functions surface before an
agent reaches for a hand-rolled version:
- New reference `references/nip05-identifiers.md`: full API surface
(`resolveUserHexOrNull`, `Nip05Client`, `Nip05Id`, `Nip05Parser`,
`KeyInfoSet`, Namecoin `.bit`, `OkHttpNip05Fetcher`) plus the
hand-rolled anti-pattern to avoid and the ✅ replacement.
- SKILL.md: new "Resolving User Input to a Pubkey" section, a
When-to-Use bullet, Bundled Resources + Quick Reference rows, and the
identifier-resolution trigger added to the skill description.
- nip-catalog.md: fix the stale NIP-05 entry (referenced a nonexistent
`Nip05Verifier.kt`) to point at `UserHexResolver.kt` / `Nip05Client.kt`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bb8txetygxSnLXHJhPZSwH
Introduce OptionalAuthPolicy, a relay-server policy that runs the full
NIP-42 challenge/verify handshake — emitting the AUTH challenge on connect
and recording verified pubkeys into the connection scope — but never
requires it: EVENT, REQ, and COUNT are always accepted, so clients that
ignore the challenge keep working.
It subclasses FullAuthPolicy and only relaxes the EVENT/REQ/COUNT gates, so
the authorize() hook and per-connection authenticatedUsers set behave
identically; downstream policies can still gate or rewrite on caller
identity. Wire it into geode via an optional_auth config option and a
--optional-auth CLI flag (ignored when require_auth/--auth is set, since
mandatory AUTH already sends the challenge).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJWy4dBBYLbqthhLvbcZh7
Brings three of PodStr's engagement/reading widgets to Amethyst's podcast
screens, reusing existing infrastructure:
- Top Supporters — a sats-ranked zap leaderboard on the show header, top 3
flagged with gold/silver/bronze medals, tap-through to profiles. Aggregates
the show note's zaps through the same LiveActivityTopZappersAggregator the
live-stream leaderboard uses. Anonymous zaps collapse into one bucket.
- In-app Chapters — fetches the Podcasting-2.0 chapters.json referenced by the
episode's `chapters` tag and renders a collapsible, tappable list; tapping a
chapter seeks the live media controller (same seek path as soundbites).
- Transcript viewer — fetches the `transcript` file and shows it in a
collapsible scrollable panel, stripping VTT/SRT scaffolding into flowing text.
Adds PodcastRemoteContent (a bounded URL text fetcher) for the two off-event
side files. Also removes the now-redundant per-episode comment chip from the
episode list rows — the show ReactionsRow and the episode thread already cover
commenting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Add a PreToolUse hook (.claude/hooks/pre-push-spotless.sh, wired in
.claude/settings.json) that runs spotlessApply before a git-push subcommand
or the create_pull_request MCP tool. If spotless reformats tracked Kotlin,
or reports a non-autofixable lint, the call is blocked so the fix is
committed first -- turning CI's spotlessCheck failure into an in-session
block. Gradle infra/network failures warn and allow, leaving CI as the
backstop. Boundary detection tokenizes the command so the words appearing
in a commit message do not trip the gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nWg1Wwq5xCE7kPFzWY95o
New KDoc blocks on isHex/isHex64 already state the "~47ns" and
"~30% faster" perf notes, so the trailing EOL comments between the
KDoc and the function now trip ktlint's standard:no-consecutive-comments
rule ("an EOL comment may not be preceded by a KDoc"). Remove them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BawgXifvcPidMMqJ719Ka2
The download/save-to-local button handed the raw content URL straight to
OkHttp. For BUD-10 `blossom:` URIs this failed with "expected scheme http
or https but was blossom" because OkHttp only speaks http/https.
Resolve `blossom:` URIs to a concrete server URL via BlossomServerResolver
(the same resolver the Coil/ExoPlayer pipeline uses) before downloading, in
both save entry points (AccountViewModel.saveMediaToGallery and the zoomable
dialog's save action). When no hosting server can be found, the save reports
an error instead of crashing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BawgXifvcPidMMqJ719Ka2
When a Podcasting-2.0 person's href points at an npub/nprofile (bare,
nostr: URI, or an njump-style link), upgrade the free-text credit to a real
Nostr profile: the standard ClickableUserPicture + UsernameDisplay, tappable
through to the profile. Plain web links keep the free-text card with the
default profile-image loader.
- quartz: PodcastPerson.nostrPubKey() resolves href → pubkey via Nip19Parser
(npub/nprofile only). Covered by PodcastPersonSoundbiteTest.
- UI: PodcastPeople branches per person — LoadUser + standard profile
components for nostr identities, free-text card otherwise — sharing one
card scaffold so both look identical in the Hosts & Guests strip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Same discoverability treatment as the Hex helpers: KDoc on the reusable,
heavily-used primitives external integrators and AIs kept missing, plus
skill coverage.
- TimeUtils: object + key-fn KDoc emphasizing Unix *seconds* (created_at)
vs the millisecond nowMillis() exception.
- RandomInstance: object + per-fn KDoc (secure random; use over kotlin.random).
- sha256(): KDoc pointing event-id work to EventHasher.
- EventHasher: object + fn KDoc on canonical id serialization / verification.
- StringUtils: KDoc on the allocation-free case-insensitive matchers + DualCase.
- Bech32: "use NIP-19 helpers unless you need a custom prefix" pointer; KDoc
on bechToBytes.
- quartz-integration skill: new "Everyday utilities" section (time/random/
hashing/bech32/base64) + quick-reference rows.
- nostr-expert skill: "Core Utilities" section.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011HM5uueF4a17umGpjC8wcz
AI agents integrating Quartz were re-implementing hex encoding or pulling
in third-party codecs because the built-in helpers weren't discoverable.
- Add KDoc to the `HexKey` typealias, its extension functions
(`toHexKey`/`hexToByteArray`/`hexToByteArrayOrNull`/`isValid`) and the
`PUBKEY_LENGTH`/`EVENT_ID_LENGTH` constants.
- Add KDoc to the `Hex` object and its public API
(`isHex`/`isHex64`/`decode`/`encode`/`isEqual`).
- Add a dedicated "Hex utilities" section + quick-reference rows to the
quartz-integration skill, and fix the wrong `HexKey.decodeHex(hex)`
snippet (no such API) to the real `hex.hexToByteArray()`.
- Add a "Hex Encoding" section to the nostr-expert skill.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011HM5uueF4a17umGpjC8wcz
Adds two Podcasting-2.0 features to the podcast stack:
Persons (podcast:person) — hosts/guests as free-text credits with role,
avatar, and link (not necessarily Nostr users):
- quartz: PodcastPerson model; PersonTag ["person", name, role, img, href]
on kind 30054 episodes; a persons[] array in the kind 30078 show JSON.
Exposed via PodcastEpisode.episodePersons() / PodcastShow.showPersons().
- UI: PodcastPeople — a "Hosts & Guests" avatar strip (robohash fallback,
tap opens href), shown on the episode card and the show header.
Soundbites (podcast:soundbite) — highlight clips:
- quartz: PodcastSoundbite model; SoundbiteTag ["soundbite", start, dur,
title?] on episodes; PodcastEpisode.episodeSoundbites().
- UI: PodcastSoundbites — "jump to the good part" chips under the audio
player that seek the live media controller to the clip's start.
Both parse leniently, round-trip through build(), and are covered by
PodcastPersonSoundbiteTest. NIP-F4 returns empty for both (no such tags),
so nothing renders there.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Give the podcast show itself the usual engagement affordance: the standard
NoteCompose ReactionsRow (comment / zap / react, with counts) now sits
between the show header and the episode list on the detail screen, bracketed
by the usual dividers like any other content detail. Acts on the resolved
show note and only renders once that event loads.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Episode comments already work end to end — replying to an episode routes to
the NIP-22 GenericCommentPost composer (kind 1111 scoped to the episode
address), the thread datasource fetches them via #a/#A, and the thread view
renders them with a full reactions row. The only gap was discoverability on
the podcast-specific surfaces.
Add a comments chip (comment glyph + live reply count via
observeNoteReplyCount) to PodcastEpisodeListItem. It reads "N comments" (or
"Comment" when empty) and opens the episode's thread, where the discussion
lives and new comments are composed. Reuses the existing thread/composer
machinery — no new event plumbing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
The single-podcast screen's top bar now shows the standard MoreOptionsButton
(the NoteCompose 3-dot menu) to the right of the bookmark button, giving the
show/episode the usual note actions (share, report, mute, etc.). Both appear
only once the show metadata has resolved.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
- The bookmark toggle was a Material IconButton (48dp minimum touch
target), so it stood taller than the titleLarge line it sits beside and
broke the side-by-side alignment. Render it as a compact clickable glyph
sized to a single title line (iconSize, default 20dp) so title and button
align.
- Drop the add/remove toast — the icon flipping filled/outline is enough
feedback — and remove the now-unused strings.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
IdleClock.bump() is called for every message the relay sends — the connection
listener bumps it per event, so a multi-million-event download bumped it millions
of times. It stored a ValueTimeMark into an AtomicReference, and since the value
class boxes when used as a generic type argument, every bump allocated a heap
object. That is needless GC pressure on the hottest path (and battery/jank on
Android).
Replace it with a single monotonic base mark taken once (stored unboxed) plus a
@Volatile Long of nanos-since-start updated on each bump — zero allocation per
bump, and only visibility (not atomicity) is needed since each relay's bumps come
from its single reader thread and the driver only reads. Behavior is unchanged;
negentropy + concurrency suites pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
The fixed `timeoutMs` (default 30s) applied to every reconcile round, but the
FIRST round on a large relay is a legitimate long silence while the relay builds
its whole negentropy fingerprint — observed at ~63-68s for a 3.5M-event kind:0
set on a real relay. So the old default spuriously failed big syncs with
UNAVAILABLE ("reconcile round timed out"), even though the relay was working fine
and would have answered seconds later.
Replace it with `idleTimeoutMs` (default 120s): the maximum time the relay may go
COMPLETELY SILENT before giving up. It resets on every message the relay sends —
each NIP-77 round and every download EOSE/event — and on connect, so a genuinely
slow but progressing sync runs for as long as it needs; only true silence trips
it. Because the watchdog is fed by a connection-level listener that sees all of
the relay's traffic, download activity extends the reconcile deadline and vice
versa. `idleTimeoutMs = 0` disables it entirely (run until the socket drops); the
initial connect and each download batch keep their own finite bounds so an
unreachable relay or a single stuck batch still can't hang the pipeline.
Liveness of a dead/half-open socket does not depend on this: the WebSocket
keep-alive (ping/pong) detects it and the disconnect is already turned into a
clean NEG-ERR abort.
Verified against wss://wot.grapevine.network: the first round took 68.4s (the old
30s default would have thrown UNAVAILABLE) and the sync sailed through it under
the 120s idle window. Negentropy unit suite passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
PodcastBookmarkButton read `note in bookmarks.public` — an identity-based
List<Note> containment that is unreliable for addressable shows/episodes
(the rendered note instance may differ from the one rebuilt from the
a-tag), so the icon never flipped and there was no way to tell a podcast
was already bookmarked.
Switch to the public bookmark id/address sets (the same reactive pattern
the working git-repository bookmark button uses): match note.address for
addressable notes and note.idHex otherwise. The icon now reliably shows
bookmarked vs not (filled/primary vs outline/onSurfaceVariant), and each
tap fires a confirming toast so the action is never silent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Revert the fresh-subId-per-page workaround (ed5c25e2) now that the underlying
double-REQ race is fixed at the root in PoolRequests. Relays cap the number of
concurrent subscriptions per connection, so a single reused id — opened per page
with the page's `until`, closed before the next page — keeps the whole download
to one subscription slot instead of churning through a distinct id each page.
Safe because the pool now serializes the "send a REQ" decision: after a page's
EOSE, the auto-resend and the loop's unsubscribe+resubscribe can no longer both
fire a REQ for the same id (guarded by PoolRequestsConcurrencyTest). Unit suites
(negentropy paging scenarios, subscriptions) pass; full-scale real-relay
verification to follow before push.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
A subscription id is driven from two threads at once: the app thread (the
subscribe/unsubscribe path) and every relay's socket-reader thread (an EOSE
that triggers an auto-resend). Both read the subscription state and both can
decide "the filters changed, send a REQ", but the decision (read state) and the
send (mark state SENT in onSent) were not atomic. So the reader could observe
the pre-send state — filters still on the previous value — while the app had
already moved the desired filters forward, and both would send a REQ for the
same sub id.
Two REQs on one id race on the wire: the relay answers with duplicate EOSEs and
events, or — if a CLOSE interleaves — an empty result that silently truncates a
paged download. This is what intermittently broke fetchAllPages on large sets
(fixed at the call site in ed5c25e2 by using a fresh sub id per page); this
commit fixes the underlying race in the relay-client layer, which could equally
corrupt any subscription that spans multiple relays (several reader threads
mutate the same RequestSubscriptionState maps concurrently).
The fix:
- Add a tiny non-reentrant spin lock (withStateLock, same AtomicBoolean
primitive BasicRelayClient uses) guarding every access to the subscription
state machine. Listener callbacks and socket sends stay OUTSIDE the lock —
they re-enter this class via onSent, so holding it across them would deadlock.
- Fold the send decision into decideCommandLocked, which runs under the lock and
pre-marks the state SENT (+ filters) the moment it decides to send a REQ. A
concurrent decider then sees SENT/updated filters and declines, so exactly one
REQ is ever produced.
Verified with a deterministic A/B repro that pins the exact interleaving open:
pre-fix 300/300 episodes produced a duplicate REQ; post-fix 0/300 (max one REQ
per episode). Kept as PoolRequestsConcurrencyTest. Full relay test suite passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
fetchAllPages reused a single subscription id across all pages
(unsubscribe + immediately re-subscribe the same id). On a real relay that
caps REQ results, the rapid same-id CLOSE→REQ races on the wire: in-flight
events from the previous page's REQ bleed into the next page's listener.
Those stale events carry a created_at above the freshly-lowered `until`, so
`match()` rejects them, the page ends with pageCount == 0, and the whole
loop breaks — silently truncating the download.
Observed against wss://wot.grapevine.network: a full kind:0 download (~3.55M
events, per a concurrent negentropy sync) stopped at 89,500. A controlled
diagnosis paging the same data with a fresh subId per page vs a shared subId
reproduced it exactly: shared stalled at ~95k with in-page duplicates and
events above `until`; fresh advanced cleanly with no duplicates. After the
fix, the real-relay fetchAllPages sails past the old stall (100k+ and
counting).
Fix: allocate the subId inside the paging loop so each page is an
independent subscription.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
Rework the negentropy download path from "reconcile fully → then download"
into a single back-pressured streaming pipeline so peak memory is independent
of the window size — built for multi-million-event syncs.
- reconcileStreaming drives the NIP-77 rounds directly (instead of via
NegentropyManager) and hands each round's ids to a bounded id-queue *before*
acking the next round, so the relay's id stream is paced to the downloader.
- Ids flow id-queue → bounded download worker pool → bounded delivery channel;
a slow consumer back-pressures the whole chain. The full id list is never
materialised.
- Drop the global event-dedup set (was O(set) ~ hundreds of MB at 4M): NIP-77
yields a distinct id set, so each event is requested once. Keep only a tiny
per-batch dedup (bounded by fetchBatch) to absorb a relay replaying a REQ.
- Pin the relay with a never-matching keep-alive subscription for the sync's
duration: a NEG-OPEN isn't a REQ, so during a reconcile round the pool would
otherwise see the relay as unwanted and disconnect it mid-sync.
- Document that timeoutMs must accommodate the relay's first-frame snapshot
latency on huge sets (a real strfry took ~73s for an unbounded kind:0 set).
Validated against wss://wot.grapevine.network: streamed 30k kind:0 events with
heap bounded at ~30-90 MB (not growing with the set) and zero duplicates. New
unit test forces many small reconcile frames to exercise the multi-round /
back-pressure path; existing windowing/cap/fallback tests still green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
Adds a Podcasts row to the Bookmark lists screen (mirroring Git
Repositories), opening a feed of just the bookmarked podcasts:
- quartz: isPodcastEvent() — a reusable predicate matching shows,
episodes (NIP-F4 + Podcasting-2.0) and trailers, used to pull the
podcast subset out of the mixed NIP-51 kind:10003 bookmark list.
- BookmarkPodcastsFeedFilter / ...FeedViewModel / BookmarkedPodcastsScreen
filter the bookmark list (public + private) down to podcast notes,
newest first, and render them with the standard podcast cards.
- Route.BookmarkedPodcasts + AppNavigation wiring; a "Podcasts" row with a
live count in ListOfBookmarkGroupsFeedView.
- The single-podcast detail screen (PodcastScreen) gains a bookmark action
in its top bar that reflects and toggles bookmarked state (reusing the
stateful PodcastBookmarkButton). TopBarExtensibleWithBackButton now
accepts an actions slot to host it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
The standard zap button now detects a Podcasting-2.0 value-for-value block
on a podcast note (episode or show) and pays that split instead of a plain
author zap — so V4V becomes a first-class zap rather than a separate, no-
receipt payment:
- AccountViewModel.zap() intercepts a note carrying a value block and routes
the chosen amount to the V4V split. Intercepting at zap() covers every
entry point (one-tap, amount popup, custom dialog, polls) with no UI churn.
- V4VPaymentHandler gains asZap/zapType: lnaddress shares now attach a NIP-57
zap request, so a Nostr-aware provider mints a zappable invoice and
publishes a receipt (driving the zap button's icon/counter). Node shares
stay keysend — no LNURL, so a receipt is impossible there by protocol.
- Per-minute streaming pays with asZap=false so it doesn't publish a receipt
every minute.
- The dedicated "Send value" button is now redundant and removed;
PodcastValueSplits becomes a pure recipient/percentage breakdown display
next to the zap button.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Per review: negentropySync should not silently switch transports. Plain
created_at paging is heavier and non-delta, and a caller who reached for
negentropy may prefer to know it failed (try another relay, narrow the
filter, abort) rather than get a surprise paged download.
So:
- negentropySync is now negentropy-only. created_at windowing on the
relay's max_sync_events cap stays automatic (it's still negentropy), but
a window that genuinely can't be reconciled — minimal window still over
the cap, or a relay with no NIP-77 support / disconnect / timeout — now
throws the typed NegentropySyncException (reason OVER_MAX_SYNC_EVENTS or
UNAVAILABLE, carrying the failing window) instead of paging. Dropped
NegentropySyncResult.fellBackToPaging.
- Added negentropySyncOrFetch (+ negentropySyncOrFetchEvents Flow form) as
the ergonomic "try negentropy, else page" combinator: runs negentropySync
and, on NegentropySyncException, falls back to fetchAllPages over the same
filter, deduping by id across both phases and honoring maxEvents. Returns
NegentropyOrFetchResult so callers can see which path ran and why.
Callers now choose explicitly: negentropySync to handle failure themselves,
negentropySyncOrFetch for automatic paging fallback.
Tests: over-cap relay with spread timestamps succeeds via windowing alone;
over-cap minimal window throws (and the caller can page); orFetch pages on
failure and uses negentropy when it works.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
Make the single-podcast detail screen spec-neutral so a Podcasting-2.0
show (kind 30078, d=podcast-metadata, e.g. "Soapbox Sessions") renders
its real cover art and title instead of the default Amethyst banner and
"Podcasts" fallback:
- PodcastHeader/PodcastScreen now take a resolved PodcastShow? instead of
a NIP-F4-only PodcastMetadataEvent?, resolving from either the kind
10154 (F4) or kind 30078 (P2.0) metadata event.
- FilterOnePodcast adds a #d=podcast-metadata filter on kind 30078 so the
P2.0 show metadata is actually fetched on deep-links/fresh loads.
- NoteMaster (thread/full view) now dispatches Podcasting20EpisodeEvent,
Podcasting20TrailerEvent, and the kind 30078 podcast-metadata variant
to the shared podcast renderers, matching NoteCompose.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Replace the cumulative `negentropySyncAsFlow(): Flow<List<Event>>` with
`negentropySyncEvents(): Flow<Event>`, which emits each event individually
as it arrives. Rebuilding an ever-growing list per event was O(events²) in
both CPU and memory and pointless for a bulk download; the stream stays
O(1) in memory and hands the caller raw events to collect however they
like.
Events are buffered with Channel.UNLIMITED because negentropySync delivers
through a non-suspending callback — a bounded buffer would drop events when
the collector lags. Callers can apply their own buffer/conflate/
collectLatest downstream.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
Claude