Two-reviewer adversarial audit of the branch found no correctness/data-loss/
crash bugs. This closes one gap and applies three robustness fixes:
- Multi-filter search ordering: a REQ whose filters all carry a search term
(e.g. the client's search-across-kinds) was created_at-ordered via the union
path. Now relevance-ordered — unionSubqueriesIfNeeded(projectRank) projects
rank per branch, UNION ALL + GROUP BY row_id MIN(rank) dedups across branches
keeping the best score. Only when every branch is a search branch; mixed
search/non-search REQs and count/delete unions stay as before.
- prepareAuthorStreams/prepareTagStreams build cursors via buildStreams, which
closes already-prepared statements if a later prepare throws (was: stranded
checked-out, un-reset handles holding read locks in the pooled connection).
- Stream counts computed as Long so authors×kinds / values×kinds can't overflow
Int back into the eligible band and route a huge fan-out into the merge.
- Renamed CachedStatement.finalize() -> finalizeStatement(): a no-arg finalize()
is the JVM Object.finalize, risking a GC double-close of the native handle.
Tests: multi-filter search relevance + cross-branch dedup + count parity;
existing merge/cache/search suites green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
From the branch audit: verifies the search-relevance path holds under a
two-tag-key filter (requires both tags, ranks by bm25, no duplicate rows),
that count(filter) matches query size under a limit smaller than the match set
(NIP-45), and that deleting a non-searchable event (no FTS row) fires the
contentless delete trigger against an absent rowid harmlessly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
Any filter with a search term must be relevance-ranked (NIP-50), but only the
tag-free shape (makeSimpleSearch) was — search + a tag fell through to the
row-id-subquery path (prepareRowIDSubQueries/makeQueryIn), which ordered by
created_at.
prepareRowIDSubQueries gains projectRank: when a search filter joins event_fts,
it also projects the bm25 score as a `rank` column and cuts its LIMIT by rank
(most relevant, not newest); makeQueryIn(orderByRank) then presents the joined
result by that rank, created_at DESC as tie-break. Off by default, so
count/delete/union/negentropy (which must stay single-column and unranked) are
untouched. toSql wires it on whenever the filter carries a search term.
SearchRelevanceOrderTest adds a tag-scoped case (stronger-but-older outranks
weaker-but-newer, wrong-tag and non-matching excluded, limit cuts by score).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
NIP-50: results are returned "in descending order by quality of search result
... not by the usual .created_at", with the limit applied after the score. The
store sorted search by created_at DESC (pre-existing), so it returned the
newest matches rather than the best ones.
makeSimpleSearch (the search [+ kinds/authors/since/until] + limit shape) now
orders by FTS5 bm25 (ORDER BY event_fts.rank, created_at DESC as a tie-break).
Verified bm25 rank works on the contentless table through the join, and that a
stronger-but-older match outranks a weaker-but-newer one. The rarer
search+specific-tag shape and the negentropy snapshot still sort by created_at
(the row-id subquery can't carry rank; negentropy is a sync set) — documented.
This is a correctness fix, not a scaling one: bm25 scores every match, so
search latency still grows with the match set. Tests: SearchRelevanceOrderTest,
Fts5CapabilityProbe.bm25RankWorksOnContentlessTableInAJoin; QueryAssemblerTest
search plans updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
searchOrderByRowId ordered NIP-50 search by the FTS rowid (ingestion order) to
get O(limit) search, but NIP-01's limit requires the newest events by
created_at. Once ingestion diverges from created_at (any historical sync) that
returns the wrong events under a limit — a spec violation — so the flag, its
QueryBuilder branch, and its test are removed. Search stays created_at-ordered;
corpus-independent search is an external-engine job, not this index.
The contentless + rowid=row_id schema stays for the reasons that don't touch
ordering: the delete trigger now seeks by rowid (O(log n)) instead of scanning
by an FTS column (O(n)) — measured ~78× faster at 8k rows and widening, on a
path every deletion hits — and the index is smaller. Benchmark reframed around
the delete win and the honest (unchanged) search cost; plan doc updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
The scale-curve report showed the SQLite store degrading with corpus size on
NIP-50 search (~18×) and the large-IN tag watcher, while point reads stayed
flat. Three read/size changes (write path and index set unchanged):
- FTS: rebuild event_fts as a contentless FTS5 table (content='',
contentless_delete=1) keyed by rowid = event_headers.row_id. Drops the
stored content copy (smaller index); external-content can't hold the
derived indexable text, so contentless is the correct primitive. Adds an
opt-in searchOrderByRowId strategy flag: ORDER BY event_fts.rowid DESC
early-terminates (O(limit), corpus-independent) at the cost of
ingestion-order results — flat ~0.22 ms vs created_at's 4.26 ms at 200k
(~19×). reindexAll ends with 'optimize'; the periodic optimize() folds in a
bounded segment 'merge'. DB version 4->5 with a drop-and-rebuild migration.
- MergeQueryExecutor: extend the k-way merge to the tag path (kinds + #e IN
[hundreds] + limit), one cursor per (value[,kind]) stream heap-merged to the
limit, deduping events that carry several queried values. O(limit + streams)
instead of collecting all matches and sorting.
- StatementCachingConnection: pool multiple handles per SQL so the merge's
many concurrent identical-SQL cursors all hit the cache (previously only the
first did) and repeated polls reuse their per-stream statements.
Tests: contentless migration (real v4 DB upgrade), rowid-order search, tag
merge correctness (incl. cross-stream dedup), statement pool, FTS5 capability
probe, and an FTS search-scaling benchmark. Plan in
quartz/plans/2026-07-21-sqlite-query-scaling.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
The remaining SmallReqFloorBenchmark waste, on the per-row replay, the
per-event live fanout, and the per-accepted-event index probe:
- LiveEventStore replay dedupe: a SeenIds holder with an inline lock
replaces the local-fn-plus-lambda that allocated one closure per
streamed row (and again per live delivery). Its HashSet is created
empty so the JVM defers the backing table to the first add — a 0-row
replay no longer allocates a 1024-slot table (was ~4 MB across the
benchmark's 1000 idle subs).
- Live fanout serializes the event body once and passes it through
onEachLive(event, body); RelaySession splices it into the per-sub
frame prefix. An event matching N live subscriptions paid N identical
Jackson passes before; now one. queryRaw's onEachLive signature gains
the body arg (EventSourceBackend default serializes inline, no
cross-sub memo, no regression). Measured: fanout 1->200 live subs
0.50 ms (2.5 us/sub).
- FilterIndex holds subscribers in one persistent map per dimension, so
candidatesFor (once per accepted ingest event) probes with the event's
own fields and allocates no IdKey/AuthorKey/KindKey/TagKey wrappers;
BucketKey now lives only in the rare register/unregister bookkeeping.
SmallReqFloorBenchmark grows a fanout stage (200 live subs, one submit)
to anchor the fanout number; it drives `live` directly and guards the
await with withTimeout so a future fanout regression fails fast.
Verified: quartz relay.server + FilterIndex suites (110 tests),
SmallReqFloorBenchmark, geode suite (126 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Three hot-path allocation cuts the SmallReqFloorBenchmark stages flagged:
- strippingSearchExtensions: index-loop guard returns the same list with
zero allocation when no filter carries a search term (every non-search
REQ/COUNT/snapshot, the overwhelming majority).
- EoseMessage/OkMessage: direct-buildString wire form on the escape-free
fast path (EOSE per REQ, OK per publish), skipping the generic
serializer's node tree; exotic subIds/reasons fall back. Shared
isEscapeFreeAscii helper in WireJson.kt, mirroring NegMsgMessage.
Verified: quartz relay.server + message-frame suites (110 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Cold boot bumped the session revision ~292 times for 3 communities, driving 22
Messages rebuilds and re-deriving every plane subscription each time. Three
compounding causes, all measured on device:
1. Every refold republished state even when the fold was identical.
ConcordCommunityState and its components were plain classes, so StateFlow
conflation never applied and a prior-epoch wrap that didn't move the
anti-rollback floor still counted as a change. Make the fold result compare
by value (AuthorityResolver holds only immutable value fields; a data class
with a private constructor is fine).
2. A control wrap bumped twice — once from ingest() returning STRUCTURAL and
once from the per-session state watcher reacting to the same refold. Add
ConcordIngestOutcome.STRUCTURAL_FOLD for the two control-plane branches so
the manager leaves those to the watcher, which (given 1) now fires only on
genuine change. Guestbook and base-rekey keep STRUCTURAL: they mutate
members/the rekey buffer, not state, so no watcher covers them.
3. refold() and controlFloorsLocked() re-opened the WHOLE wrap buffer on every
control wrap, and opening a wrap is a NIP-44 decrypt + parse — making a
backfill quadratic in decryptions (~8.6k opens to ingest 93 wraps for one
community). Memoize editions by wrap id: one open per wrap, ingest() stays
synchronous and results are unchanged.
Measured over one cold boot: revision bumps 292 -> 87, Messages rebuilds
22 -> 7, and time from first fold to all 17 channels 43s -> 7.5s.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
FilterIndex registration runs on every REQ open/close but built each
new snapshot by copying both full maps — O(S) work and allocation per
REQ with S live subscriptions. Persistent (HAMT) maps keep the
wait-free single-load reads and CAS write loop while making a write
O(keys x log S) with structural sharing.
SmallReqFloorBenchmark grows a B@1k stage (1000 idle parked
subscriptions) to make the population cost visible, and its B stage
now enters queryRaw undispatched like production does: @1000 subs the
per-REQ cost drops 0.225 -> 0.151 ms and the measured population
penalty falls below run noise (was +0.011 ms per REQ).
With this and the undispatched replay, the in-process floor above the
raw store query is ~0.11 ms (was ~0.66 ms as first measured): A 0.120,
B 0.203, C 0.239 ms on a quiet machine.
Verified: FilterIndex tests, quartz relay.server suite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
SmallReqFloorBenchmark showed the per-REQ floor on small results is
dominated by pipeline, not the store (raw query 0.125 ms vs 0.785 ms
session REQ->EOSE in-process). Half of the dispatch slice was the
scheduler hop between handleReq's launch and the query coroutine:
starting the job with CoroutineStart.UNDISPATCHED runs the stored
replay and EOSE inline on the receiving coroutine (the reader-pool
acquire doesn't suspend when a connection is free), parking only at
the live tail. Measured: dispatch+frames slice 0.397 -> 0.207 ms.
Commands on a connection are processed sequentially, so nothing can
target the subscription before the job lands in the registry at the
first suspension point.
Verified: quartz relay.server suite, SmallReqFloorBenchmark, geode
full test suite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
TagAuthorIndexBenchmark at 1M events settles the flag: the DM-room
shape (kinds + authors + #p, 65 client assembler call sites) drops
14.2 ms -> 0.66 ms (~21x, growing with corpus size) while batch-insert
cost stays inside run noise (49.0 vs 47.4 us/event). Existing relay
DBs build the index on next open via ensureOptionalIndexes.
Also refreshes the docs the numbers made stale: IndexingStrategy KDoc
now records the 200k and 1M measurements instead of a TODO,
MergeQueryExecutor's tag-merge note points at the new relayBench
reactions-watch scenario, FsQueryPlanner/FsDriverSelectionBenchmark
reflect the landed cost-based pick (149 ms -> 4.0 ms at 30k events),
and RELAY.md documents that strategy flag flips materialize indexes on
the next open.
Verified: quartz jvmTest store suites, geode test (126), desktopApp
LocalRelayStore tests (5, incl. reopening a default-strategy DB with
the new pubkey-alone flag), relayBench compiles.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Acts on the measured gaps from TagAuthorIndexBenchmark and
FsDriverSelectionBenchmark:
- FsQueryPlanner: replace the fixed tags -> kinds -> authors driver
order with a cost-based pick. Every legal driver (each tagsAll value,
each tags key's value union, the kind set, the author set) opens a
lazy directory iterator; all are drained in lockstep and the first to
exhaust (the smallest listing) drives, so a giant idx/kind tree is
never read past ~the smallest candidate's size. Fixes the 149 ms vs
3.4 ms (~44x at 30k events) authors+kinds+limit regression.
- EventIndexesModule.ensureOptionalIndexes + SQLiteEventStore: flag-
gated indexes are runtime config, not schema. An idempotent
CREATE INDEX IF NOT EXISTS pass now runs on every open, so flipping
an IndexingStrategy flag on an existing DB builds the index without
a user_version bump.
- Desktop LocalRelayStore: enable indexEventsByPubkeyAlone. Shared
ViewModels (Nip65RelayList, PrivateOutboxRelayList, VanishRequests)
replay authors-only filters that full-scanned without
(pubkey, created_at); existing DBs pick the index up on next open.
- relayBench Scenarios: add "conversation" (tag ∩ author ∩ kind, the
DM-room shape, 65 client assembler call sites) and "reactions-watch"
(kind 7 + #e IN 150 hottest notes) so the uncovered archetypes get
head-to-head numbers vs strfry.
- quartz build: forward tagBenchScale/fsBenchScale to the test JVM.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Real numbers from the two new benchmarks, so the trade-offs are on the
decision points instead of in a chat log:
- IndexingStrategy.indexTagsWithKindAndPubkey: the KDoc called the
kinds+authors+tags shape "rarely used", but the client assembler
survey found 65 call sites. TagAuthorIndexBenchmark @ 200k events:
DM-room query 9.4 ms -> 0.6 ms (~15x) with the flag on, insert cost
+14% (41.5 -> 47.3 us/event). TODO: re-evaluate defaults (geode).
- MergeQueryExecutor: tag-path analogue of the follow-feed collect-all
sort (kinds + #e IN [hundreds] + limit never merges). Measured
12.8 ms cold / 6.0 ms since-bounded at 200k events; revisit if
relayBench shows it at relay scale.
- FsQueryPlanner: fixed driver order sends authors+kinds+limit (the
most common CLI shape) through the kind tree. FsDriverSelection-
Benchmark @ 30k events: 149 ms -> 3.4 ms (~44x) driving from the
author tree; TODO: cost-based pick by directory entry counts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
The 2026-07 client filter-assembler survey mapped 551 Filter
constructions to ~12 query archetypes. Two hot shapes had no benchmark
coverage in prodbench or relayBench, and the FS store had none at all:
- TagAuthorIndexBenchmark: the DM-room shape (kinds + authors + #p,
65 assembler call sites) with indexTagsWithKindAndPubkey off vs on,
including the insert-cost delta of the extra index; plus the
reactions watcher (kinds=[7], #e IN 300, limit) cold and
since-bounded, which has no tag-side k-way merge today.
- FsDriverSelectionBenchmark: FsQueryPlanner's fixed driver order
(tags → kinds → authors) on authors+kinds+limit — the most common
CLI shape — comparing the current kind-tree driver against an
author-tree driver with kind post-filter.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
the files are now pure-ASCII and visually unambiguous:
- BlossomPaymentSafetyTest: raw U+202E/U+202C test payload -> escapes
- BlossomPaymentRequired: BIDI_OVERRIDES char array -> escapes
- Sanitizer: RTL_OVERRIDES and ZERO_WIDTH regex classes -> escapes
Verified by mutation: with BIDI_OVERRIDES stripping disabled,
reasonBidiOverridesAreRemoved fails, proving the escaped payload still
carries a real U+202E. Emoji ZWJ sequences in RichTextParserTest are
intentionally untouched (functional joiners, not bidi controls).
Addresses review findings from a merge-time audit.
- **Complete the headline multi-value `clone` fix for PRs** (was applied only to
kind:30617). GitPullRequestEvent (1618) and GitPullRequestUpdateEvent (1619)
carry `clone` with the same spec shape but still emitted repeated single-value
tags and read only the first value — so the exact interop bug this branch set
out to kill was still live for PRs, both directions (ngit keeps only the last
repeated tag; we lost every URL after the first from ngit's multi-value tag).
Now both emit one multi-value `["clone", …]` tag and read both forms. Verified
on the wire + GitNip34InteropTest + CLI harness (40 checks).
- **Android git-status spoofing (GitStatusIndex)**: newest-status-wins with no
author check meant anyone could publish a kind-1632 and make someone else's
issue render closed. Now filter statuses to the repository owner (from the
status's own `a` tag), declared maintainers (from the cached announcement), or
the target item's author — matching NIP-34 and the CLI's derivation. Pre-existing
on main; this branch made the CLI/Android divergence visible.
- **CLI robustness**: `git comment`/`git patch` no longer block forever reading
stdin on an interactive TTY (amy is non-interactive — error instead). The local
`git` subprocesses in `git init`/`git apply` now drain stdout on a side thread
under a bounded `waitFor` + `destroyForcibly`, so a wedged git can't hang the
CLI.
Left as a follow-up (cosmetic): GitBrowseCommands.candidateUrls duplicates
GitRepositoryBrowserViewModel's — worth lifting to shared code, not worth the
cross-module coupling here.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Findings from a review pass over the git-parity branch, with fixes:
- **`git init` announced a WRONG earliest-unique-commit on shallow clones**
(interop-critical). `git rev-list --max-parents=0 HEAD` returns the shallow
boundary commits, not the true root, so the repo would be announced under a
different cross-fork identity than ngit computes. Now: detect shallow clones
and omit the euc with a warning to pass `--earliest-commit`; on full clones
derive the deterministic `--first-parent` mainline root instead of an
arbitrary `tail -1`.
- **`git issues|patches|prs` silently truncated and mis-derived status** on
active repos: one single-page `drain` pulled items AND status events under a
shared cap, so status events (newer, more numerous) could crowd items out of
the window and the close-status that determines an item's state could fall
outside it → a closed item read as open. Now paginate the items
(`drainAllPages`) and fetch exactly the statuses that `e`-reference them.
Verified on the live amethyst repo: 51 PRs paginated, 19 correctly closed.
- **Pipe-buffer deadlocks** (latent): `GitInitCommand.git()` discards stderr to
the OS (a chatty command can no longer fill its stderr pipe and hang the
stdout read); `GitApplyCommand.runGit()` writes stdin on a background thread
while draining stdout, so a patch larger than the pipe buffer can't deadlock.
- Minor: `git cat` binary detection uses an index loop instead of boxing 8000
bytes; `GitRepositoryEvent.clones()/webs()` dedupe.
The harness `git init` test now runs against a fresh full checkout (this repo's
CI checkout is shallow) and adds a shallow-clone case asserting the euc is
omitted. 38/38.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Verified Amethyst's NIP-34 events byte-for-byte against the ngit reference
implementation (DanConwayDev/ngit-cli) and the spec, and fixed three real
interop divergences in quartz — so ngit/gitworkshop and Amethyst read each
other's git repos, issues, patches, and PRs without losing data.
- Repository announcement `clone`/`web` were emitted as REPEATED single-value
tags (`["clone", a]`, `["clone", b]`). The spec and ngit use ONE multi-value
tag (`["clone", a, b]`), and ngit's parser keeps only the LAST of repeated
known tags — so multi-URL repos silently lost every URL but one in both
directions. Now emitted as a single multi-value tag; `clones()`/`webs()` read
BOTH the spec form and the legacy repeated form, so old events still parse.
(`relays`/`maintainers` were already correct multi-value tags.)
- Issues (kind 1621) were missing the `["p", <repo-owner>]` tag that patches and
PRs already include — a maintainer watching `#p` wouldn't see them. The
builder now adds it (fixes both the CLI and the Android issue-creation path,
which both passed an empty notify list).
- Patch / PR / PR-update `r` tags carried the `"euc"` marker
(`["r", commit, "euc"]`). Per the spec and ngit that marker belongs only on
the kind-30617 announcement; other `r` tags are plain `["r", commit]`. A `#r`
filter matches either shape, so this is a spec-compliance/byte-parity fix.
`alt` (NIP-31) tags are intentionally still omitted — quartz treats the generic
alt client-hint as deprecated, and ngit/gitworkshop parse the structured tags,
so it isn't required for interop.
Adds `GitNip34InteropTest` (5 cases: multi-value write, tolerant read of both
forms, issue p-tag, plain patch r-tag) and 4 wire-format assertions to the CLI
git harness (37 offline). No regressions in the nip34 or Search suites.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Merges nostr proposal b07eb505 into main:
- feat(nip05): add Nip05Id.parseLenient for mention/text rendering
- feat(compose): wire NIP-05 popover mentions to nostr:nprofile1…
Also closes duplicate proposal 4b90b41f, which pointed at the same commits.
Beyond the feature, this replaces the unvalidated `Nip05Id("_", prefix)` raw
constructor in UserSuggestionState with `Nip05Id.parseLenient(prefix)`, closing
a hole where a typed mention such as `evil.com#x.bit` produced a GET to an
arbitrary host via `toUserUrl()`'s bare interpolation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the privilege escalation: any BAN holder could ban the authorities above
them — including the owner — because the Banlist gate checked only the BAN bit.
Once banned, a member loses all authority (`hasPermission` is `!isBanned && ..`)
and honest clients drop their events, so a single edition from the most junior
moderator permanently silenced every admin above them.
CORD-04 §3 requires the rank half: "One hard rule binds every action: the actor
must hold the required bit and strictly outrank its target — equal cannot act on
equal (an admin cannot ban a peer admin)", restated as §5 step 3. Only §4, which
defines the Banlist, states the bit half alone — which is why both this client
and Armada shipped the same rank-blind gate.
§3 is stated per TARGET while the Banlist is one whole-list document, so it is
enforced as a DELTA rule: an edition may only add or remove npubs its signer
strictly outranks, judged against the roster settled behind it; the owner is
never a valid target (position 0 is "supreme and unremovable"); and entries the
signer may not act on are IGNORED rather than rejecting the edition, so one bad
entry cannot discard the bulk-ban §4 recommends as the collision remedy, and a
rogue cannot grief the list by forcing rejections.
ConcordModeration.currentBanned now reads the honored banlist through the
resolver instead of decoding the raw head. Besides picking up the fork healing
it was missing, this closes a laundering path: our own next ban/unban would
otherwise re-publish an entry our fold refuses, under our signature.
BREAKING (consensus): Armada has not shipped this rule, so banlists can differ
between clients until it does — we now ignore a ban Armada honors whenever the
signer did not outrank the target. Shipping the spec-conformant behaviour was
judged better than continuing to honor an escalation. Write-up to send upstream
is docs/concord-banlist-rank-conformance.md.
The three tests added in 0ae6bc6698 as @Ignore-d documentation now pass and are
un-ignored; two companions (a moderator still bans a plain member, the owner
still bans anyone) passed throughout and pin what the fix had to preserve.
Full :quartz:jvmTest and :commons:jvmTest suites green.
Still open and documented, not addressed here: a banned BAN holder can lift
their own ban (a fixpoint-ordering question that needs a spec ruling), and a
forked ban survives an unban that does not chain onto it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ban/Remove were offered to any BAN holder against any non-owner, ignoring rank
— unlike the role picker, which routes through `canActOn`. Both the Members
roster and the message-level path (`Account.concordBanTarget`, the chokepoint
for the quick-action menu, the note dropdown, the note action sections and the
chat action sheet) now require `canActOn(me, target, BAN)`.
This is NOT the no-op it first looked like. The premise that the fold would
drop such a ban is wrong, and a test proves it: BANLIST is a single whole-list
entity, so `authorizedHeads`/`banGate` gate on the author's BAN bit alone and
never rank-check the list's *contents*. A rank-5 moderator's ban of a rank-1
admin is therefore ACCEPTED by every client, and the admin then loses every
permission, since `hasPermission` is `!isBanned && ..`. It is privilege
escalation, not a silent no-op.
The fold is deliberately left alone. Armada has the identical gap — its
`banlistGate` calls the rank-blind `isAuthorized(.., Permissions.BAN)` while
its role path uses the rank-aware `canActOnPosition` — so rank-gating our fold
would make us ignore bans every other client honors, splitting the banlist
across clients. Closing it needs a spec change, like CORD-05. Refusing to
AUTHOR such a ban restricts only what we write, never what we accept, so it
cannot diverge consensus.
Three `@Ignore`-d tests in AuthorityResolverTest state the fold-level invariant
and currently fail by design; two companions assert the gate does not
over-correct (a moderator still bans a plain member; the owner still bans
anyone). Un-ignore the first three when the spec closes the gap.
The owner short-circuits the check rather than going through `canActOn`, which
begins at `hasPermission` and is false while banned — since a rogue BAN holder
*can* currently banlist the owner, routing them through it would let them be
locked out of moderating their own community.
Device-verified on Amethyst QA Concord as Dr. Edo (QA Lead, rank 2): Bob
(Admin, rank 1) now offers only the disabled "Roles… / You don't outrank this
member" where Ban and Remove used to be enabled, while the Helper (rank 5)
still offers Roles…, Ban and Remove.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An unauthorized control edition in the middle of an entity's chain
permanently froze that entity. Observed on device for a member's GRANT:
v0 owner (grant mods)
v1 owner (grant admins) <- fold stopped here, forever
v2 MIDTIER (escalation, correctly rejected)
v3,v4,v5 owner orphaned, unreachable
`AuthorityResolver` filtered unauthorized editions out BEFORE calling
`EditionFold.foldEntity`, and the walk only advances when the next
version cites the current head's hash. Removing v2 severed the chain, so
every honest edition above it was lost. Any member could permanently
freeze any member's role assignment — including the owner's ability to
change it — with a single event, recoverable only by a Refounding. It
predates the recent rank gates (verified with a zero-role identity); the
gates only widen which editions can poison.
Armada does not have this bug, and its approach settles the design.
Reading its control-plane fold (read for semantics only — Armada is
AGPLv3, Amethyst is MIT, no code taken): the chain walk runs over the
UNFILTERED set, producing an ordered candidate list — chain-verified head
first, then every remaining edition version-descending — and authority is
applied AFTERWARDS, per candidate, picking the first admissible one. A
rejected edition is skipped during the ascending admissibility walk
without truncating it. For the chain above, Armada picks v5.
So the fix is not to filter later but to gate later: `EditionFold` gains
candidate-based gated folding, and the resolver and community state now
gate per candidate instead of pre-filtering the pool. Authority checks
themselves are unchanged — only WHEN they run moved. Applied to ROLE,
GRANT, BANLIST, CHANNEL, METADATA and the authorized-head map.
The writer had to be fixed too, for a sharper reason than expected. With
an ungated `headOf`, a rogue banlist edition at the tip is read as
current state, so the owner's next ban REPUBLISHES THE ROGUE'S CONTENT
UNDER THE OWNER'S SIGNATURE — an unauthorized empty banlist laundered
into an owner-signed one the moment the owner bans anyone else. Tolerant
reading cannot heal that, because the resulting edition is genuinely
authorized. `ConcordModeration.headOf` now folds the authority-gated
heads, and `owner` is a REQUIRED parameter rather than defaulted, since a
silently-wrong default here is a consensus footgun.
Banlist healing is preserved with one necessary change: the ancestry walk
now runs over the full pool rather than the authorized subset. Ancestry is
structural — walking only authorized editions stops at the rejected one
and misreads genuine ancestors as concurrent forks, resurrecting bans an
unban had cleared.
Six regression tests, each verified to fail without the fix. Two process
notes worth recording: the first "without the fix" run reported BUILD
SUCCESSFUL because Gradle served a stale up-to-date `jvmTest` — trusting
it would have meant concluding the tests were worthless. And the
forged-edition test initially passed both ways because the forgery's
content coincided with the honest outcome; it was rewritten so the
mid-chain arm genuinely discriminates.
The rank-gate, rogue-higher-version, floor and rollback tests all pass
unchanged.
Known gap: `headOf` gates through the per-kind permission map, which is
coarser than the resolver's rank gates, so the writer can still pick a
head the reader rejects when an in-permission but out-of-rank edition
sits at the tip. Tolerant reading makes that benign, but it is not an
exact reader/writer match; tightening it needs the resolver to expose
per-entity heads.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two problems in the remote signer, both about a client getting something
without the user meaningfully agreeing to it.
**`get_public_key` and `get_relays` answered anyone.** Every other method
runs through `ifAuthorized`; these did not, and nothing required a prior
successful `connect`. The service decrypts and dispatches any well-formed
kind-24133 envelope, so anyone holding the `bunker://` URI — pasted into a
malicious app, posted for support, leaked in a screenshot — could ask it
which account it belongs to, without the secret and without connecting.
`get_relays` additionally handed over the inbox relay set. That defeated
the transport/identity split, which otherwise works: the relay-visible
traffic really is anonymous, since the p-tag and author are a transport
key and the payload is NIP-44.
Both now require the client to be paired. The authorizer interface gains
`isPaired` with NO default, so a future authorizer has to state its own
rule rather than silently inheriting "everyone is paired".
`ping` is deliberately left open. It reveals nothing the caller does not
already have — a signer is alive at a pubkey they hold — and first-party
behaviour could be confirmed but third-party clients that ping before
connecting could not be ruled out. Breaking a legitimate handshake to
close a minor oracle is a bad trade. The choice is pinned by a test that
also asserts the pairing check is never consulted, so it stays deliberate
rather than drifting back by accident.
**Decrypt consent showed nothing at all.** The bridge populated the
content preview and raw data only for signing requests, so a decrypt
request produced an empty preview block — no ciphertext, no counterparty,
not even the "Show event" toggle — leaving "AppName wants to read your
private messages" with *Allow always* as the primary button. Meanwhile
the coordinator documented the opposite: "Amethyst decrypts first, then
asks permission to expose." That was never implemented.
Now:
- The counterparty is resolved and shown, so the prompt reads "…read your
private messages **with Alice**". It never degrades to nothing —
cached name, else a shortened npub. Knowing *whose* messages is a
categorically different decision.
- The message is decrypted BEFORE prompting and the plaintext is the
preview, as documented. It is a local operation and nothing is exposed
until approval. Failure, blank and hang all collapse to an explanatory
string under a timeout, so the dialog is never empty and cannot stall.
- A narrower grant is offered ALONGSIDE the broad one, not instead of it:
`DecryptFrom(counterparty)` keyed `decrypt:<hex>` next to `Decrypt`.
The dialog's primary button becomes "Always allow for Alice" with the
broad option demoted. Because the ledger stores an opaque op key, no
persisted decision migrates and the storage format is untouched.
Scoping decrypt per counterparty *instead* would have been worse than
the bug: a DM client would prompt once per conversation, training users
to approve everything. A narrow option beside the broad one gives
granularity without the prompt explosion.
Also fixes a latent bug found on the way: `AllowForSession` recorded the
*requested* op rather than the *granted* one, which would have widened a
narrow session grant back to broad.
Verified by three sabotage passes; the tests that stayed green under them
are the intended negative guards. One existing test asserted the buggy
behaviour outright ("public reads are never gated") and was rewritten.
Not done: the batched consent sheet still records the broad op for
"remember" — offering the narrow choice per row there is a UX design
question, not a mechanical change.
Needs a device check before release: the decrypt preview runs the account
signer before consent. That is free for a local key, but an account backed
by an external NIP-55 signer (Amber) may show Amber's own prompt ahead of
Amethyst's.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Four defects in the pay-to-upload path, all exploitable by a hostile or
merely broken media server.
**No amount cap, and the shown amount was not the paid amount.**
`amountSats()` was display-only — `pay()` never read it, so a server
asking 10,000,000 sats was handled exactly like one asking 10. `pay()`
now takes the amount the dialog showed, re-derives it from the invoice,
and refuses on over-cap, on mismatch with the dialog, and on an
amountless invoice (a payee-chosen amount must never be paid
unattended). The cap is 10,000 sats: real BUD-07 per-blob fees are
single-digit to low-hundreds, so this is one to two orders of magnitude
above legitimate use and caps one prompt's damage without blocking
anyone.
**Unbounded re-prompting.** The post-payment retry caught generic
`Exception` — including a second `BlossomPaymentException` — marked the
target missing, then re-entered the mirror path, so a server that
pocketed the preimage and replied 402 again got an indefinite pay-prompt
cycle. A new prompt ledger allows one prompt per (blob, target) per
user-initiated action; the user's own tap resets it, the automatic
continuation does not.
**Double-spend window.** `withTimeoutOrNull(90_000)` abandoned the wait
without cancelling the in-flight NWC request, so a payment settling at
second 91 was reported as failure and could be paid again. NIP-47
`sendZapPaymentRequestFor` is fire-and-forget with no cancellation and
does not return a request id, so cancelling is not reachable. Instead the
invoice is claimed before sending and released only on a definitive
wallet answer — the timeout path deliberately does not release it, so it
can never be sent twice. Known limit: the claim is process-lifetime, so a
timed-out invoice becomes payable again after a restart; persisting it
needs a real store.
**Unsanitised server text.** The server-controlled `X-Reason` header was
rendered verbatim directly above the pay button, letting a hostile server
assert its own amount ("Pay 1 sat") in what looked like Amethyst's voice.
It is now stripped of control characters and Unicode bidi overrides,
whitespace-collapsed, clamped to 200 chars, and rendered in a distinct
style as `<host> says: "…"` so it cannot be mistaken for our wording.
The two policies are small standalone classes so they are testable
without an Account graph. Verified by disabling all four checks: 12 of 15
tests fail, covering every defect; the 3 that stay green are the intended
negative controls.
Not covered: the background mirror sweep does not go through this handler
and cannot prompt, so it was never exposed to the cap or re-prompt
defects, and it inherits the invoice claim and sanitisation for free.
`pay()` itself is still not end-to-end tested — that needs Account
mocking well beyond this fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`MintHttpClient` only trimmed trailing slashes — no scheme check, no host
check — and `token.mint` comes verbatim from any pasted or posted Cashu
token. Tapping Redeem on a token in someone's note therefore made the
device issue HTTP requests to an arbitrary URL: `http://127.0.0.1:<port>`,
LAN addresses, `169.254.169.254` (cloud metadata), any scheme at all —
plus it disclosed the user's IP to whoever controlled the URL.
Validation now runs in the constructor, so no caller can issue a request
before it. The rule: `https://` to a public host, or `http://` to a
`.onion` host, and nothing else. Onion mints matter — a blanket
"https only" rule would have silently broken every Tor mint.
Rejected hosts cover the private/loopback/link-local/unique-local ranges
plus CGNAT, multicast, reserved and `0/8`: none is a public unicast host,
so allowing them buys nothing and leaks reachability.
The bypasses are what make this worth care, and each has a test:
IPv4-mapped and IPv4-compatible IPv6 (`::ffff:127.0.0.1`, `::127.0.0.1`),
the full `inet_aton` spellings (`2130706433`, `0177.0.0.1`, `0x7f000001`,
`127.1`), trailing-dot hosts, and userinfo disguise
(`https://mint.example.com@127.0.0.1/`) — handled by splitting on the
LAST `@`. The host parse is hand-rolled rather than delegated to
`java.net.URI`/`HttpUrl` precisely because those normalise these forms
inconsistently.
A mint the user added to their own wallet is exempt from the host and
https rules — a self-hosted mint on a LAN is a legitimate setup, and the
threat here is a *pasted, untrusted* token pointing inward, not a mint
the user chose. The exemption never relaxes the scheme check. It is
threaded properly rather than TODO'd: the melt path passes the wallet's
known mints and marks the mint user-configured only on a match; the
wallet ops and CLI pass it directly, since those URLs are the user's own.
Refusal gets its own message rather than reusing the mint-error string,
whose wording would have misattributed our own refusal to the mint.
DNS rebinding is out of scope and noted in a comment — the check runs
pre-resolution and cannot defend against a host that resolves differently
on the second lookup.
Verified by disabling the scheme and host checks: 11 of 20 tests fail,
every rejection case among them, and every allow case still passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The private community list (kind 13302) is documented as wire-compatible
with Armada's `communityList.ts`, whose entry type ends in
`[k: string]: unknown` — unknown keys are part of the contract, and
`ConcordJson`'s own KDoc says shapes are "deliberately client-extensible
(CORD-03/04)". But `ignoreUnknownKeys = true` plus closed `@Serializable`
DTOs meant decode dropped every unmodelled key and encode never restored
it, so **every Amethyst write of a user's list silently stripped fields
another client had written**, across every community in it.
Already proven, not hypothetical: `JoinMaterialWire` declared a
`refounder` field that nothing in the repo reads, so it was parsed and
destroyed on the first write. We only avoided destroying Armada's
`invite_ref`/`excluded_at_epoch` because those were modelled hours ago,
for stranded recovery — the anchor recovery depends on would otherwise
have been deleted on every write.
Each wire DTO's compiler-generated serializer is now wrapped in a shared
`JsonTransformingSerializer` that lifts unknown keys into a bag on
decode and merges them back on encode, with declared fields winning on
conflict. The known-key set is read from the descriptor rather than
hand-listed, so it cannot drift from the DTO. Preserved at the document
root, each entry, the `current` join material, each channel, each
held_root, each tombstone, and everything nested inside `seed`.
`refounder`'s typed field is removed so it round-trips generically.
Two further data-loss bugs surfaced while doing it, both fixed here:
- **`seed` was overwritten with `current` on every write**, destroying
the immutable join anchor. It is now kept and re-emitted verbatim as a
raw JsonObject — we never hydrate from it while `current` exists, so we
have no business rewriting it, and keeping it raw preserves everything
nested inside for free.
- **`tombstones` were re-encoded as an empty list**, which did not just
lose their unknown keys: it RESURRECTED communities another client had
deliberately removed. They are now carried verbatim.
Verified by four separate sabotage passes (no-op the transform,
re-mint `seed`, restore the empty-tombstone write, flip the merge order);
each new test fails under at least one, and every mechanism is covered.
Control-plane re-serialization was audited too and is NOT fixed here:
`compactControlPlane` is safe (it re-wraps the original seal verbatim),
but the user-facing *edit* paths — `editConcordMetadata`, `grant`, and
the channel edits — construct fresh typed entities and re-encode, so they
drop extensions the same way. Fixing those means merging into the head
edition's raw JsonObject on each edit path, which is a larger change than
this should carry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A rotation carries only (newRoot, newEpoch, rotator) — no recipient list
— so a member simply left out of the recipient set receives nothing and
is stranded on the dead epoch forever while everyone else moves on. It
applies to any member, the owner included, and cannot be prevented on the
receive side: there is nothing to check.
Armada does not prevent it either; it recovers, and this follows the same
approach. The invite link a membership was joined through is stored as
the anchor, and when that link later resolves to a HIGHER epoch, the
membership merges forward.
Uses Armada's exact wire names so the kind-13302 list stays compatible in
both directions: `invite_ref` (the link in bare `<naddr>#<fragment>`
form, host-stripped so a link minted by a different front end reduces to
the same anchor) and `excluded_at_epoch`, both at the entry level.
Details that decide whether this works at all:
- `merge()` lets a higher-epoch winner inherit the loser's `invite_ref`
when it has none. Without it a two-device merge silently discards the
only anchor recovery has, disarming it permanently.
- `adoptConcordRoot` carries `invite_ref`/`excluded_at_epoch` through a
rotation; it rebuilt the entry field-by-field and would have dropped
them at exactly the moment they matter.
- Merging forward preserves `heldRoots`, so prior-epoch history the
member legitimately holds is not lost by recovering.
- Recovery requires a strictly higher epoch and a matching community id,
so it is monotonic and cannot be steered by an unrelated bundle.
Hooked onto the existing Concord revision tick immediately after
`drainConcordRekeys`, because the two are halves of one problem: a
rotation you were included in arrives as a rekey to drain, one you were
excluded from produces no message at all and can only be found by polling
the link. Rate-limited to 15 minutes per community; an idle tick costs a
map lookup. Only a Live bundle recovers — an expired or revoked link is
not a missed rotation.
Verified by mutating the production code eight ways (dropping the anchor,
dropping heldRoots, dropping the epoch comparison, renaming the wire
keys, removing the merge inheritance, breaking bare-form parsing) and
confirming each produced exactly the expected failures.
Two things this surfaced, both left for their own change:
- Our parser does NOT round-trip unknown JSON keys — `ignoreUnknownKeys`
plus closed DTOs — while Armada's format ends in `[k: string]:
unknown`. So every Amethyst write of the community list silently strips
fields Armada added that we do not model; it already discards the
`refounder` field we parse but never re-emit. That is live interop data
loss, caused by us, independent of this work.
- `mergeForward` keeps the entry's existing private-channel grants rather
than adopting the bundle's, matching what `joinConcordViaInvite`
already does. If a recovered member should pick up new-epoch grants,
both paths need it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`ConcordRefounding.compactControlPlane` re-wraps one edition per entity
when a community rotates epoch, and the ROTATOR chooses which one
survives. The receiving side had no memory: `refold()` folds only the
wraps at the current epoch's Control-Plane address and discards the prior
epoch's buffer, and `EditionFold` accepts whatever it is handed (its
no-genesis fallback anchors at the lowest version present).
So a rotator could publish only version 1 of a chain and omit version 2 —
restoring a revoked role, clearing a banlist, reverting metadata. Every
signature is genuine; this is rollback by omission, not forgery.
Adds a per-entity floor: the version AND hash last successfully folded.
- **No floor (fresh joiner)** — unchanged: genesis anchor, else the
lowest-version edition as the legitimate compaction bootstrap.
- **With a floor** — the walk is anchored AT the floor: the offered set
must contain the exact edition already folded (version and hash; a
same-version sibling is a fork, not our chain), then walks up. A head
below the floor is structurally unreachable.
- **Gap** (the floor edition is absent) — refuse, and keep the known
head. Refusing by *retaining* matters here: this fold is recomputed
from scratch each time, so letting an entity vanish would itself be a
rollback — a dropped banlist is an unban.
The floor needs no new persistence. It is derived from `heldRoots`, the
rotated-out access roots already persisted in the NIP-44 self-encrypted
kind-13302 list: the session derives each prior epoch's Control-Plane
address from them, folds oldest-first, and takes the resulting heads as
the floor. That survives both a process restart and the session rebuild
`ConcordSessionRegistry.sync` performs at exactly the moment of a
Refounding — which would have destroyed any in-session floor. If the old
planes are not served, there is no floor and behaviour is as before.
Floors are built from AUTHORITY-GATED heads, not raw ones. Without that,
any ex-member still holding a rotated-out root could mint a high-version
edition on the old plane and freeze the entity for every honest client —
a denial of service this change would otherwise have introduced. Covered
by a test.
Verified by disabling both enforcement points: 7 of 12 quartz tests and
the end-to-end commons test fail, and the ones that still pass are
exactly the non-regression cases (fresh joiner, honest compaction,
pass-through without floors).
Known limit: `AuthorityResolver.resolve` folds authorized SUBSETS of the
edition pool and does not carry floors itself; gating happens at the pool
level before the resolver sees anything. Sound, but connectivity checked
on the full set is a weaker precondition than on each subset — passing
floors into the resolver's three folds is worth a follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Refounding recipient set was `Guestbook joins ∪ roster ∪ owner ∪ self`
— it never consulted `observedAuthors`, the members seen publishing to a
channel. `ConcordCommunitySession.allMembers()` already unions them in
(and already subtracts the banned); the rotation path just didn't use it.
Amethyst announces a Guestbook Join, but nothing in the protocol requires
one, so in practice most members of a cross-client community have never
sent one. Every member who had only ever *posted* — no role, no Join —
therefore received no rekey blob and was silently expelled by the next
rotation. Removing a single spammer would quietly strand most of the
community, and it fell hardest on Armada members, who are the bulk of the
roster in the communities we interop with.
Now uses `allMembers()`. Still a floor rather than a census, as its KDoc
says: a member who joined silently, holds no role, and has never posted
leaves no trace to re-key, and nothing here can find them. Stranded
recovery is what brings those back.
Also corrects an overclaim on `ConcordInviteBundle.validate`. Its KDoc
said self-certification stops a bundle smuggling "a false owner or a fake
key for a real community". Only the first half is true: `community_id`
commits to (owner, salt) — both public in any invite — and NOT to
`community_root`, so an attacker can mint a bundle carrying a real
community's id, owner and salt beside a root of their own. A joiner
adopts that root, believes they are in the real community, and posts into
planes the attacker can read.
That is a CORD-05 limitation, not an implementation gap: Armada's
`validateBundle` checks exactly the same thing and also leaves the root
unbound, so a stricter unilateral rule would break interop while
protecting nobody. Verifying the adopted root's Control Plane does not
close it either — sealed editions carry the owner's own signature, so an
attacker can re-wrap genuine owner editions into their plane, which is
what a legitimate compaction does. Closing it needs a spec change
(commit the root into the self-certifying id, or require the bundle to be
signed by a roster-authorized member); the KDoc now says so instead of
claiming a guarantee the code does not provide.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewed the previous commit's authority changes against Armada
(gitlab.com/soapbox-pub/armada), the reference Concord client we interop
with. Read for its rules only — Armada is AGPLv3 and Amethyst is MIT, so
no code was taken from it.
It confirmed the role rank gate (Armada checks both directions too: the
author must outrank the position being minted AND the standing position
being replaced) and the banlist check on rotation. It also showed one of
our rules was wrong and one gate was missing.
**Removes the owner's refusal of foreign rotations.** The previous commit
made the owner ignore any rotation it did not author, reasoning that a
BAN-holder could otherwise carry the owner onto a root of their choosing.
Armada does the opposite on purpose — "authority is the roster, never key
possession" — and it is right: an admin legitimately rotating to remove a
spammer would leave the owner alone on the dead epoch, self-inflicting
the strand the rule was meant to prevent, and diverging from the
reference implementation forks communities across clients. The threat is
better answered by the rank gate: with role editions gated, nobody can
escalate themselves to BAN, so BAN-holders are people the owner
deliberately trusted.
**Adds the missing rank gate on grants.** A grant was authorized if the
granter outranked every role it handed out — but a REVOKE carries no role
ids, and `all {}` over an empty list is vacuously true. So any
MANAGE_ROLES holder could strip anyone's roles, the owner's admins
included: promotion was gated, demotion was free. Armada treats a grant
as an action ON the member and requires outranking the target's standing
rank; this now does the same.
Both new tests were verified to fail with the corresponding check
disabled, and each has a companion asserting the legitimate case still
works (an admin can still revoke a moderator beneath it).
Also records what Armada does about exclusion, since we cannot prevent it
receiver-side: it does not try to. A rotation carries no recipient list
there either, so a stranded member instead re-resolves the invite link
they joined through and merges forward to the higher epoch ("stranded
recovery"). Amethyst has no equivalent, so a stranded member stays
stranded — noted at the call site as follow-up work.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two authority holes, plus the limit that remains.
**Role editions had no rank gate.** Grant editions are correctly gated —
a granter must hold MANAGE_ROLES *and* strictly outrank every role it
hands out. Role editions checked only that the author held MANAGE_ROLES.
Nothing stopped an authorized signer editing a role at or above its own
rank, including the role it holds itself.
So a moderator at position 5 with MANAGE_ROLES could publish one edition
on their own role's chain claiming position 1 and every permission bit,
then a second demoting the real admins beneath them — reaching full
authority over everyone but the owner in two editions. The
rogue-higher-version defence added earlier does not cover this: it drops
editions from *unauthorized* signers, and this signer is authorized.
Role editions are now gated in both directions: an author may not claim a
position at or above its own rank, may not touch a role that already sits
at or above it, and may not hand a role permission bits it does not hold
itself. Deleting keeps only the second rule, so retiring a role beneath
you still works. The owner is unaffected.
**A banned moderator could still rotate the community.** The rekey
receive path authorized the rotator with `effectivePermissions(...)`,
which ignores the banlist, rather than `hasPermission(...)`, which
excludes banned members. Now uses the latter.
**The owner no longer adopts a root someone else minted.** A rotation
replaces the community root, so a rotator who *includes* the owner as a
recipient hands themselves the keys to the owner's own community — the
owner would follow them onto an attacker-chosen epoch. The owner changes
epoch only by rotating themselves.
**Known limit, documented at the call site.** A rotation carries only
(newRoot, newEpoch, rotator) — no recipient list — so a receiver cannot
tell who was omitted. A BAN-holder can therefore still evict the owner by
leaving them out: everyone else adopts, the owner is stranded on the old
epoch. That is not fixable in the receive path; it needs a protocol
change (a recipient commitment the receiver can check, or owner
co-signing). Tracked for CORD-06.
The escalation test was verified to fail with the gate disabled, and a
companion test asserts an admin can still edit a role beneath it, so the
gate is not merely blocking everything.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three fixes to the Concord invite and moderation paths.
**Invite deep links redeemed with zero consent.** `ConcordInviteScreen`
called `joinConcordViaInvite` from a `LaunchedEffect` on open, and the
manifest registers `https://amethyst.social/invite/` as BROWSABLE. So a
link on any web page — or a QR code, or a push — silently caused a
connection to up to three ATTACKER-CHOSEN relay URLs decoded from the URL
fragment (disclosing the user's IP to a third party), a Guestbook JOIN
signed by the user's identity published to those relays, and a write to
their private community list. No tap, no preview.
The screen now opens in an awaiting-consent state and only joins from an
explicit Join button. The preview is built entirely from the link itself
— base64url and NIP-19 decoding, both pure in-memory — and touches the
network for nothing: no relay connection, no signing, no publishing. It
shows the relays it would contact so the user can see whom they'd be
talking to. The community name lives inside a bundle only those relays
can serve, so it is honestly reported as unknown until joining rather
than fetched.
**Invite expiry was decorative.** `ConcordInviteBundle.isExpired` had no
production callers at all — the only ones were in a test — so an expired
invite redeemed forever. Expiry is now enforced at `classify`, the choke
point every redeem path funnels through, with its own result and message
so the user knows to ask for a fresh link.
**Moderation read the wrong edition.** `ConcordModeration` used
`firstOrNull` over `controlEditions()`, which is in wrap-ARRIVAL order,
not the folded head. Once an entity had two or more editions the next one
chained off a stale predecessor, forking the chain at an already-used
version, and `EditionFold` then resolved the fork by `minByOrNull` on the
rumor id — a coin flip. Bans were masked by a down-only healing union;
UNBANS and role revocations were not, so they could silently fail to
apply. Both call sites now fold to the true head.
Regression tests assert the fold-head behaviour under two arrival orders
— a single order accidentally puts the head first and passes against the
buggy code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`computeShares` paid each fee recipient `totalMilliSats * split / 100`
with no upper bound on `split`, and `split` comes verbatim from a
kind-30054 episode event that anyone can publish (`ValueTag.parse` is a
bare `fromJson` with no validation). A single `fee:true, split:1000`
recipient was therefore paid TEN TIMES the amount the user chose. The
`remainder` clamp looked like a safety net but only zeroed the honest
recipients; it never touched the fee recipients themselves.
Proven by test before fixing: `split:1000` pays 10x, and two fee
recipients at 60% each pay 1,200,000 millisats for a 1,000,000 zap.
This was reachable in the worst possible place. Streaming V4V pays every
minute, automatically, so a modest multiplier stays under a typical NWC
budget and simply runs — and the on-screen running total tracks the
INTENDED amount, so it reads "100 sats" while 1,000 left the wallet,
while the streaming error handler suppresses the toast. The ordinary zap
button reroutes through the same path for any note carrying a value
block, so it was not limited to the streaming toggle.
Fees are a percentage off the top, so each is now clamped to 100% and the
cumulative total to the remaining budget: the payout can never exceed
what the user chose.
The existing test asserted the right invariant (`sum <= total`) but only
ever ran it on well-formed input, which is why this survived.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adversarial audit of the PR's own changes (8 finder angles, verified
before fixing). Quartz core:
- fetchAll-family drains get a wall-clock ceiling (maxTotalMs, default
10x the idle window, delay()-watchdog: cancellable and virtual-time
testable). The pure idle window was unbounded when a relay trickled
events forever — sandboxed napplet queries, set -e fetches, and
marmot await stuck inside one drain. Streaming relays still finish.
- The suspending onEvent hook no longer runs inside a cancellable
timeout scope (an expiring window could cancel verifyAndStore
mid-write and silently drop a received event); the timeout is armed
only when the channels are dry (no per-message timeout-job churn).
- fetchAll is a projection over fetchAllWithHooks: fixes its
unsynchronized events/seenIds mutation from concurrent socket
threads and deletes the duplicate loop + per-event activity channel.
- publishAndConfirmDetailed regains its only-responders contract
(synthetic no-response entries no longer render as 'relay rejected
your message' in app callers); results built by pure associateWith;
shared failure-reason constants + PublishResult.isTransportFailure.
- NIP-65 mutations: split read+write r-tags for the same URL now merge
to BOTH instead of last-wins dropping a facet (+ test).
- TcpProber's 128-thread pool drains after 60s idle.
CLI:
- publishGuard: all-transport failure exits 124 as timeout; rejected/1
is reserved for an actual OK-false answer.
- --help anywhere in argv is hoisted centrally; 'amy notes post "x"
--help' prints usage instead of publishing.
- rejectUnknown false-reject traps fixed: geochat --no-fetch behind an
early return, and 13 elvis-alias short-circuit sites read eagerly.
- Aliases load once per Context and only match name-shaped inputs (no
shadowing a real npub/NIP-05/hex); stderr color requires a
positively-known terminal (TERM sniff polluted captured logs).
- Relay-CSV strictness unified on RawEventSupport.relayFlag (post,
graperank publish/followers/register no longer silently drop
malformed URLs); Args.timeoutMs(+OrNull) replaces 27 hand-rolled
conversions, all strict; offer/debit --timeout > 3600 rejected with
a 'looks like milliseconds' hint; NPub.create idiom; stale jq .id in
the marmot reactions harness; printUsage drift (offer pay --with,
profile --clink-offer, search --kind).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CP4kfLCa3wWtE8Khy21Pkj
The fetchAll/fetchAllWithHooks accessories wrapped their whole
collection loop in one withTimeoutOrNull, so a relay actively
streaming a large backlog was cropped mid-delivery the moment the
absolute deadline hit — even though the loop already has proper
terminal conditions (per-relay EOSE / CLOSED / cannot-connect) and the
timeout's only real job is stall detection.
timeoutMs now measures the delta since the LAST message: every event
or terminal signal resets the window (fetchAll gains a conflated
activity ping so event progress is visible to its wait loop), and only
a full window of silence ends the fetch early. fetchFirst/count keep
absolute waits (single-response — idle and absolute coincide), and
subscribe's duration timeout stays absolute by design (a live stream
has no terminal state).
Since the pages/pool helpers delegate to fetchAll, pagination inherits
the semantics. This also changes app-side callers of these accessories
— in their favor: the timeout only ever fired on slow relays, exactly
when cropping loses data.
New commonTest suite pins the behavior: a relay emitting every 200ms
under a 300ms window streams to completion (10/10 events); a stall
ends one window after the last message, not after the start; EOSE
still returns immediately. CLI docs reworded (--timeout = idle window).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CP4kfLCa3wWtE8Khy21Pkj
With no external consumers yet, converge the --json surface to its
ideal shape in one pass:
- quartz gains publishAndCollectResults: the NIP-01 OK message, connect
errors, and silent timeouts now survive as PublishResult(accepted,
message) per relay instead of dying in a debug log. The existing
boolean APIs delegate unchanged. Silent relays are reported as
'no response within timeout' rather than omitted.
- Context.publish returns the rich map; the new
RawEventSupport.ackFields(ack) is the one canonical projection every
publisher emits: published_to (urls) + rejected_by as
[{relay, reason}] — 'why didn't it post' now answers itself, in
partial failures and in the rejected error alike.
- author/pubkey rule enforced module-wide: 'author' is the key that
signed an event (feed/search/dm/message list items), 'pubkey' an
identity being described; profile show and outbox add the bech32
npub beside the hex when the user is the primary subject.
- Event-list items converge on event_id/author/created_at/content
(dm, feed, search, marmot message, geochat, concord).
- Byte counts standardize on *_bytes keys (blossom/nsite size ->
size_bytes); the text renderer drops the fragile bare-'size'
heuristic and colors stderr progress independently of a piped
stdout.
- Error details are sentences everywhere (not bare gids); dead Result
class removed from the quartz publish accessory.
Docs updated (DEVELOPMENT output conventions, README rejected example).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CP4kfLCa3wWtE8Khy21Pkj
The 1500-line GrapeRankCommand (15 sub-verbs in one object, 7.5x the
module's 200-line smell threshold) becomes a 165-line dispatch that
delegates to graperank/{Crawl,Score,Publish,Operator,Support}. The TCP
reachability pre-probe with its dedicated 128-thread dispatcher is
transport infrastructure, not command code — it moves to quartz
nip66RelayMonitor/reachability (TcpProber, jvmAndroid). Pure move, no
behavior change; cli tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CP4kfLCa3wWtE8Khy21Pkj
Context.drain/drainAllPages/requestResponse re-implemented the
subscription state machine quartz already ships in
relay/client/accessories — the CLI-specific needs (per-event
verify-and-store hook, dead-relay collection, pending-on-auth) now live
in an option-rich fetchAll variant there, and Context keeps thin
adapters. The per-domain sections bolted onto Context (Cashu seed
warming/snapshot/restore counters; the Concord stream-key AUTH
registry) move to CashuContext/ConcordAuth, with the NUT-09 restore
counter rule shared via commons CashuWalletOps so the CLI and Android
can't drift. Context.kt: 1246 -> 950 lines; behavior unchanged
(cli tests green).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CP4kfLCa3wWtE8Khy21Pkj
A relay whose domain no longer exists (lapsed registration, decommissioned
host) was treated like a busy relay: the backoff doubled from 1s and spent
about ten dials climbing to the ceiling it was always going to reach. An HTTP
upgrade rejection already jumps straight there; a name that does not resolve
deserves the same.
Matching is on the exception type rather than the message because the message
is localized and platform-specific — Android says `Unable to resolve host "x"`,
JVM on macOS says `nodename nor servname provided, or not known` — while the
class name is stable. That is why onCannotConnect appends it in the first
place. Neither message ends with "Host unreachable", so the existing check
never caught DNS failures.
Being this eager is only safe because the verdict is cheap to revisit: a DNS
answer is a property of the network, not of the relay (a captive portal or a
filtering resolver forges NXDOMAIN), and both a network-identity change and a
transport change now clear the backoff outright. The test pins that round trip,
not just the classification.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A relay's reconnect backoff was process-global and network-blind: the delay
earned on one network was still being served out on the next. The only reset
signal was OkHttpClient reference identity, which is rebuilt off the metered
bit, so it fired for wifi<->cellular and nothing else. Wifi A -> wifi B, a VPN
coming up, a captive portal clearing, and metered-wifi -> cellular all left
every relay parked on a penalty earned against a network the device had left —
up to five minutes of silence on a network that might reach the relay instantly.
The transient Off that would have reset things is swallowed by the 200ms
debounce in ConnectivityFlow, so it never rescued those cases.
Key the decision on ConnectivityStatus.Active.networkId instead, which is the
same signal SurgeDns already uses to stale its cache, and treat a genuine
network change as a full pool rebuild: every socket is bound to an interface
that no longer carries traffic, and needsToReconnect() cannot see that because
it only compares the proxy and the timeouts.
Also treat a Tor policy flip as a transport change. Flipping a Tor toggle while
Tor is already up leaves both OkHttpClient references identical, so a relay
whose transport just changed kept waiting out a backoff earned on the other
transport. Only TorRelaySettings is compared, not the relay sets that
TorRelayEvaluation also carries — those churn while an account's relay lists
load (observed firing three times in one cold start), and forgiving the whole
pool every time any list updates is far more damage than it repairs.
Adds IRelayClient.resetBackoff() (default no-op, so the existing fakes and
BleNostrClient are unaffected) rather than reusing ignoreRetryDelays, which
only skips the gate for a single attempt and still doubles the stored delay —
a relay that failed that one dial came back worse off than before.
INostrClient.resetBackoff() is deliberately separate from reconnect(): the
latter debounces, so folding this into a coalescing command would let a later
request silently drop the reset.
The decision table moves into RelayProxyClientConnector.apply() so it can be
exercised directly, without a debounce and a shared StateFlow in the way.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The kind:10002 facet-merge rules (adding a write marker to a read-only
relay promotes it to BOTH; removing one facet of BOTH demotes to the
other; removing the last facet drops the relay) lived as private
helpers in the CLI's RelayCommands. Any frontend that edits a NIP-65
list needs them, so they now live in quartz nip65RelayList as
AdvertisedRelayListMutations (applyFacet/addFacet/removeFacet/setFacet)
with commonTest coverage. Behavior unchanged; the CLI rewires to the
shared functions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CP4kfLCa3wWtE8Khy21Pkj
The Threads tab loaded kind-11/1111 with a single #h since-filter and no
limit, so a group with more threads than the relay's default result cap
silently lost the older ones. Mirror the chat history stack for threads:
- RelayGroupChannel.threadsHistory: separate RelayLoadingCursors so paging
the forum doesn't move the chat's cursor.
- buildRelayGroupThreadsHistoryFilters: per-armed-relay #h + kind-11/1111
until+limit page, the forum analog of buildRelayGroupHistoryFilters.
- RelayGroupOpenThreadsHistoryFilterAssembler: the on-demand BackwardRelayPager
("relayGroup.threads.history"), bound to the open group's threadsHistory
cursors, landing on the normal ingest path (kind-11 -> addThread).
- Threads screen: mount the history subscription, eagerly backfill to a
window on open, page older content as the list nears its end, and show a
quiet loading/caught-up footer.
Tests: RelayGroupFilterBuildersTest gains the threads-history filter shape;
RelayGroupFilterServingRelayTest gains a geode backward #h + thread-kinds
walk proving every thread is covered exactly once and the walk terminates.
The screen wiring is device-untested (flagged with the other Tier-D items).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDK63toGbE7DQxKxrQnhMU
A group-scoped content event (kind-9 chat, poll, kind-11 thread) is keyed
to its RelayGroupChannel by the relay that served it, because a NIP-29
event doesn't carry its host relay. That's correct for the group's own
host-pinned subscriptions, but a message resolved from a NON-host relay --
e.g. a quoted kind-9 fetched by id during missing-event resolution -- was
filed under GroupId(groupId, strangerRelay), a channel the group's screens
never read, so the message silently vanished (the serving-relay hazard).
LocalCache.attachToRelayGroupIfScoped / attachThreadToRelayGroupIfScoped
now, when no channel is keyed to the serving relay, redirect the stray to
the group's single confirmed host channel via redirectStrayRelayGroupContent,
keyed off RelayGroupChannel.hasRelaySignedState(). A phantom channel never
has relay-signed state, so the redirect can only ever land on a real host,
never on another phantom -- the fix is strictly safe and the common
host-pinned arrival stays an untouched O(1) fast path (the scan runs only
on the rare no-channel-for-serving-relay miss).
Also add the cache-prune gap-fill can't-miss test: drive the production
RelayLoadingCursors down a real relay, rewindTo below the window, and
confirm the pruned band re-loads with no gap.
Tests: RelayGroupContentRoutingTest (pure router + the channel signal);
RelayGroupHistoryPagingRelayTest gains the rewind reload case. The
LocalCache wiring is unit-covered at the router level but still device-
untested end-to-end (flagged in the test plan).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDK63toGbE7DQxKxrQnhMU
Round out the branch's test plan across the headless-runnable tiers.
Tier B (filter shapes) — now every assembler:
- reconnect stability: a since-only bump on the state/tail filters is not a
resend (no full replay on reconnect), while a history until step is; via
FiltersChanged.needsToResendRequest.
- directory: extract buildRelayGroupDirectoryFilter into RelayGroupFilterBuilders
(kinds 39000-39003, no d/h scope, limit 500) and point the RelayGroupsOnRelay
assembler at it, with a shape test.
- ChannelPublic relay-group branch (filterRelayGroupState): state + pinned-id
backfill and crucially NO message window.
- group notifications (filterGroupNotificationsToPubkey): #p+#h scope, kind set,
empty-guards.
- discovery #p roster augmentation (filterRelayGroupsByAuthors): the author,
#p-roster and #d-backfill filter shapes.
Tier C (serves-the-shape, against the in-process geode relay): state #d,
batched preview tail, threads, a pinned message reachable by id below the tail
window, notification #p+#h scoping, and the relay directory.
Tier C3/E1 (can't-miss + resilience): drive the production RelayLoadingCursors
backward over the wire to the bottom, and pin that a short page is not
exhaustion (only an empty page + EOSE ends the walk).
The remaining hostile-relay faults (echo-newest, no-EOSE, auth/stall) are
already covered generically by RelayLoadingCursorsTest, WindowLoadTrackerIdleTest
and BackwardRelayPagerTest; Tier D (device) and E2 (real third-party relay) are
not headless-runnable and stay flagged for a human in the test plan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDK63toGbE7DQxKxrQnhMU
Extract the pure REQ-filter construction out of the NIP-29 group-chat
assemblers into RelayGroupFilterBuilders so the exact filter each screen
puts on the wire can be unit-tested without an Account or relay client,
and point every assembler at the shared builders (dropping the duplicated
per-file kind lists).
Add two test suites from the branch's test plan:
- RelayGroupFilterBuildersTest (Tier B): pins the kinds, #d/#h scope,
per-host-relay batching, since/until/limit and all-authors shape of the
state, joined chat-tail, open chat-tail, history-pager, threads and
card-warmup joined-skip filters.
- RelayGroupHistoryPagingRelayTest (Tier C3/E1): the can't-miss-messages
property against the in-process geode relay -- a backward #h + kind-9
walk delivers every group message exactly once and stops on an empty
page, and an #h-scoped walk isolates one group from another on the same
relay even when their createdAt ranges fully overlap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDK63toGbE7DQxKxrQnhMU
Adds a confirm-then-pay flow so a 402 from a paid Blossom server can be settled
from the app instead of only being reported.
- quartz: BlossomPaymentProof (settled Cashu token / lightning preimage) with the
X-Cashu / X-Lightning retry headers; BlossomClient.mirror accepts a proof.
- BlossomPaymentHandler (Android): pays the challenge's BOLT-11 invoice via the
account's existing NIP-47 (NWC) wallet and returns the preimage — it never
handles keys or funds itself, only drives the connected wallet. Decodes the
invoice amount for display.
- Blob manager: a mirror that hits 402 now raises a payment prompt; a dialog shows
the amount and, on confirm, pays and retries the mirror, then continues with the
remaining servers. Cancel leaves the blob unmirrored.
Cashu-only servers and the composer upload path still surface a clear message;
auto-settlement there can reuse this handler next. Not yet validated against a
live paid server.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ckbnz1N94W1hnNC9xpsCNP
