Two-reviewer adversarial audit of the branch found no correctness/data-loss/
crash bugs. This closes one gap and applies three robustness fixes:
- Multi-filter search ordering: a REQ whose filters all carry a search term
(e.g. the client's search-across-kinds) was created_at-ordered via the union
path. Now relevance-ordered — unionSubqueriesIfNeeded(projectRank) projects
rank per branch, UNION ALL + GROUP BY row_id MIN(rank) dedups across branches
keeping the best score. Only when every branch is a search branch; mixed
search/non-search REQs and count/delete unions stay as before.
- prepareAuthorStreams/prepareTagStreams build cursors via buildStreams, which
closes already-prepared statements if a later prepare throws (was: stranded
checked-out, un-reset handles holding read locks in the pooled connection).
- Stream counts computed as Long so authors×kinds / values×kinds can't overflow
Int back into the eligible band and route a huge fan-out into the merge.
- Renamed CachedStatement.finalize() -> finalizeStatement(): a no-arg finalize()
is the JVM Object.finalize, risking a GC double-close of the native handle.
Tests: multi-filter search relevance + cross-branch dedup + count parity;
existing merge/cache/search suites green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
From the branch audit: verifies the search-relevance path holds under a
two-tag-key filter (requires both tags, ranks by bm25, no duplicate rows),
that count(filter) matches query size under a limit smaller than the match set
(NIP-45), and that deleting a non-searchable event (no FTS row) fires the
contentless delete trigger against an absent rowid harmlessly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
Any filter with a search term must be relevance-ranked (NIP-50), but only the
tag-free shape (makeSimpleSearch) was — search + a tag fell through to the
row-id-subquery path (prepareRowIDSubQueries/makeQueryIn), which ordered by
created_at.
prepareRowIDSubQueries gains projectRank: when a search filter joins event_fts,
it also projects the bm25 score as a `rank` column and cuts its LIMIT by rank
(most relevant, not newest); makeQueryIn(orderByRank) then presents the joined
result by that rank, created_at DESC as tie-break. Off by default, so
count/delete/union/negentropy (which must stay single-column and unranked) are
untouched. toSql wires it on whenever the filter carries a search term.
SearchRelevanceOrderTest adds a tag-scoped case (stronger-but-older outranks
weaker-but-newer, wrong-tag and non-matching excluded, limit cuts by score).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
NIP-50: results are returned "in descending order by quality of search result
... not by the usual .created_at", with the limit applied after the score. The
store sorted search by created_at DESC (pre-existing), so it returned the
newest matches rather than the best ones.
makeSimpleSearch (the search [+ kinds/authors/since/until] + limit shape) now
orders by FTS5 bm25 (ORDER BY event_fts.rank, created_at DESC as a tie-break).
Verified bm25 rank works on the contentless table through the join, and that a
stronger-but-older match outranks a weaker-but-newer one. The rarer
search+specific-tag shape and the negentropy snapshot still sort by created_at
(the row-id subquery can't carry rank; negentropy is a sync set) — documented.
This is a correctness fix, not a scaling one: bm25 scores every match, so
search latency still grows with the match set. Tests: SearchRelevanceOrderTest,
Fts5CapabilityProbe.bm25RankWorksOnContentlessTableInAJoin; QueryAssemblerTest
search plans updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
searchOrderByRowId ordered NIP-50 search by the FTS rowid (ingestion order) to
get O(limit) search, but NIP-01's limit requires the newest events by
created_at. Once ingestion diverges from created_at (any historical sync) that
returns the wrong events under a limit — a spec violation — so the flag, its
QueryBuilder branch, and its test are removed. Search stays created_at-ordered;
corpus-independent search is an external-engine job, not this index.
The contentless + rowid=row_id schema stays for the reasons that don't touch
ordering: the delete trigger now seeks by rowid (O(log n)) instead of scanning
by an FTS column (O(n)) — measured ~78× faster at 8k rows and widening, on a
path every deletion hits — and the index is smaller. Benchmark reframed around
the delete win and the honest (unchanged) search cost; plan doc updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
The scale-curve report showed the SQLite store degrading with corpus size on
NIP-50 search (~18×) and the large-IN tag watcher, while point reads stayed
flat. Three read/size changes (write path and index set unchanged):
- FTS: rebuild event_fts as a contentless FTS5 table (content='',
contentless_delete=1) keyed by rowid = event_headers.row_id. Drops the
stored content copy (smaller index); external-content can't hold the
derived indexable text, so contentless is the correct primitive. Adds an
opt-in searchOrderByRowId strategy flag: ORDER BY event_fts.rowid DESC
early-terminates (O(limit), corpus-independent) at the cost of
ingestion-order results — flat ~0.22 ms vs created_at's 4.26 ms at 200k
(~19×). reindexAll ends with 'optimize'; the periodic optimize() folds in a
bounded segment 'merge'. DB version 4->5 with a drop-and-rebuild migration.
- MergeQueryExecutor: extend the k-way merge to the tag path (kinds + #e IN
[hundreds] + limit), one cursor per (value[,kind]) stream heap-merged to the
limit, deduping events that carry several queried values. O(limit + streams)
instead of collecting all matches and sorting.
- StatementCachingConnection: pool multiple handles per SQL so the merge's
many concurrent identical-SQL cursors all hit the cache (previously only the
first did) and repeated polls reuse their per-stream statements.
Tests: contentless migration (real v4 DB upgrade), rowid-order search, tag
merge correctness (incl. cross-stream dedup), statement pool, FTS5 capability
probe, and an FTS search-scaling benchmark. Plan in
quartz/plans/2026-07-21-sqlite-query-scaling.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
The README license badge label read "Apache-2.0" while LICENSE, PRIVACY.md
and every source header are MIT. Only the static label text was wrong (the
shields.io endpoint auto-detects), but it is the license on the front page.
SKILL.md had drifted from the codebase since the Kotlin DSL migration:
- All Gradle references pointed at Groovy `build.gradle` / `settings.gradle`;
the repo is `.gradle.kts` throughout. Converted the snippets to Kotlin DSL
and matched the repo's existing `getByName("release")` style.
- The plugins block listed `jetbrainsKotlinAndroid` (gone) and omitted
`serialization` and `googleKsp`.
- compileSdk is 37, not 35. Added a pointer to libs.versions.toml so the
number has a source of truth rather than drifting again.
- The client-tag section told readers to create
`nip01Core/tags/clientTag/TagArrayBuilderExt.kt` and edit both `build()`
functions in TextNoteEvent. That file already exists at
`nip89AppHandlers/clientTag/`, and the tag is now applied centrally by the
NostrSignerWithClientTag decorator — so rebranding is a one-constant edit
to CLIENT_TAG_NAME.
- Default relays pointed at `quartz/src/main/java/...`, a path that does not
exist in the KMP layout; they live in commons `defaults/`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The remaining SmallReqFloorBenchmark waste, on the per-row replay, the
per-event live fanout, and the per-accepted-event index probe:
- LiveEventStore replay dedupe: a SeenIds holder with an inline lock
replaces the local-fn-plus-lambda that allocated one closure per
streamed row (and again per live delivery). Its HashSet is created
empty so the JVM defers the backing table to the first add — a 0-row
replay no longer allocates a 1024-slot table (was ~4 MB across the
benchmark's 1000 idle subs).
- Live fanout serializes the event body once and passes it through
onEachLive(event, body); RelaySession splices it into the per-sub
frame prefix. An event matching N live subscriptions paid N identical
Jackson passes before; now one. queryRaw's onEachLive signature gains
the body arg (EventSourceBackend default serializes inline, no
cross-sub memo, no regression). Measured: fanout 1->200 live subs
0.50 ms (2.5 us/sub).
- FilterIndex holds subscribers in one persistent map per dimension, so
candidatesFor (once per accepted ingest event) probes with the event's
own fields and allocates no IdKey/AuthorKey/KindKey/TagKey wrappers;
BucketKey now lives only in the rare register/unregister bookkeeping.
SmallReqFloorBenchmark grows a fanout stage (200 live subs, one submit)
to anchor the fanout number; it drives `live` directly and guards the
await with withTimeout so a future fanout regression fails fast.
Verified: quartz relay.server + FilterIndex suites (110 tests),
SmallReqFloorBenchmark, geode suite (126 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Three hot-path allocation cuts the SmallReqFloorBenchmark stages flagged:
- strippingSearchExtensions: index-loop guard returns the same list with
zero allocation when no filter carries a search term (every non-search
REQ/COUNT/snapshot, the overwhelming majority).
- EoseMessage/OkMessage: direct-buildString wire form on the escape-free
fast path (EOSE per REQ, OK per publish), skipping the generic
serializer's node tree; exotic subIds/reasons fall back. Shared
isEscapeFreeAscii helper in WireJson.kt, mirroring NegMsgMessage.
Verified: quartz relay.server + message-frame suites (110 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
AccountSettings declares 25 backup* fields and saves each on change, but two
were never wired into LocalPreferences — neither written to nor read back from
the encrypted prefs:
- backupKeyPackageRelayList (MIP-00, Marmot/MLS)
- backupFavoriteAlgoFeedsList (kind 10090)
Both only ever existed for the lifetime of the process. Their consumers already
implement the restore-from-backup path — KeyPackageRelayListState's
normalizeKeyPackageRelayListWithBackup falls back to the field, and
FavoriteAlgoFeedsListState's init seeds the cache from it — so that code was
dead after every cold boot and the value read as empty until relays answered.
For the key-package list that matters beyond latency: it feeds
Account.publishRelaysFor(), which decides where this account's key packages are
published so others can add it to groups, and Account.updateKeyPackageRelays()
reads it as the *previous* list when computing an update.
Found by auditing all 25 backup* fields against their five LocalPreferences
wiring sites after the same gap turned up for the Concord community list; the
other 23, NIP-29's relay-group list included, are correctly wired.
Verified on device with a persist-then-cold-boot pair: the key-package list is
absent on the first boot and restores 1.6 s after the second. The favorite algo
feeds list could not be exercised on this account (it has no kind 10090, so
there is nothing to persist); it is wired identically and its type arguments
are compiler-checked, but it is not verified end to end.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
AccountSettings has held backupConcordList and saved it on change since the
feature landed, but the field was never wired into LocalPreferences: it was
neither written to nor read back from the encrypted prefs. So it only ever
existed for the lifetime of the process, concordList() returned null on every
cold boot, and the kind-13302 joined-communities list had to be refetched from
relays before a single Concord plane could be subscribed.
Every sibling list — channel, community, hashtag, geohash, ephemeral chat,
relay group, trust provider — is persisted this way; Concord was the one that
was missed. That made it the only chat type whose rooms could not appear until
the network answered, which is the bulk of the cold-boot delay: the joined list
gates the control-plane REQ, the control plane gates the fold, and the fold
gates the channels.
Measured on device, boot -> first Concord plane wrap:
- without the backup: liveCommunities sat empty for ~56 s waiting on the
13302 fetch (first arrival from nostr.mom), first wrap at +45 s
- with the backup restored: list decoded 1.6 s after boot (30 ms), first
wrap at +6 s
Wired the same five sites the other lists use (pref key, save, read, parse,
restore). Prefs are encrypted and backupCashuWallet already sets the precedent
for persisting a secret-bearing event, so the community roots in the 13302
content are stored no differently than the wallet's.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The additive feed path re-filters the existing list whenever an incoming batch
contains a kind-5, dropping notes whose event has been deleted. Event-less
notes fell into the else branch and returned false, so they were dropped too.
An event-less row is a placeholder the filter synthesizes for a room with no
message yet — a just-joined Concord channel, NIP-29 group, Marmot group or
geohash cell. It carries no event, so it cannot have been deleted. Dropping it
removed every such row from Messages the moment ANY unrelated deletion landed,
and because this is the additive path the rows stayed gone until the next full
rebuild. A community whose channels are all quiet looked like it had never
loaded at all.
Verified on device: surviving Concord placeholders in sort() went 0 -> 14, and
a community that had been absent from Messages entirely now renders all of its
channels. Not Concord-specific — the same placeholderNote() pattern backs
NIP-29, Marmot and geohash rooms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cold boot bumped the session revision ~292 times for 3 communities, driving 22
Messages rebuilds and re-deriving every plane subscription each time. Three
compounding causes, all measured on device:
1. Every refold republished state even when the fold was identical.
ConcordCommunityState and its components were plain classes, so StateFlow
conflation never applied and a prior-epoch wrap that didn't move the
anti-rollback floor still counted as a change. Make the fold result compare
by value (AuthorityResolver holds only immutable value fields; a data class
with a private constructor is fine).
2. A control wrap bumped twice — once from ingest() returning STRUCTURAL and
once from the per-session state watcher reacting to the same refold. Add
ConcordIngestOutcome.STRUCTURAL_FOLD for the two control-plane branches so
the manager leaves those to the watcher, which (given 1) now fires only on
genuine change. Guestbook and base-rekey keep STRUCTURAL: they mutate
members/the rekey buffer, not state, so no watcher covers them.
3. refold() and controlFloorsLocked() re-opened the WHOLE wrap buffer on every
control wrap, and opening a wrap is a NIP-44 decrypt + parse — making a
backfill quadratic in decryptions (~8.6k opens to ingest 93 wraps for one
community). Memoize editions by wrap id: one open per wrap, ingest() stays
synchronous and results are unchanged.
Measured over one cold boot: revision bumps 292 -> 87, Messages rebuilds
22 -> 7, and time from first fold to all 17 channels 43s -> 7.5s.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A Concord control-plane fold is what first reveals a community's channels and
makes ConcordCommunitySession.state non-null, without which
ChatroomListKnownFeedFilter emits nothing at all for that community. None of it
flows through LocalCache.newEventBundles, so the additive feed path could not
see it: a folded channel only reached the Messages tab if a message for it
happened to arrive afterwards.
Cold boot therefore showed a subset of a community's channels, or omitted a
quiet community entirely, until some unrelated invalidation fired. Measured on
device: the Concord hub reported 3 communities / 17 channels folded in memory
while Messages rendered 3 rows and omitted one community completely.
AccountFeedContentStates already forces a rebuild for the Marmot, NIP-29,
geohash, view-mode and pin flows for exactly this reason; Concord was the one
missing collector. Add it, sampled the same way Account.kt samples this flow to
drive refreshConcordChannelIndex.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
FilterIndex registration runs on every REQ open/close but built each
new snapshot by copying both full maps — O(S) work and allocation per
REQ with S live subscriptions. Persistent (HAMT) maps keep the
wait-free single-load reads and CAS write loop while making a write
O(keys x log S) with structural sharing.
SmallReqFloorBenchmark grows a B@1k stage (1000 idle parked
subscriptions) to make the population cost visible, and its B stage
now enters queryRaw undispatched like production does: @1000 subs the
per-REQ cost drops 0.225 -> 0.151 ms and the measured population
penalty falls below run noise (was +0.011 ms per REQ).
With this and the undispatched replay, the in-process floor above the
raw store query is ~0.11 ms (was ~0.66 ms as first measured): A 0.120,
B 0.203, C 0.239 ms on a quiet machine.
Verified: FilterIndex tests, quartz relay.server suite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
SmallReqFloorBenchmark showed the per-REQ floor on small results is
dominated by pipeline, not the store (raw query 0.125 ms vs 0.785 ms
session REQ->EOSE in-process). Half of the dispatch slice was the
scheduler hop between handleReq's launch and the query coroutine:
starting the job with CoroutineStart.UNDISPATCHED runs the stored
replay and EOSE inline on the receiving coroutine (the reader-pool
acquire doesn't suspend when a connection is free), parking only at
the live tail. Measured: dispatch+frames slice 0.397 -> 0.207 ms.
Commands on a connection are processed sequentially, so nothing can
target the subscription before the job lands in the registry at the
first suspension point.
Verified: quartz relay.server suite, SmallReqFloorBenchmark, geode
full test suite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
TagAuthorIndexBenchmark at 1M events settles the flag: the DM-room
shape (kinds + authors + #p, 65 client assembler call sites) drops
14.2 ms -> 0.66 ms (~21x, growing with corpus size) while batch-insert
cost stays inside run noise (49.0 vs 47.4 us/event). Existing relay
DBs build the index on next open via ensureOptionalIndexes.
Also refreshes the docs the numbers made stale: IndexingStrategy KDoc
now records the 200k and 1M measurements instead of a TODO,
MergeQueryExecutor's tag-merge note points at the new relayBench
reactions-watch scenario, FsQueryPlanner/FsDriverSelectionBenchmark
reflect the landed cost-based pick (149 ms -> 4.0 ms at 30k events),
and RELAY.md documents that strategy flag flips materialize indexes on
the next open.
Verified: quartz jvmTest store suites, geode test (126), desktopApp
LocalRelayStore tests (5, incl. reopening a default-strategy DB with
the new pubkey-alone flag), relayBench compiles.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Acts on the measured gaps from TagAuthorIndexBenchmark and
FsDriverSelectionBenchmark:
- FsQueryPlanner: replace the fixed tags -> kinds -> authors driver
order with a cost-based pick. Every legal driver (each tagsAll value,
each tags key's value union, the kind set, the author set) opens a
lazy directory iterator; all are drained in lockstep and the first to
exhaust (the smallest listing) drives, so a giant idx/kind tree is
never read past ~the smallest candidate's size. Fixes the 149 ms vs
3.4 ms (~44x at 30k events) authors+kinds+limit regression.
- EventIndexesModule.ensureOptionalIndexes + SQLiteEventStore: flag-
gated indexes are runtime config, not schema. An idempotent
CREATE INDEX IF NOT EXISTS pass now runs on every open, so flipping
an IndexingStrategy flag on an existing DB builds the index without
a user_version bump.
- Desktop LocalRelayStore: enable indexEventsByPubkeyAlone. Shared
ViewModels (Nip65RelayList, PrivateOutboxRelayList, VanishRequests)
replay authors-only filters that full-scanned without
(pubkey, created_at); existing DBs pick the index up on next open.
- relayBench Scenarios: add "conversation" (tag ∩ author ∩ kind, the
DM-room shape, 65 client assembler call sites) and "reactions-watch"
(kind 7 + #e IN 150 hottest notes) so the uncovered archetypes get
head-to-head numbers vs strfry.
- quartz build: forward tagBenchScale/fsBenchScale to the test JVM.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Real numbers from the two new benchmarks, so the trade-offs are on the
decision points instead of in a chat log:
- IndexingStrategy.indexTagsWithKindAndPubkey: the KDoc called the
kinds+authors+tags shape "rarely used", but the client assembler
survey found 65 call sites. TagAuthorIndexBenchmark @ 200k events:
DM-room query 9.4 ms -> 0.6 ms (~15x) with the flag on, insert cost
+14% (41.5 -> 47.3 us/event). TODO: re-evaluate defaults (geode).
- MergeQueryExecutor: tag-path analogue of the follow-feed collect-all
sort (kinds + #e IN [hundreds] + limit never merges). Measured
12.8 ms cold / 6.0 ms since-bounded at 200k events; revisit if
relayBench shows it at relay scale.
- FsQueryPlanner: fixed driver order sends authors+kinds+limit (the
most common CLI shape) through the kind tree. FsDriverSelection-
Benchmark @ 30k events: 149 ms -> 3.4 ms (~44x) driving from the
author tree; TODO: cost-based pick by directory entry counts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
The 2026-07 client filter-assembler survey mapped 551 Filter
constructions to ~12 query archetypes. Two hot shapes had no benchmark
coverage in prodbench or relayBench, and the FS store had none at all:
- TagAuthorIndexBenchmark: the DM-room shape (kinds + authors + #p,
65 assembler call sites) with indexTagsWithKindAndPubkey off vs on,
including the insert-cost delta of the extra index; plus the
reactions watcher (kinds=[7], #e IN 300, limit) cold and
since-bounded, which has no tag-side k-way merge today.
- FsDriverSelectionBenchmark: FsQueryPlanner's fixed driver order
(tags → kinds → authors) on authors+kinds+limit — the most common
CLI shape — comparing the current kind-tree driver against an
author-tree driver with kind post-filter.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
the files are now pure-ASCII and visually unambiguous:
- BlossomPaymentSafetyTest: raw U+202E/U+202C test payload -> escapes
- BlossomPaymentRequired: BIDI_OVERRIDES char array -> escapes
- Sanitizer: RTL_OVERRIDES and ZERO_WIDTH regex classes -> escapes
Verified by mutation: with BIDI_OVERRIDES stripping disabled,
reasonBidiOverridesAreRemoved fails, proving the escaped payload still
carries a real U+202E. Emoji ZWJ sequences in RichTextParserTest are
intentionally untouched (functional joiners, not bidi controls).
test(share): pin naddr encoding for addressable notes in QR payloads
feat(share): add strings for the QR share screen
feat(share): add fixed-height note card for the QR screen
feat(share): add display-only QR screen for notes
feat(share): register the ShareNoteAsQr destination
feat(share): add Share as QR to the note share sheet
fix(share): gate NSFW thumbnail with ContentWarningGate, prefer article title
fix(share): size the QR from available width, add a11y description
fix(share): add QR a11y strings, document QR row in ShareActionRows KDoc
fix(share): compact permanently-covered NSFW thumbnail, fix inert QR width cap
fix(share-qr): close sensitivity gate leaks and text/thumbnail bugs on the share-as-QR card
fix(share-qr): fix stale payload, unreachable controls, missing back button, and screen-wake handling
fix(share-qr): show image thumbnail for image-only notes, hide raw media URL
Merges nostr proposal 2ca4f8ae into main:
- feat(cli): full NIP-34 git collaboration parity for `amy git`
- feat(cli): `amy git grasp list|set` (NIP-34 GRASP server list, kind 10317)
- feat(cli): `amy git browse|cat|log` — read git objects over smart-HTTP
- feat(cli): `amy git init` — bootstrap a repo from the local git checkout
- feat(cli): `amy git label` (NIP-32) and `amy git apply` (patch -> working tree)
- fix(quartz): NIP-34 wire-format interop with ngit (clone/web, issue p, plain r)
- test(cli): live interop check against the real ngit-published amethyst repo
- fix(cli): shallow-clone euc, read truncation, process deadlock
- fix(cli): read routing, status perf, publish-ack + robustness
- fix: complete PR clone multi-value (kinds 1618/1620) + CLI robustness
- revert(amethyst): drop GitStatusIndex auth change; defer to separate proposal
clone/web now serialize as single multi-value tags per NIP-34 (ngit drops
repeated ones) for kinds 30617, 1618 and 1620; readers stay tolerant of the
legacy repeated form.
The GitStatusIndex status-authority guard was reverted before merge: it
authorized against the status event's own `a` tag, which the author controls,
so a forged ["a","30617:<attacker>:x"] defeated it — and unverifiable statuses
were dropped permanently (no re-reduce when the 30617 announcement arrives),
rendering closed issues as open. Refiling as its own proposal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The isAuthoritative guard added to GitStatusIndex read the repository owner
from the status event's own `a` tag (GitStatusEvent.repositoryAddress() ->
first a tag, no kind filter, no cross-check against the target's repo). That
value is attacker-controlled: a forged kind:1632 carrying
`["a", "30617:<attacker-pubkey>:anything"]` makes `status.pubKey ==
repoAddress.pubKeyHex` pass, so the spoof it meant to block still succeeds.
It also regressed reads: reduceLatestByTarget only re-runs on a new
kind 1630-1633, so a status dropped while the 30617 was uncached stayed
dropped, leaving genuinely-closed items in the Open tab with wrong counts.
Keep this series focused on the quartz + cli NIP-34 parity work. The Android
status-authorization hardening (resolve the repo from the target item, load
the cached 30617, authorize against repo.pubKey + maintainers() + the target
author, and re-reduce when a 30617 arrives) will land as its own proposal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Addresses review findings from a merge-time audit.
- **Complete the headline multi-value `clone` fix for PRs** (was applied only to
kind:30617). GitPullRequestEvent (1618) and GitPullRequestUpdateEvent (1619)
carry `clone` with the same spec shape but still emitted repeated single-value
tags and read only the first value — so the exact interop bug this branch set
out to kill was still live for PRs, both directions (ngit keeps only the last
repeated tag; we lost every URL after the first from ngit's multi-value tag).
Now both emit one multi-value `["clone", …]` tag and read both forms. Verified
on the wire + GitNip34InteropTest + CLI harness (40 checks).
- **Android git-status spoofing (GitStatusIndex)**: newest-status-wins with no
author check meant anyone could publish a kind-1632 and make someone else's
issue render closed. Now filter statuses to the repository owner (from the
status's own `a` tag), declared maintainers (from the cached announcement), or
the target item's author — matching NIP-34 and the CLI's derivation. Pre-existing
on main; this branch made the CLI/Android divergence visible.
- **CLI robustness**: `git comment`/`git patch` no longer block forever reading
stdin on an interactive TTY (amy is non-interactive — error instead). The local
`git` subprocesses in `git init`/`git apply` now drain stdout on a side thread
under a bounded `waitFor` + `destroyForcibly`, so a wedged git can't hang the
CLI.
Left as a follow-up (cosmetic): GitBrowseCommands.candidateUrls duplicates
GitRepositoryBrowserViewModel's — worth lifting to shared code, not worth the
cross-module coupling here.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Findings from a second review round (two independent reviewers), with fixes:
Read path (git issues/patches/prs/thread):
- **Reads ignored the repo's own relays** (correctness). They queried only the
account outbox/bootstrap (general relays); NIP-34 events live on the repo
announcement's advertised relays (often GRASP/git-specific), which general
relays don't mirror — so `amy git issues <repo>` with no --relay could return
empty. Now fetch the announcement once and read from queryTargets ∪ its
advertised `relays`. Verified live: `git issues`/`git prs` on the amethyst
repo now return real events (and derive `closed`) with NO --relay.
- **O(items × statuses) status rescan** with un-memoized `rootEventId()` reparse
→ pre-group statuses by root id once (O(1) lookup per item).
- **Status query could truncate / exceed relay caps**: statuses are now paged
(`drainAllPages`) and the `#e` id set is chunked to 50 (under the common
~100-value relay filter cap).
- **Latency regression**: capped the list `drainAllPages` idle timeout to 12s
(was the 30s default; `drain` had been 8s).
- **Nondeterministic status on same-second ties** → deterministic id tie-break.
- Reuse the fetched repo for the maintainer set (removes a redundant round-trip).
Write path:
- **`git init` silently reported success when the 30618 state publish failed**
— its ack was dropped. Now surfaced as `state_published_to`/`state_rejected_by`
with a stderr warning on total rejection.
- **`git apply`** feeds stdin as UTF-8 (was JVM default charset — corrupted
non-ASCII patches) and joins the stdin thread in `finally` (no leak on error).
- **`normalizeCloneUrl`** drops the port from `ssh://git@host:port/…` (it was
carried into the https URL, making it unreachable).
- **Delivery fallback** to the account outbox (repo unresolved / no advertised
relays) now warns to stderr instead of reporting silent success.
Known limitation (documented, not fixed): patch-revision-chain status derivation
follows only the root item, and `git thread` shows first-level replies only
(nested trees and 1619 PR-updates are out of scope). 38/38 harness green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Findings from a review pass over the git-parity branch, with fixes:
- **`git init` announced a WRONG earliest-unique-commit on shallow clones**
(interop-critical). `git rev-list --max-parents=0 HEAD` returns the shallow
boundary commits, not the true root, so the repo would be announced under a
different cross-fork identity than ngit computes. Now: detect shallow clones
and omit the euc with a warning to pass `--earliest-commit`; on full clones
derive the deterministic `--first-parent` mainline root instead of an
arbitrary `tail -1`.
- **`git issues|patches|prs` silently truncated and mis-derived status** on
active repos: one single-page `drain` pulled items AND status events under a
shared cap, so status events (newer, more numerous) could crowd items out of
the window and the close-status that determines an item's state could fall
outside it → a closed item read as open. Now paginate the items
(`drainAllPages`) and fetch exactly the statuses that `e`-reference them.
Verified on the live amethyst repo: 51 PRs paginated, 19 correctly closed.
- **Pipe-buffer deadlocks** (latent): `GitInitCommand.git()` discards stderr to
the OS (a chatty command can no longer fill its stderr pipe and hang the
stdout read); `GitApplyCommand.runGit()` writes stdin on a background thread
while draining stdout, so a patch larger than the pipe buffer can't deadlock.
- Minor: `git cat` binary detection uses an index loop instead of boxing 8000
bytes; `GitRepositoryEvent.clones()/webs()` dedupe.
The harness `git init` test now runs against a fresh full checkout (this repo's
CI checkout is shallow) and adds a shallow-clone case asserting the euc is
omitted. 38/38.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Adds a `--live` block that reads the actual amethyst repository ngit publishes
to relay.ngit.dev and asserts our reader parses ngit's real multi-value `clone`
tag (currently 4 URLs) plus its published issues. This is the real-world proof
of the multi-value interop fix: the pre-fix reader would have surfaced only the
first clone URL. Opt-in (needs network + the live relay), skipped by default.
Verified manually end-to-end against the live repo: repo announcement (4 clone
URLs), issues (1621), patches (1617), pull requests (1618, with a real `closed`
status derived from ngit's status event), and a NIP-22 comment via `git thread`
all read correctly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Verified Amethyst's NIP-34 events byte-for-byte against the ngit reference
implementation (DanConwayDev/ngit-cli) and the spec, and fixed three real
interop divergences in quartz — so ngit/gitworkshop and Amethyst read each
other's git repos, issues, patches, and PRs without losing data.
- Repository announcement `clone`/`web` were emitted as REPEATED single-value
tags (`["clone", a]`, `["clone", b]`). The spec and ngit use ONE multi-value
tag (`["clone", a, b]`), and ngit's parser keeps only the LAST of repeated
known tags — so multi-URL repos silently lost every URL but one in both
directions. Now emitted as a single multi-value tag; `clones()`/`webs()` read
BOTH the spec form and the legacy repeated form, so old events still parse.
(`relays`/`maintainers` were already correct multi-value tags.)
- Issues (kind 1621) were missing the `["p", <repo-owner>]` tag that patches and
PRs already include — a maintainer watching `#p` wouldn't see them. The
builder now adds it (fixes both the CLI and the Android issue-creation path,
which both passed an empty notify list).
- Patch / PR / PR-update `r` tags carried the `"euc"` marker
(`["r", commit, "euc"]`). Per the spec and ngit that marker belongs only on
the kind-30617 announcement; other `r` tags are plain `["r", commit]`. A `#r`
filter matches either shape, so this is a spec-compliance/byte-parity fix.
`alt` (NIP-31) tags are intentionally still omitted — quartz treats the generic
alt client-hint as deprecated, and ngit/gitworkshop parse the structured tags,
so it isn't required for interop.
Adds `GitNip34InteropTest` (5 cases: multi-value write, tolerant read of both
forms, issue p-tag, plain patch r-tag) and 4 wire-format assertions to the CLI
git harness (37 offline). No regressions in the nip34 or Search suites.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Close two more ngit/nak parity gaps:
- `git label TARGET LABEL[,LABEL]` — attach NIP-32 kind:1985 labels to an
issue/patch/PR (the `ngit pr label` / `issue label` surface), over quartz's
existing `LabelEvent`. Namespace defaults to `ugc`; `--namespace` overrides.
- `git apply PATCH_ID` — fetch a kind:1617 patch and apply it to the local
working tree via `git am` (the `nak git patch apply` / `ngit pr apply`
surface); `--check` dry-runs `git apply --check`, `--print` emits the patch.
Shells out to `git` like `git init`, since it operates on the local checkout.
Verified end-to-end: a patch published to a relay, fetched, and `git am`'d as a
real commit into a scratch repo; labels land as kind 1985. The harness gains 5
assertions (label + a full publish→apply round-trip), now 33 offline.
Remaining out-of-scope items are documented: git-packfile push (needs a git
write layer quartz lacks) and NIP-34 cover notes (kind 1624, no quartz builder).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Match `ngit init` / `nak git init`: read the local git repository and publish a
NIP-34 repository announcement, deriving the fields instead of making the user
type them. Shells out to `git` to determine the name (top-level dir), clone URL
(origin remote, ssh→https normalized), earliest-unique-commit (root commit),
and — for the accompanying kind:30618 state — the branch/tag tips and HEAD.
Publishes the 30617 announcement and (unless `--no-state`) the 30618 state in
one shot. Every derived value is overridable with a flag; outside a git repo
the derivation is skipped and `--name`/`--clone` are supplied manually.
This is the one `amy git` verb that shells out to `git`, since it is inherently
about the local working tree — exactly like the tools it mirrors.
Verified against the amethyst checkout itself (derives name=amethyst, the origin
clone URL, the root commit as EUC, and a 30618 with the live branches + HEAD).
The harness gains 4 assertions driving `git init` against its own checkout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Give `amy git` the git-object read side of `nak git download` / a shallow
clone. `browse` lists a repo's tree, `cat` prints (or `--out` writes) a file at
a ref, and `log` shows recent commit history — all over the git smart-HTTP v2
protocol via quartz's `GitHttpClient` (the same shallow-clone path the Android
repo browser uses). REPO may be a NIP-34 coordinate/naddr (whose announcement
supplies the clone URL) or a raw http(s) clone URL; `--clone` and `--ref`
override the URL and branch/tag.
Read-only: pushing git objects back to clone/GRASP servers stays out of scope.
Verified live against a public repo (octocat/Hello-World) — browse/cat/log all
return correct trees, blobs, and history. The harness gains a `--live` block
(28 assertions with `--live`, 24 in the default offline run) exercising these
against `$LIVE_REPO`, skipped by default since it needs a reachable git host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Declare/read a user's preferred GRASP (Git-over-Nostr hosting) servers in
preference order — the NIP-65-style list `ngit`/`nak git` consult to decide
where PR tip branches (`refs/nostr/<pr-id>`) get pushed. `set` publishes a
kind:10317 to the outbox; `list` reads it back cache-first (anonymous-capable).
Thin assembly over quartz `UserGraspListEvent`. The git push itself stays out
of scope, as with the rest of the packfile transport.
Extends the git NIP-34 harness with a grasp round-trip (24 assertions) and
updates the README/ROADMAP/help tables.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Extend `amy git` from repo announce/list/show/issue to the complete
pure-Nostr surface of `ngit` and `nak git`, so every NIP-34 collaboration
flow is scriptable without a GUI.
New sub-verbs (all thin assembly over quartz's `nip34Git` builders):
- `git state` — kind:30618 repository state (branch/tag tips + HEAD)
- `git patch` — kind:1617 patch from `git format-patch` (--file or stdin),
with --root/--root-revision, --commit, --parent-commit,
and --in-reply-to for revision chains
- `git pr` / `git pr-update` — kind:1618 pull request + kind:1619 tip update
- `git comment` — NIP-22 kind:1111 reply on an issue/patch/PR/repo (the
modern replacement for the deprecated kind:1622 git reply)
- `git open|applied|close|draft` — kind:1630/1631/1632/1633 status events
(aliases `merged`/`resolved` for applied); applied carries
--merge-commit / --commit / --patch
- `git issues|patches|prs` — list a repo's items with status derived from the
newest authoritative (owner/maintainer/author) status event,
with --open/--applied/--closed/--draft/--status filters
- `git thread` — one item plus its status timeline and comments
Shared parsing/fetch/routing glue lives in `GitSupport`; the existing
announce/list/show/issue verbs now reuse it. The git *packfile* transport
(clone/fetch/push of real objects to clone/GRASP servers) stays out of
scope — it needs a git plumbing layer, not an event builder — and is
documented as such.
Adds `cli/tests/git/git-nip34-headless.sh` (21 assertions, drives the whole
flow against `amy serve` and checks the status-deriving reads) and updates
the README/ROADMAP command tables.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
