Running the Marmot headless harness against the embedded geode relay
failed 10 of 29 scenarios, every one on the same reply: the relay
answered a resent EVENT with
["OK", <id>, false, "Error code: 2067, message: UNIQUE constraint
failed: event_headers.id"]
NIP-01 says a relay that already holds the event answers
["OK", <id>, true, "duplicate: already have this event"], and every
client here depends on that: amethyst's outbox writes an event as soon
as the socket is ready and resends it when the connection finishes
syncing, so one of the two copies is always a duplicate; MDK's wn
counts a `duplicate:` prefix as idempotent success but files an
unclassified OK false as "publish acknowledgement unknown" and keeps
retrying. Both amy's group commits and wn's KeyPackage publish were
failing on it, while nostr-rs-relay had answered the resend correctly.
SQLiteEventStore now recognises the unique-index violation on
event_headers.id and reports RejectionReason.DUPLICATE, the constant
that already carried NIP-01's exact wording but was never produced;
RelaySession sends `OK true` for a `duplicate:` reason and keeps
`OK false` for every other rejection. The store outcome stays Rejected,
so a duplicate is still not fanned out to live subscriptions or counted
as a new write by the mirror worker and importer. The filesystem store
already treated a duplicate insert as a no-op.
Two tests pinned the old OK false behaviour (NostrServerTest,
KtorRelayTest) and now assert the NIP-01 reply.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PguqnDbP2v11dtANs9xdxc
Every build printed "Deprecated Gradle features were used in this build,
making it incompatible with Gradle 10". With --warning-mode all that was
five distinct Kotlin DSL delegated-property deprecations, all in our own
scripts:
- `val x by extra(...)` / `val x: T by extra` in the root script, for the
opt-in Sonar gate that buildscript {} publishes and the body reads. Now
extra.set("x", v) and extra["x"] as T.
- `val x by getting { }` for eight of quartz's KMP source sets. Now
getByName("x") { }, which is what commons already used. None of those
vals were referenced, so the local binding goes away with them.
- `val x by tasks.registering { }` and the typed
`by tasks.registering(T::class) { }`, thirteen tasks across quartz,
commons, cli, geode, nestsClient and desktopApp. Now
tasks.register("x") { } and tasks.register<T>("x") { }, which return the
same TaskProvider, so the dependsOn / finalizedBy references to them are
unchanged.
`./gradlew --warning-mode all help` is now silent, and all nineteen
converted tasks still register and run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123kXtseu4X18hL3GMDcdER
app 1.15.1 -> 1.15.2, appCode 459 -> 460. That single edit drives Android's
versionName/versionCode, Desktop and CLI packageVersion, quartz's Maven version
and geode's RelayInfo.VERSION.
Three substantive PRs since v1.15.1, plus Crowdin translations and the packaging
syncs the bump workflows opened after the last tag:
- #4092 media previews: an extension match now requires a real dot, so a player
page whose path merely ends in the letters `_mp3` stops going to the video
player, and `og:audio`/`og:video` are read and played with the page's
`og:image` as poster. A declaration whose type is `text/html` -- YouTube's --
is refused.
- #4095 nested NIP-22 replies: engagement subscriptions asked only for the
lowercase `e`/`a` tags, so a comment two or more levels deep was invisible
until ThreadScreen opened its own subscription. Each relay gets a second,
root-scoped filter on `E`/`A`. Kind 1619 moves there too -- NIP-34 gives PR
updates only an uppercase `E`, so it had been in a filter it could never match.
- #4096 Health Connect: a rationale screen Play requires, reachable from the
composer, from Health Connect's permission screen and standalone without an
account; reads moved off the UI thread; source names memoized; and the workout
form is replaced rather than merged when a second suggestion is picked.
Verified on a Pixel 9 emulator before cutting, since two of the three are only
observable on device: the og:audio track plays in a thread with real transport
controls (00:30 / 05:00) where it used to buffer forever, and the Health Connect
rationale opens from all three routes -- including the one that matters for
review, where Health Connect's own permission screen launches our
ViewPermissionUsageActivity through the START_VIEW_PERMISSION_USAGE-guarded
filter.
RELEASE_NOTES_ID deliberately stays on the v1.15.0 note: RELEASE_OPS has it
repointed on x.y.0 only.
Left alone deliberately: everything under */packaging/ and translators.json's
tag, which the bump workflows and the Crowdin job write after the tag exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
v1.15.0 was tagged but never shipped. Its `Create Release Assets` run failed in
`deploy-android` at the first packaging task:
Execution failed for task ':amethyst:buildFdroidReleasePreBundle'
> Entry name contains invalid characters:
root/META-INF/zoomable-root:zoomable.kotlin_module
so no AAB, no APK, and none of the 47 assets were produced.
A `.kotlin_module` is named after the Gradle project path that produced it,
colons included. 14 of the 98 modules merged into the app carry one: zoomable,
Negentropy, vico, the seven coil3 artifacts, and four of ours -- Amethyst:quartz,
Amethyst:commons, Amethyst:quic and Amethyst:nestsClient -- so renaming our own
would not have been enough.
Bisected against the two toolchain bumps this cycle, since both landed after the
last good release: AGP 9.3.1 -> 9.4.0 and Kotlin 2.4.10 -> 2.4.20. With Kotlin
held at 2.4.20 and AGP reverted, both flavours' bundle tasks pass, so Kotlin is
not the trigger. The R8 output jar carries the identical 14 colon entries under
BOTH AGP versions -- 9.4.0 added the rejection rather than the names, in
JarFlinger.addJar, reached from PerModuleBundleTask.addHybridFolder.
`packaging.resources.excludes` was tried first and cannot work, at either
`META-INF/*.kotlin_module` or `**/*.kotlin_module`: with minification on, R8
emits the java resources itself and addHybridFolder hands JarFlinger its own
predicate, so those filters are never consulted. Confirmed by deleting the R8
output and re-running rather than reading a stale intermediate --
mergeJavaResource's jar holds zero kotlin_modules while R8's holds all 98.
So the entries are stripped from R8's jar in the moment before the bundle task
opens it, and the jar is put back exactly as R8 left it afterwards. Two details
carry their weight:
- the strip is doFirst on the CONSUMER rather than doLast on R8, so a
build-cache hit on R8 cannot skip it;
- the restore is what keeps R8 up to date. Without it Gradle sees a modified
output and re-runs R8 on every build -- measured here at ~2 min for an
otherwise no-op build. With it, a second run reports
minifyFdroidReleaseWithR8 UP-TO-DATE and finishes in 1s.
Pinning back to 9.3.1 was the alternative and is one line away; the catalog
comment records that, and says to drop the workaround when AGP fixes it.
Verified from a cleaned R8 output on both flavours:
buildFdroidReleasePreBundle and buildPlayReleasePreBundle both BUILD SUCCESSFUL,
each reporting "Stripped 14 colon-named entries from base.jar".
appCode 458 -> 459. RELEASE_NOTES_ID deliberately stays on the v1.15.0 note:
RELEASE_OPS has it repointed on x.y.0 only, and 1.15.1 ships that release's
contents.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
app 1.14.0 -> 1.15.0, appCode 457 -> 458. That single edit drives Android's
versionName/versionCode, Desktop and CLI packageVersion, quartz's Maven version
and geode's RelayInfo.VERSION.
RELEASE_NOTES_ID is repointed at the v1.15.0 note
(8fce45589ea44df75e828a04c7d70bb4fabedd6ffc1946a920b2f0c7c990ff9f), which
RELEASE_OPS notes happens on x.y.0 releases and not on patches. It has to ship
in this commit rather than after it: the drawer's "Release Notes" link and the
donation card both open BuildConfig.RELEASE_NOTES_ID, so a tag cut before the
repoint ships users a link to the previous release's note. Verified present on
relay.damus.io and relay.primal.net before committing.
Adds docs/changelog/v1.15.00.md, written from the 556 commits since v1.14.0,
and its index entry. The cycle's headline is the Marmot resync: the MIP
documents were deprecated in July and MDK followed, leaving our implementation
invalid under either profile the current spec defines, so it moves onto the
adopted current profile and is now interoperable with White Noise. Marmot group
chat itself is not new -- it shipped in v1.09.0 -- and the notes say so.
Also syncs the docs that state a version rather than illustrate one, since
quartz and geode both read libs.versions.app:
- README.md, quartz-integration SKILL.md and its gradle-setup.md reference
-> quartz 1.15.0
- geode/README.md install commands -> geode 1.15.0
The Homebrew/Winget status blocks in BUILDING.md and RELEASE_OPS.md were
re-verified rather than re-stamped, and the claim had gone stale in our favour:
both Homebrew packages are live upstream now. formulae.brew.sh answers 200 for
the amethyst-nostr cask (at 1.14.0) and for the amy formula, while geode-relay
404s and microsoft/winget-pkgs still has no VitorPamplona/Amethyst -- PR #422752
is open pending CLA. Both blocks now say that, RELEASE_OPS gains a geode-relay
row, and the section heading no longer claims Homebrew is not shipping.
Left alone deliberately: everything under */packaging/ and translators.json's
tag, which the bump workflows and the Crowdin job write after the tag exists
(bumping by hand would commit wrong hashes and a dead URL); and
cli/tests/marmot/state/mdk/Cargo.lock, where 1.14.0 is an unrelated crate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
Two unrelated things that both amount to not paying for something nobody
asked for.
**`MirrorSyncThroughputTest` is a benchmark, so it now opts in.** It
preloaded a million events and pulled them over a real WebSocket on every
ordinary test run: 4,584 s of `:geode:test`'s 4,636 s — 98.9% of the
module's test time for one test that asserts nothing about correctness and
reported `skipped` at the end anyway. Every other benchmark in the module
is already gated this way (`perf.LoadBenchmark`). It now bails before
building anything, and enables on `-DrunLoadBenchmark=true` OR on any of
its own sizing properties, so every invocation its kdoc documents still
runs it — naming a size is itself the opt-in. Measured after: the test
takes 5 ms, the module takes 64.8 s, and `-DsyncN=2000` still prints a
throughput number.
**The agent text stream QUIC path is kept but no longer advertised, and
nothing starts it.** Nothing in the deployed network publishes those
previews. So:
- `SUPPORTED_COMPONENTS` drops `0x8006` and the leaf capabilities drop
`0xF2D1`/`0xF2D2`/`0xF2D4`. A capability is a standing promise to every
peer that reads our KeyPackage, and one for a path nobody exercises
costs something and buys nothing. The captured reference KeyPackage in
our own conformance vector does not advertise `0x8006` either.
- The Android chat screen no longer builds a stream watcher and dials the
brokers a kind:1200 advertises. That was a UDP connection attempt to a
third-party endpoint on every feed change, on behalf of a feature with
nothing to show — a service we start, not a capability we hold.
The implementation stays and stays tested: `:marmotQuic`, the codecs,
`amy marmot stream`, the direct path, the certificate pinning and the
interop tests are all untouched. The module README records the posture and
the exact way back.
Three tests asserted the old advertisement and were reworked rather than
deleted. The role-enforcement gate is still covered — the tests now build
leaves that explicitly carry the roles, which is the better shape anyway,
since a test that exercised the gate through OUR default was really
asserting the default and stopped testing the gate the moment it changed.
A new test pins the new default: our KeyPackage carries no role and is
therefore refused by a group requiring one. That refusal is the deliberate
cost, so it is asserted rather than discovered.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`geode` can never be a homebrew-core formula: `formula_renames.json` maps
"geode" -> "apache-geode", so the token is permanently reserved and
`brew info --formula geode` resolves to Apache Geode. The previous commit
recorded that as a blocker; this removes it.
- `geode/packaging/homebrew/geode.rb` -> `geode-relay.rb`, `class Geode` ->
`class GeodeRelay` (Homebrew requires the class to track the filename).
- `bump-homebrew-geode-formula.yml` follows the path, and the three sibling
workflows' header comments now name the formula correctly.
- `geode/README.md` points at the new file and the new tap install line.
**The binary is still `geode`.** Users type `geode`, not `geode-relay`. That is
safe rather than sloppy: apache-geode installs `gfsh`, so nothing collides on
PATH. Formula token and binary name differ deliberately, which the header now
states so nobody "fixes" it later.
Verified: `brew style` clean on the renamed file (it validates class-vs-filename
agreement, so this catches a bad rename), `brew info --formula geode-relay`
resolves to this relay rather than Apache Geode, `ruby -c` passes, and replaying
the bump workflow's `sed` still changes exactly the two intended lines.
`geode/plans/2026-07-24-geode-release.md` is left alone — a dated design doc,
not live configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gs2gi3sZfQ7SHrVm2njLMw
Checked while asking whether the amethyst-nostr cask's review feedback applied
to the two formulae. It does not — but running homebrew-core's own linter over
them turned up a real defect neither had been checked for.
**The style violation, in both files.** `brew style` flags
Homebrew/FormulaPathMethods: Use formula_opt_prefix("openjdk")
instead of Formula["openjdk"].opt_prefix
on the `write_env_script` line. Fixed in `amy.rb` and `geode.rb`; both now
report no offenses. It would have been raised on submission.
**A duplicated sentence.** amy.rb opened with "Reference Homebrew formula for
`amy`, the Amethyst CLI." twice, once on line 1 and again on line 3.
**Why they must NOT be made to match the cask.** The cask lost its `livecheck`
block and inline comments on review, so the obvious next step is to do the same
here. That would be wrong, and the header now says so with the evidence:
homebrew-cask and homebrew-core differ. Sampling the live core tap, 127 of 300
formulae with GitHub-release URLs declare `livecheck` (62 using
`:github_latest`), and 109 of 200 carry indented inline comments. `livecheck`
is load-bearing in core — it is what lets BrewTestBot open version-bump PRs, so
stripping it would disable exactly the automation the block exists for.
**geode cannot be submitted under that name.** homebrew-core's
`formula_renames.json` maps "geode" -> "apache-geode", so the token is
permanently reserved and `brew info --formula geode` resolves to Apache Geode.
Submitting needs a different token (`geode-relay`, `amethyst-geode`) plus a
matching change to bump-homebrew-geode-formula.yml. Recorded as a blocker in
the header rather than discovered at PR time.
**amy is unblocked but not ready.** The one-open-AI-PR limit that gated it is
cleared now the cask has merged; the ~70 MB bundle from `:commons` pulling
Compose/Skiko onto the CLI classpath is still the likely review objection, and
`brew audit --new --formula` has not been run end to end.
Verified the enlarged headers cannot confuse the bump workflows: both anchor on
`^ url ` / `^ sha256 ` at a two-space indent, each matches exactly once, and
replaying their `sed` changes those two lines only. `ruby -c` passes on both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gs2gi3sZfQ7SHrVm2njLMw
app 1.13.1 -> 1.14.0, appCode 456 -> 457. That single edit drives Android's
versionName/versionCode, Desktop and CLI packageVersion, quartz's Maven
version and geode's RelayInfo.VERSION. RELEASE_NOTES_ID is repointed, which
RELEASE_OPS notes happens on x.y.0 releases and not on patches.
Also syncs the docs that state a version rather than illustrate one. quartz
and geode both read libs.versions.app, so their install snippets were stale
claims about what Maven Central and the release assets actually carry:
- README.md, quartz-integration SKILL.md and its gradle-setup.md reference
-> quartz 1.14.0
- geode/README.md install commands -> geode 1.14.0
The Homebrew/Winget "not bootstrapped" notes in RELEASE_OPS.md and
BUILDING.md were stamped v1.13.1. Re-verified before moving the stamp rather
than re-stamping blind: Homebrew/homebrew-cask has no amethyst-nostr.rb and
microsoft/winget-pkgs has no VitorPamplona/Amethyst, both still 404, so the
claim holds. The bump-script invocations beside them named v1.13.2, a tag
that never existed, and are now copy-pasteable.
Left alone deliberately: everything under */packaging/ and translators.json's
tag, which the bump workflows and the Crowdin job write after the tag exists
(bumping by hand would commit wrong hashes and a dead URL); BUILDING.md's
asset-name and git-checkout samples, which are illustrations; and the
"invisible until v1.13.1" line in RELEASE_OPS.md, which is history.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gs2gi3sZfQ7SHrVm2njLMw
The app dials ~190 relays in one burst from RelayPool.connect() and spikes to
~640 threads, so capping the relay client's Dispatcher.maxRequests looks like a
one-line throttle.
It is not. Against a real relay, maxRequests=4 with 20 dials opens exactly 4;
the other 16 queue forever and never fail, so nothing surfaces the stall.
Setting maxRequests=N would cap the app at N relays permanently.
Pins the behaviour so the knob is not reached for again. Throttling has to
happen above OkHttp, in RelayPool, where a settled dial can release its permit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`onDrained` was the wrong shape. It reported the one ending a coverage
caller happens to need and threw the rest away, so a CLOSED and an idle
timeout still arrived indistinguishable from a clean finish — the very
conflation this branch set out to remove, just moved one step along.
`fetchAllPages` now returns `PagedFetchResult(downloaded, end)`, where
`end` names every way the loop can stop: DRAINED, LIMIT_REACHED, IDLE,
CLOSED, CANNOT_CONNECT, UNPAGEABLE. `drained` stays as a shorthand on the
result so the meaning lives in one place. A caller can no longer ignore
the reason by accident, and the two failure endings are now reportable
rather than silently swallowed.
I argued for the callback on the grounds that ~25 call sites use the
`Int`. That was overstated: most call it as a statement and never touch
the return. Six needed a `.downloaded`, all mechanical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TNy5BsU9NErXYa3UNGTeJ
A paged band records the events it SAW, never the range it asked for, so
`legs()` can only ever say "walked this far" and keeps re-asking the leg
below the floor. Against a relay whose corpus for one kind simply starts
later than the others' that leg is unclosable: it comes back empty every
cycle, an empty fetch earns no band, so the floor never moves. Measured on
a live mirror of five NIP-65 indexers, three were in that state — kind
10002 re-walked from the beginning of time to Feb 2023 forever, because
relay lists did not exist before then.
The missing fact is why a page ended. `fetchAllPages` treated all three
terminal signals as one bare `Unit`, so an empty page could not be told
apart from silence or a CLOSED. It now carries a PageEnd, and reports
`onDrained` only for the one ending that proves absence: an EOSE on a page
that returned nothing, with no filter capped by its `limit` and no `search`
filter in play (both stop the walk short of the corpus). An idle timeout is
silence, not an answer, and recording it would durably claim coverage the
relay never served.
A callback rather than a richer return type: ~25 call sites across quartz,
geode and downstream use the `Int`, and none should have to change to learn
a fact they do not want. It follows `onNewPage`'s shape.
`SyncCoverage.record` takes `drained` and marks the kinds that produced
evidence complete — which required completeness to move from Band onto
Span. It could not stay on the band: once kinds diverge, `legs()` hands
each group its own ask, so a walk that drained `kinds: [10002]` proves
nothing about kind 0, and a band-level flag set from that leg would claim
both. That is the same over-claim per-kind spans exist to prevent, one
level up. `Band.complete` stays as a DERIVED all-kinds-complete, so both
state files keep writing the flag a pre-per-kind reader expects, and read
it back as every span's default.
A kind the walk never saw at all still earns nothing: there is no interval
to anchor a claim to, and inventing one would be the over-claim again.
Tests: five in NostrClientFetchAllPagesDrainTest pinning EOSE-empty vs
silence vs CLOSED vs cannot-connect vs a fulfilled limit, and five in
SyncCoverageTest for per-kind completeness, widening, and the deeper-floor
escape hatch that a drain must not defeat.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016TNy5BsU9NErXYa3UNGTeJ
A soft ban leaves the community_root in the ex-member's hands, so they keep
deriving the channel's stream key. CORD-01 signs every wrap with that shared
key rather than with the author, so on the wire a Concord channel looks like a
single author publishing everything — and NIP-09/NIP-62 authorize on the outer
pubkey. Read naively that hands any ex-member a one-event wipe of the whole
community's history, and geode's own Nip09DeletionTest guarantee ("a kind-5
from pubkey X cannot delete pubkey Y's events") would be vacuous inside a plane.
It is refused, but only because of a rule written for something else:
Event.owner() gives a kind-1059 to its p-tag RECIPIENT rather than its signer,
and ConcordStreamEnvelope stamps a freshly random p-tag on every wrap. Each
wrap is therefore owned by a one-time key nobody holds, attacker included.
Neither half was written with this attack in mind and either one silently
re-opens it, so both are pinned: two tests fail if ownership ever moves back to
the signer, and a counterfactual (a wrap addressed to a real key IS deletable
by its holder) fails the moment that p-tag becomes anything a member holds.
Scope: this is our relay's rule, not the protocol's. A third-party relay that
authorizes deletion by matching pubkey still hands every ex-member a wipe
button, and a Refounding only protects the future.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DrJhpFhhLjuDJQNkGvYMGj
`export`/`restore` handed back `Map<String, Band>` where the string was
the INTERNAL key — `"<relay-url> <filter-json>"`. That is fine for a file
layer that writes the key back verbatim, and nothing else. A layer that
wants its own layout — one object per relay, or per filter, or nested by
both — had to split the key apart, and the separator was folklore it
could only learn by reading this class. Two of them now do.
So the key is a pair, with the joined form kept here as `encode`/`decode`
for a file that does want one key per line. geode keeps its format
byte-for-byte and stops pattern-matching on somebody else's string.
It is also faster on the path that matters. `key()` built a new string
per lookup, so a `legs()` over a fan-out COPIED the filter's json — tens
of thousands of characters for an author-scoped filter — once per relay
per cycle, then hashed all of it, since a freshly built string carries no
cached hash. The pair hashes two halves it already holds: the url, and
the fingerprint instance the cache above it already returns.
No behaviour change: the same pairs key the same bands, a file written
before this reads back through `decode`, and the format on disk is
untouched.
Follow-up to the linux-arm64 CI leg (feat/release-linux-arm64). Extends the
release matrix to Windows in three places, all on free public-repo hosted
GitHub runners:
* build-desktop: adds windows-11-arm (arm64) alongside the existing
windows-latest (x64). jpackage/jlink on Windows arm64 produce arm64 MSIs
natively; the same packageReleaseMsi + createReleaseDistributable task
list is used unchanged and the portable-archive step already parameterises
on ${{ matrix.arch }}.
* build-cli: adds windows-latest (x64) and windows-11-arm (arm64) legs
running :cli:amyImage. Windows has no jpackageDeb/Rpm and MSI-for-CLI is
deferred (portable zip is the documented Windows install path); the
headless-lib assertion runs unchanged under git-bash. amyImage now emits
both a POSIX `bin/amy` shell launcher AND a Windows `bin/amy.bat`
launcher into the flat image so the tree layout is uniform regardless of
build host. The .bat pins UTF-8 (chcp 65001) for sun.jnu.encoding, same
reason the installDist .bat was already patched.
* build-geode: adds windows-latest + windows-11-arm legs running
:geode:geodeImage. The existing --port smoke test is generalised to pick
bin/geode.bat on Windows; NIP-11 fetch via curl works unchanged under
git-bash on GH windows runners. Same dual-launcher pattern as amy.
scripts/asset-name.sh: collect_cli_assets and collect_geode_assets now
package the flat image as .zip on Windows (7z when available, falling
back to `zip`, then a portable python3 zipfile.ZipFile invocation). Every
other OS continues to use tar.gz. Adds the expected Windows examples to
the header block.
BUILDING.md: mentions the windows-11-arm runner and updates the asset
count in the Release runbook. No asset-naming contract changes — the
existing amethyst-desktop-<v>-windows-<arch>.<ext>, amy-<v>-windows-<arch>.zip,
and geode-<v>-windows-<arch>.zip shapes were already in scope, they just
weren't produced by any CI leg before.
Local validation on macOS arm64 (build host: JDK 21, gradle 9.5.0):
./gradlew :cli:amyImage -> bin/amy + bin/amy.bat both present
./gradlew :geode:geodeImage -> bin/geode + bin/geode.bat both present
./bin/amy --help -> parses (unix launcher unbroken)
./bin/geode --port 17447 -> NIP-11 served, "supported_nips" present
collect_cli_assets windows arm64 ... -> valid .zip with bin/amy.bat
collect_geode_assets windows x64 ... -> valid .zip with bin/geode.bat
actionlint .github/workflows/create-release.yml -> no new findings
Cross-compile is impossible for jlink/jpackage, so end-to-end
Windows-runtime validation still happens on GH CI on the first PR
build; nothing in this change can be verified any harder locally.
Deep-audit pass over the branch. Nothing here changes what a band claims;
these are the defects that pass tests and bite later.
- record() read the clock twice PER KIND. A 40-kind map took 80 readings,
and worse, a span's floor and ceiling were judged against two different
instants — so a span could be accepted at one end and rejected at the
other on a clock tick. One read, one instant, for the whole call. The
aggregate path had the same double read and now shares it.
- legs() handed the SAME MutableList instance to every Filter in a group,
publishing its accumulator through a public return value. Filters are
treated as immutable everywhere else; this keeps that true by
construction rather than by nobody having tried yet.
- The state file's round trip was asserted only for the fields, never for
the behaviour. Three tests now pin it: per-kind spans survive a restart
AND still narrow per kind afterwards; the ALL_KINDS sentinel survives
its negative key through toString/toInt; and a pre-split file (min/max,
no spans) loads as the claim it always was. Plus the rollback contract
— `min`/`max` must remain the OUTER edges, since a binary from before
per-kind spans reads those and would otherwise skip ground it has not
covered.
Checked and found sound, recorded so the next reader need not re-derive
it: ConcurrentMap.snapshot() copies, so export() cannot be mutated under
a writer; Band is immutable (widen() copies its map), so a shared Band
across threads is safe; merge() keeps old.fullAt, preserving the
re-walk clock across widening; and coveringWindow does NOT regress —
a paged band gave >1 leg before this change too, and a reconciled band
still collapses to one leg and narrows the shared snapshot.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A consumer that cannot suspend has to block, and blocking here deadlocks
the whole client.
Measured on a mirror built against this library, twice, ~13 minutes after
each start: all 64 shared coroutine workers parked in `runBlocking` beneath
`trySendBlocking`, called from the websocket message callback. The consumer
draining that channel needed threads from the same pool to reach its store,
so it could never make room, so the producers never woke. Every stream, the
health reporter, all of it stopped, at 2% CPU with a healthy, idle backend.
A full queue was the symptom; producers eating the threads the drain needed
was the cause.
The coroutine context was already there — BasicOkHttpWebSocket has always
processed messages inside `scope.launch { for (message in incomingMessages) }`
— so the only thing forcing a blocking hand-off was that the hops in between
were declared non-suspend. Now they are not:
WebSocketListener.onMessage
RelayConnectionListener.onIncomingMessage
PoolRequests/PoolCounts/PoolEventOutbox.onIncomingMessage
SubscriptionListener.onEvent
fetchAllPages / negentropy accessories' onEvent parameter
A consumer that fills its buffer now suspends and releases its thread rather
than holding it, which is the same reasoning BasicOkHttpWebSocket already
documents for keeping its own channel UNLIMITED so a slow consumer cannot
block OkHttp reader threads. This extends it one layer down.
BLE is the one transport whose callback genuinely cannot suspend — the
platform hands notifications to a plain callback — so BleNostrClient gets
the same treatment the websocket transport already had: an UNLIMITED
hand-off channel so the BLE stack is never blocked, drained by ONE coroutine
so message order survives the boundary.
Tests that drove these entry points directly now do so from `runTest`, or
from `runBlocking` where the call sits inside a raw thread or Runnable that
models a platform callback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A band held ONE created_at interval per (relay, filter). For a filter
naming several kinds that is a claim no walk can support: ask for
`kinds: [0, 30382]`, find profiles going back years and score cards only
from last month, and the band records 2020..now for the pair. The next
run then skips that whole interior for BOTH — so score cards written
inside it are never asked for again, and nothing anywhere says so. A
long-lived kind vouched for a short-lived one.
Band.spans is now per kind. Each carries only the evidence actually
collected for it, so the profile kind keeps its wide interval and the
score kind keeps its narrow one, and legs() re-opens the interior for
the second while still skipping it for the first.
Three things keep the cost of that where it was:
- legs() REGROUPS kinds by the windows they want. Identical coverage —
the common case, and the only case until they diverge — collapses back
into one ask, so a filter that produced two legs still produces two
rather than two per kind. Only a kind whose evidence genuinely differs
earns its own.
- A finished reconcile needs no per-kind evidence and is given none:
negentropy compares the filter's whole id set in one pass, so it
covers every kind in the filter or none. Only the PAGED path changed.
- Filters naming no kinds keep a single span under ALL_KINDS, which is
the same claim as before, correctly scoped to the case where it is the
only claim available.
record() takes observedByKind, and SyncCoverage.observe() accumulates it
as events arrive — replacing the pair of hand-rolled vars each caller
kept, and moving the per-event isPlausible guard in with it. A paged
walk over a MULTI-kind filter that supplies none earns no band at all,
loudly, once: attributing one interval to every kind is exactly the
over-claim this removes, and a band that over-claims skips events
silently, which is worse than re-reading them. Single-kind filters are
untouched — there the aggregate always was the per-kind answer.
The state file gains a per-kind `spans` object and keeps `min`/`max` as
the outer edges, so a rollback to a binary from before this reads the
file and behaves as it always did. A file written BEFORE this loads its
one interval under ALL_KINDS — the old, wider claim, kept rather than
discarded because discarding it would re-download every upstream's
corpus once on upgrade. The first per-kind walk replaces it.
All 26 existing SyncCoverage tests pass unchanged, which is the evidence
that single-kind behaviour did not move. The five new ones were checked
against the pre-fix rule reinstated in place: the two behavioural ones
fail there and pass here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
quartz:
- SQLiteEventStore: classify per-row savepoint errors — policy refusals
(blocked:/constraint/not allowed) stay Rejected, everything else is now
Failed, so disk-full no longer masquerades as 2M duplicate rejections
- IEventStore.batchInsert default: rethrow CancellationException and map
unknown throws to Failed (re-offering a duplicate is idempotent;
dropping a good event on a transient store error is not)
- IngestQueue: rethrow CancellationException instead of stamping a
cancelled batch Failed and continuing
- HostStrikes: make the eviction verdict exactly-once under concurrency
(deadHosts.add is the atomic gate) and re-check produced before
publishing
- SyncCoverage: bound the identity fingerprint cache (a caller minting
fresh Filter instances per cycle could grow it forever); legs() gains a
floor parameter so a complete band re-opens its older span when the
caller's window deepens; coveringWindow no longer treats a fully
covered relay as needing the whole filter
- PagingWindowProgress: accept single-second windows (a band's re-read
edge leg is exactly that shape)
geode:
- MirrorWorker: cap reconciledThrough at the leg's own ceiling — the
older leg of a resumed catch-up no longer stamps the band complete
through 'now' before the newer leg has run (silent event loss for up
to fullResyncSeconds if that leg failed)
- MirrorWorker: run negentropy and the paged fallback by hand instead of
negentropySyncOrFetch: drops the O(delivered-ids) dedup set from the
mirror path, and a fallback resets the observed span so a band never
claims interior ranges only a half-finished reconcile scattered over
- MirrorWorker: clamp a paged band's ceiling to the snapshot instant so
one future-dated event cannot suppress the next boot's newer leg
- MirrorWorker.close(): join the workers (bounded) so the final coverage
flush carries the last records
- Main: gate the coverage file on the store actually being persistent —
database.file with in_memory=true (the default) persisted bands over a
volatile store, and the next boot skipped the backfill over an empty
database; honor --db overrides
- SyncCoverageFile: request ATOMIC_MOVE explicitly; fix the restore/dirty
comment
- Import summary now prints the failed count; document
mirror_sync_state_file in config.example.toml
Renames from review: SyncBands -> SyncCoverage ("sync" reads negentropy-ish
in quartz, and coverage is the role — the bands are the records), and
PagingProgress moves into relay.client.paging as PagingWindowProgress,
beside RelayLoadingCursors and RelayPagingProgress, with its docs swept from
"walk" dialect to quartz's pagination vocabulary and cross-references
delineating the three: cursors are in-memory positions for demand-driven UI
paging, the window progress is fraction/ETA for a bulk pagination over a
known window, coverage is persistent intervals that license skipping work.
geode adopts both halves of the new contract. MirrorWorker counts
InsertOutcome.Failed in its own `failed` counter instead of folding it into
`rejected`, and the down catch-up gains resume memory: SyncCoverageFile
persists SyncCoverage next to the event database (admin state-file
convention, temp-file + atomic move, daemon flush), and runCatchUpDown asks
only for the legs outside the covered band. Bands are keyed on the stable
scoped filter — never the boot window, whose since/until change every start
— and clamped to the window, which only slides forward, so an old band can
never license skipping a range an earlier boot could not ask about. A clean
reconcile records completeness through its snapshot instant; a paged
fallback earns only the span it saw. For an upstream without NIP-77 this
turns the every-boot full re-download of the backfill window into a
resumed walk.
Off unless wired: MirrorWorker's coverage parameter defaults to null and
in-memory stores keep no state file, so existing tests and setups are
unchanged. Full :quartz:jvmTest and :geode:test pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4Pi9YYMhdzTFxRiV2jF9R
Bisecting existed to reconstruct per-event attribution after a store threw a
batch-wide exception. Better to never lose the attribution: batchInsert's
contract now requires per-row isolation, with a third outcome telling whose
fault a miss was. Rejected is the EVENT's fault (duplicate, expired, invalid,
blocked) and is final; Failed is the STORE's fault (schema drift, a failed
feed, a resource error) — the event was good, it is lost unless re-offered,
and a rising Failed count means the store is broken rather than that
upstreams send junk. Throwing is reserved for failures with no per-event
answer (engine unreachable, transaction never started), readable as "nothing
in this batch was written".
Consumers updated: RelaySession maps Failed to OK false with NIP-01's
"error:" prefix; IngestQueue converts a thrown batch and a missing outcome to
Failed instead of Rejected; NdjsonImportExport counts failed apart from
rejected; geode's MirrorWorker logs store failures at warn instead of
folding them into debug-level rejections. BisectingInsert and its test are
removed — with attribution guaranteed by the contract, retry-by-splitting
has nothing left to do.
Full :quartz:jvmTest passes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4Pi9YYMhdzTFxRiV2jF9R
multiRelayPoolReturnsContentFromEachRelay flaked with
"expected:<from-b> but was:<null>": the SubscriptionListener wrote the
per-relay results into a plain HashMap/HashSet, but each relay delivers
its EVENT/EOSE on its own InProcessWebSocket scope (Dispatchers.Default)
and PoolRequests dispatches the listener callbacks outside any lock. Two
relays therefore call `received[relay] = ...` concurrently, and a
HashMap.put racing a rehash can drop an entry, leaving a relay's value
null and failing the assertion.
Use ConcurrentHashMap and ConcurrentHashMap.newKeySet() for the shared
collections. Reproduced within 7 runs before the fix; 80 stress runs
clean after.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeAjLDNBvGPjb5bfViU3ad
The first cut appended :geode:test to the build-desktop matrix command.
That was wrong twice over: geode is JVM-only, so it ran 3× across the
ubuntu/macos/windows matrix, and — because org.gradle.parallel=true —
its default suite's CPU-heavy throughput benchmarks (a 1M-event mirror
sync, WireReqFloor, NegentropyServerReconcile) ran concurrently with the
timing-sensitive quartz relay-client tests, flaking
NostrClientReqBypassingRelayLimitsTest.denseSecondBeyondCapIsSteppedPastWithoutStalling.
Move :geode:test into its own test-geode job (needs: lint, ubuntu, JVM
21) so it runs once and its benchmark load can't starve another module's
timing assertions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCdJwdhGtmLZ12ViS56S3k
geode was runnable only via ./gradlew :geode:run and was absent from CI.
Give it the same release process as the amy CLI (it's the same kind of
application-plugin JVM module), plus the pieces a long-running server
daemon needs that a one-shot CLI does not.
- Main.kt: add terminal --version/-V and --help/-h flags so a packaged
binary has a fast, exit-0 command (Homebrew test block, package smoke
checks, Docker healthcheck).
- build.gradle.kts: jlinkRuntime + geodeImage (portable flat app-image
with a bundled JRE, plus config.example.toml + geode.service under
share/) + jpackageDeb/jpackageRpm, mirroring cli/. No Compose to
exclude — geode depends only on :quartz.
- Dockerfile + .dockerignore: multi-stage image (gradle installDist ->
temurin JRE), the primary channel for relay operators.
- packaging/: systemd unit, macOS hardened-runtime entitlements, and a
reference Homebrew formula.
- scripts/asset-name.sh: geode_asset_name/collect_geode_assets under the
canonical geode-<version>-<family>-<arch>.<ext> scheme.
- create-release.yml: build-geode matrix (tarball + deb/rpm + no-JRE jvm
bundle, with a serve+NIP-11 smoke test of the jlink image) and a
docker-geode job pushing ghcr.io/<owner>/geode:<version> (+ :latest).
- bump-homebrew-geode-formula.yml: auto-sync the reference formula on
stable releases.
- build.yml: run :geode:test in CI (it ran in no workflow before).
- README.md + plans/2026-07-24-geode-release.md: operator docs + design.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCdJwdhGtmLZ12ViS56S3k
Geode hard-wired the SQLite EventStore. Add a `[database].backend`
selector (and `--store` CLI flag) so an operator can choose the store
implementation:
- "sqlite" (default): the SQLite EventStore, unchanged.
- "fs": quartz's filesystem FsEventStore, rooted at [database].file.
- any other value: a fully-qualified class name of a custom
IEventStore on the classpath, instantiated reflectively via one of
`(NormalizedRelayUrl?, IndexingStrategy)`, `(NormalizedRelayUrl?)`,
or `()` — the "plug in anything" escape hatch.
Store construction moves into a new StoreFactory (mirrors cli's
StoreFactory) shared by the serve path and the import/export verbs, so
both open the same store from the same config. The SQLite-only
`PRAGMA optimize` maintenance loop now runs only when the resolved
store is the SQLite one.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GG3TBvLUv5uB5js1naG5sc
TagAuthorIndexBenchmark at 1M events settles the flag: the DM-room
shape (kinds + authors + #p, 65 client assembler call sites) drops
14.2 ms -> 0.66 ms (~21x, growing with corpus size) while batch-insert
cost stays inside run noise (49.0 vs 47.4 us/event). Existing relay
DBs build the index on next open via ensureOptionalIndexes.
Also refreshes the docs the numbers made stale: IndexingStrategy KDoc
now records the 200k and 1M measurements instead of a TODO,
MergeQueryExecutor's tag-merge note points at the new relayBench
reactions-watch scenario, FsQueryPlanner/FsDriverSelectionBenchmark
reflect the landed cost-based pick (149 ms -> 4.0 ms at 30k events),
and RELAY.md documents that strategy flag flips materialize indexes on
the next open.
Verified: quartz jvmTest store suites, geode test (126), desktopApp
LocalRelayStore tests (5, incl. reopening a default-strategy DB with
the new pubkey-alone flag), relayBench compiles.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
RelayAuthenticator.signWithAllLoggedInUsers gained an `interactive`
Boolean parameter, but these two geode auth tests still passed a
two-arg lambda and no longer compiled. Accept and ignore the flag.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0129yvP2hmVeDFfuKKy94tqX
Drive a real NostrClient against an in-process geode relay running
FullAuthPolicy, publishing a NIP-17 gift wrap through the PoolEventOutbox
retry queue. The first EVENT races ahead of AUTH and is rejected
`auth-required`; a RelayAuthenticator answers the challenge and the
still-pending wrap is resent on the post-AUTH resync and stored. This is
the integration counterpart to PoolEventOutboxAuthTest and exercises the
"auth-required must not burn the retry budget" fix end-to-end. A control
test (no authenticator) proves the relay genuinely gates, so the delivery
assertion isn't vacuous.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EZjmYpgHP4pf79Sav5QT8a
Drop the flaky publish-into-large-relay warmup from DeletionSettleBenchmark
(it timed out the measured reconcile at N=100k — the container noise the
docstring already warns against) and remove the throwaway ScratchSettleTiming
investigation tool. Record in the docstring what the phase breakdown proved:
the settle's extra time over a bare reconcile is O(K) relay-ingest of the K
residual deletions, dominated by one-time JVM/JIT warmup of the publish path
(consecutive K-note batches fell ~3100->570ms), not the deletion algorithm and
not O(N).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgL1WTV4Hkp2uuXcUHCHGt
relayBench measures relay-to-relay reconcile, not the amy/quartz client feature,
so the deletion-settle perf claim belongs in an in-process benchmark of
negentropySettleDeletions itself.
Models the post-content-settle state: a relay with N notes, a local store with
the same N except K it deleted (keeping the K kind-5s). The reconcile residual is
exactly those K, so a sendUp settle fetches K — not N. Asserts residual==K,
sentUp==K, and relay convergence (correctness guard at the small default N),
and prints one-reconcile vs full-settle so the deletion overhead reads as
"a few reconciles + K", never "+ a content re-download". Measured:
N=2000 K=20: settle ~2x one reconcile, fetched K=20 not N
N=100000 K=20: settle ~5x one reconcile, fetched K=20 not N=100000
The growth is the relay rebuilding its negentropy index after the deletions
(O(N) once) — inherent to applying deletions, and still far cheaper than
re-fetching the need set, which the old per-need-fetch approach did.
Scale with -DdelBenchN / -DdelBenchK (forwarded by the geode test task).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgL1WTV4Hkp2uuXcUHCHGt
The two-pass deletion convergence is protocol logic, not CLI assembly, and the
geode mirror is a near-term second consumer — so move it out of SyncCommand into
a reusable accessory alongside the rest of the negentropy family.
quartz: negentropySettleDeletions(relay, filter, store, sendUp, applyDown, …) —
re-reconciles after a content settle and resolves only the residual: publishes
our covering deletions up (sendUp) and/or ingests the relay's kind-5 down
(applyDown, vanish never auto-applied), looping until a round resolves nothing.
Returns DeletionSettleResult(sentUp, appliedDown, rounds). Everything it needs is
already quartz (negentropyReconcileIds, fetchAll, deletionsCovering,
publishAndConfirm, Event.verify, IEventStore), so it carries no CLI dependency.
SyncCommand's pass 2 collapses to a single call; pass 1 (content) is unchanged.
Catalogued in the accessories README.
Tests: DeletionSyncTest drives the accessory end-to-end both ways (sendUp → relay
converges to gone; applyDown → local converges to gone), on top of the existing
deletionsCovering unit + manual-wiring cases.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgL1WTV4Hkp2uuXcUHCHGt
Replace the per-need-event fetch (which pulled the whole need set just to read
metadata — an O(db) regression on large syncs) with a second reconcile pass over
the residual, per the "settle, then diff, then explain what didn't converge" idea.
Pass 1 is the plain content sync again (drain needs, publish haves) — zero
deletion overhead. Pass 2+ re-reconciles; the leftover diff is exactly the
deletion mismatches, and only that (tiny) set is fetched:
- residual need (relay has it, we still lack it after --down) = we deleted it →
publish our covering deletion up so the relay drops it;
- residual have (we have it, relay still lacks it after --up) = the relay deleted
it → pull the relay's covering kind-5 down and apply locally (vanish is NOT
auto-applied on pull — account-wide blast radius).
Loops until a round resolves nothing (converges + self-verifies).
So `amy sync` makes the relay honor our deletions; `--up` makes us honor the
relay's; `--up --down` converges both ways. Cost is one cheap reconcile + the
residual regardless of database size — the large-DB bottleneck is gone by
construction, not by heuristics.
quartz: deletionsCovering is now source-agnostic (takes a query lambda) so the
same coverage rule runs against the local store (up) or the relay (down); the
IEventStore overload is the local convenience.
Tests: DeletionSyncTest gains the down-direction end-to-end (relay deleted →
local removes) alongside the up-direction and the per-form unit cases.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgL1WTV4Hkp2uuXcUHCHGt
Refine the sync deletion rule to what was asked: for the events the relay HAS
that we LACK (the reconcile need set), publish only the local deletions that
would actually make the relay remove them — and nothing else, not other
deletions by the same author.
Determining coverage needs the need event's author/address/created_at, which we
don't have for an id we lack, so we fetch the need events (raw — no verify, no
store) purely for metadata. quartz gains IEventStore.deletionsCovering(events,
relay), which maps server-held events to the covering local deletions across all
three forms:
- NIP-09 id-based: a kind-5 with an `e` tag naming the event id;
- NIP-09 address-based: a kind-5 with an `a` tag naming the event's
addressable/replaceable coordinate, at/after it (created_at <= deletion);
- NIP-62 vanish: a kind-62 by the event's author, targeting this relay, issued
after it (created_at < vanish).
SyncCommand's need workers now fetch each need batch once (Context.fetchRaw),
publish its covering deletions (deduped across workers), and — when --down —
store the rest; anything we deleted is rejected by the store's own tombstone.
Nothing is pulled down or applied locally, so it cannot over-delete the store.
DeletionSyncTest covers each form (with cutoff and wrong-relay negatives) plus an
end-to-end reconcile → cover → publish that removes the note on the relay.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgL1WTV4Hkp2uuXcUHCHGt
Per the actual requirement, deletion propagation is exactly: for the ids the
relay HAS that we LACK (the negentropy need set), if we hold a kind-5 deletion
targeting one of them, publish that deletion up — so a note we deleted is
deleted on the relay too instead of being re-downloaded. Only the need ids,
only kind-5, up only.
This removes all the machinery the earlier approach accreted and that the audit
flagged as over-broad / data-loss-prone:
- deleted NostrClientDeletionSyncExt (the bidirectional side-channel, author
scoping, vanish gating, kind selection);
- reverted geode MirrorWorker to base (no deletion side-channel, live-sub
changes, catch-up ordering, or convergence changes);
- dropped the 3-phase SyncCommand flow (deletions-first pull, author-scope
derivation, reject-reaction backstop, --sync-vanish, deletions_* output).
The new path pulls nothing down and applies nothing locally, so it cannot
over-delete the store, and it needs no author scoping — the need set already
bounds it. Kind-62 is intentionally excluded: a vanish is not "of an id".
Emits deletions_sent. DeletionSyncTest now exercises the exact wiring
(reconcile → look up local kind-5 by its e tag for the need ids → publish),
including the negative case (a need id we never had sends nothing).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgL1WTV4Hkp2uuXcUHCHGt
An audit (adversarial-verified) found the deletion side-channel over-deletes
and over-propagates. Root cause: deletionSideChannelFilter fell open to
authors=null for any non-author-scoped content sync, so `amy sync --kind 1`
reconciled the RELAY'S ENTIRE kind-5/62 history and applied it to the personal
FsEventStore — every kind-5 deleting its targets + installing an id-tombstone
for every target id, every ALL_RELAYS kind-62 wiping all of a pubkey's events
(all kinds), and pushing our whole local deletion history up. Data loss plus a
full-history reconcile on every scoped sync.
Fixes:
- Bound the side-channel to the authors we actually hold content for (filter
authors ∪ local matched-set authors), never the relay's population. Skip when
that scope is empty; Phase 3's reject-reaction covers the author-less case.
- Kind-5 (precise, owner-scoped) propagates by default; kind-62 vanish is opt-in
via --sync-vanish (its blast radius always exceeds a content sync's scope).
- excludesDeletionKinds() now checks each deletion kind independently
(`--kind 1,5` no longer silently drops kind-62); the side-channel reconciles
only the missing kinds.
- amy Phase 1 is best-effort: a deletion-reconcile failure records deletions_error
and falls through to content, never aborting the primary sync (matches geode).
- Mirror up-catch-up converges on whether a PUBLISHABLE event was pushed, not raw
haveCount — a vanish targeting another relay no longer burns all 8 rounds every
startup. Mirror keeps its (correct) global scope for relay-to-relay replication.
Helper API: negentropyPropagateDeletions gains scopeAuthors + deletionKinds;
deletionSideChannelFilter takes authors + deletionKinds and returns only the
missing kinds. Tests updated for the new semantics.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgL1WTV4Hkp2uuXcUHCHGt
Reorder amy sync so both sides fully reflect each other, including deletions:
1. Deletion side-channel now runs FIRST, before content, in both directions.
The content snapshot is taken AFTER it, closing a resurrection bug: a
deletion pulled down mid-sync removes a local event, but the old top-of-run
snapshot still listed it and would re-offer it up — resurrecting it on a
relay that also lacked the deletion.
2. Content reconcile, over the post-deletion snapshot.
3. Reject-reaction backstop: when the relay blocks a content push (usually it
holds a deletion we lack), pull that author's kind-5/62 and ingest locally
so we stop re-offering the dead event. Verify-by-fetch — only a real
deletion the store accepts has any effect; fires only on an actual reject.
The up-push of deletions is already verified per-event: ctx.publish awaits the
relay's OK, and ingesting the kind-5 runs the delete synchronously, so OK=true
confirms the remote applied it.
Mirror catch-up gets the same deletions-first ordering (down and up), so a
deletion lands, or the reject-trigger is armed, before its target — no
add-then-delete churn.
Tests: MirrorDeletionSyncTest gains scopedUpMirrorPushesDeletion (authoritative
push — local holds the deletion, remote drops the note).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgL1WTV4Hkp2uuXcUHCHGt
NIP-77 reconciles by event id over the content filter, so a scoped sync
(`--kind 1`) never carries the kind-5/62 that deletes one of those notes:
the deletion stays stuck on whichever side issued it while the target
lives on forever on the other. Add a deletion side-channel that reconciles
kinds 5 & 62 on their own, independent of the content filter.
quartz: NostrClientDeletionSyncExt — DELETION_PROPAGATION_KINDS,
Filter.excludesDeletionKinds()/deletionSideChannelFilter() (kinds 5/62 scoped
to the same authors, no time window since a deletion's created_at is not its
target's), shouldPropagateDeletionUp() (kind-5 always; kind-62 only to a relay
it targets, honoring the vanish's declared relays), and
negentropyPropagateDeletions() — one bidirectional reconcile that streams
have→upload and need→download.
amy sync: run the side-channel bidirectionally regardless of --up/--down
whenever the filter excludes 5/62; emits deletions_{need,have,downloaded,
uploaded}; --no-sync-deletions opts out.
geode MirrorWorker: thread a per-upstream deletionScope through both catch-up
phases and both live subs (down + up), in the mirror's configured direction;
relax down containment to accept in-scope deletions, gate kind-62 pushes by
target relay, and carry the deletion filter on re-subscribe so a reconnect
never drops it.
Tests: DeletionSyncTest (up/down propagation + filter/vanish-gate units) and
MirrorDeletionSyncTest (a kind-scoped down mirror still removes the note).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgL1WTV4Hkp2uuXcUHCHGt
The `import`/`export` engine is pure protocol/store logic — it operates only on
the `IEventStore` interface and Quartz event types (Event, OptimizedJsonMapper,
verify, Filter), with zero geode dependency — so per the sharing philosophy
("quartz = Nostr business logic, protocol, data") it belongs in Quartz, not in
the geode app. Any Quartz consumer (a relay, the `amy` CLI, a desktop
backup/restore) can now reuse it.
- move `com.vitorpamplona.geode.ImportExport` →
`com.vitorpamplona.quartz.nip01Core.store.NdjsonImportExport` (commonMain,
next to IEventStore); rename for a clear library-level name.
- geode keeps only the CLI glue (verb dispatch, arg parsing, file/stdin/stdout,
the stderr summary) in Main.kt, delegating to the Quartz engine.
- move the test into quartz jvmTest, rebuilt on Quartz's own EventFactory +
NostrSignerSync (real Schnorr signing) instead of geode fixtures.
No behavior change — `geode import`/`export` work exactly as before (verified
end-to-end previously); this is purely where the code lives.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Bulk NDJSON import/export as first-class geode subcommands, mirroring
`strfry import` / `strfry export` (one JSON event per line — the interchange
format for seeding a relay, migrating between relays, or taking a backup):
geode import [--db …] [--no-verify] [FILE…] # files, or stdin when none
geode export [--db …] # NDJSON to stdout
Both stream — memory is bounded to one batch (import) / one event (export), so
a multi-million-event corpus round-trips in roughly constant memory. `import`
verifies signatures by default (same `Event.verify()` the relay's VerifyPolicy
uses), upholding the relay's verify-by-default stance rather than trusting the
file; `--no-verify` is the trusted-input escape hatch. Verb dispatch is
backward-compatible: a bare `geode --port …` (no verb) still serves.
This makes the benchmark-only `CorpusServerMain` redundant — a corpus source is
now just `geode import` into a DB, then a normal `geode` serve — so it's
deleted, removing benchmark-only code from the production geode artifact (the
question that started this). The 1M sync-throughput plan is updated to describe
sources via `geode import` + serve.
Also fixes a native-target CI break: MergeQueryCorrectnessTest used the
deprecated `String(CharArray)` (error-level on Kotlin/Native) — switched to
`CharArray.concatToString()`.
Verified end-to-end through the packaged `geode` binary: import (file + stdin,
--no-verify), export round-trip, and verify-on rejecting bad signatures.
ImportExportTest covers the counts, duplicate handling, malformed-line
skipping, and verify accepting a freshly-signed event while rejecting bad sigs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Follow-up to the audit fixes so nothing describes the pre-fix behavior:
- CorpusServerMain: drop the leftover "reuses an already loaded DB … skips
the reload" comment above `val dbFile` — the sentinel-gated reuse it
described is now spelled out in the block just below it.
- sync-throughput-1m plan: the up-catch-up now streams `negentropyReconcile`
(publishing each onHaveIds batch) instead of materializing the full diff
via negentropyReconcileIds; note the O(batch) memory win at 1M.
- follow-feed plan: the k-way merge dedups repeated authors/kinds, and its
id-ASC tie-break is byte-exact vs the single-SQL path only when the store
indexes id (useAndIndexIdOnOrderBy) — otherwise ties fall in rowid order.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Vitor PamplonaandGitHub
Claude
vitorpamplonaandgithub-actions[bot]
Vitor PamplonaandGitHub
mstrofnoneandVitor Pamplona
davotoula