Follow-up to the toolchain update, from an audit of that diff.
Build script:
- The NDK pin rejected machines that have the pinned revision installed.
An exported ANDROID_NDK_HOME or ANDROID_NDK_ROOT short-circuited the
search and then failed the revision check, and GitHub runners export both
at their own bundled NDK. Every candidate is now checked against its own
source.properties and a mismatch moves on, so the build fails only when
the pinned revision is genuinely absent, listing what it found instead.
- verify_jni_symbols printed missing exports and exited 0, so a library that
would throw UnsatisfiedLinkError on every call could ship. It now fails the
build, and checks only the ABIs this run built.
- Both post-build checks now use the pinned NDK's own llvm-readelf and
llvm-nm. The stamp check silently skipped on macOS, which has no readelf,
and Apple's nm cannot read ELF at all, so the symbol check would have
reported every symbol missing there.
- The stamp check read its note through `readelf | grep -q`, the same
SIGPIPE-plus-pipefail shape this branch removed from the symbol check.
- $HOME is expanded with a default, so `set -u` no longer aborts before the
"NDK not found" message in an environment without HOME.
verify-reproducible.sh hashed every .so under jniLibs, so --release, which
rebuilds arm64 only, hashed the untouched x86_64 library identically in both
runs and reported the whole tree reproducible and matching the commit. It now
hashes and diffs only the ABIs the run builds, and prints which those are.
lib.rs:
- initialize() signalled "already initialized" out of the JNI closure as an
empty string, re-tested after it. A destroy() landing in between would let
the empty string through as the data directory, which resolves to relative
state/ and cache/ paths against the process working directory. The check
now reads the whole Option outside the closure and no sentinel exists.
- Corrected the comments claiming the error policy keeps a panic from
crossing extern "C". It does not: the policy's panic arm runs through
catch_unwind, which catches nothing under this crate's panic = "abort"
profile. The Err arm, which is what the code relies on, is unaffected.
README: the troubleshooting section still told readers to install cargo-ndk
unpinned and to export ANDROID_NDK_HOME at an arbitrary revision, which was
the exact way to trip the old gate.
Verified: two clean builds byte-for-byte identical, both ABIs stamped r30,
JNI exports present, 16 KiB alignment kept. JVM tier-3 smoke test green, and
a scratch harness drove getVersion, setLogCallback, initialize, a second
initialize on a live client (the reuse path the sentinel used to carry) and
destroy over real JNI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cSuXeu4bUTNRAUCZJcLLW
Moves every pin in tools/arti-build forward and rebuilds all three shipped
libraries from them. Arti 2.5.0 carried two medium-severity fixes that a
client reaches in normal use, so the shipped 2.3.0 was the reason to do this
now: TROVE-2026-24, where a malicious directory mirror crashes the
tor-netdoc parser and eventually stops tor-dirmgr, and TROVE-2026-27, an
inefficient algorithm an attacker can drive into a CPU stall.
Pins:
- Arti 2.3.0 -> 2.6.0 (arti-client / tor-rtcompat 0.42 -> 0.46). New MSRV is
1.91, satisfied by the pinned toolchain.
- Android NDK 27.3.13750724 (r27d) -> 30.0.16248370 (r30), the current LTS.
clang and lld move 18.0.4 -> 21.0.0.
- rustc 1.94.1 -> 1.98.1.
- jni 0.21 -> 0.22.
- Cargo.lock regenerated, so every transitive dependency moves to its latest
semver-compatible release. cargo-ndk was already on the pinned 4.1.2.
Source changes the upgrades required:
- Arti 2.4.0 made every TorClient constructor return an Arc<TorClient> and
dropped Clone from TorClient, so the wrapper no longer wraps it itself.
- jni 0.22 splits the FFI environment pointer (EnvUnowned) from the API type
(Env), which is only borrowed inside a closure. Native methods now acquire
it via with_env and map failures through an ErrorPolicy instead of
unwinding out of extern "C", which aborts. initialize() reads everything
JNI-owned up front and resolves through Option<String>, because the policy
default for jint is 0, the value that API reports as success. GlobalRef
became Global<JObject>, thread attachment takes a closure (which also
scopes a local-reference frame per log line), and the method name and
signature are encoded at compile time via jni_str! / jni_sig!.
Verification:
- verify-reproducible.sh: two clean builds byte-for-byte identical, JNI
symbols exported, 16 KiB LOAD alignment kept, both ABIs stamped r30, same
libc/libm/libdl dependency set as before.
- JVM tier-3 smoke test passes against the rebuilt host shim, and a scratch
harness drove setLogCallback, getVersion, initialize and destroy through
real JNI: log lines arrive over the migrated callback and initialize
returns 0.
- cargo audit: rsa 0.9.10 (RUSTSEC-2023-0071) remains, with no fixed version
published upstream; it arrives via ssh-key-fork-arti and needs RSA private
key operations, which a client without hosted onion services never does.
The event-listener unsound and spin yanked warnings are gone.
Not verified here: the network-dependent integration tier and the on-device
instrumented test. This container blocks most outbound TCP (directory
authority port 9131 among them), so Tor circuits time out regardless of
which library is loaded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cSuXeu4bUTNRAUCZJcLLW
The committed libraries were built with NDK r25b (25.1.8937393) while the
build docs told everyone to install r27. Nothing pinned the NDK, so
build-arti.sh took the first directory matching ~/Android/Sdk/ndk/*/. The
NDK supplies the clang that compiles Arti's C dependencies (ring, zstd-sys,
libsqlite3-sys) and the lld that links the cdylib, so its revision is baked
into the output bytes exactly like rustc's is. The reproducible-build
promise therefore only held by accident of which NDK a verifier happened to
have installed.
- Pin the revision in ANDROID_NDK_VERSION (27.3.13750724, r27d) and resolve
it by name. A different revision now fails the build with the sdkmanager
line that fixes it, instead of silently producing unverifiable bytes.
- Record the verified cargo-ndk release in CARGO_NDK_VERSION. Warning only:
it wraps the NDK rather than generating code.
- Re-read .note.android.ident after each build, so the output has to carry
the pinned NDK's stamp to pass.
- Rebuild both ABIs on r27d (clang 18.0.4, lld 18.0.4, rustc 1.94.1).
verify-reproducible.sh: two clean builds byte-for-byte identical, all 8
JNI symbols exported, 16 KiB LOAD alignment kept, same libc/libm/libdl
dependency set as before.
- Fix verify_jni_symbols reporting every exported symbol as missing: piping
nm into `grep -q` per symbol lets grep exit first, nm dies of SIGPIPE, and
`set -o pipefail` fails the pipeline. Pre-existing, reproduces on the old
binary too.
- Docs: the 16 KiB page alignment comes from rustc's Android target spec,
not from "NDK 25+".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cSuXeu4bUTNRAUCZJcLLW
The adapters compared the socket OkHttp named in each callback against
the one they held, and took a monitor to make the compare-and-null
atomic and to close a window in connect() where a callback could arrive
before the field was assigned. Neither is needed: the relay client builds
a fresh adapter per dial and OkHttp binds exactly one socket to the
listener created in connect(), so anything that reaches that listener is
from this session by construction. The only question a callback has to
ask is whether the session already ended, which is one AtomicBoolean
claimed by whichever of onClosed, onFailure or disconnect() gets there
first. A compare-and-set keeps the "exactly one terminal report"
guarantee without a monitor, and there is no assignment window left to
guard.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014cq6vrfQkASwxgqXXY4py8
**Tor's control flow was being pinned alive for the process lifetime.** The
eviction wiring subscribed to `torManager.activePortOrNull` from a never-cancelled
coroutine on `applicationIOScope`, in both managers. That flow chains to
`TorManager.status`, whose upstream is `WhileSubscribed` and runs
`launch { service.start() }` when collected — so a permanent subscriber starts
Arti at process construction and never lets it unsubscribe on background.
AppModules documents this exact hazard for the battery ledger and deliberately
watches the raw `TorService.status` instead; I wired the poisoned well four lines
away from the warning.
Rewired from signals that are plain StateFlows and therefore free to observe:
`torPrefs.torType`, `torPrefs.externalSocksPort`, and `torService.status`'s socks
port. It moves to AppModules, which is where those live and where the precedent
is; commons had no business knowing Tor's subscription hazards anyway.
**`drop(1)` promised more than it delivered.** In the manager it skipped whatever
was present when the *coroutine started*, not when the call was made, so a route
change landing in that window was swallowed. In its new home the two coincide —
this runs during AppModules construction, before anything is pooled — so the
operator now means what the comment says.
**Animated media in a Crop cell flashed its raw URL.** The loading ladder keyed
its last branch on `ratio != null`, but `mediaSizingModifier` also bounds the
height for `ContentScale.Crop`. MyAsyncImage passes dimensions/blurhash/thumbhash
all null, so every gif in a card slot — DVM covers, long-form headers,
follow-set/calendar/music cards — hit the unbounded branch on first load and drew
URL text where it used to draw nothing. The predicate wanted "is the height
bounded", so it now says so: `contentScale == Crop || ratio != null`.
Drops ProxyRouteChange.kt and its tests with the rewire. The trigger is now three
stdlib flow operators; what needed judgement was which signals are safe to watch,
and that is recorded in the comment rather than in a test of combine().
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TdidqyhRni5L3ft5h8qti5
The first commit on this branch added INostrClient.connectedRelays(), a
snapshot read from the pool members, because connectedRelaysFlow could not
be trusted: a relay the pool had dropped could sit in it for minutes. That
has since been fixed at the source -- the pool clears the flow itself when
it lets a relay go, and every transport reports its session end exactly
once -- so the flow's value and the members' isConnected() move on the
same transitions, and re-reading the members on every emission was a
redundant pool walk. The notification takes its count from the emitted
set, the breakdown and the Active Subscriptions screen go back to the
flow, and the snapshot API is removed. The pool test keeps pinning what
the app actually reads: the flow drops a relay on removal and on
disconnect even when the socket layer never confirms the close.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014cq6vrfQkASwxgqXXY4py8
Nothing in the version catalog or any module references an
info.guardianproject artifact anymore (the old tor-android/jtorctl
dependencies were replaced by kmp-tor from Maven Central). Resolving
every module's dependency graph with and without the repository yields
an identical result, so the repository is dead weight that only adds
a network lookup to every miss.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014pmSZEn7Ub9URvxow4etZc
setExecutable's result was ignored (Sonar). The runner launches the wrapper
via sh -c <path>, so a missing exec bit only surfaced later as an opaque
permission-denied; check it at extraction time instead.
Clears Sonar 'Using HTTP protocol is insecure' hits. None were real requests:
six are @Preview dummy strings, four are picture-URL field hints that
suggested http:// to users. The Namecoin RPC onion placeholder stays
http:// on purpose (Tor encrypts; Namecoin Core RPC has no TLS).
`strict lookup keeps the strict contract for aliases outside the vault`
drove the miss through the macSecurityLookup seam, which getPrivateKeyOrThrow
only consults when isMacOs(). On Linux and Windows the strict path is
javakeyring, which cannot tell a miss from a denial and deliberately throws,
so the test's assertNull failed there — red on the Android job and the
Linux/Windows desktop builds of main.
Branch on the host like SecureKeyStorageOrThrowTest does: off macOS assert
the miss throws SecureStorageException with the PasswordAccessException in
its cause chain (coroutine stack-trace recovery may wrap it); on macOS keep
the null-on-miss / throw-on-ambiguous assertions.
Verified with os.name forced to Linux and natively on macOS (29/29
SecureKeyStorage* tests).
SecureKeyStorageVaultTest's "strict lookup keeps the strict contract for
aliases outside the vault" fails on Linux (red on main's own CI run for
38bbfa8c and reproduced in a clean worktree of origin/main): the test wires a
fake mac `security` probe through macSecurityLookup, but the strict lookup
gates that path on the real os.name, so on a Linux runner it takes the
javakeyring branch, where an unknown alias throws instead of answering null.
Make the OS check injectable like the other test hooks (keyringFactory,
macSecurityLookup) and have wireMacProbe force it on, so the suite exercises
the probe path it was written for on any host.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N56KzPSYiN5edMRamvKEgD
Removing the per-rebuild eviction left one sliver: when the user switches Tor ON,
the direct client's idle sockets to real hosts stayed pooled for the 5-minute
keepalive. Nothing could route a request through them — OkHttp keys the pool by
`Address`, which includes the proxy — but they are real connections to real hosts
outliving the moment the user asked for everything to go through Tor.
[evictOnProxyRouteChange] closes that by watching the one signal that means the
route actually moved: `torManager.activePortOrNull`. Tor coming up (null -> 9050),
going away (9050 -> null), or moving (9050 -> 9150) each evict exactly once.
`drop(1)` keeps subscribing from counting as a change.
Deliberately not driven by the two things that misled the old code:
- Client rebuilds. One factory mints both the proxied and the direct client and
they share a pool, so a per-rebuild check fired on every isMobileDataProvider
emission and every resubscribe — constantly, and never specifically on a Tor
toggle.
- The per-feature Tor switches (imagesViaTor, videosViaTor, …). Those change
which of the two existing clients a request picks, not the route either one
uses, so no pooled connection goes stale.
Wired in both managers' init, so the media and relay pools behave identically.
Six tests cover the trigger, including that re-emitting the same port never
evicts — the regression the old design had.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TdidqyhRni5L3ft5h8qti5
Merges nostr proposal 1c931194 (v4) into main:
- fix(commons): consolidate desktop keychain items into a single vault-v1 item
- fix(desktop): wire two-phase vault bootstrap into AccountManager
- fix(commons): make the keychain vault authoritative without blinding lookups
- fix(desktop): migrate the keychain vault before the first account-store read
- style(desktop): import CancellationException instead of inlining its name
Every nsec, bunker ephemeral and NWC URI now lives in one vault-v1 keychain
item behind a single ACL, so cold boot prompts once. One approval releases
every secret; this single-ACL model is an accepted maintainer decision.
v2-v4 fixed two account-store wipes (strict lookup ignoring the vault;
migration running after refreshAccountListOnStartup's first read), a
missing legacy fallback, and orphaned nsecs on logout. Verified with 17
mutation-checked tests and three consecutive launches against a real
macOS Keychain.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WHCtgMDNkAHvRDXnthuXQc
The vault bootstrap added two more inline
kotlin.coroutines.cancellation.CancellationException references next to
two existing ones; CLAUDE.md forbids fully-qualified names in function
bodies. One import, no behaviour change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WHCtgMDNkAHvRDXnthuXQc
Found by running the migration against a real macOS Keychain: the first
launch migrated and loaded fine, the second launch wiped the account store.
bootstrapConsolidatedVault() was called from loadSavedAccount(), and its
comment claimed it ran "before any other keychain read on the hot startup
path". It did not. Main.kt's startup DisposableEffect calls
refreshAccountListOnStartup() first, which reads accounts.json.enc and so
needs the metadata AES key.
On the first launch that read still found the legacy per-alias item and
cached the key, so the migration that followed looked harmless. On the
second launch the item was gone -- migrated into vault-v1 and deleted --
and the vault had not been activated yet, so the strict lookup answered
"definitively absent", getOrCreateKey minted a fresh AES key, wrote it back
as a legacy item, and the decrypt that followed failed with a GCM tag
mismatch. Observed exactly that: accounts.json.enc renamed to
.corrupt.<ts>, account-metadata-key resurrected as a per-alias item, and
the original key still sitting unused inside vault-v1.
Phase 1 is now a run-once, mutex-guarded ensureVaultMetadataKeyMigrated()
that every account-store entry point calls, including refreshAccountList().
Phase 2 still runs from loadSavedAccount() once the npub list is known.
Verified on macOS against the real Keychain and the real account store:
three consecutive launches across the migration boundary all load the
account, no corruption events, accounts.json.enc byte-identical throughout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
Three defects found reviewing the vault consolidation against current main.
1. The vault wiped every account on the first cold boot after upgrade.
getPrivateKeyOrThrow was the only accessor without a vaultActive branch
(savePrivateKey, getPrivateKey and deletePrivateKey all had one). The
migration deletes the legacy per-alias items after writing vault-v1, so
the strict path probed the OS for an item that no longer existed, read
macOS exit 44 / NotFound as "definitively absent", and let
DesktopAccountStorage.getOrCreateKey mint a fresh AES key over the one
that decrypts accounts.json.enc -- the exact silent wipe proposal
5d31b68e added that method to prevent.
It was also unrecoverable: phase 1 preserves the original key inside the
vault, but getOrCreateKey then persists the new key over the same alias,
destroying the only key that could decrypt the .corrupt backup. And
because phase 2 calls loadAccounts(), the wipe happened inside the
bootstrap itself, so no nsec was ever folded in.
The strict path now consults the vault first. A vault miss still falls
through to the strict per-alias probe, so uncovered aliases keep the
strict contract.
2. The documented legacy fallback did not exist. getPrivateKey was
`vaultActive -> vaultGet(npub)` with no fallback, so once the vault was
active any alias it did not cover read as absent. Phase 1 activates it
with only the metadata key, and a phase 2 that throws is swallowed, so
the real worst case was every nsec reading null rather than the
advertised "old two-prompt behaviour". A vault miss now falls back to
the legacy per-alias item.
3. deletePrivateKey left orphaned secrets. vaultDelete returned false for
an alias outside the vault and never touched the legacy item, so logging
out of an account whose nsec had not been folded in left the nsec in the
OS keychain indefinitely -- still readable via the fallback in 2. It now
unlinks the legacy item as well.
Also logs the swallowed vault-bootstrap failure; silently discarding it
made a half-migrated keychain impossible to diagnose from a user report.
All three are pinned by new tests in SecureKeyStorageVaultTest and
mutation-checked: reverting any one fix fails exactly its own test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
Runs SecureKeyStorage.enableConsolidatedVault at the top of
loadSavedAccount, before any other keychain read on the cold-boot
hot path, so migrated setups pay exactly one Keychain Access prompt
regardless of how many accounts, per-account bunker-ephemerals, and
NWC URIs the user has.
Phase 1 migrates only account-metadata-key (the AES key that
decrypts accounts.json.enc). Nothing else can be enumerated before
that file is readable, so this phase runs against a single-alias
candidate list. It is a no-op on fresh installs (no legacy item)
and on already-migrated setups (vault-v1 exists).
Phase 2 runs after accounts.json.enc has been decrypted and the
full npub list is known. For each npub we add the nsec alias
itself, the per-account bunker-ephemeral alias, and the NWC alias.
The legacy shared bunker-ephemeral alias is included for the
pre-per-account migration compatibility branch in
loadBunkerAccount. Phase 2 is idempotent so it is safe to run on
every startup, including when the vault already covers every alias.
DesktopAccountStorage.METADATA_KEY_ALIAS is promoted from private
to internal so AccountManager.bootstrapConsolidatedVault can name
it without duplicating the alias string. The alias itself, its
storage, and its read/write path all still live in
DesktopAccountStorage.
Bootstrap failures are swallowed: SecureKeyStorage falls back to
legacy per-alias reads for anything the vault does not cover, so
the worst case is the old two-prompt behaviour. Nothing on the
account-load path breaks.
No API change on AccountManager (bootstrapConsolidatedVault is
private) and no visible behaviour change on the fallback
(no-keyring) storage path, which already uses a single encrypted
file and does not have the per-item ACL problem the vault exists
to solve.
Follow-up to ae3218249a. Caching the Keyring handle collapsed
Keyring.create() calls but did not fix the double prompt on macOS,
because macOS Keychain gates access per item, not per session.
Amethyst's cold-boot path reads at minimum two distinct items,
account-metadata-key (the DesktopAccountStorage AES key) and the
active account's nsec, so the OS still surfaces two Keychain Access
prompts unless the user explicitly picked "Always Allow" on every
single item, which many users don't.
Fix: consolidate all Amethyst-managed keychain items into a single
vault-v1 item, so the OS sees exactly one item to gate.
SecureKeyStorage.enableConsolidatedVault(candidateAliases):
1. If vault-v1 already exists, load it and mark the vault active.
Legacy per-alias items in candidateAliases are still folded in
on this pass so a crash-during-migration leaves nothing stranded.
Aliases not in candidateAliases are ignored, so pre-existing
items from unrelated accounts do not get pulled in and re-prompt.
2. If vault-v1 is absent, batch-read each candidate alias in the
legacy per-item layout (paying the migration prompt once), pack
the recovered entries into vault-v1, and delete the originals.
The vault item is written first, legacy items are deleted only
after that write succeeds, so a crash mid-migration leaves the
legacy items in place and the next run retries cleanly.
3. Fresh installs write an empty vault so future savePrivateKey
calls stay inside it.
Migration is idempotent and cheap when the vault already exists
(one keychain read plus a JSON parse). Safe to call on every cold
boot, and safe to call twice per process for the two-phase
bootstrap pattern (see the AccountManager change in the next
commit).
Once the vault is active, savePrivateKey / getPrivateKey /
deletePrivateKey / hasPrivateKey read and write the in-memory
LinkedHashMap and persist the whole map back to the vault item on
mutation. Fresh SecureKeyStorage instances that have not opted into
the vault continue to use the legacy per-alias layout, so callers
that never call enableConsolidatedVault keep working unchanged.
The vault contents are stored as a JSON envelope of the form
{"schemaVersion":1,"entries":{alias: base64(secret), ...}}.
schemaVersion reserves room for future migrations. Values are
base64-encoded so alias / secret contents that contain quotes,
backslashes, control chars, or non-ASCII round-trip cleanly through
the hand-rolled JSON codec (kept hand-rolled so the keystorage
module does not need Jackson; Jackson lives in desktopApp and
quartz). Amethyst does not layer additional crypto over the OS
keyring for legacy per-item storage; the OS keychain is the trust
boundary. The vault follows the same policy.
Tests (SecureKeyStorageVaultTest, hermetic CountingKeyring
KeyringHandle fake, no real OS keychain):
- fresh install writes an empty vault item
- legacy items are migrated into one vault-v1 item and originals
deleted
- an existing vault is loaded without re-probing aliases it already
contains
- legacy leftovers from an interrupted earlier migration get
absorbed on the next boot when still named in the candidate list
- savePrivateKey after vault enabled persists to the vault
- getPrivateKey after vault enabled reads from in-memory contents
without further keychain traffic
- enableConsolidatedVault is idempotent across repeated calls
- two-phase migration folds in aliases discovered after phase 1
- partial legacy migration survives a relaunch (two SecureKeyStorage
instances against one backing store)
- delete removes from the vault and unlinks the whole item when the
last alias goes
- vault round-trips keys with quotes, newlines, backslashes, and
Unicode
All existing SecureKeyStorageKeyringCacheTest cases still pass;
the changes are additive and back-compatible when
enableConsolidatedVault is never called.
Both factories emptied the whole connection pool whenever the proxy differed
from the last one they were handed. It was defensive, and it was not free.
It is not needed. OkHttp keys the pool by `Address`, and `Address.equalsNonHost`
compares `proxy` — so a call is only ever given a connection opened through the
very same route. A connection left over from an old proxy is already unreachable
by anything using the new one; it just ages out of the pool on its own. There was
never a stale-route connection to protect against.
The cost was real, though. `evictAll()` empties the ENTIRE shared pool, and each
factory mints BOTH clients: `DualHttpClientManager` builds defaultHttpClient
(always SOCKS, since buildLocalSocksProxy falls back to 9050 rather than
returning null) and defaultHttpClientWithoutProxy (always null) from one
instance, and they share one `rootClient.connectionPool`. A single "last proxy"
field therefore alternated forever, and every rebuild read as a route change and
dropped every warm connection the other client was relying on. Both are
stateIn(WhileSubscribed(1000)) flows collected from a composable, so it fired on
each isMobileDataProvider change and each foreground round trip — and the next
image then paid a fresh DNS + TCP + TLS. Plausibly the "first image after a pause
takes forever" stall the pingInterval above it was added for.
DualHttpClientManagerForRelays has the identical shape, so the relay factory is
fixed the same way. Less damaging there — evictAll spares connections with active
calls, so live relay sockets survived — but it was still discarding idle pooled
connections on every network flap.
This replaces the ProxyRouteTracker approach from earlier on this branch, which
kept the eviction and merely made its bookkeeping correct. Deleting the mechanism
is the better answer, and it takes the class and its tests with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TdidqyhRni5L3ft5h8qti5
availableRelaysFlow was merged in when connectedRelaysFlow could not be
trusted to move on a removal: a relay the pool had dropped could sit in it
for minutes, so the one flow that did move on membership changes was used
as an extra wake-up and the count re-read from the pool members. The pool
now clears the connected flow itself whenever it lets a relay go, and every
transport reports its session end exactly once, so that flow emits on every
change the count can reflect and the extra trigger only added wake-ups on
membership churn. The count still comes from client.connectedRelays(),
the members' live socket state, which is what it is meant to show.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014cq6vrfQkASwxgqXXY4py8
The previous commit taught BasicRelayClient to remember which listener it
had wired and to ignore callbacks from any other. That put the knowledge
"this report is about a socket I already threw away" in the wrong layer:
the transport adapter is the one that owns the socket, and OkHttp names
the socket in every callback, so the adapter can tell for free.
BasicRelayClient goes back to exactly its previous code. The contract it
relies on is now written on the WebSocket interface and kept by every
transport: a session ends with exactly one terminal callback, and
disconnect() reports onClosed synchronously and forwards nothing from that
socket afterwards -- what InProcessWebSocket has always done. Both OkHttp
adapters (quartz's BasicOkHttpWebSocket and the Android app's
OkHttpWebSocket) now check ownership on every callback, claim the terminal
report under a lock so a session cannot be reported twice, and answer
disconnect() themselves instead of waiting for OkHttp, which raises nothing
for a cancel when no reader is left to fail and otherwise raises it later
on its own thread. The reconnect race this closes is the same one as
before: with disconnect()+connect() back to back, the old socket's late
failure used to land on the new connection and cancel it.
The client-level stale-socket test is replaced by adapter-level tests on
both modules: disconnect() reports once and synchronously, OkHttp's own
reaction to the cancel never surfaces, and a relay-initiated close followed
by disconnect() is still one report.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014cq6vrfQkASwxgqXXY4py8
One conflict, in BlossomReadAuthTokenProvider.signOnce(). Main already carries
the same fix via PR #4108 (davotoula), landed while this branch was open. The two
are functionally identical — same third cache look, in the same place, for the
same reason — so the conflict is resolved by taking main's wholesale and dropping
mine. The file is now byte-identical to main; my add7bfe3 contributes nothing
beyond it.
What this branch still adds over main:
- GifVideoView loading fallback (the invisible no-imeta note)
- ProxyRouteTracker + its tests (the shared connection pool being evicted
whenever the proxied and direct clients alternate)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TdidqyhRni5L3ft5h8qti5
I lowered these to 32/8 on reasoning that does not survive reading the source.
`maxRequests` is the thread ceiling — Dispatcher's executor really is
corePoolSize=0 / maxPoolSize=MAX_VALUE over a SynchronousQueue, and its own kdoc
notes a pool sized exactly to maxRequests is not even sufficient. That much was
right. The conclusions drawn from it were not:
- "128 concurrent TLS handshakes" is wrong. These hosts are HTTP/2, so concurrent
calls to one host multiplex over a single connection; handshake count is bounded
by distinct hosts and the pool, not by maxRequests.
- "A tighter total lets the visible images finish first" is backwards.
promoteAndExecute walks readyAsyncCalls as a strict FIFO with no priority, and
PrefetchFeedMedia enqueues notes before the user reaches them — so a lower cap
makes the on-screen image queue behind those prefetches rather than start
immediately. That is the very symptom under investigation.
- Halving maxRequestsPerHost also halves HTTP/2 stream concurrency against the
single Blossom host a feed pulls from, which is what these were tuned for.
What is left is thread memory, and blocked threads commit little. No measurement
justified the change, so the values go back as they were. The comment now carries
the analysis so the next reader does not re-derive the same wrong intuition.
The ProxyRouteTracker fix from the same commit is unaffected and stands.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TdidqyhRni5L3ft5h8qti5
test-quartz-ios failed on :commons:compileKotlinIosSimulatorArm64 with
"Unresolved reference 'okio'" in service/image/DeferredDeleteFileSystem.kt.
The file always used okio, but commons only ever got it transitively:
through Coil on every target, and through OkHttp on JVM. Moving Coil to
:commonsUI removed the only Apple-side provider, and the JVM builds kept
passing, so nothing caught it locally.
Declare okio (3.18.1, the version already resolved everywhere; Apache-2.0
per its POM) in commons commonMain. Reproduced and re-verified on Linux with
:commons:compileCommonMainKotlinMetadata, which resolves commonMain against
the shared-dependency set exactly like the iOS compile. That task, for all
three KMP library modules, is now part of the lint job so this class of gap
fails fast on Linux.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N56KzPSYiN5edMRamvKEgD
Follow-ups from an audit of the two previous commits on this branch.
The Android app does not use quartz's BasicOkHttpWebSocket: AppModules wires
its own OkHttpWebSocket, a near-twin that decides needsReconnect() from the
OkHttpClient in use. The onClosing answer therefore only reached desktop,
CLI and geode. Mirror it here, with the same loopback RFC 6455 test. The
unit-test android.util.Log stub gets a primitive-signature isLoggable (the
boxed one it had was a different method to the JVM, which is why OkHttp
could never be built in this module's tests) plus println, so the socket can
be exercised for real.
RelayPool now clears its connected flow itself on every path where it lets
a relay go -- removeRelay, removeAllRelays and disconnect -- instead of
waiting for a callback the socket layer may never deliver. The previous
commit added a members-read snapshot and migrated three callers; the other
consumers of connectedRelaysFlow (drawer status, connection-time accounting,
"wait until connected" loops) were still reading the stale set.
BasicRelayClient retires its listener before tearing a socket down and
ignores anything a retired socket reports afterwards. OkHttp delivers
onClosed from its writer thread and a cancelled socket's failure later
still; with the pool rebuilding sessions via disconnect()+connect(), a late
callback from the old socket could null the new one, orphaning a live
connection and dialing a third. disconnect() also reports onDisconnected
itself now, so a client-initiated teardown never depends on the socket
layer confirming it.
KDoc and comments that described the unanswered-close behaviour in the
present tense, or claimed a still-desired relay reads isConnected()=false
on a silent close, are corrected.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014cq6vrfQkASwxgqXXY4py8
The merge of origin/main resolved quartz/build.gradle.kts with the branch
side wholesale, which kept the shared purity-gate apply but dropped the
`by getting` -> `getByName(...)` cleanup from 6e3af61e. Re-applied on top.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N56KzPSYiN5edMRamvKEgD
Merges nostr proposal 5d31b68e into main:
- fix(desktop): stop silent account wipe on keychain errors and upgrade races
- fix(desktop): allow first-launch key bootstrap and stop caching unwritten state
Desktop lost every logged-in account whenever the OS keychain answered
ambiguously. getOrCreateKey() treated any failed lookup as "key absent" and
minted a fresh AES key, orphaning accounts.json.enc; the read path then
renamed the unreadable file to .corrupt.<ts>; and nothing serialised access,
so a Homebrew upgrade race could interleave two instances.
Now: getPrivateKeyOrThrow() distinguishes confirmed-absent from refused or
ambiguous (macOS via /usr/bin/security exit codes) and only the former
rotates; genuine corruption (AEAD tag, bad padding, malformed JSON) is
separated from transient failures, which preserve the file and surface as
StorageCorruption.TransientError; and a cross-process advisory lock plus
in-process mutexes guard the file.
Review follow-ups in the second commit: non-macOS keyring backends throw for
a genuinely absent credential, so a fresh Linux/Windows install could never
mint the key -- creation is now allowed when accounts.json.enc does not yet
exist, where there is no ciphertext to orphan. And the metadata cache is
populated only after the disk write succeeds, so a failed save no longer
leaves the session serving accounts that were never persisted.
Verified on macOS against the real Keychain and the real account store:
security lookup exit 0, accounts load and survive a restart, ciphertext
byte-identical, zero corruption events.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
signOnce() looked at the cache a second time to catch a leader that had already
finished, but it looked too early — before claiming the in-flight slot. The gap
between that look and putIfAbsent still spans signerProvider() and an allocation,
and a leader caches its token and retires its entry inside it. A straggler in
that gap therefore put into a map the leader had just emptied, won the slot, and
signed a duplicate.
Winning the slot is not proof that nobody signed; only a look from inside it is.
Once we hold the entry no one else can be leader, and our successful put observed
the map after that leader's removal, which its cache write is ordered before — so
a token visible at that point is the last word. Hand it over and stand down.
This is pre-existing, not fallout from the dispatcher change: the same test fails
on an unmodified origin/main worktree, and aFastSignerStillSharesOneSignature
already says the window is "microseconds wide, so one round hits it only now and
then" and runs 200 rounds to catch it. It cost a duplicate signature — with a
NIP-55 external signer that is a second IPC round trip, and potentially a second
prompt, for a burst of images from one gated host.
Verified with 8 consecutive runs of the suite (1600 signing rounds), all green,
against a baseline that failed inside the first run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TdidqyhRni5L3ft5h8qti5
OkHttp fires onClosed only once both peers have sent a CLOSE frame, and
sending ours is the application's job (WebSocketListener KDoc; RealWebSocket
emits onClosed solely from the writer once our Close is dequeued with the
peer's code already set; the bundled WebSocketEcho recipe answers onClosing
with close(1000, null)). BasicOkHttpWebSocket never implemented onClosing,
so a relay-initiated close left the socket half-closed: no onClosed, no
onFailure, send() still accepted and silently discarded, and a later
cancel() silent as well because no reader was left to fail. The relay
client kept believing it was connected, with its REQs live, until the 120s
ping path finally failed up to two intervals later.
Answer onClosing with close(1000, null). Verified against OkHttp 5.5.0:
onClosed then fires at once whether the relay still holds the TCP session
or has already dropped it, and the existing onClosed path in
BasicRelayClient marks the connection closed and lets the pool reconnect
under its normal backoff. Always 1000 rather than echoing the relay's code,
since close() validates the code it writes and relays may send reserved
ones.
The test drives the wrapper against a minimal RFC 6455 server on a loopback
ServerSocket (no new dependency): the relay sends CLOSE, the client must
answer with its own CLOSE frame and report onClosed with the relay's code.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014cq6vrfQkASwxgqXXY4py8
Second finding from the Marmot headless harness on geode. wn's `keys
publish` mints a KeyPackage (kind 30443, same `d` tag) in the same second
as the one its bootstrap already published; NIP-01's lowest-id-wins tie
keeps the stored one, and the insert of the loser trips the addressable
unique index. The store classified that as a rejection carrying SQLite's
text — "UNIQUE constraint failed: event_headers.kind, event_headers.pubkey,
event_headers.d_tag" — so the relay answered OK false with a reason no
client can classify, and MDK filed it as "publish acknowledgement
unknown" and retried forever.
nostr-rs-relay, which this harness was validated against, does not even
attempt the insert when a newer version exists and acknowledges the event
as `OK true "duplicate: ..."` (its Duplicate status maps to true). Match
that: the replaceable and addressable unique-index failures now classify
as RejectionReason.SUPERSEDED, a `duplicate:`-prefixed reason the session
already answers with OK true. The stored version is untouched, nothing is
fanned out, and the STORE-W01/W02 contract in the event-store-semantics
skill is updated to say so.
Tests: NostrServerTest covers an older kind-0 re-insert and the
same-second kind-30443 tie, asserting the OK true duplicate: reply and
that the winner remains the only stored version.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PguqnDbP2v11dtANs9xdxc
Running the Marmot headless harness against the embedded geode relay
failed 10 of 29 scenarios, every one on the same reply: the relay
answered a resent EVENT with
["OK", <id>, false, "Error code: 2067, message: UNIQUE constraint
failed: event_headers.id"]
NIP-01 says a relay that already holds the event answers
["OK", <id>, true, "duplicate: already have this event"], and every
client here depends on that: amethyst's outbox writes an event as soon
as the socket is ready and resends it when the connection finishes
syncing, so one of the two copies is always a duplicate; MDK's wn
counts a `duplicate:` prefix as idempotent success but files an
unclassified OK false as "publish acknowledgement unknown" and keeps
retrying. Both amy's group commits and wn's KeyPackage publish were
failing on it, while nostr-rs-relay had answered the resend correctly.
SQLiteEventStore now recognises the unique-index violation on
event_headers.id and reports RejectionReason.DUPLICATE, the constant
that already carried NIP-01's exact wording but was never produced;
RelaySession sends `OK true` for a `duplicate:` reason and keeps
`OK false` for every other rejection. The store outcome stays Rejected,
so a duplicate is still not fanned out to live subscriptions or counted
as a new write by the mirror worker and importer. The filesystem store
already treated a duplicate insert as a no-op.
Two tests pinned the old OK false behaviour (NostrServerTest,
KtorRelayTest) and now assert the NIP-01 reply.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PguqnDbP2v11dtANs9xdxc
Three read-only sweeps over commons (feeds, relayClient, model, viewmodels,
services) and commonsUI produced 44 candidates; each was re-read in the
source, 38 held up, 34 are fixed here, 4 are deferred with rationale in
commons/plans/2026-09-12-audit-findings.md.
Correctness highlights:
- SecureKeyStorage (jvm): AES-GCM was initialised with IvParameterSpec, which
every JDK rejects, so the no-keyring fallback never worked. GCMParameterSpec.
- OnionLocationInterceptor cached any Onion-Location header and re-pointed all
Tor traffic at it for 24h over plain http; only .onion targets now.
- BasicBundledInsert wedged forever after one exception (no finally).
- CachedRichTextParser keyed parses on a 32-bit hash without checking inputs.
- noProtocolUrlValidator backtracked exponentially per composer keystroke.
- Base83 indexed a 255-entry table with any char code from an imeta tag.
- MetadataRateLimiter never flushed a batch smaller than 20 pubkeys.
- FeedMetadataCoordinator mutated six HashSets from two threads and marked
pubkeys 101+ as requested without asking for them.
- Note: two discarded boolean/relay expressions, an NPE window in flow(),
removeReport leaving empty buckets, unlocked read-modify-write on
replies/boosts/edits/reactions/reports/labels.
- EventCollectionState restarted its flush timer on every insert.
- Chatroom prune outside the lock; top-zappers publish outside the mutex;
hashCode used as dedup/feed keys; OnlyLatestVersionSet.addAll always true.
- UI: ClickableTexts remembered a stale onClick, ZonedSwipeModifier a stale
openDrawer, two robohash light-theme predicates thrashed one cache, a chess
remember key summed two counters.
Performance: user-cache search off the Compose dispatcher, regexes hoisted
in SearchResultSorter, frame-rate animation reads moved out of composition
in Shimmer/LoadingAnimation/BunkerHeartbeat, GlowingCard allocations cached,
emoji inlineContent remembered, itemsIndexed in the search pickers.
Ten regression tests added (rate limiter flush/dedup/rate, GCM round trip,
regex timing, Base83 bounds, CosineCache key). Verified: commons and
commonsUI jvmTest, compiles of cli, desktopApp, nappletHost, amethyst.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N56KzPSYiN5edMRamvKEgD
Follow-ups from a review of the commons/commonsUI split:
- `verifyKmpPurity` was copy-pasted into quartz, commons and commonsUI and the
three copies had already drifted (checked dirs, hint text). It now lives
once in gradle/kmp-purity.gradle.kts and each module applies it; the
checked-dir list is the union, filtered by existence. Verified the shared
task still fails on a deliberate java.util.UUID reference in commons.
- amethyst/src/main/res/CLAUDE.md and the compose-expert catalog reference
still pointed at the pre-split composeResources / ui paths.
- Same-package imports left behind in the files moved to commons.feeds and
commons.model are removed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N56KzPSYiN5edMRamvKEgD
Two defects found reviewing the strict-keychain fix.
1. Fresh Linux/Windows installs could never persist an account.
getPrivateKeyOrThrow turns any PasswordAccessException into a
SecureStorageException on non-macOS backends, but every backend
java-keyring ships throws that exact exception for a *genuinely absent*
credential:
- WinCredentialStoreBackend: CredReadA false (ERROR_NOT_FOUND) -> throw
- FreedesktopKeyringBackend: empty object paths ->
throwNoExistingCredentialException
- KWalletBackend: hasEntry false -> "Password is not in wallet"
So the strict lookup structurally cannot report "definitively absent"
there, the create branch in getOrCreateKey was unreachable, and nothing
between it and AccountManager.addAccountToStorage catches the throw.
getOrCreateKey now bootstraps a fresh key when the strict lookup fails
*and* accounts.json.enc does not exist. With no ciphertext on disk there
is nothing a new key can orphan, so the invariant the strict contract
protects is untouched: once the file exists the exception propagates
exactly as before.
2. writeCachedMetadata updated the in-memory cache before the disk write,
so a failed write (keychain refusal, I/O error, disk full) left the
session serving accounts that were never persisted -- a save that
reported success and vanished on the next launch. Persist first, cache
second.
Both are pinned by new tests, and both were mutation-checked: reverting
either fix fails exactly one of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
Three independent bugs in DesktopAccountStorage / SecureKeyStorage could
turn one ambiguous macOS Keychain reply, one transient read error, or
one Homebrew upgrade race into permanent account-metadata loss on
~/.amethyst/accounts.json.enc.
1. Silent AES metadata-key rotation on ambiguous keychain miss.
getOrCreateKey() treated null from getPrivateKey("account-metadata-
key") as "no key exists" and generated a fresh AES key. On macOS
javakeyring collapses errSecItemNotFound (-25300), errSecAuthFailed
(-25293), errSecUserCanceled (-128), and errSecInteractionNotAllowed
(-25308) into the same PasswordAccessException; getFromKeyring turns
them all into null. A single Deny click on the OS Keychain dialog
silently rotated the AES key and destroyed the ability to decrypt
the existing accounts.json.enc.
Fix: add a new strict SecureKeyStorage.getPrivateKeyOrThrow(npub)
on the common expect. On JVM/macOS it wraps /usr/bin/security
find-generic-password whose exit codes (0 = found, 44 = not found,
others = ambiguous) are documented and unambiguous. On JVM
Windows/Linux it uses javakeyring but throws on any
PasswordAccessException from the strict path. On Android it uses
EncryptedSharedPreferences.contains(). On iOS it mirrors the
existing "pending (iOS Phase 4)" stub. getOrCreateKey now calls
getPrivateKeyOrThrow and propagates SecureStorageException without
ever rotating the key. The permissive getPrivateKey(npub) is
unchanged; its callers (per-account nsec, ephemeral bunker keys)
still tolerate null on any error.
2. Any read failure resets the file. readMetadataFromDisk() used to
rename to accounts.json.enc.corrupt.<ts> and return empty
AccountMetadata() on any exception, including transient IO and
the newly-throwing keychain path from bug 1.
Fix: distinguish exception types.
- AEADBadTagException / BadPaddingException: back up to
.corrupt.<ts>, reset, fire StorageCorruption.FileCorrupted
(unchanged).
- JacksonException: back up but to .jsonerror.<ts> so it is
distinguishable from ciphertext corruption; fire
StorageCorruption.JsonMalformed.
- Anything else (IO error, OOM, thrown keychain path): do NOT
rename; rethrow to caller and fire a new
StorageCorruption.TransientError(cause) subtype. The file stays
untouched. AccountManager.loadSavedAccount already wraps in
try/catch and turns the throw into Result.failure.
- Truncated file (size < GCM IV size): still backup + reset,
genuinely unusable.
3. No cross-process advisory lock. Homebrew replacing the .app while
the old process is mid-save, or an accidental double-launch of
Compose Desktop (no built-in single-instance guard), could produce
a truncated file that trips bug 2.
Fix: withAccountsFileLock helper (mirrors SecureKeyStorage.
withFileLock) wraps read + write in a
RandomAccessFile(lockFile, "rw").channel.lock() on
~/.amethyst/accounts.json.enc.lock (0600). Because FileChannel.lock
is per-JVM, an in-process Mutex is held before acquiring the
channel lock. A separate stateMutex guards the read-modify-write
cycle in saveAccount / deleteAccount / setCurrentAccount so two
concurrent writers cannot each read the same base metadata and
each rewrite it.
Backward compatibility: existing keychain items are read unchanged;
no schema migration for accounts.json.enc; the file lock adds a
.lock sidecar older builds ignore.
Tests: new SecureKeyStorageOrThrowTest (pure exit-code parser,
mac lookup Found/NotFound/Ambiguous, non-mac keyring hit and
throw-on-PasswordAccessException). DesktopAccountStorageTest gains
five cases: getOrCreateKey ambiguous-error preserves file and does
not rotate; getOrCreateKey definitive-not-found happy path;
readMetadataFromDisk transient-IO preserves file with no backup
sibling; GCM tag mismatch keeps .corrupt.<ts> backup; JSON malformed
uses new .jsonerror.<ts> suffix; eight concurrent saveAccount calls
serialize under the file lock with no lost updates. All existing
AccountManager* MockK setups extended to also stub
getPrivateKeyOrThrow.
Local verify: :desktopApp:test + :commons:jvmTest, 2490 tests, all
pass. Spotless clean.
Closes the last two items of documented debt from the commons/commonsUI split.
ParentNote (replyingDirectlyTo, isCommunityDefinition) and ReplyContext are
pure thread logic used by ViewModels, so they move from the misleading
`ui.note` package to `commons.model`, next to ThreadAssembler, together with
their tests and the StubCache fixture that shared the package. No `ui.*`
package is left in commons. The `ui.note` composables in commonsUI gain
explicit imports; consumer imports rewritten.
Whether commons still needs the Compose compiler plugin was an open question;
it is now measured. With compiler reports on the three GUI modules and full,
non-incremental recompiles in both configurations, removing the plugin flips
composable parameters typed with unannotated commons classes (TopFilter,
TorSettings, ProfileBroadcastStatus, ScheduledPost, EmojiPackState, ...) from
runtime-stable to unstable: 20→28 in commonsUI, 33→65 in desktopApp,
90→149 in amethyst. The plugin stays; the numbers are recorded in the build
file, ARCHITECTURE.md and the split plan so the question is not reopened.
Verified: JVM compiles for commons, commonsUI, cli, desktopApp; Android debug
compiles for nappletHost and amethyst; commons/commonsUI/cli/desktopApp JVM
test suites.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N56KzPSYiN5edMRamvKEgD
Audit of the sync path the new geode-backed EventSyncTest exercises found
two bugs in EventSync itself, plus review nits on the harness changes.
- runSync closed its client (`use {}`) the moment the last page arrived,
while `publish` is fire-and-forget through the client's outbox. Events
forwarded from the final page of the last relay were still waiting for
a socket or an OK when the outbox was destroyed, so the sync reported
Done and silently never delivered them. Wait, bounded by the existing
per-relay timeout, until no forwarded event has a relay left pending.
- The "events sent" counters incremented on every onSent, including the
failed write to a destination still connecting and the outbox's
at-least-once resend of an unacknowledged event after the connection
syncs. Every cold destination therefore reported at least one extra
event sent. Count only successful writes, once per (event, relay).
The test now asserts the sent total equals the routed total.
- Harness: the 127.0.0.2 rationale claimed it survives Quartz's
isLocalHost() strip; that filter now covers all of 127.0.0.0/8, so
say so and note what it means for the strict-inbox DM cases. The
interactive Marmot harness gets an overridable RELAY_HOST/RELAY_BIND
and documents the loopback/RFC1918 stripping limit it inherits, and
its new --port guards a missing value instead of dying on set -u.
- EventSyncTest builds both scenarios through one helper.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PguqnDbP2v11dtANs9xdxc
Two problems in the shared non-relay client, both of which make every HTTP
request cost more than it should.
The connection pool was being emptied constantly. buildHttpClient() evicted the
whole pool whenever the proxy differed from the last one it was handed, tracked
in a single field. But one factory mints BOTH long-lived variants:
DualHttpClientManager builds defaultHttpClient (always SOCKS, since
buildLocalSocksProxy falls back to 9050 rather than returning null) and
defaultHttpClientWithoutProxy (always null) from the same instance, and the two
share one rootClient.connectionPool. So the field alternated between the proxy
and null forever, and every rebuild read as a route change and wiped the pool
they share. Both are stateIn(WhileSubscribed(1000)) flows collected from a
composable, so that happened on every isMobileDataProvider change and every
foreground round trip — and the next image then paid a fresh DNS + TCP + TLS.
This is a plausible source of the "first image after a pause takes forever"
stall the pingInterval above it was added for.
ProxyRouteTracker narrows it to what the eviction was actually for: a direct
build never evicts (null is that variant's permanent route), and a proxied build
evicts only when the Tor port really moved. Nothing is lost by being this narrow
— OkHttp's Address, the pool key, already includes the proxy, so proxied and
direct connections to the same host are distinct entries that can never be
handed to each other's calls.
Second, maxRequests was 128. Dispatcher's executor is an unbounded cached pool,
so that is the thread ceiling: up to 128 threads running 128 concurrent TLS
handshakes on a handset. Nothing upstream bounds the arrival rate either —
Coil's enqueue is unbounded and PrefetchFeedMedia warms ±3 notes on both sides
of the viewport on every visible-range change — so a fast scroll really does
reach it. Past saturation, more concurrency slices the same bandwidth thinner
and pushes every image's completion out together, the on-screen one included.
32 total / 8 per host keeps the per-host lift that feeds need while letting
visible images finish and paint.
The pool fix is covered by tests. The dispatcher numbers are a reasoned choice,
not a measured one — worth a run against benchmark/ before release.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TdidqyhRni5L3ft5h8qti5
GifVideoView's Loading branch emitted DisplayBlurHash unconditionally, and
DisplayBlurHash renders nothing at all when both hashes are absent —
placeholderModel(null, null) returns null and the composable early-returns.
That is exactly the state a post with no imeta lands in. With no `dim` tag and
nothing in MediaAspectRatioCache, `ratio` is null, so mediaSizingModifier falls
to a bare fillMaxWidth() with no height constraint. The container then wraps an
empty loading state and the whole note collapses to zero height: no picture, no
URL, no spinner, just a gap in the feed for however long the fetch takes, and
then the image appearing from nowhere. Seen on a kind-1111 comment from Sidecar
whose content is a single blossom .gif URL.
UrlImageView already has the ladder this needs, so mirror it: blurhash/thumbhash
when there is one, a spinner in the reserved box when only a ratio is known, and
otherwise the URL plus a loading symbol via WaitAndDisplay. Every branch now
emits something with a height.
This is the missing feedback, not the latency — a slow fetch still takes as long
as it takes, it just stops being invisible while it does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TdidqyhRni5L3ft5h8qti5
MainActivity.onPause() calls debugState() unconditionally, and every line it
emits is Log.d built through the *eager* overload — so the arguments are
evaluated before the level check. Those arguments are the expensive part:
nine materialising LargeCache.filter scans over notes/addressables/users and
the three channel maps, nested sums over every channel's and chatroom's
notes, two sorted passes, and three passes calling Event.countMemory(), which
itself walks every tag and tag element of every cached event.
Release builds sit at WARN, so all of that ran on every backgrounding and the
result was discarded. Gated on `Log.minLevel > LogLevel.DEBUG` rather than
`isDebug`, because that is exactly the condition under which the lines are
dropped: benchmark builds are `isDebug` but sit at INFO, so an isDebug gate
would have left the one variant whose numbers are meant to be trustworthy
still paying the cost. The function only reads and logs — no mutation — so
the early return cannot skip a side effect.
The same shape is already gated elsewhere: AppModules builds relayReqStats
and bootDiagnostics as `if (isDebug) ... else null`, and BootRelayDiagnostics
does comparable work. debugState was the one that was not.
fix: rethrow CancellationException in six catch blocks
All six sat in suspend functions whose try body contains a suspension point,
so a cancelled scope surfaced as CancellationException and was swallowed,
against the `if (e is CancellationException) throw e` convention this
codebase follows in 192 files. What cancellation used to mean:
- VanishRequestsState: published ComplianceStatus.ERROR, showing a relay as
having answered badly when it was never asked.
- NappletResourceFetcher: reported ERROR_NETWORK to the sandboxed page, so a
cancelled fetch was indistinguishable from an upstream failure.
- MarmotAgentStreamWatcher (two sites): the per-candidate handler does
`continue`, so a cancelled watcher kept dialling the remaining broker
candidates.
- CallSession: logged as a PeerConnection creation failure.
- NamecoinSharedPreferences: returned emptyList(), i.e. "no pinned certs".
Uses kotlin.coroutines.cancellation.CancellationException, the import that
also works from commons/commonMain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123kXtseu4X18hL3GMDcdER
- Only retry a refused BOLT12 offer over BOLT11 when the wallet rejected it
before attempting a payment (EXPIRED, NOT_FOUND, BAD_REQUEST,
NOT_IMPLEMENTED, UNSUPPORTED_PAYMENT_INSTRUCTION, UNSUPPORTED_NETWORK).
NIP-47 defines PAYMENT_FAILED as possibly "due to a timeout", so an HTLC
can still settle after that reply; retrying on it, or on INTERNAL / OTHER
/ a missing code, could pay the recipient twice. The decision is now an
allowlist and the test locks the non-retry set.
- NwcInfoCache exposes an `updates` counter bumped on every stored entry.
The zap picker keys its rail recompute on it, so a BOLT12-only recipient's
bolt appears when the wallet's kind:13194 info lands after the popup
opened, instead of only after closing and reopening it.
- A BOLT12 refusal with neither message nor code no longer toasts the raw
"%1$s" placeholder; the code name (or OTHER) fills the detail.
- One abbreviateBolt12Offer() replaces three copies of the lno1 truncation
in the profile chip, the offers dialog and the settings screen.
- Reword the "these four" recompute-key comment so it covers the BOLT12
reads added alongside it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LCxV2YvpegwUEKUhN13Mvq
The pre-insert cache look added in 6dc631e85d narrowed the single-flight
gap but didn't close it: a fast leader can insert, sign, cache and retire
its entry entirely between a straggler's cache read and its putIfAbsent,
so the straggler wins an empty map and signs a second time. That is the
intermittent `expected:<1> but was:<2>` in aFastSignerStillSharesOneSignature
failing main CI.
Check the cache again once this caller owns the slot. A leader always caches
before retiring its entry, so any token minted before the insert is visible
there; take it and give the slot back instead of re-signing.
Reproduced locally at round 2431 of 5000; two 5000-round runs pass with the fix.
`aFastSignerStillSharesOneSignature` failed on a loaded machine (rounds 11,
18 and 31 across three runs, on the branch head and on the already-pushed
commit alike): a straggler could miss the in-flight map, miss the cache,
get descheduled, and then win `putIfAbsent` only because the fast leader
had already cached *and* retired its entry — and sign a second time.
Take one more look at the cache after winning the in-flight slot. A prior
leader's cache write happens before its `remove`, and winning the slot
after that `remove` goes through the same ConcurrentHashMap bin, so the
just-minted token is visible there; hand it out and retire the slot instead
of launching a duplicate signature. The test class now passes four runs in
a row where it previously failed three in a row.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N56KzPSYiN5edMRamvKEgD
The always-on notification read its "Connected to N relays" from
RelayPool.connectedRelays, a set that only moves on the onConnected /
onDisconnected callbacks. That set over-reports: a relay that sent a
WebSocket CLOSE frame never produces a callback (the app does not answer
onClosing, so OkHttp fires neither onClosed nor onFailure, and the later
cancel() is silent too), so the URL lingers until the 120s ping path
finally fails. After the feeds tore down in the background this left
hundreds of already-dropped relays in the count for minutes, with no
subscription in the "show details" breakdown to justify any of them.
Expose the pool's ground truth instead: RelayPool.connectedRelayUrls()
reads each member's isConnected(), surfaced as INostrClient.connectedRelays()
(defaulting to the flow's value for pool-less clients). The notification
keeps the flows only as a trigger, merging in availableRelaysFlow because
that is the flow that moves when the pool drops such a relay, and re-reads
the live count on each sample. The "show details" breakdown and the
Active Subscriptions screen read the same source so all three agree.
A pool test pins the drift: with a socket layer that never confirms the
close, removing a relay leaves it in the flow but out of the snapshot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014cq6vrfQkASwxgqXXY4py8
The wallet resolves the offer itself, so a stale or unreachable kind:10058
offer surfaces as a NIP-47 error reply, which by the spec means nothing was
paid. When that recipient also publishes a lightning address, re-send the
same share as a regular zap through the BOLT11 lane instead of toasting the
BOLT12 error; the toast stays for a recipient with no other route.
Bolt12LightningFallback keeps the decision pure and tested: every refusal
retries except the ones about our own wallet (insufficient balance, quota,
rate limit, restricted, unauthorized, unsupported encryption), which would
fail the same way over BOLT11. A paid-but-no-receipt outcome is never
retried, and neither is a wallet that never answers: sendBolt12Zap now
passes a timeout handler, so a silent wallet reports a timeout and steps
the progress instead of leaving the zap hanging.
The BOLT11 lane moves into zapOverLightning so the main zap and the
fallback share one path, and Bolt12Recipient carries the lnAddress and
relay hint the retry needs.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LCxV2YvpegwUEKUhN13Mvq
Follow-ups to the commons/commonsUI split.
Feed data-access layer: FeedFilter, AdditiveFeedFilter, AdditiveComplexFeedFilter,
ChangesFlowFilter, FeedContentState, FeedState, InvalidatableContent,
DefaultFeedOrder, RepostRenderability and friends move from the misleading
`ui.feeds` package to the root of `commons.feeds`, next to `feeds/custom`.
The `ui.feeds` composables (NewPostsChip, RelayReachMarker, DM history cards)
stay in commonsUI and gain explicit imports. Consumer imports rewritten.
Chess: the eleven composable files in commonsUI move from the flat
`nip64Chess` package to `nip64Chess.ui`; the logic stays in
`commons/…/nip64Chess`. Consumer imports rewritten.
CLI size budget: measured after the split (1.15.2, Linux x64) the JVM
tarball is 55 MB and the jlink image tarball 80 MB, so the release gate
drops from 200 MB to 120 MB per asset. BUILDING.md, the architecture docs,
the feed-patterns skill and the split plan record the new state.
Verified: JVM compiles for commons, commonsUI, cli, desktopApp; Android debug
compiles for nappletHost and amethyst; commons/commonsUI/cli/desktopApp JVM
test suites.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N56KzPSYiN5edMRamvKEgD
aFastSignerStillSharesOneSignature still failed about two runs in five: a
straggler that read the in-flight map and the cache as empty could then win
putIfAbsent because the leader had already signed, cached and retired its
entry in between, and would sign a second token.
The leader's cache put happens-before its removal of the same key, so once
a caller has claimed the slot a cached token, if any, is visible. Look once
more there: hand the cached token to this caller and to any follower that
already picked up the fresh deferred, retire the entry, and skip the
signature. Ten reruns of the class pass where two in five failed before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LCxV2YvpegwUEKUhN13Mvq
Lint reports 874 warnings / 0 errors per amethyst variant. 512 of those are
in Crowdin-managed values-*/strings.xml (mostly MissingQuantity — a
translator's plural missing a quantity) and are not ours to hand-edit; this
takes the ones that are mechanical and behaviour-preserving:
- EmptySuperCall (24 in amethyst, plus 2 in commons the amethyst run cannot
see): ViewModel.onCleared is documented empty, so drop the super calls.
- UseKtx (2): Canvas.withTranslation for the LaTeX drawable, and
Bitmap.toDrawable for the map pin — the KTX form CLAUDE.md asks for, and
both compile to the same calls.
- ConstantLocale (1): CalendarEventListCard held its "MMM" formatter in a
file-level val, which captures Locale.getDefault() once — month
abbreviations stayed in whatever language was active at class init. The
formatter is now cached per locale, which keeps the property the original
comment was protecting (a formatter per recompose was 500 allocations while
scrolling), and the locale comes from LocalLocale.current.platformLocale so
the read is observable: Locale.getDefault() inside a composable is not, and
Compose's own NonObservableLocale check rates that an error.
- UnusedResources (6): the Android Studio new-project wizard's leftover
colors (purple_200, teal_200, teal_700, black, white, transparent), each
verified unreferenced from Kotlin and XML. purple_500/700 are in use and
stay.
- UseTomlInstead (3): the debug-only Compose/Perfetto tracing dependencies
move into the version catalog. Same coordinates and versions; the catalog
already carries BOM-managed versionless entries.
playDebug goes from 874 warnings to 838, still 0 errors.
Deliberately left, because each is a decision rather than a cleanup:
AppLinkWarning (autoVerify only works if the domains serve a matching
assetlinks.json), the 126 unused source strings and 10 PluralsCandidate
(both churn the translation surface), GradleDependency /
NewerVersionAvailable (dependency bumps need the license check), VectorRaster
/ VectorPath / IconDensities / IconXmlAndPng (redrawing assets), BatteryLife
(the battery-optimization helper working as designed), and InlinedApi /
ClickableViewAccessibility / DiscouragedApi / InsecureBaseConfiguration (each
needs its surrounding intent read first).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123kXtseu4X18hL3GMDcdER
The zap picker gated its Lightning bolt on the recipient's lud16/lud06, but
the send path has routed a recipient with a kind:10058 offer over BOLT12 for
a while (when our default NWC wallet advertises `pay`). A recipient who
published only an offer therefore had no bolt in the popup and no one-tap
zap, even though ZapPaymentHandler could pay them.
Teach RailCapabilityResolver about the BOLT12 route: hasLightning is now true
for a recipient with an offer when our wallet can pay offers. The rail keeps
its single bolt — which flavour is used stays a send-time decision. The
popup observes the recipient's offer list and the default wallet URI so the
bolt appears as those load, and the one-tap fast path uses the same check.
The sender-side test moves into AccountZapActions.canZapViaBolt12 so the
handler, the picker and the profile dialog share it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LCxV2YvpegwUEKUhN13Mvq
The Crowdin workflow now passes import_eq_suggestions: true, so a value
equal to the English source is no longer skipped on upload — the repo's
locale files are the seed for Crowdin's database. The skill said the
opposite ("Don't add source-identical fallbacks"), which now leaves keys
untranslated in the UI forever.
- Background rewritten for the flag, with the confirming evidence: run
34706537802 -> PR #4107 round-tripped 330 identical values with zero
net key changes.
- Items 1-3: missing keys are actionable in the repo; copy English
verbatim, except <plurals> (trips MissingQuantity) and words a locale
would genuinely translate.
- Item 4 reconciled with item 3: seeding a key Crowdin holds nothing for
sticks; overwriting a value it holds differently still loses.
- Records the diff-reading trap: compare key sets per file, never -/+
lines separately, or a reorder reads as a mass strip.
Every build printed "Deprecated Gradle features were used in this build,
making it incompatible with Gradle 10". With --warning-mode all that was
five distinct Kotlin DSL delegated-property deprecations, all in our own
scripts:
- `val x by extra(...)` / `val x: T by extra` in the root script, for the
opt-in Sonar gate that buildscript {} publishes and the body reads. Now
extra.set("x", v) and extra["x"] as T.
- `val x by getting { }` for eight of quartz's KMP source sets. Now
getByName("x") { }, which is what commons already used. None of those
vals were referenced, so the local binding goes away with them.
- `val x by tasks.registering { }` and the typed
`by tasks.registering(T::class) { }`, thirteen tasks across quartz,
commons, cli, geode, nestsClient and desktopApp. Now
tasks.register("x") { } and tasks.register<T>("x") { }, which return the
same TaskProvider, so the dependsOn / finalizedBy references to them are
unchanged.
`./gradlew --warning-mode all help` is now silent, and all nineteen
converted tasks still register and run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123kXtseu4X18hL3GMDcdER
BlossomReadAuthTokenProviderTest.aFastSignerStillSharesOneSignature still
fails about one run in three (round N: expected 1 signature, was 2), which
the pre-push hook turns into a hard block on every push.
The second cache look added in 6dc631e8 closes the leader-finished-early
window for callers that reach it after the leader retired its entry, but
not for a caller parked between that look and its own putIfAbsent: a
leader that starts after the caller's miss can sign, cache and retire in
that gap (a local key does it in microseconds), so the parked caller's
putIfAbsent then succeeds against an empty map and mints a second token.
Once the caller holds the in-flight slot, any earlier leader has already
cached, because a leader caches before it retires. So a cache hit taken
at that point is definitive: hand the cached token to ourselves and to
every follower already parked on our deferred, and retire the slot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PguqnDbP2v11dtANs9xdxc
BOLT12 offers saved in Settings were only reachable through a small bolt
button in the profile header action row, next to a second NIP-A3 wallet
button, while every other way to pay a profile (Lightning, CLINK, on-chain,
Cashu, NIP-A3 targets) rendered as a chip in the payment rail below the bio.
Render one chip per NIP-B1 offer in that rail: tap opens the existing
copy / pay-with-wallet / pay-via-intent dialog for that offer, long-press
copies the raw lno1 string. Remove both header buttons, since the rail
already lists every NIP-A3 target with the same tap-to-pay and long-press
copy behaviour the dialog rows have. The two dialogs stay (the reaction row
still opens the NIP-A3 one), so their files are renamed after what is left.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LCxV2YvpegwUEKUhN13Mvq
`:commons` is on the CLI classpath, yet it declared Compose UI, Coil, Compose
resources, markdown and desktop Compose as dependencies, dragging ~40 MB of
UI/Skiko jars into every `amy` distribution. This moves every
Compose-dependent file into a new KMP module, `:commonsUI`, that
`api`-depends on `:commons`; `:commons` keeps only the Compose runtime
(stability annotations + snapshot state) and lifecycle-viewmodel.
Files keep their `com.vitorpamplona.amethyst.commons.*` packages, so the split
is a build-graph boundary and no consumer import changed. 236 files were
`git mv`'d (composables, icons, robohash, theme, Coil fetchers, the
`@Composable` relay-client entry points, `composeResources`, and the tests
that exercise them). Two headless files needed surgery instead of a move:
`GalleryParser` lost a vestigial foundation `@OptIn`, and the
`LocalPrivacyLockState`/`lockStateFor` CompositionLocal accessor moved out of
`PrivacyLockState` into its own commonsUI file. The feed DAL under `ui/feeds`
and `ui/note/ParentNote`+`ReplyContext` stay in `commons` because ViewModels
depend on them.
`amethyst`, `desktopApp`, `nappletHost` (NappletWebContract serves the shell
from composeResources) and `benchmark` now depend on `:commonsUI`; `cli`,
`geode` and `marmotBench` do not. commons' androidMain gains an explicit
androidx.core KTX dep it previously got transitively through Compose UI.
CI, crowdin, the icon-font tools and the escaping hook point at the new
composeResources location; CLAUDE.md, commons/ARCHITECTURE.md, a new
commonsUI/ARCHITECTURE.md, CONTRIBUTING, BUILDING and the affected skills
document the boundary. A plan doc under commons/plans records the
classification method and follow-ups.
Verified: JVM compiles for commons, commonsUI, cli, desktopApp; Android debug
compiles for nappletHost and amethyst; commons/commonsUI/cli JVM test suites;
both verifyKmpPurity gates; the cli runtime classpath no longer resolves
Compose UI, material3, Skiko or Coil.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N56KzPSYiN5edMRamvKEgD
Every test that used to open a socket to something outside the repo now
talks to geode, the relay this project ships, either in-process or as the
embedded `amy serve`.
- amethyst: the Android instrumented EventSyncTest dialed vitor.nostr1.com,
pyramid.fiatjaf.com and the nos.lol / nostr.mom defaults and asserted
nothing. Replaced by a JVM unit test on geode's InProcessRelays that
preloads a source relay and asserts what lands on the outbox, inbox and
DM relays, plus a NIP-42 variant with the source behind FullAuthPolicy
to cover the RelayAuthenticator wiring. Wires :geode, its testFixtures
and the JVM SQLite driver into amethyst's unit-test classpath, the same
way quartz's jvmAndroidTest already does.
- cli/tests: the cache, dm and marmot headless harnesses cloned and
cargo-built nostr-rs-relay on first run. They now boot `amy serve`
(geode) from the amy binary they already build, via a shared
start_local_relay / stop_local_relay in headless/helpers.sh. Rust is no
longer needed for the cache and dm suites at all.
- cli/tests/marmot/marmot-interop.sh: the interactive harness defaulted
to relay.damus.io / nos.lol / primal / bitcoiner.social / nostr.mom,
with `--local-relays` pointing at MDK's docker stack. It now boots the
embedded relay on 0.0.0.0 by default (the phone reaches it over the
LAN) and keeps the public set behind an explicit `--public-relays`.
- docs: cli/tests/README.md, CONTRIBUTING.md, cli/DEVELOPMENT.md and
cli/ROADMAP.md no longer describe a loopback nostr-rs-relay.
The quartz prodbench probes (NegentropyStallRepro, CursorTerminationProbe,
NegentropyMultiRelayLiveTest, ProductionReceiverBenchmark, ...) are left
as they are: they are opt-in diagnostics of production relay behaviour,
gated behind PROD_RELAY_BENCH / NEG_MULTI / NEG_STALL_REPRO, and would
measure nothing against a local relay.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PguqnDbP2v11dtANs9xdxc
Second half of the warning sweep, covering :amethyst (both flavors, all
build types, unit + instrumented tests), :desktopApp and :benchmark:
- GitReplyEvent (NIP-34 kind 1622) is deprecated in favour of NIP-22
comments, but events already on relays still arrive and still have to be
routed, rendered and surfaced. @Suppress("DEPRECATION") with that reason
at the five production sites and the coverage test that pins them.
- The four instrumented Compose tests move to
androidx.compose.ui.test.junit4.v2.createComposeRule. The v2 factory
returns the same ComposeContentTestRule, so mainClock, setContent and the
node assertions are unchanged; only the effect dispatcher differs.
- RelayAuthPromptBusTest / RelayAuthSessionGrantsTest: @OptIn for the
ExperimentalCoroutinesApi members (testScheduler.currentTime, runCurrent)
they already use, matching the annotation the file's other tests carry.
- MarmotFileUploader: drop a nullable alias of a non-null cipher, left
behind when the v2 reference stopped being conditional.
- LocalCacheSearchParityTest: hoist the Json format out of the loader.
- LivesSection: FlowRowOverflow and FlowRow's overflow parameter are
deprecated; the non-deprecated overload already clips beyond maxLines.
- HexBenchmark: drop a bare `null` expression statement from the measured
lambda.
Also fixes :commons:compileCommonMainKotlinMetadata, which did not compile
at all: shared code called BigDecimal.toLong(), which resolves in every
platform compilation (every actual is a Number) but not in the common
metadata one, where only the expect class's own members are visible.
`expect class BigDecimal : Number` cannot work — java.math.BigDecimal leaves
toByte()/toShort() abstract, so the JVM typealias fails the expect/actual
modality check — so the conversion is a top-level expect/actual extension
instead, with actuals next to each BigDecimal actual.
Verified warning- and error-free across the jvm, android (play/fdroid ×
debug/release/benchmark), linuxX64, and the common/jvmAndroid/native/apple/
ios metadata compilations. The apple actuals are checked by
compileAppleMainKotlinMetadata, which runs the frontend against the Apple
klibs without needing a macOS host.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123kXtseu4X18hL3GMDcdER
Fixes the Kotlin warnings the compiler reports for these modules across the
jvm, android, linuxX64 and metadata compilations:
- PartialTokensTest / marmotBench: drop redundant casts. kotlin.test's
assertTrue carries a `returns() implies` contract, so the `is` check
already smart-casts; the benchmark values were never nullable-typed.
- MarmotPublish*Test, LastResortKeyPackageReuseTest: name overridden
parameters as the supertype does (`retainedSecrets`, `snapshot`), so
named-argument calls through the interface stay correct.
- AuthOutcomeTest: PersistentMap.put is deprecated in favour of putting(),
which is what the rest of the codebase already uses.
- IndexableContentGoldenTest: drop an unnecessary !! on a non-null String.
- Nip46Test: the generic encode/decode round trip cannot be checked at
runtime, so suppress UNCHECKED_CAST with a note on why it is safe.
- InternTradeoffBenchmark: hoist the liveness anchor from a local to a
field. As a local its assignments were visible to data flow, which folded
the trailing `check(sink != null)` into a constant.
- Http3GetClient: an empty `else -> {}` branch instead of a bare `Unit`
expression statement.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123kXtseu4X18hL3GMDcdER
BlossomReadAuthTokenProvider.header() reads the token cache, then signOnce()
reads the in-flight map — two separate reads. A leader caches its token before
retiring its in-flight entry, so a caller sitting between those two reads sees
an empty cache (its read came first) and an empty in-flight map (the leader
already finished), and signs a second token for the same host.
A 300ms test signer never opens that window, which is why the provider's own
concurrency test missed it. A local in-process key signs in microseconds, so
BlossomReadAuthFetcherTest.aBurstOf401sSharesOneSignature — 16 fetchers that
all 401 and all retry — hit it and intermittently saw two distinct tokens.
signOnce() now takes a second look at the cache once it finds no in-flight
entry: an absent entry proves the leader's cache write is already visible, so
the straggler reuses that token instead of starting another signature.
Covered by a new aFastSignerStillSharesOneSignature, which runs the 16-caller
burst against an instant signer over many rounds — the existing test's slow
signer cannot reach the window.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011APd48WJ5pZVK4Ltpj3YpL
app 1.15.1 -> 1.15.2, appCode 459 -> 460. That single edit drives Android's
versionName/versionCode, Desktop and CLI packageVersion, quartz's Maven version
and geode's RelayInfo.VERSION.
Three substantive PRs since v1.15.1, plus Crowdin translations and the packaging
syncs the bump workflows opened after the last tag:
- #4092 media previews: an extension match now requires a real dot, so a player
page whose path merely ends in the letters `_mp3` stops going to the video
player, and `og:audio`/`og:video` are read and played with the page's
`og:image` as poster. A declaration whose type is `text/html` -- YouTube's --
is refused.
- #4095 nested NIP-22 replies: engagement subscriptions asked only for the
lowercase `e`/`a` tags, so a comment two or more levels deep was invisible
until ThreadScreen opened its own subscription. Each relay gets a second,
root-scoped filter on `E`/`A`. Kind 1619 moves there too -- NIP-34 gives PR
updates only an uppercase `E`, so it had been in a filter it could never match.
- #4096 Health Connect: a rationale screen Play requires, reachable from the
composer, from Health Connect's permission screen and standalone without an
account; reads moved off the UI thread; source names memoized; and the workout
form is replaced rather than merged when a second suggestion is picked.
Verified on a Pixel 9 emulator before cutting, since two of the three are only
observable on device: the og:audio track plays in a thread with real transport
controls (00:30 / 05:00) where it used to buffer forever, and the Health Connect
rationale opens from all three routes -- including the one that matters for
review, where Health Connect's own permission screen launches our
ViewPermissionUsageActivity through the START_VIEW_PERMISSION_USAGE-guarded
filter.
RELEASE_NOTES_ID deliberately stays on the v1.15.0 note: RELEASE_OPS has it
repointed on x.y.0 only.
Left alone deliberately: everything under */packaging/ and translators.json's
tag, which the bump workflows and the Crowdin job write after the tag exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
Three defects found auditing the Health Connect workout path.
applyPrefill only assigned a field when the incoming route carried a value,
so picking a second suggestion merged into the first instead of replacing it.
Tapping a run (5 km, 380 kcal) and then a gym session left the run's distance
and calories in the form — the user publishes numbers from a workout that
never happened. Every metric is now assigned on both branches. Notes are
deliberately left alone: they are typed by the user, never carried by a route.
The `source` tag published the writing app's display label ("Samsung Health"),
or its raw package name when that app is not installed. SourceTag defines a
vocabulary — gps / manual / health_connect — and the feed badge uppercases
whatever is in the tag, so an imported workout rendered as "SAMSUNG HEALTH"
or "COM.HUAWEI.HEALTH" beside other clients' "GPS". It now publishes
SourceTag.HEALTH_CONNECT, which existed for this and had no callers.
DetectedWorkout.source keeps the friendly name for the UI.
readNewWorkouts ran entirely on Dispatchers.Main: the callers launch into a
composable's rememberCoroutineScope, and nothing in the feature switched
dispatcher. Health Connect's own calls suspend, but resolveSourceName's
PackageManager lookup is a blocking binder call made once per session, so a
week of sessions blocked the UI thread once each. The read now runs on
Dispatchers.IO and the label lookups are memoized per writer package.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019egdJyBHnrATZHjs86up8f
NIP-34's PR Update example carries only `["E", <pull-request-event-id>]` —
there is no lowercase `e` tag on a 1619 at all, and
GitPullRequestUpdateEvent.build() writes only RootEventTag to match. Listing
1619 in RepliesAndReactionsKinds2 (the `#e` filter's kind list) therefore
never pulled a single PR revision, and the comment claiming it was "rooted at
the target patch/PR/issue via a `root`-marked `e` tag" was wrong for that kind
(it is correct for the 1630-1633 statuses beside it).
The `#E` filter added in the previous commit is what actually makes a PR's
revision chain reachable from an on-screen PR row, so 1619 moves there and
comes out of the `e` list.
Nip34NotificationCoverageTest asserted the false half, which is how this
survived: it now checks each kind against the filter that can actually match
it, and pins 1619 out of the `e` list so it cannot drift back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ARAmQQWbT2gGVo9tysdAgM
Per CONTRIBUTING-WITH-AI.md, the branch got a review pass before the PR. All
six findings were real; none of them fire in the nostr.build case the feature
was built against, which is exactly why the tests missed them.
- Only a root-relative `og:audio`/`og:video` was resolved. A document-relative
`media/x.mp3` -- and any value at all when the page URL fails to parse --
reached the player verbatim as a relative string, which can only hang. Now
every reference resolves against the page, and an unresolvable one is refused
rather than passed on.
- Nothing checked the scheme. `og:audio = file:///...` with `audio/mpeg` passed
the gate, letting a remote page aim the local player at a local URI. Playable
media must now be http(s).
- The branch keyed on "is this playable", not "is this a player page", so an
article that merely embeds a clip lost its entire card -- headline, summary,
host, tap-through -- and got a bare player instead. `og:type` is now read and
required to be `music.*` or `video.*`; an article, or a page that declares no
type, keeps its card. The fixtures already carried the tag; nothing read it.
- Relaxing fetchComplete() broke the unstated "Loaded implies a non-empty
image" invariant, so a cover-art-less track page painted a blank 180dp box in
UrlPreviewCard (and through it UrlScreen and DisplayExternalId).
WebBookmarksScreen guarded with `!= null` on a non-null String, which is
vacuous. Both now test for blank.
- The new `audio/*` branch in UrlPreview produced Loaded states that the
composer's two preview handlers did not cover, so a direct audio URL was
handed to Coil to decode as an image and showed an empty square. Audio joins
video there, as it already had in LoadUrlPreview.
- The video/audio branch never forwarded the fetched Content-Type to the
player, contradicting its own new comment. An HLS playlist served as
`audio/x-mpegurl` from an extension-less URL was then unrecognisable to
MediaItemCache and played as progressive, i.e. not at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8uqrrzDpqUYxV9R7CJgxW
Google rejected the Health Connect declaration for "Insufficient Information
to Determine App Functionality": neither the store listing, the privacy
policy, nor the in-app experience explained what Amethyst does with the seven
read permissions it asks for.
All seven are used by HealthConnectManager, so none can be dropped. What was
missing was the explanation, on every surface a reviewer looks at:
- The rationale intents the manifest declares
(ACTION_SHOW_PERMISSIONS_RATIONALE, ACTION_VIEW_PERMISSION_USAGE +
CATEGORY_HEALTH_PERMISSIONS) pointed at MainActivity, which handles neither
— tapping "privacy policy" in Health Connect just opened the feed. They now
land on HealthConnectRationaleActivity, a static, account-free screen that
lists each data type, what it fills in, and what the app never does. It is
also reachable from a "What Amethyst reads" link on the Connect card, so
the rationale is available before the permission request, not only after.
- PRIVACY.md gains a "Health and fitness data (Health Connect)" section: a
per-type purpose table plus the limits (read-only, foreground-only, 7-day
window, no route/background/history permissions, no secondary use).
- The Play listing description now covers the app's features and the
Workouts flow, so the listing reflects what the permissions are for.
- docs/health-connect-play-declaration.md holds the paste-ready Play Console
text: app functionality, a reviewer walkthrough, and a per-permission
justification — including that CyclingPedalingCadence and StepsCadence come
bundled with READ_EXERCISE and READ_STEPS and are never read.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019egdJyBHnrATZHjs86up8f
The per-note engagement subscription only asked relays for the lowercase
`e`/`a` tags. NIP-22 puts the conversation root in the **uppercase** `E`/`A`
tags and only the direct parent in lowercase, so a kind-1111 comment two or
more levels deep never carries `e`=<rootId> (nor `a`=<address> when the root
is an article). Those comments therefore never matched the feed's REQs and
only showed up once ThreadScreen opened its own `E`/`A` subscription — which
is exactly the "half the replies appear later" behaviour on note cards.
Kind-1 threads were unaffected: NIP-10 repeats the root `e` tag on every
descendant, so the existing filter already caught them.
LocalCache was never the problem: CommentEvent.tagsWithoutCitations()
already returns root + reply ids, so a nested comment is wired into the root
note's replies as soon as it arrives — it simply never arrived.
Adds an `E` filter (kinds 1111 + 1619, which also anchors with `E`) to
filterRepliesAndReactionsToNotes and an `A` filter (kind 1111) to
filterRepliesAndReactionsToAddresses. They are separate filters because tag
names inside one filter are ANDed — folding `E` into the `e` filter would
only match comments carrying both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ARAmQQWbT2gGVo9tysdAgM
nostr.build mirrors its audio player page for video, confirmed against the live
host: `e.nostr.build/v_<id>_mp4` answers `text/html` and declares
og:video https://v.nostr.build/<id>.mp4
og:video:type video/mp4
— the same shape as the `og:audio` page, and the same `_mp4`-without-a-dot trap
the previous commit stopped misrouting into ExoPlayer. So og:video gets the same
treatment: parsed, type-checked, and played with the page's og:image as poster.
Reading og:video is not the same as trusting it, and video is where that
distinction earns its keep. YouTube declares
og:video:url https://www.youtube.com/embed/dQw4w9WgXcQ
og:video:type text/html
which is markup, not a video — playing it would reproduce the exact bug this
series exists to fix. UrlInfoItem therefore accepts a declaration only when its
type holds up (classifyMedia == VIDEO, so `video/*`, `audio/*` and the HLS
playlist MIMEs all pass, `text/html` does not) or, absent a type, when the URL
carries a real media extension. YouTube falls through to its link card, as it
should. Both cases are pinned as tests from the live tags.
`playableAudioUrl` collapses into `playableMediaUrl` + `playableMediaType`, so
the renderer has one branch and the preference rule — video wins a page that
declares both, since the audio path deliberately strips the picture — lives in
one testable place. A refused og:video still falls back to a playable og:audio.
Also reads the `:url`/`:secure_url` aliases, since hosts disagree on which to
emit (nostr.build the bare key, YouTube only `:url`), and drops the parser's
early exit outright: it stopped at title+description+image, a set every page
completes *before* it reaches the media tags, so it could only ever skip them.
The scan is bounded regardless — MetaTagsParser stops at `</head>`.
Verified: the nostr.build and YouTube tags above are from curl against the live
pages. commons jvmTest (2163) and amethyst fdroid unit tests (1471) pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8uqrrzDpqUYxV9R7CJgxW
A note linking nostr.build's audio player page rendered a dead player:
https://e.nostr.build/a_ETvKzX2OdOGmFEp1avRlm5_mp3?t=Aria&by=The+Fishcake
That URL is not an mp3. It answers `Content-Type: text/html` — it is the
human-facing player page, and the track it is about lives at
`a.nostr.build/ETvKzX2OdOGmFEp1avRlm5.mp3` (`audio/mpeg`), named in the page's
`og:audio`. We fed the HTML to ExoPlayer anyway, which can only buffer forever
and then show the browser-fallback overlay.
Two independent defects, fixed together:
1. Extension matching ignored the dot. `isAudioUrl`/`isVideoUrl`/`isImageUrl`/
`isPdfUrl`/`classifyMedia`/`parseImageOrVideo` trimmed the query and then
asked `endsWith("mp3")`, so a path ending in the *letters* of an extension
matched — `_mp3` here, and any prose slug such as `/my-thoughts-on-mp3`.
They now share one allocation-free matcher, `RichTextParser.hasExtensionIn`,
which requires a real dot-introduced extension in the last path segment and
compares case-insensitively (so `.Mp3`, which the doubled lower/UPPER lists
never covered, resolves too). SupportedContent carried a second copy of the
same dot-less scan — it admitted the player page into the video feed — and
now delegates to the shared matcher instead.
On its own this change turns the broken player into the correct OpenGraph
link card, because an unclassifiable URL falls through to LoadUrlPreview.
2. The preview pipeline ignored og:audio. OpenGraphParser now reads `og:audio`,
`og:audio:secure_url` and `og:audio:type` alongside title/description/image,
and UrlInfoItem exposes `playableAudioUrl` — the resolved URL, but only when
the page vouched for the type (an `audio/` MIME, or a genuine audio
extension when the MIME is absent), so a junk `og:audio` still falls through
to the card. LoadUrlPreview plays that file with the page's `og:image` as
cover art. Note the parser's early exit had to wait for the audio pair:
pages emit `og:audio` after `og:image`, so stopping at title+description+
image skipped the one field that makes the page playable.
So the note now plays, rather than merely stopping at a broken player.
Also: a URL that serves `audio/*` without an audio extension no longer throws
"unknown encoding" out of UrlPreview and lose the note's link, and fetchComplete()
counts playable audio as evidence so a track page with no cover art is not
discarded as Empty.
Verified: the Content-Type and og:audio claims above are from curl against the
live URLs. commons jvmTest (2154) and amethyst fdroid unit tests (1471) pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8uqrrzDpqUYxV9R7CJgxW
v1.15.0 was tagged but never shipped. Its `Create Release Assets` run failed in
`deploy-android` at the first packaging task:
Execution failed for task ':amethyst:buildFdroidReleasePreBundle'
> Entry name contains invalid characters:
root/META-INF/zoomable-root:zoomable.kotlin_module
so no AAB, no APK, and none of the 47 assets were produced.
A `.kotlin_module` is named after the Gradle project path that produced it,
colons included. 14 of the 98 modules merged into the app carry one: zoomable,
Negentropy, vico, the seven coil3 artifacts, and four of ours -- Amethyst:quartz,
Amethyst:commons, Amethyst:quic and Amethyst:nestsClient -- so renaming our own
would not have been enough.
Bisected against the two toolchain bumps this cycle, since both landed after the
last good release: AGP 9.3.1 -> 9.4.0 and Kotlin 2.4.10 -> 2.4.20. With Kotlin
held at 2.4.20 and AGP reverted, both flavours' bundle tasks pass, so Kotlin is
not the trigger. The R8 output jar carries the identical 14 colon entries under
BOTH AGP versions -- 9.4.0 added the rejection rather than the names, in
JarFlinger.addJar, reached from PerModuleBundleTask.addHybridFolder.
`packaging.resources.excludes` was tried first and cannot work, at either
`META-INF/*.kotlin_module` or `**/*.kotlin_module`: with minification on, R8
emits the java resources itself and addHybridFolder hands JarFlinger its own
predicate, so those filters are never consulted. Confirmed by deleting the R8
output and re-running rather than reading a stale intermediate --
mergeJavaResource's jar holds zero kotlin_modules while R8's holds all 98.
So the entries are stripped from R8's jar in the moment before the bundle task
opens it, and the jar is put back exactly as R8 left it afterwards. Two details
carry their weight:
- the strip is doFirst on the CONSUMER rather than doLast on R8, so a
build-cache hit on R8 cannot skip it;
- the restore is what keeps R8 up to date. Without it Gradle sees a modified
output and re-runs R8 on every build -- measured here at ~2 min for an
otherwise no-op build. With it, a second run reports
minifyFdroidReleaseWithR8 UP-TO-DATE and finishes in 1s.
Pinning back to 9.3.1 was the alternative and is one line away; the catalog
comment records that, and says to drop the workaround when AGP fixes it.
Verified from a cleaned R8 output on both flavours:
buildFdroidReleasePreBundle and buildPlayReleasePreBundle both BUILD SUCCESSFUL,
each reporting "Stripped 14 colon-named entries from base.jar".
appCode 458 -> 459. RELEASE_NOTES_ID deliberately stays on the v1.15.0 note:
RELEASE_OPS has it repointed on x.y.0 only, and 1.15.1 ships that release's
contents.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
app 1.14.0 -> 1.15.0, appCode 457 -> 458. That single edit drives Android's
versionName/versionCode, Desktop and CLI packageVersion, quartz's Maven version
and geode's RelayInfo.VERSION.
RELEASE_NOTES_ID is repointed at the v1.15.0 note
(8fce45589ea44df75e828a04c7d70bb4fabedd6ffc1946a920b2f0c7c990ff9f), which
RELEASE_OPS notes happens on x.y.0 releases and not on patches. It has to ship
in this commit rather than after it: the drawer's "Release Notes" link and the
donation card both open BuildConfig.RELEASE_NOTES_ID, so a tag cut before the
repoint ships users a link to the previous release's note. Verified present on
relay.damus.io and relay.primal.net before committing.
Adds docs/changelog/v1.15.00.md, written from the 556 commits since v1.14.0,
and its index entry. The cycle's headline is the Marmot resync: the MIP
documents were deprecated in July and MDK followed, leaving our implementation
invalid under either profile the current spec defines, so it moves onto the
adopted current profile and is now interoperable with White Noise. Marmot group
chat itself is not new -- it shipped in v1.09.0 -- and the notes say so.
Also syncs the docs that state a version rather than illustrate one, since
quartz and geode both read libs.versions.app:
- README.md, quartz-integration SKILL.md and its gradle-setup.md reference
-> quartz 1.15.0
- geode/README.md install commands -> geode 1.15.0
The Homebrew/Winget status blocks in BUILDING.md and RELEASE_OPS.md were
re-verified rather than re-stamped, and the claim had gone stale in our favour:
both Homebrew packages are live upstream now. formulae.brew.sh answers 200 for
the amethyst-nostr cask (at 1.14.0) and for the amy formula, while geode-relay
404s and microsoft/winget-pkgs still has no VitorPamplona/Amethyst -- PR #422752
is open pending CLA. Both blocks now say that, RELEASE_OPS gains a geode-relay
row, and the section heading no longer claims Homebrew is not shipping.
Left alone deliberately: everything under */packaging/ and translators.json's
tag, which the bump workflows and the Crowdin job write after the tag exists
(bumping by hand would commit wrong hashes and a dead URL); and
cli/tests/marmot/state/mdk/Cargo.lock, where 1.14.0 is an unrelated crate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
Fill the strings and plurals missing from the cs, de-rDE, sv-rSE and
pt-rBR locale files in both resource trees:
- amethyst: git status notifications, relay login dialogs, Marmot
disband/encrypted attachments, pay-to hand-off, napplet encryption,
library/ratings labels, library_directory_* plurals
- commons: Marmot avatar link, disband, disappearing messages and
system event lines, search empty states, publication/profile
labels, publication_* and birdex_species_more plurals
Source-identical entries (format strings, placeholders, brand names,
loanwords) are left for Crowdin to decide.
A swipe-from-edge back stopped sliding and started shrinking the outgoing
screen toward its centre. A back *button* press still slid, which is the tell:
only the gesture takes the changed branch.
navigation-compose 2.10.0 split the gesture off from the button. NavHost was:
if (composeNavigator.isPop.value || inPredictiveBack) { … popExitTransition }
and is now:
if (inPredictiveBack) { … predictivePopExitTransition.invoke(this, swipeEdge) }
else if (composeNavigator.isPop.value) { … popExitTransition }
with defaults of `fadeIn` opposite `scaleOut(targetScale = 0.7f)`
(DefaultNavTransitions.android.kt). We pass no predictive pair, so every
gesture back ran that 0.7 scale instead of the per-route slides. Arrived with
the 2.9.8 -> 2.10.1 bump in 86d96dda06; not in any release.
The per-destination overrides are `internal` in 2.10.1, so the only public
lever is the NavHost-level pair — and it is handed the entries but not the
builder that declared them. PopFamilies records that as the graph is built, so
predictivePopEnter/predictivePopExit can reproduce the existing rules exactly:
a modal leaves downward, a drill-in leaves toward the end, a tab entry fades.
They call popExitToEnd/popExitToBottom/popEnterFromBehind rather than respell
them, so the large-screen tier still applies and the two paths cannot drift.
navShellFadeIn/Out are now shared with the NavHost's own enter/exit instead of
being spelled twice.
Verified on emulator-5554 (Pixel 9, API 36), gesture held at the same x=540
with `input motionevent` and screencapped mid-drag:
- pre-fix build, Profile: screen scaled down, Home visible around all edges
- fixed, Profile (fromEnd): screen translated right, full size, Home revealed
- fixed, Edit Profile (fromBottom): translated down, Profile revealed above
The bottom-family case is the one that proves the lookup resolves per family
rather than answering END for everything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VgVDQQXAg4cmzsWHoJj61k
`concurrentGetFreshCallersShareOneFetch` failed about one run in four under
full-suite load with `expected:<1> but was:<2>`, and passed every time when
run alone -- so it read as a cache bug that was not one.
The callers joined on `Dispatchers.IO`, which only *schedules* them. The
test then opened the gate immediately, so the winner could finish first:
its `finally { inFlight.remove(key, ours) }` cleared the slot, a caller
arriving afterwards found nothing to join, and started a second fetch. The
coalescing the test is about had never been exercised on those runs.
`Dispatchers.Unconfined` runs a coroutine's body inline until its first
real suspension, so `async` now returns only once the caller has executed
`inFlight.putIfAbsent` and parked on the winner's deferred. The
interleaving the GatedFetch KDoc promises -- "caller one is provably inside
the fetch before the others arrive" -- is ordered by construction rather
than by timing.
`getFreshJoinsFetchAlreadyStartedByRefreshIfStale` carried the identical
race and is fixed the same way; it had not been observed failing. The three
remaining `Dispatchers.IO` callers in this file are left alone, because
there the caller IS the fetch winner and has nothing to join late.
Verified: the full fdroid suite (1464 tests), where the failure originally
surfaced, is green twice over, and the class passes 6/6 in isolation. The
isolated runs prove little on their own -- the unfixed test passed 5/5 that
way too -- so the load runs are the evidence that matters.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bumps every catalog entry that has a newer release, verified with the
JVM/Android unit test suites and the desktop Compose UI tests.
Highlights:
- Kotlin 2.4.10 -> 2.4.20 (and kotlinTest, ksp 2.3.12)
- Compose Multiplatform 1.11.1 -> 1.12.0, androidx compose BOM 2026.09.00,
runtime-annotation 1.12.1
- coil 3.5.0 -> 3.6.2, media3 1.11.1, navigation-compose 2.10.1,
camera 1.6.2, sqlite 2.7.1, firebase BOM 34.19.0, vico 3.3.1
- benchmark 1.5.0-rc01 -> 1.5.0 (final), kotlinx-collections-immutable 0.5.2,
slf4j 2.0.19, spotless 8.10.2, sonarqube plugin 7.5.0.8588
- CI: actions/cache v4 -> v6 in build.yml, matching create-release.yml
Two entries needed more than a version change:
- material3 1.9.0 -> 1.12.0-alpha03. coil 3.6.2 requires Compose
Multiplatform 1.12.0, whose foundation changed the CustomStyle.applyStyle
signature. Every material3 release built against foundation 1.11.x (the
stable 1.9.0, and the 1.11.0-alpha07 kodein-emoji drags in) then throws
AbstractMethodError from OutlinedTextFieldDefaults at runtime; two desktop
UI tests caught it. 1.12.0-alpha03 is the first release built against
foundation 1.12.x. Android still takes androidx material3 from composeBom,
so this ref only drives desktop/iOS.
- cachemap stays at 0.2.4. 0.3.0 makes CacheMap.entries/keys/values
DeprecationLevel.ERROR and throws at runtime; quartz's appleMain
LargeCache/ConcurrentHashCache read those views directly and through the
stdlib Map operators. Porting them to the new forEach/forEachKey/
forEachValue API needs a build on an Apple target, so it is left for a
follow-up. Rationale recorded in the catalog.
commons/build.gradle.kts: DisableCacheInKotlinVersion.2_4_10 no longer
exists in KGP 2.4.20 (the enum keeps a sliding window on purpose, to force a
re-check each release), so the iOS ui-uikit prebuilt-cache workaround moves
to 2_4_20.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXfvJi9QshDv9Umzp3ZYgQ
StreamCommands read the "stream-id" flag name in four places and
MarmotBenchmarks repeated the "wss://bench.invalid" relay URL three
times. Each is now a private const in its file.
Sonar flagged both delete() calls for discarding the Boolean result.
save() now warns when the temp file left behind by a failed rename
cannot be removed, and clear() only deletes a file that exists and
warns when the delete is refused.
Every NIP-51 relay list update rebuilt the event by stripping all `relay`
tags from the public tag array and writing the whole new relay set into the
NIP-44 encrypted content. A 10012 created elsewhere (Jumble, for instance)
keeps its relays in plain tags, so unfollowing a single relay in Amethyst
moved every survivor into the encrypted content and the list read as empty
in that other client.
splitRelayListUpdate() now decides where each relay goes: relays that were
public stay public, relays that were private stay private, and relays that
weren't on the list follow its convention -- public when the earlier version
was public-only, private otherwise. Applied to all nine lists that shared
the update path (relay feeds, blocked, trusted, broadcast, indexer, proxy,
search, private outbox and relay sets).
Fixes#4075
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HKnV2t1sQDTRjYNptjpXAd
The transport retry assumed the relay it is retrying is still in the connection
pool. `reconnect()` walks the pool's current relays, so when it is not, the
retry is issued, logged, dials nothing, and is reported at the deadline as the
same hang-up we already knew about — the one failure mode a retry must not
have, because it is indistinguishable from having tried.
A relay leaves the pool when nothing wants it any more. `NostrClient` recomputes
that set through `combine(...).sample(300)` and `RelayPool.updatePool` retires
whatever the sampled snapshot omits, socket included. A snapshot taken before
this publish claimed the relay therefore retires a relay with an event in
flight. The outbox still holds the event and would re-send it on the next
connect, so the only thing actually missing is pool membership.
`ensureInPool` restores it before the reconnect. Defaults to a no-op on
`INostrClient` rather than reusing `getOrCreateRelay`, which throws for clients
that expose no pool, and it is a no-op in the ordinary case where the relay
never left.
Not covered by a new test, deliberately rather than by omission: every
deterministic route to "relay absent from the pool" runs through the outbox
exhausting its own retry budget, and at that point the event has been abandoned
and NOT re-sending is correct. The one route that reaches this branch is the
300ms sampling window, which cannot be forced through the public API. So this
is defence in depth on an inferred cause, and the evidence for the inference is
the interop harness's `disconnected before OK` on test 22: the event
`a153a582…` IS stored in the harness relay's database, so the relay took it and
only the OK was lost; and the relay did not hang up (no rate limits configured,
a 20-minute idle timeout, and a 1024-message slow-client queue against a
database holding 107 events total), which leaves a client-side teardown.
Existing publish suites green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Test 29 asserted that wn reaches amy's post-disband epoch within one fixed
window. That is stricter than the protocol promises, and it failed on a
full-suite run for a case `group-lifecycle-v1.md` explicitly allows.
MDK rotates its own leaf shortly after joining, so it can commit between the
epoch-agreement gate and amy's disband — and then the two have forked. What the
spec guarantees from there is not "the disband lands first time" but that the
REQUEST survives, is regenerated against whichever branch was selected, and
lands eventually. The old assertion could only pass in the race-free case, and
a busier machine widens the race.
The loop now re-reads AMY's epoch each round, because regeneration advances it,
and drives amy's own sync, which is what carries a pass to settlement and
re-issues a disband that lost. It also adds a check the old version lacked: once
the two agree, amy must still read the group as disbanded. "Epochs agree but the
group is live" is the outcome actually worth catching, and counting epochs alone
never would have.
Evidence this is a race and not a regression from the audit fixes: the two
commits in the failing window carry different `h` tags, so they are commits in
two different groups rather than a fork, and the test passes in isolation
(`epoch 1 -> 2, wn at 2`). A full run with this change is the confirmation and
is not in yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
A review of the full branch diff turned up five, all verified against the code
before changing anything.
**A departure gate nothing ever cleared.** `leaveGroup` raises a durable
`LEAVING` gate and the only `clearGate` calls were inside
`resolveDisbandRequest`, so a member who left and was invited back held a gate
against a membership that no longer existed. Harmless until this branch, where
gates began blocking outbound work AND surviving restarts — which turned it
into a group that reads fine and can never be written to again, permanently. An
authenticated re-join now clears it, which is the rule `REMOVED` already
stated.
**An SSRF hole in IPv6 avatar hosts.** `isNonRoutableIpv6` compared TEXT, so
`::1` was caught and `0:0:0:0:0:0:0:1` — the same address, expanded — was not,
and `::ffff:127.0.0.1` shares no prefix with anything it looked for. A group
avatar URL could make every member fetch from their own machine. Addresses are
now parsed to their 16 bytes and judged numerically, with IPv4-mapped and
-compatible forms delegated to the existing IPv4 rules and an unparseable
literal refused rather than waved through.
**A message on a branch the group then adopted was never rendered.** An app
payload that decrypted only on a candidate branch had its id recorded as
processed, so a later redelivery hit the `Duplicate` early-return — even though
that result is itself a witness FOR the branch, which convergence may go on to
select. It is now retryable like `UndecryptableOuterLayer`. Safe to
re-process: witnesses are a set keyed by sender, so a resent payload adds
nothing to a branch's standing.
**One dropped socket wedged a group until app restart.** An unconfirmed publish
pins `PendingPublish`, and the only caller of `retryPendingPublishObligations`
was `restoreAll`. A blocked commit now retries that group's obligations on its
way through, so the next attempt is the recovery.
`MarmotPublishBeforeApplyTest` measured "no replacement commit" by counting
sends, which the retry breaks without violating anything: the re-send carries
the SAME event id, and a fork means a second DIFFERENT commit for the held
epoch. It now counts distinct ids, which is the property it always meant.
**`forget()` left two of the gate's three copies behind.** It dropped the map
but not the snapshot non-suspending readers see, nor the record on disk, so
`restore()` resurrected a gate for a forgotten group. Latent — no production
caller yet.
11,001 tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Four follow-ups from the disband work.
**A settled pass no longer reads as `Recovering` forever.** `settle()` cleared
the pass and terminalized a disband but never restored `Stable`, so after any
fork the group reported a recovery that had already finished. The publish gate
is what actually decides whether a commit may be prepared, so nothing locked up
— it was a lie in the reporting, which is worse in its way, since the next
reader to gate on it would have found a group that looked permanently stuck.
`endRecovery` clears only `Recovering`; `Unrecoverable` is not a pass outcome
and `Disbanded` is absorbing.
**The composer is disabled when an outbound gate is up.** Sending into a
disbanding, leaving or removed group throws behind the gate, and the UI found
out by tapping. The gate is mirrored onto `MarmotGroupChatroom` and the chat
view replaces the input with the reason. The gate map is guarded by a mutex and
the refresh path is not suspending, so `MarmotPublishGate` now publishes an
immutable snapshot for non-suspending readers rather than pushing `suspend` up
through every caller for one flag.
**A retry now outlives the deadline it was issued at.** The transport retry
already existed (7e187e39) and was still losing publishes: it shared the
caller's original budget, so the retry was issued, the clock ran out, and the
publish was reported failed having done the work and thrown the answer away —
which is how a healthy loopback relay kept costing the interop harness a
message a run to `disconnected before OK`. Issuing a retry now extends the
deadline by `TRANSPORT_RETRY_GRACE_MS`. Bounded by construction: only a relay
that gave a transport failure earns it, and only as often as the retry budget
allows. The new test fails without the change with exactly the harness's error.
**The three blocked interop directions are closed, with the reason recorded.**
The obvious next idea is to bypass `wn`'s missing verbs through its daemon, and
it does not work: `wnd`'s protocol carries `Ping`, `Status`, `Shutdown`, four
`*Subscribe` variants and `Execute { cli: Box<Cli> }` — the same clap tree `wn`
parses. A verb missing from `Cli` is unreachable through the socket too, so
closing them needs a verb upstream or a driver linked against
`marmot-uniffi`/`marmot-c`. Written down in cli/tests/README.md so nobody
re-investigates.
10,998 tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Two defects found auditing the NIP-44 change.
The in-app browser mints its own per-origin launch token and never consulted
HostProfile, so it still granted IDENTITY+RELAY. Those are exactly the surfaces
that set __nappletNip07, so the shim advertised window.nostr.nip44 and the
broker then denied every call -- worse than not advertising it, since apps stop
falling back. The website set now lives once, in NappletCapability, and both
mints read it.
Second, the broker recorded a consent grant under the REQUESTED op rather than
the op the grant itself carried. Nip44Decrypt is the first napplet-side request
with a narrower alternative (DecryptFrom(peer)), so a user tapping "always allow
for Alice" would have been stored as a broad "allow decrypt" -- every
conversation, forever, from one tap. Recording now goes through
NostrSignerPermissionLedger.record, which uses the grant's own op, and a
standing narrow grant is honoured on later requests instead of re-prompting.
This mirrors the NIP-46 authorizer, which already got both right.
With the recording fixed, the consent dialog can safely name the counterparty:
Nip44Decrypt now supplies it, so the prompt reads "read your private messages
with Alice" and offers the scoped grant beside the broad one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Hge2jR1BPnyZse75VQ4kg
The convergence-based disband landed in `commons` and `quartz` but three things
it depends on only exist per front end, and the app had none of them.
**Nothing would have carried the pass to settlement.** Terminalization now
happens when the convergence pass SETTLES, and the settler is started by
inbound traffic that detected a fork — but a disband opens its pass from a
local outbound Commit with no fork, so no carrier ever started. On Android the
group would have sat in `Recovering` behind its own `Disbanding` gate forever:
nothing sendable, never ending. The CLI never showed it because the harness
drives settlement explicitly. `disbandGroup` and the regeneration path now
start the carrier themselves.
**The gate was not durable anywhere real.** Gate storage is defaulted on
`MarmotPublishObligationStore` so existing stores keep compiling, and neither
`AndroidPublishObligationStore` nor `FilePublishObligationStore` overrode it —
so the previous commit's durability claim held only for the in-memory store the
test used. Both now persist gates: one file per group beside the obligations.
Android writes them unencrypted, unlike an obligation, because the value is one
enum name and the filename is a group id the device already stores in the clear
— no key material, no message content. `FileStoresGateTest` pins the round trip
on the real file store, including that a gate write is not mistaken for an
obligation on reload.
**The UI announced an ending that may not have happened.** The toast said
"Group disbanded" as soon as the call returned, which used to be true because
an unacknowledged publish threw. It no longer throws — the request stays
durable and pending — so `disbandMarmotGroup` now returns whether the group is
terminal, and the screen says "Ending the group" when it is not. Leaving the
screen is right either way: the group takes no further outbound work.
Verified: quartz 4836, commons 1886, cli 53 green, and the full MDK interop
harness is 29/29 at the new 0.9.21 pin with all of this built in — including
test 05, whose earlier failure was the loopback relay dropping a websocket
before OK rather than anything in 0.9.21.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
**The disband was terminalizing on application.** `group-lifecycle-v1.md`
("Convergence and realization") says a valid disband Commit is never
terminalized through ordinary linear advancement: admitting one moves the
lifecycle to `Recovering` EVEN WITH NO DIVERGENT EDGE, and only a SELECTED
disband Commit moves it to `Disbanded`. We went straight to `Disbanded` in
`recordApplied`, so a disband that lost a branch race had already destroyed the
group locally — and `Disbanded` is absorbing, so that client stops processing
group traffic and can never learn the branch it lost was the one everyone else
kept.
Most of the machinery for this was already written and never wired:
`ConvergencePass.markDisbandCandidateAdmitted`, `isRecovery`, and
`LocalOutboundGate.DISBANDING` all existed with no callers outside one unit
test.
- `recordApplied` now admits the disband for selection instead of
terminalizing; `settle` stays the only path to `Disbanded`.
- The pass opens WITHOUT `markForkDetected` — the spec is explicit that the
forced transition does not assert a fork.
- `settleUncontested` resolves a pass with no divergent material. Every pass
used to be opened BY a divergent commit, so `freezeInputs` could assume one
existed; with none it returns null and `settle` bailed WITHOUT clearing the
pass, leaving it open forever and spinning every caller polling for
settlement. A no-fork disband is exactly that shape.
**The request is now durable.** The `Disbanding` gate goes up first and is
persisted (gate storage added to the obligation store as default methods, so
existing stores keep compiling), because it has to outlive a publish no relay
acknowledged, a crash, a restart and a losing branch. An unacknowledged publish
no longer throws the intent away, and `requireOutboundAllowed` honours the gate,
so a group with a pending disband refuses new messages instead of carrying on as
if nothing had been asked.
**Regeneration is bounded to one attempt per epoch.** `resolveDisbandRequest`
runs at settlement and regenerates against the selected state when an active
branch won — but regenerating opens a fresh pass, and settling that pass calls
back in, so without the bound the two spin against each other forever. (MDK
bounds the same loop with `DisbandRequest.last_prepared_epoch`.) Waiting for a
new epoch is also right on the merits: a commit authenticating against the same
parent that just lost would lose again.
**MDK pin → 0.9.21 (`fdd398a8`).** The two shipping apps have diverged —
android is on 0.9.21, ios still on 0.9.20 — so the comment claiming they agree
was false. The rule is now written down: take the newer, because that is where
new validation lands and a client satisfying it satisfies the older one.
Also renames three shared-source test functions that contained a comma.
Kotlin/Native rejects those outright, which is why `test-quartz-linux-native`
and `test-quartz-ios` failed while every JVM run passed — the pre-push hook runs
JVM tasks only, with the native ones disabled on this host.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`lint` fails on the branch: `compose_escaping_check` flags one `\'` in the
Compose resource catalog. Android's parser resolves that escape; Compose's does
not, so the string rendered with a literal backslash in it.
Repaired with the repo's own tool, as the check instructs:
python3 tools/strings-migrate/fix_escapes.py --no-unwrap-quotes \
commons/src/commonMain/composeResources
`marmot_retention_footer` was the only entry affected. Both lint hooks pass now,
which unblocks the five jobs the workflow gates behind them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Adds the three harness directions the reference CLI can actually drive, and
the implementation each one needed.
**Deletions wn->amy (test 27).** We sent kind:5 and MDK applied it (test 23),
but nothing on our side applied an inbound one — a message its sender believed
was gone stayed on screen. `MarmotManager.deletedIds` mirrors `editOverlays`
with the same account-identity rule MDK uses for a self-retraction; `amy marmot
message list` reports `deleted` and blanks the body. Android already applied
these through LocalCache's NIP-09 path, which enforces the same author check.
MDK also honours an admin *moderation* delete carrying an authenticated grant
frozen at ingest. We issue no such grant, so a cross-author delete is ignored
rather than guessed at.
**Retention wn->amy (test 28).** Test 26 proved MDK accepts a group requiring
0x8005 with our bytes; the read side was untested, and that is where the
epoch-pinning rule lives — a message keeps the retention of the epoch that
DELIVERED it. Adds `MarmotManager.setMessageRetention` (the component's
explicitly-allowed mid-life update, admin-only), `amy marmot group
set-retention`, and a pinned `expires_at` on every message row. The test has
wn send under one policy, amy re-time the group, wn send again, and asserts the
two messages carry different pinned expiries.
**Disband amy->wn (test 29).** Our disband staged a bare lifecycle update.
`group-lifecycle-v1.md` fixes the whole Commit — the lifecycle update, an
admin-policy replacement naming only the committer, and a Remove for every
other leaf, all inline — and MDK rejects anything else as an unsupported
lifecycle transition, which would have left the group live for every member
while reading as ended here. `MlsGroupManager.stageDisband` now builds that
shape, `stageEnableDisbanding` covers a group predating the component, and
`amy marmot group disband --yes` drives it.
Also labels `MarmotIngestResult.Ignored` with the branch that produced it.
Four very different situations collapsed into one unlabelled result, which
made a client stuck in convergence indistinguishable from a quiet one — that
ambiguity cost most of the time spent diagnosing test 29.
Three directions stay uncovered because the reference CLI cannot originate
them: edits wn->amy (no `wn messages edit`, and kind 1009 is reserved against
`send-event`), setting retention from wn, and disband wn->amy. MDK's runtime
and uniffi surfaces expose all three; only its CLI does not. Documented in
cli/tests/README.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
NIP-52 requires a `D` tag on time-based calendar events -- the day-granularity
unix timestamp floor(unix_seconds / 86400), one per UTC day the start..end
range spans. Amethyst emitted none, so events it authored were invisible to
clients that discover calendar events by date (#D queries) rather than by
scanning every 31923 in existence.
CalendarTimeSlotEvent.build now emits the full set, which covers editing too:
the create/edit screen rebuilds the whole event through build(), and the
builder extension removes before it adds, so shortening an event drops the
days it no longer covers instead of leaving them claiming it forever.
end is exclusive per the spec, so an event finishing exactly at midnight does
not tag the following day. The range is capped so a mistyped end date cannot
produce an event too large to publish.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Hge2jR1BPnyZse75VQ4kg
The injected NIP-07 provider offered only getPublicKey/getRelays/signEvent, so
a page hosted in the nSite viewer or the in-app browser could sign but never
seal: a NIP-59 kind:13 seal is NIP-44 ciphertext authored by the real key, and
signEvent alone cannot produce one. That put NIP-17 DMs and every gift-wrapped
app protocol out of reach of any web app logging in with Amethyst.
Adds nostr.nip44Encrypt/nip44Decrypt to the broker, behind a new SIGNER
capability. SIGNER is website-only by construction: no NIP-5D domain maps to
it, so resolveRequiredCapabilities can never hand it to a locked napplet
however its manifest is written. Individual calls still pass the per-operation
signer ledger, reusing the vocabulary NIP-46 already uses for the same ops --
encrypt auto-allows under REASONABLE, decrypt always asks.
nip04 stays deliberately absent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Hge2jR1BPnyZse75VQ4kg
Every image Amethyst sent into a Marmot group was invisible to every other
implementation. The upload was encrypted with the MIP-era scheme and
described with a MIP-era `imeta` -- `url`, `x`, `n`, `v mip04-v2` -- and
MDK 0.9.21, which White Noise embeds, knows only `encrypted-media-v1|v2`:
`locator`, `ciphertext_sha256`, `plaintext_sha256`, `nonce`. Its typed
parser rejects anything else and the caller drops the tag without a word,
so the message arrived carrying no attachment at all.
The encoder for the adopted shape was already here and already correct.
What sent the old one was the gate: v2 was used only when the group
carried the `encrypted-media-v2` component, and groups are created
WITHOUT it on purpose, so epoch 0 matches the reference implementation
byte for byte. The default path was therefore always the dead dialect.
Receivers do not require that component -- MDK pins the opposite, that an
out-of-policy locator is "kept, not dropped on ingest", because media is
authenticated by its hashes and AEAD rather than by where it sits. Only a
SENDER's own outbound validation is constrained by policy. So the cipher
is now chosen unconditionally, and a media type too malformed to
canonicalize becomes `application/octet-stream` rather than falling back:
an attachment labelled imprecisely still renders, one in a dialect nobody
reads does not.
MIP-04 stays on the READ side for messages older builds already sent.
The test pins the tag we write against MDK's own fixture
(`crates/marmot-app/src/media/tests.rs`, `valid_v2_imeta_tag`) instead of
round-tripping through our own parser, which is what let this drift
through a suite that already claimed media interop: both ends drifted
together and agreed with each other.
Verified on device, Amethyst -> White Noise Android: the chat list shows
`Photo` (their `classify_chat_list_attachments` only says that once the
imeta parses) and the conversation renders the image.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three assumptions the harness makes are Linux-only, and each stops it
before a single test runs:
- it binds the relay to the URL host, and macOS has no 127.0.0.2 alias
("Can't assign requested address"). `RELAY_BIND` now separates the bind
address from the advertised host, so the relay can listen on 127.0.0.1
while the URL names something else.
- `wnd` refuses a socket path it considers too long, and rejects any path
containing a symlink or a directory it does not consider trusted-owned.
`B_SOCKET`/`C_SOCKET` are overridable now, so the sockets can live in a
short real directory while state stays in the repo.
Note for whoever runs this next: the `127.0.0.2` trick the comments
describe no longer buys anything. `RelayUrlNormalizer.isLocalHost` parses
the address now instead of matching literals, so the whole 127.0.0.0/8 is
stripped from the DM and KeyPackage relay lists -- `amy relay add
ws://127.0.0.2:8080` lands in nip65 only, and KeyPackage publishing falls
back to the public defaults. A host that resolves to loopback without
looking like it (lvh.me) survives our strip, but `wnd` rejects plain `ws://`
for a non-local host, so the two stacks currently admit no shared cleartext
relay url. That needs solving before this suite can pass end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"Use outbox relays" launched `saveKeyPackageRelayListFromOutbox()` into its
own coroutine and called `proceedWithCreate()` beside it, so the write the
creation depends on raced the creation itself. The loser was silent:
`isCreating` had already latched true, and the top bar gates on
`isActive = { !isCreating }`, so Create went inert for the rest of the
screen's life -- no group, no error, and only Cancel could leave.
`proceedWithCreate` now takes an optional `prepare` step that runs inside
the same coroutine and is awaited, which also puts a failure to save on the
same Toast path as a failure to create instead of dropping it in a
coroutine nobody reads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
No Amethyst account could mint a KeyPackage. Every attempt died in
`AccountIdentityProofV2.create` with "signer altered the account identity
proof event tags", so nothing was published to relays, no group could be
created, and the failure was invisible -- the exception surfaced no toast
and, with `VERBOSE_LOGS` off, no log line either.
The account signer is a `NostrSignerWithClientTag`, which appends the
NIP-89 client tag to everything it signs; the setting defaults to on, so
this was the ordinary path on device rather than an exotic one. The check
it tripped exists so a substituting external signer cannot authorize a key
the caller never asked to authorize, and it cannot tell a decorator's
addition apart from a hostile one -- correctly, since both rewrite the
bytes about to be hashed into an id.
So the proof is signed by the signer with that decorator peeled off, which
is what `withoutClientTag()` is for and what every other "these exact bytes
were requested" caller already does. A proof is a fixed statement about a
key rather than a post: it carries no client attribution, and a verifier
would have to strip the tag again anyway.
Verified on device: the app now publishes a framed kind:30443 (466 bytes,
`0001 0005 0001 0001`), White Noise Android finds it, and a group created
there reaches the app.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
encrypted-media-v2 was reachable in theory and unreachable in practice on
Android. `marmotUsesEncryptedMediaV2` keys off the group's `0x800b` policy, and
nothing in the app ever set one: createMarmotGroup does not pass it, and
setEncryptedMediaPolicy had no caller. Every group this app created therefore
fell back to MIP-04 forever, and the v2 code could only ever run in groups
created by amy or the reference client.
Leaving it off at creation is deliberate and stays that way —
CurrentProfileGroupFactory explains that carrying it at epoch 0 would make our
GroupContext differ from the reference's for identical inputs and would force
every joiner to advertise `0x800b` before it could be added. The spec's answer
is that "a group that wants a media policy commits one". This adds the thing
that commits one.
Admin-only, current-profile-only, and offered only while the group lacks the
component. Enable-only: changing the policy later is the same commit, but
REMOVING it is a question the component does not answer, and inventing a
removal that strands members mid-upload is not something to guess at. The
explainer says plainly that every member sees the change and it cannot be
undone.
The endpoints come from the account's own Blossom server list rather than a
constant, because a policy naming servers the uploader does not use would
describe a group nobody can actually post media to; with none configured it
refuses and says where to add one.
MarmotGroupChatroom gains hasEncryptedMediaPolicy so the action disappears once
it has been taken, populated in syncMetadataTo alongside isCurrentProfile.
Verified against the reference implementation, not just our own tests: the
headless MDK interop harness passes 26/26, including media-v2 in both
directions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The people search is one NIP-50 string against kind 0, so every other
token in the box is dropped without a word the moment the scope includes
People: `#bitcoin since:2025-01-01` came back with people named "bitcoin"
and no hint that the date had been thrown away.
Only `kind:` admitted this, which left the other fourteen fields keeping
the toggle enabled while doing nothing. `isEventOnly` becomes
`pinsToNotes`, and the rule is now the widest one that cannot lie: free
text and its operators (`OR`, `-exclude`, quoted phrases) leave the
toggle alone, and anything else -- authors, `from:`, kinds, pseudo-kinds,
`since:`/`until:`, hashtags, `lang:`, `domain:`, `to:`, `label:`, NIP-73
scopes, `group:` -- greys All and People.
A few of those (`domain:` against a nip05, an npub `from:`) could be
answered by a people search built to; none are today, and offering a
scope that ignores half the box is worse than not offering it.
The screen needed no wiring: `enabled = !pinnedToNotes || s == NOTES`
already covered both halves. The scope stays derived rather than written
back over the reader's pick, so deleting the chip gives back the scope
they chose -- verified on device along with each case in the table.
Tests take one case per pinning field, so a field added to SearchQuery
and forgotten there fails rather than silently doing nothing again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twenty commits, one conflict, in the file both sides had reason to touch.
Main pinned kind 11998 — the DVM heartbeat — into the golden test's KINDS
list as a `<not searchable>` row, so that a heartbeat quietly gaining
indexable content would show as a diff. This branch had already replaced
that hand-kept list with `SearchableKinds.ALL`, which holds only kinds
that are searchable, so the pinned row has nowhere to live and is
dropped.
Nothing is lost by that, which is the whole reason to resolve it this
way: `SearchableKindsTest` builds every kind from 0 to 65535 through
`EventFactory` and asserts the recorded set is exactly what came back
searchable. A heartbeat that started indexing text fails there, by
number, without anyone having remembered to list it — the same guarantee
the pinned row gave for one kind, for all of them. The KDoc says so, so
the next person doesn't re-add a row the list cannot hold.
Main's other change to that file is kept as-is: the golden comparison
normalizes CRLF, and `.gitattributes` forces LF on `*.golden`, so a
Windows checkout can't fail the suite on invisible line endings.
Everything else merged clean. The heartbeat kind is not searchable, so
`SearchableKinds`, `RenderableKinds` and the `kind:` vocabulary are
unaffected — verified by running their drift tests against merged main
rather than by reading the diff.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
SSRF guard walked past by IPv4 shorthand. isSafeToContact only recognised a
literal when the host split into exactly four integer parts, so `127.1`,
`2130706433`, `0x7f.0.0.1` and `0177.0.0.1` all fell through to "safe" — every
one of them loopback to a resolver, which uses inet_aton and does not need four
parts. It gates the real avatar fetch in MarmotGroupIconDisplay, so a group
admin could make every member's device probe its own network: exactly what the
function's own comment says it prevents. Now any notation is packed to its
32-bit value and judged once, and a numeric-looking host that cannot be
evaluated is refused rather than allowed — "we could not tell" must not mean
"go ahead".
encrypted-media-v2 was write-only. The send path existed; nothing rendered it.
The v2 imeta carries `locator <kind> <value>` pairs and no `url`, because the
same ciphertext may live in several places and none is privileged — but
IMetaTag.parse anchors on `url` and returns null without one, so hasMip04Media
was false and a v2 attachment drew as its caption with the image missing and no
error. Adds a v2 branch that parses the tag directly, plus the receiving
constructor EncryptedMediaV2Cipher lacked: the existing one populates the nonce
and plaintext hash from encrypt(), so it could only ever decrypt what the same
instance had just encrypted — the sender's case and nobody else's.
A confirmed publish RETRY forked the group against itself. commitAndPublish
marks the message processed, records the local commit, pins retention and syncs
system rows; retryPendingPublishObligations installed the state and did none of
it. The relay's echo of our own republished commit was therefore admitted as an
unknown kind:445 at an epoch already merged, opening a convergence pass against
ourselves — a restart could put a healthy group into Recovering by succeeding.
The framed commit is recovered by reopening the stored event with the
pre-commit exporter secret that priorState derives, so no persisted record
had to change shape to carry bytes it already implies.
The convergence settler could drop its carrier. Between the loop finding
nothing left and the finally clearing the flag, a new pass's
startConvergenceSettler() lost the compareAndSet and returned, leaving an open
pass with nobody polling it until unrelated traffic arrived. The carrier now
re-checks after releasing the flag and reclaims it if work appeared in the gap.
And a wasted write: persistGroup re-encoded and rewrote the whole retained-epoch
window on every call, including every application message — one TlsWriter and
one array per retained epoch plus a store write, for bytes identical to the ones
already there. The window only moves when an epoch advances, so it is now gated
on a revision counter. The group state itself still persists on every send; that
is what keeps the sender's ratchet generation durable, and nothing here changes
it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The spinner could only ever say "still going", and it said it off a stopwatch:
`settled` means "1200ms since you last typed", not "the relays answered". So a
search that hung looked the same as one that had finished, and neither said why.
The sub-assemblers knew all along. They build the per-relay filters and they are
handed every EOSE — the information was reaching `EOSEByKey` for `since`
bookkeeping and going no further. `SearchQueryState` now records both halves:
- `asked`, written as the filters are built
- `answered`, written from `newEose`
Additive and keyed by the query, because the posts and the people assemblers run
off the same state; a call that replaced the set would leave whichever ran second
looking like the only one that asked anything. `newEose` had to become `open` for
a subclass to see it.
Tapping the spinner now lists the relays outstanding, with the ones already in
below them dimmed, under "Waiting on 2 of 5 relays".
The spinner itself changed with it. The timer is now only the floor — something
has to show in the moment before any relay can reply — and after that the real
signal takes over: it turns while a relay still owes an EOSE. Otherwise it would
vanish 1.2s in and the popup would be untappable, which is how I found this
worth doing.
A relay that never sends EOSE would spin forever, so there is a 12s ceiling on
how long the spinner will admit to waiting. Past it the spinner stops and the
list still names who never answered.
Verified on device: "Waiting on 2 of 5 relays", antiprimal.net and
relay.ditto.pub outstanding, nostr.wine / relay.noswhere.com / search.nos.today
dimmed as answered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two commits landed on the branch while this audit was running, and one of
them is a crash the audit missed: `searchDataSourceState` had ended up
below the eagerly-shared collectors that call `updateDataSource`, so
opening search from anywhere died on a null field. I found the same
hazard on `listState` and fixed that one, then failed to check what else
that function touches. Their version keeps the field order and states the
rule as a rule; taken as-is.
The spinner was found and fixed twice, independently. Theirs is kept: the
mirror is driven from an `init` block rather than an eagerly-shared
collector, which is one less field whose declaration order matters in a
class that has now been bitten by that twice.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Four bugs and four costs, none of which any test was watching for.
**The spinner never stopped.** `isRefreshing` was
`derivedStateOf { searchValue.isNotBlank() && !state.settled.value }`.
`derivedStateOf` invalidates on snapshot reads and a `StateFlow`'s
`value` is a plain field, so it computed the right answer once and never
recomputed until the text changed — the spinner appeared on the first
keystroke and stayed up for as long as there was anything in the box.
`settled` is mirrored into Compose state now, declared above the
collector that writes it for the same reason `listState` is.
**A search remembered before disk answered was thrown away.** The
restore assigned over whatever had been remembered in the window between
the screen opening and the file being parsed — in memory and then on
disk, because the next write persists what survived. It merges now, and
writes back when the merge changed anything.
**Two saved searches could share an id.** The id was the timestamp plus
the list's size, so two saves in the same second — or one after a delete
— collided, and an id is what `forget` deletes by and what the restore
de-duplicates by. Found by the restore test, which is how it came to have
one of its own.
**A stored history with duplicate rows crashed the list.** Two entries
that serialize alike are two rows with the same key. De-duplicated on
read, not just on write.
Costs, in the order they matter:
- The cache-driven rescan walked the whole cache for traffic that could
never appear in search. A running app takes in chat, DMs, reactions and
zap receipts continuously; each bundle cost up to four full scans. They
are dropped before the sample now, on the kinds the result lists are
actually built from.
- Three parses per keystroke: the two debounced views re-parsed the text
the immediate one had already parsed. They are debounces of the parse
now, so one keystroke is one parse and the same `SearchInput` reaches
all three.
- `KindRegistry.tokenize` re-sorted the alias map on every call, and it
is called for every chip on every recomposition and for every
serialization of a query. Sorted once.
- Desktop re-ranked its whole result list on every keystroke, against a
query no relay had been asked yet. It ranks against the debounced query
— the one the results were fetched for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
`isRefreshing` read `state.settled.value` inside `derivedStateOf`. A StateFlow
read is invisible to the snapshot system, so the derivation re-ran only when
`searchValue` changed -- the one snapshot input it had. The spinner therefore lit
on the first keystroke and stayed lit for as long as the box held text, long
after the results were on screen.
Observed: type a query, wait forty seconds with results fully rendered, and the
arc is still turning.
The same flow already has a correct reader two hundred lines down, where the
empty state collects it with `collectAsStateWithLifecycle`. That is why "nothing
found" timed out properly while the spinner did not -- one consumer observed the
flow, the other sampled it once and never looked again.
Mirrored into snapshot state in the view model so both of `isRefreshing`'s
inputs invalidate it. Verified on device: spinner present a few seconds in, gone
by twenty-eight, where before it ran past forty.
Worth noting what this does not fix: `settled` is a timer, not knowledge. It says
"long enough since the query changed", not "the relays answered" -- there is no
EOSE on this path. The spinner is now honest about the heuristic it has, not
about the search.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Opening search crashed again, one field along from last time:
NullPointerException: SearchQueryState.getSearchQuery()
on a null object reference
at SearchBarViewModel.updateDataSource(SearchBarViewModel.kt:504)
at SearchBarViewModel$searchTerm$2.invoke(SearchBarViewModel.kt:162)
Same mechanism as the `listState` crash: `searchTerm` and `sourceWatcher` are
shared `Eagerly`, so they run `updateDataSource` inside the constructor, and the
pipeline refactor left `searchDataSourceState` declared below them. Kotlin
initialises in declaration order, so it was still null.
Worse than the first one, though. `listState` is only touched on the non-blank
branch, so an empty box survived; `searchDataSourceState` is touched on *both*
branches, so this crashed opening search from anywhere -- seeded or not, Home
included.
Moved above the collectors, next to `listState`, and the comment there is now a
rule rather than a note about one field: everything `updateDataSource` touches
is declared above that line. Two occurrences in two passes over this class say
the ordering is not obvious from reading it, so it is worth stating plainly for
whoever tidies these fields next.
Verified on device both ways: search from Home (unseeded) and from Reads
(seeded, `kind:article`) — no crash on either, and the seed still chips.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
vocabulary that can answer
Three smells from the audit, and the first one was a bug.
**Results stopped at the first scan.** Every result flow combined the
debounced query with an invalidation counter that only the lifecycle
touched. So a search ran once, ~100ms after the last keystroke, against
whatever happened to be cached — and the events its own REQ brought back
never appeared, because nothing re-ran the scan. Typing another character
or leaving and returning to the screen was the only way to see what the
relays had sent. The cache has published what it takes in all along
(`ICacheEventStream`); search just never listened. Now it does.
Sampled rather than debounced, and the distinction is the whole of it: a
debounce waits for quiet, and a cache taking in a search's own results
plus everything else the app subscribes to may not go quiet for seconds
— the list would have stalled exactly when it had the most to show.
Sampling caps the cost at one rescan per 400ms and guarantees progress
while events are still arriving. Lifecycle refreshes stay immediate.
**`isRefreshing` meant "the box has text".** Every other implementor of
`InvalidatableContent` uses it for "a refresh is running", and three call
sites here read a name that said one thing and meant another. Both
meanings now have their own names, and the honest one earns its keep:
the field shows a spinner beside the clear button while a search is still
under way. Android had no way of saying results were still coming, so a
half-filled list looked like the answer — desktop has had per-relay
progress since its bar was written.
**Two `kind:` names could never return anything.** `kind:repost` and
`kind:profile` both drew a chip and narrowed the query to a kind the
result scan drops. `kind:profile` was the worse of the two: it also
pinned the scope to Notes, taking away the People scope, which is the
only place a profile is ever found.
The chain of twelve `is` checks that dropped them was in `CacheSearch`,
where the vocabulary could not see it. It is now
`RenderableKinds.NEVER_IN_RESULTS` in commons, `CacheSearch` reads it,
and a test fails if any alias names one. Three more tests pin the rest of
the table: every alias writes itself back as its own name, every alias
names kinds `EventFactory` can build, and the fifteen renderable kinds
with no name of their own are listed so adding a kind is a decision about
whether it earns a word.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Nothing in the Android Marmot UI knew what a legacy MIP-01 group was —
grepping isCurrentProfile across the whole surface returned nothing. So an
admin of a group created before the current profile saw the Disband button and
the avatar-URL field, used them, and got a toast: those components have no
carrier in a legacy group, and MarmotManager refuses before building the
commit.
Surface it as state rather than as an error. MarmotGroupChatroom gains
isCurrentProfile, populated in syncMetadataTo from the group view, and the two
screens read it:
- Disband is hidden unless the viewer is an admin AND the group can express a
lifecycle at all. An admin of a legacy group gets a line saying why, because
a silently missing action is the more confusing outcome — the surprising part
is that a NEW group would have it.
- The avatar-URL field is replaced by its reason instead of shown and then
rejected on save. The uploaded Blossom image still works there, which makes
this a missing option rather than a missing feature. The save call is guarded
too, so a later edit to the form cannot turn a hidden field back into a
refused commit.
Both strings say the group cannot be upgraded and a new one is the route. That
is the part a user cannot infer: the identity proof lives in each member's own
LeafNode and covers that leaf's signature key, so it cannot be added to leaves
that already exist.
isCurrentProfile defaults to TRUE on purpose. Creating a group does not run a
metadata sync, and every group created now is current-profile, so a
just-created room must show its full feature set immediately; a restored legacy
room is corrected by the startup sync, which runs for every group before any
screen reads it.
setEncryptedMediaPolicy has the same legacy refusal but no Android UI, so
disband and the avatar URL were the only two exposed paths.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The staged-proposal pool moved STATE_VERSION from 3 to 4, and nothing covered
the case that decides what happens to every group already on disk: a blob the
PREVIOUS build wrote being read by this one. There was a v1 test, from an older
bump, and then nothing.
v4 only appends, so for a group with nothing staged the new section is exactly
one uint32 zero. That makes a genuine v3 blob obtainable by stripping those four
bytes and moving the version word back — byte-for-byte what the previous build
would have written, rather than a re-implementation of the old encoder that
could drift from it. The test asserts the tail really is a zero count before
relying on that.
Asserts the blob parses, invents no staged proposals, keeps its epoch, and is a
WORKING group afterwards: it encrypts and decrypts, and its exporter secret is
unchanged. A parse that succeeds but derives different secrets would be the
worse failure, and would look like a passing test without that last check.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Groups created before the current profile existed are still on disk, and they
can never become current-profile groups: the account identity proof lives in a
member's own LeafNode and covers that leaf's signature key, so it cannot be
added to leaves that already exist. AccountIdentityProofV2 states it outright —
"There is no fallback and no in-place migration". The only route from a legacy
room to a current-profile one is a new group and a re-invite.
That makes the legacy contract worth pinning rather than rediscovering by hand
each time someone tests an upgraded install. Messaging still works; disband
(0x800c) and the URL avatar (0x8007) are GroupContext components a legacy group
has nowhere to put, and both refuse up front with a message naming the reason
rather than failing somewhere inside the commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The claim this whole pass rests on is that the same query reaches a relay
as the same REQ from either front end. Nothing checked it — every bug it
started from was a violation of it, and each one was invisible because
the two answers were never compared.
`desktopApp` is the only module that can see both paths, so the test
lives there: 16 query shapes, each asked through Android's
`searchPostsByText` and Desktop's `SearchFilterFactory`, compared arm by
arm on kinds, authors, tags, search, since and until.
One assumption of mine was wrong and the test said so: a bare `from:`
does build filters on both sides. It is bounded by its author, so unlike
a bare `kind:` it is a real search rather than an unbounded feed.
The plan is updated to what shipped, including the two findings left for
someone else: twelve event classes missing from `EventFactory`, and the
searchable-kinds reference table now nine kinds behind the code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
MDK's prepare_app_send builds its group with create_request(vec![]), so its
sender is alone in the group. Now that our send row is parameterised by member
count, the row that lines up is send_app_message/0 — naming it explicitly keeps
the comparison matched instead of leaving a reader to assume the unparameterised
number still applies. Ratio goes 6.5x to 7.1x faster on the matched pair.
Also records why ingest_app_message has no MDK column: their bench binary panics
in bench_deferred_outbound_preflight_matrix before reaching
bench_app_message_ingest, so there is no reference number — not one we declined
to use.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Desktop has had a search history since the advanced bar was written.
Android had none, for no reason other than where the code sat — and an
empty search box on Android rendered nothing at all, which is the one
place a reader's last searches are worth offering.
The behaviour came over with `SearchHistory`; what is added here is
Android's half. `DataStoreSearchHistoryStorage` is the file the two
strings live in — device-global like the drawer's collapse state, since
what you searched for is a property of this phone and is never published
to a relay. The empty box now lists the recent searches, each one tap
from being re-run: an entry is stored as the same text the box holds, so
putting it back in the field is the whole of loading it.
Recorded on the Enter key rather than on every pause in typing. The field
already had an `onSubmit` slot that nothing on Android was passing;
recording every settled query would fill the list with the prefixes of
the word being typed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Two gaps, both of which turned out to be hiding something.
**The message path ran on groups of one and two.** Latency is genuinely flat
across group sizes — an application message is sealed under the sender's own
ratchet and never touches the tree — so the head-to-head claim against MDK
survives being parameterised. Allocation is not flat on the send side: 68.7 KB
at zero members to 253.1 KB at 32, while the receive side moves 61.5 to
67.4 KB. The asymmetry is exact rather than mysterious. MlsGroupManager.encrypt
calls persistGroup unconditionally, serialising the whole group state on every
message sent; decrypt calls it only when the message was a Commit that advanced
the epoch, so decrypting an application message persists nothing. Sending in a
32-member group therefore spends a full state serialisation to record what
amounts to a generation-counter bump. The write is necessary — a sender
generation reused after a crash is a nonce-reuse-class problem — but writing
all of the state for it is heavier than the invariant needs. Recorded as a
finding, not changed: send-path persistence is security-sensitive.
**Commit ingest was measured by nobody**, here or in MDK, despite being the
operation every member performs on every membership or settings change and the
only one whose cost is meant to scale with the group. It grows 1 950 -> 2 367
-> 3 009 us from 1 to 32 members: 1.5x for a 32x bigger group, which is the
log2 shape MLS predicts, since the UpdatePath carries one node per LEVEL of the
tree. Allocation grows 5.7x to 1.4 MB, making it the largest single allocator
in the suite and the row to watch on a phone.
Two measurement lessons are written into the README rather than left implicit.
An isolated --only=ingest_commit run reported no trend at all and put the
one-member case slowest; that was JIT warm-up on the shared group builder,
and the full-suite numbers are the trustworthy ones. And create_group/32 is
now reported as a range (77.5 - 195.7 ms) instead of a figure: five runs
produced four within 11% and one more than twice the rest, which is what the
fewest-iterations row on a shared vCPU looks like.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The cap, the most-recent-first ordering, the de-duplication and the
encoding were all written into a desktop `object` bolted to
`java.util.prefs`. None of it is desktop-specific — it is a capped list
and a labelled list — and that placement is the entire reason Android has
never had a search history and the entire reason none of this was tested.
`SearchHistory` in commons owns the behaviour; `SearchHistoryStorage` is
the two-method seam for where the bytes land. `SearchHistoryStore` keeps
its API and becomes the `Preferences` half, so Desktop is unchanged and
an existing stored history keeps loading.
Two bugs the tests found on arrival:
A saved search whose label contained a tab lost the query it named — the
label is the one free-text field and the tab is the field separator, so
everything after it shifted by one. Labels are now escaped, and only for
the two separators and the backslash, so an already-stored label that
contains none of them encodes to exactly itself.
Re-running a query typed in a different token order made a second history
entry. The comparison is on the serialized form now, so two queries that
mean the same thing are one entry however they were typed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
`AdvancedSearchBarState` kept its own text, parse, debounce and sort
orders. It now delegates all of that to `SearchState` and keeps what is
genuinely Desktop's: relay callbacks, raw `Event` results, per-relay sync
status, and the expanded form panel.
Two things the shared holder fixes on the way in:
Relevance ranked against the whole box. `sortEvents(notes, order,
rawText)` scored the literal text of the query's own tokens, so a search
with a `from:` or a `kind:` in it ranked on noise — Android had been
fixed to score the leftover terms, Desktop had not. Going through
`SearchPipeline.rank` fixes it and gives Desktop the stable tiebreak too.
`ChangeSource` is gone. It existed to decide whether the field should
show what was typed or the serialized query, because a form edit wrote to
the query while typing wrote to the text and the two could disagree.
`SearchState.edit` writes the token into the box instead, so a button
press and a typed token produce the same state — which is the rule the
whole chip language rests on.
`SearchResultSorter.sortEvents` had no callers left after that, so it is
deleted and its tests now exercise `SearchPipeline.rank`. Two orderings
of the same list, one of them without a tiebreak, was the duplication
this whole pass exists to remove.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
marmotBench did not compile after the founding-add change — it was outside the
modules I rebuilt when the commit event became nullable, so two call sites in
it still dereferenced. Both are the same shape as the ones already fixed
elsewhere: the ingest_app_message setup re-ingested its own commit as an echo,
and the epoch probe printed the commit's kind. Neither has a commit to speak of
now, and the probe says so:
after createCurrentProfileGroup: epoch=0
after addMember: epoch=1
commits published by addMember: 0 (founding add, merged locally)
bob after joining: epoch=1
The measured effect is smaller than "we removed a whole published event"
suggests, and the README now says so precisely. Allocation per operation:
create_group/0 unchanged at 81.3 KB — with no invitees there is no founding Add
to skip — then -11% at one invitee (543.6 -> 482.5 KB), -6% at eight, and -1.4%
at 32. The absolute saving grows with the invitee count because the commit that
is no longer built carries N Adds; the fraction shrinks because the rest of the
operation grows faster.
Latency moved within run-to-run noise, so no latency claim is made for it. The
reason to want the change is that creating a group with initial members no
longer depends on a relay acknowledgement the spec never asked for.
Full table refreshed from two runs: against MDK, create_group/1 is 1.5-1.6x
slower, create_group/8 1.7x, create_group/32 2.6x, while join_welcome is 2.5x
and send_app_message 6.5x FASTER.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`protocol-core/publish-lifecycle.md`: "When founding creation includes initial
invitees, the creator next prepares and locally merges one founding Add Commit
from epoch 0 to epoch 1. That Commit also has an empty group-message
publication obligation: the creator is the only pre-existing member, so no peer
can be forked by failure to publish it." And: "The empty-obligation exception
is limited to the epoch-0 creation and, when applicable, its immediately
following founding Add Commit."
We implemented the first half and missed the second. createGroup already
satisfied an empty obligation for epoch 0, but the Add that follows went down
the ordinary commitAndPublish path, which applies a commit only once a relay
acknowledges it. So creating a group with initial members against an
unreachable relay silently produced an empty epoch-0 group — the members were
never added and the Welcomes were never sent — where the spec makes the Add
canonical immediately and each Welcome an independent retryable delivery that
"does not affect canonical group state".
addMemberInvites now detects the founding case (epoch 0, creator the sole
member), merges the Add locally, and returns a null commit event with the
Welcomes. Nothing is published, which also stops spending a signature and an
outer encryption on bytes with no audience, and stops leaving a kind:445 on
relays that a joiner can receive before its Welcome — the reference calls that
a "welcome-before-commit AlreadyAtEpoch bounce" and drops the commit for the
same reason.
The commit event is nullable rather than absent so every call site had to be
looked at: the CLI reports founding_local_merge and publishes nothing, and the
Android action logs the case instead of dereferencing.
Test fallout was all one shape — suites that used create + addMember as SETUP
were testing the exception rather than the rule. MarmotPublishBeforeApplyTest
and MarmotPublishDurabilityTest now get past the founding add first, and gain
direct coverage that a founding add merges even when the publisher rejects
everything, and that the very next commit is ordinary.
publish-fail/v1 is refused rather than passed. It fails the FOUNDING creation's
outbound and expects epoch 0 with one member, which is the legacy lifecycle:
MDK resolves the profile from application_profile and `None | Some("legacy")`
means legacy, where create_group returns GroupCreated { pending }. Our old
behaviour happened to match that. Weakening the fix to keep the vector green
would reinstate the bug, so the runner refuses it by name via
LegacyOnlyScenario and the test asserts the refusal. invite-publish-fail/v1
still runs in full: a rejected later invite is an ordinary commit either way.
Also corrects this file's own README claim that create_group/N costs "one more
commit" than MDK. Both do exactly one; only the publishing differed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Both front ends had grown their own copy of the same state — the text,
the parse, the debounce, the scope, the sort orders, "is it fair to say
nothing was found yet" — and the copies disagreed on every one of them.
Android parsed the same string about nine times per keystroke across four
debounce windows; Desktop parsed once and had no scope at all. Neither
difference was a decision.
`SearchState` in commons owns all of it. `SearchBarViewModel` keeps only
what a shared holder cannot: a LazyListState, a FocusRequester, invite
routing, NIP-05 resolution, and the seven result flows — the acquisition
of results, which is a cache scan here and a relay callback on Desktop.
615 lines down to 508.
Three things fall out of it:
`SearchInput` carries the text and its parse as one value. Most callers
want the query; a couple genuinely want the characters (a relay finder
matching wss://, an id lookup deciding whether the box holds a pointer or
a phrase), and reading those from a separately debounced flow lets a
collector pair one keystroke's text with another's parse. It also carries
`nameTerms`, which was the most-repeated parse of all — every people and
channel finder re-parsed the whole box to ask for it.
Two debounce windows instead of ten, each named for what it protects:
100ms before a cache scan, 300ms before a REQ, because a REQ opens a
subscription on every search relay and withdrawing it a keystroke later
is traffic nobody wanted. Eight collectors had been declaring the 100
separately, which is not eight policies but one policy that could drift.
`edit {}` writes a filter into the box as text rather than holding it
beside the text. Desktop carried a `ChangeSource` flag to decide whether
to show what was typed or the serialized query; with the box
authoritative there is nothing to decide, and a chip stays something the
reader can delete by editing the words.
One fragility fixed on the way: `listState` is now declared above every
eagerly-shared collector that scrolls it. It was above the only one that
did, and `debouncedForRelays` — a StateFlow with a seeded value, where
the old flow waited out 300ms first — added another.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
The golden test's KINDS list was missing the new heartbeat kind, and the
comparison read the golden raw — on Windows CI (autocrlf checkout) every
internal CRLF counted as a diff, failing the suite with a full-table
ComparisonFailure. Adds 11998 as a pinned <not searchable> row (a heartbeat
is a machine signal: no indexable content, per the NIP-50 eligibility
policy), normalizes CRLF in the comparison, and forces LF for *.golden in
.gitattributes so checkouts can never break it. No reindex needed: nothing
is indexed for this kind.
The README claimed create_group/N "includes one more commit on our side"
than MDK's founding creation. Measured, that is wrong: create leaves epoch 0
having published nothing, addMember produces exactly one kind:445, and the
invitee joins at epoch 1. MLS allows nothing else — RFC 9420 section 11
requires a group to be created with a single member, so MDK's founding
creation commits Adds internally too.
The difference is that we publish that founding Add Commit and MDK
deliberately does not. Its own comment: "we intentionally do NOT emit the
commit ... every other member lands in the group via welcomes, which carry
the post-commit state directly."
--epoch-probe prints the shape so the claim stays checkable rather than
remembered. The README correction and the behaviour change belong in their
own commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The All/People/Notes toggle was applied as seven separate
`if (scope == …) return emptyList()` lines, one written into each result
flow. Seven copies of a three-row table is how `ALL` came to mean
"everything" in six of them and something slightly different in the
seventh: the note flow let hashtags through under Notes, the channel
flows did not, and nothing said which was intended.
`SearchScope.shows(SearchResultKind)` is that table, once. A new result
kind now answers the question by appearing in the `when` rather than by
someone remembering to guard it, and the pinned test is written from the
old guards rather than from the new enum, so it records what the toggle
actually did on the day it was collapsed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
One X25519 scalar multiplication cost 541us, and create_group/1 is about two
dozen of them, so the curve primitive was not part of the gap against MDK — it
was the gap.
The cause was the representation rather than the language. Curve25519Field
used TweetNaCl's 16 limbs of radix 2^16, so a schoolbook multiply spent 256
limb products. SunEC's X25519 — also pure Java, same JIT, same machine — ran
the same operation in 160us on ~26-bit limbs in 10 words, which is 100
products. The ratio of products matched the ratio of times, which rules out
"managed language" as the explanation and names the fix.
Rewrite the field to 10 limbs of radix 2^25.5, the layout ref10,
curve25519-donna and SunEC all use. A multiply is 100 products, a square 55
(each off-diagonal pair once, doubled), and the ladder's a24 constant gets a
dedicated scalar multiply instead of a general one against nine zero limbs.
Limbs stay signed and denormalised between operations; only pack25519
produces a canonical value. Straight-line locals mean mulInto and sqrInto need
no scratch accumulator at all, so that parameter is gone from every caller.
x25519_dh 541us -> 121us ed25519_sign 1018us -> 259us
x25519_base 535us -> 121us ed25519_verify 2113us -> 536us
At 121us the scalar multiplication is faster than SunEC's 160us, which is the
sanity check on the result: it lands where a good managed implementation
should rather than somewhere suspiciously better. Against MDK, create_group/1
goes from 4.7x slower to 1.5x, join_welcome from 1.3x slower to 2.5x FASTER,
and send_app_message from 2.5x to 6.5x faster.
Re-encoding curve constants is where one mistyped limb yields code that runs
and is silently wrong, so none were transcribed by hand. Each was re-derived
from its existing encoding and checked against its mathematical definition:
d == -121665/121666, d2 == 2d, By == 4/5, I^2 == -1. The multiply and square
formulas were generated from the representation's weight bookkeeping and
diffed against an independent reference over 20000 random limb vectors before
any Kotlin was written; the carry chain and the canonical encoder were
validated the same way, including at p, p-1 and on non-canonical inputs.
RFC 7748, RFC 8032, HPKE, the MDK crypto-interop vectors and the full quartz
and commons suites all pass unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`SearchBarViewModel` asks `LocalCache.search` five questions, and
`LocalCache` is an `amethyst` singleton — which is the whole reason
desktop wrote a second search rather than reusing this one, and so the
whole reason the two drifted. The five now sit on `ICacheProvider`, the
port the commons migration sweep already routes cache access through, so
a state holder in commons can ask them without naming the module.
They default to returning nothing rather than being abstract. Desktop's
cache holds notes and live channels but has no public-chat or ephemeral
store, and a port that forced it to implement those would be asking it to
lie; a front end renders what its cache can answer and gains the rest
when its cache does.
`CacheSearch` now takes the mute list as `LiveHiddenUsers` rather than the
`HiddenUsersState` holder it lives in. The holder cannot be named from
commons, and the value is what was wanted anyway: `findNotesStartingWith`
was re-reading `.flow.value` five times down one scan, so a mute arriving
mid-scan could hide a note from one branch and not the next.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
A JFR profile of create_group put 75% of all execution samples in
car25519. Rewriting car25519 to drop its modulo and its branch then changed
nothing measurable — which is the profile telling on itself. JFR's execution
sampler is safepoint-biased and the counted loops in the field arithmetic
carry no safepoint polls, so samples land on whichever method follows the
poll rather than the one burning the time.
Time the primitives end to end instead. That needs no profiler to be
believed, and multiplying by how many of them an operation performs says how
much of it is curve work:
x25519_dh 579us ed25519_sign 1004us
x25519_base 582us ed25519_verify 2102us
create_group/0 is ~4.0ms, about seven scalar multiplications; create_group/1
is ~14ms, about twenty-four. The curve primitive is essentially the whole
cost, so that is the only place a create_group speedup can come from.
Square by symmetry: in a*a every off-diagonal pair is computed twice, so
taking each once and doubling turns 256 multiplications into 136. Worth a
measured 6.6% on the scalar multiplication (579us to 541us). Ed25519 is
unchanged, as extended-coordinate point addition contains no squarings.
The car25519 simplification is kept but explicitly claims no speedup: C2 was
already strength-reducing what it removes, and ART is the target that might
not. Its KDoc now records that honestly, and warns the next reader off
profiling this file.
Adds the primitive benchmarks and a --only= filter to marmotBench, which is
what made the profiler's story falsifiable in the first place.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
"Which kinds can a search return?" was answered in three places that
disagreed. Android's relay subscription listed 34 kinds; desktop's filter
factory listed its own copy, which had lost the calendar slots and code
snippets; the local cache scan had no kind window at all, so it matched
kinds no relay was ever asked for. Same query, three answers.
Quartz already knew the protocol half of it — SearchableEvent, 142
reachable kinds, referenced zero times by the client — but only as an
interface nothing could enumerate. So it gains SearchableKinds.ALL, and a
test that sweeps the whole 16-bit kind space through EventFactory and
asserts the recorded list is exactly what came back searchable. A kind
added to quartz now lands as a failing test naming the number.
Commons gains RenderableKinds: the 46 of those Amethyst has a card for,
plus three (pin lists, poll responses, NNS records) that match on content
alone and are named as such. Both front ends and the local scan read it.
The audit that produced it found fifteen searchable, renderable kinds the
search never asked for: pictures, all four video kinds, workouts, git
repos, sites, napplets, meeting spaces and rooms, calendar events and
software applications. Searching for your own pictures returned nothing.
Two findings on the way, from the sweep rather than from reading:
- quartz's indexable-content golden test pinned 126 kinds when 142 were
searchable, so seventeen — every video kind among them — had their
indexed text unpinned. It now reads SearchableKinds.ALL; the regenerated
golden adds those seventeen and drops 31890, which is not buildable.
- twelve event classes are absent from EventFactory, so their events parse
as a plain Event and nothing ever calls indexableContent(). One of them,
FeedDefinitionEvent, implements SearchableEvent. Left alone and written
down in the plan: a factory bug is not search's to fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Beat notes live in LocalCache.addressables (WeakReference values) with no
strong holder on the Discover screen: every GC sweep cleared them all at
once, the freshness gate failed for every DVM simultaneously, and the list
collapsed and rebuilt one beat at a time (worst on refresh — an allocation
spike). The detail screen survived only because its own composable held the
note.
DvmHeartbeatRegistry (Address -> latest createdAt, strong, one entry per
DVM) is now the freshness source: recorded on every beat consumption, read
by the gate and the liveness composables. Stale entries are inert; the map
is bounded by distinct DVM addresses.
An allocation profile of the Marmot benchmarks put 93% of every sampled
allocation in Curve25519Field.mul/add/sub. The pure-Kotlin field arithmetic
returned a fresh LongArray(16) from every operation, and a Montgomery ladder
runs ~18 of them per scalar bit across 255 bits, so one X25519 scalar
multiplication produced over a megabyte of garbage. Ed25519 was worse: its
extended-coordinate point addition needs ten temporaries and a scalar
multiplication calls it 512 times.
Give each field operation an *Into twin that writes into a caller-owned
output and shares one 31-limb accumulator, then rewrite both hot paths around
them. The X25519 ladder allocates its eleven-array working set once before the
loop and overwrites a/b/c/d in place after their last read; Ed25519 creates a
single PointAddScratch per scalar multiplication and reuses it for all 512
additions, including the aliasing doubling step. Every *Into is safe when the
output aliases an input, because mulInto fully accumulates into the scratch
before it touches the output.
The allocating functions stay. They are still used off the hot path, where
clarity is worth more than the bytes, and keeping them means the in-place
versions can be differentially tested against them.
Allocation per operation drops 10x to 80x depending on the benchmark
(create_group/0 6958.7 KB to 88.3 KB, ingest_app_message 5640.3 KB to
70.9 KB), reproducing to four significant figures across runs, and p50
latency improves on every row that is not dominated by measurement noise.
No behaviour changes: the RFC 7748 and RFC 8032 vector suites, the HPKE
tests and the full quartz suite pass unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Persisting the staged-proposal pool moved STATE_VERSION from 3 to 4, but
this assertion still pinned 3. It is the only test that reads the version
word directly, so nothing else caught it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Pull-to-refresh only re-read the cache (invalidateData), so when cached
beats were stale it showed the loss instead of fixing it. The DVM tab's
refresh now also invalidates the discovery assembler, re-issuing the 31990
REQ and both heartbeat streams (global + outbox batches) with fresh rolling
windows — beats repopulate in seconds instead of whenever the open REQs
happen to deliver.
Beats arrive every 300s, so the old 420s window tolerated barely one
delivery hiccup — live DVMs dropped in oscillations and flickered back on
the next beat. The gate now keeps a DVM visible through several missed
deliveries (relay reconnects, REQ churn) before dropping it; a dead DVM
still disappears after 15 minutes. Single constant, all surfaces consistent.
Every piece of a search already existed in commons and was already tested. What
did not exist was the *sequence* — each front end assembled parse → filters →
keep → rank by hand, and each forgot a different part.
That is not a coincidence to be fixed four times. Android never called the
post-filters, so `-term` and the pseudo-kinds did nothing; it never used the
relevance scorer, so Relevance sorted by date; it dropped the query's kinds on
the way to the filters, so a `kind:` chip narrowed nothing. None of those were
visible as a missing call, because there was nothing they were missing *from*.
SearchPipeline is that something. Both filter paths and the Android result path
now go through it, so the fixes are inherited by construction rather than by
remembering.
Generic over the item via two accessors — what event does this carry, what is
it worth — because Android holds `Note` and desktop holds `Event`, and that is
the entire difference between them. The zap accessor is a Double rather than
the BigDecimal a Note carries: quartz declares that type `expect` with no
Comparable and its JVM actual is java.math.BigDecimal, so a comparator over it
cannot be written in common code. A double is exact past any zap that will
exist.
Sort keys are snapshotted per item, which the Android feed order already did
and desktop did not need: a Note is a mutable box, a newer addressable event
arriving mid-sort moves createdAt under the comparator, and TimSort answers
that with "Comparison method violates its general contract!".
15 tests, one per bug that shipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Adds `marmotBench` — the quartz half of a head-to-head against MDK's
`cgka-engine --bench group_lifecycle`, case for case: create_group/N,
join_welcome, send_app_message, ingest_app_message. Both sides exclude
transport crypto, run over in-memory storage, and keep setup outside the
measured window, so what is compared is the engine's own CPU cost.
Every row also reports BYTES ALLOCATED PER OPERATION, from the JDK's own
`ThreadMXBean` — no dependency added. Latency alone cannot answer "are we
avoiding GC": a JVM can win a microbenchmark and still hand the user a
dropped frame later. The counter is per-thread, so benchmark bodies run
inline via `runBlocking` rather than on a dispatcher, where the allocation
would go uncounted.
First results, same host, nothing else running:
operation MDK quartz ratio
create_group/1 3.61 ms 16.93 ms 4.7x slower 27 MB/op
create_group/8 9.93 ms 51.47 ms 5.2x slower 95 MB/op
create_group/32 31.64 ms 190.08 ms 6.0x slower 333 MB/op
join_welcome 4.77 ms 6.22 ms 1.3x slower 10 MB/op
send_app_message 4.28 ms 1.72 ms 2.5x FASTER 2.8 MB/op
So the steady-state path a user actually exercises — sending a message — is
already faster than the reference. The gap is concentrated in key-agreement
work, and a JFR allocation profile says exactly where: 93% of all allocation
samples are `long[]` from `Curve25519Field.mul/add/sub`, which return a
freshly allocated field element on every single field operation inside
255-iteration scalar-multiplication loops. That one shape explains both the
5x latency gap and the MB-per-op allocation.
The fix (in-place field ops over caller-supplied scratch) is left as its own
change so it can be verified against the RFC vectors on its own merits.
Note: MDK's `ingest_app_message` has no number here. Their bench binary
panics in `bench_deferred_outbound_preflight_matrix` (an assertion on peeler
attempts) before reaching it, and criterion's filter does not skip that
bench's fixture construction. Reporting the gap rather than inventing a
comparison.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Two defects, both found by `leaver-removal-secrecy` and both isolated with a
failing test first. Between them a departing member stayed in the tree —
holding the group's keys and reading everything sent after they left — while
the group believed the departure had been processed.
A peer's proposal must be REFERENCED, not inlined
--------------------------------------------------
`commit()` inlined every staged proposal, including ones another member
authored. An inline proposal carries no sender, so a receiver attributes it to
the committer. For a `SelfRemove` that is not cosmetic: the proposal means
"remove my leaf", so inlining someone else's says "remove the COMMITTER's
leaf". Now only our own proposals go inline; a peer's goes in by
`ProposalOrRef.Reference`, which resolves against the receiver's own pool where
their copy of the same standalone proposal already sits with the original
proposer's leaf index. `PendingProposal.authenticatedContentBytes` documented
this contract all along; the code did not implement it.
A path-less commit must contribute a ZERO commit secret
--------------------------------------------------------
`commit()` derives path secrets unconditionally — it needs them to build the
UpdatePath when there is one — and then keyed the commit secret on whether
those secrets existed rather than on whether the path was actually SENT. A
SelfRemove-only commit omits the path (RFC 9420 §12.4.1), so every receiver
used the zero vector while the committer used a derived one: different epoch
secrets, and every witness rejected the commit with a confirmation-tag
mismatch and fell an epoch behind. The same branch also overwrote
`pathPrivateKeys` with keys that were never published, discarding the ones that
could still decrypt commits addressed to our ancestors.
The pool is an obligation, so it is durable
--------------------------------------------
`MlsGroupState` gains `pendingProposals` (STATE_VERSION 4; older blobs decode
with an empty pool), and staging a peer's standalone proposal now persists the
group the way an epoch change does — it only mutated memory before. Losing the
pool to a restart does not lose a message, it loses the obligation: nobody is
left holding the proposal that evicts the leaver.
That also makes `stageCommit`'s explicit hand-off of the pool redundant, so it
goes back to the shared `stage` helper and `adoptPendingProposals` is removed.
`leaver-removal-secrecy` now replays instead of asserting its own divergence.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The outbox fetcher read its announcement set from the gated DVM feed list —
a death spiral: a DVM dropped for a stale beat left the fetch batch, its
beats were never fetched again, and the drop became permanent. The fetcher
could only ever help DVMs that were already visible.
The assembler now takes cache-backed sources from the front end:
LocalCache.cachedDvmAnnouncements (every cached k=5300 announcement,
newest-first, capped at 100 — the gate's other eligibility checks, WITHOUT
the freshness gate), a per-author relay lookup unioning the NIP-65 outbox
with the cached relay hints (same mix as the event finder), and 31990/NIP-65
observation flows as re-issue drivers.
A proposal, not a change. Four bugs of one shape turned up during the search
feature work — a control live on one platform and dead on the other — and
fixing them individually has no end while the structure keeps producing them.
The finding that matters for scheduling is that this is not a laziness
problem. LocalCache, CacheSearch and Account live in `amethyst/`; Note, User
and the whole query layer live in `commons/`. Desktop could not reuse
Android's search, so it wrote a second one over relay callbacks alone. Every
divergence follows from that single module boundary.
Which means the commons migration sweep already plans the move that dissolves
it — LocalCache is its step 3, and deleting DesktopLocalCache is called the
largest duplication in the repo. So the plan explicitly refuses to invent a
search-side cache port: that would be scaffolding the sweep deletes. It either
waits for step 3 or extends the sweep's own ICacheProvider, and the sequencing
call is the maintainer's.
Phase 1 is independent of both and does the part that actually caused the
bugs: one pipeline object owning parse → build → post-filter → rank, so a
caller cannot skip a step, plus one kind-set source derived from quartz's
SearchableEvent instead of three hand-kept lists that have already drifted.
Non-goals, the pieces that are genuinely not shareable, and the parity test
that would have caught all four bugs are written down too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Alive DVMs whose heartbeats never reach the user's discovery relays stayed
hidden from the Discover list: the freshness gate only saw beats delivered
by the global heartbeat REQ on the selection's relays, while the beats
existed on each DVM's own outbox (their detail screens proved it).
DiscoveryDvmHeartbeatSubAssembler joins the discovery assembler group and
batches the DVM list's announcement authors per DVM outbox relay
(coverage-ranked, capped at 12 relays; unknown outboxes fall back to the
global REQ). Re-issues on DVM-list membership changes via the inner feed
flow, since FeedState.Loaded reuses its wrapper.
Selecting a Relay (or favorite-algo-feed) chip in the Discover menu, or a
relay whose per-relay slice was momentarily empty, dispatched no kind-31990
filters — and the heartbeat REQ rode along, so beats stopped arriving while
the Content tab kept rendering cached announcements. The 60s staleness timer
then dropped every DVM as its cached beat aged past 420s.
The heartbeat filter now rides the selection's own relays (new
IFeedTopNavPerRelayFilterSet.relays()) instead of the 31990 dispatch result.
`MlsGroup.saveState()` does not serialize the staged-proposal pool. `stageCommit`
prepares its commit on a CLONE restored from that state, so the clone always
started with an empty pool: `commit()` produced an empty proposal list, the
epoch advanced, and every proposal the commit was called to apply was silently
dropped.
The case that bites is a departing member. MIP-03 makes a departure a
standalone `SelfRemove` PROPOSAL — the leaver cannot evict themselves — so it
sits in the pool until an authorized member commits it. That commit ran, looked
successful, and left the leaver IN THE TREE, still holding the group's keys and
still able to decrypt everything sent after they left.
`stageCommit` now hands the clone the live group's pool. The other `stage*`
entry points are untouched: each of those has a proposal of its own to make,
and folding a peer's pending SelfRemove into an unrelated commit would also
trip MIP-03's no-mixing rule for a non-admin committer.
Also here:
- `MarmotManager.commitPendingProposals` — the commons-level entry point for
"commit what a peer proposed", returning null when nothing is staged so a
caller can drive it unconditionally after ingest.
- `MlsGroup.hasPendingProposals` / `MlsGroupManager.hasPendingProposals` — a
public way to ask whether there is work to commit, where the proposals
themselves stay module-internal.
- Scenario runner: `restart_client` (rebuilds the manager over the same stores
and calls `restoreAll`, so nothing may depend on state that only lived in
memory) and `leave`. `restart-delivery-faults` replays.
- `removed_members` observations are now checked, including the evictor's own
commit — the actor was the one participant who did not remember doing it.
Two gaps stay open and asserted rather than deleted, so they fail loudly when
fixed: a peer that has the same proposal staged does not apply the commit
carrying it inline (`leaver-removal-secrecy`), and the proposal pool still does
not survive a restart.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Disband (`marmot.group.lifecycle.v1`, 0x800c)
--------------------------------------------
We enforced the Disbanded state but nothing on this side could enter it, so
the enforcement was only reachable from a peer's commit. `disbandGroup` writes
the component in a Commit every member replays. It refuses what is
structurally impossible — a non-admin, a legacy MIP-01 group with no carrier
for a lifecycle state, a second disband — and, unlike every other metadata
setter, refuses to report success when no relay acknowledged the commit: the
caller is about to tell a human the conversation is over, and a group that is
still live for everyone else must not be announced as ended. The obligation
stays queued either way.
On Android it is an admin-only action in the group header behind its own
confirmation, worded for what it is: for everyone, and not reopenable.
Avatar link (`marmot.group.avatar-url.v1`, 0x8007)
--------------------------------------------------
`setGroupAvatarUrl` existed but only `amy` could reach it — the renderer
already preferred a URL avatar over the Blossom image, and no Android screen
could set one. Edit Group Info now carries the field; it commits separately
from the profile so saving a rename does not rewrite the avatar state, and
clearing it falls the group back to the uploaded image.
Seven more scenario vectors
---------------------------
New steps: `update_group_data`, `remove_members`, and the delivery-fault
family — `omit_message`, `duplicate_message`, `reorder_messages`,
`withhold_message`, `release_withheld`. Selectors are matched key by key and
an unknown key is refused, because a silently widened selector injects a
different fault from the scripted one while still reporting under the
vector's name. `group_profile` outcomes are checked too.
Replaying now: group-data-update, deferred-tick-catchup, incremental-growth,
drop-queued, queue-faults, delayed-past-epoch-app-message,
readd-after-eviction. 18 vector tests, all green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Batched Adds
------------
MDK — and therefore both White Noise clients — turns a `create_group` with N
invitees into ONE Commit: N Add proposals and one Welcome carrying N
`EncryptedGroupSecrets`, keyed by KeyPackage reference (RFC 9420 §12.4.3.1).
We staged one Add per commit, so the same group landed at epoch N instead of
epoch 1 and cost a round trip per invitee.
`MlsGroup.addMembers` / `MlsGroupManager.stageAddMembers` propose-N then commit
once; `MarmotManager.addMembers` publishes that single commit and fans the same
Welcome bytes out to each invitee. The singular entry points delegate, so no
caller changes.
Scenario vectors: two shapes, one of them unread
------------------------------------------------
The conformance vectors state their expectations two ways —
`expected_trace.observations` and `expected_outcomes` — and the parser only
read the first. Seven of the nine vectors here use the second, so they parsed
to ZERO expectations, replayed their steps and reported green without checking
anything. `in_group` was also treated as a leaf, which silently skipped every
step nested inside it, and `send_app_message` built a message the runner never
queued for delivery.
Now parsed and checked: `client_state`, `clients_converged`,
`pending_resolution`, `no_pending_work`, inline `assert`/`payload_count`
(the forward-secrecy and isolation assertions), `added_members`, `clear_events`,
and multi-group clients. `convergence_decision` has no counterpart in our
engine, so `convergence-committer-selected` is now REFUSED via
`UnsupportedScenarioOutcome` rather than passed on the parts that happen to be
modelled. A new guard test fails any vector that parses to nothing to check.
three-client-message-exchange and conversation now replay for real.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
**Disappearing messages are settable.** A picker at group creation — off, 1
hour, 1 day, 1 week — and a read-only line on the group info screen so a member
who cannot change the setting still knows their messages are on a clock. Fixed
values rather than a free-form duration: the number is committed into group
state every member's client reads, and an arbitrary one buys nothing. Creation
is the only place it can be chosen, because promoting a component to required
after epoch 0 takes two commits and that screen makes one. Until now we obeyed
a setting neither Amethyst nor `amy` could set.
**The harness tests the engine users run.** It cloned mdk master and got
whatever was tip; both shipping White Noise clients embed an immutable
MarmotKit artifact and name its `mdk-sha` in a lockfile. It now checks out that
commit, and rebuilds `wn` when the checkout moves — a pinned tree beside a
binary built from a different commit would report a version it did not test.
**A scenario-vector runner.** Their manifest marks 31 artifacts `portable`,
meaning written for one engine and meant to be replayed by another. Three are
byte fixtures we already consume; the rest are scripts. `MarmotScenarioRunner`
replays them against our own stack — publishing captured into a queue,
`deliver_all` moving it into inboxes, `tick` draining them — and checks the
expected trace. Six pass. This is a different claim from the interop harness:
that proves we can TALK to `wn` over a relay, this proves the same events land
us in the same group state.
It refuses an unimplemented step by name rather than skipping it, because a
runner that ignored steps would report a pass for a script it never executed.
That refusal immediately found something. Three vectors create a group with
several invitees, and the reference adds them all in ONE commit — epoch 1.
`MarmotManager.addMember` stages one Add per commit, so we reach epoch N. Both
are valid MLS and any peer processes either, but our traces cannot match, and
creating a group costs an extra round trip per invitee. Asserted as a named
divergence so it stays visible and fails the day batched adds land.
**Two smaller things.** `DispatchStageBenchmark` is opt-in behind
`-DrunLoadBenchmark=true`: it pushed 30k events through six variants twice
inside a `runTest` whose cutoff is one minute, so on a loaded runner it failed
having measured the machine rather than the code. And encrypted-media-v1
(`0x8008`) is closed as not-needed — required only on the Legacy profile, which
strict cutover now forbids joining, so no group that asks for it is reachable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Three related pieces.
**The pinning was approximate.** Expiry was pinned from the group's CURRENT
retention at persist time, which is right for a message delivered under the
current epoch and wrong for one delivered under an older one — a kind:445 held
back as a retained candidate, or replayed after a restart, is decrypted under
an epoch the group has since moved past. Pinning that to today's setting is
precisely what the component forbids. The store now keeps a small epoch →
retention history, written wherever the epoch may have advanced, and the pin
reads the delivering epoch's value. The fallback to the current value is not a
shrug: a group whose setting never changed has one value at every epoch, which
is the overwhelmingly common case. MDK carries `source_epoch` on its rows for
the same reason.
**`amy` can set it.** `marmot group create --disappearing-secs N`, on both
profiles: component `0x8005` for a current-profile group, and the legacy blob
for `--legacy`, where asking for it bumps the version to 3 because v1/v2
deliberately omit the field to stay byte-compatible with MDK's older parser.
**Interop test 26.** Nothing proved MDK accepts a GroupContext that REQUIRES
`0x8005` with our bytes, and retention has the nastiest encoding in the set:
eight big-endian bytes with no length prefix, unlike almost every other Marmot
field, and MIP-01 spelled it differently. amy creates such a group, wn joins
it, and messaging round-trips.
It asserts acceptance rather than read-back, because wn's CLI `group_json`
does not surface the retention value — that field is on the uniffi struct the
apps consume, not this surface. Acceptance is still the encoding test: a
required component whose bytes wn cannot decode leaves the group unreadable,
so `groups show` returning it at all means the eight bytes parsed. The
direction is one-way for the same reason as push — `wn groups` has no command
that sets retention.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Two things the search screen showed that were not true.
**Relevance sorted by date.** `SearchSortOrder.RELEVANCE` shared a branch with
NEWEST, so picking it changed nothing. SearchResultSorter.scoreEvent — which
weighs an exact phrase, token boundaries and an article's title, and is fully
tested — was wired only into AdvancedSearchBarState, the desktop path, and had
no references in this module at all. It ranks here now, so the two front ends
agree.
Scored on the leftover terms rather than the whole box: `from:npub1…` and
`kind:article` are filters, and hunting for their literal text inside an
event's content ranks on noise. A query that is all chips has nothing to be
more or less relevant to, and falls back to newest. The sort still lives here
rather than calling sortEvents wholesale, because POPULAR ranks on a note's zap
total that a raw Event cannot see — the sorter says as much itself.
**An empty result list said nothing at all.** This was the only search surface
in the app without an empty state; settings, git repositories, the location
picker and app recommendations all have one. It is also the one that needs it
most: a screen that seeds only its kind opens holding a chip and showing
nothing, because SearchFilterBuilder rightly refuses "every recent article" as
an unbounded REQ rather than a search.
So the two cases are told apart. A query that asked nothing gets "type
something to search"; a query that asked and came back empty gets "no results".
Blaming the network for a search the reader has not written yet is the failure
worth avoiding, and the method that decides is pinned by tests so the two
messages cannot swap places.
"No results" waits out a grace period first. No EOSE from the search
subscription reaches this screen, so nothing actually knows the relays have
finished; without the delay an empty list would announce failure in the gap
before the first event arrives, which is every search, for a moment.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Tests 18 and 19 failed inside TLS before a single frame was written:
CRYPTO_ERROR (TLS alert 42): certificate chain validation failed:
PKIX path building failed: unable to find valid certification path
The reference broker generates a self-signed certificate — its startup JSON
says so, `"tls":"generated_self_signed"` — and there is no CA anywhere in this
picture, so chaining it to the JDK trust store could never have worked. The
binding anticipates exactly this: a client MAY pin the endpoint certificate by
SHA-256 instead of chaining, which is what `--pin-sha256` and
`PinnedCertificateValidator` are for. The broker prints the fingerprint the
pin needs, in the same JSON line the harness already waits on; the harness
just never read it.
So `start_quic_broker` now captures `server_cert_sha256_fingerprint` and fails
loudly if it is absent, and the three `amy marmot stream send|watch` calls pass
it. `wn` reaches the same place with `--insecure-local`; pinning is the better
half of that trade, since the peer still has to sign the TLS transcript with
the pinned certificate's private key.
25 passed, 0 failed, 0 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Paging through the `kind:` list slid the top and bottom bars away as if the
results had been scrolled.
A picker is a scrollable that lives inside the *top bar*, and
DisappearingBarNestedScroll moves the bars on `consumed.y + available.y` — the
total scroll that entered the chain, deliberately, so the bars also ride
overscroll at a list's edges. That sum is conserved as a scroll walks up the
nested-scroll chain: a node hands its parent `consumed + myConsumed` and
`available - myConsumed`. So no connection placed under the picker can hide its
scrolling from the scaffold, however much it consumes — the scaffold has to be
told instead.
It already takes `allowBarHide`, and reads it through rememberUpdatedState, so
the search screen pins its bars while a picker is open and releases them when
it closes. Which is what you want anyway: chrome that slides around under a
dropdown the reader is aiming at is its own bug.
The flag is reported upward from the field rather than read downward, because
the scaffold that has to stop moving is composed above the field that knows.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Reverts 490cd4fa. The row offered two things the text field already does on its
own — put the caret in the token and press backspace — so it bought no new
capability, and it appeared whenever the caret so much as touched a chip, which
made it noise rather than an affordance.
The chips stay tappable in the sense that matters: the caret goes where you tap
and the token is ordinary editable text, which was always the point of keeping
the query as text rather than as a set of widgets.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
A `kind:` window is a property of an event, and a person is not an event of any
kind — so All and People cannot answer such a query. Leaving them selectable
offers the reader a scope guaranteed to come back empty, which reads as the
search being broken rather than as the filter doing its job.
The toggle now shows Notes and greys the other two while a kind is in the box.
Greyed rather than merely unselected: an option that is present and silent is
worse than one that is visibly unavailable.
The applied scope is derived from the reader's pick rather than written back
over it, so dropping the `kind:` chip returns them to whatever they had chosen
before instead of leaving them pinned by a filter that is no longer there.
Every consumer of `scope` — the people, note, channel and hashtag result flows
— reads the derived value, so none of them can disagree with the toggle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
`marmot.group.message-retention.v1` decoded into `MarmotGroupState.retention`
and then nothing read it. In a group with disappearing messages enabled, every
member's copy vanished on schedule except Amethyst's, which kept the plaintext
indefinitely — not a wire incompatibility, the group still worked, but a
privacy divergence from what that group was told it had.
Expiry is pinned per message when it enters the log, never recomputed. That is
the component's rule and it is the easy one to get wrong: a message keeps the
retention of its OWN source epoch, so changing the setting later must not
shorten, extend, or restore the expiry of a message that already exists.
Recomputing from the current setting would let one member retroactively
shorten everyone's history, or resurrect what should already be gone. First
write wins for the same reason — the ratchet rewinds on restart and relays
replay recent kind:445s, so the same message really is persisted twice, and a
second write that re-timed it would let a message postpone its own expiry
every time it was replayed.
The retention itself is read from the `0x8005` component with a fallback to a
legacy group's `0xF2EE` field, because the two profiles express the same
setting in different places and reading only one would silently treat half the
groups as having no expiry.
Expiring deletes rather than hides. This store is the only copy — the ratchet
moved past the ciphertext it came from long ago — so a message that is merely
filtered out of a read is still on disk, and a disappearing message that is
gone from disk but still on screen has not disappeared either. Both stores
rewrite their logs, reads prune first so a restart cannot show something that
fell due while the app was closed, and the front end is told what went so it
can drop those rows from a conversation already open.
Traffic is the clock: a group being read is a group whose expired messages
should already be gone. There is no timer, so a group nobody opens keeps its
messages until someone does — worth knowing, and better than a wakeup that
exists only to delete.
Expiry stays advisory by design, as the component says: the duration is
authenticated but the base is the sender's own `created_at`, so it inherits
the trust already placed in an MLS-authenticated sender and is not a guarantee
against a hostile one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Opening search from any seeded screen crashed the app on the spot:
NullPointerException: Attempt to invoke virtual method
LazyListState.scrollToItem(...) on a null object reference
at SearchBarViewModel.updateDataSource(SearchBarViewModel.kt:479)
at SearchBarViewModel.<init>(SearchBarViewModel.kt:131)
`sourceWatcher` shares its flow `Eagerly`, so `onEach { updateDataSource(...) }`
runs while the constructor is still executing, and `updateDataSource` scrolls
`listState` -- which was declared *after* it. Kotlin initialises properties in
declaration order, so at that moment the field is still null.
It was latent until this branch. `updateDataSource` returns early on a blank
term, and the box always opened blank, so the scroll was never reached during
construction. Seeding the field with the screen's own filter makes the term
non-blank on the very first pass, which turns the ordering bug into a crash the
moment search opens from any seeded feed.
Moving the declaration above the collector fixes it, and the comment says why it
has to stay there -- the next person to tidy these fields alphabetically would
put it back.
Reproduced deterministically before the change (Reads -> search, fresh crash
buffer, one FATAL) and confirmed gone after, on the same tap sequence. The
feature it was blocking now works: search opened from Reads seeds `kind:article`
as a chip, tapping the chip offers Change/Remove, and Change cuts it to `kind:`
and opens the new kind picker.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three pieces of hardening taken from the reference client's `:fuzz` module.
**Parse bounds.** An app payload reaches a decoder only after MLS has
authenticated that a group MEMBER sent it — never that it is well-intentioned,
and deep nesting or a huge collection costs a parser far more than it costs
whoever sent it. `MarmotJson` now pre-scans for the same three limits the
reference draws, at the same values: 64 KiB, depth 16, 64 elements per
container. The scan is linear and runs before any JSON library sees the string,
and it deliberately does NOT double as a validity filter — malformed input
inside the limits still reaches the parser, so its error paths keep being
exercised. `MarmotAppEvent.decode` is the choke point, so every inner kind is
covered, with kind:1210 checked again at its own entry point because
`fromAppEvent` can be reached without it.
The byte limit counts UTF-8 rather than UTF-16 code units, which is the
difference between a 64 KiB cap and a 256 KiB one for a payload of emoji.
**Two ported targets.** Neither could be a like-for-like copy, because the
reference fuzzes code we do not have in that shape — their metadata walkers are
deliberately Android-free byte functions, ours is `ExifInterface` over a `Uri`.
What ports is the set of oracles:
- Identity references, against `Nip19Parser`: never throws, deterministic,
idempotent on what it canonicalises, and never emits a key that is not
32 bytes of lowercase hex. The corpus is their grammar — `nostr:`, profile
links, percent-encoded separators, truncated and over-long bech32 bodies,
clipboard text with several references run together.
- Container sniffing, against `ShareHelper`: never throws, deterministic,
always names a declared kind, a mismatched walker does not claim the
container, and — the one that matters most — only the header decides.
A sniffer that read past its header would let bytes deep inside a file
relabel it, which is what content-type confusion needs.
Seeded rather than Jazzer-driven, so no fuzzing engine joins the build and a
failure reproduces from the printed seed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Seventeen more feeds hand their kind window to the search box: communities,
workouts, music tracks and playlists, podcasts and episodes, software apps,
shorts, products, follow packs, live streams, polls, stories, calendars and
calendar sets, long videos. Each also carries whatever its list spinner
narrowed it to, on the same terms as the feeds wired earlier — a hashtag or a
geohash says itself as a token, a follow set does not and seeds nothing.
Home seeds only its spinner: its feed spans every kind Amethyst can render, so
there is no one window to hand over.
Thirteen new KindRegistry aliases so those chips read as names rather than
numbers, which also makes each one a row in the `kind:` picker. The four video
feeds are nested windows over the same kinds — `short` is 34236, `longvideo`
34235, `stories` both, `video` all four — and the alias matcher already refuses
to widen a window into a larger group that merely contains it. Pinned per
screen, because a window that serialized to a name the parser read back as a
different set would hand the reader a search their screen never asked for, and
nothing else in the round trip would notice.
`baseFilter` loses its default. It had one, and the result was that two thirds
of the app's search buttons opened bare without anyone noticing — an omission
that looks exactly like a decision. Now a screen must decide, and the four that
genuinely have nothing to say pass null and say why: the location-channel list
and the group-discovery list each show many, not one, and NIP-17 messages are
encrypted, so no relay can search them and no kind window would return anything
the reader could read.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Rendering kind:1210 rows, added earlier today, trusted the wrong thing. MLS
authenticates that a member SENT an inner payload; it says nothing about
whether the payload is true. The renderer read `actor` and `subject` straight
out of that payload, so any member could send a well-formed 1210 saying "X
removed Y" or "X renamed the group" and Amethyst would draw it as a system
caption — attributed, styled as history, indistinguishable from a real one,
in the part of a conversation a reader trusts most.
`syncGroupSystemRows` already documented the rule ("one that arrives over the
wire is an assertion by its sender, not a derived fact"); the render path
simply did not honour it. The reference client draws the same line from the
other side — its raw 1210 parser nulls attribution outright and marks every
result unauthenticated, with a fuzz target asserting exactly that.
The rule now lives at the one choke point every row passes through:
`MarmotGroupList` shows a 1210 only when this client authored it. That is the
right test because a derived row is diffed from MLS-authenticated state and is
always authored by the account itself. It has to be there rather than at
ingest, because rows arrive by two routes — live decryption and the restart
re-read of the local log — and the log holds received payloads too, so an
ingest-only guard would have let a forgery back in on the next launch.
Dropping the sender's version costs nothing: every client that applied the
same commits derives the same rows.
The same feature had a second defect, which the first one was hiding. Derived
rows were persisted but never surfaced, so they appeared only after a restart,
and Android derived them solely for its own commits. In practice the only 1210s
reaching the feed live were the untrusted ones. Rows now surface as they are
derived, and every accepted commit derives them, not just ours.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
A chip is drawn text inside one BasicTextField, not a composable of its own,
so tapping it can only move the caret — there is nowhere to hang a ✕ on the
chip itself. So the caret landing on a finished token is what stands in for
"the reader tapped this one", and a row under the field turns that into the
two things anyone wants from a filter they can see: change it, or drop it.
Change cuts the token back to its prefix — `kind:article` becomes `kind:` —
which leaves the field in exactly the state typing `kind:` and stopping would
produce, so the picker opens on the spot and every existing rule about what it
offers and what a pick splices in still holds. Nothing new had to learn how to
edit a token. A chip with no picker (`#tag`, `-term`, `"a phrase"`) is selected
instead, so the next keystroke replaces it.
Remove takes the one space the token leaves with it. That is not cosmetic: the
field's text round-trips through the parser on every keystroke, so a doubled
space compounds every time a filter is dropped, and a seeded query — which
arrives with a trailing space so its chip settles — has to come back to a
genuinely empty box rather than one that looks used.
The editor and the picker can never both be up: a picker only opens on a token
that is not finished, and the editor only on one that is. That is asserted
rather than assumed, along with every offset of a chip resolving to that chip,
since a tap can land on either edge.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Advertising a capability and running a service are different claims, and
conflating them made an Amethyst user un-addable to any group a White Noise
user starts.
The reference client installs agent-text-stream-quic-v1 with
`required_member_roles = receive` into the required component set of EVERY
group it creates, then refuses an invitee whose KeyPackage omits either the
component or the `0xF2D1` role — checked before negotiation, so negotiation
cannot rescue it, and the invite path applies the same rule. Dropping the
advertisement on the grounds that nothing in the deployed network publishes
previews was right about the traffic and wrong about the capability: a
capability says "this client can handle it", never "this group uses it".
So the component and the receive role go back. `send` and `fanout` stay off
and no watcher is started — we can be shown a preview, we do not originate
one, and nothing dials a broker.
Two tests had encoded the old premise, one of them asserting the refusal as
if it were a feature. They now assert the rule that actually decides interop:
our default leaf is admitted by the reference's own stream policy, and is
refused by a group that requires `send`. The KDoc on
`currentProfileLeafCapabilities` had described the role as present the whole
time — it was the code that had drifted from it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Kind 451 had no callers, and the reason turned out to be everything around
it: the 447/448/449 events in this tree were the exploratory shape the spec
now names as not interoperable — tokens in `token` tags with empty content,
the sender's leaf implicit, no removals at all, and no owner authentication.
The token encryption derived its key from the old `mip05-v1` salt, and the
446 trigger still carried the `encoding` tag the adopted rumor dropped.
Wiring the proof into that would have produced records no peer can read.
So the gossip is now content-JSON under `marmot-push-v1`, and the version
string is the gate: the old value is refused rather than translated, because
the two versions are not predecessor and successor.
The design the rewrite is really about is owner authentication. A record's
authority comes from its own `owner_sig` and current membership, never from
who carried it — which is what lets one member relay another's records so a
group converges without every owner being online, while stopping the relayer
from repointing, re-signing or restamping what it carries. `PushSignedRecord`
is the canonical byte string that makes both halves computable; it uses the
spec's fixed-width fields rather than this codebase's usual QUIC varints,
which look identical locally and are wrong on the wire.
The part that costs real machinery is revocation. A removal does not merely
delete: it leaves a tombstone at its own `(owner_ts, digest)` stamp, and that
stamp has to be durable. Any current member can re-emit a revoked but still
validly-signed record in a fresh kind 448 at any later epoch, so its carrying
epoch is unbounded and no retained-message window can bound it. The stored
stamp is the only thing that recognises such a record as stale, which is why
`MarmotPushStateStore` exists and why Amethyst backs it with a file.
Everything here is advisory end to end. A bad entry, an unverifiable
signature, a stale list — each drops on its own and none of it may reach the
validity of the kind:445 that carried it. The decoders return what they could
read instead of throwing, and the coordinator catches at its boundary, so a
surprise cannot escape into ingest.
Not wired: announcing a token of our own. That needs Amethyst's own
notification-server public key, which is a deployment decision rather than
something the protocol discovers — a server can only wake the app whose push
credentials it holds. Until it exists this client participates correctly in
other members' routing and announces nothing.
MDK's `wn` exposes no push commands, so the harness cannot drive this against
the reference. Coverage is the spec's published removal fixture, byte-layout
assertions written independently of the encoder, and the ordering and
tombstone rules.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`-scam` parsed into excludeTerms, serialized back out, and then did nothing at
all on Android. NIP-50 has no negation operator, so an exclusion can only ever
be applied to the results — and SearchResultFilter, which does exactly that,
was called from desktop and from nowhere else. The same held for the
`kind:reply` and `kind:media` pseudo-kinds, which describe a shape of an
event's tags rather than anything a relay indexes; they now matter more,
because the new `kind:` picker offers both by name.
SearchResultFilter gains a per-event `matches` predicate so a caller holding
its own list type can apply the same rules without going through `filter`,
which is what the Android search screen needs — it works in Notes, not Events.
`filter` is rewritten in terms of it, so desktop keeps the behaviour it had.
Quoted phrases needed no fix: the parser keeps the quotes in the text, they
travel to the relay's NIP-50 `search` as typed, and locally
EventSearchMatcher already reads a quoted span as one term rather than two.
Both are now chips as well. These two are drawn only — the tokenizer lifts
them out for the renderer and hands the raw text straight back to the pass
that already reads them, so what the query means is untouched (QueryParserTest
and QuerySerializerTest pass unchanged). An exclusion draws struck through in
the error colour rather than a tint, because it is the one token whose effect
a reader can misread as its opposite; a phrase drops its quotes, since the
chip itself is what says the words travel together.
`-#bitcoin` is deliberately left to the hashtag splitter: claiming it here
would quietly turn "the bitcoin hashtag" into "not the word #bitcoin", which
is a decision about the language rather than about how it is drawn.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Two app payload kinds already round-tripped through the protocol layer and
then stopped at the feed: a kind:1009 edit was indexed but nothing drew it,
and a kind:1210 system row was filtered out of the message list entirely.
Edits reuse the overlay rail the other chat protocols already use.
`latestMarmotEdit()` picks the winner off a message's own `edits` children,
and it is deliberately read-side: the transport cannot stop a member from
sending a well-formed 1009 that names someone else's message, so the reader
is the one that has to check the author matches. Ties on the same second
resolve by event id, otherwise two devices of one account could leave two
readers rendering different text for the same message forever, with neither
of them wrong. `RenderConcordEditedNote` was already exactly the renderer
this needs, so it loses the protocol from its name and gains a Marmot caller.
System rows needed somewhere to go. `ChatFeedRowRenderer` is a hook the feed
consults per item: a renderer claims a note, or the ordinary bubble draws it.
That keeps a Marmot-shaped row out of the generic chat feed, which serves
four other protocols. `MarmotSystemRowRenderer` reads the row's structured
fields rather than its `text`, so the caption is localized here instead of
being whatever string the sender happened to compose; `text` stays as the
fallback for a row whose fields we cannot read.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`from:`, `since:` and `group:` each open a picker as they are typed; `kind:`
did not, so the only way to write one was to already know the alias by heart.
That was tolerable while `kind:` was invisible in the field. Now that it pills
— and now that screens seed it — a chip a reader cannot discover how to type
is the wrong half of the feature.
The kind picker is the one this component can fill in itself. People and
groups are account-scoped, relay-backed questions that `commons` deliberately
refuses to answer, which is why they arrive as caller-supplied rows; "which
kinds are there" is a constant in KindRegistry. So every caller of
TokenizedSearchField — Android's search screen and the desktop spotlight —
gets the picker with no wiring at all.
A bare `kind:` offers the whole vocabulary, because a reader who typed the
prefix and stopped is asking what the options are. A partial offers what it
could still become, prefix-matched rather than substring-matched so the list
stays a completion instead of a search. Each row says the kinds it will
actually ask a relay for — the difference between `kind:video` (four kinds)
and `kind:21` (one) is worth seeing before picking — and a pseudo-kind says it
filters on results instead, because it asks a relay for nothing.
Unlike `group:`, the picker stands down on an exact match: the vocabulary is
closed, so `kind:article` really is finished, where `group:gen` is both a
plausible id and a prefix of `general`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
A screen is already a filter — the articles feed is a kind window, a profile
is one author, a location channel is one geohash — and none of that survived
tapping search: the box opened empty and the reader retyped what the screen
already knew.
Every search button now hands over a SearchQuery, serialized into the route
and seeded straight into the field. It is ordinary field text from there on,
so it draws as chips through the path anything typed takes, and a backspace
drops it — rather than a base filter held beside the box that the reader
cannot reach.
Three things had to be true first, and were not:
- `kind:`, `lang:` and `domain:` were never tokenized. They were read only by
QueryParser's second pass over the leftover text, so they filtered but drew
as plain words. They are tokens now, and pill like the rest. An alias the
registry cannot resolve, a language that is not a code and a domain that is
not a hostname stay text, so a chip still cannot claim a filter that is not
sent.
- `query.kinds` never reached the REQ on Android: searchPostsByText always
fanned out over the three fallback kind groups, and the local cache scan
asked for every kind. A `kind:` chip promised a narrowing that stopped at
the field. Desktop already honoured it; both paths now do.
- QuerySerializer named kinds one at a time, so the two-kind `channel` alias
wrote itself twice. KindRegistry.tokenize matches alias groups whole and in
input order, and refuses to widen: 30312+30313 is `kind:nest`, but 30312
alone stays `kind:30312` rather than borrowing the larger `live`.
Seeded screens: profile (from:), articles/pictures/highlights/nests/nsites/
emoji/badges/public chats/napplets/git repos (their kind window), the hashtag
and geohash feeds and the geohash chat (#tag, geo:), relay group chats
(group:, not on DMs), and notifications (to:me). Where a feed has a top-nav
list spinner, what that spinner narrowed to seeds too — but only when the
token language can say it. A follow set seeds nothing rather than some of its
authors: spelling one out fills the box with chips, and truncating it would
quietly seed a narrower query than the feed the reader was looking at.
Profile, hashtag, geohash and relay-group chat gain a search action; they had
none before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66
Two unrelated things that both amount to not paying for something nobody
asked for.
**`MirrorSyncThroughputTest` is a benchmark, so it now opts in.** It
preloaded a million events and pulled them over a real WebSocket on every
ordinary test run: 4,584 s of `:geode:test`'s 4,636 s — 98.9% of the
module's test time for one test that asserts nothing about correctness and
reported `skipped` at the end anyway. Every other benchmark in the module
is already gated this way (`perf.LoadBenchmark`). It now bails before
building anything, and enables on `-DrunLoadBenchmark=true` OR on any of
its own sizing properties, so every invocation its kdoc documents still
runs it — naming a size is itself the opt-in. Measured after: the test
takes 5 ms, the module takes 64.8 s, and `-DsyncN=2000` still prints a
throughput number.
**The agent text stream QUIC path is kept but no longer advertised, and
nothing starts it.** Nothing in the deployed network publishes those
previews. So:
- `SUPPORTED_COMPONENTS` drops `0x8006` and the leaf capabilities drop
`0xF2D1`/`0xF2D2`/`0xF2D4`. A capability is a standing promise to every
peer that reads our KeyPackage, and one for a path nobody exercises
costs something and buys nothing. The captured reference KeyPackage in
our own conformance vector does not advertise `0x8006` either.
- The Android chat screen no longer builds a stream watcher and dials the
brokers a kind:1200 advertises. That was a UDP connection attempt to a
third-party endpoint on every feed change, on behalf of a feature with
nothing to show — a service we start, not a capability we hold.
The implementation stays and stays tested: `:marmotQuic`, the codecs,
`amy marmot stream`, the direct path, the certificate pinning and the
interop tests are all untouched. The module README records the posture and
the exact way back.
Three tests asserted the old advertisement and were reworked rather than
deleted. The role-enforcement gate is still covered — the tests now build
leaves that explicitly carry the roles, which is the better shape anyway,
since a test that exercised the gate through OUR default was really
asserting the default and stopped testing the gate the moment it changed.
A new test pins the new default: our KeyPackage carries no role and is
therefore refused by a group requiring one. That refusal is the deliberate
cost, so it is asserted rather than discovered.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Our Marmot tests agreed with nobody but themselves. They were written from
the same reading of the spec as the code they test, so a parser that read
a tag differently from every other client would pass all of them.
MDK ships fixtures built for exactly this. `fixtures/encrypted-media/
imeta-v2.json` says so in its own description: "Shared by marmot-app,
marmot-uniffi, and wn-cli tests so every layer agrees on validation
verdicts and exact wire round-trips." We are another layer and were not
using it. Same for the byte-level component vectors under
`cgka-conformance-simulator/vectors/byte-fixtures/`, whose manifest marks
31 of its 41 artifacts `"status": "portable"`.
Copied verbatim and wired to our codecs:
- **10 imeta v2 cases** — 5 golden, 5 rejections. The rejections are the
half that matters: a merely lenient parser passes every golden case and
still cannot be interoperated with, because it accepts tags a conformant
sender never emits and then renders media another client refuses. The
fixture also distinguishes an absent hint from a present-but-empty one,
which is a real wire distinction we now assert rather than assume.
- **10 imeta v1 cases as NEGATIVE cases.** `0x8008` is frozen and "MUST
NOT be reinterpreted as v2"; the two share enough field layout that a
parser keying only on fields would read one as the other and derive a
file key under the wrong scheme. Every v1 case now has to bounce off the
v2 parser, including the ones v1 itself calls valid.
- **3 nostr-routing byte fixtures.** These are the first tests we have
that pin a component's wire bytes against another implementation instead
of against our own encoder — a round trip proves we can read what we
wrote, which is a different and much weaker claim. The invalid fixture
is the sharp one: a decoder that deduplicated the relay list rather than
refusing it would hold bytes no peer agrees with.
All 8 tests pass unmodified, so this is coverage rather than a fix — but
it is coverage that can now fail for a reason our own tests never could.
The fixtures carry a README with their provenance and a refresh command,
because a copied artifact drifts silently. It also records what is still
missing: the 19 portable scenario vectors need a runner that drives our
client through a scripted trace and projects state per
`foundation/conformance.md`, and that is where the convergence and
crash/restart coverage lives.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Six new harness tests against MDK, all green in one clean run:
20 avatar-url amy->wn 21 avatar-url wn->amy
22 edit amy->wn 23 deletion amy->wn
24 media-v2 amy->wn 25 media-v2 wn->amy
The media pair needed a blob store, so the harness now runs a loopback
Blossom server of its own (`blossom-server.py`, PUT /upload + GET
/<sha256>). It holds nothing but ciphertext — the file key comes from each
group's MLS exporter — so a download that hashes back to the original
bytes is proof both implementations derived the same key. That is the
whole point of tests 24 and 25, and they pass in both directions.
Test 20 sends a URL that is deliberately NOT normalized
(`https://Example.COM:443/a/./avatars/../pic.png`) and asserts both what we
store and what wn reads back. Normalization is the wire format for this
component — a decoder rejects bytes that differ from its own serialization
— so a disagreement here is a group the other side cannot read at all,
not a cosmetic difference.
Tests 22 and 23 assert what the protocol actually says rather than what a
renderer happens to do. For the edit that means a well-formed kind:1009
reaching wn (one `e` tag naming the target, the replacement as its body,
the right author) plus our own reader applying the overlay — MDK's storage
deliberately leaves the original row's body alone and lets the client
compute the chain, so asserting on painted text would be testing its TUI.
For the deletion it means the `deleted` flag on wn's materialized
timeline, which is its user-visible truth.
Two harness bugs surfaced while getting there, both of the kind that make
a failure unreadable rather than wrong. `run.env` — where tests hand each
other group ids — survived the per-run state wipe, so a `--tests` subset
that consumed without re-creating failed on "not a member" for a group id
from a previous run. And `amy_json` read `$?` inside `if ! cmd`, where it
is the status of the negation, so every failure reported "exit 0".
Push (kind 451) has no cross-implementation test here and cannot: the
owner proof is an UNPUBLISHED event handed to a push service, the
reference CLI exposes no command that emits one, and the harness runs no
push service. There is nothing for two implementations to disagree about
on the wire.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`marmot.group.encrypted-media.v2` (0x800b) had a codec, a policy type and
a key schedule in quartz, and not one caller. The group could not carry a
media policy, nothing derived a v2 key, and Android still sent MIP-04
attachments — which peers on the current profile no longer create.
The policy is now group state: read through `MarmotGroupState`, committed
through `setEncryptedMediaPolicy`, and honoured by the sender. Both of its
lists are ordered on purpose — `default_blob_endpoints` order IS the
upload/fetch fallback priority — so reordering one is changing where the
group uploads, not reformatting it.
Sending and receiving run through the group's own MLS exporter, so the
blob store sees ciphertext and its hash and is storage rather than a party
to the conversation. `amy marmot media` drives the whole loop
(policy/set-policy/send/get), and Android picks v2 per group: a group
carrying `0x800b` gets a v2 reference, one that does not keeps MIP-04, and
the frozen v1 policy at `0x8008` is never reinterpreted as v2.
The URL normalizer is now shared rather than approximated twice. The media
policy says its base URLs use the same WHATWG normalization the avatar
component defines, and it was instead checking a hand-rolled structural
subset — which rejected `https://host//double/`. That URL is normalized:
WHATWG keeps the path as a segment list and only `.` and `..` are special,
which the reference `url` crate confirms. So the old check refused group
state the reference implementation produces, the exact failure the shared
normalizer exists to prevent. `http` is permitted for a blob store and not
for an avatar, because the components differ there and a self-hosted store
on a private network is real.
Two smaller things the reading turned up. `0x8007` was implemented but
missing from the advertised supported-component list, so a group requiring
an avatar URL would have refused our KeyPackage. And the note claiming we
do not advertise the agent-stream send/fanout capabilities has been false
since the sequence store landed.
Epoch 0 deliberately does NOT carry the media policy: the reference
implementation's own epoch-0 GroupContext does not, and adding it unasked
would both diverge from that and require every joiner to advertise
`0x800b` before it could be added. The conformance vector caught that when
the default went the other way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Kinds 1009 and 1210 had codecs in quartz and not one reference anywhere
above them. So an edit from a peer landed in the log and changed nothing,
and a group state change produced no row at all.
**Edits (1009).** An edit is not chat: it replaces the target's text in
place, must never render as its own row, and must not advance an unread
count — a reader caught up with the original is caught up with the edit.
Two rules decide which text a reader sees, and both are enforced at READ
time because a sender cannot be trusted to have applied them: only the
account that wrote a message may replace it (by account, not by leaf, so
a second device of the same account still qualifies), and the latest edit
wins with the event id breaking a tie. The tie-break is not decoration —
two devices of one account can stamp the same second, and without it two
readers would render different text for the same message forever.
**System rows (1210).** These are synthesized locally from canonical group
state, never received: a row derived from an MLS-authenticated commit
cannot be forged by one member, and every client that applied the same
commits derives the same rows. The derivation is a pure diff of two
snapshots with a fixed output order, because two clients ordering rows by
hash iteration would show the same history differently.
Diffing against a PERSISTED baseline rather than against the pre-commit
state in hand is what makes it safe: it is idempotent, it survives a
restart mid-transition, and it cannot write a second caption for a change
it already described. The first look at a group establishes the baseline
and writes nothing — a joiner announcing every existing member as newly
added would be a timeline full of events that did not happen.
Two bugs the tests found rather than the reading did. The row content's
quote escape was written as the literal text `ESC"`, which produced a
content string no decoder could read back — and since a 1210's content is
inside the app event's id preimage, a peer would have rejected the row
outright rather than merely mis-rendering it. And the snapshot read the
group name off the current profile's components alone, so every legacy
MIP-01 group — which keeps its name inside `0xF2EE` — looked permanently
nameless and no rename ever derived a row.
Android now also keeps 1009, 1200 and 1210 out of the chat feed. None of
them is a chat bubble: an edit would show the same sentence twice, a
stream anchor has an empty body and would render blank, and a system row
would render as a bubble of JSON. Rendering 1210 in its own style, and
applying the edit overlay in the bubble, still needs a renderer that
knows about them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Two halves of the same gap in `transports/quic.md`.
**The direct path.** The binding has a second delivery mode we had not
built: the sender dials the receiver, opens one unidirectional stream and
writes records with no control envelope at all. It is deliberately smaller
than the broker path — the dialed endpoint is already the one receiver, so
there is no room to claim — and it negotiates its own ALPN so an
incompatible change to either mode cannot reach the other. Note the
inverted direction: here the RECEIVER listens and the SENDER dials, which
is also why v1 gives it no start-payload discovery and it is only usable
against an endpoint known out of band.
Only the sending half is here. `:quic` is a client stack with no server
role, so this module can dial a direct receiver but cannot be one; that is
recorded in the README rather than half-built.
**The pin.** Preview endpoints and brokers are commonly self-signed and
the binding expects that, saying a client MAY pin by exact DER or SHA-256
fingerprint. What we had instead was `PermissiveCertificateValidator` on
the CLI path, which is not a weaker trust model — it is none, and anyone
on the path can be the broker. `PinnedCertificateValidator` replaces the
chain and the hostname check and nothing else: the peer still has to sign
the TLS transcript with the pinned certificate's private key, so copying a
public certificate off the wire buys an attacker nothing. `amy marmot
stream send|watch` takes `--pin-sha256`, and `--insecure` still exists for
a throwaway local broker but now has to be asked for by name.
Both are verified against the reference implementation, which is the only
thing that can tell an ALPN string, a stream direction, an absent envelope
and a frame prefix from an implementation agreeing with itself: our direct
sender against `wn stream receive`, and the pin — accepted and refused —
against a real handshake with `marmot-quic-broker`.
One thing that only showed up under a real handshake: a certificate the
validator refuses closes the connection before it is established, and the
transport was reporting that as PeerClosed. A caller walking a candidate
list reads that kind to decide what to do next, and "never connected" is
not "the peer hung up on us", so it is classified on the connection's
actual status now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`marmot.group.avatar-url.v1` (0x8007) is the lightweight alternative to
the encrypted Blossom blob: a link, two opaque render hints, no key
material. We had neither the codec nor a carrier for it, so a group MDK
gave a URL avatar rendered as if it had none.
The hard part is not the struct, it is that the URL is canonical state.
The spec makes normalization a producer-side encoding rule and requires a
decoder to re-run the WHATWG parse-and-serialize and REJECT bytes that
differ — never repair them, because two members repairing differently
hold different bytes for the same group. So `MarmotHttpsUrl` is a WHATWG
serializer, not a validator with a regex: lowercased scheme and host, the
default port dropped, dot-segments resolved against a segment list (a
trailing slash is a final empty segment, which is also why `/a/.` keeps
one), percent-encoding normalized with existing triplets left verbatim.
The vectors in the test come from the Rust `url` crate the reference
implementation uses, so the two agree byte for byte. Non-ASCII hosts are
refused rather than guessed at: IDNA is not implemented here, and a wrong
punycode encoding would be worse than a refusal.
Contact safety is deliberately a separate function. A URL can be perfectly
valid group state and still be somewhere this client refuses to go, and
the spec is explicit that the fetch decision "MUST NOT affect component or
commit validity" — so the SSRF check lives at the renderer, where an
unsafe destination falls back to the Blossom image instead of erroring.
Clearing writes the canonical empty state rather than removing the
component. Removal is not a free substitute: a component MUST NOT be
removed while `app_components` still lists it as required, so a remove is
only legal in the same Commit that stops requiring it — and the empty
state is what the reference implementation writes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Both kinds land in LocalCache as regular Notes, so the search screen's
scan over `cache.notes` could match their `content` (a kind-5 deletion
reason, a kind-62 vanish reason) or an id prefix and render them as
result cards, which they can't meaningfully be. Add them to
`excludeNoteEventFromSearchResults` alongside the other non-renderable
kinds (reposts, reactions, zaps, metadata, contact lists, app data).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LeT6g4wLvn2sJofnntvsZ1
`shellTemplateKeepsTheOpaqueIframeAndSourceChecks` sat in `commonTest`, so it ran
on both targets: green under `:commons:jvmTest`, and red under
`:commons:testAndroidHostTest` with
MissingResourceException: ... files/napplet/shell.html.
Android context is not initialized.
`NappletWebContract.shellHtml()` goes through Compose Resources, whose Android
reader needs an initialised `Context`. A host unit test has none and commons does
not use Robolectric, so the same assertion was red or green depending only on
which target happened to run it. That is the worst shape for a test: it fails on
a developer's machine having passed in whatever ran last, and it kept the whole
module red regardless of the change under review.
Moved to `jvmTest` as `NappletShellResourceTest`. Nothing about it is
platform-specific -- `shell.html` is one shared file, so reading it once on the
JVM checks its contents everywhere. What is genuinely not covered is the Android
resource plumbing, which needs a `Context`; that wants an instrumented test, and
the KDoc says so rather than leaving the gap silent.
The other two contract tests never touch a resource and stay in `commonTest`,
still running on both targets. `:commons:testAndroidHostTest` now passes 1530
tests with no failures, and `:commons:jvmTest` 1913 -- the shell assertion among
them, so it is still enforced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Six tests failed on any machine outside UTC, by exactly that machine's offset:
QueryParserTest sinceDate, sinceDateYearOnly, combinedQuery
QuerySerializerTest sinceDate, untilDate, combinedQuery
They hardcoded 1735689600L -- 2025-01-01 midnight *UTC* -- but the parser now
returns the reader's own midnight, and the serializer formats through
`DateUtils.localDay`. In America/New_York the parser answered 1735707600 and the
serializer rendered "2024-12-31", both correct. The production code is right; the
assertions were left behind when the bound became local.
The same file already had it right further down, where the newer cases assert
`LocalClock.startOfDay(SearchDate(...))` under a comment explaining that a bound
is the reader's midnight and not UTC's. These six now say the same thing, so they
state the intended behaviour rather than the behaviour of a UTC build machine.
`timestampToDate2025` keeps its UTC literal deliberately: `timestampToDate` is
plain epoch arithmetic in `DateUtils`, not the local formatter `serialize` uses.
Changing it would have broken a passing test -- the two paths in `QuerySerializer`
genuinely differ.
Verified green in America/New_York, UTC, Pacific/Auckland and Asia/Kolkata --
37 + 19 tests, zero failures in each. Full quartz, commons and amethyst suites
pass locally, which they did not before this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Our leaf advertised `receive` only, which was honest while nothing could
originate a stream and is not any more. It now advertises `receive`,
`send` and `fanout` — the same set MDK puts on every KeyPackage it
publishes — so a group requiring any of them admits us. A capability is a
claim about what the client supports, not a duty to stream: a member that
never originates one is a quiet member, not a broken one.
The role-gate test asserted the old behaviour, so it was testing our own
capability set rather than the gate. It now builds a deliberately reduced
leaf and checks that THAT is refused, which keeps working whatever we go
on to advertise; a second case pins the new fact that we fill every role
the profile defines.
`MarmotAgentStreamWatcher` in commons follows the newest kind:1200 in a
group, folds the QUIC records behind it under the receive discipline, and
settles the result against the durable kind:9 — confirmed when the
transcript agrees, dropped when it does not, because a disagreement means
we rendered something the publisher never sent. Resolving the final
message lives here rather than in the UI so a front end only has to say
"the feed moved", and so the whole decision is testable without a UI.
Android shows it as an italic, labelled row between the transcript and
the composer. Provisional content has to look provisional: preview text
is not durable history until the final message vouches for it, and the
row disappears the moment it is confirmed or contradicted. Progress and
status records render as separate chrome, never as answer text, which is
what the spec requires of them.
Every failure path ends as "no preview" rather than as a broken group: no
stream, no broker candidate, an unreachable broker, an unimplemented
stream type, or a platform with no QUIC at all. `receive` explicitly does
not require the QUIC data plane.
The desktop app has no Marmot chat screen to render into — its chat UI is
NIP-17 only — so there is nothing to wire there yet. The watcher is in
commons and speaks only quartz's transport port, so desktop inherits it
the day that screen exists.
Interop unchanged at 19 of 19 with all three roles advertised.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Typing `from:` or `to:` in the spotlight opened nothing — the field was
constructed without `people`, `displayName` or `onPeopleQuery`, so the
picker had no rows to offer and a key had to be pasted in as a raw npub.
It now builds the same UserSearchEngine the full search screen does, with
the same DesktopRelayUserSearchDelegate behind it: cache hits first, then
whatever the account's search relays answer with. The built-in `people`
list is used rather than the `peoplePicker` slot, because that path is the
keyboard-walkable one — arrows walk the rows and Enter takes the highlighted
one, which is what a spotlight is driven by. `displayName` also gives
finished key chips a name instead of a short npub.
The input row switches from centre to top alignment. The picker opens below
the field inside the same column, so centring would drag the leading search
icon halfway down it; with no picker up the two are the same height and the
resting layout is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
firstDayOfWeek() returned a hardcoded ISO Monday on iOS. It now reads
NSCalendar.currentCalendar.firstWeekday, converting Foundation's 1-based
Sunday index to this API's 0-based one, so a reader in the Americas, East
Asia or the Middle East gets the column order their region actually uses.
Read per call, so a settings change lands without a restart.
The rest of the day arithmetic moves to ZoneMath in commonMain, behind a
ZoneOffsets fun interface, leaving the iOS actual holding only Foundation
lookups. The reason is that an Apple source set compiles off a Mac but
never runs off one, so anything expressed in platform calls is unexercised
until someone opens Xcode.
Moving it also fixed a bug the old two-pass had. Resolving local midnight
is a fixed point, not a subtraction, and the two probes oscillate when
midnight does not exist at all — a zone that springs forward *at* midnight,
which Santiago, Havana and Tehran have all done. The old code took the
earlier probe, an hour before the day starts; it now detects the
non-convergence and takes the later one, the first instant of the day that
exists. The repeated-midnight case already converged on the earlier of the
two, matching java.time's atStartOfDay, and still does.
ZoneMathTest covers a fixed-offset zone, a 23-hour spring-forward day, a
25-hour fall-back day, both midnight edges, dayAt either side of local
midnight, and a full-year sweep asserting every day starts exactly where
the one before it ended.
Verified: 7/7 ZoneMathTest, 12/12 SearchCalendarTest, and both
compileKotlinIosArm64 and compileKotlinIosSimulatorArm64 build clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
`directRouteResolver` auto-navigated on any nip19 code found *anywhere* in the
box. `Nip19Parser` extracts a code from inside a string, which is right for
spotting a mention in a note and wrong for reading a search field — so once
`from:`/`to:` existed, typing `from:npub1…` looked exactly like pasting a profile
and threw the reader onto that person's page halfway through writing a filter.
The profile/post auto-navigation is gone. Invite links stay: they cannot be typed
by accident (both need a URL carrying `/invite/`, which no token produces) and
they open a redeem flow rather than a profile or a post.
Deleting it alone would have regressed pasting, though — the old code read
"navigate on hit without displaying results", so the jump *was* the only path to a
pasted code, and a paste of somebody the cache had never seen would have surfaced
nothing at all. A whole-input code now resolves into the results list instead:
consumed into the cache and offered as an ordinary user or note row. Still one tap
away, but the reader chooses when to leave.
`wholeInputNip19` in commons is the rule that separates the two cases — a paste is
the only thing in the box, a token always carries its prefix. Desktop's
`parseSearchInput` had the same extract-from-anywhere behaviour and now goes
through it too, so a `from:` filter no longer produces a direct-lookup row for
somebody the reader was merely filtering by.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
The transport was there and the codecs were there; nothing joined them to
a group. Now `amy marmot stream start|send|watch|finish` does: a hidden
kind:1200 anchors the stream over MLS, records ride raw QUIC through a
broker, and a kind:9 closes it carrying the transcript a receiver checks
its own fold against.
`AgentTextStreamSubscriber` is the receive discipline the binding spells
out, and it matters because a preview that quietly diverges is worse than
no preview: `seq` accepted at most once and never folded out of order, a
replayed record (which a broker WILL send from the start of its replay
window on reconnect) discarded silently and never stream-fatal, a gap
that cannot be backfilled marking the preview unverifiable because the
transcript hash can no longer complete. Only TextDelta and Checkpoint
reach the answer text — progress and status are chrome the spec forbids
from ever reaching notifications, indexes or automation input.
The start payload also grew the tags it was missing: `stream-type`,
`final-kind` and the optional `parent`, plus the rule that a final
payload whose kind disagrees with `final-kind` is ignored.
Verified in both directions against MDK in harness tests 18 and 19: `wn
stream verify` confirms our transcript from our own kind:1200 + kind:9,
and our subscriber folds MDK's stream to a transcript hash identical to
the one `wn stream send` computed. That equality is the key schedule, key
context, AEAD, framing and transcript construction all agreeing with an
implementation that is not ours. 19 of 19 harness tests pass, twice.
Two defects only that exercise could have found:
- The epoch belongs to the stream, not to the clock. The record key
context binds mls_epoch, and both sides were resolving it as "the
group's current epoch" at each command, so a commit landing between
the start and the send put them on different keys and produced an
empty preview. The epoch that DELIVERED the kind:1200 is the
stream's; it is persisted with the message now and read back by
publisher and receiver alike.
- close() dropped the tail of a stream. enqueue only fills the send
buffer, so tearing the connection down before the driver flushed it
lost records silently — the publisher had already counted them. QUIC
ACKs a FIN only once everything ahead of it arrived, so finish() now
waits for finAcked. This is exactly why the test passed alone and
failed inside a full run.
The `send` (0xF2D2) and `fanout` (0xF2D4) role capabilities stay
unadvertised: a role is a promise to the whole group, and only the CLI
originates a stream so far.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`forEachIndexableField` shipped on seven kinds — the ones local search happened to
scan — leaving 126 on the default that falls back to `indexableContent()`. All 133
implementors now have one.
The conversion is worth nothing if it changes what a store indexes: the SQLite and
filesystem stores index through `indexableContent()`, and
`references/searchable-kinds.md`, which external engines mirror at version bumps,
is a transcription of it. A silent change there ships stale results downstream and
needs a `reindexFullTextSearch()` on every existing database.
So the output of all 126 kinds was recorded BEFORE any edit and diffed after every
wave. `indexable-content.golden` keeps that recording as a permanent guard, and it
is byte-identical to the pre-change baseline. Two more tests join the golden one:
every kind's visitor rejoins to exactly its `indexableContent()`, and every kind
actually stops when the visitor says stop — a class that ignored the stop signal
would still agree about its content, so nothing else would catch it.
Most of the work was mechanical and done by script — 31 content-only, 65
`listOfNotNull(…).joinToString`, 14 list-append and nullable-single shapes, 10
JSON-backed `?.let { … }`. The last six were hand-written because their joined
form is not a plain list: `LabelEvent` filters empty strings, so the visitor skips
rather than visits them; the two poll kinds append `content` unconditionally, so
the visitor must offer it even when empty or the separator it produced goes
missing; and `NIP90TextGenerationRequestEvent` filters by input type.
`indexableSeparator()` is added to the interface for the handful of metadata-ish
kinds that join with a space rather than a newline. Nothing on the read path uses
it — it exists so the agreement test can rejoin what the visitor hands over and
prove it reproduces the indexed string exactly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
The transport binding was the last piece missing from agent text streams,
and it did not need a new QUIC implementation — `:quic` already had the
whole hard part. What it needed was entering that stack at the right
layer.
`nestsClient` speaks WebTransport: HTTP/3, Extended CONNECT, QPACK,
SETTINGS. Its `WebTransportSession` abstraction begins above all of that.
Marmot's binding is raw QUIC — it negotiates its own ALPN
(`marmot.quic_broker.v1` / `marmot.quic_stream.v1`) and writes frames
straight onto QUIC streams, with no HTTP/3 anywhere in it. So this reuses
everything below that line — connection, TLS 1.3, ALPN negotiation,
stream multiplexing, loss recovery, the UDP socket — and none of the
WebTransport wrapper.
The codecs are in quartz next to the rest of agent-text-stream, because
they are pure bytes and that is where the conformance risk lives: the
control envelope with its literal 21-byte protocol string and its
trailing-byte rejection, the uint32 frame codec with both the broker's
blind cap and a policy-aware one, `quic://` candidate parsing down to
ignoring everything after the authority and never sending an IP literal
as SNI, and the first record's stream id pinning the rest.
`:marmotQuic` is the connection layer, mirroring how `:nestsClient` sits
on `:quic`. A publisher claims a room on a uni stream, a subscriber reads
the fan-out on a bidi one, and an endpoint that does not take our ALPN is
reported as unusable so the caller moves to the next candidate rather
than waiting on records that never come.
Verified against MDK's own `marmot-quic-broker`, which is the only way to
know a wire format is right: our publisher and subscriber meet inside the
reference broker, the records come back, open under the group-derived key
and fold to the publisher's transcript hash, and the broker keeps rooms
apart. Opt in with -DmarmotQuicBroker=host:port; the cases skip visibly
without one, so an ordinary test run needs no broker.
Still not wired at the app layer: nothing yet mints a kind-1200 start,
picks a candidate, or renders a live preview, so `send` (0xF2D2) and
`fanout` (0xF2D4) stay unadvertised. The direct path has no start-payload
candidate format in v1 and is unimplemented.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
A relay that drops the socket between our EVENT frame and its OK has told
us nothing: the event may be stored, or it may not. We recorded that as
the relay's verdict and stopped waiting — even though the pool's own
outbox still owed the relay the event and would have flushed it on
reconnect. Nobody was listening by then, so the publish came back failed
and the event landed on the relay a second later anyway.
publishAndCollectResults now holds a transport failure as provisional for
one retry: it drops the tentative verdict, ignores the echoes of the same
drop, clears the backoff and dials, and takes the OK when the pool's
flush earns it. Everything happens inside the caller's existing timeout,
so no publish waits longer than it used to, and a relay that keeps
hanging up is still reported as a transport failure rather than a
success. transportRetries = 0 restores the old behaviour exactly.
Found through the Marmot interop harness, which was losing a message
every few runs to a loopback relay that was healthy a second later. The
same race is every publish that meets a network change on mobile.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Publishing preview records was blocked on one thing: the spec requires a
publisher to never restart or reuse a `seq` for one key context —
"including after reconnect, retry, process restart, or daemon resume" —
and to stop publishing entirely when it cannot prove which value is next.
That is a cryptographic requirement, not bookkeeping. `seq` is XORed into
the ChaCha20-Poly1305 record nonce and the key is fixed for the stream,
so a repeated `seq` repeats a (key, nonce) pair, which leaks the XOR of
the two plaintexts and forfeits authentication for every record under
that key.
`AgentTextStreamPublisher` owns that discipline. Sequence values are
reserved in the durable store before a record is handed out, in windows
so a chatty stream is not a write per record — a crash then skips the
unused tail of a window rather than replaying it, and a gap is something
the transport binding already handles while a repeat is a nonce
collision. `resume` returns null, rather than starting over at 1, both
when nothing was retained and when the stream was finished or aborted;
the caller falls back to the authoritative final kind:9 and a later
preview needs a fresh stream id. A frame the group's
`max_plaintext_frame_len` refuses is rejected before it claims a value,
so a refused frame does not leave every receiver with a permanent gap
where no record ever existed.
Only an in-memory sequence store ships here. `send` (0xF2D2) and
`fanout` (0xF2D4) stay unadvertised: there is no QUIC data plane behind
them yet, and claiming a role we cannot serve is worse for a group than
not claiming it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
MIP-00 replaces a KeyPackage as soon as a Welcome consumes it, so rotation
is not a rare path — it runs right after the first group we are ever
invited to. It went through the legacy generator, so from that moment on
the only KeyPackage on relays for us was a MIP-era one with no account
identity proof. A current-profile peer refuses that outright ("member
KeyPackage identity or profile is invalid") and keeps inviting from
whatever stale copy it still has cached, so an account went silently
uninvitable one join after it was set up.
Rotation and first publication now share one mint path, so a replacement
cannot land on a different profile than the KeyPackage it replaces.
Harness, two tests that were reporting our bugs as theirs and one that
was reporting the reverse:
- Test 16 asked `wn keys publish` to rotate. That verb is the
idempotent retry of the durable stable-slot replacement — with
nothing pending it republishes the same event id, so there is no
rotation to observe. `wn keys rotate` is the one that mints.
- Test 13 swallowed `wn keys check`'s output, so "no prior KP for A"
read as a missing fixture when it was MDK refusing what we had
published. The raw answer goes to the log now and the message says
what actually happened.
- Test 09 polled `reactions.by_emoji`, which belongs to the
materialized timeline; `wn messages list` reads the raw app-event
log, where a reaction is its own kind:7 entry with an "e" tag naming
the anchor. The reaction had been arriving and being stored
correctly the whole time.
The MDK 0.9.20 interop harness is now green, 17 of 17, twice in a row
from a clean state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
We publish every KeyPackage marked last resort, and then threw away its
private keys the moment one Welcome consumed it. Those two things cannot
both be true. OpenMLS is explicit about the contract — on the Welcome
path it deletes the consumed bundle only `if !key_package.last_resort()`
and otherwise logs "KeyPackage has a last-resort marker, not deleting" —
and MDK leans on it: it marks all of its own KeyPackages last resort,
caches the peer KeyPackage it resolved in its user directory, and invites
from that same cached copy every time after. So the first invite
addressed to us worked and every one after it died on "No matching
KeyPackageBundle", which is four of the interop harness's failures.
Consumed bundles now stay reachable when the KeyPackage says they may
be, bounded on both axes: at most eight of them, and never past the
KeyPackage's own not_after. That retention is the entire forward-secrecy
cost of the last-resort marker, and it is a cost we already accepted by
publishing the marker.
Two things had to be right for it to work at all:
- `isLastResort()` has to read both carriers. The MIP-era profile sets
MLS extension type 0x000A on the KeyPackage; the current profile —
the one we actually publish — carries a `last_resort_key_package`
component inside the KeyPackage-level app_data_dictionary. Reading
only the first made every KeyPackage we ship look single-use.
- The Welcome lookup has to trust the MLS refs over the Nostr "e" tag.
RFC 9420 addresses each EncryptedGroupSecrets to a KeyPackageRef and
the joiner takes the first it holds keys for; the "e" tag is a
routing hint an inviter can get wrong, and MDK gets it wrong exactly
here — it stamps the event id of its cached copy, which is stale the
moment we rotate. Refs first, tag as fallback.
The restore path also stopped throwing the whole snapshot away when the
eventId→slot index is empty. It drops the active bundles that index made
unreachable, and keeps the retained bundles (keyed by event id, always
reachable) and the named slot d-tags (a fresh d-tag would republish into
a new addressable slot and orphan the old one).
Harness: reset A's amy home and the relay database at the start of every
run, keeping the relay build. wnd already wiped B's and C's data dirs,
but A's store and the relay's events survived, and the leftovers are not
inert — a KeyPackage from an earlier run is still on the relay to be
invited with, and old kind:445 events still arrive undecryptable. That
drift alone accounted for tests 03 and 08. `--reuse-state` opts out and
`--tests "..."` runs a subset.
Interop: 10 → 14 of 17 passing. 05, 12, 14 and 15 (every "A never
received invite") now pass, as do 03 and 08.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Two MLS defects that between them meant nobody but a group's creator could
successfully invite anyone into a group with three or more members.
**GroupSecrets.path_secret was neither sent nor used.** A Commit's
UpdatePath refreshes every node from the committer's leaf to the root, and
a refreshed node has no unmerged leaves — so a member added by that same
Commit is MERGED at their common ancestor the instant it joins. RFC 9420
§12.4.1 also excludes newly-added leaves from the copath resolution, so
that ancestor's secret is not in the UpdatePath at all. The only place it
exists is `GroupSecrets.path_secret` (§12.4.3.1), and we sent `null` and
ignored what MDK sent us. The joiner therefore held nothing above its own
leaf, and the first commit from the other side of the tree — which
resolves the joiner's sibling subtree to that merged ancestor — was
undecryptable. MDK reported it exactly:
UpdatePath at common ancestor carries no ciphertext for us
(my_leaf=1, my_node=2, resolution=[1], held_path_nodes=[])
**Parent-hash validation was stricter than RFC 9420 and rejected valid
trees.** We re-derived every COMMIT-source leaf's `parent_hash` top-down
from the CURRENT tree and demanded a match. §7.9.2 makes a much weaker
claim, per PARENT node: for each non-blank parent P, exactly one of its
subtrees must contain a node whose `parent_hash` equals `ParentHash(P,
other_subtree)`. The strong version cannot hold — a later commit
refreshes ancestors and a later Add changes the tree's shape, so a leaf
set two epochs ago legitimately no longer re-derives — and it rejected the
GroupInfo of every group whose inviter was not the last committer.
Both halves need the RFC's `original_sibling_tree_hash`: the sibling
subtree's tree hash with the parent's `unmerged_leaves` removed. Those are
exactly the leaves added since the parent was populated, so excluding them
reconstructs the tree as the parent's author saw it. `RatchetTree` gains
`originalTreeHash` and `resolutionExcluding` for it.
The regression test builds the case that no two-party test can reach: a
member added by its own sibling, so its ancestor is merged on arrival,
followed by a commit from the other subtree. It fails on either half of
this change alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
RFC 9420 §7.6 does not say the committer encrypts the path secret to your
leaf. It says the committer encrypts one secret per node in the copath
RESOLUTION, and each member decrypts at whichever of those nodes it holds
a private key for. A merged subtree resolves to its PARENT, so from three
members on the ciphertext meant for us stops naming our leaf at all.
We kept only our own leaf key and looked ourselves up by leaf index. That
worked for two members and failed for three, which is exactly why it
survived every test we own: two-party tests never produce the case. MDK
did, on its first commit after a three-member Add:
UpdatePath at common ancestor carries no ciphertext for us
(my_leaf=1, my_node=2, resolution=[1], encrypted_path_secrets=1)
Node 1 was the parent we had held a key for since the commit that merged
us, and we had thrown it away.
`MlsGroup` now keeps the private halves for its whole direct path — filled
on our own commits from the path secrets we mint, and on inbound commits
from the secret we recover at the common ancestor — and scans the
resolution for a key it holds rather than assuming its leaf. Candidates
are tried in order rather than committing to the first: an Add or Remove
renumbers nodes, and a stale key fails the AEAD instead of producing a
wrong secret, so trying the next one is exact. `MlsGroupState` v3
persists them; losing them to a restart would make the same group stop
decrypting on relaunch with nothing tying the failure to the restart.
Also completes the durability and lifecycle work:
- `PublishOutcome.UNKNOWN`. A non-confirmed publish used to discard its
obligation and return the group to Stable, which let a REPLACEMENT
commit be prepared for the same epoch. "No OK arrived" is not "no
peer took it" — a timeout or a dropped connection leaves it unknown,
and a second commit for an epoch a peer already holds is precisely
the fork this gate exists to prevent. The obligation now stays
durable and the group stays held. `FAILED` remains for the case where
retrying is genuinely impossible.
- Publish obligations are durable in the CLI and on Android, and
`restoreAll` republishes each unresolved one VERBATIM. The same bytes,
not a fresh commit: a peer that already has the event deduplicates it.
- `Disbanded` and `Unrecoverable` now gate rather than describe.
Convergence terminalizes a group when an applied commit's lifecycle
component says so, and marks one unrecoverable when a selected branch
cannot be rebuilt from retained material — the one thing a client must
not do there is keep its own losing branch and call that settled.
Outbound work and inbound application are both refused in those
states, and `MarmotManager.lifecycle` now merges the publish gate's
view with convergence's instead of reading only the former (which
reported Stable for a disbanded group).
- Durable ingest markers. A relay `since` cursor cannot skip a backdated
event, and NIP-59 wraps are backdated by up to two days on purpose, so
every wrap in that band was unwrapped, decrypted and re-decided on
every single sync — forever. Only outcomes that cannot change are
marked: a Welcome we joined from, and one naming a KeyPackage whose
private half we never held. An event that is merely undecryptable
right now is not marked, because a kind-445 under a future epoch
becomes readable the moment its commit arrives.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Everything that touched a group's name, admins, relays or avatar went
through `groupMetadata`, which decodes ONLY the legacy `0xF2EE`
extension. It returns null for every current-profile group, so:
- `amy marmot group show` / `list` / `admins` printed a blank name and
an empty admin set;
- `amy marmot await group --name X` never matched, which is what test
03 was actually reporting — we had joined MDK's group, we just could
not find it by name;
- the Android chatroom showed no name, no admins, no relays, no avatar;
- `group rename` / `promote` / `demote` / `set-image` BOOTSTRAPPED a
legacy blob and committed it into a current-profile group, so the
rename appeared to work locally while every peer kept the old name.
Adds `MarmotManager.groupView` (read) and `setGroupProfile` /
`setGroupAdmins` / `setGroupImage` (write). The setters dispatch on the
group's actual profile: a current-profile group takes an
`app_data_update` naming ONE component, so a concurrent admin-policy
change does not lose its work to a rename; a legacy group has no such
separation and its single extension is rewritten whole. Every call site
in the CLI, the Android app and the relay-subscription manager now goes
through them.
`createMarmotGroup` creates a CURRENT-profile group. The profile is
decided once, at creation, and cannot be migrated later — a legacy
group's existing leaves have no account identity proofs to add — so a
group made the old way is joinable only by other legacy clients. The
name and description are passed in at creation because the routing
component has to exist from epoch 0 anyway: it carries the
`nostr_group_id` every kind-445 event in the group is addressed to.
`MarmotGroupIconUpload` gains `mediaType`. MIP-01's image blob never
carried one; the current profile's `0x8002` component requires it on a
present image and binds it into the AEAD's AAD, so a receiver cannot be
steered into decoding the plaintext as a different type than the
uploader meant.
Also: a gift wrap is no longer broadcast to the public default relay set
when the recipient advertised an inbox we declined to reach. "Advertised
nothing" and "advertised only local-network relays" are different facts,
and treating the second as the first sends someone's invite to a relay
set they never chose — the opposite of what the filter is for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
MDK requires component `0x8006` in every group it creates, and its
`required_member_roles` mask names MLS leaf capabilities each member must
advertise. A client that carries neither is refused at the Add — which is
why `wn groups create "Interop-03" <us>` failed outright, taking five
interop scenarios with it. The `0xf2d1`/`0xf2d2`/`0xf2d4` extensions MDK
advertises are exactly those role capabilities.
Implements the component and the record layer it gates:
- `AgentTextStreamQuicPolicyV1` — the 12-byte component state, decoded
strictly (a short, long, or out-of-range payload is rejected, never
defaulted: these bytes sit in signed group state and a guessed role
mask admits a member the group refuses).
- `AgentTextStreamRecordV1` — the wire record, with the QUIC varint
length prefixes the Marmot binary profile uses. Unknown record types
decode fine on purpose; a newer advisory record must not tear down an
otherwise valid preview stream.
- `AgentTextStreamCrypto` — HKDF-Expand-only key and nonce derivation
over the full key context, `nonce_base XOR uint96_be(seq)`, and the
record AAD. `seq` is in both the nonce and the AAD, so a replayed or
reordered record fails to open rather than being noticed afterwards.
- `AgentTextStreamTranscriptV1` — the rolling hash the final kind-9
chat publishes, so a receiver can tell that it saw exactly the stream
the publisher sent.
- `AgentTextStreamStart` / `AgentTextStreamFinal` — the kind-1200 anchor
tags and the kind-9 closing tags.
We advertise the RECEIVE role only, and the group state validator refuses
a Welcome whose policy requires a role we do not advertise. Publishing
would need durable per-stream sequence state to avoid reusing an AEAD
nonce across a restart, and we have none — advertising `send` without it
would be a claim we cannot keep.
Also fixes three places that read only the legacy `0xF2EE` extension and
therefore did nothing at all on a current-profile group:
- The Welcome's `nostr_group_id`, which is the `h` tag every kind-445
event carries. Without it a joiner cannot subscribe, so MDK's welcome
decrypted and was then discarded — "GroupContext is missing the
NostrGroupData extension" — for a routing id that was present the
whole time in the `0x8004` component.
- The admin gate on GroupContextExtensions changes, which read
`adminsConfigured` as false for every current-profile group and so
skipped the check instead of failing closed.
- Disappearing-message expiration, which silently never applied.
And `syncMetadataTo`, which left every current-profile group with a blank
name, no admins, no relays and no avatar in the UI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The harness was parsing a wire format `wn` no longer speaks, and every
mismatch failed silently as "nothing arrived".
`wn --json groups invites` now answers
`{"ok":true,"result":{"account_id":…,"invites":[…],"npub":…}}`. The
harness iterated `(.result // .) | .[]?`, which walks that object's three
VALUES — two strings and an array — so `jq_group_id` matched nothing on
every poll and the pending count printed 3 forever. Test 02 reported "B
never received invite" for welcomes that had in fact arrived.
Named collections moved the same way (`members`, `admins`, `messages`),
per-entry id fields were renamed (`member_id`, `admin_id`,
`message_id`), the decrypted body is `plaintext` rather than `content`,
the group display name lives in the profile component
(`.group.profile.name`), and `keys check` nests its event id under
`.key_package`.
Adds two helpers so this is fixed in one place rather than at 20 call
sites: `jq_list <name>` peels the envelope and names the collection, and
`jq_member_ids` reads whichever id field the collection uses. `jq_group_id`
now searches `.result.group_id`, `.result.group.group_id` and the bare
element shape, keeping the older serde encodings so a run against an
older `wn` still reports a real mismatch instead of an empty string.
Also fixes test 05, which fed wn's MLS group id to `amy marmot message
send`; amy indexes by the MIP-01 nostr_group_id, which it only learns
from its own `await group`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
A Commit replaces the committer's leaf through the UpdatePath. It is the
same member with new key material, so everything the leaf says about the
member has to survive — but `buildLeafNode` was called without either
`capabilities` or `leafExtensions`, so it fell through to the legacy
defaults every time.
That cost us every invitation we have ever sent to MDK. A current-profile
leaf carries `marmot.member.account-identity-proof.v2` inside an
`app_data_dictionary` LEAF extension, and no proposal can put a leaf
extension back, so our very first Commit silently demoted the group
creator out of the current profile. The rebuilt leaf also stopped
advertising the `app_data_dictionary` extension (0x0006) and the
`app_data_update` proposal (0x0008) that the group's own
`required_capabilities` demands, which makes the resulting tree fail
RFC 9420 §7.3 leaf validation for every receiver. MDK reported
`PublicGroupError(LeafNodeValidation(UnsupportedExtensions))` and dropped
the Welcome minted by that same commit — the invitee simply never saw an
invite, with nothing logged on either side.
The same omission was in `proposeSigningKeyRotation`, where an Update
proposal replaces our leaf for forward secrecy, and in `externalJoin`,
which had no way to express a current-profile joiner leaf at all.
Verified against MDK 0.9.20 on the interop harness: before, the invitee's
pending-invite list stayed empty; after, our group arrives with its
routing, profile and admin policy intact.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Findings from auditing the branch against the real corpus (897 kind-30040
indexes and 820 kind-30041 sections pulled off thecitadel, theforest, damus and
nos.lol), not against the sample events.
**Every table of contents read "Untitled section".** NKBIP-01 lets an index
carry each entry's title in the `a` tag's slot 2, and parsing it is why that
slot is disambiguated from a relay hint and a nesting level. Across those 897
indexes, not one of 11,176 entries uses it: every publisher writes
`["a", coord, relay, <event id>]`. So a 240-chapter book showed 240 rows of
"Untitled section" until 240 separate fetches came back. The coordinate's own
`d` is a slug of the title (`pg59225-chapter-1-introduction`) and
`humanizeIdentifier` already existed, so rows now name themselves immediately
and for free. The section's real title still wins the moment it lands.
**`[%hardbreaks]` was dropped, running verses together.** Asciidoctor honours
it and the reference client renders a break per line; the converter treated it
as an ordinary block-attribute line and CommonMark then joined every line into
one paragraph. The KJV chapters are published exactly this way. Both the block
option and the `:hardbreaks:` document attribute now emit CommonMark hard
breaks, with the block form lapsing at the end of its own block.
**The thread view composed the whole contents at once.** It passed
`Int.MAX_VALUE`, and that table is a plain `Column` inside one lazy-list item,
so every row composed together and each opened its own event observation and
relay subscription. 234 of the 897 indexes list more than 12 sections, 21 list
more than 100, the largest 240. Capped at 60, which still shows almost every
publication whole; past it the count is shown and the pager walks the rest.
**The AsciiDoc conversion ran uncached on the composition thread.** `remember`
is dropped when a card leaves a lazy list, so scrolling back re-converted.
Measured over the 638 non-empty sections (1.9 MB of prose) on a desktop JVM:
p50 114us, p95 1.4ms, worst 7.2ms for a 13.5 KB section — and a phone is
several times slower than that. Now behind an LRU keyed by event id, trimmed
with the other caches under memory pressure.
Also checked and found clean: the converter is stable on the whole corpus (no
unbalanced emphasis introduced, no output corruption) and has no catastrophic
backtracking — every adversarial input (2–10 KB of `*`, `_`, unterminated
macros and fences) converts in under 1ms.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
`amy marmot group create` now builds a current-profile group; `--legacy`
keeps the MIP-era path for reproducing groups already on disk. The
difference is what a group REQUIRES of a joining leaf — the account
identity proof, which every conformant peer's KeyPackage carries, versus
`0xF2EE`, which none of them do. With this, `group add` accepts an MDK
KeyPackage where it previously failed the capability gate outright.
Making that work surfaced two bugs that would each have been fatal on
their own.
`MarmotManager.groupRelays` read only the legacy `0xF2EE` extension, but
a current-profile group routes through `NostrRoutingV1` (`0x8004`). Every
current-profile group therefore had an EMPTY recipient scope — so under
publish-before-apply no commit could ever be acknowledged, and no such
group could ever advance past epoch 0. It now reads both.
And the local-network relay filter turned up a third time, in NIP-65:
`parseReadNorm`/`parseWriteNorm` dropped loopback entries, so our own
outbox and inbox read as empty while `nip65` showed the relay. Everything
then published to the default relay set — which is why commits and
Welcomes were going to public relays instead of the harness's loopback.
Same split as before: filtered for someone else's list (it is
attacker-supplied input, and it is what exempts a relay from Tor),
unfiltered for reading back our own.
Two conformance fixes came with it. The Welcome rumor carried an
`encoding` tag, which the binding forbids outright for every event shape
it defines — a receiver that switched decoders on one could be steered
into a different parse of the same bytes. And we implemented
encrypted-media v2 last commit but never advertised `0x800b` in the leaf,
so a group requiring it would refuse us; the advertised list now carries
it, with a note that an id belongs there only when the component is
actually implemented.
`MarmotMipBehaviorTest` asserted the MIP-era rule that a rumor MUST carry
an encoding tag. The adopted binding reverses it, so the test now asserts
the current rule.
Interop test 01 still passes. Test 02 (invite MDK into our group) reaches
MDK but is not yet ingested; a control run confirms MDK->MDK invites work
in this harness, so the remaining defect is ours, in the Welcome.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
These run on every filter rebuild, several times a second, and the previous shape
was a chain of operators that each allocated an intermediate: `partition`,
`groupBy`, `map`, `distinct`, `sorted`, `chunked`, `map`, then a list concat.
`groupBy { kind to pubKeyHex }` allocated a `Pair` and a boxed `Int` per address
on top of that.
Now one pass into nested maps, sorting in place, appending straight to the output
list. `distinct()` went entirely: the input is a `Set<Address>` and `Address` is a
data class, so two entries in one (kind, author) group cannot share a `d` -- it
was dead work every call. The two group maps are allocated only if that kind of
address turns up, and `forEachChunk` hands the whole list through untouched when
it already fits, which is nearly always, instead of `chunked` building an outer
list and a copy.
Measured, same machine, JVM, against the previous implementation:
8 addresses (typical), 200k reps : 368.8ms -> 75.8ms 4.9x
240 addresses (a book), 5k reps : 132.5ms -> 52.0ms 2.5x
The small case gains most, which is the one that runs constantly -- 1.84us to
0.38us per call. Behaviour is unchanged: same 7 tests, and the 240-section index
still renders all its rows on device.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
The previous commit justified the chunk with "relays cap the values they will
accept in a tag filter". No relay was observed doing that, and no such cap
exists: neither NIP-11 nor `RelayLimits` has a values-per-filter field, and
`LimitsPolicy` never checks one. That was inferred from the `chunked(100)` the
other bulk builders use, and stated as fact. It was not.
The mechanism that does exist is the limit clamp. `LimitsPolicy.applyLimits`
rewrites a filter's `limit` down to the relay's `maxLimit`, so the addressable
group -- which sets `limit` to its coordinate count -- would ask for 240 and be
answered with `maxLimit` of them, the remainder missing silently. Chunking keeps
each `limit` under the clamp. That is the real reason, and it is now what the
comments say.
The chunk size itself stays convention, and says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Test 01 (bidirectional KeyPackage discovery with MDK) now passes. It was
failing on a four-byte omission.
`foundation/key-packages.md`: "a transport publication is unambiguously
the framed MLSMessage, not a bare KeyPackage struct." We published bare
bytes. That is not a cosmetic difference — a reader expecting the envelope
reads a bare KeyPackage's leading 0x0001 0x0001 as version 1, wire format
1 (mls_public_message), parses on as a PublicMessage, and dies several
fields later on a byte that means nothing. MDK reported
`UnknownValue(112)`, a number that appears nowhere in a KeyPackage, which
is why this was invisible from our side: our own decoder round-tripped our
own bytes perfectly. Only a second implementation could find it.
The KeyPackageRef stays over the INNER KeyPackage, as RFC 9420
MakeKeyPackageRef defines — framing the ref too would make our `i` tag
disagree with everyone else's. Bare bytes are still accepted on read:
every KeyPackage we published before this is bare and still inside its
lifetime, and refusing them would leave our own users unable to invite
each other until all of them rotated.
With framing fixed MDK got one field further and rejected the next thing:
"mls_extensions tag does not exactly match decoded KeyPackage metadata".
Those id-list tags duplicate metadata already inside the KeyPackage, and
we were writing them by hand — so adding one leaf capability (the legacy
0xF2EE, added so our KeyPackages stay addable to existing groups) silently
invalidated every KeyPackage we published. They are now derived from the
KeyPackage itself and cannot drift. `app_components` lists the Marmot
registry ids only; the upstream MLS-extensions component ids below 0x8000
are not app components being advertised.
The harness needed one more fix: MDK 0.9.x reports a found KeyPackage
under `result.key_package`, and the harness probed two older shapes, so a
successful check read as a failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Audit of the previous commit turned up three problems.
A stray `i` tag was being read as the next species' reference. Birdstar
writes `i`/`n` as adjacent pairs, but the pairing only tracked "the last
`i` seen", so an unrelated reference earlier in the event — a NIP-73
identity for the event itself, say — attached to whatever name came next
and pointed its link at the wrong page. Adjacency is now required.
Clicking a link could take the app down. Compose's own handling of
LinkAnnotation.Url swallows IllegalArgumentException and nothing else,
while DesktopUriHandler builds a java.net.URI (URISyntaxException on a
malformed URL) and can throw UnsupportedOperationException or IOException
besides — and these URLs come from a stranger's event. The click now goes
through the same runCatching that ClickableUrl has always used.
Expanding no longer dumps the whole list into one feed row: Compose lays
out an interaction region per link, so "+N more" reveals a page of 30 at a
time and a full expansion no longer costs hundreds of composables in a
single tap. "Show less" folds it back.
Also: speciesCount() counted `n` tags by building the list of names and
taking its size; it now counts them in place. Rebuilding the shown sublist
on every recomposition is gone with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUzRwL1XtMad5xZepJHock
The preview suffix was replaced by a standalone "+N more" toggle, so the
old key is gone from the default catalog. Retiring it in the same push
keeps lint from reporting 55 [ExtraTranslation] orphans.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUzRwL1XtMad5xZepJHock
A Birdex (kind 12473) carries its species list as positional `i`/`n` tag
pairs — a Wikidata entity URL next to each scientific name — but the card
rendered the names as one flat, comma-joined string, so the references in
the event were unreachable from the UI.
BirdexEvent gains species(), which pairs each `n` with the `i` next to it
(either side, since the pairing is positional) and keeps only references a
UI can open, matching what BirdDetectionEvent already does for a single
sighting; both now share that http(s) check.
The card renders those pairs as italic scientific names — every one with a
reference a link to its Wikidata entry — and, because a life list grows
without bound, keeps showing 6 up front behind a "+N more" toggle that
expands the rest in place instead of truncating them away.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUzRwL1XtMad5xZepJHock
The harness had never actually been run. It now builds MDK 0.9.20 —
against the same OpenMLS fork rev our vector generator pins — boots a
local relay, brings up both wnd daemons and amy, and executes all 17
scenarios. They all still fail, downstream of MDK not finding A's
KeyPackage, but "it runs" is the difference between having an interop
signal and not having one.
Four environment blockers stood between preflight and a run: protoc is
now a build prerequisite; MDK 0.9.x needs WN_ALLOW_LOOPBACK_RELAYS=1
before it will accept a ws:// loopback relay at all; it refuses to create
its socket unless the parent directory is 0700; and `wn --json whoami`
moved to {"ok":true,"result":{"accounts":[…]}}, which the harness's
extractor probed right past.
Two defects in our own code came out of it.
`amy relay add` reported success from its DECISION to write rather than
from the store's answer, so a rejected or no-op write printed
`added: yes` and the caller only discovered otherwise much later.
The more consequential one: we read our OWN relay lists back through the
local-network filter. That filter is correct for someone else's list — it
is attacker-supplied input, and it is also what exempts a relay from Tor
— but applied to a list we published ourselves it made a deliberately
configured local relay look like no configuration at all. The publisher
then fell back to a default set, and the harness sent A's KeyPackage to
five PUBLIC relays instead of its loopback, which is the exact opposite
of what a "nothing leaves the machine" harness is for. `allRelays()` now
exists for reading back our own lists; the KeyPackage publish goes only
to the configured relay.
Test 01 is still blocked on a narrower puzzle: the kind-10051 list
persists under `relay key-package set` but not under `relay add`, while
kind 10050 works through the identical code path. That is a storage/CLI
thread, not a protocol one, and it needs its own pass.
Separately, and not a bug on either side: MDK accepts ws:// only for a
loopback host while quartz strips exactly those hosts from relay lists.
No address satisfies both, so a loopback-relay harness cannot pass until
one side moves — and changing a Tor-adjacent privacy guard is a
maintainer call, not one to make in passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The Quartz half of the current profile was ready and tested; nothing in
the app layer called it. KeyPackage publishing now defaults to the
current profile, which is the half that decides whether anyone running
the adopted spec can invite us at all — a leaf without the 0x8009
account identity proof is simply not addable to a current-profile group.
`createCurrentProfileGroup` builds the group through
`CurrentProfileGroupFactory` and hands it to the manager via `adoptGroup`.
Group creation is the one place a group cannot be built through the
manager: a leaf's identity proof covers its OWN signature key, so the
keypair has to be generated and authorized by the account signer before
the leaf exists.
Switching the KeyPackage path surfaced the mirror image of the bug this
whole effort started from. A current-profile leaf advertised only the
draft app_data_dictionary extension, and a legacy group REQUIRES 0xF2EE —
so our new KeyPackages were un-addable to every group that already
exists. Capabilities say "this client can handle it", not "this group
uses it", so the leaf now advertises both. Advertising more than a group
requires is always fine; advertising less is what gets a leaf rejected.
The reference-shape test now states that rule rather than asserting
byte-equality with MDK's leaf.
The current-profile KeyPackage event also carries neither a `relays` nor
an `encoding` tag, both per transports/nostr.md: a KeyPackage is fetched
from the account's own inbox relay set, so repeating the relays would be
a second drifting source of truth, and the binding forbids `encoding`
outright because a receiver that switched decoders on one could be
steered into a different parse of the same bytes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`filterMissingAddressables` built a filter per address. Opening a 240-section
publication therefore put 240 filters -- each `kinds`+`authors`+one `#d`, `limit`
1 -- into a single REQ per relay, when every one of them shares a kind and an
author and differs only in `d`.
Grouping by (kind, author) turns that into one filter carrying every `d`. The
`limit` becomes the number of coordinates asked for rather than 1: these are
replaceable, so a relay holds exactly one event per coordinate and that count is
the ceiling.
Both bulk builders now chunk at 100 values, the size the other bulk filter
builders in this module already use. Relays cap the values they accept in a
filter, and a filter silently truncated loses its tail with no error to notice --
which is the failure this was reported as. The id path had the same exposure: it
already coalesced ids into one filter per relay, but unbounded.
Generic by construction: `filterMissingEventsForThread` calls these same two
functions, so threads get it without touching the thread assembler. Replaceables
with no `d` keep their own group, since kind and author alone address them.
Measured on the 240-section Aeschylus index, cold start each time: a black screen
for ~2 minutes at 120% CPU before, first paint at ~22s after. All 240 rows still
resolve to real titles, with zero "Untitled section" -- and those titles can only
come from fetched sections, since the index's `a` tags carry a relay hint in slot
2, which `fromAddressTag` correctly refuses to read as a title.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Two Stage 7 surfaces, both verified against fixtures the spec publishes
rather than against my own reading of it.
**encrypted-media v2 (component 0x800b).** Supersedes the frozen v1
policy at 0x8008, which must never be reinterpreted as v2. The unusual
rule here is that neither list is sorted: `default_blob_endpoints` order
IS the upload/fetch fallback priority, so sorting it — as nostr-routing
and admin-policy both do — would silently change which server a group
uploads to. Two policies differing only in order are different canonical
values, and the decoder preserves what the producer wrote.
Its field checks look excessive until you see why they exist.
`plaintext_sha256`, `m` and `filename` all feed both the key derivation
and the AEAD AAD, joined by single 0x00 bytes with no length prefixes.
That is unambiguous only because each field excludes 0x00 — fixed-width
hash, ASCII-token media type, filename profile forbidding U+0000. It is
also why a duplicate single-occurrence `imeta` field is rejected rather
than resolved: a first-wins decoder and a last-wins decoder would derive
different keys from the same authenticated tag, so one sender could hand
two conformant clients tags that decrypt to different content.
**Push owner proof (kind 451).** A BIP-340 signature over the id of an
exact, never-published Nostr event. The event id is a ready-made
canonical digest over the tuple that needs binding, and binding it is the
whole point: because the id covers group_id, server_pubkey, relay_hint,
the encrypted token and owner_ts, a member who merely RELAYS someone's
record cannot move it to another group, repoint it at a different
notification server, swap the token, or restamp it. A record's authority
comes from owner_sig and current membership, never from who carried it.
A current-profile group accepts only kind 451; a legacy group also
accepts the superseded kind-450 form so upgraded and un-upgraded members
can share a group. That split is a security boundary, not a courtesy —
in a group where every leaf already carries a 0x8009 identity proof,
accepting the weaker form would let anyone able to produce one bypass the
stronger binding.
Both fixtures reproduce exactly: the spec's published removal event id
and its owner_sig verify under our tag construction, which is what proves
tag order, arity and value formatting are right rather than merely
self-consistent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Marmot app payloads are a Nostr event MINUS the signature, and we were
sending them WITH one. A conformant decoder rejects a payload carrying a
`sig` member at all, so every message we sent was refusable by any peer
following the adopted spec — and a signed inner event is a valid
standalone relay event, so one leaked plaintext could be republished
publicly as a signed statement by its author.
`MarmotAppEvent` is that shape, with the strict decoder the spec
requires. Each rejection closes a different hole: a `sig` member for the
reason above; an unknown top-level member, because two implementations
that disagree about what to ignore disagree about the id preimage; a
duplicate key, because "last one wins" and "first one wins" are both
defensible and yield different events from identical bytes; and a
mismatched id, because the id is what edits, history and dedup all
reference.
Duplicate-key detection needed its own scan. Every JSON library here
resolves duplicates before the caller sees them, so `MarmotJson` walks
the raw text tracking nesting depth and string boundaries — it has to be
right about exactly one thing, where a top-level key sits.
The id is unchanged by the switch. NIP-01 hashes
[0, pubkey, created_at, kind, tags, content], which never covered the
signature, so message identity survives and existing history still lines
up. The Android pipeline already treated inner events as unsigned rumors
with an empty sig and skipped verification, so the app layer needs no
change: the empty `sig` is re-added at the inbound boundary instead of
travelling on the wire.
Also adds the two Stage 7 kinds. Kind 1009 edits carry the deterministic
tie-break the spec implies but does not spell out — two devices of one
account can stamp the same second, and without it two readers would
render different text for the same message forever. Kind 1210 system rows
are synthesized from canonical state rather than received, which is what
makes them unforgeable by a single member.
Verified against the spec's published fixture: our canonical serialization
hashes to the exact event id the spec prints for its kind 1210 example.
That is the only check that distinguishes correct from merely
self-consistent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
A locally generated group-state change was becoming canonical the moment
it was prepared, before anything had been published. The spec forbids
that, and the reason is not bookkeeping: if the publish then fails, this
client holds an epoch no peer has, and every message it sends next is
undecryptable to the group.
Apply-then-undo would not have fixed it. Between the apply and the undo
there is a window in which we are already forked, and a crash inside that
window makes the fork permanent. So a local commit is now prepared on a
CLONE restored from the current state: the live group does not move, keeps
its pending proposals (which is exactly the "proposal stays available for
retry" rule on failure), and the pending state becomes canonical only via
installState once publication is acknowledged.
Acknowledged means what the spec says it means — at least one endpoint in
the recipient scope returning an accept, which over Nostr is OK true from
a relay. `MarmotPublisher` makes that explicit and the manager owns the
publish, because a caller handed bytes may or may not report back. Its
default refuses everything: a client that never configures a publisher
can read a group but never advance it, which is the safe direction to
fail. The recipient scope comes from the group's own relay list, so the
accept has to come from an endpoint the GROUP names.
The obligation record is durable before the publish, not after. The other
order leaves a crash window in which peers have accepted a commit this
client has no memory of preparing — and on restart it would generate a
replacement, forking itself at its own epoch.
Group creation keeps its exception: a one-member epoch-0 group has no peer
that failure to publish could fork, so its obligation is empty and
immediately satisfied. Departure keeps its own shape too — a SelfRemove is
a proposal, not a local commit, so it has no pending state, but it raises
the LEAVING outbound gate, and a gated group refuses new commits rather
than preparing epochs it has no standing to publish.
Also carries a quiet fork to its cutoff. Inbound traffic ticks convergence
opportunistically, but a group where the fork was the last thing to arrive
had nothing to settle it; the settler runs only while a pass is open, so a
quiet client still does no periodic work.
Two test premises of mine were wrong and the code was right: a sole admin
cannot SelfRemove without demoting first, and a payload for an unretained
branch is transport-deferred rather than rejected. Both tests now assert
the actual rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Convergence could score witnesses but nothing ever produced one, so the
witness steps of the branch comparison were dead code. This wires the
producer, and it turned out to need two layers rather than one.
The MLS layer is the obvious half: an app message that decrypts on no
canonical epoch is tried against the candidate states convergence
retains. That set is bounded by the rollback horizon, so a flood of
undecryptable ciphertext costs a bounded number of attempts rather than
an unbounded key search. Candidate states are now replayed when a
divergent commit is admitted, not only at resolution, because witnesses
have to accumulate DURING a pass to influence the selection that pass
makes.
The transport layer is the half that is easy to miss. Marmot's outer
ChaCha20 layer is keyed by a per-epoch exporter secret, so an event
published on a fork does not merely fail to decrypt — it does not peel
at all, and never reaches the MLS layer to be tried. Retained candidate
states now also contribute outer keys, derived on demand rather than
stored, so the release condition stays in one place: when the state goes,
the key goes with it.
A payload that decrypts on a candidate branch is NOT delivered — the
canonical state contradicts it — but it is not dropped either. It is
reported as living on a branch, and counted as a witness only if it
passes the same author check a delivered payload does. Decryption alone
is not a witness: without that check one member could mint many sender
identities and buy the witness quorum outright.
Canonical decryptions now witness too. The incumbent is rebuilt and
rescored at every resolution, so counting only divergent branches would
have let any fork win the witness steps unopposed.
Writing the tests corrected one of my own premises: a payload for a
branch we do not retain is `transport_deferred`, not a terminal error.
The commit that makes it readable may still arrive, and retaining that
branch is exactly the change of transport decryption context the spec
says must trigger a retry — so the test now asserts the deferral and
then the successful retry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
No — the card only showed the raw URL as a blue link. These events attach their
material through an `encoding:*` group (Blossom URL, MIME type, size, sha256)
and name it nowhere in the body, and most of the corpus attaches a PDF: a
worksheet, an instruction sheet, the thing the event is actually about.
The file now goes through the same classify-then-view path kind 1063 uses, so a
PDF gets `PdfPreviewCard`'s inline first page and `PdfViewerDialog` on tap, an
image gets its picture, and a webxdc bundle or an archive — anything no viewer
can show — gets `FileAttachmentCard` with its type and size rather than a URL to
squint at. The `encoding:sha256` is passed through as the content hash, which is
also what the viewer verifies its download against.
Also reads `datePublished`, a fourth spelling of the publication date: the
Caesar-Scheibe event that prompted this carries no `author` and no `published`,
so without it the byline came out empty. Its `image` stays the cover, distinct
from the attached material.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
A section opened on its own said nothing about what it was a section *of*, and
offered no way to the next one -- so a book was readable only by returning to the
index and picking the next row by hand.
Sections carry the back reference already: `T` holds the index's bare `d` (59 of
the 60 live sections surveyed carry it; `c` is a second spelling the same
publisher emits). It is an identifier, not a coordinate, so `publicationAddress()`
reconstructs the index from it plus the section's own author -- an index and its
sections share one, and the alternative, scanning the cache for an index that
lists this section, costs a walk per chapter.
- A crumb above the title names the book and opens it. Deliberately a crumb and
not `PublicationHeader`: a cover, blurb and 34-row contents on top of the
chapter you just opened would bury it.
- A pager below the body moves to the neighbours in the index's own order. Drawn
only when the index actually lists this section, since that listing is the only
thing that defines an order. The ends stay blank rather than disabled -- there
is no chapter before the first, and a greyed control invites a tap that cannot
do anything.
Verified on device across Wuthering Heights: Chapter II shows "Wuthering Heights"
above its title and CHAPTER I / CHAPTER III below, tapping CHAPTER III lands on
it, and Chapter I correctly offers CHAPTER II with no previous.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
`MarmotInboundProcessor` decided same-epoch races by the superseded MIP
rule: lowest outer `created_at`, then lowest Nostr event id. Both are
transport metadata the sender picks and MLS does not authenticate, so a
member could win every race by backdating. `CommitOrdering.kt` is deleted
and `MarmotConvergenceEngine` takes over, running the bounded pass, the
candidate graph and the six-step comparison end to end.
The wiring decision worth recording is that convergence does NOT hold
every commit for the quiescence window. A literal reading of the bounded
pass would tax the overwhelmingly common single-commit case with a second
of latency for nothing. It does not have to, because MLS is its own fork
detector: once a commit is applied, a competitor authored against the
same parent stops authenticating against the new tip but still
authenticates against the RETAINED parent. So linear commits apply
eagerly, the state each was applied to is retained, and a commit that
authenticates a retained state rather than the tip IS the fork — only
then does a pass open.
That does not change the answer, and the reason is which state resolution
treats as the base. It is the newest retained state a divergent commit
authenticates against, not the current tip, and the canonical commits
applied at or after it are replayed back into the graph. So the incumbent
is rebuilt as a branch and scored by the same rule as its challengers
instead of winning by having been applied first. Eager application only
decides which branch is provisionally displayed while the pass runs.
Supporting pieces: `MlsGroupManager.snapshot` takes a state without
touching storage, and `installState` is the rewind primitive — it pushes
the outgoing epoch's secrets into the retention window first, so messages
already sent on the abandoned branch still decrypt. `MarmotManager`
records locally-authored commits too: our own commit is half of any fork
we are party to, and without it a peer's competitor would look like an
unplaceable orphan and be deferred rather than compared.
The headline test builds a real same-epoch fork between two admins, feeds
two observers the same two commits in opposite orders, and asserts they
land on the same GroupContext with exactly one of them having rewound.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
A 32176 is a manifest for a file split across Blossom servers, but the parser
stopped at five tags, so the card showed a title, a summary, a poster and a raw
byte count -- and a tap did nothing, because no URL had been read.
Everything that makes it a *piece* index went unparsed: `r` (a whole-file URL),
`blossom` (the servers holding the pieces), `x` (the file hash) and every `b`
(one piece's hash and byte count). Those are now accessors, with `pieces()`
keeping publication order because that is reassembly order and must not be
sorted.
The card gains what a reader can act on:
- the byte count humanized -- `31838839` was a number you had to decode
- the piece count, the thing the kind exists for
- the Blossom servers on the byline; without them the hashes name something
unreachable
- a tap opens the whole-file URL. Nothing in-app reassembles pieces yet, so this
is the one address that plays. With no `r` there is nothing to open and the
card stays inert rather than pretending otherwise.
Verified against silberengel's "Glyfada evening tide": reads `file - 32 MB -
31 pieces` over `files.sovbit.host`, and tapping opens the video. The 31 `b`
tags sum to exactly the declared 31838839 bytes, so the piece list parses whole.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Convergence had a selector with nothing to select over. This adds the
candidate-graph builder that turns retained group states plus a bag of
inbound commits into the branches `BranchSelector` compares.
The central rule is that parentage is DERIVED, never declared. A commit
carries no parent pointer, and it must not be believed if it did, so the
builder finds a parent by asking which retained state the commit's
membership tag authenticates against. That makes it a fixed point rather
than a sweep: replaying a commit produces a state that may be the parent
of a commit nothing could place a moment earlier, so it keeps sweeping
the unplaced set until a pass produces no new edge.
`CandidateStateEngine<S>` splits the graph algebra from MLS so each half
is testable on its own terms; `MlsCandidateStateEngine` is the real
adapter. A state id is SHA-256 over the serialized GroupContext, not the
epoch number — two states can share an epoch number and be different
states, which is exactly what a fork is. Every trial replay restores a
fresh group from the retained snapshot, because a candidate parent gets
tried by several competing commits and "advance it then roll it back"
works right up until an exception escapes halfway through.
Dispositions keep "I cannot place this" apart from "I caught you
misbehaving": an unauthorized commit whose parent IS known is terminal
`authorization_failed`, while a commit nothing authenticates is
`deferred`, and only `stale` once the live canonical tip passes the
rollback horizon. A resulting state that breaks a component invariant
produces no edge at all, so convergence can never select it.
`MlsGroup` grows three non-mutating helpers for this —
`resolveCommitProposals`, `isCommitAuthorized`, `isSelfOnlyCommit` —
reusing the same gates the local commit path runs, so an inbound commit
and one we authored are held to one rule instead of two that drift.
Writing the real-MLS test found a bug in the builder: an unattributable
tip was zero-filled into a 32-byte committer, which would have handed it
the lowest possible account pubkey and won it a step-5 tie-break it never
earned. Such a tip is now dropped.
Tests: 12 over a fake engine for the graph algebra, 9 over real MLS
groups and a genuine same-epoch fork.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Two fixes from looking at the real corpus (69 kind-30142 events off theforest,
damus, nos.lol and thecitadel) rather than at the one sample event.
**Learning resources rendered as a title and a summary line.** The card only
read title/summary/image, and the wire has two publisher families that carry
neither pair fully:
- *Codices* (25 of 69) publish books: `title`, `author`, `published`, `image`,
and an empty body. The card showed a title over a cover with nothing to say
who wrote it or when.
- *Edufeed / EKW* (41 of 69) publish schema.org: `name`/`description` plus a
vocabulary of facets spelled as flat tag pairs — `about:id` with the URI and
`about:prefLabel:de` with the label, repeated per value, plus
`learningResourceType:*`, `educationalLevel:*`, `creator:name`,
`inLanguage`, `license:id`, `isAccessibleForFree` and an `encoding:*` group
naming the file the resource *is*. None of it was read.
`LearningResourceEvent` now parses all of it, and the card shows a byline
(author · year), a chip row of facets, and a link to the attached file. Facets
resolve to one language — a real event carries `learningResourceType` in six —
and deduplicate, because the vocabularies overlap and a chip row that repeats
itself reads as a bug. With this, every one of the 69 events shows at least one
fact beyond its title; before, 44 of them showed none.
This is a superset of what the reference Android client renders: it maps 30142
onto its generic hero card, which reads `author`, `published_on`, `type` and
`l` but none of the schema.org facets, so its cards for the Edufeed corpus are
title + description + cover.
**Rating stars sat above the rated thing.** A verdict a reader meets before the
card it judges has nothing to attach to; they now sit under it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
Routing the name searches through the leftover terms was half right and I said so
too confidently last time: `#bitcoin` does not reach the channel finder as
"#bitcoin" any more, but nor does it reach it as "bitcoin" — the tokenizer lifts
the tag out, nothing is left over, and the finders return early on a blank
string. So the fix that was meant to make a channel called "Bitcoin" findable
made it unfindable, along with every user.
`SearchQuery.nameSearchTerms()` gives these finders the word the reader actually
typed: the leftovers when there are any, and otherwise the value inside the token.
Somebody typing `#bitcoin` into a search box means "bitcoin" by it, and
`from:vitor` means "vitor"; neither means "search for nothing". One term, never a
join — these match a single name, so `"bitcoin lightning"` would match nothing at
all.
Desktop had the same hole through a different door: its people subscription fell
back to serializing the whole query, so the relay's people index was asked for the
literal "#bitcoin". Same function, same answer.
Checked the other surfaces rather than assuming: the hashtag suggestion row, the
NIP-05 resolver, the bech32 auto-navigate and the relay-URL row all read the raw
box on purpose — an npub or a `wss://` url is the whole input, not a word inside
it — and none of them were touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
Every previous change read the reference. This one writes, which is the half
that decides whether MDK accepts anything we author.
CurrentProfileGroupFactory assembles current-profile leaves, KeyPackages and
groups. The order it enforces is forced by the protocol rather than chosen: a
leaf must carry an account identity proof over its OWN MLS signature key, and
only an account signer — possibly a remote bunker or an external app — can
produce that proof. MLS leaf construction is synchronous and normally generates
its signature keypair internally, so the proof cannot be attached afterwards by
code that only sees a finished leaf. The keypair is therefore generated first,
authorized, and only then built into a leaf. MlsGroup.create and createKeyPackage
gained leaf-extension, capability and required-capability parameters to allow it.
Writing the producer side immediately found two bugs that the reader-side tests
could not have found:
buildLeafNode accepted a leaf-extensions parameter and then wrote
extensions = emptyList(). Every leaf we built would have silently dropped its
identity proof — the exact component that makes a group classifiable at all.
Fresh KeyPackages carried Lifetime(0, Long.MAX_VALUE). That fails the bound
Stage 4 had just started enforcing, so every KeyPackage we published would have
been rejected by any conformant peer, including by us. Now it spans now-1h to
+84 days: the backdate gives a peer with a slow clock a window where the package
is already valid, and 84 days leaves the spec's whole one-hour skew allowance as
headroom rather than sitting on the limit.
Six tests, including the two directions that matter for interop: a KeyPackage we
build has the same component set, capabilities and dictionary layout as the one
the OpenMLS fork emits, and a KeyPackage the fork authored can be added to a
group we created. Full quartz jvmTest: 4,600 tests, 0 failures. commons and cli
compile.
What this does NOT do, recorded in the plan rather than implied: the app layer
still creates MIP-era groups — nothing in commons, amethyst, desktopApp or cli
calls this factory yet. The Quartz half is ready and tested; the wiring is not
written.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
The other half of the convergence machinery: the input-collection window that
decides which inputs a resolution sees.
A pass exists so a client resolves a fixed batch rather than chasing a moving
one. It snapshots pass_base_epoch when it opens, closes at the earlier of the
quiescence window and the absolute deadline, and then freezes: resolution
reaches a deterministic fixed point using only what was admitted, without
waiting for a fetch or admitting later input.
Two asymmetries carry real weight, and both are tested.
Only selection-relevant input restarts quiescence. Ordinary chat traffic that
cannot change branch selection must not, because outbound work is gated on the
group settling — if it did, a busy group would never send anything.
Neither a detected fork nor an admitted disband candidate restarts anything. A
pass that becomes a recovery is the same pass; restarting its timers or
resnapshotting its base epoch would let a steady trickle of forks hold it open
indefinitely. The disband case forces Stable -> Recovering even on a linear edge
with no fork, so terminalization can only happen after branch selection.
Deferred-commit expiry deliberately tracks the LIVE canonical tip, so obsolete
input ages out as state advances across completed passes, while branch
eligibility uses the FROZEN pass_base_epoch, so an open pass cannot move its own
rollback horizon while comparing candidates. Using one epoch for both would make
the horizon shift underneath a pass.
The timers are scheduling, not semantics: input arrival time, cutoff time and
pass membership never enter candidate validity or the branch score, and there is
a test that splitting the same inputs across passes reaches the same answer.
Driven by an injected monotonic clock — a wall clock would make these tests
flaky and prove less, and a clock adjustment must not be able to shorten or
extend a real pass either.
Twelve tests. Full quartz jvmTest: 4,594 tests, 0 failures.
Two of these tests started out asserting my own bad arithmetic — the tick at
exactly the absolute deadline is refused, not admitted, and a pass cannot be
kept alive to 4800ms without feeding it — and now assert the rules explicitly.
Still open: the candidate-graph builder that replays MLS bytes against retained
states, and wiring the pass and selector into MarmotInboundProcessor.
CommitOrdering's transport-metadata tiebreak still stands until that exists.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Every new kind whose defining semantic is replacement was reading its event
once, with `note.event as? X ?: return`. `Note` is `@Stable` but `Note.event`
is a plain `@Volatile var`, not snapshot state, so that read registers no
snapshot dependency: when a newer version of an addressable event supersedes
the old one on the same `Note` instance, the card keeps drawing the version it
first composed with. The one-time read also skips the relay side —
`observeNoteEvent` opens the EventFinder subscription, so an unobserved card
never asks for what it is missing.
Switched to `observeNoteEvent<T>` in the eight addressable renderers:
30040 publication index, 30041 section, 30045 directory, 30142 learning
resource, 30819 wiki redirect, 31987 relay review, 32176 Blossom piece index,
34259 entity rating. Left the regular kinds alone (17, 31/32/33 citations,
818/819 wiki merge): a non-addressable event is immutable once it lands, and
`WatchNoteEvent` already covers the not-yet-arrived case.
Also fixed the one unobserved User read: the rated-note card named its author
from `targetNote.author?.toBestDisplayName()`, keyed on the note's own state.
A kind-0 routinely lands long after the note it signs, so the card froze on
the hex. `observeUserInfo` both watches the cache and asks for the metadata.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
Stages 5 and 6, protocol cores.
The lifecycle model: six canonical states with their legal-transition table,
four derived convergence statuses with the legal-combination table, and the
durable local gates (Leaving, Disbanding, realized removal) that restrict
outbound work without being lifecycle states.
Two table entries are load-bearing rather than bookkeeping, and both have tests.
There is no Merging -> Recovering edge: a competing branch observed while
applying our own confirmed commit is retained, the merge completes to Stable,
and admission into a bounded pass then triggers Stable -> Recovering. Diverting
mid-merge would leave a half-applied epoch. And Disbanded has no outgoing edge
at all — no later branch supersedes a terminalized disband.
Branch selection replaces the superseded MIP-03 rule, which broke a same-epoch
tie on the outer Nostr created_at and then the event id. Both are transport
evidence: timestamps are chosen by senders, and each transport copy of one MLS
message carries a different event id. The replacement reads only authenticated
values.
Three details that decide whether two clients agree:
Byte ordering is unsigned. Account keys and SHA-256 digests are uniformly
distributed, so a signed comparison inverts roughly half of all final ties, and
two implementations would disagree that often.
raw_commit_depth gets no comparison step of its own — it is already inside
effective_commit_depth, so once effective depth and quorum status tie, a further
raw-depth comparison is necessarily tied too. A widely circulated write-up of
this algorithm lists raw depth as a step; the spec does not, and there is a test
that fails if it is added.
Witnesses count distinct sender ACCOUNTS per branch epoch, capped at the quorum
size, and epochs at or before fork_epoch do not count. Counting by account stops
a multi-device member counting twice; counting distinct senders stops one member
inflating a branch by sending a lot; the per-epoch cap stops one busy epoch
outweighing several quiet ones.
The policy constructor enforces max_witness_override_depth <= max_rewind_commits,
because without that bound app-payload traffic could push a branch past the
rollback horizon and beat an arbitrarily longer valid commit branch.
Twenty-five tests, including the worked example: a three-commit branch with
witness quorum ties a four-commit branch without one at effective depth four and
then wins on quorum, while a five-commit branch beats both because the boost is
capped at one. Selection is asserted invariant under input order, reversal,
shuffling and every rotation. Full quartz jvmTest: 4,582 tests, 0 failures.
Still open in these stages: the bounded pass scheduler and the candidate-graph
builder that replays MLS bytes against retained states, plus wiring the
lifecycle states into MlsGroup so they gate anything. CommitOrdering's
transport-metadata tiebreak therefore still stands — deleting it is only safe
once something replaces it end to end, and selection alone does not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Two things a real bookshelf turned up.
**A nested directory read "Untitled section".** `Library.kt` reuses
`PublicationSectionRow`, whose title `when` knew only the publication kinds, so a
directory listing another directory fell to `else -> null` and then to the
section placeholder -- both the wrong name and the wrong noun. A directory lists
whatever it likes, so the row now names the library kinds too, learning resources
and piece indexes included.
**A learning resource read as its `d` slug.** `title()` looked only at `title`,
but publishers that tag themselves `type: LearningResource` follow schema.org and
emit `name`/`description`. Both spellings are accepted now, `title`/`summary`
winning where an event carries both, so nothing that renders today changes.
Not a bug, checked and left alone: Laeserin's directory shows `my-book-collection`
because that event carries no `title` *or* `name` at all -- the `d` fallback is
the right answer there.
Verified on device against the same two events that showed the defects: entry 3
of `my-book-collection` now reads "nostr" instead of "Untitled section", and the
German resource reads "Caesar-Scheibe - 30 Buchstaben: A-Z plus Ae Oe Ue ss"
instead of "17xu8qb7". Two tests added for the vocabulary split, including the
precedence case; LibraryEventsTest 11/11 green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Finishes Stage 3 and lands most of Stage 4.
Image crypto (0x8002). GroupBlossomImageCrypto implements the current-profile
scheme, which breaks from MIP-01 in three ways: image_key IS the AEAD key rather
than an HKDF seed, image_upload_key IS the Blossom-auth secret rather than a
seed, and the AAD is domain-separated and binds the media type where MIP-01 used
an empty AAD. That last one closes a real hole — with no AAD a blob could be
replayed as a different media type. MarmotMediaType implements the frozen
canonicalization; it uses ASCII case folding explicitly, because a locale-aware
lowercase would map a dotted capital I to a dotless one and change the AAD
bytes. Decryption verifies the content hash before attempting the AEAD: the blob
is addressed by hash, so a store returning different bytes is broken or hostile,
and finding out through an authentication failure loses that distinction. The
MIP-01 scheme stays for groups already on disk and nothing falls back between
them, because the component id is the version.
Transport. kind:30443 gains a current-profile builder emitting the required tag
set and omitting the two tags the spec forbids: encoding (a receiver decodes
each field by the rule that defines it, never by a negotiated marker) and relays
(discovery is the author's NIP-65 write set). Validation is profile-aware, told
apart by the presence of app_components rather than a version tag.
KeyPackage relay discovery moves to the NIP-65 write set. publishRelaysFor no
longer prefers a kind:10051 list — publishing only where a now-removed list
points would make us invisible to a conformant peer, which looks in the NIP-65
set and nowhere else. The legacy list is unioned in rather than substituted, so
peers that have not migrated keep finding us.
Deduplication now uses SHA-256 over the recovered MLS bytes rather than the
Nostr event id, which the transport spec forbids as a dedup key. The old scheme
collapsed nothing it was supposed to: relays redeliver, and every transport copy
of one MLS message carries its own fresh ephemeral pubkey and therefore a
different event id — so cross-relay duplicates always got through, and a hostile
republisher could mint unlimited distinct ids for a single message. Dedup
necessarily moved after outer decryption, since there is nothing to hash before
that, and OutboundGroupEvent now carries the id so a publisher suppresses its own
echo by MLS identity.
Also enforces the KeyPackage Lifetime bound (present, current, at most
7,261,200 seconds) and validates the embedded account identity proof on inbound
current-profile KeyPackages — the app_components tag is only an advertisement, so
the decoded LeafNode is what decides.
Fifteen new tests. Full quartz jvmTest: 4,557 tests, 0 failures. commons and cli
compile.
Left for Stage 6: bounded retained-candidate trial decryption for kind:445. Its
rule is defined over the retained-state set convergence owns, so it cannot land
ahead of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`RenderNoteRow` and `NoteMaster` are two independent dispatch chains, and the
library, citation, section and relay-review kinds only reached the first. So
every one of them rendered in a feed row and then fell apart when you tapped it.
Six kinds were affected -- 31/32/33 citations, 30045 directories, 30142 learning
resources, 32176 piece indexes, 30041 publication sections and 31987 relay
reviews -- degrading two ways depending on whether the kind carries `content`:
- 30045 rendered as an avatar and an action row with nothing between them, since
a directory's whole substance is its `a` tags.
- 30041 fell through to the generic body: the prose showed, but with no title,
no AsciiDoc conversion and no wikilink resolution, so `RenderPublicationSection`
never ran. That one is on a path the feature builds itself -- a publication's
table of contents links straight into it.
Citations get one branch on the `CitationEvent` supertype, matching the feed.
Verified on device against live events for the four kinds that have any:
Laeserin's `my-book-collection` now lists its 7 entries by title, Wuthering
Heights' CHAPTER I renders titled and converted, the German learning resource and
the "Glyfada evening tide" file record both reach their own renderers. Nothing
has published a kind 31/32/33 on any of the eight relays checked, so those three
are wired and compiled but unverified against real data.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Audited the whole branch diff. Seven real defects, the first of them the one
that mattered.
SECURITY. Nine kinds had been added to REASONABLE_SIGN_KINDS — the set a
connected app may sign WITHOUT asking the user — as part of mechanically
working through a registration checklist, never against that list's own stated
test ("when a kind's blast radius is unclear, it is left out"). Four fail it and
are removed: a kind-30045 directory is a replaceable *list* one bad write wipes
(exactly the case the KDoc excludes), and ratings (34259), relay reviews (31987)
and wiki redirects (30819) attach a replaceable opinion or redirect to the
user's identity. The addressable *content* kinds stay, matching 30023/30818.
AsciiDoc, both proven with failing tests first:
- A trailing space defeated matchEntire on a block image, so `image::url[Alt] `
fell through to the inline rule, which captured `:url` and emitted
`` — a dead link, breaking the file's "never mangled output"
promise. Block constructs now match the trimmed body.
- A `[source,kotlin]` attribute with no block after it kept its language and
labelled an unrelated later `....` block as Kotlin. The pending language is
now cleared once ordinary content intervenes.
Rendering:
- A rating using the spec's DEFAULT `event` mark names its target by event id,
and the card printed the raw 64-hex string as its bold title. It now loads
the note and shows its author and opening, and opens it on tap.
- Relay reviews printed a redundant "relay" chip beside the stars, directly
under the relay URL — the comment claimed MARKS_WITH_A_CARD suppressed it,
and it did not.
- The directory overflow row reused the total-count plural, so a 20-item shelf
read "20 items … 8 items". It has its own "N more items" now.
Parsing:
- Slot 2 of a section `a` tag was dropped as a title whenever it parsed as any
integer, losing a chapter genuinely called "1984"; only a value inside the
level range is a level now. A schemeless relay hint (`relay.example.com`) was
read as a title, putting a hostname in the table of contents.
Also two CLAUDE.md style violations: inline fully-qualified arrayOfNotNull in
WikilinkTag and ExternalTargetTag, now imported.
Tests: 4 new, all written to fail against the old behaviour first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
Stage 3 of the Marmot resync. MIP-01 packed name, description, admins, routing,
image and retention into one 0xF2EE extension, so any change rewrote the whole
blob. The adopted spec splits them into six independently versioned components
in the GroupContext app_data_dictionary; this implements all six, plus
MarmotGroupState as the read view that replaces MarmotGroupData.
Several are not mechanical translations of the MIP-01 fields:
- message-retention is a fixed uint64 with no length prefix, and 0 now means
disabled where MIP-01 rejected it outright;
- blossom-image gains media_type, and that type is bound into the AEAD's AAD;
- profile equality is byte equality, so nothing may Unicode-normalize a group
name before hashing, comparing or storing it;
- admin keys sort by unsigned byte value, which matters because roughly half
of all x-only keys start above 0x7f.
Decoders reject unsorted, duplicate, ragged and trailing bytes rather than
repairing them, for the same reason the dictionary codec does: these bytes sit
in the signed GroupContext, and normalizing on the way in would leave two peers
each holding bytes they consider valid and disagreeing about which is canonical.
This also closes the authorization gap Stage 1 left open and flagged. A
current-profile group keeps its admin list in marmot.group.admin-policy.v1
(0x8003), not in marmot_group_data, so both gates had nothing to read and let
everything through. They now resolve admins through currentAdminIdentities(),
which prefers 0x8003 and falls back to 0xF2EE, and depletion resolves an
admin-policy change carried by an AppDataUpdate rather than only by a
GroupContextExtensions proposal.
The admin lookup deliberately decodes only 0x8003 and not the whole component
set. The first version went through MarmotGroupState, which made a malformed
profile component freeze the group by taking the admin check down with it — its
own tests caught that. Authorization must not depend on components it does not
read.
Thirty-five tests: the MDK-generated GroupContext dictionary decoded component
by component and re-encoded byte-identically, per-component validation rules the
happy-path fixture cannot reach, and authorization driven through real groups
rather than by calling the gates directly. Two of those tests started as wrong
premises of mine and became real coverage: naming an admin who holds no member
leaf is rejected by the admin/leaf coupling rule, and removing the admin policy
is rejected because it is the group's sole admin authority for its lifetime.
Full quartz jvmTest: 4,542 tests, 0 failures. commons and cli still compile.
The 0xF2EE decoder stays as the legacy read path. Left for a follow-up: the
image ENCRYPTION still uses an empty AAD and treats image_key/image_upload_key
as HKDF seeds rather than the keys themselves, which touches the Android and CLI
image paths.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Adds a recorded corpus of 64 real events from search-staging.brainstorm.world — a
vespa-relay, the same software this token language was ported from — and 18 tests
over it: 8 on the engine in :commons, 10 on `LocalCache.filter` in :amethyst.
Recorded, not fetched. `tools/search-parity/fetch_fixtures.py` drives `amy fetch`
against the relay by hand; the tests read the committed fixture, so `./gradlew
test` stays offline and the pre-push hook does not depend on somebody else's
uptime. The fixture stores each case's filter FIELDS rather than a prebuilt
filter, so the test rebuilds the Filter in view of the reader and a wrong rebuild
cannot quietly make the assertions vacuous.
What the relay can and cannot referee turned out to be the whole design, and it
was measured rather than assumed:
- **NIP-01 it can.** Given kinds, #t, since and until there is exactly one right
answer, and across all 64 events our matcher agrees with the relay about every
one it chose to return — 0 violations. That is now a hard assertion, with a
converse test so it cannot pass by matching everything.
- **NIP-50 it cannot.** This relay retrieves topically: asked for `bitcoin` it
returns a block-height summary that never says "bitcoin". Eight of 64 events
carry no literal occurrence of the term that fetched them. Asserting our
substring matcher reproduces that would encode someone else's semantic
expansion as a requirement on a lexical one — a test that fails on correct
code. So text results are deliberately not compared, and the divergence is
pinned as a range instead: zero would mean the relay turned lexical and the
comparison should be rewritten, a quarter would mean we regressed.
The fixture uses the `include:spam` lens, which waives the web-of-trust gate.
Also measured: it makes the corpus reproducible, where `observer:<pubkey>` ties
every answer to one account's moving trust graph — but it does not make retrieval
lexical, and in fact widens the divergence from 4 events to 8 by letting more
topical matches through.
The LocalCache tests cover what the relay knows nothing about and where the bugs
actually were: the regular/addressable split, the viewer-policy predicate
composing with rather than replacing the filter, and the result cap keeping the
newest — the ordering whose absence let `take(limit)` run before the sort.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
The six kinds that have real renderers in jumble but none here. None is defined
by a NIP; the shapes come from the publishing clients' draft builders and
manifests, which is stated on each class rather than implied.
31/32/33 citations get one card, because they are one idea aimed at three sorts
of source and a reader should not learn three layouts. A shared CitationEvent
base carries what they have in common; each subclass adds only its own fields —
a URL and timestamp for the web, a containing work and page range for print, a
model name for a prompt. The provenance line is assembled from whichever fields
the citer actually recorded rather than templated, since a fixed layout would be
mostly blank labels.
Kind 30, "internal citation", is deliberately NOT modelled: Quartz already
registers kind 30 as a Jester chess move, so the two vocabularies collide on the
wire and a kind-30 citation already parses as chess here. Picking a winner is a
protocol decision, not a parsing one. A test states the collision rather than
leaving it to be rediscovered.
Two wire details worth pinning: a hardcopy citation's VOLUME has no tag of its
own and rides in the second slot of `published_in`, and the reference client's
manifest names the external URL tag `url` while its card and draft builder use
`u` — `u` is the wire truth, `url` is accepted so a publisher who followed the
manifest is not dropped.
30045 directory is a curated shelf, and its items use exactly the `a`/`e`
grammar a publication index uses for its table of contents — including the
uppercase-is-a-source rule. So it reads them through PublicationSectionRef and
renders them with the same rows, rather than growing a parallel parser that
would drift.
30142 learning resource renders its body in full: the reference Android client
marks the kind `reader = true`.
32176 blossom piece index keeps `size` as the string it is published as, with a
separate sizeInBytes() that returns null rather than guessing units on anything
that is not a byte count.
Uses AutoAwesome and Collections from the existing subset instead of adding
SmartToy and School, so the font does not need regenerating for two glyphs.
Tests: 22 new (13 citations, 9 library).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
Two things wrong with how a kind-30040 index rendered as its own post.
**The summary was a bare `Text`.** It is authored prose -- a blurb, unbounded,
with no `maxLines` -- so a plain Text gave it no translate offer and rendered any
url, mention or hashtag inside it as dead literal characters. It now goes through
`TranslatableRichTextViewer`, the same overload the rating and relay-review bodies
use, which wraps `ExpandableRichTextViewer` and so also brings the Show-More
collapse that a blurb of this length wants. Threading it needs `makeItShort`,
`canPreview`, `quotesLeft` and `backgroundColor`, which puts `PublicationHeader`
and `RenderPublicationIndex` on the same parameter shape as `RenderEntityRating`.
The visible asymmetry this removes: a German *review* of a book offered
translation while the book's own German blurb did not.
**The card framed the whole post.** `replyModifier` draws the quote border used
for something cited inside a note. A 30040 carries no content beside this header,
so the border wrapped the entire post and read as a citation of something else --
a card inside the note row's own frame, with nothing outside it. Now a plain
Column, and the card's horizontal insets go with it since the note row already
indents. This is the split LongForm already makes: `LongFormHeader` keeps the
card for a feed row, `RenderLongFormHeaderForThread` drops it for the thread.
Verified on device against two of `npub1m4ny6...`'s publications: the English
Wuthering Heights index renders flush with the note frame with a Show-More
summary, and the German "Odysseus: Mythos und Wahrheit" renders its blurb with
"Auto-translated from German to English" -- which the bare Text could not do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Stage 1 of the Marmot resync, and the foundation the rest of it needs. The
current profile keeps every piece of application-owned group state in the
draft-ietf-mls-extensions `app_data_dictionary` rather than in a bespoke
extension, so nothing downstream can be built without it.
Adds ComponentData, AppDataDictionary (extension 0x0006) and the ComponentsList
payload shared by app_components (0x0001) and safe_aad (0x0002); the
AppDataUpdate proposal (0x0008) with its update/remove operations, wired
through MlsGroup on both the committing and receiving paths; and reads
last-resort as the KeyPackage-level 0x0004 component rather than an MLS
extension type, which is what the MIP-era profile used 0x000a for — a value
that now means the self_remove proposal.
Building a dictionary sorts its entries; decoding refuses to. A receiver that
silently sorted would accept two encodings of one dictionary, and since the
dictionary sits inside signed LeafNodes and the GroupContext, two peers would
then hold bytes they each considered valid and disagree about which is
canonical. Same reasoning for ComponentsList and for rejecting trailing bytes.
Two application rules are taken from openmls rather than inferred, because both
change the resulting GroupContext and therefore the epoch key schedule:
AppDataUpdate applies after the rest of the proposal list, so a
GroupContextExtensions proposal in the same commit is already reflected
regardless of list order; and the dictionary extension is added-or-replaced in
place and never dropped, so removing the last component leaves an empty
dictionary rather than no extension.
MLS leaves update-payload semantics to the application — openmls hands the
proposals back unresolved because a payload can be an arbitrary diff. Every
Marmot component document defines its update as a full replacement state, so
resolution here is the identity function; that assumption is documented where a
future diff-shaped component would have to break it.
Twelve tests parse the MDK-generated KeyPackage in marmot-current-profile.json
end to end — MLSMessage, KeyPackage, LeafNode, dictionary, components — and
re-encode the dictionary byte-identically. A JSON echo of the component map
could not establish that: the generator would just be handing back what it was
told to write. Nine more cover the proposal wire format and its group-level
application, including two members converging on the same dictionary across the
separate receive path. Full quartz jvmTest: 4,507 tests, 0 failures.
One gap is left open on purpose and marked at the source: the MIP-era
authorization gates read marmot_group_data (0xF2EE), which a current-profile
group does not have, so both return without enforcing anything there. The
admin-policy component closes that in Stage 3.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
A rating only reached its author's profile if somebody had reposted it. Both
profile gates were missing the kinds, so the events never arrived and would not
have been accepted if they had:
- `UserProfilePostKinds2` did not request 34259 or 30040, so the profile REQ
never asked the author's outbox relays for them.
- `UserProfileNewThreadFeedFilter.acceptableEvent` did not accept either type.
The branch wired the Home feed's equivalent pair but not this one. The
addressable scan here has no kind allow-list of its own (unlike Home's
`ADDRESSABLE_KINDS`), so the DAL side is just the two type checks.
The rating check mirrors Home's: a rating with nothing to point at cannot be
rendered, so `hasTarget()` gates it rather than leaving an empty row on the
author's own profile.
Kinds deliberately left out: 30041 publication sections, because they are
chapters rather than posts and one book would bury a profile under 34 entries;
and 31987 relay reviews, which have the same shape as ratings but were never in
any feed and should be a decision of their own.
Verified on device against `npub1m4ny6...`, whose two ratings differ usefully:
the Wuthering Heights one has three kind-16 reposts and so was already visible,
while "Am Fluss der Zeiten" has none. The latter now renders on the profile as a
plain entry with no repost header, which it could only do by arriving through
these gates.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Stage 2 of the Marmot resync. A member leaf carries two unrelated keys — the
MLS BasicCredential identity, which is the member's Nostr account key, and the
MLS leaf signature key MLS generates per device — and MLS never checks that the
account agreed to the leaf key next to it. Without a proof, anyone able to
author a leaf can claim any account's identity.
This is also what makes us classifiable at all. MDK decides Legacy vs Current
purely on whether a group requires extension 0xf2f1 or component 0x8009; we
required neither, so profile classification errored out before any component
check ran.
Adds the common authorization-proof envelope (foundation/authorization-proofs.md):
104 fixed-width bytes of signer pubkey, big-endian uint64 timestamp and BIP-340
signature, with event-id reconstruction. The created_at bounds are load-bearing
twice over — the lower bound rejects zero, and the upper bound (2^53-1) catches
a uint64 whose top bit is set, which reads back negative as a Kotlin Long.
Deliberately absent: any comparison of created_at against a local clock. A proof
authorizes a long-lived key binding, not a one-time operation, and a wall-clock
rule would let skew make two members reach different verdicts on the same Commit.
The component itself signs a kind-450 template through NostrSigner rather than
raw BIP-340, which is the whole point of the indirection: a NIP-46 bunker or
NIP-55 app can produce a proof without exposing arbitrary signing. create()
therefore re-verifies everything the signer returned — pubkey, timestamp, kind,
tags, content, recomputed id, signature — since an external signer is free to
substitute a stale or altered event.
Also adds the app-component id registry, and the RFC 9420 signature-scheme
mapping to MlsCiphersuite (declared outside the companion: an enum's entries
initialize before its companion object, so entry constructor arguments cannot
read companion properties).
Tested two ways. Sixteen tests pin the spec's published fixture — canonical
event serialization, event id, signature, the 104-byte layout — and check that
every signed input actually binds, including a ciphersuite change that leaves
the signature scheme untouched. Six more validate the proofs in
marmot-current-profile.json: those come from a separate implementation, for
randomly generated keys, which is the interop property a fixed vector cannot
establish. Full quartz marmot suite: 395 tests, 0 failures.
Nothing reads or writes these on a real leaf yet — the carrier is the
app_data_dictionary, which is Stage 1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
Four more of the kinds imwald publishes that Amethyst dropped on arrival.
818 wiki merge request (NIP-54 Appendix 1) — target article, destination
author, the version to merge, the version it was based on, and the
explanation. Renders both ends as rows that resolve to the articles they name.
The spec and the implementations disagree on one thing, and it is the load
bearing one: NIP-54 writes the merge-source marker as `source`, while the
client actually publishing these writes `fork`. Reading only the spec's word
turns every merge request in the wild into one with nothing to merge, so both
are accepted. The marker slot is read directly rather than through
MarkedETag's enum — `source` is NIP-54 vocabulary, and adding it to the NIP-10
threading markers would imply it takes part in threading, which it does not.
819 merge acceptance — NOT a NIP-54 kind. The spec stops at the request and
says the author answers with a NIP-25 +/- reaction; 819 is an extension that
records the acceptance and the version it produced, which a bare reaction
cannot carry. Documented as such rather than presented as spec.
30819 wiki redirect — the spec section is literally `[INSERT EVENT EXAMPLE]`,
so the publishing clients are the only reference: `d` is the slug redirected
from, `a` the article redirected to.
Writing the slug normalizer from first principles produced a real interop bug,
caught by its own test: folding every non-alphanumeric to a dash gives
`C++ Programming` the slug `c--programming`, where the rest of the network
computes `c-programming` — a redirect published under it would match nothing.
The rule is separators to a dash, letters and digits kept (Unicode-wide, so
non-Latin titles survive), everything else DROPPED, runs collapsed. Pinned by
tests. One documented divergence: the reference NFC-normalizes first, which
commonMain cannot portably do.
17 external reaction (NIP-25) — a like on something that is not a nostr event:
a web page, a podcast episode, a book. Borrows the activity-card frame so a
like reads the same whatever it targets, but there is no note to quote and no
recipient pubkey, so the card names the NIP-73 target instead — as a link when
one is openable, as plain text when it is an ISBN or a GUID that only looks
like one.
Tests: 25 new (18 wiki, 7 reactions).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
Stage 0 of the Marmot resync: get a live reference back, so the later stages
are written against real bytes instead of a careful reading of the spec.
The vector generator pinned stock crates.io openmls 0.8. MDK builds against
erskingardner/openmls with the `extensions-draft` feature, and the whole
current Marmot profile is expressed in terms of what that feature adds —
app_data_dictionary (0x0006), app_components (0x0001), safe_aad (0x0002),
app_data_update (0x0008). Vectors from the published crate cannot reach any of
it. Pinned to MDK's exact rev instead.
Adds `marmot-profile-gen`, which builds a group the way cgka-engine does:
required capabilities of extension 0x0006 plus proposal 0x0008; GroupContext
dictionary carrying the required-component list, group profile, admin policy,
Nostr routing and lifecycle; per-leaf dictionaries carrying the supported list,
an empty safe_aad list and the 104-byte account-identity-proof v2 component;
last resort as the empty-data 0x0004 component in the KeyPackage dictionary,
not an extension type; PublicMessage handshakes. It emits the Add commit, the
Welcome, and exporter KATs for both group-event and the conformance commitment.
The identity-proof encoder is hand-rolled from the spec rather than lifted from
MDK, and asserts itself against the fixture published in
account-identity-proof-v2.md before emitting anything — so if the generator
runs at all, the kind-450 canonical serialization, its id, the BIP-340
signature and the component layout are known to match.
The interop harness cloned marmot-protocol/whitenoise-rs, which was archived on
2026-08-05 pinned to mdk-core 0.8.0: it was testing us against a frozen
MIP-era client, which is part of how the drift went unnoticed. Repointed at
marmot-protocol/mdk, building -p wn-cli. Both source patches are dropped —
mock-keyring is replaced by MDK's native --secret-store file, and
skip-unprocessable-retry targeted a path MDK does not have. The daemon socket
is now pinned via wnd --socket rather than guessed from a derived default.
The harness changes are read off MDK's DaemonArgs and wn-cli manifest, not off
a passing run; building MDK's workspace needs its pinned toolchain and a local
relay. A human run of marmot-interop-headless.sh is the acceptance test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
A kind-30040 card announced "34 sections" and offered no way to open one.
Sections parsed and rendered individually, but nothing ever linked to them, so
the reader was unreachable. This lists them.
Reviewing how imwald reads the same events turned up two interop gaps in our
parser, both of which silently lose most of a real book:
- Sections are listed with `e` tags as well as `a` tags, interleaved in tag
order. We only read `a`, so an index that lists its chapters by event id
looked empty.
- Slot 2 of an `a` entry is often a TITLE, not the documented relay hint, and
slot 3 is often a nesting LEVEL rather than the documented event id. The
title is the valuable one: it lets the whole table of contents render from
the index alone, with no round trip.
PublicationSectionRef models all of that, and keeps the case-sensitivity that
matters — uppercase `A`/`E` name the original source of a derivative work, not
its contents, so reading them as sections would splice in the wrong book.
Section rows resolve lazily and upgrade in place: the index's own title shows
immediately, observing the section drives the fetch, and the event's better
title replaces it on arrival. Nested entries indent by level, so a
part-and-chapter structure reads as one. Titles are taken from every kind an
index may list — 30041, nested 30040, long-form, wiki and spec — not just
30041.
Feed cards cap at 12 entries with a "+N more" line; the thread view lifts the
cap, since that is where a publication is actually read. The cap is a fetch
budget as much as a layout one — each row carries its own subscription.
Not adopting imwald's assembly model: they render each section with its own
kind's markup rules rather than concatenating, and their own docs say so, which
is what this does too.
Tests: 15 new, covering tag order, interleaving, the uppercase exclusion, title
vs relay hint in slot 2, event id vs level in slot 3, and clamping.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
Marmot deprecated the MIP documents on 2026-07-02 and MDK followed. Our
implementation still targets MIP-00..MIP-05, so it is no longer a valid
Marmot client under either profile the current spec defines.
The decisive break is identity. MDK classifies a group as Legacy or Current
purely by RequiredCapabilities: legacy requires MLS extension 0xf2f1
(account-identity-proof v1), current requires app component 0x8009
(account-identity-proof v2). We require neither, so
protocol_profile_of_group_extensions errors out before any component check
runs. We never implemented an account identity proof at all.
Records what changed upstream, what that costs us surface by surface
(app_data_dictionary components replacing marmot_group_data, convergence
replacing the timestamp+event-id tiebreak, NIP-65 replacing kind 10051,
group disbanding, the durability contract), and stages the work. Also notes
that whitenoise-rs — the reference our interop harness clones — was archived
on 2026-08-05 and moved into mdk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
`RepostIcon` defaults to `tint = Color.Unspecified`, so the boost button drew
its glyph in the ambient content colour while every other idle icon in the row
-- reply, like, zap -- is drawn in the `grayTint` the row is handed. Passing it
explicitly puts the un-boosted state back in line; the boosted state is
unaffected, since `RepostedIcon` keeps its own `RepostedColor` default.
Follow-up to a31364b999, which fixed the liked and reposted colours.
Authored by Vitor Pamplona; committed from a Claude Code session.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
A review of the branch turned up eight, four of them wrong answers rather than
rough edges.
**An unresolved name asked for everything.** `from:vitor` — a name the picker has
not turned into a key — made `SearchQuery.isEmpty` false while contributing no
authors and no search text, so the builder emitted filters constrained by nothing
but their kinds: three unbounded REQs to every search relay, and locally the first
200 notes the cache happened to walk, presented as results. A query that can
express nothing now asks nothing.
**The result cap cut before sorting.** `LocalCache.filter` did
`(addressables + notes).take(limit).toSortedSet(...)`, and both halves arrive in
hash-walk order — so it dropped whichever matches the walk reached last, the
newest as often as not, and 200 addressable matches pushed out every regular note.
Sorted first, then cut; the comparator is descending, so the cap now keeps the
newest.
**A quoted phrase could never match.** Splitting the NIP-50 string on whitespace
turned `"hello world"` into `"hello` and `world"`, two terms each carrying a quote
character. Terms are now split respecting quotes, with an unterminated quote
running to the end as a lexer reads it.
**The group-metadata arm inherited the query.** A kind-39000 event is written by
the host relay and carries the room's name — not the searched author, not the
searched words — so inheriting `search`/`authors` made the room-naming lookup
return nothing exactly when a `group:` query had any other content.
And four narrower ones: the desktop clear button cleared the field but left the
results, relay states, sort orders and deduplicator behind; the advanced panel
rendered its now-local bounds with the UTC formatter, showing a day off in most
timezones; the `group:` picker's channel list was a keyless `remember`, frozen at
first composition; and the interaction source feeding the desktop field's outlined
chrome reached no text field at all, so it never showed focus or hover.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
Kind 30041 parsed and rendered as a titled plain body, with the two things that
make it a publication section deferred. This does them.
AsciiDocToMarkdown converts the common AsciiDoc subset — headings, lists,
emphasis, link/image macros, source and literal blocks, quote blocks,
admonitions — and the body then goes through the same CommonMark renderer that
draws kind-30023 long-form, which brings media, imeta and nostr-link handling
with it.
Deliberately NOT the reference implementation's approach: imwald-android runs
Asciidoctor.js in a headless WebView, because JRuby cannot run on Android. That
is reasonable for an Android-only app and wrong here — it is Android-only while
Quartz targets JVM, iOS and native; it vendors a JS bundle; and it puts a
WebView on the text path of every article. The trade is fidelity: no tables,
includes, conditionals or cross-references. Anything unrecognized passes
through unchanged, so an unsupported construct degrades to the plain text it
already was rather than to mangled output.
The correctness rule the reference implementation also learned the hard way:
every inline rewrite is skipped inside ----, .... and ++++ blocks, so a code
sample containing *stars* or [[brackets]] survives verbatim. An unterminated
block still closes its fence, or the whole tail would render as code.
Wikilinks now resolve. WikilinkTag parses the positional addressing slots and
drops malformed ones rather than shifting them — an event id read as a pubkey
would address the wrong thing. A [[target]] becomes an nevent when the tag
names an exact revision, an naddr when it only names an author, and falls back
to the bare label otherwise. imwald points these at its own web wiki; pointing
them at kind-30818 pages keeps the reader in the app.
Tests: 28 new (21 converter, 7 event), covering verbatim-block protection,
snake_case surviving emphasis, unterminated fences, unknown-construct
passthrough, and the case/separator-insensitive body-to-tag matching.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
Three seams the token language opened and left open.
**People and channels were still searched with the raw box.** Those flows read
`searchValueFlow` directly, so `#bitcoin` reached `findPublicChatChannelsStartingWith`
as the literal string and a channel actually named "bitcoin" did not match. That
was true before too, but it did not matter while nothing honoured the tokens —
now notes do, so one box meant two things depending on which result list you
looked at. All four name searches take the leftover terms.
**A `group:` chip drew the raw id.** A group id is a stranger's opaque string,
and the name is the only part a reader can check against the room they meant —
doubly so because there was no picker, making the token type-from-memory. The
chip now draws the name and the picker offers the rooms from
`LocalCache.allRelayGroupChannels()`, with its relay underneath and a "shared id"
warning where two relays mint the same id, which a `#h` filter cannot tell apart.
The id stays the value, so the query is unchanged.
**A `geo:` chip drew the raw geohash.** "9q8yy" says nothing about what was
filtered on. It now reads the same `CachedReversedGeoLocations` cache that the
feed spinner and thread view already reach through `LoadCityName`, so the chip
says "San Francisco". Resolution is asynchronous and a text transformation cannot
wait, so the resolver is synchronous by contract and returns null until the cache
has an answer — which leaves the geohash showing rather than a guess.
All three names come in as caller-supplied resolvers, like `displayName` for keys:
`commons` has no business reverse-geocoding or reading an account's group list.
Offset mapping is fuzzed over the renamed tokens too, since a name is a different
length to the id it replaces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
Audited every event kind imwald-android and its jumble web client know (95
distinct kinds across ImwaldConstants.kt and jumble's ExtendedKind) against
Amethyst's parsers. 22 had no event class; these are the two that mattered most.
30041 — NKBIP-01 publication section. The prose a 30040 index points at, so
without it the publication support added earlier indexes chapters it cannot
open. Renders as a titled body. Two limits are deliberate and documented: the
spec allows AsciiDoc, which we have no renderer for and which degrades to
readable plain text rather than mangled markup; and `wikilink` tags are parsed
but not resolved, which belongs to a reader that does not exist yet.
31987 — relay review. The same gesture as the kind-34259 entity rating aimed at
a relay, and it reuses the machinery: RatingStars is now shared between the two
so a reader does not have to learn two star vocabularies. It needs one new tag
accessor, CategoryRatingTag, because a relay review may carry several `rating`
tags — one overall, plus per-aspect ones with the category in the third slot.
Unlike 34259 there is no scale ambiguity here: every publisher of 31987 uses
the 0..1 fraction, so a value outside that range is malformed rather than a raw
star count, and parse() rejects it instead of guessing. Per-aspect scores render
as percentages; five more star rows would drown the overall one.
Tests: 13 new (9 relay review, 4 section), covering the overall-vs-category
split, both boundaries, out-of-range rejection, the d/relay fallback order and
URL normalization.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
87a44b97 added variable-axis support to MaterialSymbolPainter and aa54508e
switched the star row to `filled = isOn`. The KDoc and the plan still claimed
tint was the only lever available, which is no longer true and reads as an
argument against the fix that just landed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
Three defects found testing the branch on device against the two real kind-34259
events in the wild (both `m=books`, both five stars, from `npub1m4ny6...`).
**The stars were never filled.** Section 11.1 is right that `Star` and
`StarBorder` sharing U+F09A is not a bug, but the conclusion drawn from it --
that tint alone can carry the score -- does not survive contact with the screen:
a five-star review drew five *hollow* stars, and since `showLabel` is false
without a decimal there was no numeral either, so a full score read as an empty
row. Worse, `StarHalf` (U+E839) *is* a distinct half-solid glyph, so 4.5 would
have drawn four hollow stars beside one half-filled one -- the half star looking
more earned than the full ones. Now passes `filled = isOn` to draw the earned
stars solid off the font's FILL axis.
**Tapping a rating showed no rating.** The `EntityRatingEvent` branch went into
`NoteCompose.RenderNoteRow`, but a thread's focused note is drawn by `NoteMaster`
in `ThreadFeedView`, which has its own dispatch chain -- so opening a rating fell
through to the generic body: no stars, no cover, no publication. `HighlightEvent`,
the precedent section 4 cites for the no-`computeReplyTo` design, *is* in that
chain; the rating was not. Added beside it.
**The rated publication was a dead end.** `PublicationIndexEvent` is parsed but
had no renderer at either seam, so the one tappable thing on a rating card led to
a note showing nothing but a hashtag. `Publication.kt` renders the index: cover,
title, author, `type - version - N sections`, summary and topics. Built on the
`LongFormHeader` idiom but with a 2:3 portrait cover -- NKBIP-01's default type
is `book`, and the wide hero letterboxes every jacket. Wired at both seams.
It renders the card, not a reader: a 30040 carries no content of its own, and the
30041 sections it points at are still unparsed, exactly as sections 7 and 11.3
intended. `type` is shown as published rather than mapped through a string table,
because the spec calls that vocabulary open-ended and a table would blank every
value it has not enumerated. The publication branch sits in the body chain rather
than the header `when` above it: a 30040 has no body, so the generic renderer
would otherwise add a second copy of the topics the card already shows.
No font regeneration: the cover placeholder reuses `MaterialSymbols.MenuBook`,
already in the subset and already this feature's book icon.
Verified on device (Pixel 9, API 36) against the Wuthering Heights rating: solid
stars in both the thread and list paths, and the publication resolving to its
title, author, cover, blurb and `Book - 1.0 - 34 sections`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
Material Symbols expresses fill through a variable **FILL axis**, not through a
second codepoint: `star`, `star_border`, `star_outline` and `grade` all map to
`f09a`. The bundled variable font carries that axis (`FILL 0..1`), but
`ProvideMaterialSymbols` pinned it to `MaterialSymbolsDefaults.FILL` (0) for the
whole tree, so there was no way to draw any symbol solid — a "filled" icon and
an "empty" one differed only by tint.
Adds a second family at FILL=1 on its own CompositionLocal, built beside the
outline one so both are allocated once per subtree rather than per call site,
and threads `filled: Boolean = false` through `rememberMaterialSymbolPainter`
and `Icon`. The two Font constructions collapse into one `symbolFont(weight,
fill)` helper so the variants cannot drift on the other three axes.
`filled` is ignored when the caller supplies its own `family`: that font is the
caller's (Amethyst's own icon font, via `AmethystIconGlyph`) and need not have
the axis at all.
Additive with a default, so no existing call site changes. Requires API 26 for
`FontVariation`, which is the project minSdk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwk8tDEaEvsNoGbtrjZavz
The first cut fell back to `findNotesStartingWith` whenever the filter path came
up empty, so every zero-result keystroke scanned the whole cache twice — while
typing, which is exactly when it is worst.
The fallback exists for one reason: an id matches on `idHex`, which is not content
and so nothing a filter's `search` can reach. So only text that could name an
event takes it — a bech32 pointer, or a run of at least eight hex characters. An
ordinary query now scans once.
Also records the outcome in the plan: what shipped, the two things that changed on
contact with the code (step 7 became moot, so `FilterMatcher` stays untouched and
its blast radius never opens), and what is deliberately left — the user and channel
finders, which are name-prefix lookups rather than event filters and would be worse
expressed as one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
The token language shipped drawing chips that nothing acted on outside desktop's
relay path. `SearchBarViewModel` handed the raw box — chips and all — to both the
local scan and the NIP-50 `search` string, so `from:npub1… bitcoin` asked relays
for the literal text of its own tokens and matched nothing, and `since:`/`#t`
narrowed neither side.
Both paths now go through the one builder:
- `searchPostsByText` parses the text and builds its three kind-group REQs from
`SearchFilterBuilder`, so `from:`/`to:` become `authors`/`#p`, dates become the
window, and only the leftover terms travel as `search`.
- `LocalCache.filter` grows a predicate overload — the place for everything a wire
Filter cannot say — and `CacheSearch.findNotesMatching` runs the same filters
against the cache under it. Local and relay results stop disagreeing about what
a query means.
The predicate carries the two things that are not filter fields: the NIP-50
`search`, via one `EventSearchMatcher` per filter reused across the scan, and
viewer policy — mute list, unsearchable kinds, encrypted content — which is the
reader's business and not a relay's.
Notably this means `FilterMatcher` never had to learn `search`, so the blast
radius the plan worried about (33 feed filters, FilterIndex, geode's MirrorWorker
all silently narrowing) never opens. Search opts in by composing a matcher; every
other caller is untouched. Plan step 7 is moot.
A full-text query still falls back to the old scan when the filter path finds
nothing, so no existing search gets worse while the two are compared in the wild.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
`indexableContent()` is a write-path API. A store calls it once per event on
insert, so joining a string there is free — and 86 of the 126 implementations do
exactly that, via `listOfNotNull(...).joinToString("\n")`: a list, a
StringBuilder and a joined String per call. The JSON-backed kinds are worse;
`MetadataEvent.contactMetaData()` has no cache, so kind 0 reparses its profile
every single call.
Matching a query against the whole cache inverts that cost — once per event per
keystroke — so this adds a read path beside it rather than changing it:
fun interface IndexableFieldVisitor { fun visit(field: String?): Boolean }
fun forEachIndexableField(visitor: IndexableFieldVisitor)
A `fun interface` rather than a lambda parameter, because an interface method
cannot be inline and a lambda written at the call site would allocate per event —
which is the entire thing being avoided. One visitor is built per scan and
carries the term, so the walk allocates nothing and stops at the first field that
matches: a hit on the title never builds the body.
`indexableContent()` is untouched and stays the store contract, so the
externally-mirrored kind table needs no reindex. The default visitor falls back to
it — already free for the ~28 kinds whose indexable content is `content` itself —
and the kinds local search actually scans override it: text notes, long-form,
wiki, highlights, classifieds, live activities and community definitions.
`EventSearchMatcher` matches an event against a NIP-50 search string in memory:
terms ANDed, each a case-insensitive substring of a tag value or an indexable
field, with unsupported extensions ignored per the spec so an extensions-only
search matches everything rather than nothing. Substring because that is what
Amethyst's local search has always done and tokens would silently stop matching
mid-word; AND because that is what a relay does with the same string.
A test pins the two paths together: for every overriding kind, the visitor's
fields rejoined must equal `indexableContent()` byte-for-byte, including the
subsets where a null field is what makes a join drift.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
Two flakes in ImageDiskCacheReconcilerTest, both mine, both green locally and
red on CI.
aStartAfterTheIntervalIsDueAgain measured the interval from the clock it
sampled, but isDue() compares against the marker's file system mtime. A file
system that keeps mtime at whole-second resolution reads the marker back up to
a second before the write that made it, so `now + interval - 1` was already past
the interval and the pass ran when the test expected it skipped. Reproduced
exactly by truncating the marker's mtime to its whole second: same result object
CI reported, ceilingBytes and all.
It now pins the recorded pass to a whole second and measures from that, so the
boundary holds at any mtime resolution — and asserts the file system kept the
value, so an environment that cannot would fail loudly instead of flaking.
Renamed to theIntervalIsMeasuredFromTheRecordedPass, which is the property.
aSecondStartWithinTheIntervalSkipsTheWalk asserted the directory's byte total
was unchanged across the skipped call. Coil evicts asynchronously on its own
scope and the drainer unlinks behind it, so the two measurements raced both:
CI saw 24103 where the test had recorded 25127. What the test needs to rule out
is a wipe, and clear() takes DiskCache.size to zero — so it asserts on that
instead, which no amount of eviction churn can move.
Verified by running the class ten times, and by running CI's own task list
locally (both lintBenchmark variants and both unit-test variants — the pre-push
hook covers only testPlayDebugUnitTest, which is how these reached CI).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RYCgbvhtBCBNVLMxSWoCJ
`FilterMatcher.match` runs once per event per candidate filter. On a full-cache
scan that is tens of thousands of calls per keystroke, and it allocated three
different ways inside that loop:
- `tags.forEach { tag.value.toSet() }` built a Set per event, per tag key.
- `tagsAll` built a MutableSet per event per key, via a full walk of the event's
tags, before checking anything.
- `event.tags.any { }` allocated an iterator over the tag array each time, where
the repo's hot-path rule calls for the in-place `fast*` operators.
All three are replaced by indexed loops that read the tag array in place. The
filter's own value lists are short — a handful per key — so a linear `contains`
over them beats hashing and needs no allocation to do it.
No API change, so all 33 feed filters, FilterIndex and geode's MirrorWorker get
this without touching their call sites.
Guarded by a differential test that keeps the previous implementation verbatim as
an oracle and fuzzes 20,000 random event/filter pairs against it — including the
edges a hand-written set would have missed: empty value lists, absent tag keys,
duplicate values, short and empty tags, and `tagsAll` values spread across two
tags. Plus explicit since/until inclusivity, which the rewrite restates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
Records why local search cannot use the generic path today (FilterMatcher takes
every NIP-01 field except `search`), what CacheSearch does that a wire Filter
cannot express, and the two performance findings that shape the design:
- `FilterMatcher.match` allocates inside the per-event loop — a Set per event per
tag key, a MutableSet plus a full tag scan for tagsAll, and stdlib `any` on the
tag array where the hot-path rule calls for `fastAny`. Tolerable at feed-rebuild
rates, not on a full-cache scan per keystroke, and the search field's own
multi-spelling tag filters make it worse. The fix — hoisting loop-invariant work
into a prepared matcher — pays for every existing caller, so it leads.
- `indexableContent()` is a write-path API: 86 of 126 implementations allocate a
list, a StringBuilder and a joined String per call, and the JSON kinds reparse
on every call with no cache. Proposes a `fun interface` field visitor that
allocates once per scan rather than once per event and short-circuits on the
first hit, with `indexableContent()` kept and derived so its output stays
byte-identical for the stores and the externally-mirrored kind table.
Also records the correctness hazard that decides sequencing: `Filter.match` is
reached by 33 feed filters, FilterIndex and geode's MirrorWorker, so honouring
`search` must be opt-in until those are audited.
Three decisions are left open for a human: substring vs token matching locally,
where viewer policy applies, and whether relevance ordering needs a score out of
the matcher.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
The first pass was functional but plain: it parsed PublicationIndexEvent.image()
and .summary() and then used neither, so a book review card had no book cover —
just a generic MenuBook glyph sitting in the slot where the artwork belongs.
- Cover-led card. MyAsyncImage in a 2:3 portrait thumbnail (a book cover and a
film poster share that ratio), with the repo's existing loading/error
fallbacks, falling back to a tinted plate carrying the mark's icon.
- The icon now follows the mark rather than always being a book: movies, a
profile, a relay and a hashtag each get their own, with Article as the
generic. Adds MaterialSymbols.Movie (U+E404) and regenerates the subset font.
- Subtitle line shows the author, or the summary when there is no author.
- One RatedTargetCard for both the resolvable and unresolvable target, so the
two cannot drift apart visually. The cover is a slot rather than three
nullable image parameters.
- The star row was invisible to screen readers: five icons with a null
contentDescription announce nothing. It now carries one merged
contentDescription ("Rated 4.5 out of 5").
- The numeral appears only for a fractional score. It was redundant next to
five filled stars, and it was previously shown only for exact halves, which
silently dropped the .2 of a 4.2.
- Empty stars are tinted at 0.4 alpha so the filled ones carry the eye.
- Theme size tokens instead of hardcoded dp.
- Second @Preview for the whole card, including the long-title and
no-cover cases.
Not a <plurals>: the score is fractional, and plurals need an integer quantity.
The string avoids a counted noun so no locale has to decline it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
Kind 34259 was dropped at LocalCache's "Event Not Supported" fallback, so it
never became a Note and nothing could render it. Same for kind 30040, the
publication these ratings point at.
The upstream spec (XYZ.md in abh3po/nostr-polls, "Pollerama") is a generic
"rate anything" addressable kind with three tags — d, m, rating — so
EntityRatingEvent is generic too; books is just the first `m` mark rendered
richly. The a/A/e/k/p/s/c tags publishers add on top are parsed as extensions.
Two parsing traps drove the design:
- `["rating", "1"]` is ambiguous: 1.0-of-1 to a spec publisher, 1-of-5 to a
raw-scale one. stars() resolves it with a four-step ladder — an `s` tag
inside 1..5 wins outright, then a 0..1 fraction (closed interval, because a
full score really is published as "1.000" despite the spec's "less than 1"),
then a raw 1..5 count, then null. Never zero, which would misreport the
author.
- `d` carries a `<mark>:` prefix, so the coordinate has to be stripped back
out before it parses as an Address. `a`/`A` are preferred when present.
Feed visibility needs three gates opened, all of them: the REQ kind list, a
HomeFeedType group (which drives both the REQ strip and the DAL), and
HomeNewThreadFeedFilter's ADDRESSABLE_KINDS — the last is required because
feed() only scans LocalCache.notes for kind < 10000.
No computeReplyTo branch: the a/e tags name what is rated, not a parent, and
populating replyTo would flip isNewThread() and silently drop the card out of
the New Threads tab. A test pins that.
PublicationIndexEvent (30040, NKBIP-01) is parsed so the rated work resolves
to a real title; the reader and kind 30041 sections are deliberately not
implemented.
Adds MaterialSymbols.StarHalf (U+E839, its own glyph) and regenerates the
subset font. Note that star/star_border/star_outline sharing U+F09A is NOT a
bug — Material Symbols expresses fill through the FILL variable axis — so
filled and empty stars are one glyph at two tints.
Tests: 44 new, full suites green (quartz 4491, commons 1788, amethyst 1415).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
The search field shipped with its own people lookup — a `mentionQuery` flow over
`LocalCache.search.findUsersStartingWith` and a hand-rolled row list. Amethyst
already had a better one: `UserSuggestionState` + `ShowUserSuggestionList`, the
@-mention picker behind the post composer, the channel and Concord composers,
badge awards and the group member screens.
That duplicate was not merely redundant, it was worse. It resolved no NIP-05, so
`from:vitor@nostr.com` found nobody while `@vitor@nostr.com` in a post found the
right person; it never asked the search or indexer relays, only the local cache;
and it ranked nothing, so follows did not come first.
`commons` cannot call that stack — it is built on `Account` and
`AccountViewModel` — so `TokenizedSearchField` takes a `peoplePicker` slot and
keeps only what is genuinely shared: when the picker opens, and what a pick
splices into the text. Android fills the slot with the composer's list. Desktop
keeps its `UserSearchEngine` path through the built-in, keyboard-walkable list,
which the slot deliberately does not replace: a slot owns its own selection
affordance, so the arrow keys stay with the caret while one is up.
Also fixes an inconsistency in the desktop advanced panel: its "Until" box
resolved to the *start* of the named day, so a day typed there excluded almost
all of itself while the same day written as an `until:` token included it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
The relay's web client parses its search box into NIP-01 filter fields —
`from:`/`to:`, `since:`/`until:`, `#tag`, `label:`, `group:` and the NIP-73
scopes — rather than handing everything to NIP-50. Filter fields are index
lookups, so a chip narrows a search without competing with the relay's own
relevance ranking. This brings that language, and the field that draws it, to
Amethyst.
The scanner is the single source of both the chips and the REQ, so a chip can
never claim a filter the query does not send:
- `SearchTokenizer` scans the typed string into segments that cover it exactly,
each carrying its raw text so the field can draw over precisely the
characters it stands for. A corrupt value — a failed bech32 checksum, a day
that does not exist, a scope that asks nothing — stays plain text.
- `SearchFilterBuilder` turns a query into the filters it sends. A hashtag is
three questions (`#t` on the event, `#l` on a label, `#i`/`#I` on comments
written about it) and one filter ANDs its tag fields, so those fan out into a
union with a smaller limit on the secondary arms.
- `PartialTokens` says which half-written token the caret is in, and therefore
which picker belongs under the field. Derived from the text and the caret, so
a blurred field picks up where it left off.
- Dates are the reader's own local day, inclusive at both ends: a search saved
in one timezone and reopened in another still names the same day. `LocalClock`
is the only platform surface; the calendar arithmetic is pure and tested.
`QueryParser` now runs on top of the scanner and keeps its looser second pass
(`kind:`, `lang:`, `domain:`, `OR`, `-exclusions`, quoted phrases) over what is
left, so nothing that parsed before stops parsing.
On the UI side `TokenizedSearchField` draws tokens as chips over a plain text
value — the value is never rewritten, so undo, IME, selection and copy-paste
keep working and a query survives being shared as text. Wired into the Android
search bar, the desktop search screen and the desktop spotlight; the desktop
`SearchFilterFactory` now delegates to the shared builder and keeps only its
kind window.
Adds 68 tests, including every offset of every fixture through the field's
offset mapping in both directions — Compose crashes on an out-of-bounds answer.
Not verified: the iOS `LocalClock` actual. The Kotlin/Native toolchain cannot be
fetched in this environment, so that file is written against Foundation APIs the
repo already exercises and is unbuilt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017yKjw2WqwZpSzsqcYZMnkV
Kind 34259 has no parser, no LocalCache branch and no renderer today, so
the events are dropped at LocalCache's "Event Not Supported" fallback.
The plan covers: a generic EntityRatingEvent in quartz/experimental/ratings
(the upstream Pollerama spec is entity-agnostic — books is only one `m`
mark), the two parsing traps (`rating` is ambiguous at exactly 1 unless the
`s` tag disambiguates; `d` carries an `m:` prefix), the LocalCache
addressable branch, and the three separate gates that must all open for a
kind to reach the Home feed (the REQ kind lists, a HomeFeedType group, and
HomeNewThreadFeedFilter's ADDRESSABLE_KINDS + acceptableEvent).
Also records why v1 skips computeReplyTo (it would flip isNewThread and
drop the card out of the New Threads tab), the kind-30040 dependency for
resolving the rated publication's title, and that MaterialSymbols.Star and
StarBorder currently share one codepoint so a star row needs the subset
font regenerated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013NSumuEaxE4D1jvStUFw7M
The reconciler ran on every AppModules.initiate(), which is every process
start — including the WorkManager wake-ups that cold-start the whole graph, and
which the ledger already counts because that churn is a known problem. Even the
healthy pass is a readdir plus a stat per file: on a full 1 GB cache that is
tens of thousands of syscalls, paid on every start, to look for drift that
accrues only when a process dies with unlinks still queued.
reconcileIfDue() gates the walk on the mtime of an empty `.reconciled` marker in
the cache directory, so a start inside the interval costs one stat instead. The
marker lives with the thing it describes: clearing the app's cache from Settings
takes it too, and the next start reconciles a directory whose history we no
longer know.
A marker dated in the future — a clock that jumped back, or a restored backup —
counts as due, so it cannot park the check until real time catches up.
The marker is a plain file in the swept directory, so it joins the journal files
in the preserved set. reconcileIfDue() rewrites it after a pass anyway, which is
exactly why theMarkerSurvivesAWipe drives reconcile() directly — through
reconcileIfDue() the rewrite masks the deletion and the test guards nothing.
Verified by mutation: dropping the marker from the preserved set fails that
test, and short-circuiting isDue() fails both interval tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RYCgbvhtBCBNVLMxSWoCJ
DeferredDeleteFileSystem moves Coil's eviction unlink() off the DiskLruCache
lock by queueing it in memory. Anything still queued when the process dies is
never unlinked — and Coil cannot recover it: DiskLruCache.processJournal()
derives `size` purely from the journal's recorded lengths and never scans the
directory for files it does not know about. The orphan counts toward neither
`size` nor eviction, so the cache directory keeps the residue of every killed
process, forever, drifting past its own cap.
Measured against the real Coil DiskCache: 48 unlinks lost to one process death
left Coil reporting 16 KB against a 16 KB budget while the directory actually
held 48 KB, and reopening as a fresh process reclaimed none of it. With
maxSizePercent(0.2) capped at 1 GB, that drift is unbounded over the app's life.
ImageDiskCacheReconciler runs once at startup on the IO scope: it walks the
cache directory and, only if it holds more than its budget plus slack, empties
it. Two steps, because DiskCache.clear() alone is not enough — evictAll() walks
lruEntries, exactly the set of files the journal knows about, so it goes right
past the orphans. So: clear() to make the journal's truth empty, then unlink
every non-journal file left in the directory, which is by then unreferenced by
definition. The unlinks go through the same deferred file system as any other
eviction; the pending set dedupes the paths clear() already queued.
It is deliberately blunt — it costs the whole cache — so the trigger sits well
past what normal operation needs: budget + max(25%, 4 MiB). Async eviction and
the dirty files of writes in flight both put a healthy directory a little over
budget; only real drift is wiped. On a healthy cache the pass is one directory
walk and nothing else, and it never reads DiskCache.size, so it does not force
the journal parse on the happy path or couple to Coil's on-disk format.
The startup call is also the one place that forces the `diskCache` lazy, so its
build (a statvfs for the size budget) and this walk both land on IO rather than
on whichever thread happens to load the first image.
Tests cover the end-to-end leak (inert drainer stands in for the dead process,
then a fresh cache over the same directory reconciles it back under budget), the
healthy no-op, drift inside the slack, a missing directory, and the shipped
ceiling arithmetic. clearAlone_wouldNotHaveReclaimedThem pins the reason this
class exists — if Coil ever learns to sweep, it fails and the class can go.
Verified by mutation: dropping the orphan sweep fails the leak test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RYCgbvhtBCBNVLMxSWoCJ
The identity check that cost animated images their file source cost still
images a decoder outright: `StaticImageDecoder.Factory` returns null when it
cannot get an `ImageDecoder.Source`, so with our `DeferredDeleteFileSystem`
on the source every network still image fell through to
`BitmapFactoryDecoder`.
Move the re-home from the animated decoder to the fetcher, where it fixes
both at one seam. `SystemFileSystemFetcher` wraps the three network-backed
fetchers the app builds (`OkHttpFactory`, `BlossomFetcher`,
`ProfilePictureFetcher`) and re-points a disk-cache-backed result at
`FileSystem.SYSTEM` before any decoder sees it.
Doing it here rather than at each decoder:
- One wrapper covers every decoder, including Coil's own registered
`StaticImageDecoder.Factory`. Adding a second static factory would instead
have introduced a second decode-parallelism semaphore alongside the one
Coil's bitmap decoders share, and put us in charge of registry ordering
against SVG and video frame decoding.
- It is the only place that still knows the disk cache key.
`FileImageSource.diskCacheKey` is internal to Coil and cannot be copied off
an existing source, but `NetworkFetcher` derives it as
`options.diskCacheKey ?: url` — so the wrapper is handed the same value and
`SuccessResult.diskCacheKey` survives the swap.
The re-homed source takes ownership of the one it replaces (`closeable =
this`), since the engine closes exactly one source per fetch and that is now
the replacement — without it the disk-cache snapshot would leak.
`AnimatedImageDecoderFactory` keeps only the frame-delay scan, which is a
separate matter: Coil's sub-threshold rewrite is stream-backed and would
forfeit the file source again for GIFs on API < 34.
Tests: `SystemFileSystemFetcherTest` covers the swap, the pass-throughs
(already on SYSTEM, unknown wrapper, stream-backed, non-source result,
declining fetcher), the ownership transfer and the carried key — the last
two verified by mutation. `SystemFileSystemImageDecoderInstrumentedTest`
pins the platform half the JVM tests cannot reach: that
`toImageDecoderSourceOrNull` really does return null on our disk cache's
file system and non-null once re-homed, and that `StaticImageDecoder.Factory`
declines the former and accepts the latter.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RYCgbvhtBCBNVLMxSWoCJ
A 69.8 MB, 1138x640, 201-frame GIF from a Ditto note froze the feed. The
media is pathological, but the app made it far worse than it had to be.
Coil hands `android.graphics.ImageDecoder` a *file* only when the image
source's file system is **referentially** `FileSystem.SYSTEM`
(`ImageSource.toImageDecoderSourceOrNull`, Coil 3.5.0):
if (fileSystem === FileSystem.SYSTEM) {
val file = fileOrNull()
if (file != null) return ImageDecoder.createSource(file.toFile())
}
`NetworkFetcher` stamps the source with `diskCache.fileSystem`, and ours is
a `DeferredDeleteFileSystem` wrapper, so that identity check fails for every
image we fetch from the network. For an animated image the fallback is
`ImageDecoder.createSource(source.squashToDirectByteBuffer())`: the entire
encoded animation is pulled onto the heap and then copied into an equally
large direct `ByteBuffer` that stays alive for as long as the
`AnimatedImageDrawable` does. Replaying that path over the reported GIF
measured 66 MB of heap plus 66 MB of native memory, and ~700 ms of pure
copying on desktop x86. Animated results are never memory-cached
(`DrawableImage.shareable` is false), so the feed paid it again on every
scroll back into view.
Coil then compounds it below API 34: it wraps every GIF in a stream-backed
`FrameDelayRewritingSource` to clamp sub-threshold frame delays, which
forfeits the file fast path even when the identity check would have passed.
So:
- `onSystemFileSystem()` re-points a disk-cache-backed source at the real
`FileSystem.SYSTEM` before it reaches a decoder. Only deletes are deferred
by the wrapper; reads already go straight through.
- `hasSubThresholdGifFrameDelay()` streams a 64 KiB window over the file and
asks for the rewrite only when a graphics control block really declares a
delay below 2/100 s. The reported GIF declares 5 on all 201 frames, as do
the overwhelming majority of GIFs, so they now decode from the file with
no heap copy at all. Files that do need the clamp keep Coil's behaviour.
- `AnimatedImageDecoderFactory` replaces Coil's factory with the same sniff
and those two changes; `AvifAnimatedDecoderFactory` shares it.
Still images take a similar hit from the same identity check (they fall back
from `StaticImageDecoder` to `BitmapFactoryDecoder`) — left alone here since
it changes the decode path for every image in the app.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RYCgbvhtBCBNVLMxSWoCJ
Kotlin/Native rejects backtick identifiers containing "," ("Name contains
illegal characters"), so `:commons:compileTestKotlinIosSimulatorArm64` has
failed since #4059 moved these relay-group tests from the app's JVM-only
test set into commonTest. JVM never minded, which is why the pre-push hook
stayed green. Reworded the three names; behaviour unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J836W9ZjSUJUQ1d23TpiJo
The relay-group, connected-apps and poll-responses files moved into commons
still linked to classes by their old amethyst paths (or to app-only screens
commons cannot see). Point them at the commons class where one exists and
use plain text where the target stays in the app.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J836W9ZjSUJUQ1d23TpiJo
The four families that read one relay-set state off `Account` now carry that
set as a `StateFlow` on their key, the way `SearchQueryState` does:
`ProfileAppRecommendationsQueryState(outboxRelays, defaultGlobalRelays)`,
`ProfileBadgesQueryState(notificationRelays)`,
`ConnectedAppsQueryState(homeRelays)` and `ThreadQueryState(defaultRelays)`.
Thread also takes the cache: `ThreadFilterAssembler(cache, client)`, and its
two sub-assemblers run `ThreadAssembler(cache)`.
The home feed moves whole. Its `FilterHomePosts*` dispatchers were already
import-clean; `HomeOutboxEventsEoseManager` becomes a
`TopNavFeedSubAssembler<HomeQueryState>` whose key carries the new-threads
and replies floors plus `enabledHomeFeedTypes`. The Settings › Home toggle
stays an unsampled invalidator (still `drop(1)`) via `extraInvalidators`,
and the disabled-kinds stripping is unchanged. Same deltas as the other
feeds: watchers on the screen scope, follows sampler 500 ms (was 1000 ms).
The commented-out alternative sub-assemblers in `HomeFilterAssembler` are
dropped.
The app keeps one `*Subscription.kt` per feature, building the key from
`AccountViewModel`. The algo-feed filter test moves to commons commonTest
on `kotlin.test`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J836W9ZjSUJUQ1d23TpiJo
Review findings on the three migration commits, all verified:
- The shared feed-floor watcher sampled at 5 s for every feed, but the
video swipe feed used 1 s before the refactor, so paging older videos
waited up to five times longer. The interval is now a `floorSampleMs`
override and Video keeps its 1 s cadence.
- `endSub` cancelled the watcher jobs but never dropped the `userJobMap`
entry, leaving a cancelled job list per account ever subscribed on every
app-lifetime assembler. It now removes the entry.
- `DesktopLocalCache.consume(nip19)` was a copy of the Android body. The
NIP-19 seeding is now one default on `ICacheProvider`, built only from
interface members, with a single `consumeEmbedded(event)` hook for the
`nembed` branch (Android verifies via `justConsume`, Desktop via its
`consume(event, relay)`). The default uses `checkGetOrCreateNote`, so an
invalid id yields no placeholder instead of an exception.
- `allRelayGroupChannels()` dumped the group cache through a predicate that
always returned true; it now snapshots `values()`.
- The music/podcast kind lists go back to `internal`: their only callers
moved into commons with the dispatchers.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J836W9ZjSUJUQ1d23TpiJo
The datasource families whose keys are a User, an AddressableNote, a string,
or an IAccount-compatible account — chess, relay feed, NIP-66 relay info,
url, geohash, the channel assembler, communities, git repo, profile, one/my
podcast, onchain zaps and poll responses — move wholesale into
`commons/relayClient/<feature>/`, along with the pure thread filter
functions and `FilterPostsByScopes.kt` (`CommentKinds`), which those filters
share with the home feed. Three Subscription composables that never touched
`AccountViewModel` (community, repository, profile) come along; the ones
that do stay in the app.
The only seam was `LocalCache`, read for `relayHints` and
`checkGetOrCreateUser`, both already on `ICacheProvider`. The affected
assemblers take `cache: ICacheProvider` first, thread it into their
sub-assemblers, and the filter functions that need it take it as their first
parameter; `RelaySubscriptionsCoordinator` passes the cache it already holds.
`PollResponsesQueryState` is typed on `IAccount`.
The URL filter test moves to commons commonTest on `kotlin.test`. The plan
doc records what stays behind and why.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J836W9ZjSUJUQ1d23TpiJo
The 26 datasource families that dispatch to the Batch-2 filters — an
`XFilterAssembler`, an `XSubAssembler` and an `Account`-typed `XQueryState`
per top-nav feed — were 26 copies of the same ~90 lines, differing only in
the `makeXFilter` dispatcher and which feed floor bounds `since`. They now
sit on three shared classes in `commons/relayClient/topNavFeeds/`:
- `TopNavFeedQueryState(account: IAccount, listName, followsPerRelay, scope,
feeds)` is the one key type. Discovery and SoftwareApps subclass it for
their seven named tabs and the blocked-relay set respectively.
- `TopNavFeedSubAssembler<K>` carries the shared EOSE-manager wiring (list
watcher, per-relay follows sampler, combined feed-floor sampler, plus
`onListChanged` / `extraInvalidators` hooks); `SingleTopNavFeedSubAssembler`
is the concrete one every single-dispatcher feed uses, with the calendars'
EOSE reset as a flag.
- `TopNavFeedFilterAssembler<K>` owns the sub-assemblers, so each feature's
assembler is a one-liner naming its dispatcher and the coordinator and the
`*Subscription` composables keep their types.
The pure `makeXFilter` dispatchers moved next to their filters. The app keeps
only the Compose glue: each `XFilterAssemblerSubscription` builds the key via
`AccountViewModel.topNavFeedQueryState(...)`, which is where the `Account`,
`AccountSettings` and `AccountFeedContentStates` reads now live. Video is
refit onto the shared key, replacing the `VideoQueryState` from the previous
batch, with its inline dispatch extracted to `makePictureAndVideoFilter`.
Small deliberate deltas: all watchers run on the screen scope (the floor
watcher used `account.scope`), the follows sampler is 500 ms everywhere
(two feeds had 1000 ms), and Video gained the list-name watcher.
The plan doc records this batch and the desktop `FilterBuilders.kt`
investigation: it is a plain-`Filter` broadcast factory, not a copy of the
subassemblies, and the three ways to reconcile the two models are written
up for a decision.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J836W9ZjSUJUQ1d23TpiJo
The audit's Batch 2 — the 161 `ui/screen/**/subassemblies/` filter functions
that turn a top-nav selection into per-relay REQs — was never executed by the
earlier waves. They are import-clean (no Android, no R, no Account), so they
move mechanically into `commons/relayClient/<feature>/…`, dropping the
`subassemblies` path segment to match how Batch 1 landed.
Pure companions the 161 needed came along: the picture/video kind lists
(`FeedBasis.kt`), `RelayGroupFilterBuilders.kt` with its test, and the two
`*_PAGE_LIMIT` constants. The three query-state keys the sub-assemblers are
typed on now live in commons on `IAccount`: `ChannelQueryState`,
`SearchQueryState` (carries the search / indexer / follow-plus-mine relay
sets as flows) and `VideoQueryState` (carries the list name, per-relay
follows and feed-floor flows), so nothing in commons reaches into `Account`
or `AccountFeedContentStates`.
`ICacheProvider` grows the seams the six `LocalCache`-coupled files needed:
`consume(nip19)` for search-query seeding, plus `allRelayGroupChannels()` /
`getRelayGroupChannelsOnRelay()` for the relay-group discovery back-fill.
`DesktopLocalCache` implements `consume`; the relay-group lookups default to
empty for caches without NIP-29 support. The search `filterBy*` and the
set-level relay-group filters take the cache as a parameter instead of the
`LocalCache` singleton.
The four tests moved to `commons/src/commonTest` on `kotlin.test`, and the
misnamed `shorts/FilterPollsByAllCommunities.kt` is now
`FilterShortsByAllCommunities.kt`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J836W9ZjSUJUQ1d23TpiJo
UI_APPLIED_NIGHT_MODE existed to skip a repeat setApplicationNightMode call
on launches where the theme had not changed. Reading the platform, that
saves much less than it looked like, and costs more.
What the call actually does when the mode is unchanged: nothing expensive.
PackageConfigPersister.updateFromImpl compares the new mode against the
stored record and returns early without writing when it matches, and
ActivityRecord.applyAppSpecificConfig gates the activity reconfiguration on
having actually changed. What remains per launch is one Binder round trip,
already off the main thread via flowOn(Dispatchers.IO).
Against that, the key was a private copy of state the app does not own.
There is no public getter for the per-application override, so it could
only ever be a shadow -- and a shadow that drifts (anything resetting the
override out from under us) makes the app skip the call precisely when it
is needed, leaving the splash silently wrong with no way to recover.
Re-sending the value every launch converges instead.
This is also why the deduplication in applyLanguage does not generalise to
here, though the two look alike: that one compares against
getApplicationLocales(), the authoritative value, and was justified by a
measured ~220ms main-thread cost with a StrictMode violation behind it.
Neither property holds for night mode.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDAVkn1krKjijoBdja1Ruo
The previous commit made the splash follow the phone's light/dark setting,
which is correct for the default ThemeType.SYSTEM but still wrong for
someone who pins LIGHT or DARK against their phone: the system composites
the splash from the manifest theme before the process starts, resolving it
against the app's configuration, and the in-app preference lives in a
DataStore the system knows nothing about.
UiModeManager.setApplicationNightMode is the mechanism that closes this.
It commits a persisted *per-package* configuration override — the service
hands the mode to ActivityTaskManagerInternal.PackageConfigurationUpdater —
which the system then applies when it launches the app, splash included.
This is the per-application setter, not the device-wide setNightMode that
726f3e39 removed. That removal was right: setNightMode changes every app
on the device and is gated behind MODIFY_DAY_NIGHT_MODE, which Amethyst
does not hold, so it was a silent no-op. The per-application setter is the
alternative the framework docs point app developers at, and it is not
permission-checked — UiModeManagerService.setApplicationNightMode only
validates the mode argument before committing.
Mapping, per the same service method: DARK -> MODE_NIGHT_YES, LIGHT ->
MODE_NIGHT_NO, SYSTEM -> MODE_NIGHT_AUTO. AUTO is not "auto" here; the
service maps everything other than YES/NO onto UI_MODE_NIGHT_UNDEFINED,
which clears the override so the app falls back to the device config —
exactly what SYSTEM wants when a user un-pins.
Wired next to applyLanguage in UiSharedPreferences, which already does the
same shape of work for AppCompat locales: an eagerly-started flow on the
preference, off the main thread, deduplicated so an unchanged value costs
nothing. Deduplication matters because the call is a Binder round trip that
pushes a configuration change into every running activity of the package.
The applied mode is recorded only after the call returns, so a failure is
retried next launch rather than remembered as done, and it is kept out of
UiSettings because it is bookkeeping rather than a user setting.
Two properties that make this safe, both checked rather than assumed:
MainActivity already declares `uiMode` in its configChanges, so the
resulting configuration change goes to onConfigurationChanged instead of
recreating the activity; and the write is guarded to API 31+, where
setApplicationNightMode exists (verified present in android-37).
The effect lands on the next cold start — the current launch's splash is
already painted by the time any app code runs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDAVkn1krKjijoBdja1Ruo
726f3e39 pinned `windowSplashScreenBackground` to `@color/purple_700`
(#3700B3) in both `values/` and `values-night/`. That fixed a real bug —
the splash used to inherit the AppCompat window background and could flash
the wrong colour — but by writing the same value into both qualifier
buckets it also removed the system's ability to pick one, so every launch
showed the same deep indigo regardless of the phone's theme. #3700B3 is
also a leftover from the Android Studio Material 2 template: it appears
nowhere in the Compose colour schemes, whose brand purple is #9A82DB.
The system composites the splash from this theme before the process
starts, resolving it against the phone's UI-mode configuration, so the
day/night resource qualifiers are already the mechanism for following the
phone. Restore that by giving each bucket its own colour, and point both
at the value MaterialTheme itself uses for `background`:
values/ splash_background = #FDFDFD (lightColors.background)
values-night/ splash_background = #000000 (darkColors.background)
Matching the scheme rather than a brand colour means the splash hands off
to the first composed frame with no visible step.
`windowBackground` is set to the same colour in both buckets — it is what
paints the splash below API 31 (minSdk is 26), so pre-31 devices now track
the phone's theme too. It stays an opaque colour, so the window remains
opaque and SurfaceFlinger can still skip the layers beneath it; that was
the ~17% frame-P90 regression 726f3e39 measured when the background was
cleared, and it is unaffected here.
Verified by dumping the linked resource table with aapt2: color/
splash_background resolves to #fffdfdfd by default and #ff000000 under
(night), and Theme.Amethyst carries the attribute in both the () and
(night) variants.
Note this follows the *phone's* setting, which is correct for the default
ThemeType.SYSTEM. A user who pins LIGHT or DARK against their phone still
gets a splash matching the phone, because the in-app preference is not
visible to the system at launch time. Making the splash follow the pinned
preference needs UiModeManager.setApplicationNightMode, which is a
behavioural change and is left for a separate commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDAVkn1krKjijoBdja1Ruo
Both entries in crowdin.yml are declared `type: android`, so Crowdin's
Android serializer escapes apostrophes on the way down: `l'URL` comes
back as `l\'URL`. That is right for amethyst/src/main/res/, which aapt
un-escapes at build time, and wrong for commons/.../composeResources/,
where Compose resolves only \uXXXX, \n and \t and leaves \' \" \? \@
alone -- so the backslash reaches the screen.
Nothing prevented this, so every sync reopened the same regression and
CI's compose_escaping_check.py failed on the bot's own PR. It happened
three times (f9baab0e, 1685d7c0, e223d505), most recently 2,888
occurrences across 40 locale files, each time repaired by hand after the
fact. Convert on the way in instead, so the PR is born clean.
Two steps, placed after the ownership fix (the Crowdin container writes
as root, so the tree is not writable before it) and before the PR is
opened:
- Convert: runs the documented repair over the Compose catalog only, so
the Android res tree keeps the escaping it needs. --no-unwrap-quotes
is mandatory -- escape conversion is idempotent, quote-unwrapping is
not, and a second unwrap would strip the real display quotes from
values like import_follows_tips.
- Verify: re-runs the check that guards main, so a case the converter
cannot repair fails the sync loudly here instead of opening a red PR.
Verified by replaying both steps against the real Crowdin output on
l10n_crowdin_translations (df930817): the check reproduces the failure
at 2,888 occurrences, the convert step fixes 1,996 entries across 40
files, the verify step then exits 0, amethyst/src/main/res/ is left
untouched, and the resulting catalog is byte-identical to main -- so
with this in place that sync would have carried no string changes at
all.
Known gap, documented inline on the verify step: fix_escapes.py only
rewrites text inside <string>/<item> elements while the check scans the
whole file, so an escape in an XML comment (comments do propagate into
the locale files) would fail the gate without the converter being able
to repair it. No such comment exists today; it has to be fixed at the
source string by hand if one ever appears.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVrW3p8NGWHgLbN673Fet2
The Crowdin sync in b0ff30b0 reintroduced Android-only escaping into the
Compose resource catalog for the third time: 2,888 occurrences across 40
locale files (\' x2,832, \" x~470, plus a handful of \? and \@).
Compose's handleSpecialCharacters resolves only \uXXXX, \n and \t, and
collapses \\. It leaves \' \" \? \@ untouched, so these render with a
visible backslash -- "Utiliser l\'URL directe" instead of "Utiliser
l'URL directe". The apostrophe-heavy locales are hit hardest: uz-rUZ
(949), fr-rFR (341), fr-rCA (326), tr-rTR (189).
Repaired with the documented command:
python3 tools/strings-migrate/fix_escapes.py --no-unwrap-quotes \
commons/src/commonMain/composeResources
--no-unwrap-quotes is required here: quote-unwrapping is not idempotent,
and the two remaining quote-wrapped values (import_follows_tips in
values-es, messages_new_message_to_caption in values-tr) carry real
display quotes that a second unwrap would strip.
Verified: the diff is exactly the escape removal and nothing else (each
old file, with unescaped \' \" \? \@ backslashes stripped, is byte-equal
to its new version), all 40 files still parse as well-formed XML, no file
outside composeResources is touched, and both resource CI checks --
compose_escaping_check.py and orphan_strings_check.py -- now exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVrW3p8NGWHgLbN673Fet2
Follow-up to the crash fix, which changed what cancellation costs.
Inferences can no longer be recalled: awaitDetached() detaches instead of
cancelling, because cancelling the future is what killed the process. So
abandoning a running batch no longer stops anything — it leaves seven rewrites
burning on-device compute for text the user has already moved past, and since
precomputeAiResults() runs on every keystroke, each later pause stacked seven
more on top with nothing bounding the pile.
The composer now coalesces instead of abandoning. The debounce window stays
freely cancellable (nothing has reached the model yet), but once a batch's
inferences are under way it is left to finish and the newest draft text is
stashed in aiPendingText, picked up when that batch ends. In-flight work is
bounded at one batch however fast the user types, and per-batch latency is
untouched — the seven tones still run concurrently.
MLKitImageLabelService moves off ListenableFuture.get() onto awaitDetached().
Describing an image takes seconds, and get() held an IO thread for all of it
uninterruptibly, so backing out of the composer left the thread pinned until
AICore answered. This needs a CancellationException rethrow ahead of the
existing catch-all: now that the awaits suspend, a cancelled caller lands there
and must not be swallowed as "no suggestion".
Also drops MIN_CONFIDENCE/MAX_LABELS, which nothing has referenced since the
keyword image-labeling path was removed, and corrects the class KDoc that still
described that fallback.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E1mgCe29aaWUwGmGHKnFKo
Cancelling a `genai-rewriting` 1.0.0-beta1 inference future kills the process
from a thread we don't own:
Thread: AiCoreClientWorker-thread-5
java.lang.NullPointerException: Attempt to invoke interface method
'void com...mlkit_genai_rewriting.zzp.zzd()' on a null object reference
at com...mlkit_genai_rewriting.zzby.zzk
at com...mlkit_genai_rewriting.zzbt.run
at java.util.concurrent.ThreadPoolExecutor.runWorker
Disassembling the library pins it down exactly. `zzw.zzf` — the
`IMagicRewriteService` AIDL proxy — reads the returned `ICancellationCallback`
with `Parcel.readStrongBinder()`, which yields null when AiCore answers without
one, and passes that null on. `zzbh.attachCompleter` then registers it as the
future's cancellation listener with no null check
(`addCancellationListener(new zzbt(handle), ...)`), so cancelling the future
runs `zzby.zzk(null)` → `null.zzd()`. `zzk` catches only `RemoteException`, and
it all happens on ML Kit's own worker pool, so nothing we wrap can see it: the
NPE reaches the default uncaught handler and takes the app down.
The composer cancelled these routinely — a keystroke replaces the in-flight
batch of seven tones via `aiComputeJob.cancel()`, and leaving the composer
cancels `viewModelScope` — which turned a beta-library race into a routine
crash.
There is nothing to upgrade to: genai-rewriting, genai-proofreading and
genai-image-description have each published exactly one version. So the future
bridge now detaches instead of cancelling — `awaitDetached()` drops
`invokeOnCancellation { cancel(true) }` and skips reading the result once the
caller is gone. A cancelled batch's inferences finish with nobody listening,
which spends a little on-device compute where cancelling spent the process; the
composer's 1s debounce already keeps most stale batches from starting.
MLKitImageLabelService blocks on `.get()` and never cancels, so it is unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E1mgCe29aaWUwGmGHKnFKo
Two follow-ups from reviewing the artwork cap:
The platform MediaSession reads its metadata bitmap ceiling from the resources
of whatever context builds it, while the cap is measured against the app
context. PlaybackService happens to pass applicationContext today, so the two
agree; building the session with appContext directly makes that structural
rather than incidental, and stops a pooled session from holding an Activity.
Decoding to just under 2x the ceiling leaves the pre-scale bitmap holding up to
4x the pixels of the copy that is kept. It is exclusively ours — the delegate
decodes a fresh bitmap per request, and media3's caching wrapper sits above
this loader — so it is recycled as soon as the scaled copy exists instead of
waiting for the collector.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2qse1pZsCAsQ5XASgeqSK
`ConcordInviteBundle.classify` resolved a revocation tombstone by `createdAt`
(newest wins, per CORD-05 §2 replaceable semantics) but picked the live bundle
with `firstNotNullOfOrNull` — i.e. whichever copy happened to decrypt first in
relay arrival order. `wraps` comes straight off `fetchAll` at every call site
(`joinConcordViaInvite`, `amy concord join`, `refreshConcordInviteLinks`), and
`fetchAll` gives no ordering guarantee.
A Refounding re-mints every live link at its OWN coordinate carrying the new
epoch's root. Until now, a relay still serving the pre-Refounding bundle could
hand a joiner the root of the epoch the community had just left: they would
join, post into planes nobody reads, and see no error explaining why. The file's
own KDoc and the CLI's redeem comment both already claimed newest-wins here.
Sorting newest-first also decrypts fewer bundles in the common case, since the
current edition is now tried first.
Regression test covers both fetch orders.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcaCNiqN8T9izve4AXcake
Audit of the previous commit. limitToRouteTextArg measured the whole
value and then walked it again to find the cut, so a five-megabyte share
payload was scanned twice on the main thread to produce twenty kilobytes
— O(input) work in the one function whose job is to defend against
unbounded input. One pass now, returning the moment the budget is blown,
which costs the same for a megabyte as for a value at the limit.
Three smaller things from the same pass:
- A budget too small to hold the truncation marker returned the bare
marker, which is longer than the limit the function promises. It now
yields nothing.
- The measurement-free fast path divided by twelve, the cost of a
supplementary code point — but that cost covers two chars, so the real
worst case per char is nine. Drafts of 1,667 to 2,222 characters were
being measured for nothing.
- limitToRouteTextArgOrNull is gone: "OrNull" reads as "returns null on
failure", which it doesn't, and `?.` at the one call site says it
better. The marker's own cost is hoisted out of the call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gCs4kkfMP7vFPGRwHg65p
Sending a crash or resource-usage report to the dev crashed the app:
IllegalArgumentException: Navigation destination that matches route
…Route.Room/<pubkey>?message=<the whole report>… cannot be found in
the navigation graph
The destination is in the graph — it just cannot be matched. androidx
finds a destination by regex-matching the generated route string, and
NavDeepLink ends every path pattern with `($|(\?(.)*)|(#(.)*))`: a
capturing group inside a `*` loop, so the engine pushes one backtracking
frame per character of the query. On Android `java.util.regex` is
ICU-backed (com.android.icu.util.regex.MatcherNative), ICU caps that
stack at 8 MB, and on overflow it reports *no match* rather than an
error. Run against ICU with the Route.Room pattern, matching flips at
about 100,000 encoded characters:
uriLen 100091 -> match=1 status=U_ZERO_ERROR
uriLen 100191 -> match=0 status=U_REGEX_STACK_OVERFLOW
The same route matches fine on the JVM at 13 MB, which is why this only
shows up on device. The reports are exactly that size, and it compounds:
a crash report quotes the route it failed on, and re-encoding turns each
`%` into `%25`, so every round through the dialog roughly triples.
So keep oversized text out of the route in the first place:
- limitToRouteTextArg cuts a value down to a fifth of the measured
ceiling, measured in encoded characters (a space costs three, an emoji
twelve) and never through a surrogate pair.
- routeToMessage caps the chat draft — the funnel every prefilled Room
route goes through: crash and resource-usage reports, error toasts,
shares.
- Intent.sharedText caps what another app hands us, which is bounded
only by Binder and feeds all six share targets.
- ReportAssembler caps the throwable headline at 1,000 chars, which is
what breaks the compounding; the stack trace, the useful part, stays
whole.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gCs4kkfMP7vFPGRwHg65p
MediaSession.setMetadata() re-builds metadata through
MediaMetadata.Builder.build(), which scales every bitmap-valued key larger
than config_mediaMetadataBitmapMaxSize. media3 stores the *same* Bitmap
instance under both METADATA_KEY_DISPLAY_ICON and METADATA_KEY_ALBUM_ART, so
build() scales that one instance twice. AOSP leaves the source untouched, but
on ROMs that recycle it while scaling the second pass throws
IllegalArgumentException: cannot use a recycled source in createBitmap
on the main thread, inside a Guava future callback the app cannot intercept.
Artwork comes from arbitrary nostr imeta URLs, so it routinely arrives well
above the ceiling. The previous fix capped the decode at that ceiling, but
DataSourceBitmapLoader only subsamples by powers of two, so a 1080px image
under a 900px ceiling decoded to 540px — correct, but half the resolution the
session would have accepted. Decode to just under 2x and scale precisely
afterwards instead (the same recipe media3 uses for its own default loader),
via a small BitmapLoader wrapper that mirrors the platform's own scaling math.
The ceiling is also re-read per load rather than memoized once: it is a dp
value, and the app survives display-size changes without restarting, so a
density drop would otherwise leave the cap stale and too large.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2qse1pZsCAsQ5XASgeqSK
`ConcordInviteLink.encodeFragment` refused to encode more than
`MAX_RELAYS = 3` bootstrap relays, but both mint call sites —
`AccountConcordActions.mintConcordInvite` and `amy concord invite` — hand it
the community's full relay list. A community with more than three relays
therefore blew up the invite button with
`IllegalArgumentException: at most 3 relays, was 5`.
The cap was self-imposed: nothing in the fragment format needs it. The layout
is `[version][flags][count][relays...][token:16]` and the relay count is a
whole byte, so the format's own ceiling is 255. `MAX_RELAYS` is deleted and
the only remaining guard is that ceiling — without it a 256-relay list would
wrap the count byte to 0 and silently strand every relay in the fragment.
The stock set still collapses to flag `0x01` and zero relay bytes, so the
common invite is unchanged in length.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcaCNiqN8T9izve4AXcake
Wires compose_escaping_check.py into the two layers that already police the
identical orphan-strings desync, mirroring pre-push-orphan-strings.sh:
- the fast `lint` job in build.yml, next to the orphan check
- a PreToolUse hook, so a push or PR from an agent session is gated too
CI is the layer that matters here. As amethyst/src/main/res/CLAUDE.md records for
the orphan desync, these arrive through bot-authored PRs -- the Crowdin sync
reintroduced 2,068 escaped apostrophes across 40 locales twice in two days, with
no local session anywhere in the path.
Verified end to end: a payload with no push exits 0 without spawning python; a
push against a clean tree exits 0; a push with values-tr-rTR restored to its
pre-repair state exits 1 and names the file with a per-escape breakdown.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1CzYQvWyHfipSW7x3j4Yo
The Crowdin sync reintroduced Android escaping into the Compose catalog a second
time (PR #4046, commit 2129e16044): the same 2,068 escaped apostrophes and 749
escaped quotes across 40 locale files, always the apostrophe-heavy regional
variants -- uz-rUZ 949, fr-rFR 341, fr-rCA 326, tr-rTR 189. Compose resolves
neither, so those strings render with a literal backslash.
Repairing after each sync is not a fix: Crowdin holds the Android-escaped source,
so every import brings it back. Add .claude/hooks/compose_escaping_check.py, a
sub-second scan of the Compose catalog for \' \" \? \@ and tools: attributes,
mirroring orphan_strings_check.py -- same shape of bug, same fix. \n, \t, \uXXXX
and \\ are left alone because Compose resolves those itself, and Android res trees
are not scanned because there the escaping is correct.
Verified in both directions: clean tree exits 0; restoring the regressed
values-fr-rFR makes it exit 1 and name the file with a per-escape breakdown.
Not yet wired into CI or the pre-push hook -- that is a maintainer call about
where the gate lives. As amethyst/src/main/res/CLAUDE.md notes for the identical
orphan-strings desync, the CI lint job is the layer that matters: these arrive
through bot-authored PRs with no local session in the path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1CzYQvWyHfipSW7x3j4Yo
@@ -47,7 +47,7 @@ Walk the imports. The usual offenders:
| `android.util.Log` | Replace with `quartz``PlatformLog` (already multiplatform). |
| `android.graphics.Bitmap` | Almost never needed by Amy. Keep in Android and split the function. |
| `android.net.Uri` | Replace with `kotlinx.io` path types or a plain `String`. |
| `androidx.compose.*` | Must stay out of `commons/commonMain` unless you're in a Compose-Multiplatform module. Amy doesn't depend on Compose. |
| `androidx.compose.*` | Compose UI (`ui`/`foundation`/`material3`), Coil and `Res` must stay out of `commons` entirely — they belong in `:commonsUI`, which Amy never depends on. Only the Compose *runtime* (`@Stable`, snapshot state) is allowed in `commons`. |
### Step 3 — Pick a migration strategy per dependency
@@ -66,7 +66,7 @@ Walk the imports. The usual offenders:
# Target location depends on what it is:
# - Protocol → quartz/src/commonMain/kotlin/…
# - Business logic → commons/src/commonMain/kotlin/…
@@ -24,7 +24,7 @@ Visual UI patterns for sharing composables across Android and Desktop.
## Philosophy: Share by Default
**Default to `commons/commonMain`** unless platform experts indicate otherwise.
**Default to `commonsUI/commonMain`** (shared composables live in `:commonsUI`, the Compose half of the shared layer; headless state/ViewModels stay in `:commons`) unless platform experts indicate otherwise.
### Always Share
@@ -416,7 +416,7 @@ fun DataScreen(uiState: UiState) {
This catalog documents shared UI components in `commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/ui/`.
This catalog documents shared UI components in `commonsUI/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/ui/` (the Compose half of the shared layer; headless state stays in `commons`).
## Directory Structure
```
commons/src/commonMain/kotlin/.../commons/ui/
commonsUI/src/commonMain/kotlin/.../commons/ui/
├── components/ # Reusable UI components
├── screens/ # Screen-level composables
├── theme/ # Theming and styling
@@ -232,7 +232,7 @@ private val pathData1 = PathData {
description:Feed composition and data-access layer patterns in Amethyst. Use when adding or modifying a feed (home, profile, hashtag, bookmarks, notifications, DMs, communities), working with the shared `FeedFilter` / `AdditiveFeedFilter` / `ChangesFlowFilter` / `FeedContentState` in `commons/.../ui/feeds/`, the Android-only `AdditiveComplexFeedFilter` / `FilterByListParams` in `amethyst/.../ui/dal/`, or extending the `FeedViewModel` family in `commons/.../viewmodels/`. Covers how feeds scan `LocalCache`, react to changes, apply ordering, and render through Compose.
description:Feed composition and data-access layer patterns in Amethyst. Use when adding or modifying a feed (home, profile, hashtag, bookmarks, notifications, DMs, communities), working with the shared `FeedFilter` / `AdditiveFeedFilter` / `ChangesFlowFilter` / `FeedContentState` in `commons/.../feeds/`, the Android-only `AdditiveComplexFeedFilter` / `FilterByListParams` in `amethyst/.../ui/dal/`, or extending the `FeedViewModel` family in `commons/.../viewmodels/`. Covers how feeds scan `LocalCache`, react to changes, apply ordering, and render through Compose.
---
# Feed Patterns
@@ -25,7 +25,7 @@ Amethyst's "feed" abstraction is: a `FeedFilter` that decides which notes belong
│ ◄── MarmotGroupFeedViewModel │
│ │
│ │
│ commons/.../ui/feeds/ (shared, KMP) │
│ commons/.../feeds/ (shared, KMP) │
│ IFeedFilter / FeedFilter<T> (abstract base) │
│ IAdditiveFeedFilter / AdditiveFeedFilter<T> │
│ ChangesFlowFilter │
@@ -68,7 +68,7 @@ Amethyst's "feed" abstraction is: a `FeedFilter` that decides which notes belong
- **`FeedFilter.kt`** — `abstract class FeedFilter<T> : IFeedFilter<T>`. Has `feed(): List<T>` (the sync query against the cache), `feedKey(): String` (identity used to cache), `limit()`, and `loadTop()`.
- **`AdditiveFeedFilter.kt`** — `abstract class AdditiveFeedFilter<T> : FeedFilter<T>(), IAdditiveFeedFilter<T>`. Adds incremental updates (the "additive" part): `updateListWith(oldList, newItems)` runs `applyFilter(newItems)` and grafts accepted items onto the existing list (re-`sort` + `take(limit())`) without recomputing everything.
- The filter **base classes** (`FeedFilter`, `AdditiveFeedFilter`, `ChangesFlowFilter`) and feed state (`FeedContentState`) are in `commons/.../ui/feeds/` — **shared**. ViewModels are in `commons/.../viewmodels/` — **shared**.
- The filter **base classes** (`FeedFilter`, `AdditiveFeedFilter`, `ChangesFlowFilter`) and feed state (`FeedContentState`) are in `commons/.../feeds/` — **shared**. ViewModels are in `commons/.../viewmodels/` — **shared**.
- The **concrete** filters are platform-local: Android's in `amethyst/.../ui/screen/loggedIn/*/dal/`, Desktop's in `desktopApp/.../feeds/`. `amethyst/.../ui/dal/` keeps Android-only helpers (`AdditiveComplexFeedFilter`, `FilterByListParams`, `DefaultFeedOrder`) plus back-compat typealiases.
- When porting a feed, share the concrete filter only if both platforms need identical inclusion rules.
@@ -7,7 +7,7 @@ description: Use when comparing Android strings.xml locale files to find untrans
## Overview
Extract string resource keys from a default `values/strings.xml` that are absent in a target locale's `strings.xml`, excluding non-translatable entries. Outputs missing keys and offers to translate them.
Extract string resource keys from a default `values/strings.xml` that are absent in a target locale's `strings.xml`, excluding non-translatable entries. Outputs the missing keys, then offers the two things that close them: **translate** the ones needing translation, and **copy the English value verbatim** for the ones a locale deliberately keeps in English (since 2026-09-12 that copy is what seeds Crowdin — see Background).
The repo now has **two independent Crowdin-managed resource trees** — you must scan **both** (see "Resource trees" below).
@@ -24,9 +24,9 @@ There are two separate `strings.xml` trees, each with its own default `values/`
The `commons` tree appeared when shared event-renderer composables were extracted out of `amethyst/` into `commons/` (Compose Multiplatform `stringResource`). It is **not** a copy of the amethyst tree — the vast majority of its keys are commons-only; only a small handful overlap. Every diff/count/translate command below works on either tree by swapping the base path — **run the whole technique once per tree** and report them separately (each maps to its own Crowdin file, so the counts should reconcile against two different Crowdin UI numbers).
The `commonsUI` tree appeared when shared event-renderer composables were extracted out of `amethyst/` into `commons/` — now `commonsUI/` since the UI split (Compose Multiplatform `stringResource`). It is **not** a copy of the amethyst tree — the vast majority of its keys are commons-only; only a small handful overlap. Every diff/count/translate command below works on either tree by swapping the base path — **run the whole technique once per tree** and report them separately (each maps to its own Crowdin file, so the counts should reconcile against two different Crowdin UI numbers).
**Locale-qualifier caveat:** `commons` uses the same region-qualified locale dirs as amethyst for our four targets (`values-cs`, `values-de-rDE`, `values-sv-rSE`, `values-pt-rBR`), but the *full* set of locale dirs differs between trees. Enumerate `values-*` under each tree's own base rather than assuming they match.
@@ -37,7 +37,7 @@ The `commons` tree appeared when shared event-renderer composables were extracte
Detect name-overlap **and flag value mismatches** in one pass:
@@ -54,7 +54,7 @@ Only `SAFE-COPY` keys may be copied verbatim. For `VALUE-DIFFERS`, translate the
**Whitespace-quote convention differs between trees.** Android string resources use surrounding double-quotes to preserve leading/trailing whitespace (`"replying to "`). The **commons Compose-resources tree does NOT use this convention** — it authors trailing/leading spaces raw and unquoted (`replying to `). So when copying/translating a commons string with edge whitespace, **match the commons source: raw spaces, no wrapping quotes.** (Mistake we made: we copied amethyst's quoted `"replying to "` into commons, where the quotes would render literally.) A quick check for stray quote-wrapping you introduced:
**Do not** treat the value-overlap as something to deduplicate during a translation pass. Migrating amethyst's own screens onto the shared `action_*` strings is a *separate, optional* refactor and a maintainer call — out of scope for this skill. Just translate each tree correctly and independently.
## Background: Crowdin strip-identical behavior
## Background: source-identical translations and the `import_eq_suggestions` flag
This repo syncs translations via Crowdin (branch `l10n_crowdin_translations`). Crowdin's default export behavior **omits any translation that exactly equals the source**, so a key that the translator deliberately kept as English (common for brand terms like `"Nowhere Drop"`, single-word loanwords like `"Apps"` / `"Feed"` / `"Issues"`, or version prefixes like `"v%1$s"`) will not appear in the locale's `strings.xml` even though the Crowdin UI shows it as 100% translated.
This repo syncs translations via Crowdin (branch `l10n_crowdin_translations`). Crowdin does not *store* a translation that exactly equals the source unless it is told to, so historically a key a translator deliberately kept as English (brand terms like `"Nowhere Drop"`, single-word loanwords like `"Apps"` / `"Feed"` / `"Issues"`, version prefixes like `"v%1$s"`) never appeared in the locale's `strings.xml`, even though the Crowdin UI showed it as 100% translated.
**That changed on 2026-09-12.** `.github/workflows/crowdin.yml` now passes `import_eq_suggestions: true` to `crowdin/github-action`, so `upload_translations` no longer skips values equal to the source — whatever sits in the repo's locale files is seeded into Crowdin's database, identical values included. `auto_approve_imported` stays at its default `false`, so they arrive as **pending** translations for a translator to approve.
Confirmed end-to-end the same day: the first sync after the flag landed (workflow run `34706537802` → PR #4107) rewrote all five touched locale files in Crowdin's own key order with **zero net key changes** — 323 additions and 323 removals that pair up exactly. All 330 identical values pushed that morning came back down intact, unapproved included. Since Crowdin's download *replaces* file content with its export, a value it did not hold would have vanished; none did.
**Reading such a sync diff: compare key *sets* per file, never `-`/`+` lines separately.** A reorder looks identical to a mass strip under `grep '^-'`, and it will convince you the mechanism failed when nothing changed at all.
What this means for this skill:
1. **The raw on-disk diff is the candidate set.** A key missing from a locale file is either genuinely untranslated *or* a source-identical entry Crowdin stripped. Both are reported; the human decides which to skip. The Crowdin web UI ("N untranslated") is the ground truth for what genuinely needs work.
2. **Source-identical entries are a small, recognizable minority.** Brand terms (`Nowhere X`), single-word loanwords (`Apps` / `Feed` / `Issues`), and bare version/format strings (`v%1$s`) are the usual cases. Skip these by inspection rather than translating them to something identical.
3. **Don't add source-identical fallbacks.** Android falls back to `values/strings.xml` at runtime, so a key intentionally kept as English already renders correctly, and Crowdin's next sync would strip a local duplicate anyway.
1. **The raw on-disk diff is the candidate set.** A key missing from a locale file is genuinely untranslated,*or* a source-identical entry stripped before 2026-09-12 that no sync has re-seeded yet. Both are reported, and both are now actionable in the repo — translate the first, copy English into the second. The Crowdin web UI ("N untranslated") remains the ground truth for what needs human work.
2. **Source-identical entries are still recognizable, but no longer skipped.** Brand terms (`Nowhere X`), loanwords (`Apps` / `Feed` / `Issues`), symbol- or format-only values (`v%1$s`, `+%1$d`, `%1$d/%2$d`, `∞`, 👀) and example placeholders (`iPhone 13`, `https://example.com`) are the usual cases. Copy the English value into the locale file verbatim so the upload can seed it.
3. **DO add source-identical values — that is now the mechanism, not churn.** A key absent from a locale file is invisible to `upload_translations`; writing the English value in is what gets it into Crowdin, so a translator approves it once in bulk instead of typing it into the UI ~70 times per locale. (Runtime behaviour is unchanged either way: Android still falls back to `values/strings.xml`.) Two exclusions:
- **Never for `<plurals>`.** Copying English `one`/`other` into cs/pl trips `MissingQuantity`, which is a CI error (cs needs `one`/`few`/`many`/`other`). Plurals stay a Crowdin-UI job.
- **Not for words a locale would genuinely translate.** German `buzz_dm_workspace` ("Arbeitsbereich"), `workout` ("Training"), `relay_group_threads_title` ("Themen"), `calendar_rsvp_section` ("Zusagen") are *gaps*, not deliberate English keeps. Copying English there seeds a wrong pending suggestion — list those for the human to translate rather than approve.
4. **A repo-side edit to a translated value only sticks where Crowdin's database
doesn't contradict it.** Download replaces file content with Crowdin's current
@@ -96,6 +104,13 @@ What this means for this skill:
from `values/strings.xml` removes it project-wide, and attributes declared
there propagate into every export.
**This does not contradict item 3 — the two cases differ.** Seeding a key
Crowdin holds *nothing* for (the identical-value copy) sticks, because there is
no stored value to contradict it; that is exactly why the copy pass works.
*Overwriting* a value Crowdin already holds differently — including an empty
one — still loses on the next sync. Add missing entries in the repo; change
existing translations in the UI.
> **Historical note:** an earlier version of this skill tried to auto-filter the
> candidate list with a git "sync-timestamp" heuristic (skip any key added before
> the last `New Crowdin translations` commit). It was **dropped** because it
A convenient way to run the whole technique twice is to loop over the two base dirs:
```bash
for base in amethyst/src/main/res commons/src/commonMain/composeResources; do
for base in amethyst/src/main/res commonsUI/src/commonMain/composeResources; do
echo "########## tree: $base ##########"
# ... run the diff/count/value-extraction commands with $base/values[...] ...
done
@@ -172,7 +187,7 @@ comm -23 \
This gives two lists of missing key names — keep them separate; `<plurals>` translations need the per-locale CLDR category set (see Step 5 → "Plurals: handle with care").
Crowdin can asymmetrically strip keys across locales (each translator independently chose source-identical for different keys), so **cs is not a reliable upper bound**. Diff **every** target locale and union the results — don't assume the cs set covers the others. A quick per-locale count is a useful sanity check against the Crowdin UI's "N untranslated":
Locale files are asymmetric — legacy pre-2026-09-12 strips and uneven translator progress both leave different keys missing in different locales — so **cs is not a reliable upper bound**. Diff **every** target locale and union the results — don't assume the cs set covers the others. A quick per-locale count is a useful sanity check against the Crowdin UI's "N untranslated":
```bash
for locale in cs de-rDE sv-rSE pt-rBR; do
@@ -190,7 +205,7 @@ for locale in cs de-rDE sv-rSE pt-rBR; do
done
```
The combined `strings + plurals` total should line up with the Crowdin web UI's untranslated count for that locale. If it does, the raw diff is your actionable set (minus any source-identical entries you skip by inspection — see Background).
The combined `strings + plurals` total should line up with the Crowdin web UI's untranslated count for that locale. If it does, the raw diff is your actionable set: translate what needs translating, and copy the English value verbatim for the entries a locale keeps in English (see Background).
### 3. Get English values for missing keys
@@ -258,8 +273,8 @@ Flag and offer to fix:
# hardcode "1" (or other literal digits) instead of using a placeholder.
# Looks at default + all values-* locales, in BOTH resource trees.
for f in amethyst/src/main/res/values/strings.xml amethyst/src/main/res/values-*/strings.xml \
Three things this scan taught us, all of which it now encodes:
@@ -458,7 +473,7 @@ When adding translated strings to locale files:
- **Append new strings at the bottom** of the file, just before the closing `</resources>` tag.
- Do NOT try to insert them in alphabetical or matching order — a separate process handles ordering.
- **Insert into each locale ONLY the keys missing from *that* locale — never a shared "union" block.** Because Crowdin strips keys asymmetrically (Step 2), a key you translate may already exist in some target locales. If you compute one union set of missing keys, translate it, and paste the *same* block into every locale, you will create **duplicate keys** in whichever locales already had them. Drive the insertion off the **per-locale** diff, not the union:
- **Insert into each locale ONLY the keys missing from *that* locale — never a shared "union" block.** Because locale files are asymmetric (Step 2), a key you translate may already exist in some target locales. If you compute one union set of missing keys, translate it, and paste the *same* block into every locale, you will create **duplicate keys** in whichever locales already had them. Drive the insertion off the **per-locale** diff, not the union:
```bash
# For each locale, insert only the keys comm -23 reports missing FOR THAT LOCALE.
@@ -528,15 +543,15 @@ When adding translated strings to locale files:
## Common Mistakes
- **Scanning only the amethyst tree** — there are now **two** Crowdin-managed `strings.xml` trees (`amethyst/src/main/res` and `commons/src/commonMain/composeResources`). A key extracted into `commons/` will never show up in the amethyst diff. Run the whole technique once per tree (see "Resource trees") and report each separately.
- **Scanning only the amethyst tree** — there are now **two** Crowdin-managed `strings.xml` trees (`amethyst/src/main/res` and `commonsUI/src/commonMain/composeResources`). A key extracted into `commonsUI/` will never show up in the amethyst diff. Run the whole technique once per tree (see "Resource trees") and report each separately.
- **Copying an overlapping `commons` translation by key name alone** — a shared key name does NOT mean shared English. `napplet_card_permissions` is "What it can access" in commons but "Permissions:" in amethyst; copying by name produced the wrong string. Diff the English *values* first; copy verbatim only when they're byte-identical, else translate fresh (see "Overlap" in Resource trees).
- **Applying amethyst's `"…"` whitespace-quote convention to a commons string** — the commons Compose-resources tree authors edge whitespace raw and unquoted; wrapping quotes copied from amethyst render literally there. Match the commons source format.
- **Trying to "dedupe" the amethyst↔commons value-overlap** — it's required architecture (commons can't depend on amethyst, so shared composables need their own `Res.string` catalog), not an error. Don't fold consolidation into a translation pass.
- **Forgetting `translatable="false"`** — these should never appear in locale files
- **Diffing only `<string name=`** — `<plurals>` is a separate resource type; a source `<plurals>` missing from a locale will never show up in a `<string>` diff. Always run the diff twice (once per resource type) as shown in Step 2. The same goes for `<string-array>` if the project uses it.
- **Trusting a git "sync-timestamp" heuristic to pre-filter the list** — this skill used to skip keys added before the last `New Crowdin translations` commit, on the theory that Crowdin had already "decided" them. It was dropped: a key added shortly before an export that translators hadn't reached yet is genuinely missing, so the heuristic silently dropped real work. Use the raw on-disk diff and reconcile against the Crowdin web UI's untranslated count instead.
- **Adding source-identical fallbacks locally** — they get overwritten on the next Crowdin sync. Android falls back to `values/strings.xml` at runtime anyway, so a key intentionally kept as English already renders correctly. Skip these by inspection (brand terms, loanwords, `v%1$s`-style strings); don't translate them to an identical value.
- **Skipping per-locale diffs when only diffing cs** — Crowdin can strip different keys in different locales (each translator's choice), so cs is not a reliable upper bound. Diff each target locale and union the results.
- **Skipping source-identical entries instead of copying them in** — correct before 2026-09-12, wrong now. With `import_eq_suggestions: true` the repo file is the *seed* for Crowdin's database, so a key you leave out stays untranslated in the UI forever and reappears in every future scan. Copy the English value verbatim, except for `<plurals>` (trips `MissingQuantity`) and words the locale would really translate. (Confirmed by PR #4107: 330 identical values survived the next sync with zero net changes.)
- **Skipping per-locale diffs when only diffing cs** — different keys are missing in different locales (legacy strips plus uneven translator progress), so cs is not a reliable upper bound. Diff each target locale and union the results.
- **Pasting the union set of missing keys into every locale → duplicate keys** — the union is the right set to *translate*, but the wrong set to *insert*. A key missing in only some locales, inserted into all of them, duplicates in the ones that already had it. Drive each file's insertion off its own per-locale diff (see Step 6). In `commons`, a duplicate key is build-breaking: `convertXmlValueResourcesForCommonMain` fails with `Duplicated key '…'`. **Always run the post-insertion duplicate + XML-wellformedness gate in Step 6 before declaring done.** (Happened 2026-07-21 with `ps1_save_block` / `podcast_value_for_value` / `chats_history_relays`.)
- **Declaring the pass done without running `:amethyst:lintPlayBenchmark`** — the duplicate-key + XML + `convertXmlValueResourcesForCommonMain` gate is necessary but nowhere near sufficient. `MissingQuantity` and `ImpliedQuantity` are errors, there is no lint baseline, and `abortOnError` is on, so a change that compiles and passes every check in Step 6's first half can still take CI red. Compiling is not evidence. (Happened 2026-08-13: 3 lint errors after a clean duplicate/XML gate and a green `compileFdroidDebugKotlin`.)
- **Converting a `<string>` to `<plurals>` with `other` only** — "Crowdin fills the rest" is false; `MissingQuantity` errors immediately and CI fails before any sync. Supply every category the locale uses at conversion time, and re-check the declension rather than reusing the old text for `one`.
@@ -17,7 +17,7 @@ The layer between `LocalCache`/`Account` and the raw relay connection. Ensures c
## Layout
All under `commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/`:
All under `commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/` (the `@Composable` entry points — `observeUser*`, `*FilterAssemblerSubscription`, `KeyDataSourceSubscription` — sit in the same package but in `commonsUI/src/commonMain/…`, the Compose half of the shared layer):
| Artifact | Committed at | Regenerate when | Guide |
|---|---|---|---|
| **Material Symbols subset font** | `commons/src/commonMain/composeResources/font/material_symbols_outlined.ttf` | You add/remove a `MaterialSymbol("\uXXXX")` codepoint in `MaterialSymbols.kt`, or bump the upstream font | [`tools/material-symbols-subset/README.md`](tools/material-symbols-subset/README.md) — run `./tools/material-symbols-subset/subset.sh` |
| **Material Symbols subset font** | `commonsUI/src/commonMain/composeResources/font/material_symbols_outlined.ttf` | You add/remove a `MaterialSymbol("\uXXXX")` codepoint in `MaterialSymbols.kt`, or bump the upstream font | [`tools/material-symbols-subset/README.md`](tools/material-symbols-subset/README.md) — run `./tools/material-symbols-subset/subset.sh` |
| **Arti (Tor) native libs** | `amethyst/src/main/jniLibs/*.so` | You update the pinned Arti version, change the JNI wrapper, or want to reproduce the binaries | [`tools/arti-build/README.md`](tools/arti-build/README.md) |
> **Material Symbols is mandatory after icon changes.** The bundled font is a
@@ -105,8 +105,9 @@ and each has its own guide:
> change. Reusing an existing codepoint needs no regeneration.
Both tools have their own prerequisites (`fonttools`/`brotli` for the font; a
Rust toolchain + Android NDK 25+ for Arti) documented in their READMEs — they
are **not** required to build Amethyst from the committed sources.
Rust toolchain + the exact Android NDK revision pinned in
`tools/arti-build/ANDROID_NDK_VERSION` for Arti) documented in their READMEs —
they are **not** required to build Amethyst from the committed sources.
---
@@ -466,8 +467,8 @@ Homebrew removes the quarantine attribute on its own downloads.
> with `dry_run=true` — the sign+notarize step runs regardless of `dry_run` and
> now prints the per-file notary log on a non-`Accepted` verdict. If it comes
> back `Invalid`, the fix is to codesign the dylibs *inside* those jars before
> zipping (and/or strip the unused `skiko`/Compose jars from the CLI image — the
> `:commons`core/ui split the size budget already flags). The **desktop** app
> zipping (the unused `skiko`/Compose jars left the CLI image with the
> `:commons`/ `:commonsUI` split). The **desktop** app
> bundles the same jars through Compose/jpackage notarization, so run a desktop
> dry-run too; its in-jar handling differs and is likewise unverified.
@@ -533,14 +534,17 @@ reads an optional per-release changelog from
## Bootstrap runbook (one-time)
> **Status as of v1.14.0:**Winget has been submitted — [microsoft/winget-pkgs#422752](https://github.com/microsoft/winget-pkgs/pull/422752), pending CLA + review. Neither Homebrew package (`amethyst-nostr` cask, `amy` formula) has been submitted yet.
> `https://formulae.brew.sh/api/cask/amethyst-nostr.json` and
> `microsoft/winget-pkgs/manifests/v/VitorPamplona/Amethyst` both 404, so
> **Amethyst does not currently ship through either channel.** The bump
> workflows detect this and skip with a `::warning::` instead of failing, so a
> green release run does *not* mean Homebrew/Winget shipped. The two subsections
> below are the work that activates them; until then treat the desktop app as
> GitHub-Releases-only on macOS and Windows.
> **Status as of v1.15.2:**both Homebrew packages are now live upstream — the
> `amethyst-nostr` cask (`Homebrew/homebrew-cask`, at 1.14.0) and the `amy`
> formula (`Homebrew/homebrew-core`) both answer 200 on `formulae.brew.sh`, so
> `bump-homebrew.yml` finally has something to bump. **Winget is still not
Amethyst is free, open-source software (MIT License — see `LICENSE`). It is not a service. There is no Amethyst server, no Amethyst account, and the developer has no access to data stored on your device.
@@ -20,6 +20,7 @@ Using the app causes the following data to leave your phone:
- **Nostr events** you publish, sent to the relays you have configured.
- **Subscriptions** (filters describing what you want to read), sent to those relays.
- **Media uploads** (images, audio, video), sent to the media server you select.
- **Workout summaries**, when you choose to publish one — see [Health and fitness data](#health-and-fitness-data-health-connect) below.
- *(Google Play build, push notifications enabled)* a per-device push token, your public key, and a preferred relay, registered with Google Firebase Cloud Messaging so a notification proxy can wake the app.
- *(F-Droid build, push notifications enabled)* a per-device token registered with whichever UnifiedPush distributor you install (e.g. ntfy).
@@ -29,6 +30,36 @@ The developer does not run any server that aggregates or stores this data.
Configuration, cached events, keys, drafts, and other operational data live in the app's local storage. Other apps cannot read it on a standard, non-rooted Android device. You can wipe it by clearing the app's storage or uninstalling.
### Health and fitness data (Health Connect)
Amethyst's **Workouts** section lets you publish a summary of a finished workout to the Nostr relays you choose (a NIP-101e kind 1301 event), so the people who follow you can see it. To save you typing the numbers in by hand, Amethyst can read the workout your watch or fitness app already saved to **Android Health Connect** and pre-fill the post.
The feature is optional and off until you grant the permissions. Amethyst asks for them only when you open the New Workout composer — never on first launch.
**What Amethyst reads, and what each type is for:**
| Health Connect data type | Permission | What it is used for |
| --- | --- | --- |
| ExerciseSession | `READ_EXERCISE` | The workout itself: activity type, start time and duration — the title, date and duration of the post. |
| Distance | `READ_DISTANCE` | The distance of the run, ride, walk or swim. |
| ActiveCaloriesBurned | `READ_ACTIVE_CALORIES_BURNED` | The energy the workout burned. |
| TotalCaloriesBurned | `READ_TOTAL_CALORIES_BURNED` | Fallback energy figure for sources that only record total energy. |
| HeartRate | `READ_HEART_RATE` | Average and maximum heart rate over the workout — how hard the effort was. |
| Steps | `READ_STEPS` | The step count of a run, walk or hike. |
| ElevationGained | `READ_ELEVATION_GAINED` | How much you climbed. |
Health Connect groups a few data types under one permission: `READ_EXERCISE` also covers CyclingPedalingCadence and `READ_STEPS` also covers StepsCadence. Amethyst does not read, store, or publish cadence — those types come attached to the permissions above and are never requested separately.
**Limits on this access:**
- **Read-only.** Amethyst never writes to Health Connect.
- **Foreground only.** Reads happen only while the New Workout composer is on screen. Amethyst does not request `READ_HEALTH_DATA_IN_BACKGROUND` and has no background health worker.
- **Last 7 days only.** Only sessions that finished in the previous 7 days are offered. Amethyst does not request `READ_HEALTH_DATA_HISTORY`.
- **No location.** Amethyst does not request `READ_EXERCISE_ROUTE`, so it never receives the GPS track of a workout.
- **Nothing is uploaded automatically.** Health data stays on your device until you pick a suggestion, review the pre-filled post, and publish it yourself. The developer runs no server; a published post goes to the Nostr relays you configured, and those numbers then become public like any other post you make.
- **No other use.** Health data is never used for advertising, analytics, profiling, or sale, and is never shared with third parties. It is not used to determine your eligibility for insurance, credit, or employment, and is not transferred to any such party.
- **Revocable.** Turn the feature off under Settings → Compose → "Suggest workouts to share", or revoke the permissions in Health Connect at any time. Amethyst keeps the workout suggestions it has already shown only in memory; revoking access stops all reads immediately.
| **Homebrew formula** | `Homebrew/homebrew-core` → `geode-relay` | Not submitted — renamed from `geode`, which is permanently reserved for Apache Geode |
| **Winget** | `microsoft/winget-pkgs` → `VitorPamplona.Amethyst` | **Submitted at v1.14.0** — [PR #422752](https://github.com/microsoft/winget-pkgs/pull/422752), still open pending CLA + review |
Until each lands, that channel delivers nothing and macOS/Windows users get the
desktop app from GitHub Releases only. Re-check before assuming — the state
above is a snapshot, and `gh api repos/microsoft/winget-pkgs/contents/manifests/v/VitorPamplona`
(404 = still absent) answers it in one call.
Until each lands, that channel delivers nothing and its users get the desktop
app or CLI from GitHub Releases only. Re-check before assuming — the state above
is a snapshot, and two calls answer it:
`gh api repos/microsoft/winget-pkgs/contents/manifests/v/VitorPamplona` and
│ This is my very favorite book. The fast- │ content, TranslatableRichTextViewer
│ paced and mysterious plot … │
└──────────────────────────────────────────────┘
```
- Target resolution: `LoadAddressableNote(targetAddress(), …)` — same helper
`RenderPostApproval.kt:61` and `Attestation.kt` use.
- **Title fallback while (or if) the target never loads:** derive a label from the
coordinate's `d` (`wuthering-heights` → "Wuthering Heights"), as jumble's
`publicationTitleHintFromRatingEvent` does. Without this the card is a row of
stars attached to nothing — see §7.
- Unknown/absent `m`: render stars + comment + a generic `nostr:` link to the
target. The kind is generic; the card must degrade, not blank.
- Dispatch: add `is EntityRatingEvent -> RenderEntityRating(...)` to
`NoteCompose.kt`'s render `when`, **before** the `else` at :1658 (which
currently routes unknown kinds to `RenderTextEvent`).
### The star icons need a font change
`MaterialSymbols.kt:233-234` defines **both**`Star` and `StarBorder` as
`\uF09A` — the same codepoint. Filled and outline stars are currently
indistinguishable (which also means `FavoriteAlgoFeedToggle.kt:90/97` and
`RelayGroupDiscoveryScreen.kt:400` are drawing the same glyph for on and off
today).
A 3.5-of-5 star row needs three distinct glyphs. So:
1. Give `StarBorder` its real outline codepoint and add `StarHalf`.
2. Regenerate the subset — **mandatory**, per `.claude/CLAUDE.md`:
`./tools/material-symbols-subset/subset.sh`
3. Commit the regenerated `material_symbols_outlined.ttf` with the
`MaterialSymbols.kt` change, or the new glyphs render as tofu.
Fixing the duplicate is a small pre-existing-bug fix that rides along; call it
out in the PR body since it changes two unrelated toggles' appearance.
---
## 7. The kind-30040 dependency
The rated object here is a **kind-30040 NKBIP-01 publication index**. Amethyst
has no class for 30040 either, so `LocalCache` drops it by the same `else` at
:3886. `LoadAddressableNote` will therefore resolve to a permanently empty
`Note`, and the target card in §6 will never show a real title.
Two ways to close this, and they are separable:
**(a) v1 — slug fallback only.** Ship §6's derived-title fallback and no 30040
class. The card reads "Wuthering Heights" from the coordinate. Cheap, honest,
and correct for the common case where the `d` is a slug. Fails softly (shows the
raw identifier) when the `d` is a hash or an opaque id.
**(b) Phase 2 — minimal `PublicationIndexEvent` (30040).** A `title`/`author`/`d`
parser plus `consumeBaseReplaceable` wiring, so the target card shows the real
title and links somewhere. **Explicitly not** the publication *reader* — 30040 is
an index over kind-30041 sections, and rendering a book is a separate feature an
order of magnitude larger than this one.
Recommend shipping (a) in v1 and (b) as an immediate follow-up. Do **not** let
(b)'s scope pull the reader in.
---
## 8. Out of scope for v1, and what each would cost
| Deferred | Why | Rough shape |
| --- | --- | --- |
| **Publishing a rating** | Needs a compose surface (star picker + comment) and a "what am I rating" entry point, which does not exist until 30040 objects are browsable. The quartz builder DSL lands in v1 anyway, so this is UI-only later. | `NewPostScreen` variant + a rate action on the target card |
| **Aggregate rollups** | Needs a per-target index in `LocalCache` (like the zap/reaction indices) plus a REQ by `#a`/`#d`. Meaningful only once there is a target screen to put the average on. | New index + `FeedMetadataCoordinator` assembler |
| **Rating relays / profiles / hashtags** | The kind is generic and these marks are in the spec, but each needs its own target card and entry point. The event class supports them from day one. | Per-mark `RenderEntityRating` branch |
| **Relay reviews (kind 31987)** | Same `rating` tag convention, also unsupported in Amethyst today. Sharing `stars()` between the two is the natural next step. | Reuse `tags/RatingTag.kt` |
---
## 9. Order of work
1. **quartz** — `EntityRatingEvent` + tags + builder + `RatingMark`, with unit
tests covering all four `stars()` branches, the `d`-prefix strip, `a`-over-`d`
preference, and a round-trip of the real imwald event above as a fixture.
# DVM heartbeat liveness — only show DVMs with a fresh kind-11998 heartbeat
_Status: **implemented** (Android; desktop wiring deliberately out of scope — §8)._
## 0. The shape of the thing
Amethyst shows Data Vending Machines (DVMs) in three places: the Discover "Content" tab
(kind 31990 NIP-89 announcements advertising kind 5300), DVM feeds pinned to the top-nav
(`FavoriteAlgoFeedsOrchestrator`), and the per-DVM content-discovery screen. Today all of
these treat every announced DVM as alive, forever — a DVM that went down months ago still
renders as a usable feed.
DVM operators are now sending a **heartbeat event (kind 11998) every 300 seconds**. The
event is plain-text (`content = "Alive and kicking"`) with three tags:
- `status` — free-text status line (e.g. "My heart keeps beating like a hammer")
- `d` — the DVM's **NIP-89 DTAG**, tying the heartbeat to the announcement's address
- `expiration` — `createdAt + 300` (NIP-40), so relays drop the beat once the next one lands
Kind 11998 sits in the replaceable range (10000–19999), so relays keep only the latest beat
per author. There is no NIP for this yet — the shape above comes from the operator-side
builder and is treated as the wire contract.
The feature: **a DVM counts as alive only if its latest heartbeat is at most 900 seconds
old** (one missed 300s beat plus slack). Dead DVMs disappear from the Discover list; pinned
feeds and the detail surface show an offline state instead.
## 1. Decisions taken
1. **Approach: cache-backed heartbeats.** The heartbeat is a real event class stored through
the standard replaceable path in `LocalCache` (newest per address, standard invalidation).
Rejected alternatives: a side-state registry (duplicates invalidation plumbing) and
regular-note storage (no address matching, pollutes the notes index).
2. **Scope: all three surfaces** — Discover list (hide), pinned feeds (offline state, chip
stays), DVM detail screen (offline banner, requesting still allowed). The manage screen
(`FavoriteAlgoFeedsListScreen`) also gets the badge.
3. **Pinned chips stay when offline** — the user pinned them deliberately; they gray out
with an offline badge rather than vanishing, and tapping still opens the feed.
4. **Threshold: 900 seconds** (raised from the original 420 after field testing: beats arrive every 300s, and a 420s window tolerated barely one delivery hiccup, dropping live DVMs in oscillations). Exactly 900s old counts as fresh.
5. **Strict from cold start.** No grace period: the Discover list starts empty and fills
within ~1–2s as heartbeat REQs return (same behavior as the existing 31990 load).
## 2. Event model (quartz)
New `quartz/.../nip90Dvms/dvmHeartbeat/DvmHeartbeatEvent.kt`:
| [2026-09-07-generic-local-filter.md](2026-09-07-generic-local-filter.md) | Local note search moved onto `LocalCache.filter(Filter)` so the search box's tokens narrow local and relay results alike; steps 1–6 shipped, `FilterMatcher` left untouched via a predicate parameter. |
| [2026-05-24-ios-support.md](2026-05-24-ios-support.md) | Incremental KMP-to-iOS port; quartz/commons iOS targets are configured (Phase 1) but no `iosApp` module exists yet. |
Log.i("AppModules"){"Image cache was over budget: wiped ${result.reclaimedFiles} files (${result.bytesOnDisk} bytes on disk, ${result.budgetBytes} budget)"}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Vitor PamplonaandGitHub
Claude
David KasparandGitHub
vitorpamplonaandgithub-actions[bot]
davotoulaandgithub-actions[bot]
mstrofnoneandVitor Pamplona
Dr. Tobias Baur
