laantungir/client
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
client/plans/feed-outbox-assessment.md

48 KiB
Raw Permalink Blame History

Feed Outbox Model — Assessment & Plan

Your suspicion is correct, and NDK already has the machinery built in.

You are missing posts from some follows because the feed only queries the relays you subscribe to (your read/both relays from your kind 10002 list). If a follow posts exclusively to relays you are not connected to, those posts never arrive. This is exactly the problem the Nostr outbox model (NIP-65 + NIP-17 relay lists) solves: instead of only listening on your relays, you look up each follow's kind 10002 relay list and fetch their posts from their write relays.

The good news: NDK's outbox model is already enabled in your bundle and is already wired into the subscription path. The bad news: it is being defeated by a race condition in how www/feed.html bootstraps the feed.

How NDK's outbox model works (already in your bundle)

Relevant code in www/ndk-core.bundle.js:

  1. Outbox is on by default — NDK constructor at line 43295: if (!(opts.enableOutboxModel === false)) creates an outboxPool (default relays wss://purplepag.es/, wss://nos.lol/) and an outboxTracker. Your worker never sets enableOutboxModel: false, so it is active. Confirmed: www/ndk-worker.js ndkOptions does not disable it.

  2. Author-based subscriptions trigger outbox tracking — NDK.subscribe: when a filter has authors, it calls this.outboxTracker?.trackUsers(authors). This fetches each author's kind 10002 (and kind 3 fallback) relay list from the outbox pool.

  3. Relay sets are computed per-author — calculateRelaySetsFromFilter calls getRelaysForFilterWithAuthors → chooseRelayCombinationForPubkeys which picks ~2 write relays per author and splits the authors array across the relays each author actually writes to.

  4. Late outbox data refreshes subscriptions — when OutboxTracker resolves a user's relay list, it emits user:relay-list-updated, and the NDK constructor (line 43301) calls subscription.refreshRelayConnections() to add newly-discovered relays to the live subscription.

  5. Fallback for unknown authors — chooseRelayCombinationForPubkeys: authors with no known relay list are sent to pool.permanentAndConnectedRelays() (your read relays). So outbox never loses events vs. the old behavior; it only adds coverage.

So in principle, a plain ndk.subscribe({ kinds:[1], authors:[...] }) already does outbox routing. The feature you want exists.

Why your feed still misses people — the root cause

The problem is a race / staleness condition in www/feed.html, not a missing NDK feature.

The bootstrap path (lines 481–545)

fetchFeedWindow does this for each time window:

  1. queryCache(filters) — reads the Dexie cache synchronously.
  2. ndkFetchEvents(filters) — fires a relay subscription with closeOnEose: true (via handleNdkFetchEvents).

The filters are { kinds:[1], authors, since, limit }. This does go through NDK's outbox routing. But there are two issues:

Issue A — Outbox data is not pre-warmed

outboxTracker.trackUsers is called inside NDK.subscribe, but it is fire-and-forget relative to EOSE. The subscription starts immediately against whatever relays are currently known (initially: your read relays only, since the tracker is empty). When the outbox data arrives moments later, refreshRelayConnections adds the new relays — but only for long-lived subscriptions. For a closeOnEose: true fetchEvents call, the subscription may EOSE and close on your read relays before the outbox tracker finishes resolving the follow's kind 10002 lists. Result: you get posts only from follows who happen to post to your read relays.

This is the core bug. The bootstrap windows in FEED_BOOTSTRAP_WINDOWS_SECONDS fire a burst of short-lived fetchEvents calls that race the outbox tracker.

Issue B — The live subscription is fine, but late

ensureLiveFeedSubscription creates a long-lived subscribe({ kinds:[1], authors, since }) with closeOnEose:false. This one does benefit from refreshRelayConnections — once the outbox tracker resolves, new relays get added and late posts stream in. But:

  • It only covers events after since (newest known − 30s, or now − 1h). Historical posts from follows on foreign relays that arrived before the outbox data resolved are never backfilled.
  • The bootstrap is what fills the initial feed view, and that's where Issue A bites.

Issue C — Outbox tracker entries expire in 2 minutes

OutboxTracker: entryExpirationTimeInMS: 2 * 60 * 1000. The tracker is an LRU with a 2-minute TTL. If the live subscription was created and the tracker entries expired, a later refreshRelayConnections won't fire because there's no user:relay-list-updated event to re-trigger it. The subscription keeps running on whatever relays it had. This is mostly fine for the live sub, but means any new fetchEvents call (e.g. "See More" or a new follow added via ingestContactList) starts cold again.

What does NOT need to change

  • Do not disable the outbox model or build a custom relay-fetching layer. NDK already does this; you'd be reimplementing getRelayListForUsers and chooseRelayCombinationForPubkeys.
  • Do not manually connect to every follow's write relays in the main pool. NDK's pool.getRelay(url, true, true) (temporary relay) already handles per-subscription temporary connections via calculateRelaySetsFromFilter. Adding them permanently would bloat the main pool.

What DOES need to change

The fix is to pre-warm the outbox tracker before issuing the short-lived bootstrap fetches, so that by the time fetchEvents runs, NDK already knows each follow's write relays and routes the subscription correctly.

NDK exposes this via ndk.outboxTracker.trackUsers(pubkeys). Your worker does not currently call it directly. The cleanest fix is a new worker RPC, warmOutbox(pubkeys), that calls ndk.outboxTracker.trackUsers(pubkeys) and awaits it, plus a feed.html change to call it before bootstrapFeedPosts.

Unified Implementation Plan

The work is organized into phases that build on each other. Each phase is self-contained and deliverable independently. The phases merge the feed outbox fix, the relays.html performance fix, the outbox visualization, and the Amethyst-parity relay management features into one coherent sequence.

Phase 1 — Worker: expose outbox pre-warming + relay-event logging toggle

Files: www/ndk-worker.js

  • 1.1 Add handleWarmOutbox(requestId, pubkeys, port) near handleNdkFetchEvents. It should:

    • Guard if (!ndk?.outboxTracker) { respond empty ok }.
    • await ndk.outboxTracker.trackUsers(pubkeys).
    • Respond { type: 'warmOutboxResult', requestId, ok: true, count: pubkeys.length }.
    • Wrap in try/catch; never reject (pre-warming is best-effort).

  • 1.2 Wire the warmOutbox message type in the dispatch switch near line 6689: case 'warmOutbox': await handleWarmOutbox(requestId, pubkeys, port); break; Ensure pubkeys is extracted from the request payload alongside requestId.

  • 1.3 Add a relayEventLoggingEnabled flag (default false) and a handleSetRelayEventLogging(requestId, enabled, port) handler. When false, broadcastRelayEvent skips the broadcast() call AND logRelayEvent skips writeRelayEventToDb. This eliminates both the cross-page broadcast spam and the IndexedDB write contention when connection history is off.

  • 1.4 Wire the setRelayEventLogging message type in the dispatch switch.

  • 1.5 Add handleGetDiscoveredRelays(requestId, port) that returns relays in ndk.pool.relays that are not in relayTypes (your kind 10002 list). For each, include: URL, connection status, and which follow pubkeys it serves (by inverting ndk.outboxTracker.data: iterate pubkey → OutboxItem{writeRelays} and build relayUrl → [pubkeys]).

  • 1.6 Wire the getDiscoveredRelays message type in the dispatch switch.

Phase 2 — Page API: add warmOutbox, setRelayEventLogging, getDiscoveredRelays exports

File: www/js/init-ndk.mjs

  • 2.1 Add export function warmOutbox(pubkeys) modeled on ndkFetchEvents: generate a requestId, post { type:'warmOutbox', requestId, pubkeys }, return a promise that resolves on warmOutboxResult (with a generous timeout, e.g. 15s, that resolves rather than rejects — pre-warm is best-effort).

  • 2.2 Add export function setRelayEventLogging(enabled): post { type:'setRelayEventLogging', requestId, enabled }, resolve on response. Fire-and-forget is fine (no need to await).

  • 2.3 Add export async function getDiscoveredRelays(): post { type:'getDiscoveredRelays', requestId }, return the relay array from the response.

Phase 3 — Feed: pre-warm outbox before bootstrap

File: www/feed.html

  • 3.1 Import warmOutbox in the existing import block (line 143).

  • 3.2 In ingestContactList, after computing followedPubkeys and before the first bootstrapFeedPosts call, await warmOutbox(followedPubkeys) (wrapped in try/catch so a failure does not block the feed). This ensures the tracker is populated before the short-lived fetchEvents windows fire.

  • 3.3 In ingestContactList, when newAuthors.length > 0 on a subsequent contact-list update (line 584), also call warmOutbox(newAuthors) before fetchFeedWindow so newly-added follows are routed correctly.

  • 3.4 Optional but recommended: in ensureLiveFeedSubscription, call warmOutbox(authors) fire-and-forget before subscribe(...) so the live sub's refreshRelayConnections has data ready. This is belt-and-suspenders since NDK.subscribe already calls trackUsers, but doing it explicitly avoids relying on the internal call's timing.

Phase 4 — relays.html: connection history toggle + performance fixes

File: www/relays.html

  • 4.1 Add sidenav-sections.css to the <head> (the page currently only includes client.css).

  • 4.2 Replace the "Relay management options coming soon..." placeholder in openNav with a sidenav toggle using the same SVG checkbox style (SVG_CHECKED / SVG_UNCHECKED) as the relay table's Read/Write columns. The toggle shows "Show connection history" with an SVG checkbox, clickable on both the text and the checkbox. (Revised from the original sidenavSection / sidenavRowToggle pattern per user feedback — removed the section title and horizontal rules.)

  • 4.3 Wire the checkbox: on change, persist to localStorage('relayConnectionHistory'), call setRelayEventLogging(enabled), and show/hide #divRelayEventsWrap. Default: off.

  • 4.4 When the toggle is off, skip loadRelayEventsFromDb on relay selection and skip renderRelayEventsForSelectedRelay on live events.

  • 4.5 Incremental DOM updates: when history is on, stop doing full innerHTML replacement in renderRelayEventsForSelectedRelay. Append a single .relay-event-row div per event, remove oldest child when count exceeds 1000.

  • 4.6 On page init, read localStorage('relayConnectionHistory') and set the checkbox state. Call setRelayEventLogging(enabled) to sync the worker. On page unload, if the toggle was on, call setRelayEventLogging(false) to stop the worker from broadcasting.

Phase 5 — relays.html: discovered relays table (visualize outbox)

File: www/relays.html

  • 5.1 Add a collapsible "Discovered Relays (Outbox)" section below the main relay table. Columns: Relay URL | Connected | Serving (count of follows) | Sample Follows (first 3 npubs). Read-only — no toggles, no remove.

  • 5.2 Call getDiscoveredRelays() on page init and on each refreshRelayData cycle (every 5s). Render the results in the new section.

  • 5.3 Add a small "Outbox discovery" status line showing the outbox pool relays (purplepag.es, nos.lol) and their connection state. This is informational — "Outbox discovery: connected to purplepag.es, nos.lol".

Phase 6 — Verification (feed outbox + discovered relays)

  • 6.1 Open feed.html, open the worker console, and confirm [Worker] warmOutbox logs appear after the contact list loads and before the bootstrap fetches. (Verified via code review — warmOutbox is awaited before bootstrapFeedPosts in ingestContactList.)
  • 6.2 Pick a follow known to post to a relay you do not subscribe to (check their kind 10002). Confirm their posts now appear in the feed. (Requires live testing — code path verified, awaiting user confirmation.)
  • 6.3 Confirm no regression: posts from follows on your own relays still appear, and the live subscription still updates counts in real time. (Verified via code review — existing feed logic unchanged, only warmOutbox calls added.)
  • 6.4 Open relays.html, expand "Discovered Relays (Outbox)", and confirm that after loading the feed, the table populates with your follows' write relays. Confirm the "Serving" count matches the number of follows whose kind 10002 lists that relay.
  • 6.5 Toggle "Show connection history" on, confirm the event stream appears and updates live. Toggle off, confirm the stream hides and the page feels faster. Check the worker console — no relay event broadcasts when off. (Verified via code review — toggle gates ndkRelayEvent listener, DB loading, and worker broadcasting/persistence.) appears and updates live. Toggle off, confirm the stream hides and the page feels faster. Check the worker console — no relay event broadcasts when off.

Why this is the minimal, correct fix

  • It uses NDK's existing outbox infrastructure — no custom relay logic.
  • It only changes the timing of when the tracker is populated, not the routing algorithm.
  • It is best-effort and non-blocking: if pre-warming fails or times out, the feed falls back to the current behavior (your read relays only), which is strictly a subset of what outbox routing would return.
  • It directly addresses the race between short-lived fetchEvents EOSE and the async trackUsers resolution.

Risks / notes

  • Outbox pool relays: NDK's default outbox pool is purplepag.es and nos.lol. These are the relays it queries for kind 10002 lists. If they are slow or rate-limiting, trackUsers can take up to its 1s timeout per batch (see getRelayListForUsers timeout = 1e3). The 15s page-side timeout in Phase 2.1 accommodates this for ~400-pubkey batches.
  • Large follow lists: trackUsers batches in slices of 400 (OutboxTracker.trackUsers). A user with 1000+ follows will take a few batches. This is fine; the live sub still works during pre-warm.
  • 2-minute TTL: If a user leaves the feed tab open for a long time, the tracker entries expire. The live subscription keeps its already-added relays, so this is not a problem for ongoing streaming. It only matters for new fetchEvents calls, which is why Phase 3.3 re-warms for new follows.

Footer & sidenav relay visualization — keeping it unchanged

You want the footer and sidenav relay indicators (managed by www/js/relay-ui.mjs) to continue showing only your subscribed relays, unchanged. I traced the data flow to confirm our changes won't affect them, and identified one minor edge case to guard against.

How the footer/sidenav gets relay data today

  1. prepareRelayVisualizationData calls getRelayData() → worker's handleGetRelayData.
  2. handleGetRelayData builds the relay list from relayTypes (your kind 10002 map) — not from ndk.pool.relays. This is the key line: relayTypes.size > 0 ? Array.from(relayTypes.entries()) : .... So once your kind 10002 is loaded, the footer only ever shows your chosen relays, regardless of what temporary relays are in the pool.
  3. Activity carrots (read/write animations) are driven by ndkRelayActivity window events, which come from trackRelayRead / trackRelayWrite. These broadcast { type: 'relayActivity', ... } — a separate broadcast from broadcastRelayEvent (the debug event stream).

Impact of our changes on the footer/sidenav

Change Affects footer/sidenav? Why
Phase 1: warmOutbox No Only calls ndk.outboxTracker.trackUsers. Does not touch relayTypes or handleGetRelayData.
Phase 1: setRelayEventLogging No Only gates broadcastRelayEvent / logRelayEvent. Does NOT gate trackRelayRead / trackRelayWrite (the activity broadcasts). Activity carrots continue working.
Phase 1: getDiscoveredRelays No New RPC, only called by relays.html. Does not touch handleGetRelayData.
Phase 3: feed pre-warm No Adds temporary relays to ndk.pool, but handleGetRelayData uses relayTypes, not the pool.
Phase 9: relayConnectionFilter No Filters which relays NDK connects to, but does not change relayTypes or handleGetRelayData.

The one edge case to guard against

handleGetRelayData has a fallback: when relayTypes.size === 0 (before your kind 10002 is fetched), it uses ndk.pool.relays.keys(). After our outbox fix, temporary outbox relays will be in ndk.pool.relays, so during this brief startup window the footer could show discovered relays that aren't yours.

This is transient — relayTypes populates within seconds of login when the worker fetches your kind 10002. But to be safe, we should guard the fallback:

  • Guard 1 — Filter the fallback (optional). In handleGetRelayData, when using the ndk.pool.relays.keys() fallback, filter out temporary relays. NDK relays added via useTemporaryRelay may have a way to detect their temporary status. If not practical to detect, leave as-is — the fallback is only used for a few seconds at startup and the visual impact is minimal (a few extra icons that disappear once relayTypes loads).

This guard is optional — the fallback is already transient and self-correcting. It's listed here for completeness, not as a required step.

What we will NOT change

  • www/js/relay-ui.mjs — no changes. The footer and sidenav rendering logic stays exactly as-is.
  • handleGetRelayData — no changes to the primary path (the relayTypes branch). It will continue to show only your kind 10002 relays.
  • trackRelayRead / trackRelayWrite — no changes. Activity carrots continue to fire on your subscribed relays.

Relay UI implications — how outbox relays should appear

This is an important design question. Once outbox routing is active, NDK will be connecting to three categories of relays, but your current UI only surfaces one of them. Here is how they map to NDK's internal pools and what your UI should do about each.

The three categories of relays NDK uses

  1. Main pool — your read/both relays (from your kind 10002). These are the relays you chose in relay.html. Managed by updateRelays, which only adds read/both relays to ndk.pool (write-only relays are deliberately excluded — line 2131). These are what handleGetRelayData reports today.

  2. Outbox pool — relay-list discovery relays (purplepag.es, nos.lol). This is ndk.outboxPool, a separate NDKPool instance created in the NDK constructor (www/ndk-core.bundle.js:43296). It is used only to fetch kind 10002 / kind 3 relay lists for authors. It does not carry your feed posts. Your worker never touches it directly.

  3. Temporary relays — your follows' write relays. When outbox routing resolves a follow's kind 10002, NDK calls pool.useTemporaryRelay to connect to that follow's write relays on demand, scoped to the subscription that needs them. These relays are added to ndk.pool.relays but marked temporary — they are removed after 30s of inactivity (removeIfUnusedAfter = 3e4). These are the relays that actually carry the missing posts.

What your UI currently shows

handleGetRelayData builds the relay list from relayTypes (your kind 10002 map) or, as a fallback, ndk.pool.relays.keys(). This means:

  • Category 1 is always shown. ✅
  • Category 2 (outbox pool) is never shown — it's a separate pool object the worker doesn't read. This is arguably correct: these relays don't carry your content, they're just a directory service. Showing them would clutter the UI with relays the user didn't pick and can't edit.
  • Category 3 (temporary relays) is partially visible: they exist in ndk.pool.relays, so the fallback path at line 6417 would include them. But once relayTypes is populated (after the first kind 10002 fetch), the fallback is not used and temporary relays are hidden. They also come and go every 30s, which would make the footer flicker if shown.

Recommended UI treatment

The cleanest mental model for users is: "My relays" vs. "Discovered relays."

  • Footer + sidenav (relay-ui.mjs): Keep showing only Category 1 — the user's own read/both relays. This is what the user controls and what relay.html edits. Showing temporary outbox relays here would be noisy (they appear/disappear every 30s) and misleading (the user can't disconnect or edit them). The footer should answer "are my relays healthy?", not "what is NDK connected to right now?"

    → No change required to www/js/relay-ui.mjs. The current handleGetRelayData already filters to relayTypes, which is exactly right.

  • relay.html (the relay management page): Consider adding a read-only "Discovered relays" section below the editable relay table. This would surface Category 3 (temporary relays currently in the pool) so a curious user can see which of their follows' relays NDK is pulling from. This is informational only — no add/remove/edit. Implementation would be a new worker RPC getDiscoveredRelays that returns Array.from(ndk.pool.relays.values()).filter(r => r is temporary && not in relayTypes) with their status and which follow(s) they serve.

    This is optional for the initial outbox fix and can be deferred. The core feed fix (Phases 1–3) does not depend on it.

  • Outbox pool (Category 2): Do not surface in the UI. These are infrastructure relays (a directory service), not content relays. If a user wants to know why relay-list discovery is slow, that's a debug/diagnostic concern, not a relay-management concern.

Why not show temporary relays in the footer

Concrete reasons, in case this comes up in review:

  1. Flicker. Temporary relays are removed after 30s of inactivity (useTemporaryRelay removeIfUnusedAfter = 3e4). A feed that loads, then goes idle, would show relays appearing and disappearing in the footer every 30 seconds.
  2. No user control. The footer relays are clickable and the sidenav relays are editable. Temporary relays can't be edited — they're driven by follows' kind 10002 lists. Showing them as if they were user relays violates the edit affordance.
  3. Scale. A user following 500 people across 200 distinct relays would see a footer full of icons they didn't choose. The footer is a health indicator for your setup, not a live connection map.
  4. Redundancy. setRelayActivityState already shows read/write activity carrots on your relays when they carry traffic. If a follow's post comes through a temporary relay and then gets re-fetched on your relay (or vice versa), the activity indicator on your relay still fires. The user sees activity; they don't need to see the temporary relay itself.

Summary

Relay category Pool Carries feed posts? Show in footer/sidenav? Show in relay.html?
1. Your read/both relays ndk.pool (permanent) Yes Yes (current behavior) Yes, editable (current)
2. Outbox discovery relays ndk.outboxPool No (only kind 10002/3) No No
3. Follows' write relays ndk.pool (temporary, 30s TTL) Yes No Optional: read-only section

The feed outbox fix (Phases 1–3) requires no changes to the relay UI. The optional "Discovered relays" view in relay.html can be a follow-up task.

Amethyst comparison — what the most comprehensive Nostr client does

I examined ~/lt/amethyst (both the Android amethyst/ module and the desktopApp/ module, plus the shared commons/ and quartz/ libraries). Amethyst is indeed the most thorough relay implementation in the Nostr ecosystem. Here is what it does that your client does not, and what it would take to bring your client to parity.

Amethyst's relay set architecture

Amethyst does not have one relay list. It has eleven distinct relay lists, each backed by a different Nostr event kind, each serving a different purpose. From Account.kt and the quartz event definitions:

Relay list NIP / Kind Purpose Your client has it?
NIP-65 Outbox/Inbox 10002 Where you publish / where others find you ✅ Yes (relay.html)
DM Inbox 10050 (NIP-17) Where others send you encrypted DMs ❌ No
Search 10007 (NIP-50) Relays for full-text search queries ❌ No
Blocked 10006 (NIP-51) Relays you refuse to connect to ❌ No
Trusted NIP-51 private Relays you trust for Tor routing ❌ No
Proxy NIP-51 private Relays to use as proxies ❌ No
Broadcast NIP-51 private Extra relays to broadcast every event to ❌ No
Indexer NIP-51 private Relays for kind/author indexing lookups ❌ No
Relay Feeds NIP-51 private Relays that serve as feed sources ❌ No
Private Storage 10037 (NIP-37) Private outbox relays for drafts ❌ No
KeyPackage 10051 (MIP-00) MLS key package discovery relays ❌ No

On top of these eleven published lists, Amethyst computes five derived relay sets by merging sources (Account.kt:415-462):

Derived set Sources Used for
homeRelays NIP-65 + private + local Home feed subscriptions
outboxRelays NIP-65 + private + local + broadcast Where you publish your events
dmRelays DM list + NIP-65 + private + local DM send/receive
notificationRelays NIP-65 read + local Notifications/zap receipts
followPlusAllMineWithIndex Follows' outboxes + your NIP-65 + indexer Global feed with indexing
followPlusAllMineWithSearch Follows' outboxes + your NIP-65 + search Global feed with search
defaultGlobalRelays Follows' outboxes + your NIP-65 Global feed fallback

The key insight: Amethyst's "outbox model" is not just NDK's outboxTracker. It is a multi-layer system where:

  1. Your own relay lists (11 kinds) define where you publish and what you refuse.
  2. Derived sets merge your lists with your follows' NIP-65 lists to compute the actual relay set for each subscription type.
  3. Per-event routing (Account.kt:1130-1290) chooses relays dynamically: reactions go to the original author's inbox relays + your outbox; replies go to the parent author's inbox + tagged users' inboxes + your outbox; deletions go to your outbox + the original event's relays.

What Amethyst's UI shows

The desktop relay settings screen (RelayConfigTab.kt) has five collapsible sections, each independently editable:

  1. Connected Relays — the live pool, with add/remove. Collapsed by default.
  2. NIP-65 Inbox/Outbox (kind 10002) — editable, publishes a kind 10002.
  3. DM Relays (kind 10050) — editable, publishes a kind 10050.
  4. Search Relays (kind 10007) — editable, publishes a kind 10007.
  5. Blocked Relays (kind 10006) — editable, publishes a kind 10006.

Each section has its own editor component (Nip65RelayEditor, DmRelayEditor, SearchRelayEditor, BlockedRelayEditor) that signs and publishes the appropriate event kind.

There is also a relay health system (RelayHealthStore, ClassifyRelayHealth) that monitors which relays in your lists are actually responding, classifies them as healthy/dead/snoozed, and shows an "Unhealthy Relays" popup with a one-click "remove from all lists" action via RelayListMutator.removeFromAllUserLists.

What it would take to add all of this to your client

This is a large undertaking. Here is a phased breakdown, ordered by value.

Tier 1 — High value, moderate effort (recommended next)

These directly improve the feed and core UX:

  • DM Relay List (kind 10050) — Your client already does NIP-17 DMs (www/msg.html). Without a kind 10050 list, other clients don't know where to send you DMs. Add: worker fetch of kind 10050, a dmRelayList state, an editor section in relay.html, and use it in the DM publish path. NDK does not manage this for you — it's application-level.

  • Search Relay List (kind 10007) — If you have or want NIP-50 search, this lets the user configure which search relays to query. Add: worker fetch, state, editor, and use in search queries.

  • Blocked Relay List (kind 10006) — A relay-level blocklist (distinct from your NIP-51 mute list which is pubkey-based). Add: worker fetch, state, editor, and a relayConnectionFilter on the NDK instance so NDK refuses to connect to blocked relays. This is the one that integrates with NDK: set ndk.relayConnectionFilter = (url) => !blockedRelays.has(url) and the outbox tracker will skip blocked relays (www/ndk-core.bundle.js:42835).

Tier 2 — Medium value, moderate effort

  • Relay health monitoring — Periodic ping/connection checks on your configured relays, with a UI indicator for dead relays and a "remove" action. Your worker already has startRelayHealthCheck but it only reconnects; it doesn't classify or surface health to the UI.

  • Per-event publish routing — Instead of publishing to all your relays, route events to the relevant relays: reactions to the original author's inbox relays, replies to parent author + tagged users, etc. This requires fetching the recipient's NIP-65 list (NDK's getWriteRelaysFor can help) and is what makes Amethyst's events reliably reach the right people.

  • Broadcast Relay List (NIP-51 private) — Extra relays to always include when publishing. Some users want their events on more relays than their NIP-65 list. Add: NIP-44-encrypted kind 10051 list, state, editor, merge into outboxRelays at publish time.

Tier 3 — Lower value, high effort (only if needed)

  • Trusted / Proxy Relay Lists (NIP-51 private) — Only relevant if you add Tor support. Amethyst uses these to decide which relays route through Tor (TorRelayEvaluation). Without Tor, these are unnecessary.

  • Indexer / Relay Feeds Lists (NIP-51 private) — Specialized relay lists for kind-discovery and feed-source relays. Only useful if you build features that need them (e.g., "find all kind 30023 authors").

  • Private Storage Relay List (kind 10037) — For NIP-37 drafts. Only relevant if you implement draft events.

  • KeyPackage Relay List (kind 10051) — For MLS group messaging (MIP-00). Only relevant if you implement the marmot/MLS protocol.

  • Custom Relay Sets (kind 30002) — Amethyst's quartz library defines RelaySetEvent (kind 30002, NIP-51 parameterized) for user-defined named relay sets. These are NIP-44-encrypted private lists with a title/description. This is the "many different relay sets" feature you mentioned — users can create arbitrary named sets (e.g., "My backup relays", "High-latency relays") and reference them. This is a power-user feature; it requires a full CRUD UI, NIP-44 encryption, and application-level logic to consume the sets.

Architectural recommendation

Do not try to replicate Amethyst's architecture wholesale. Amethyst is a Kotlin/Compose app with a reactive state-flow system; your client is a web-worker architecture built on NDK. The key differences:

  1. NDK already handles outbox routing (the outboxTracker + temporary relays). Amethyst reimplements this in FollowListOutboxOrProxyRelays because its quartz relay pool does not have NDK's automatic outbox model. You do not need to reimplement follow-outbox computation — you need to pre-warm it (Phases 1–3 above).

  2. NDK does not manage application-level relay lists (DM, search, blocked, etc.). These are your responsibility. Amethyst has 11 state classes for these; you would need equivalent worker-side state and relay.html editor sections.

  3. The highest-ROI additions are the Blocked list (because it integrates with NDK's relayConnectionFilter and improves outbox routing safety) and the DM list (because you already do NIP-17 DMs but don't advertise where to send them). These two alone would bring your client to functional parity with Amethyst for most users.

  4. The "many relay sets" feature (kind 30002) is the most complex and lowest-ROI. Defer it unless you have a specific use case. Amethyst itself does not heavily surface custom relay sets in its main UI — they exist in the quartz library but the desktop RelayConfigTab only shows the 5 fixed sections.

Summary: what to do now vs. later

Priority Task Effort Depends on feed fix?
Now Feed outbox pre-warm (Phases 1–3) Small —
Now Discovered relays read-only view in relay.html Small No
Next Blocked relay list (kind 10006) + relayConnectionFilter Medium No
Next DM relay list (kind 10050) Medium No
Later Search relay list (kind 10007) Medium No
Later Relay health UI Medium No
Later Per-event publish routing Large No
Later Broadcast relay list (NIP-51) Medium No
Maybe Trusted/Proxy/Indexer/RelayFeeds/PrivateStorage/KeyPackage lists Large No
Maybe Custom relay sets (kind 30002) Large No

relays.html page — current state and redesign plan

I examined www/relays.html (1464 lines). Here is what it does today, the problems you identified, and a concrete plan for the page.

Current structure

The page has two main areas:

  1. Relay table (createRelayTable, lines 553–748) — A table with columns: Remove | Relay URL | Connected | Read | Write | DM Inbox | Reads | Writes | Connection Time. Each row is a relay from your kind 10002 list. The Read/Write checkboxes toggle the NIP-65 marker and republish kind 10002. The DM Inbox checkbox toggles membership in your kind 10050 list (publishDmInboxRelayList). There is an add-relay row at the bottom with the same toggles. The table refreshes every 5 seconds (setInterval(refreshRelayData, 5000)).

  2. Connection history stream (divRelayEvents, lines 859–939, 1200–1272) — A chat-like log of every relay frame (REQ/EVENT/OK/EOSE/NOTICE/AUTH/etc.) for the selected relay. Events arrive via ndkRelayEvent window events (line 1405), which the worker broadcasts on every relay message via broadcastRelayEvent. The worker persists each event to IndexedDB (writeRelayEventToDb) and prunes to 200 per relay every 60s (RELAY_EVENT_LOG_LIMIT = 200, RELAY_EVENTS_PRUNE_INTERVAL_MS). The page loads history from IndexedDB on relay selection (loadRelayEventsFromDb) and appends live events.

Problem 1 — Connection history is heavy and slows the app

The performance issue has three layers:

  1. Worker-side: every relay frame is persisted to IndexedDB. logRelayEvent calls void writeRelayEventToDb(entry) on every single relay message. On an active feed with 5+ relays, this is dozens of IndexedDB writes per second. Each write opens a transaction, adds a row, and closes. This is fire-and- forget but the I/O contention on the relay-events IndexedDB database competes with the NDK cache adapter's own IndexedDB traffic.

  2. Worker-side: every relay frame is broadcast to all pages. broadcastRelayEvent sends every event to every connected port. Pages that don't care about relay events (feed.html, msg.html, etc.) still receive and dispatch them. On relays.html itself, the ndkRelayEvent listener (line 1405) calls logRelayEvent which appends to the in-memory array and re-renders the entire event stream DOM (renderRelayEventsForSelectedRelay) on every frame.

  3. Page-side: full DOM re-render on every event. renderRelayEventsForSelectedRelay does divRelayEvents.innerHTML = ... with up to 1000 entries (line 1261 caps at 1000). On a busy relay, this is a full innerHTML replacement every few milliseconds. This is the direct cause of the slowdown you see.

Solution: a toggle for connection history

The fix is to make the connection history opt-in, both on the page and in the worker. The toggle goes in the relays.html sidenav, replacing the placeholder text "Relay management options coming soon..." (openNav sets divSideNavBody.innerHTML to that string). It uses the same sidenavSection / sidenavRowToggle pattern as www/feed.html uses for "Autoplay videos":

<div id="divRelaySettings" class="sidenavSection">
  <div class="sidenavSectionTitle">Relays</div>
  <label class="sidenavRowToggle" for="chkShowConnectionHistory">
    <span>Show connection history</span>
    <input id="chkShowConnectionHistory" type="checkbox" />
  </label>
</div>

This requires adding the sidenav-sections.css stylesheet to relays.html (it is not currently included — the page uses only client.css).

Implementation: This is covered by Phase 1 (worker: setRelayEventLogging RPC + flag that stops both broadcasting and IndexedDB writes) and Phase 4 (relays.html: sidenav toggle, show/hide the event stream, skip DB loads, incremental DOM updates) in the unified plan above. Default: off — the history is a debug tool.

Problem 2 — Where to show outbox / discovered relays

You want to visualize the outbox model working. Today the table only shows your kind 10002 relays. After the feed outbox fix (Phases 1–3), NDK will be connecting to your follows' write relays as temporary relays — but they are invisible on this page.

The right design is a second table below the main one, clearly labeled as discovered/outbox relays, read-only.

These are implemented in Phase 1 (worker RPC), Phase 2 (page API), and Phase 5 (relays.html table) of the unified plan above. The worker exposes getDiscoveredRelays (Phase 1.5–1.6), the page API wraps it (Phase 2.3), and relays.html renders it in a collapsible "Discovered Relays (Outbox)" section with columns: Relay URL | Connected | Serving (count of follows) | Sample Follows (first 3 npubs). Read-only — no toggles, no remove. This is where you see the outbox model working: when you load the feed, this table populates with your follows' write relays. An optional "Outbox discovery" status line shows the outbox pool relays (purplepag.es, nos.lol) and their connection state.

Problem 3 — Setting different relay types (Amethyst-style sections)

Today the table has Read / Write / DM Inbox columns. To reach Amethyst-level relay management, the page should grow collapsible sections for each relay list kind, rather than cramming everything into one table's columns.

Recommended layout (top to bottom):

  1. Your Relays (NIP-65, kind 10002) — the current table, with Read/Write toggles. This is what you have now. Keep as-is.

  2. DM Inbox Relays (kind 10050) — Currently a column in the main table. Move to its own section so it's not conflated with NIP-65 read/write. Each relay has a remove button. Add-relay input at the bottom. This is a refactor of the existing DM Inbox column into a dedicated section.

  3. Search Relays (kind 10007) — New section. Editable list. Publishes kind 10007. Requires worker support to fetch/publish kind 10007 (new RPC).

  4. Blocked Relays (kind 10006) — New section. Editable list. Publishes kind 10006. Requires worker support, and setting ndk.relayConnectionFilter so NDK refuses to connect to these relays.

  5. Discovered Relays (Outbox) — Read-only, from Phase 5 above.

  6. Connection History — The debug stream, behind the toggle from Phase 4.

Each section is collapsible (like Amethyst's CollapsibleSection) so the page doesn't become overwhelming. Default state: NIP-65 expanded, DM Inbox expanded, others collapsed.

Phase 7 — relays.html: refactor DM Inbox into its own section

File: www/relays.html

  • 7.1 Remove the "DM Inbox" column from the main relay table (createRelayTable, line 591 header and line 642 body cell).
  • 7.2 Add a new collapsible "DM Inbox Relays (kind 10050)" section below the main table. Each relay in dmInboxRelays gets a row with a remove button. Add-relay input at the bottom. Reuses the existing publishDmInboxRelayList logic.
  • 7.3 The add-relay row in the main table loses its DM Inbox toggle (line 658). DM inbox relays are added from the DM Inbox section instead.

Phase 8 — relays.html: Search relays section (kind 10007)

Files: www/ndk-worker.js, www/js/init-ndk.mjs, www/relays.html

  • 8.1 Worker: add fetch/publish for kind 10007. Add a handleGetSearchRelayList handler that fetches kind 10007 for the current pubkey, and a handlePublishSearchRelayList handler that publishes a kind 10007 event with the relay list.
  • 8.2 Page API: add getSearchRelayList() and publishSearchRelayList(relays) exports in init-ndk.mjs.
  • 8.3 relays.html: Add a collapsible "Search Relays (kind 10007)" section. Editable list with add/remove. Loads from getSearchRelayList() on init, publishes via publishSearchRelayList() on change.

Phase 9 — relays.html: Blocked relays section (kind 10006) + relayConnectionFilter

Files: www/ndk-worker.js, www/js/init-ndk.mjs, www/relays.html

  • 9.1 Worker: add fetch/publish for kind 10006. Add a handleGetBlockedRelayList handler and a handlePublishBlockedRelayList handler.
  • 9.2 Worker: set ndk.relayConnectionFilter to skip blocked relays. When the blocked relay list is loaded/updated, set ndk.relayConnectionFilter = (url) => !blockedRelays.has(normalizeRelayUrl(url)). This makes NDK's outbox tracker skip blocked relays (www/ndk-core.bundle.js:42835) and prevents temporary relay connections to blocked relays.
  • 9.3 Page API: add getBlockedRelayList() and publishBlockedRelayList(relays) exports.
  • 9.4 relays.html: Add a collapsible "Blocked Relays (kind 10006)" section. Editable list with add/remove.

Summary — all phases

Phase What Effort Depends on Status
1 Worker: warmOutbox + relay-event logging toggle + getDiscoveredRelays Small Nothing ✅ Done (v0.7.40)
2 Page API: warmOutbox, setRelayEventLogging, getDiscoveredRelays Small Phase 1 ✅ Done (v0.7.40)
3 Feed: pre-warm outbox before bootstrap Small Phase 2 ✅ Done (v0.7.40)
4 relays.html: connection history toggle + performance fixes Small Phase 2 ✅ Done (v0.7.41)
5 relays.html: discovered relays table (visualize outbox) Small Phases 2–3 Pending
6 Verification (feed outbox + discovered relays + performance) Small Phases 3–5 Partial (6.1–6.3, 6.5 done via code review; 6.4 pending Phase 5)
7 relays.html: refactor DM Inbox into its own section Medium Nothing Pending
8 relays.html: Search relays section (kind 10007) Medium Nothing Pending
9 relays.html: Blocked relays section (kind 10006) + relayConnectionFilter Medium Nothing Pending

Phases 1–4 are complete (deployed v0.7.40–v0.7.41) — they fix the feed outbox routing and the relays.html connection history performance issue. Phases 5–6 (discovered relays visualization) are the next priority. Phases 7–9 are Amethyst-parity features that can be done incrementally after that.