client/plans/long-running-memory-bloat-fix.md

Long-Running Memory Bloat & Unresponsiveness Fix Plan

Problem Statement

When the client runs for extended periods, pages become bloated and unresponsive. The root causes fall into four categories:

1. Unbounded in-memory growth — arrays/Maps/Sets that accumulate events without eviction
2. Listener/event accumulation — DOM listeners and worker subscriptions that are never cleaned up
3. Over-aggressive polling — 1-second intervals on every page, each triggering multiple worker round-trips
4. Full re-renders — innerHTML = '' + rebuild on every incoming event, discarding DOM and re-attaching all listeners

The SharedWorker broadcasts every subscription event to every connected tab, and pages process all of them regardless of which subscription they originated from. Combined with full-DOM-rebuild rendering, this causes O(n) work on every event and linear memory growth over the session lifetime.

Scope

Pages affected: index.html, feed.html, post.html, notifications.html, relays.html Worker affected: ndk-worker.js Shared module affected: js/init-ndk.mjs

Architecture Overview

flowchart LR
    subgraph Worker[ndk-worker.js SharedWorker]
        sub[activeSubscriptions Map]
        stats[relayActivityStats Map]
        ports[connectedPorts array]
        wallet[lightwalletTokenEvents Map]
        broadcast[broadcast to all ports]
    end

    subgraph Tabs[Open Tabs - each gets ALL broadcasts]
        index[index.html]
        feed[feed.html]
        post[post.html]
        notif[notifications.html]
        relays[relays.html]
    end

    sub -->|event/eose per sub| broadcast
    stats -->|relayActivity per read/write| broadcast
    broadcast -->|unfiltered| index
    broadcast -->|unfiltered| feed
    broadcast -->|unfiltered| post
    broadcast -->|unfiltered| notif
    broadcast -->|unfiltered| relays

    feed -->|posts array grows forever| MemLeak[Memory Leak]
    notif -->|notificationsById grows forever| MemLeak
    notif -->|document.click leaked per row| ListenerLeak[Listener Leak]
    feed -->|innerHTML rebuild per event| CPUSpike[CPU Spike]
    relays -->|table rebuild every 5s| CPUSpike

Implementation Phases

Phase 1: P0 — Critical Rendering & Listener Leaks

Goal: Eliminate the two most severe causes of user-visible jank and memory growth.

1.1 Append-only rendering for feed and post pages

Files: www/feed.html, www/post.html

Current behavior: renderFeed() does divFeed.innerHTML = '' and rebuilds all cards on every incoming event.

Changes:

• Split renderFeed() into two functions:
  • renderFeed() — full rebuild, only called on initial load, "See More", sort changes, or mute removal
  • prependPostCard(post) — creates a single card, inserts it at the top of divFeed if newer than the first visible post, and wires interactions for just that one card
• In the ndkEvent handler, call prependPostCard(evt) instead of renderFeed() when a new kind-1 event arrives and the feed is already rendered
• Cap the visible DOM: if divFeed.children.length > displayCount + SEE_MORE_INCREMENT, remove the last child
• Keep posts[] sorted by inserting new events in the correct position (binary insert or simple unshift + periodic sort)

Acceptance criteria:

• New posts appear at the top without rebuilding existing cards
• Existing images/videos are not reloaded when a new post arrives
• "See More" still does a full re-render to show additional posts

1.2 Append-only rendering for notifications page

Files: www/notifications.html

Current behavior: renderNotifications() does divNotifications.innerHTML = '' and rebuilds all rows on every new notification.

Changes:

• Split into renderNotifications() (full rebuild for filter changes / "View more") and prependNotificationRow(item) (single new row)
• In addNotificationFromEvent, call prependNotificationRow when the notifications list is already rendered and the new item passes the filter
• Cap visible DOM to visibleNotificationsCount + NOTIFICATIONS_PAGE_SIZE

Acceptance criteria:

• New notifications appear without rebuilding existing rows
• Filter toggles and "View more" still do full re-renders

1.3 Fix notification document.click listener leak

Files: www/notifications.html

Current behavior: renderNotifications() registers a document.addEventListener('click', ...) for every notification row, and these are never removed.

Changes:

• Remove the per-row document.addEventListener('click', ...) entirely
• Register a single document-level click listener in main() that closes any open menu panel by checking document.querySelector('.notif-menu-btn[aria-expanded="true"]') or similar
• Alternatively, use a module-level openMenuPanel variable; the single listener checks if the click target is outside openMenuPanel and closes it

Acceptance criteria:

• Only one document click listener exists regardless of notification count
• Menu panels still close when clicking outside

1.4 Cap posts[] and postIds with rolling eviction

Files: www/feed.html, www/post.html

Current behavior: posts array and postIds Set grow without bound.

Changes:

• Add const MAX_STORED_POSTS = 500 constant
• In upsertFeedPost / upsertPost, after pushing a new event, if posts.length > MAX_STORED_POSTS, sort by created_at descending, slice to MAX_STORED_POSTS, and rebuild postIds from the remaining array
• Also remove evicted post IDs from renderedPostIds
• In post.html event mode, skip this (single event display)

Acceptance criteria:

• posts.length never exceeds MAX_STORED_POSTS
• Evicted posts are removed from postIds and renderedPostIds
• "See More" still works within the capped array

1.5 Cap notificationsById and unreadNotificationsById with periodic pruning

Files: www/notifications.html

Current behavior: Both Maps grow without bound; pruneNotificationsByCurrentSettings only runs on filter toggles.

Changes:

• Add const MAX_STORED_NOTIFICATIONS = 500
• Add a pruneNotificationMaps() function that:
  • Sorts notificationsById entries by created_at descending
  • Keeps only the top MAX_STORED_NOTIFICATIONS
  • Deletes evicted entries from both notificationsById and unreadNotificationsById
  • Also prunes postImageCache to an LRU of 200 entries
• Call pruneNotificationMaps() at the end of addNotificationFromEvent if notificationsById.size > MAX_STORED_NOTIFICATIONS + 100
• Also call it after hydrateNotificationsFromCache and hydrateNotificationsFromRelays

Acceptance criteria:

• notificationsById.size never exceeds MAX_STORED_NOTIFICATIONS + 100
• postImageCache.size never exceeds 200
• Filter counts remain accurate after pruning

---

Phase 2: P1 — Subscription Hygiene & Polling Reduction

Goal: Stop the cross-tab event storm and reduce worker message queue saturation.

2.1 Filter ndkEvent by subId on the page side

Files: www/feed.html, www/post.html, www/notifications.html, www/js/init-ndk.mjs

Current behavior: The worker includes subId in event broadcasts (ndk-worker.js:1094), but pages' ndkEvent handlers process all events regardless of source subscription.

Changes:

• In init-ndk.mjs, the ndkEvent CustomEvent already includes the full message.data — verify subId is in event.detail
• Each page tracks its own active subscription IDs in a Set:
  • feed.html: track the kind-3 sub and the live feed sub
  • post.html: track the kind-1 author/event sub
  • notifications.html: track the notifications sub
• In the ndkEvent handler, check if (!mySubscriptionIds.has(event.detail.subId)) return; before processing

Acceptance criteria:

• Pages only process events from their own subscriptions
• Events from other tabs' subscriptions are ignored (no CPU work, no DOM updates)

2.2 Close old live subscriptions on follow-list change

Files: www/feed.html

Current behavior: ensureLiveFeedSubscription creates a new subscription when follows change but never closes the old one.

Changes:

• Add a module-level let liveFeedSub = null; variable
• In ensureLiveFeedSubscription, before creating the new subscription:

  if (liveFeedSub?.close) {
    liveFeedSub.close();
  }
  

• Assign the return of subscribe() to liveFeedSub
• Also track the kind-3 subscription similarly and close it on logout

Acceptance criteria:

• Only one live feed subscription exists at a time in the worker
• activeSubscriptions Map in the worker doesn't accumulate stale feed subscriptions

2.3 Increase footer polling interval to 5 seconds (or event-driven)

Files: www/index.html, www/feed.html, www/notifications.html, www/relays.html

Current behavior: setInterval(UpdateFooter, 1000) on every page.

Changes:

• Change all 1-second footer intervals to 5 seconds (matching post.html which already uses 5s)
• index.html line 850: setInterval(UpdateFooter, 5000)
• feed.html line 692: setInterval(updateFooter, 5000)
• notifications.html line 1390: setInterval(UpdateFooter, 5000)
• relays.html line 1794: setInterval(UpdateFooter, 5000)
• Future enhancement (not in this phase): Make footer fully event-driven by reacting to ndkRelays and ndkRelayActivity events instead of polling

Acceptance criteria:

• No page polls the worker for footer data more than once per 5 seconds
• Footer still shows current relay status (5s staleness is acceptable)

2.4 Diff-update relay table instead of full rebuild

Files: www/relays.html

Current behavior: createRelayTable does divRelays.innerHTML = html every 5 seconds, destroying all rows, inputs, and listeners.

Changes:

• Split createRelayTable into:
  • buildRelayTable(relays) — initial build (full innerHTML)
  • updateRelayTableRows(relays) — in-place update of existing rows:
    • Update status icon, read/write counts, connection time text
    • Update checkbox SVGs if relay type changed
    • Add/remove rows if relay count changed
    • Preserve the add-relay input value and focus
• refreshRelayData calls updateRelayTableRows if the table already exists and the relay count matches; otherwise calls buildRelayTable

Acceptance criteria:

• Relay table updates without losing input focus or re-attaching all listeners
• Add-relay input value is preserved across refreshes (already partially implemented)

2.5 Throttle relayActivity broadcasts in the worker

Files: www/ndk-worker.js

Current behavior: trackRelayRead and trackRelayWrite broadcast relayActivity on every single event read/write.

Changes:

• Add a relayActivityBroadcastTimer and a pendingRelayActivity Set of relay URLs that have had activity since the last broadcast
• In trackRelayRead / trackRelayWrite, add the relay URL to pendingRelayActivity and schedule a 500ms batched broadcast if not already scheduled
• The batched broadcast sends one relayActivity message per pending relay with current stats, then clears the set
• This reduces broadcast frequency from per-event to per-500ms-batch

Acceptance criteria:

• relayActivity broadcasts are batched to at most once per 500ms
• Stats counts remain accurate (they're cumulative)
• Footer/sidenav relay indicators still update within 1s of activity

---

Phase 3: P2 — Worker Cleanup & Cache Eviction

Goal: Prevent unbounded growth in worker-side state and clean up dead connections.

3.1 Prune connectedPorts on tab close

Files: www/ndk-worker.js

Current behavior: connectedPorts accumulates dead ports; broadcast() iterates all of them.

Changes:

• In the onconnect handler (ndk-worker.js:7600), add:

  port.onclose = () => {
      connectedPorts = connectedPorts.filter(p => p !== port);
      if (activeSigningPort === port) activeSigningPort = null;
  };
  

Acceptance criteria:

• connectedPorts only contains live ports
• broadcast() doesn't iterate dead ports

3.2 Evict inlineComposerInstances on re-render

Files: www/feed.html, www/post.html

Current behavior: inlineComposerInstances Map retains entries with detached hostEl after renderFeed() clears the container.

Changes:

• At the start of renderFeed() (the full rebuild path), after divFeed.innerHTML = '':

  inlineComposerInstances.clear();
  

• Since the DOM is rebuilt, all composer host elements are gone; the Map entries are pure garbage

Acceptance criteria:

• inlineComposerInstances is empty after a full re-render
• Composers can still be re-created on demand via getOrCreateInlineComposer

3.3 Prune temporary relay Maps after broadcast publish

Files: www/ndk-worker.js

Current behavior: relayActivityStats, relayConnectAttemptStartedAt, relayLastDisconnectMeta accumulate entries for every temporary broadcast relay.

Changes:

• After the broadcast publish phase: 'done' block (ndk-worker.js:6410), add a cleanup step:

  const userRelayUrls = new Set(Array.from(relayTypes.keys()).map(normalizeRelayUrl));
  for (const url of relayActivityStats.keys()) {
      if (!userRelayUrls.has(url)) relayActivityStats.delete(url);
  }
  relayConnectAttemptStartedAt.clear(); // only used during active connect attempts
  // Keep relayLastDisconnectMeta for reconnect detection
  

• Only prune relayActivityStats — the disconnect meta is needed for reconnect subscription rebuilding

Acceptance criteria:

• relayActivityStats only contains entries for the user's own relays after a broadcast completes
• Reconnect detection still works (disconnect meta preserved)

3.4 Periodic reconciliation of lightwalletTokenEvents

Files: www/ndk-worker.js

Current behavior: lightwalletTokenEvents and lightwalletDeletedIds grow monotonically; deleted entries are only removed during publishDirectProofs.

Changes:

• Add a 5-minute interval timer lightwalletReconcileTimer that:
  • Iterates lightwalletTokenEvents and removes any entry whose id is in lightwalletDeletedIds
  • Optionally trims lightwalletDeletedIds to the most recent 500 entries (keep recent ones to prevent re-processing)
• Clear this timer in handleDisconnect

Acceptance criteria:

• lightwalletTokenEvents doesn't retain deleted token events
• lightwalletDeletedIds is bounded to ~500 entries

3.5 Clear health-check and prune intervals on disconnect

Files: www/ndk-worker.js

Current behavior: startRelayHealthCheck (ndk-worker.js:1262) and scheduleRelayEventPruning (ndk-worker.js:789) create intervals that are never cleared.

Changes:

• Store the health-check interval ID in a module-level variable relayHealthCheckTimer
• Store the prune interval ID in relayEventsPruneTimer (already exists)
• In handleDisconnect() (ndk-worker.js:7350), add:

  if (relayHealthCheckTimer) { clearInterval(relayHealthCheckTimer); relayHealthCheckTimer = null; }
  if (relayEventsPruneTimer) { clearInterval(relayEventsPruneTimer); relayEventsPruneTimer = null; }
  

Acceptance criteria:

• No duplicate intervals after re-init
• Intervals are cleared on disconnect

---

Phase 4: P3 — Minor Leaks & Polish

Goal: Clean up remaining minor leaks and harden against edge cases.

4.1 Clear refreshTimes interval on notifications logout

Files: www/notifications.html

Current behavior: setInterval(refreshTimes, 30000) is never assigned to a variable and can't be cleared.

Changes:

• Assign to let refreshTimesIntervalId = setInterval(refreshTimes, 30000)
• Clear it in Logout()

4.2 Remove duplicate window.addEventListener('message', ...) in relays.html

Files: www/relays.html

Current behavior: relays.html:1777 has a fallback message listener that duplicates the ndkRelayActivity handler.

Changes:

• Remove the window.addEventListener('message', ...) block at lines 1777–1783
• The ndkRelayActivity handler at line 1762 already covers this

4.3 Add beforeunload cleanup for subscriptions on all pages

Files: www/feed.html, www/post.html, www/notifications.html

Current behavior: Pages don't close their subscriptions on beforeunload, leaving stale entries in the worker's activeSubscriptions Map until the worker detects the port close.

Changes:

• Add window.addEventListener('beforeunload', () => { disconnect(); }) (or close individual subscriptions) on each page
• The disconnect() function in init-ndk.mjs already sends a disconnect message and closes the port

Acceptance criteria:

• Worker's activeSubscriptions is cleaned up when a tab closes

---

Testing Strategy

Manual Testing

1. Open index.html, feed.html, post.html, notifications.html, relays.html in separate tabs
2. Leave them running for 30+ minutes with active subscriptions
3. Check browser DevTools Memory tab:
   • Take heap snapshots at 5min, 15min, 30min
   • Verify heap growth is sublinear (not growing with event count)
4. Check Performance tab:
   • Record a 30s profile while events are flowing
   • Verify no full renderFeed / renderNotifications on every event
5. Check that relay footer updates within 5s of connection changes
6. Check that notification menu panels still open/close correctly

Automated Checks

• After each phase, run the existing test suite (tests/broadcast-relay-test.sh etc.)
• Add a memory regression test if feasible (puppeteer heap snapshot comparison)

Risk Assessment

Phase	Risk	Mitigation
Phase 1 (append-only)	Sort order bugs if new event is older than visible posts	Only prepend if newer than first visible; fall back to full render otherwise
Phase 1 (listener fix)	Menu panel might not close in all cases	Test thoroughly; use single delegated listener
Phase 2 (subId filter)	Might miss events if subId tracking is wrong	Log filtered vs processed events during testing
Phase 2 (polling change)	Footer might feel less responsive	5s is still fast; relay status rarely changes in seconds
Phase 3 (worker cleanup)	Might prune stats too aggressively	Only prune temp relay stats, not user's own relays

Files Modified

File	Phases
www/feed.html	1, 2, 3, 4
www/post.html	1, 2, 3, 4
www/notifications.html	1, 2, 3, 4
www/relays.html	2, 4
www/index.html	2
www/ndk-worker.js	2, 3
www/js/init-ndk.mjs	2

Implementation Checklist

Phase 1: P0 — Critical Rendering & Listener Leaks

• 1.1 Append-only rendering for feed and post pages
  • Split renderFeed() into full rebuild + prependPostCard()
  • New posts prepend without rebuilding existing cards
  • Cap visible DOM to displayCount (trims last child on overflow)
  • "See More" still does full re-render
  • Verified: data-post-id attribute set by renderPostItem for trim logic
  • Reviewed: no errors found, fallback paths correct
• 1.2 Append-only rendering for notifications page
  • Split renderNotifications() into full rebuild + prependNotificationRow()
  • Extracted buildNotificationRow(item) helper shared by both functions
  • New notifications prepend without rebuilding existing rows
  • Filter toggles and "View more" still do full re-renders
• 1.3 Fix notification document.click listener leak
  • Remove per-row document.addEventListener('click', ...)
  • Register single delegated document-level click listener in main()
  • Added notif-menu-panel class for panel discovery
  • Menu panels still close when clicking outside
• 1.4 Cap posts[] and postIds with rolling eviction
  • Add MAX_STORED_POSTS = 500 constant
  • Evict oldest posts when array exceeds cap (+50 batch threshold)
  • Rebuild postIds and renderedPostIds after eviction
  • Skip pruning in post.html event mode
• 1.5 Cap notificationsById / unreadNotificationsById with periodic pruning
  • Add MAX_STORED_NOTIFICATIONS = 500
  • Add pruneNotificationMaps() function
  • Call prune on overflow, after cache/relay hydration
  • LRU-cap postImageCache to 200 entries

Phase 2: P1 — Subscription Hygiene & Polling Reduction

• 2.1 Filter ndkEvent by subId on the page side
  • Verify subId is in event.detail from init-ndk.mjs
  • Track active subscription IDs per page
  • Ignore events from foreign subscriptions
• 2.2 Close old live subscriptions on follow-list change
  • Store live feed subscription handle
  • Close previous subscription before creating new one
• 2.3 Increase footer polling to 5 seconds
  • index.html: 1s → 5s
  • feed.html: 1s → 5s
  • notifications.html: 1s → 5s
  • relays.html: 1s → 5s
• 2.4 Diff-update relay table instead of full rebuild
  • Split createRelayTable into build + update
  • In-place update of status icons, counts, checkboxes
  • Preserve add-relay input focus across refreshes
• 2.5 Throttle relayActivity broadcasts in the worker
  • Add batched broadcast with 500ms timer
  • Coalesce per-event broadcasts into per-batch
  • Stats counts remain accurate

Phase 3: P2 — Worker Cleanup & Cache Eviction

• 3.1 Prune connectedPorts on tab close
  • Add port.onclose handler in onconnect
  • Clear activeSigningPort if it was the closed port
• 3.2 Evict inlineComposerInstances on re-render
  • Clear Map after innerHTML = '' in full rebuild path
• 3.3 Prune temporary relay Maps after broadcast publish
  • Remove non-user relay entries from relayActivityStats
  • Clear relayConnectAttemptStartedAt
  • Preserve relayLastDisconnectMeta for reconnect detection
• 3.4 Periodic reconciliation of lightwalletTokenEvents
  • Add 5-minute interval timer
  • Remove deleted entries from lightwalletTokenEvents
  • Trim lightwalletDeletedIds to 500
  • Clear timer in handleDisconnect
• 3.5 Clear health-check and prune intervals on disconnect
  • Store health-check interval ID
  • Clear both intervals in handleDisconnect

Phase 4: P3 — Minor Leaks & Polish

• 4.1 Clear refreshTimes interval on notifications logout
• 4.2 Remove duplicate message listener in relays.html
• 4.3 Add beforeunload subscription cleanup on all pages