# Long-Running Memory Bloat & Unresponsiveness Fix Plan

## Problem Statement

When the client runs for extended periods, pages become bloated and unresponsive. The root causes fall into four categories:

1. **Unbounded in-memory growth** — arrays/Maps/Sets that accumulate events without eviction
2. **Listener/event accumulation** — DOM listeners and worker subscriptions that are never cleaned up
3. **Over-aggressive polling** — 1-second intervals on every page, each triggering multiple worker round-trips
4. **Full re-renders** — `innerHTML = ''` + rebuild on every incoming event, discarding DOM and re-attaching all listeners

The SharedWorker broadcasts every subscription event to every connected tab, and pages process all of them regardless of which subscription they originated from. Combined with full-DOM-rebuild rendering, this causes O(n) work on every event and linear memory growth over the session lifetime.

## Scope

**Pages affected:** `index.html`, `feed.html`, `post.html`, `notifications.html`, `relays.html`
**Worker affected:** `ndk-worker.js`
**Shared module affected:** `js/init-ndk.mjs`

## Architecture Overview

```mermaid
flowchart LR
    subgraph Worker[ndk-worker.js SharedWorker]
        sub[activeSubscriptions Map]
        stats[relayActivityStats Map]
        ports[connectedPorts array]
        wallet[lightwalletTokenEvents Map]
        broadcast[broadcast to all ports]
    end

    subgraph Tabs[Open Tabs - each gets ALL broadcasts]
        index[index.html]
        feed[feed.html]
        post[post.html]
        notif[notifications.html]
        relays[relays.html]
    end

    sub -->|event/eose per sub| broadcast
    stats -->|relayActivity per read/write| broadcast
    broadcast -->|unfiltered| index
    broadcast -->|unfiltered| feed
    broadcast -->|unfiltered| post
    broadcast -->|unfiltered| notif
    broadcast -->|unfiltered| relays

    feed -->|posts array grows forever| MemLeak[Memory Leak]
    notif -->|notificationsById grows forever| MemLeak
    notif -->|document.click leaked per row| ListenerLeak[Listener Leak]
    feed -->|innerHTML rebuild per event| CPUSpike[CPU Spike]
    relays -->|table rebuild every 5s| CPUSpike
```

## Implementation Phases

### Phase 1: P0 — Critical Rendering & Listener Leaks

**Goal:** Eliminate the two most severe causes of user-visible jank and memory growth.

#### 1.1 Append-only rendering for feed and post pages

**Files:** [`www/feed.html`](www/feed.html), [`www/post.html`](www/post.html)

**Current behavior:** [`renderFeed()`](www/feed.html:460) does `divFeed.innerHTML = ''` and rebuilds all cards on every incoming event.

**Changes:**
- Split `renderFeed()` into two functions:
  - `renderFeed()` — full rebuild, only called on initial load, "See More", sort changes, or mute removal
  - `prependPostCard(post)` — creates a single card, inserts it at the top of `divFeed` if newer than the first visible post, and wires interactions for just that one card
- In the `ndkEvent` handler, call `prependPostCard(evt)` instead of `renderFeed()` when a new kind-1 event arrives and the feed is already rendered
- Cap the visible DOM: if `divFeed.children.length > displayCount + SEE_MORE_INCREMENT`, remove the last child
- Keep `posts[]` sorted by inserting new events in the correct position (binary insert or simple unshift + periodic sort)

**Acceptance criteria:**
- New posts appear at the top without rebuilding existing cards
- Existing images/videos are not reloaded when a new post arrives
- "See More" still does a full re-render to show additional posts

#### 1.2 Append-only rendering for notifications page

**Files:** [`www/notifications.html`](www/notifications.html)

**Current behavior:** [`renderNotifications()`](www/notifications.html:886) does `divNotifications.innerHTML = ''` and rebuilds all rows on every new notification.

**Changes:**
- Split into `renderNotifications()` (full rebuild for filter changes / "View more") and `prependNotificationRow(item)` (single new row)
- In `addNotificationFromEvent`, call `prependNotificationRow` when the notifications list is already rendered and the new item passes the filter
- Cap visible DOM to `visibleNotificationsCount + NOTIFICATIONS_PAGE_SIZE`

**Acceptance criteria:**
- New notifications appear without rebuilding existing rows
- Filter toggles and "View more" still do full re-renders

#### 1.3 Fix notification `document.click` listener leak

**Files:** [`www/notifications.html`](www/notifications.html)

**Current behavior:** [`renderNotifications()`](www/notifications.html:1046) registers a `document.addEventListener('click', ...)` for **every** notification row, and these are never removed.

**Changes:**
- Remove the per-row `document.addEventListener('click', ...)` entirely
- Register a **single** document-level click listener in `main()` that closes any open menu panel by checking `document.querySelector('.notif-menu-btn[aria-expanded="true"]')` or similar
- Alternatively, use a module-level `openMenuPanel` variable; the single listener checks if the click target is outside `openMenuPanel` and closes it

**Acceptance criteria:**
- Only one document click listener exists regardless of notification count
- Menu panels still close when clicking outside

#### 1.4 Cap `posts[]` and `postIds` with rolling eviction

**Files:** [`www/feed.html`](www/feed.html), [`www/post.html`](www/post.html)

**Current behavior:** `posts` array and `postIds` Set grow without bound.

**Changes:**
- Add `const MAX_STORED_POSTS = 500` constant
- In `upsertFeedPost` / `upsertPost`, after pushing a new event, if `posts.length > MAX_STORED_POSTS`, sort by `created_at` descending, slice to `MAX_STORED_POSTS`, and rebuild `postIds` from the remaining array
- Also remove evicted post IDs from `renderedPostIds`
- In `post.html` event mode, skip this (single event display)

**Acceptance criteria:**
- `posts.length` never exceeds `MAX_STORED_POSTS`
- Evicted posts are removed from `postIds` and `renderedPostIds`
- "See More" still works within the capped array

#### 1.5 Cap `notificationsById` and `unreadNotificationsById` with periodic pruning

**Files:** [`www/notifications.html`](www/notifications.html)

**Current behavior:** Both Maps grow without bound; [`pruneNotificationsByCurrentSettings`](www/notifications.html:570) only runs on filter toggles.

**Changes:**
- Add `const MAX_STORED_NOTIFICATIONS = 500`
- Add a `pruneNotificationMaps()` function that:
  - Sorts `notificationsById` entries by `created_at` descending
  - Keeps only the top `MAX_STORED_NOTIFICATIONS`
  - Deletes evicted entries from both `notificationsById` and `unreadNotificationsById`
  - Also prunes `postImageCache` to an LRU of 200 entries
- Call `pruneNotificationMaps()` at the end of `addNotificationFromEvent` if `notificationsById.size > MAX_STORED_NOTIFICATIONS + 100`
- Also call it after `hydrateNotificationsFromCache` and `hydrateNotificationsFromRelays`

**Acceptance criteria:**
- `notificationsById.size` never exceeds `MAX_STORED_NOTIFICATIONS + 100`
- `postImageCache.size` never exceeds 200
- Filter counts remain accurate after pruning

---

### Phase 2: P1 — Subscription Hygiene & Polling Reduction

**Goal:** Stop the cross-tab event storm and reduce worker message queue saturation.

#### 2.1 Filter `ndkEvent` by `subId` on the page side

**Files:** [`www/feed.html`](www/feed.html), [`www/post.html`](www/post.html), [`www/notifications.html`](www/notifications.html), [`www/js/init-ndk.mjs`](www/js/init-ndk.mjs)

**Current behavior:** The worker includes `subId` in event broadcasts ([`ndk-worker.js:1094`](www/ndk-worker.js:1094)), but pages' `ndkEvent` handlers process all events regardless of source subscription.

**Changes:**
- In `init-ndk.mjs`, the `ndkEvent` CustomEvent already includes the full `message.data` — verify `subId` is in `event.detail`
- Each page tracks its own active subscription IDs in a `Set`:
  - `feed.html`: track the kind-3 sub and the live feed sub
  - `post.html`: track the kind-1 author/event sub
  - `notifications.html`: track the notifications sub
- In the `ndkEvent` handler, check `if (!mySubscriptionIds.has(event.detail.subId)) return;` before processing

**Acceptance criteria:**
- Pages only process events from their own subscriptions
- Events from other tabs' subscriptions are ignored (no CPU work, no DOM updates)

#### 2.2 Close old live subscriptions on follow-list change

**Files:** [`www/feed.html`](www/feed.html)

**Current behavior:** [`ensureLiveFeedSubscription`](www/feed.html:548) creates a new subscription when follows change but never closes the old one.

**Changes:**
- Add a module-level `let liveFeedSub = null;` variable
- In `ensureLiveFeedSubscription`, before creating the new subscription:
  ```js
  if (liveFeedSub?.close) {
    liveFeedSub.close();
  }
  ```
- Assign the return of `subscribe()` to `liveFeedSub`
- Also track the kind-3 subscription similarly and close it on logout

**Acceptance criteria:**
- Only one live feed subscription exists at a time in the worker
- `activeSubscriptions` Map in the worker doesn't accumulate stale feed subscriptions

#### 2.3 Increase footer polling interval to 5 seconds (or event-driven)

**Files:** [`www/index.html`](www/index.html), [`www/feed.html`](www/feed.html), [`www/notifications.html`](www/notifications.html), [`www/relays.html`](www/relays.html)

**Current behavior:** `setInterval(UpdateFooter, 1000)` on every page.

**Changes:**
- Change all 1-second footer intervals to 5 seconds (matching `post.html` which already uses 5s)
- `index.html` line 850: `setInterval(UpdateFooter, 5000)`
- `feed.html` line 692: `setInterval(updateFooter, 5000)`
- `notifications.html` line 1390: `setInterval(UpdateFooter, 5000)`
- `relays.html` line 1794: `setInterval(UpdateFooter, 5000)`
- **Future enhancement (not in this phase):** Make footer fully event-driven by reacting to `ndkRelays` and `ndkRelayActivity` events instead of polling

**Acceptance criteria:**
- No page polls the worker for footer data more than once per 5 seconds
- Footer still shows current relay status (5s staleness is acceptable)

#### 2.4 Diff-update relay table instead of full rebuild

**Files:** [`www/relays.html`](www/relays.html)

**Current behavior:** [`createRelayTable`](www/relays.html:795) does `divRelays.innerHTML = html` every 5 seconds, destroying all rows, inputs, and listeners.

**Changes:**
- Split `createRelayTable` into:
  - `buildRelayTable(relays)` — initial build (full innerHTML)
  - `updateRelayTableRows(relays)` — in-place update of existing rows:
    - Update status icon, read/write counts, connection time text
    - Update checkbox SVGs if relay type changed
    - Add/remove rows if relay count changed
    - Preserve the add-relay input value and focus
- `refreshRelayData` calls `updateRelayTableRows` if the table already exists and the relay count matches; otherwise calls `buildRelayTable`

**Acceptance criteria:**
- Relay table updates without losing input focus or re-attaching all listeners
- Add-relay input value is preserved across refreshes (already partially implemented)

#### 2.5 Throttle `relayActivity` broadcasts in the worker

**Files:** [`www/ndk-worker.js`](www/ndk-worker.js)

**Current behavior:** [`trackRelayRead`](www/ndk-worker.js:828) and [`trackRelayWrite`](www/ndk-worker.js:850) broadcast `relayActivity` on every single event read/write.

**Changes:**
- Add a `relayActivityBroadcastTimer` and a `pendingRelayActivity` Set of relay URLs that have had activity since the last broadcast
- In `trackRelayRead` / `trackRelayWrite`, add the relay URL to `pendingRelayActivity` and schedule a 500ms batched broadcast if not already scheduled
- The batched broadcast sends one `relayActivity` message per pending relay with current stats, then clears the set
- This reduces broadcast frequency from per-event to per-500ms-batch

**Acceptance criteria:**
- `relayActivity` broadcasts are batched to at most once per 500ms
- Stats counts remain accurate (they're cumulative)
- Footer/sidenav relay indicators still update within 1s of activity

---

### Phase 3: P2 — Worker Cleanup & Cache Eviction

**Goal:** Prevent unbounded growth in worker-side state and clean up dead connections.

#### 3.1 Prune `connectedPorts` on tab close

**Files:** [`www/ndk-worker.js`](www/ndk-worker.js)

**Current behavior:** [`connectedPorts`](www/ndk-worker.js:260) accumulates dead ports; [`broadcast()`](www/ndk-worker.js:2479) iterates all of them.

**Changes:**
- In the `onconnect` handler ([`ndk-worker.js:7600`](www/ndk-worker.js:7600)), add:
  ```js
  port.onclose = () => {
      connectedPorts = connectedPorts.filter(p => p !== port);
      if (activeSigningPort === port) activeSigningPort = null;
  };
  ```

**Acceptance criteria:**
- `connectedPorts` only contains live ports
- `broadcast()` doesn't iterate dead ports

#### 3.2 Evict `inlineComposerInstances` on re-render

**Files:** [`www/feed.html`](www/feed.html), [`www/post.html`](www/post.html)

**Current behavior:** `inlineComposerInstances` Map retains entries with detached `hostEl` after `renderFeed()` clears the container.

**Changes:**
- At the start of `renderFeed()` (the full rebuild path), after `divFeed.innerHTML = ''`:
  ```js
  inlineComposerInstances.clear();
  ```
- Since the DOM is rebuilt, all composer host elements are gone; the Map entries are pure garbage

**Acceptance criteria:**
- `inlineComposerInstances` is empty after a full re-render
- Composers can still be re-created on demand via `getOrCreateInlineComposer`

#### 3.3 Prune temporary relay Maps after broadcast publish

**Files:** [`www/ndk-worker.js`](www/ndk-worker.js)

**Current behavior:** `relayActivityStats`, `relayConnectAttemptStartedAt`, `relayLastDisconnectMeta` accumulate entries for every temporary broadcast relay.

**Changes:**
- After the broadcast publish `phase: 'done'` block ([`ndk-worker.js:6410`](www/ndk-worker.js:6410)), add a cleanup step:
  ```js
  const userRelayUrls = new Set(Array.from(relayTypes.keys()).map(normalizeRelayUrl));
  for (const url of relayActivityStats.keys()) {
      if (!userRelayUrls.has(url)) relayActivityStats.delete(url);
  }
  relayConnectAttemptStartedAt.clear(); // only used during active connect attempts
  // Keep relayLastDisconnectMeta for reconnect detection
  ```
- Only prune `relayActivityStats` — the disconnect meta is needed for reconnect subscription rebuilding

**Acceptance criteria:**
- `relayActivityStats` only contains entries for the user's own relays after a broadcast completes
- Reconnect detection still works (disconnect meta preserved)

#### 3.4 Periodic reconciliation of `lightwalletTokenEvents`

**Files:** [`www/ndk-worker.js`](www/ndk-worker.js)

**Current behavior:** `lightwalletTokenEvents` and `lightwalletDeletedIds` grow monotonically; deleted entries are only removed during `publishDirectProofs`.

**Changes:**
- Add a 5-minute interval timer `lightwalletReconcileTimer` that:
  - Iterates `lightwalletTokenEvents` and removes any entry whose id is in `lightwalletDeletedIds`
  - Optionally trims `lightwalletDeletedIds` to the most recent 500 entries (keep recent ones to prevent re-processing)
- Clear this timer in `handleDisconnect`

**Acceptance criteria:**
- `lightwalletTokenEvents` doesn't retain deleted token events
- `lightwalletDeletedIds` is bounded to ~500 entries

#### 3.5 Clear health-check and prune intervals on disconnect

**Files:** [`www/ndk-worker.js`](www/ndk-worker.js)

**Current behavior:** `startRelayHealthCheck` ([`ndk-worker.js:1262`](www/ndk-worker.js:1262)) and `scheduleRelayEventPruning` ([`ndk-worker.js:789`](www/ndk-worker.js:789)) create intervals that are never cleared.

**Changes:**
- Store the health-check interval ID in a module-level variable `relayHealthCheckTimer`
- Store the prune interval ID in `relayEventsPruneTimer` (already exists)
- In `handleDisconnect()` ([`ndk-worker.js:7350`](www/ndk-worker.js:7350)), add:
  ```js
  if (relayHealthCheckTimer) { clearInterval(relayHealthCheckTimer); relayHealthCheckTimer = null; }
  if (relayEventsPruneTimer) { clearInterval(relayEventsPruneTimer); relayEventsPruneTimer = null; }
  ```

**Acceptance criteria:**
- No duplicate intervals after re-init
- Intervals are cleared on disconnect

---

### Phase 4: P3 — Minor Leaks & Polish

**Goal:** Clean up remaining minor leaks and harden against edge cases.

#### 4.1 Clear `refreshTimes` interval on notifications logout

**Files:** [`www/notifications.html`](www/notifications.html)

**Current behavior:** [`setInterval(refreshTimes, 30000)`](www/notifications.html:1392) is never assigned to a variable and can't be cleared.

**Changes:**
- Assign to `let refreshTimesIntervalId = setInterval(refreshTimes, 30000)`
- Clear it in `Logout()`

#### 4.2 Remove duplicate `window.addEventListener('message', ...)` in relays.html

**Files:** [`www/relays.html`](www/relays.html)

**Current behavior:** [`relays.html:1777`](www/relays.html:1777) has a fallback `message` listener that duplicates the `ndkRelayActivity` handler.

**Changes:**
- Remove the `window.addEventListener('message', ...)` block at lines 1777–1783
- The `ndkRelayActivity` handler at line 1762 already covers this

#### 4.3 Add `beforeunload` cleanup for subscriptions on all pages

**Files:** [`www/feed.html`](www/feed.html), [`www/post.html`](www/post.html), [`www/notifications.html`](www/notifications.html)

**Current behavior:** Pages don't close their subscriptions on `beforeunload`, leaving stale entries in the worker's `activeSubscriptions` Map until the worker detects the port close.

**Changes:**
- Add `window.addEventListener('beforeunload', () => { disconnect(); })` (or close individual subscriptions) on each page
- The `disconnect()` function in `init-ndk.mjs` already sends a `disconnect` message and closes the port

**Acceptance criteria:**
- Worker's `activeSubscriptions` is cleaned up when a tab closes

---

## Testing Strategy

### Manual Testing
1. Open `index.html`, `feed.html`, `post.html`, `notifications.html`, `relays.html` in separate tabs
2. Leave them running for 30+ minutes with active subscriptions
3. Check browser DevTools Memory tab:
   - Take heap snapshots at 5min, 15min, 30min
   - Verify heap growth is sublinear (not growing with event count)
4. Check Performance tab:
   - Record a 30s profile while events are flowing
   - Verify no full `renderFeed` / `renderNotifications` on every event
5. Check that relay footer updates within 5s of connection changes
6. Check that notification menu panels still open/close correctly

### Automated Checks
- After each phase, run the existing test suite (`tests/broadcast-relay-test.sh` etc.)
- Add a memory regression test if feasible (puppeteer heap snapshot comparison)

## Risk Assessment

| Phase | Risk | Mitigation |
|-------|------|------------|
| Phase 1 (append-only) | Sort order bugs if new event is older than visible posts | Only prepend if newer than first visible; fall back to full render otherwise |
| Phase 1 (listener fix) | Menu panel might not close in all cases | Test thoroughly; use single delegated listener |
| Phase 2 (subId filter) | Might miss events if subId tracking is wrong | Log filtered vs processed events during testing |
| Phase 2 (polling change) | Footer might feel less responsive | 5s is still fast; relay status rarely changes in seconds |
| Phase 3 (worker cleanup) | Might prune stats too aggressively | Only prune temp relay stats, not user's own relays |

## Files Modified

| File | Phases |
|------|--------|
| `www/feed.html` | 1, 2, 3, 4 |
| `www/post.html` | 1, 2, 3, 4 |
| `www/notifications.html` | 1, 2, 3, 4 |
| `www/relays.html` | 2, 4 |
| `www/index.html` | 2 |
| `www/ndk-worker.js` | 2, 3 |
| `www/js/init-ndk.mjs` | 2 |

## Implementation Checklist

### Phase 1: P0 — Critical Rendering & Listener Leaks

- [x] **1.1** Append-only rendering for feed and post pages
  - [x] Split `renderFeed()` into full rebuild + `prependPostCard()`
  - [x] New posts prepend without rebuilding existing cards
  - [x] Cap visible DOM to `displayCount` (trims last child on overflow)
  - [x] "See More" still does full re-render
  - [x] Verified: `data-post-id` attribute set by `renderPostItem` for trim logic
  - [x] Reviewed: no errors found, fallback paths correct
- [x] **1.2** Append-only rendering for notifications page
  - [x] Split `renderNotifications()` into full rebuild + `prependNotificationRow()`
  - [x] Extracted `buildNotificationRow(item)` helper shared by both functions
  - [x] New notifications prepend without rebuilding existing rows
  - [x] Filter toggles and "View more" still do full re-renders
- [x] **1.3** Fix notification `document.click` listener leak
  - [x] Remove per-row `document.addEventListener('click', ...)`
  - [x] Register single delegated document-level click listener in `main()`
  - [x] Added `notif-menu-panel` class for panel discovery
  - [x] Menu panels still close when clicking outside
- [x] **1.4** Cap `posts[]` and `postIds` with rolling eviction
  - [x] Add `MAX_STORED_POSTS = 500` constant
  - [x] Evict oldest posts when array exceeds cap (+50 batch threshold)
  - [x] Rebuild `postIds` and `renderedPostIds` after eviction
  - [x] Skip pruning in post.html event mode
- [x] **1.5** Cap `notificationsById` / `unreadNotificationsById` with periodic pruning
  - [x] Add `MAX_STORED_NOTIFICATIONS = 500`
  - [x] Add `pruneNotificationMaps()` function
  - [x] Call prune on overflow, after cache/relay hydration
  - [x] LRU-cap `postImageCache` to 200 entries

### Phase 2: P1 — Subscription Hygiene & Polling Reduction

- [ ] **2.1** Filter `ndkEvent` by `subId` on the page side
  - [ ] Verify `subId` is in `event.detail` from init-ndk.mjs
  - [ ] Track active subscription IDs per page
  - [ ] Ignore events from foreign subscriptions
- [ ] **2.2** Close old live subscriptions on follow-list change
  - [ ] Store live feed subscription handle
  - [ ] Close previous subscription before creating new one
- [ ] **2.3** Increase footer polling to 5 seconds
  - [ ] index.html: 1s → 5s
  - [ ] feed.html: 1s → 5s
  - [ ] notifications.html: 1s → 5s
  - [ ] relays.html: 1s → 5s
- [ ] **2.4** Diff-update relay table instead of full rebuild
  - [ ] Split `createRelayTable` into build + update
  - [ ] In-place update of status icons, counts, checkboxes
  - [ ] Preserve add-relay input focus across refreshes
- [ ] **2.5** Throttle `relayActivity` broadcasts in the worker
  - [ ] Add batched broadcast with 500ms timer
  - [ ] Coalesce per-event broadcasts into per-batch
  - [ ] Stats counts remain accurate

### Phase 3: P2 — Worker Cleanup & Cache Eviction

- [ ] **3.1** Prune `connectedPorts` on tab close
  - [ ] Add `port.onclose` handler in `onconnect`
  - [ ] Clear `activeSigningPort` if it was the closed port
- [ ] **3.2** Evict `inlineComposerInstances` on re-render
  - [ ] Clear Map after `innerHTML = ''` in full rebuild path
- [ ] **3.3** Prune temporary relay Maps after broadcast publish
  - [ ] Remove non-user relay entries from `relayActivityStats`
  - [ ] Clear `relayConnectAttemptStartedAt`
  - [ ] Preserve `relayLastDisconnectMeta` for reconnect detection
- [ ] **3.4** Periodic reconciliation of `lightwalletTokenEvents`
  - [ ] Add 5-minute interval timer
  - [ ] Remove deleted entries from `lightwalletTokenEvents`
  - [ ] Trim `lightwalletDeletedIds` to 500
  - [ ] Clear timer in `handleDisconnect`
- [ ] **3.5** Clear health-check and prune intervals on disconnect
  - [ ] Store health-check interval ID
  - [ ] Clear both intervals in `handleDisconnect`

### Phase 4: P3 — Minor Leaks & Polish

- [ ] **4.1** Clear `refreshTimes` interval on notifications logout
- [ ] **4.2** Remove duplicate `message` listener in relays.html
- [ ] **4.3** Add `beforeunload` subscription cleanup on all pages
