client/plans/relay-disconnect-and-reread-fix.md

Relay Disconnect & Excessive Re-Read Fix Plan

Problem Summary

Two related issues with the local relay (ws://127.0.0.1:7777/):

1. Relay disconnects — The WebSocket connection drops with code 1005 (no status code present) after ~81 seconds of silence.
2. Massive re-reads after reconnect — After reconnecting, hundreds of kind:1 events are re-fetched from the relay, duplicating data already received.

---

Root Cause Analysis

Issue 1: Relay Disconnect

Timeline from logs:

• 13:49:31 — Last batch of RECV events
• 13:50:52 — Disconnect (81 seconds of silence)
• Disconnect code: 1005 (no status code), wasClean: true

Root cause: The local relay at ws://127.0.0.1:7777/ is closing idle WebSocket connections. The NDK keepalive/silence detection in ndk-core.bundle.js uses a 120-second (12e4 ms) NDKRelayKeepalive timer, but the local relay times out before that threshold is reached.

The NDK probe mechanism (which sends {"kinds":[99999],"limit":0}) only fires AFTER silence is detected — it doesn't prevent the disconnect, it only detects it after the fact.

Key code path:

setupMonitoring() → NDKRelayKeepalive(120000ms) → probeRelayConnection()

But the relay disconnects at ~80s, before the 120s keepalive fires.

Issue 2: Massive Re-Reads

Two contributing factors:

Factor A: NDK Subscription Re-Fire on Reconnect

In ndk-core.bundle.js, when a WebSocket reconnects, ALL open subscriptions are re-fired:

const isReconnection = this.reconnectAttempts > 0;
this.reconnectAttempts = 0;
for (const sub of this.openSubs.values()) {
    sub.eosed = false;
    if (isReconnection) {
        for (let f = 0; f < sub.filters.length; f++) {
            if (sub.lastEmitted) {
                sub.filters[f].since = sub.lastEmitted + 1;
            }
        }
    }
    sub.fire();
}

The since filter is only set if sub.lastEmitted exists. If a subscription never received an event (or lastEmitted wasn't tracked), the subscription re-fires with its original filter — fetching ALL matching events again.

Multiple pages create closeOnEose: false subscriptions for kind:1 events:

• post.html — { kinds: [1], authors: [...], limit: 50 } with closeOnEose: false
• post.html — { kinds: [1], authors: [targetPubkey], limit: 50 } with closeOnEose: false
• post.html — { kinds: [1], authors: [...], limit: 50 } with closeOnEose: false
• links.html, template.html, strudel.html — similar patterns

Factor B: Probe Filter and Local Relay limit:0 Handling

The NDK probe sends {"kinds":[99999],"limit":0} via probeRelayConnection. However, the log shows:

[13:52:52.797] CLIENT ➜ RELAY SEND
["REQ","probe-9yft3",{"kinds":[1],"limit":0}]

This shows kinds:[1] not kinds:[99999]. This could be:

1. The relay event log is capturing a different subscription being re-fired (not the probe itself)
2. The probe filter is being modified somewhere

Regardless, the local relay appears to interpret limit:0 as no limit rather than zero results, causing it to dump all matching events.

---

Fix Plan

flowchart TD
    A[Issue 1: Relay Disconnect] --> B[Add WebSocket keepalive pings in worker]
    B --> B1[30s interval sends REQ/CLOSE probe to keep connection alive]
    
    D[Issue 2: Excessive Re-Reads] --> E[Detect reconnect via relayLastDisconnectMeta]
    E --> F[Use disconnect timestamp as since filter]
    F --> F1[Re-create active subscriptions with since = disconnect time]
    
    D --> G[Add event deduplication guard per subscription]
    G --> G1[Track seen event IDs - skip duplicates on broadcast]
    
    D --> H[Track lastEventCreatedAt per subscription as fallback]
    H --> H1[Use max created_at if disconnect meta unavailable]
    
    D --> I[Investigate local relay limit:0 behavior]
    I --> I1[NIP-01 says limit:0 = zero results - relay may be non-compliant]

Fix 1: Prevent Relay Disconnect with Keepalive Pings

File: www/ndk-worker.js

Add a periodic WebSocket ping mechanism in the worker for local/private relays. Since the NDK keepalive is 120s but the relay times out at ~80s, we need to send activity more frequently.

Approach: After relay connect, start a 30-second interval that sends a lightweight REQ/CLOSE probe to keep the connection alive. This is cheaper than modifying the NDK bundle.

// In attachRelayEventListeners() or after relay connect:
function startKeepalive(relay) {
    const KEEPALIVE_INTERVAL = 30000; // 30 seconds
    const intervalId = setInterval(() => {
        if (relay.status !== 5) { // Not connected
            clearInterval(intervalId);
            return;
        }
        // Send a no-op REQ/CLOSE to keep connection alive
        const keepaliveId = `ka-${Date.now()}`;
        try {
            relay.send(JSON.stringify(["REQ", keepaliveId, {"kinds":[99999],"limit":0}]));
            setTimeout(() => {
                try { relay.send(JSON.stringify(["CLOSE", keepaliveId])); } catch(e) {}
            }, 100);
        } catch(e) {
            clearInterval(intervalId);
        }
    }, KEEPALIVE_INTERVAL);
}

Fix 2: Use Disconnect Timestamp as since on Reconnect

File: www/ndk-worker.js

The worker already tracks disconnect timestamps per relay in relayLastDisconnectMeta — a Map of relay.url → { timestamp, code, reason, wasClean }. We can use this to distinguish reconnects from first connects and inject a since filter.

How to tell the difference between first connect and reconnect:

• First connect: No entry in relayLastDisconnectMeta for that relay URL
• Reconnect: Entry exists with a timestamp (milliseconds)

Approach — Reconnect-aware subscription re-creation:

In the relay connect event handler at attachRelayEventListeners():

1. Check if relayLastDisconnectMeta has an entry for this relay → this is a reconnect
2. Get the disconnect timestamp (convert from ms to Nostr seconds: Math.floor(timestamp / 1000))
3. For each active subscription in activeSubscriptions, stop the old NDK sub and create a new one with since = disconnect timestamp injected into the filters
4. Clear the disconnect meta entry after processing

This way, after a reconnect, the relay only sends events created after the disconnect — not the entire history.

// In the relay 'connect' handler:
relay.on('connect', () => {
    const normalized = normalizeRelayUrl(relay.url);
    const disconnectMeta = relayLastDisconnectMeta.get(normalized);
    
    if (disconnectMeta && disconnectMeta.timestamp) {
        // This is a RECONNECT — patch active subscriptions with since filter
        const sinceSec = Math.floor(disconnectMeta.timestamp / 1000);
        console.log(`[Worker] Reconnect detected for ${relay.url}, will use since=${sinceSec}`);
        
        for (const [subId, sub] of activeSubscriptions.entries()) {
            // Stop old subscription
            sub.stop();
            // Re-create with since filter
            const patchedFilters = sub.filters.map(f => ({ ...f, since: sinceSec }));
            const newSub = ndk.subscribe(patchedFilters, sub.opts);
            // Re-attach event handlers...
            activeSubscriptions.set(subId, newSub);
        }
        
        relayLastDisconnectMeta.delete(normalized);
    }
});

Additionally, track lastEventCreatedAt per subscription as a fallback:

• In handleSubscribe(), on each event, record Math.max(lastEventCreatedAt, event.created_at)
• Use this as an alternative since value if disconnect meta is unavailable

Fix 3: Event Deduplication Guard

File: www/ndk-worker.js

Add a per-subscription seen-event-ID set in the worker's event handler to prevent broadcasting duplicate events to pages.

// In handleSubscribe(), add:
const seenEventIds = new Set();

sub.on('event', async (event) => {
    // Dedup guard
    if (seenEventIds.has(event.id)) return;
    seenEventIds.add(event.id);
    // Cap the set size to prevent memory leaks
    if (seenEventIds.size > 5000) {
        const entries = Array.from(seenEventIds);
        for (let i = 0; i < 1000; i++) seenEventIds.delete(entries[i]);
    }
    // ... rest of event handling
});

Fix 4: Investigate Local Relay limit:0 Behavior

The NIP-01 spec says limit:0 should return zero events. If the local relay interprets it as "no limit," this is a relay-side bug. Options:

1. Fix the local relay to respect limit:0 per NIP-01
2. Change the probe to use limit:1 instead of limit:0 in the NDK bundle

Fix 5: Add Reconnect Logging

File: www/ndk-worker.js

Add logging around subscription re-fire to make future debugging easier:

• Log when subscriptions are re-sent after reconnect
• Log whether since filter was applied
• Log count of events received per subscription after reconnect

---

Priority Order

1. Fix 1 (Keepalive) — Prevents the disconnect entirely, which eliminates the cascade
2. Fix 3 (Dedup guard) — Quick safety net against duplicate events regardless of cause
3. Fix 2 (Since filter) — Ensures reconnect re-fetches are incremental
4. Fix 4 (Relay limit:0) — Investigate and fix relay-side behavior
5. Fix 5 (Logging) — Observability for future issues