Fix permanent zombie relay connections: detect dead transport in check_connection_health

When a relay closes the TCP connection without a clean WebSocket CLOSE
frame, the ws client's cached state remains NOSTR_WS_CONNECTED forever
(it only transitions on a clean close). Previously:

- nostr_ws_ping() send failures were silently ignored (return code
  unchecked), so the pool never learned the transport was dead
- the pong-timeout branch marked the relay DISCONNECTED but left the
  stale ws_client alive, so ensure_relay_connection() short-circuited
  on the cached CONNECTED state and never reconnected

Result: a single relay-side disconnect left the pool with zero live
sockets indefinitely while still reporting all relays connected —
all publishes dropped and all inbound events silently missed.

Fix: on ping-send failure or pong timeout, close and destroy the stale
ws client (ws_client = NULL) and mark the relay DISCONNECTED so the
reconnect logic performs a fresh connect. Recovery now takes one ping
interval + pong timeout + backoff (~1-2 min) instead of forever.

Observed in production on a didactyl agent: relays dropped the
connection at 00:08 UTC; the agent ignored all inbound DMs for 13+
hours while /api/status reported 3/3 connected.
This commit is contained in:
Laan Tungir
2026-08-27 07:00:41 -04:00
parent c1587ce2a1
commit 14e0dc5b7c
+17 -2
View File
@@ -548,11 +548,23 @@ static void check_connection_health(relay_connection_t* relay) {
now - relay->last_ping_sent >= relay->pool->reconnect_config.ping_interval_seconds &&
!relay->ping_pending) {
if (nostr_ws_ping(relay->ws_client) == 0) {
int ping_rc = nostr_ws_ping(relay->ws_client);
if (ping_rc == 0) {
relay->last_ping_sent = now;
relay->ping_pending = 1;
// Store high-resolution start time for latency measurement
relay->pending_ping_start_ms = get_current_time_ms();
} else {
// Ping send failed: the underlying transport is dead (e.g. the
// remote closed the TCP connection). The ws client's cached
// state may still claim CONNECTED because it only transitions
// on a clean WebSocket CLOSE frame. Close the client so the
// cached state resets, and mark the relay disconnected so the
// reconnect logic performs a fresh connect.
nostr_ws_close(relay->ws_client);
relay->ws_client = NULL;
relay->status = NOSTR_POOL_RELAY_DISCONNECTED;
relay->ping_pending = 0;
}
}
@@ -560,7 +572,10 @@ static void check_connection_health(relay_connection_t* relay) {
if (relay->ping_pending &&
now - relay->last_ping_sent > relay->pool->reconnect_config.pong_timeout_seconds) {
// No pong received - connection is dead
// No pong received - connection is dead. Close the client for the
// same reason as above: the cached ws state cannot be trusted.
nostr_ws_close(relay->ws_client);
relay->ws_client = NULL;
relay->status = NOSTR_POOL_RELAY_DISCONNECTED;
relay->ping_pending = 0;
}