Fix permanent zombie relay connections: detect dead transport in check_connection_health
When a relay closes the TCP connection without a clean WebSocket CLOSE frame, the ws client's cached state remains NOSTR_WS_CONNECTED forever (it only transitions on a clean close). Previously: - nostr_ws_ping() send failures were silently ignored (return code unchecked), so the pool never learned the transport was dead - the pong-timeout branch marked the relay DISCONNECTED but left the stale ws_client alive, so ensure_relay_connection() short-circuited on the cached CONNECTED state and never reconnected Result: a single relay-side disconnect left the pool with zero live sockets indefinitely while still reporting all relays connected — all publishes dropped and all inbound events silently missed. Fix: on ping-send failure or pong timeout, close and destroy the stale ws client (ws_client = NULL) and mark the relay DISCONNECTED so the reconnect logic performs a fresh connect. Recovery now takes one ping interval + pong timeout + backoff (~1-2 min) instead of forever. Observed in production on a didactyl agent: relays dropped the connection at 00:08 UTC; the agent ignored all inbound DMs for 13+ hours while /api/status reported 3/3 connected.
This commit is contained in:
@@ -548,11 +548,23 @@ static void check_connection_health(relay_connection_t* relay) {
|
||||
now - relay->last_ping_sent >= relay->pool->reconnect_config.ping_interval_seconds &&
|
||||
!relay->ping_pending) {
|
||||
|
||||
if (nostr_ws_ping(relay->ws_client) == 0) {
|
||||
int ping_rc = nostr_ws_ping(relay->ws_client);
|
||||
if (ping_rc == 0) {
|
||||
relay->last_ping_sent = now;
|
||||
relay->ping_pending = 1;
|
||||
// Store high-resolution start time for latency measurement
|
||||
relay->pending_ping_start_ms = get_current_time_ms();
|
||||
} else {
|
||||
// Ping send failed: the underlying transport is dead (e.g. the
|
||||
// remote closed the TCP connection). The ws client's cached
|
||||
// state may still claim CONNECTED because it only transitions
|
||||
// on a clean WebSocket CLOSE frame. Close the client so the
|
||||
// cached state resets, and mark the relay disconnected so the
|
||||
// reconnect logic performs a fresh connect.
|
||||
nostr_ws_close(relay->ws_client);
|
||||
relay->ws_client = NULL;
|
||||
relay->status = NOSTR_POOL_RELAY_DISCONNECTED;
|
||||
relay->ping_pending = 0;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -560,7 +572,10 @@ static void check_connection_health(relay_connection_t* relay) {
|
||||
if (relay->ping_pending &&
|
||||
now - relay->last_ping_sent > relay->pool->reconnect_config.pong_timeout_seconds) {
|
||||
|
||||
// No pong received - connection is dead
|
||||
// No pong received - connection is dead. Close the client for the
|
||||
// same reason as above: the cached ws state cannot be trusted.
|
||||
nostr_ws_close(relay->ws_client);
|
||||
relay->ws_client = NULL;
|
||||
relay->status = NOSTR_POOL_RELAY_DISCONNECTED;
|
||||
relay->ping_pending = 0;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user