From d3cf1d6f25314a8033498984fc7402764e03d398 Mon Sep 17 00:00:00 2001 From: Johnathan Corgan Date: Sun, 14 Jun 2026 17:23:44 +0000 Subject: [PATCH] Land v0.4.0 pre-release source content: docs, changelog, packaging Bring the source tree to its finished v0.4.0 content state ahead of the release candidate. Documentation, changelog, release notes, and packaging metadata only; no code or version-string changes. CHANGELOG.md: backfill the operator-visible changes that landed since v0.3.0 (the show_metrics scraper query and fipsctl stats metrics, the discovery dedup-cache-full counter, the off-rx_loop control read surface and its new daemon-resolved fields, the fipstop TUI overhaul, the TCP inbound cap now honoring max_connections, host-map hot-reload, log-noise demotions, and the net bug fixes), topic-grouped rather than replayed per commit; intra-cycle fixes folded into their feature entries; the duplicate Unreleased Fixed section merged into one. Release notes: author docs/releases/release-notes-v0.4.0.md to the operator-upgrade bar and mirror it byte-for-byte into the root RELEASE-NOTES.md. Attribute each feature to its author in Contributors (Nym transport and the mixnet demo to @oleksky, opt-in mDNS LAN discovery to @mmalmi). README.md: add the Nym transport to the support matrix (Linux, macOS, Windows; OpenWrt pending verification), the multi-transport bullet, and the "What works today" list; fold mDNS LAN discovery into the existing Nostr-discovery bullets; refresh the status narrative to v0.4.0 over a global, public test mesh. packaging/common/fips.yaml: add commented Nym transport and mDNS LAN discovery example stanzas with verified field names and defaults. Cargo.toml: add homepage, keywords, and categories crate metadata. docs/reference: update the cli-fips version example to the released form; document the new control-socket output fields and the typed RejectReason families in control-socket.md; sync cli-fipstop.md with the overhauled TUI keybindings. --- CHANGELOG.md | 177 +++-- Cargo.toml | 3 + README.md | 35 +- RELEASE-NOTES.md | 935 +++++++------------------- docs/reference/cli-fips.md | 2 +- docs/reference/cli-fipstop.md | 51 +- docs/reference/control-socket.md | 36 +- docs/releases/release-notes-v0.4.0.md | 279 ++++++++ packaging/common/fips.yaml | 29 + 9 files changed, 774 insertions(+), 773 deletions(-) create mode 100644 docs/releases/release-notes-v0.4.0.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 23dc52d..d3b6137 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,17 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -### Fixed - -- The Tor transport now increments its `connect_refused` statistic (the - "Refused" line in fipstop) when a SOCKS5 connection is actively - refused, instead of recording every connect failure as a generic - SOCKS5 error. The counter previously stayed at zero. -- MMP sender metrics now ignore duplicate or regressed receiver reports - before updating RTT, loss, goodput, or ETX. Receiver reports also - suppress timestamp echo when dwell time overflows, so stale reports - cannot inflate SRTT. - ### Added - Nym mixnet transport (`transports.nym`) for outbound peer links @@ -70,6 +59,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 is updated at every pool-insert and receive-loop-exit site, plus on transport stop and on send-failure-driven removal. Surfaces through `TcpStatsSnapshot` and `TorStatsSnapshot` for `show_transports`. +- `fipsctl stats metrics`, backed by a new counter-only `show_metrics` + control query that dumps the atomic metric registry as flat counter + name/value pairs. Serves a Prometheus-style scraper that samples node + counters without contending the receive loop. +- Discovery now counts `LookupRequest`s dropped when the dedup cache is + full. A saturated `recent_requests` cache + (`MAX_RECENT_DISCOVERY_REQUESTS`) previously dropped requests + silently; a new `DiscoveryStats::req_dedup_cache_full` counter (typed + reject reason `DiscoveryReject::ReqDedupCacheFull`) makes the drop + visible through `show_routing`. - [`PR-REVIEW.md`](PR-REVIEW.md) — the 13-criteria PR review checklist the maintainer runs against every incoming PR, published at the repo root so contributors can run the same pass on their own change @@ -118,6 +117,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 restores headroom toward the fixed-filter capacity limit without materially weakening the antipoison gate: a saturated or poisoned filter is ~100% FPR and still rejected. +- TCP inbound connection cap now honors `node.limits.max_connections`. + The per-transport TCP inbound accept ceiling was hardwired to 256 and + never read `max_connections`, so raising it was a silent no-op for + inbound TCP. The effective cap now resolves with precedence: explicit + per-transport `max_inbound_connections`, then node-wide + `max_connections`, then the built-in default of 256. Established peers + remain bounded node-wide by `add_connection`. +- The control-socket read surface is now served off the `rx_loop`. + Every pure-read `show_*` query — `show_status`, the `show_stats_*` + family, `show_listening_sockets`, the new `show_metrics`, + `show_tree`/`show_bloom`/`show_cache`/`show_routing`/ + `show_identity_cache`, `show_peers`/`show_sessions`/`show_links`/ + `show_connections`/`show_transports`/`show_mmp`, and `show_acl` — now + renders in the control accept task from ArcSwap-published read + snapshots instead of round-tripping the data-plane receive loop; only + the mutating `connect`/`disconnect` commands still reach the loop. + This removes the head-of-line coupling where a busy or slow `rx_loop` + could time out `fipsctl` and `fipstop` observability (the five-second + query pattern operators saw on loaded nodes). Per-entity snapshots + reuse unchanged rows by pointer, so per-tick publish cost stays + bounded as peer/session count grows. New daemon-resolved fields + surface through the snapshots: effective persistence, root/is-root, + and a per-transport-type peer-count map in `show_status`; per-peer + `effective_depth` in `show_peers`; `root_npub` in `show_tree`; and the + last-sent uptree filter fill ratio and subtree estimate in + `show_bloom`. +- `fipstop` TUI overhaul: reworked rendering, navigation model, and the + control read surface it draws from, surfacing the new daemon-resolved + snapshot fields above. Built on a ratatui `TestBackend` + render-snapshot harness that asserts the text grid and per-cell + style of every `ui::draw_*` against canned `show_*` JSON. +- Static host aliases in `/etc/fips/hosts` now hot-reload on mtime + change instead of only at daemon startup, so `fipsctl`/`fipstop` + display names reflect edits without a restart. The peer ACL and host + map both reload once per node tick through a new lock-free + `Reloadable` snapshot. +- Steady-state log noise reduced on saturated public-mesh nodes. + Routine per-peer connection-lifecycle and capacity-cap events are + demoted from info/warn to debug — FMP K-bit cutover promotion, + connection-promoted-to-active-peer (a redundant duplicate + promotion line removed), peer-restart-detected, peer-removed-and- + cleaned-up, the TCP `max_inbound_connections`-reached rejection, and + the congestion-CE-flag line — so genuinely notable info/warn lines + are no longer drowned out. An exhausted FMP-msg1 / FSP-msg3 rekey + retransmission-budget abort (an expected, self-limiting outcome on + lossy or high-latency links) is likewise demoted from warn to debug. - Sidecar example (`examples/sidecar-nostr-relay`): `udp.mtu` is now overridable via the `FIPS_UDP_MTU` environment variable, defaulting to 1472 (preserving prior behavior). Plumbed through `docker-compose.yml` @@ -219,7 +264,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 medians, Linux x86_64, docker-bridge mesh): A→D 1379→2708 Mbps (1.96×), A→E 1394→2663 Mbps (1.91×), E→A 1406→2624 Mbps (1.87×); RTT +0.11–0.19 ms from the worker queue handoff. Windows - continues on the existing tokio-based send/recv path. + continues on the existing tokio-based send/recv path. Two issues in + the off-rx_loop drain path are resolved as part of the overhaul: the + per-peer drain worker is now detached on `Drop` rather than joined + synchronously (a synchronous join from the runtime thread could wedge + the whole daemon when a peer was removed with an in-flight worker), + and the connected-UDP drain no longer busy-spins on a poll error + (#106). - The Debian package no longer ships `/etc/fips/fips.yaml` as a dpkg conf-file. The default configuration is installed as an example at @@ -256,6 +307,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- The Tor transport now increments its `connect_refused` statistic (the + "Refused" line in fipstop) when a SOCKS5 connection is actively + refused, instead of recording every connect failure as a generic + SOCKS5 error. The counter previously stayed at zero. +- MMP sender metrics now ignore duplicate or regressed receiver reports + before updating RTT, loss, goodput, or ETX. Receiver reports also + suppress timestamp echo when dwell time overflows, so stale reports + cannot inflate SRTT. - Six discovery counters (`req_decode_error`, `req_duplicate`, `req_ttl_exhausted`, `resp_decode_error`, `resp_identity_miss`, `resp_proof_failed`) no longer double-count. Each was incremented both @@ -337,25 +396,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 accounting change. Operators with mixed outbound + inbound deployments no longer see legitimate inbound peers rejected once outbound connections fill the pool past the configured cap. -- `PeerRecvDrain::drop` no longer calls `std::thread::join` on the - worker thread. The drain worker uses `packet_tx.blocking_send(...)` - on a tokio mpsc Sender, which internally parks the worker in - `tokio::block_on` on the same `current_thread` runtime that drives - `rx_loop`. Joining synchronously from inside `remove_active_peer` - (which runs on the runtime thread, the runtime's sole driver) - produced a circular wait: rx_loop blocked in libc futex via - `Thread::join`, the worker unable to observe the stop flag because - the runtime that polls it is the very thread now blocked joining - it, and all other peer-drain workers parked on the same runtime - via `block_on`. Full daemon wedge, fipsctl unresponsive, SIGTERM - ignored. Trigger was peer-removal via the 30-s link-dead-timeout - cleanup path with any in-flight worker, with statistical likelihood - amplified by aggressive multi-npub-from-one-NAT reconnect patterns - but not bounded to them. Fix: detach the std::thread (drop the - `JoinHandle` without joining); the stop flag + self-pipe write - already signal the worker to exit; the kernel-level `libc::poll()` - inside the drain loop sees the wake, checks the flag, exits, and - the OS reclaims the thread state independently. - Outbound connection initiation now honors the `node.limits.max_peers` cap that was previously only checked on inbound msg1 admission. Four paths gated: auto-reconnect retries (`process_pending_retries`), @@ -379,13 +419,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 saturation: ~3.6 cap-denials/s × Msg2 (~104 B + AEAD compute) each. Bigger win is cleaner peer-side semantics — no fake-completed handshake whose subsequent data frames fail decryption on this side. -- Mesh-size estimator (`compute_mesh_size`) no longer double-counts the - parent's bloom cardinality during the transient cache window after a - local parent-switch. Symptom: `fipsctl show status` / fipstop displayed - mesh size nearly-but-not-exactly doubling during tree rebalancing. - Fix: explicit parent-skip at the head of the children loop, making the - disjoint-subtree invariant structural rather than dependent on - `peer_declaration` cache freshness. Per-peer 500 ms rate-limiter and +- The mesh-size estimator (`compute_mesh_size`) no longer over-counts + under filter overlap. It previously summed the per-filter cardinality + of the parent and each child filter, which assumes the filters are + perfectly disjoint; a stale or oversized parent filter or a routing + loop inflated the reported mesh size to several times the true value, + and dropping the parent on a tree rebalance collapsed the upward leg + and flapped the count (the symptom operators saw as the size + nearly-but-not-exactly doubling during rebalancing). The estimator now + computes the cardinality of the OR-union over self plus every + connected peer's inbound filter, dropping the parent/child tree gating + entirely. + OR is idempotent, so any overlap is deduplicated — the result equals + the old sum in the disjoint case, stays correct under overlap, damps + the parent-switch flap, and removes the estimate's dependence on + tree-declaration cache freshness. The per-peer 500 ms rate-limiter and overall recompute cadence are unchanged. - Spanning-tree state distribution is now eventually-consistent. Previously every `send_tree_announce_to_all` call site fired only @@ -412,6 +460,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 so any partition healed by the periodic broadcast at T+60 lands inside the convergence window; `wait_for_full_baseline` early-exits on PASS, so successful reps see no extra wall-clock. +- A single-uplink node stranded out of the tree now re-attaches within + a round-trip instead of waiting for the periodic re-broadcast cadence. + A node with one tree peer has periodic parent re-evaluation disabled, + so a lost one-shot attaching `TreeAnnounce` left it self-rooted and + unreachable until the next periodic re-broadcast + (`reeval_interval_secs` later). Tree-position exchange is now + self-healing on the receive path: when an accepted `TreeAnnounce` + advertises a root strictly worse (higher NodeAddr; election is + smallest-wins) than our own, we echo our current declaration back to + that peer, provoking the better-rooted peer to re-push its real + position immediately. The echo fires only in that one direction and is + bounded by the existing per-peer rate limiter. - `rx_loop` tick-arm stall under convergence-phase mesh pressure is eliminated. Previously, the tick body's per-peer `check_*` @@ -583,6 +643,47 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 future caller bypasses the outer dispatch check. As a side benefit, narrows a cooldown-poisoning vector previously available to an attacker injecting stale failure events for an active peer. +- A manual `fipsctl disconnect` now notifies the peer so teardown is + symmetric. Previously a manual disconnect tore down only the local + side and sent the peer nothing, so the peer kept its session and never + re-emitted its tree and filter announcements; on reconnect it was + never re-adopted as a child and its bloom filter was never recorded. + The local side now sends the disconnected peer a scoped `Disconnect` + (the same message graceful shutdown sends), so both ends tear down and + re-handshake cleanly on the next connection. +- `fips-gateway` no longer drops long-lived or DNS-cached client + mappings while traffic is still flowing. The virtual-IP pool's TTL + clock advanced only on DNS re-query, never on traffic, and the mapping + TTL is wired equal to the DNS TTL, so an in-use mapping was forced to + drain at TTL and reclaimed at the first zero-conntrack tick — breaking + long-lived, bursty, or DNS-cached clients. The tick now refreshes the + mapping's last-referenced time whenever conntrack reports active + sessions, and recovers a draining mapping to active (with a fresh + grace window) when traffic resumes; only genuinely idle mappings + drain. +- Transport-layer mutex poisoning no longer cascades. Ten + `Mutex::lock().unwrap()` sites across the UDP, BLE, and Ethernet + transports would turn a single panic (poisoning the mutex) into a + cascade of panics on every subsequent lock. Each is replaced with + `lock().unwrap_or_else(|e| e.into_inner())`, recovering the guarded + data with no new dependency and no call-graph change; four + `local_addr.unwrap()` calls on the UDP start/adopt paths get a + provably-safe sentinel fallback. The critical sections are short, + locally-scoped, and not reachable from peer input, so this is + robustness hardening, not a remotely-triggerable fix. +- `fipstop` no longer renders a garbled screen on startup or leaves + stray bytes on quit, most visible over SSH and inside tmux. Startup + forces a full repaint (`terminal.clear()`) before the first draw so + prior alternate-screen contents no longer show through; quit gives the + stdin-poll thread a stop flag and joins it before restoring the + terminal, so post-raw-mode keystrokes or terminal query responses no + longer echo onto the restored screen. +- macOS `.fips` name resolution now works on a fresh install: the + shipped resolver shim points at `::1`, matching the daemon's default + IPv6 DNS listener, instead of `127.0.0.1`. The mismatched shim + (`nameserver 127.0.0.1` while the daemon listens on `::1`) broke + `getaddrinfo` for `.fips` on every macOS install since the resolver + was introduced. ## [0.3.0] - 2026-05-11 diff --git a/Cargo.toml b/Cargo.toml index 4b09f82..80dfa76 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -6,7 +6,10 @@ description = "A distributed, decentralized network routing protocol for mesh no license = "MIT" authors = ["Johnathan Corgan "] repository = "https://github.com/jmcorgan/fips" +homepage = "https://fips.network" readme = "README.md" +keywords = ["mesh", "p2p", "decentralized", "overlay-network", "nostr"] +categories = ["network-programming", "command-line-utilities", "cryptography"] [dependencies] ratatui = "0.30" diff --git a/README.md b/README.md index 66e69aa..66c3d61 100644 --- a/README.md +++ b/README.md @@ -40,8 +40,8 @@ same way it would on a local network. - **Self-organizing mesh routing.** Spanning-tree coordinates with bloom-filter-guided discovery; no global routing tables, no flooding. -- **Multi-transport.** UDP, TCP, Ethernet, Tor, and Bluetooth (BLE - L2CAP) ship today; transports compose on a single mesh and a +- **Multi-transport.** UDP, TCP, Ethernet, Tor, Nym, and Bluetooth + (BLE L2CAP) ship today; transports compose on a single mesh and a node may run several at once. - **Two-layer encryption.** Noise IK between peers (hop-by-hop) and Noise XK between mesh endpoints (independent end-to-end), with @@ -55,7 +55,8 @@ same way it would on a local network. - **Nostr-mediated discovery and NAT traversal.** Peers publish endpoint adverts on public Nostr relays, exchange candidates via NIP-59 gift-wrapped offers and answers, and establish direct - paths through NATs using STUN-assisted hole punching. + paths through NATs using STUN-assisted hole punching. On the local + network, mDNS LAN discovery finds peers directly without relays. - **LAN gateway.** Optional `fips-gateway` service folds an entire unmodified LAN into the mesh: outbound (LAN clients reach mesh destinations through a DNS-allocated virtual IPv6 pool and @@ -120,6 +121,7 @@ supported; transport availability varies by platform. | TCP | ✅ | ✅ | ✅ | ✅ | | Ethernet | ✅ | ✅ | ❌ | ✅ | | Tor | ✅ | ✅ | ✅ | ✅ | +| Nym | ✅ | ✅ | ✅ | ❌ | | BLE | ✅ | ❌ | ❌ | ❌ | On Linux, a source build requires `libclang` — the LAN gateway's @@ -136,6 +138,11 @@ gated on a build-script probe — install the dependencies first and the `cargo build` line above picks it up. The OpenWrt ipk omits BLE because libdbus is not available on the target. +Nym (mixnet) transport builds on all desktop platforms. The OpenWrt +❌ is provisional, pending verification of `nym-socks5-client` +availability on the target; it will flip to ✅ only if confirmed +buildable there. + ## Documentation `docs/` is organised by reader purpose: @@ -197,14 +204,14 @@ testing/ Docker-based integration test harnesses + chaos simulation ## Status & roadmap -FIPS is at **v0.3.0**. The core protocol works end-to-end over -UDP, TCP, Ethernet, Tor, and Bluetooth on a small live mesh of -deployed nodes. v0.3.0 is the testing-and-polishing track for -everything accumulated since v0.2.0 on the v0.2.x wire format — -Nostr-mediated peer discovery, UDP NAT traversal, peer ACL, the -DNS-responder fix, packaging hardening, and discovery rate-limit -retuning. New wire-format work is staged on the `next` branch for -the post-v0.3.0 release line. +FIPS is at **v0.4.0**. The core protocol works end-to-end over +UDP, TCP, Ethernet, Tor, Nym, and Bluetooth on a global, public test +mesh of thousands of nodes. v0.4.0 builds on the v0.3.0 testing-and-polishing +track, adding the Nym mixnet transport and mDNS LAN discovery +alongside the existing Nostr-mediated peer discovery, UDP NAT +traversal, peer ACL, and packaging hardening. New wire-format work +continues to be staged on the `next` branch for the subsequent +release line. ### What works today @@ -222,10 +229,10 @@ the post-v0.3.0 release line. estimation. - ECN congestion signaling (hop-by-hop CE relay, IPv6 CE marking, kernel-drop detection). -- UDP, TCP, Ethernet, Tor, and BLE transports (BLE via L2CAP CoC - with per-link MTU negotiation). +- UDP, TCP, Ethernet, Tor, Nym (mixnet), and BLE transports (BLE + via L2CAP CoC with per-link MTU negotiation). - Nostr-mediated overlay endpoint discovery and UDP hole punching - for NAT traversal. + for NAT traversal, plus mDNS LAN discovery for local peers. - LAN gateway (`fips-gateway`) with both outbound (LAN-to-mesh) and inbound (mesh-to-LAN port-forwarding) modes. - Peer ACL: per-npub allow / deny admission control at the link diff --git a/RELEASE-NOTES.md b/RELEASE-NOTES.md index fe533e9..0d62767 100644 --- a/RELEASE-NOTES.md +++ b/RELEASE-NOTES.md @@ -1,696 +1,262 @@ -# FIPS v0.3.0 +# FIPS v0.4.0 -**Released**: 2026-05-11 +**Released**: 2026-06-DD (provisional) -v0.3.0 is the testing-and-polishing release on the v0.2.x wire format. -It widens the platform reach of FIPS from Linux-only to Linux, macOS, -Windows, and OpenWrt; adds two large new mesh capabilities (Nostr-mediated -peer discovery with UDP NAT traversal, and the `fips-gateway` LAN bridge); -ships a default-deny security baseline for the mesh interface; introduces -mesh-peer access control; substantially speeds up session-layer crypto and -the Linux receive path; and tightens packaging across every supported -distribution channel. +v0.4.0 is the throughput-and-observability release on the v0.3.x wire +format. It adds two new ways for nodes to find and reach each other (the +Nym mixnet transport and opt-in mDNS LAN discovery), overhauls the data +plane for higher single-node throughput and lower per-packet CPU, moves +the entire operator read surface off the data-plane hot path so +observability stays responsive under load, ships a reworked `fipstop` +TUI, and hardens FMP and FSP rekey to be hitless under packet loss in +both directions. It also folds in the accumulated mesh-convergence, +admission-control, and packaging fixes from the maintenance line. -v0.3.0 is wire-compatible with v0.2.x. Mixed meshes interoperate; there -is no flag-day upgrade. - -v0.3.0 also rolls forward all changes from the v0.2.1 maintenance -release. The sections below cover the cumulative v0.2.0 → v0.3.0 -delta; the per-section intros call out which entries first shipped -in v0.2.1. +v0.4.0 is wire-compatible with v0.3.0. Mixed meshes interoperate; there +is no flag-day upgrade. A deployed v0.3.0 node and an upgraded v0.4.0 +node peer, rekey, and route normally, so you can roll the upgrade out +across a mesh in any order. ## At a glance -- 123 commits since v0.2.0 (109 non-merge), spanning 307 files with - +44,186 / -4,078 lines. -- 10 committers plus 3 issue reporters across feature work, fixes, - packaging, and reviews. -- 5 new GitHub Actions CI workflows (Linux Package, macOS Package, - Windows Package, OpenWrt Package, AUR Publish) plus expanded - integration matrices (gateway, NAT-cone, NAT-symmetric, NAT-LAN, - rekey-accept-off, `.deb` install across Debian 12/13 + Ubuntu - 22/24/26, multi-backend `.fips` DNS resolver across the same five - distros). -- The long-standing systemd-resolved DNS-responder silent-drop is - closed end-to-end. -- Pre-1.0 control-socket JSON schema change for two query fields; - see [Upgrade notes](#upgrade-notes). +- New outbound Nym mixnet transport with a single-container demo and a + new mixnet-relay example. +- Opt-in mDNS / DNS-SD discovery on the local link. +- Data-plane overhaul: off-task encrypt and decrypt worker pools, GSO, + connected-UDP send path, copy-avoidance on receive, batched macOS + receive. +- The full `show_*` read surface now serves off the receive loop, so + `fipsctl` and `fipstop` stay responsive on loaded nodes; a new + counter-only `show_metrics` query enables a Prometheus scraper at no + hot-path cost. +- Reworked `fipstop` TUI on a machine-verified render-snapshot base. +- Rekey is now hitless under loss and reordering in both directions. ## What's new -### Mesh discovery and NAT traversal +### Nym mixnet transport -Previously, two FIPS nodes could only become peers if they had a way -to find each other beforehand: a configured address, a shared LAN -segment, or a Bluetooth radio range. v0.3.0 introduces a Nostr-based -overlay-discovery channel that lets nodes find each other through any -public Nostr relay set, plus a STUN-assisted UDP hole-punching path -that connects peers across most consumer NATs. +FIPS can now peer over the [Nym](https://nymtech.net/) mixnet for +metadata-resistant connectivity. The new `transports.nym` transport +makes outbound connections through a `nym-socks5-client` SOCKS5 proxy +that you run alongside the daemon (for example as a service running +alongside the fips daemon, or as a sidecar container). The transport +waits at startup for the nym-socks5-client to become ready before giving +up. -Each participating node publishes a signed overlay advert as a Nostr -**Kind 37195** parameterized replaceable event. (The kind sits in the -application-defined replaceable range and the digits visually spell -*FIPS*: 7=F, 1=I, 9=P, 5=S.) The advert lists reachable transport -endpoints (UDP, TCP, Tor) and is consumed by other nodes to populate -fallback addresses for `via_nostr` peers. Under `policy: open`, the -advert cache is also dialed for non-configured peers within a budget -cap. +This is a privacy and anonymity deployment mode chosen for its own +properties. It mixes your FIPS traffic into the Nym cover-traffic +network so that link-level observers cannot correlate which mesh peers +are talking. A new `examples/sidecar-nostr-mixnet-relay/` demonstrates a +FIPS-reachable Nostr relay peered across the mixnet end to end, and a +single-container demo ships with the transport. -When both peers are behind NAT, the daemon coordinates a UDP hole -punch using NIP-59 gift-wrap signaling for the offer/answer exchange -and STUN for reflexive address discovery. A candidate-pair punch -planner attempts LAN-private and reflexive paths in parallel; on -success the live socket is handed into the standard FIPS UDP transport -via a bootstrap-handoff API. +Enable it by adding a `transports.nym` instance and pointing it at your +running nym-socks5-client. See the transports reference for the field +set. -Operators turn this on with `node.discovery.nostr.enabled: true` and -the configured relay set. `policy: open` adds best-effort dialing of -non-configured peers seen on the relays. New `peers[].via_nostr` and -per-transport `advertise_on_nostr` / `public` flags control what each -endpoint contributes to the published advert. Cross-field validation -runs at startup to catch mis-configured combinations early. +### mDNS LAN discovery -A Docker NAT lab covering cone, symmetric, and LAN scenarios is wired -into the integration CI matrix. A daemon-side failure-suppression -layer (per-npub cooldown after consecutive failures, ±60s clock-skew -tolerance, rate-limited WARN logs) keeps relay traffic well-mannered -when peers come and go from the open discovery cache. A separate -structural cooldown (`protocol_mismatch_cooldown_secs`, default 24h) -suppresses retraversal when a punched peer turns out to be running an -FMP version this daemon cannot handshake with: the punch completes at -the UDP layer, the rx loop spots the version-mismatched packet, -reverse-maps to the originating npub, and removes the peer from the -next sweep until either side upgrades. +Nodes on a shared local link can now find each other with zero address +configuration. The opt-in `node.discovery.lan` path runs an mDNS / +DNS-SD responder and browser: each node advertises a FIPS service record +on the link and adopts the peers it discovers. This complements the +existing Nostr-mediated overlay discovery for the common case where the +peers are simply on the same LAN. -The auto-connect retry loop pins itself to relay ground truth. Each -retry attempt refetches the cached overlay advert against the -configured `advert_relays` (one filter query, 2s timeout) before -dialing, so a peer whose NAT rebound to a fresh endpoint is recovered -on the next retry rather than looping on a stale cached address. -`NoTransportForType` triggers a fire-and-forget re-fetch that either -replaces or evicts the cache entry. A startup peer-init failure (no -operational transport, all addresses unreachable) now schedules a -retry instead of leaving the peer in a dead state until the daemon is -restarted. Adopted NAT-traversed UDP transports inherit the operator's -primary `[transports.udp]` listener config (MTU, recv/send buffer -sizes) instead of falling back to the 1280 IPv6-minimum default. +Turn it on with `node.discovery.lan.enabled: true`. `service_type` and +`scope` tune the advertised service record and which interfaces +participate. Discovery on the local link needs no relay and no STUN. -### Cross-platform reach +### Data-plane throughput overhaul -FIPS now ships first-class binaries for **Linux, macOS, Windows, and -OpenWrt**. +The receive and send paths were reworked for higher single-node +throughput and lower per-packet CPU, building on the v0.3.0 +crypto-backend swap: -- **macOS** support uses the native `utun` TUN interface, raw - Ethernet via BPF, a `.pkg` installer with a launchd plist and - uninstall script, and an x86_64 cross-compile from arm64 build - hosts. A new CI matrix entry runs build and unit-test jobs on - macOS hosts. -- **Windows** support uses [wintun](https://www.wintun.net/) for the - TUN device, a TCP control socket on `localhost:21210` (replacing - the Unix domain socket Linux and macOS use), Windows Service - lifecycle (`fips.exe --install-service`, `--uninstall-service`, - `--service`), and a ZIP package with PowerShell install/uninstall - scripts. -- **MIPS** atomic-ABI portability lets the daemon build for 32-bit - MIPS targets (`mips`, `mipsel`, MIPS32r2) by routing through - `portable_atomic`. This unblocks OpenWrt deployments on - consumer-grade MIPS routers. -- **OpenWrt** packaging gets a procd init with dnsmasq forwarding, - proxy NDP, RA route advertisements, and IPv6 forwarding sysctls. - The `fips-gateway` is enabled by default in the OpenWrt build. +- **Off-task encrypt and decrypt.** Per-peer encrypt and decrypt now run + on dedicated worker tasks rather than inline on the receive loop, so a + single busy peer no longer serializes the whole node's crypto. +- **GSO and connected-UDP send.** The Linux send path uses generic + segmentation offload and a connected-UDP socket where available, + cutting syscall overhead on bulk flows. +- **Copy-avoidance on receive.** The receive hot path avoids buffer + copies it previously made per packet. +- **Batched macOS receive.** macOS gains a `recvmsg_x` batched receive, + mirroring the Linux `recvmmsg` batching from v0.3.0. +- **Shared immutable-state context and an atomic metric registry.** + Immutable per-node state moved into a single shared context, and + counters live in an atomic metric registry that the new `show_metrics` + query reads without touching the hot path. -### FIPS gateway +These are all internal to the data plane and require no operator action. -The new `fips-gateway` binary lets unmodified LAN hosts reach FIPS -mesh destinations without running the FIPS daemon themselves. Two -flows ship together: +### Observability off the hot path -- **Outbound (LAN -> mesh)**: a virtual-IP pool (default - `fd01::/112`) is allocated on demand from `.fips`-name DNS lookups. - A state-machine lifecycle, conntrack-backed session tracking, proxy - NDP on the LAN interface, and TTL-based reclamation handle the - bookkeeping. A LAN host that resolves `peer.fips` gets a virtual - address it can reach over IP, and the gateway translates the flow - to the mesh. -- **Inbound (mesh -> LAN)**: new `gateway.port_forwards` config - installs prerouting DNAT rules so mesh peers can reach a configured - `host:port` on the gateway's LAN. A LAN-side masquerade is added - automatically when any forwards are configured, so replies flow - back through conntrack. +Every read-only control query now renders from a snapshot published once +per tick into a lock-free `ArcSwap`, served from the control accept task +instead of round-tripping the data-plane receive loop. This covers +`show_status`, `show_stats_*`, `show_peers`, `show_sessions`, +`show_links`, `show_connections`, `show_transports`, `show_mmp`, +`show_tree`, `show_bloom`, `show_cache`, `show_routing`, +`show_identity_cache`, `show_acl`, `show_listening_sockets`, and the new +`show_metrics`. Only the mutating `connect` and `disconnect` commands +still reach the loop. -A dedicated control socket at `/run/fips/gateway.sock` exposes -`show_gateway` and `show_mappings`. `fipstop` adds a Gateway tab with -a pool gauge and mappings table. +The practical effect: on a loaded node where the receive loop was busy, +`fipsctl` and `fipstop` queries previously stalled or timed out (the +five-second query pattern operators saw). They now answer promptly +regardless of data-plane load. Per-entity snapshots reuse unchanged rows +by pointer, so the per-tick publish cost stays bounded as peer and +session counts grow. -The gateway's `dns.listen` source default is now `[::1]:5353`, -matching the canonical deployment model: the gateway sits on a host -already serving DHCP and DNS to a LAN segment (an OpenWrt AP, a Linux -router), port 53 there is taken by the existing resolver, and `.fips` -queries are forwarded to the gateway over loopback. The OpenWrt ipk -previously overrode the prior `[::]:53` source default in its packaged -config; that override is now redundant and has been dropped. -Operators on a host without a pre-existing resolver on port 53 can -opt back into the wildcard bind by setting `dns.listen: "[::]:53"` -explicitly. The new default binds IPv6 loopback only, so forwarders -that reach the gateway over IPv4 loopback need an explicit IPv4 -listen address. +A new **`show_metrics`** query (surfaced as `fipsctl stats metrics`) +returns a counter-only snapshot of every metric family. It is the +enabler for a Prometheus scraper that pulls node counters at no hot-path +cost. -The cold-boot startup race between `fips.service` and -`fips-gateway.service` is handled by a systemd `After=fips.service` -ordering, an `ExecStartPre` poll loop that waits up to 30 seconds for -the `fips0` interface to appear, and a DNS upstream probe in the -gateway itself that retries up to 5 times with 1-second backoff. +### Reworked fipstop TUI -Packaging covers systemd, Debian, AUR, and OpenWrt. The full design -is in [`docs/design/fips-gateway.md`](../design/fips-gateway.md). +`fipstop` gets a rendering, navigation, and read-surface overhaul on a +machine-verified base: a render-snapshot harness asserts the exact text +grid and per-cell style of every view against canned control-socket +output. New daemon-resolved fields surface through the snapshots, +including effective persistence, root and is-root state, a +per-transport-type peer-count map, per-peer effective depth, the root +npub, and the last-sent uptree filter fill ratio with the subtree size +estimate. -### Mesh-interface security baseline +A separate fix clears a garbled-screen problem on startup and stray +bytes on quit, most visible over SSH and inside tmux: startup now forces +a full repaint before the first draw, and quit stops and joins the +stdin-poll thread before restoring the terminal, so post-raw-mode +keystrokes no longer echo onto the restored screen. -The FIPS mesh is a flat layer-3 segment. Every authenticated peer can -route packets to every other peer's `fips0` address. Peer identity is -authenticated end-to-end by the FMP and FSP Noise handshakes, but -identity is not authorization. A service on a mesh host that binds to -a wildcard address is, by default, reachable from every peer in the -mesh. +### Rekey reliability -v0.3.0 ships an opt-in default-deny baseline that closes this gap on -Linux: +FMP and FSP session rekey are now hitless under packet loss and +reordering in both directions: -- **`/etc/fips/fips.nft`** is installed as a documented operator - conffile. It defines a single `inet fips` nftables table with one - chain hooked at `input`, default-denies inbound traffic on - `fips0`, and is a no-op for every other interface. -- **`fips-firewall.service`** loads it. The unit ships **disabled by - default**; activation is an explicit - `systemctl enable --now fips-firewall.service`. -- Per-service allowances live in **`/etc/fips/fips.d/*.nft`** - drop-ins that the baseline includes. +- Inbound frames are authenticated against the pending session before + the K-bit cutover promotes it, so a spoofed or stale frame cannot + derail a rekey in progress. +- Rekey message-1 retransmission is bounded, and the link-dead heartbeat + is rekey-aware so an in-flight rekey is not mistaken for a dead link. +- FSP session rekey holds connectivity across the rekey window under + loss and reordering. +- Dual-initiation races (both peers starting a rekey at once on a + high-latency link) are desynchronized with symmetric jitter so the two + sides converge on one session rather than fighting. +- An exhausted retransmission-budget abort, an expected and self-limiting + outcome on lossy or high-latency links, is logged at debug rather than + warn. -Choosing opt-in keeps the mesh quick to bring up for evaluation while -giving operators a documented, packaged path to lock it down for -production. The full design (threat model, rule layout, conntrack -handling, drop-in mechanism, and the rationale for a conffile rather -than an auto-loaded package side-effect) is in -[`docs/design/fips-security.md`](../design/fips-security.md). - -`fipstop`'s Node tab gains a **"Listening on fips0" panel** that -surfaces the answer to the operational question "what services on -this host are reachable from the mesh, and what does the firewall -currently say about each of them?" The panel lists every IPv6 -listening socket bound to either the wildcard address or this node's -`fd00::/8` address, paired with its classification against the -running `inet fips` baseline chain: `OPEN` (canonical accept rule), -`filt` (falls through to drop), or `filt?` (referenced with matchers -the panel cannot fully decompose, e.g. saddr filters or jumps). When -`fips-firewall.service` is inactive, a yellow banner above the table -reminds the operator that every listener is mesh-exposed; wildcard -binds carry a trailing `*` in the Process column. The classifier is -built on a new `show_listening_sockets` control query (Linux-only), -which is also useful from `fipsctl` for scripting. - -### Peer access control - -Operators can now restrict which mesh peers a node will form direct -links with. Optional `/etc/fips/peers.allow` and `/etc/fips/peers.deny` -files (TCP-Wrappers style) match against npub, hex pubkey, host -alias, or `ALL`. Enforcement runs at three points: - -1. Outbound connect (before dialing). -2. Inbound msg1 (the first FMP handshake message from a new peer). -3. Outbound msg2 (the response). - -Files reload automatically on mtime change; a new `fipsctl acl show` -query reports the effective rule set. A six-node Docker integration -harness (`testing/acl/`) exercises allowlist and denylist patterns -end-to-end. - -**Important scope distinction**: peer ACLs are an FMP-layer -restriction. They control who can establish a *direct link* with this -node. They do **not** control session-layer (FSP) reachability through -the mesh. A node that denies peer X with an ACL can still receive FSP -traffic from X relayed via other peers. - -### Bluetooth Low Energy transport (experimental, Linux) - -A new BLE L2CAP Connection-Oriented Channel transport lets FIPS nodes -peer over Bluetooth Low Energy without any IP infrastructure in -between. The transport handles per-link MTU negotiation, continuous -scan/probe peer discovery with cooldown-based deduplication, -continuous advertising, deterministic NodeAddr cross-probe -tie-breaker, and a configurable connection pool with eviction. - -This transport is **experimental in v0.3.0**. It is implemented and -functional on Linux (BlueZ via `bluer`), but the reliability follow-up -logic (probe cooldown, cross-probe tie-breaker, pubkey timeout, -continuous advertising semantics, probe-promotion, fail-fast send) is -not yet behaviorally tested in CI. Its maturity path is field-driven; -please file issues with field reports. macOS BLE support is in -development as a separate track and is not part of v0.3.0. - -### UDP transport profiles - -The UDP transport gains posture flags organized around deployment -patterns: - -- **Public-facing inbound nodes**: `bind_addr: "0.0.0.0:2121"`, - `accept_connections: true` (default), `public: true` for advert - publication. v0.3.0 adds STUN-based public-IP autodiscovery so - cloud nodes (AWS EIP, GCP, Azure 1:1 NAT) advertise the right - address even when the public IP isn't on a host interface. -- **Ephemeral leaf nodes**: `outbound_only: true` binds an ephemeral - port (`0.0.0.0:0`), refuses inbound msg1, and is never advertised - on Nostr regardless of `advertise_on_nostr`. Use this for client - postures that should connect outbound only, without exposing an - inbound listener on a known port. -- **General-purpose nodes**: `accept_connections: false` mirrors the - Ethernet/BLE knob without changing the bind address. The Node-level - handshake gate carves out msg1 from peers already established on - this transport so rekey continues to work. - -Startup validation now rejects `bind_addr` set to a loopback address -when at least one peer has a non-loopback UDP address, closing a -silent-failure trap from v0.2.0 where Linux's source-address routing -check would drop outbound flows from the loopback-bound socket. - -A new `external_addr` field on `transports.udp.*` and -`transports.tcp.*` lets operators specify the advertise-as address -explicitly. This is useful for UDP as a deterministic alternative to -STUN, and required for TCP on cloud-NAT setups (where binding to the -public IP fails with `EADDRNOTAVAIL` because the IP isn't on a host -interface). - -### `.fips` DNS resolver overhaul - -The IPv6 adapter's `.fips` name resolution has been rebuilt around -the constraints of contemporary systemd-based hosts. The default -`dns.bind_addr` is now `::1` (IPv6 loopback), and a setup script -picks one of five backends in priority order: - -1. systemd-resolved global drop-in - (`/etc/systemd/resolved.conf.d/fips.conf`). -2. systemd dns-delegate (per-link configuration handed off to - systemd-resolved). -3. `resolvectl` per-link configuration. -4. Standalone `dnsmasq`. -5. NetworkManager's dnsmasq plugin. - -Teardown reverses only what setup applied, recorded in a state file -at `/run/fips/dns-backend`. A new `testing/dns-resolver/` harness -exercises every backend across Debian 12, Debian 13, Ubuntu 22.04, -Ubuntu 24.04, and Ubuntu 26.04, so a regression in any of the five -backends shows up in CI rather than in the field. - -This overhaul resolves the long-standing silent-drop case where the -`resolvectl dns fips0 []:5354` target collided with the -daemon's mesh-interface filter on certain systemd-resolved -deployments (typically Ubuntu 22 with systemd 249's interface-scoped -routing). - -### Operator tooling additions - -A handful of additions land in `fipsctl`, `fipstop`, and the daemon's -configuration surface: - -- **`node.log_level`** config field replaces the hardcoded - `RUST_LOG=info` previously baked into systemd units and the - OpenWrt procd init. The daemon now loads config before - initializing tracing so the configured level takes effect. - `RUST_LOG` still overrides when set. -- **`fipsctl show identity-cache`** is a new query that lists every - cached node identity (npub, IPv6 address, display name, LRU age) - alongside the configured cache capacity. -- **`fipsctl show peers / sessions / cache / routing`** are - substantially extended: per-peer security signals (replay - suppression count, consecutive decrypt failures), Noise session - counters, session indices, rekey lifecycle state, handshake resend - counts, K-bit epoch, coords-warmup remaining, drain state, per-peer - retry state, per-target lookup detail (attempt, age, last sent), - and pending TUN packet queue depth. -- **Historical statistics**: in-memory time-series rings on the - daemon (1-second × 3600 fast, 1-minute × 1440 slow) cover per-node - and per-peer metrics. New `show_stats_*` control-socket queries, a - `fipsctl stats list / peers / history` subcommand with Unicode - sparkline rendering, and a `fipstop` Graphs tab with btop-style - sparklines surface them to the operator. - -### Performance - -Two independent perf threads land in v0.3.0: a session-layer crypto -backend swap, and a Linux receive-path overhaul. - -**Session-layer crypto backend.** The ChaCha20-Poly1305 backend used -by every FIPS Noise session (end-to-end FSP traffic and link-layer -FMP traffic alike) has been swapped from RustCrypto's -`chacha20poly1305` crate to `ring 0.17`. ring wraps BoringSSL's -hand-tuned ChaCha20-Poly1305 implementation, which dispatches to NEON -on aarch64 and AVX2 / AVX-512 on x86_64. Typical throughput is in the -3-5 GB/s/core range, versus the ~600-800 MB/s/core RustCrypto soft -path on the same hardware. - -Wire format is unchanged. ChaCha20-Poly1305 is byte-deterministic for -a given `(key, nonce, plaintext, aad)`, so any correct AEAD -implementation produces identical ciphertext. A mixed mesh with some -nodes pre-swap and some post-swap interoperates without protocol -awareness; v0.3.0 can roll out across a mesh in any order. - -Measurements on an aarch64 Apple Silicon docker target: - -- Two-node TCP single-stream: 437 -> 1097 Mbps (about 2.5×). -- Two-node UDP at 1000 Mbit: 599 Mbps with 40% loss -> lossless at - line rate. -- Three-node ping under bulk-traffic load: 7.68 ms avg / 215 ms max - -> 0.72 ms / 3.6 ms max as the relay path stops being crypto-bound. - -No operator-visible action is required; the swap is internal to the -session layer. - -**Linux UDP receive path.** The Linux UDP receive path now uses -`recvmmsg(2)` with a 32-packet batch in place of single-packet -`recvmsg(2)`. A single `readable()` wakeup drains up to 32 datagrams -in one syscall before yielding back to the reactor, eliminating the -per-packet scheduler-hop and futex cost that previously capped -inbound rate at one event per scheduler quantum independent of CPU. -`SO_RXQ_OVFL` is sampled once per batch and surfaced through -`AsyncUdpSocket::recv_batch` so the existing 1Hz transport-congestion -detector continues to feed the per-transport `dropping` flag. macOS -and Windows fall through to the per-packet path; `recvmmsg` is -Linux-specific. - -**Inner rx-loop drain batching.** `Node::run_rx_loop` drains up to -256 additional ready items via `try_recv()` after each -`tokio::select!` await fires on the packet and TUN-outbound branches, -in a tight inner loop before yielding. Previously the select cost a -full scheduler hop and futex per packet, capping throughput at one -event per scheduler quantum with the worker near-idle. `biased` -ordering keeps data-plane branches priority over tick / control / DNS -under sustained load; the 256 cap keeps the worker on a busy stream -between yield points (about 400 KB of contiguous traffic) while still -bounding the inner loop so a flood on one branch cannot starve the -periodic tick or control socket. - -**Eager `pubkey_full` precompute.** `PeerIdentity::pubkey_full()` -precomputes the parity-aware full secp256k1 public key at -construction in `from_pubkey`. Previously the method fell through to -an EC point parse on every call when the full key wasn't passed at -construction (i.e. for every peer constructed from an npub or x-only -key), about 6% of per-packet CPU on the bulk-data send path for a -value that never changed after construction. The same parse already -runs at construction inside `NodeAddr::from_pubkey`, so the cost is -paid once where it would be paid anyway. - -These three changes are a coordinated set: the syscall batching -removes the per-packet kernel cost, the inner-loop drain removes the -per-packet scheduler cost, and the pubkey-cache change removes the -per-packet crypto-derivation cost. Like the AEAD swap, they are all -internal and require no operator action. - -### Examples - -- **macOS WireGuard companion** ([#51](https://github.com/jmcorgan/fips/pull/51)): - run FIPS in a local Docker container and route `.fips` traffic - from the macOS host through a WireGuard tunnel to the container's - `fips0`. Only traffic destined for `fd00::/8` transits the - companion; regular internet traffic continues to use the host - network. Persistent FIPS and WireGuard key material is generated - on first run. - -### Documentation - -- **`docs/design/port-advertisement-and-nat-traversal.md`** - documents how nodes find each other through Nostr relays and the - STUN-assisted UDP hole punch. -- **`docs/design/fips-gateway.md`** documents the gateway's virtual - IP pool, lifecycle, control surface, and packaging. -- **`docs/design/fips-security.md`** documents the mesh-interface - security posture, threat model, default-deny baseline, and drop-in - workflow. -- **`CONTRIBUTING.md`** has been expanded with build prerequisites, - Rust toolchain setup, and first-build steps. - -The `docs/` tree has been reorganized end-to-end into four sections -(*tutorials / how-to / reference / design*) with a new -[`docs/getting-started.md`](../getting-started.md) and per-section -landing pages. Content was reconciled against current source: -protocol-layer details, wire-format diagrams, configuration knobs, -and CLI references were brought back into agreement with the -implementation. See [Documentation pointers](#documentation-pointers) -below for entry points by reader intent. +The net operator takeaway: rekey completes cleanly without dropping +traffic, even on lossy or high-latency links, and the log no longer +cries wolf when a rekey gives up and retries. ## Behavior changes worth flagging -These default-config changes affect every operator on upgrade, even -those with no explicit configuration. Two items below — bloom-filter -fill-ratio validation and TreeAnnounce ancestry validation — first -shipped in v0.2.1 and roll forward into v0.3.0; the rest are -v0.3.0-net-new. +These affect operators on upgrade. -- **Discovery rate-limiting** has been retuned to be less aggressive - at cold start. v0.2.0 used a single-lookup-with-internal-retry - model where a timed-out lookup during bloom-filter propagation - could suppress retries for 30 seconds while none of the reset - triggers fired on a stable post-handshake topology. v0.3.0 - replaces this with a per-attempt timeout sequence - (`node.discovery.attempt_timeouts_secs`, default `[1, 2, 4, 8]`, - 15s total). Each attempt sends a fresh `LookupRequest` with a new - `request_id`, letting successive attempts take different - forwarding paths as the bloom and tree state evolve. Post-failure - suppression is **off by default**; operators with chatty - applications can opt back in via `backoff_base_secs` / - `backoff_max_secs`. -- **MMP report intervals** are retuned for constrained transports. - The steady-state floor moves from 100ms to 1000ms, the ceiling - from 2000ms to 5000ms, with a cold-start phase running 200ms for - the first 5 SRTT samples. This reduces BLE overhead by roughly - 10× while keeping reports well above the EWMA convergence - threshold. Session-layer MMP intervals are unchanged. -- **Bloom filter fill-ratio validation** runs on every inbound - `FilterAnnounce`. Filters whose derived false-positive rate - exceeds `node.bloom.max_inbound_fpr` (default 0.05) are rejected - silently on the wire, logged at WARN, and counted in a new - `bloom.fill_exceeded` counter. A rate-limited WARN also fires - when the local outgoing filter exceeds the cap. -- **TreeAnnounce ancestry validation** is now run before tree-state - mutation, enforcing ancestry-self-match, root-single-entry, - parent-second-entry, and root-is-minimum-NodeAddr. Non-conforming - announces are rejected with a WARN. Mixed v0.2.0 / v0.2.1 / v0.3.0 - meshes may produce WARN log lines on the v0.2.1+ side until all - peers upgrade; behavior is correct, log noise only. -- **Log noise reduction**: 35 info-level log messages have been - demoted to debug (handshake cross-connection mechanics, periodic - MMP telemetry, TUN/transport shutdown, retry scheduling). The - default `RUST_LOG` in systemd units is now `info`, where it - previously ran at `debug`. Operator-visible info output now - focuses on lifecycle events, peer promotions, session - establishment, parent switches, and transport start/stop. +- **Bloom filter antipoison cap raised.** `node.bloom.max_inbound_fpr` + moves from 0.05 to 0.10, accepting filters with a higher derived + false-positive rate before rejecting them. This reduces spurious + filter rejections on larger meshes while keeping the antipoison + protection in place. +- **TCP inbound cap honors `max_connections`.** The TCP inbound accept + ceiling now resolves from explicit per-transport + `max_inbound_connections`, then node-wide + `node.limits.max_connections`, then the built-in default of 256. + Previously the TCP inbound ceiling was hardwired to 256 and ignored + `max_connections`, so raising it had no effect on inbound TCP. +- **Static host aliases hot-reload.** `/etc/fips/hosts` now reloads on + mtime change once per tick rather than only at startup, so display + names in `fipsctl` and `fipstop` reflect edits without a daemon + restart. The peer ACL reloads through the same lock-free snapshot + mechanism. +- **Quieter logs on busy public-mesh nodes.** Routine per-peer + connection-lifecycle and capacity-cap events, no-route session-datagram + drops, and exhausted rekey-budget aborts are demoted to debug, so + genuinely notable info and warn lines are no longer drowned out. +- **More visible drops.** Receive-path silent rejections now flow + through typed reject-reason counters, and discovery counts requests + dropped when the dedup cache is full (`req_dedup_cache_full`, visible + via `show_routing`). Drops that were previously silent are now + countable. +- **Tor connect-refused accounting.** The Tor transport increments its + `connect_refused` statistic (the "Refused" line in `fipstop`) on an + actively-refused SOCKS5 connect, instead of recording every connect + failure as a generic SOCKS5 error. ## Notable bug fixes -These pre-existing v0.2.0 bugs are worth singling out because they -either affected real-world deployments or produced misleading -operator experiences. The CHANGELOG has the exhaustive list; this is -the operator-relevant subset. Four items below first shipped in -v0.2.1 and roll forward into v0.3.0: auto-connect Disconnect-reconnect, -`fipsctl connect` mesh-address rejection, `fd00::/8` routing -protection from Tailscale interception, and bloom-filter routing -greedy-tree fallback. The control-socket path-detection fix landed -in v0.2.1 as well, and the unified resolver below is the v0.3.0 -refactor that builds on it. +The CHANGELOG has the exhaustive list. This is the operator-relevant +subset of fixes for behavior that shipped in v0.3.0. -- **DNS responder silent-drop on systemd-resolved** is fixed: the - responder no longer drops queries on Ubuntu 22 / Debian 13 and - similar deployments where systemd applies interface-scoped - routing. Default bind moves to `::1`; new global drop-in backend - available ([#52](https://github.com/jmcorgan/fips/issues/52), - [#77](https://github.com/jmcorgan/fips/issues/77)). -- **Auto-connect peers reconnect after a graceful Disconnect.** - Previously, a clean upstream shutdown left the auto-connect peer - orphaned; only the link-dead, decrypt-fail, and peer-restart - paths scheduled a reconnect - ([#60](https://github.com/jmcorgan/fips/issues/60), reported by - [@SwapMarket](https://github.com/SwapMarket)). -- **`fipsctl connect` rejects FIPS mesh addresses** (`fd00::/8`) - for `udp`, `tcp`, and `ethernet` transports with a clear error - message, instead of echoing success while the daemon silently - failed the bind with `EAFNOSUPPORT` - ([#61](https://github.com/jmcorgan/fips/issues/61), reported by - [@SwapMarket](https://github.com/SwapMarket)). -- **Default control-socket path resolution unified.** Daemon and - client tools now share a single resolver, eliminating a divergence - where `fipsctl` / `fipstop` could connect to a socket the daemon - never bound (notably on dev runs with `XDG_RUNTIME_DIR` set, or - after a prior packaged install left a root-owned `/run/fips` - behind). Canonical order is - `/run/fips` -> `$XDG_RUNTIME_DIR/fips/` -> `/tmp/fips-`. The - `/run/fips` arm is selected by directory existence; the kernel - enforces actual access at `connect(2)` time, so users not yet in - the `fips` group get a clear `EACCES` rather than a silent path - mismatch and a misleading `No such file` fallback to - `$XDG_RUNTIME_DIR`. `XDG_RUNTIME_DIR` is validated as an existing - directory before being used so stale post-logout values are - treated as missing. The deployed fleet is unaffected: packaged - configs set `node.control.socket_path` explicitly - ([#30](https://github.com/jmcorgan/fips/issues/30), reported by - [@Sebastix](https://github.com/Sebastix)). -- **`fd00::/8` routing protected from Tailscale interception.** The - daemon installs an IPv6 routing-policy rule - (`ip -6 rule to fd00::/8 lookup main priority 5265`) at TUN - setup, so Tailscale's table 52 default route can no longer divert - mesh traffic. -- **TCP-over-FIPS reliability on mixed-MTU paths** is markedly - improved. Four interlocking changes ship together: - `Node::transport_mtu()` is now deterministic across daemon - restarts (min across operational transports rather than - insertion-order-dependent); the TCP MSS clamp at the TUN boundary - reads per-destination path MTU instead of a single global ceiling; - reactive `MtuExceeded` from forwarders is mirrored back into the - TUN-side `path_mtu_lookup` so later flows pick up forward-path - bottlenecks without re-discovery; and the proactive end-to-end - `PathMtuNotification` echoed by the destination is mirrored into - the same TUN-side store. Without that fourth piece, on long-lived - stable paths where the destination's echo had tightened the - session MTU but no transit router had emitted a fresh - `MtuExceeded`, new TCP flows opened in that window were clamped by - the staler discovery-time value. The proactive mirror uses the - same tighter-only semantics as the reactive mirror, so it never - loosens the clamp. The Windows TUN reader receives the same - per-destination plumbing. -- **Bloom filter routing greedy-tree fallback.** `find_next_hop` no - longer returns `NoRoute` when the bloom candidate set is non-empty - but no candidate is strictly closer than the current node; it - falls through to greedy tree routing instead. Previously, this - caused dropped packets in topologies where the tree parent was - closer but not a bloom candidate. -- **`fipstop` graceful tty-init failure.** `ratatui::try_init()` - produces a clean error message instead of a hard crash when - terminal initialization fails (Docker on macOS Sequoia, ttyless - environments). -- **TreeAnnounce ancestry on self-root transitions.** When a node - had no smaller-NodeAddr peer to use as a parent, the spanning-tree - state correctly promoted it to root, but the ancestry advertised - on the next `TreeAnnounce` still referenced its previous parent's - path. Receiving peers rejected the announce as - `invalid ancestry: advertised root X is not the minimum path entry - Y`, blocking mesh transit on any path that needed to traverse the - node. The self-root transition is now detected explicitly in - `TreeState::become_root` and the advertised ancestry rebuilt to - start from self; the MMP receive handler corrects stale ancestry - inherited across reconnect eagerly rather than waiting for the - next observation tick. -- **Spanning-tree internal-path updates** that change only the - internal path between root and leaf (without changing the root or - the depth) now propagate to leaves correctly. Previously, a leaf - could continue routing against a stale internal path until the - parent or depth also changed. +- **Symmetric peer teardown on manual disconnect.** A manual + `fipsctl disconnect` now sends the peer a scoped Disconnect so both + ends tear down and re-handshake cleanly. Previously a manual + disconnect tore down only the local side, leaving the peer with a + stale session that was never re-adopted as a child and whose bloom + filter was never re-recorded. +- **Gateway holds long-lived and DNS-cached mappings.** `fips-gateway` + no longer drops a virtual-IP mapping while traffic is still flowing. + The mapping TTL clock previously advanced only on DNS re-query, so a + busy long-lived or DNS-cached client could have its mapping reclaimed + mid-flow. The tick now refreshes the mapping whenever conntrack reports + active sessions and recovers a draining mapping to active when traffic + resumes; only genuinely idle mappings drain. +- **Accurate mesh-size estimate under filter overlap.** The mesh-size + estimator now estimates the cardinality of the OR-union of self plus + every connected peer's inbound filter, instead of summing per-filter + cardinalities of tree peers. Summing assumed the filters were disjoint, + so a stale or oversized parent filter or a routing loop inflated the + reported mesh size and a tree rebalance flapped the count. OR-union + deduplicates overlap, equals the old result in the disjoint case, and + removes the estimate's dependence on tree-declaration cache freshness. +- **Single-uplink node reattaches within a round-trip.** A node with one + tree peer, which has periodic parent re-evaluation disabled, was left + self-rooted and unreachable if its one-shot attaching TreeAnnounce was + lost, until the next periodic re-broadcast. Tree-position exchange is + now self-healing on the receive path: a node that hears an announce + advertising a strictly worse root echoes its own declaration back, + provoking the better-rooted peer to re-push its real position + immediately. ## Upgrade notes -Operator-actionable items when moving from v0.2.x to v0.3.0: +Operator-actionable items moving from v0.3.0 to v0.4.0: -- **Control socket JSON schema (breaking, pre-1.0).** - - `show_cache` response field `entries` has changed type from a - `u64` count to an array of entry objects. The previous scalar - value is now in a new `count` field. - - `show_routing` response field `pending_lookups` has changed - type from a `u64` count to an array of per-target lookup - objects. - - External tooling parsing these fields as numbers must be - updated. In-tree `fipstop` is adjusted to the new schema. The - control-socket interface remains pre-1.0 and is not covered by - stability guarantees. +- **Wire-compatible, no flag day.** v0.4.0 peers with v0.3.0. Upgrade + nodes in any order. During a rolling upgrade you may see some log lines + on the upgraded side as it interacts with not-yet-upgraded peers; + behavior is correct, log noise only. +- **Bloom antipoison cap default changed.** `node.bloom.max_inbound_fpr` + now defaults to 0.10 (was 0.05). If you set this explicitly, review + whether you still want the old value. +- **New optional config surfaces.** `transports.nym` (outbound Nym + mixnet) and `node.discovery.lan` (mDNS LAN discovery) are both opt-in + and off by default. Adding them is the only way to turn the new paths + on. +- **TCP inbound cap.** If you relied on the old hardwired 256 inbound-TCP + ceiling, note it now honors `max_inbound_connections` then + `node.limits.max_connections` then 256. +- **New observability query.** `fipsctl stats metrics` (the + `show_metrics` control query) returns a counter-only snapshot suitable + for a scraper. -- **Cargo feature flags removed.** `tui`, `ble`, `gateway`, and - `nostr-discovery` are gone. Subsystem inclusion is now driven by - platform `cfg` gates, so plain `cargo build` compiles everything - available on the target without `--features` invocations. - Source-build tooling that passed any of these features should be - updated to omit them. - -- **Discovery rate-limiting defaults changed.** Post-failure - suppression is **off by default** - (`node.discovery.backoff_base_secs: 0`, `backoff_max_secs: 0`). - Operators relying on the prior 30s base / 300s cap behavior must - set those fields explicitly. The per-attempt sequence - (`attempt_timeouts_secs`, default `[1, 2, 4, 8]`) now governs - cold-start lookup behavior. - -- **`.fips` DNS bind address default changed.** The default - `dns.bind_addr` is now `::1`. Operators with explicit overrides - of this field should review them; many existing overrides were - workarounds for the silent-drop bug that this release fixes - properly. - -- **Gateway `dns.listen` source default changed.** The - `fips-gateway` `dns.listen` default is now `[::1]:5353` (was - `[::]:53`), matching the canonical deployment model where a - pre-existing resolver on the host already owns port 53. The - OpenWrt ipk previously overrode this in its packaged config; the - override is now redundant and has been dropped. Operators on a - host without a pre-existing resolver on port 53 can opt back into - the wildcard bind by setting `dns.listen: "[::]:53"` explicitly. - The new default binds IPv6 loopback only, so forwarders that - reach the gateway over IPv4 loopback need an explicit IPv4 listen - address. - -- **systemd unit log level.** The shipped systemd units no longer - hardcode `RUST_LOG=info`; the daemon's effective log level is - driven by `node.log_level` (default `info`). `RUST_LOG`, when - set, still overrides. - -- **UDP transport `bind_addr` validation.** Startup now rejects a - `bind_addr` set to a loopback address when at least one peer has - a non-loopback UDP address. Operators who configured a loopback - UDP bind as a workaround should switch to `outbound_only: true` - for the same effect, plus the correct semantics (kernel-assigned - ephemeral port, refuses inbound, never advertised). - -- **Tor advert port.** If the Tor `HiddenServicePort` virtual port - isn't 443, set `transports.tor.advertised_port` to match. The - default is 443 and matches the conventional virtual-port choice. - -## Documentation pointers - -v0.3.0 ships a `docs/` tree reorganized into four sections -(*tutorials / how-to / reference / design*). A new top-level -[`docs/getting-started.md`](../getting-started.md) and per-section -landing pages anchor the entry points. - -Entry points by reader intent: - -- **New users**: [`docs/getting-started.md`](../getting-started.md) - and [`docs/tutorials/`](../tutorials/) cover guided introductions - for bringing up your first node, joining the test mesh, - advertising a node over Nostr, hosting a service, deploying a - gateway, walking through the IPv6 adapter, and resolving peers - via Nostr. -- **Operators with a specific task**: - [`docs/how-to/`](../how-to/) holds task-driven guides for enabling - Nostr discovery, deploying the gateway, troubleshooting the - gateway, deploying a Tor onion, hosting aliases, persistent - identity, running unprivileged, setting up a Bluetooth peer, - enabling the mesh firewall, tuning UDP buffers, and diagnosing - MTU issues. -- **Reference lookups**: [`docs/reference/`](../reference/) holds - the config field reference, control-socket query reference, the - `fips`, `fipsctl`, `fipstop`, and `fips-gateway` CLI references, - and the protocol diagram set. -- **Architectural background**: [`docs/design/`](../design/) holds - design rationale for FIPS as a whole, FMP and FSP, the spanning - tree, bloom-filter discovery, transports, the IPv6 adapter, the - Nostr discovery layer, and the gateway. -- **Security**: [`docs/design/fips-security.md`](../design/fips-security.md) - documents the mesh-interface security baseline, threat model, and - drop-in workflow. - -## Getting v0.3.0 +## Getting v0.4.0 - **Linux x86_64 / aarch64**: `.deb` and tarball at the - [v0.3.0 release page](https://github.com/jmcorgan/fips/releases/tag/v0.3.0). + [v0.4.0 release page](https://github.com/jmcorgan/fips/releases/tag/v0.4.0). - **Arch Linux**: `fips` from the AUR. -- **macOS**: `.pkg` at the v0.3.0 release page. -- **Windows**: ZIP at the v0.3.0 release page. -- **OpenWrt**: `.ipk` at the v0.3.0 release page. -- **From source**: `cargo build --release` from a checkout of the - v0.3.0 tag. +- **macOS**: `.pkg` at the v0.4.0 release page. +- **Windows**: ZIP at the v0.4.0 release page. +- **OpenWrt**: `.ipk` at the v0.4.0 release page. +- **From source**: `cargo build --release` from a checkout of the v0.4.0 + tag (Rust 1.94.1 per `rust-toolchain.toml`; `libclang-dev` is a + required Linux build prerequisite). The full per-commit changelog lives in [`CHANGELOG.md`](../../CHANGELOG.md). Issues and discussion at @@ -698,67 +264,16 @@ The full per-commit changelog lives in ## Contributors -Thanks to everyone who contributed code, packaging work, bug reports, -or reviews to this release. +Thanks to everyone who contributed code, packaging work, bug reports, or +reviews to this release. -**Code and packaging**: - -- [@jcorgan](https://github.com/jmcorgan): release shepherd, Nostr - discovery / NAT traversal, `fips-gateway`, ACL infrastructure, - packaging, security baseline, BLE follow-ups. -- [@Origami74](https://github.com/Origami74): macOS platform support, - from-source Docker companion build and `fipstop` terminal-init - handling, gateway co-development, OpenWrt BLE-feature build fix, - AUR-workflow follow-ups. -- [@jodobear](https://github.com/jodobear): Linux release-artifact - workflow and target-aware build scripts, CONTRIBUTING.md - expansion, rekey integration-test stabilization. -- [@tidley](https://github.com/tidley): Nostr-mediated overlay - discovery and UDP NAT traversal - ([#53](https://github.com/jmcorgan/fips/pull/53)). -- [@alexxie16](https://github.com/alexxie16): peer ACL enforcement - ([#50](https://github.com/jmcorgan/fips/pull/50)), - macOS WireGuard companion example - ([#51](https://github.com/jmcorgan/fips/pull/51)), - follow-up ([#67](https://github.com/jmcorgan/fips/pull/67)). -- [@osh](https://github.com/osh): diagnostic queries for security - validation and mesh debugging - ([#42](https://github.com/jmcorgan/fips/pull/42)). -- [@OceanSlim](https://github.com/0ceanSlim): Windows platform - support ([#45](https://github.com/jmcorgan/fips/pull/45)). -- [@mmalmi](https://github.com/mmalmi): ring AEAD backend - ([#80](https://github.com/jmcorgan/fips/pull/80)), - hot-path drain batching + recvmmsg + eager pubkey_full - ([#81](https://github.com/jmcorgan/fips/pull/81)), - TreeAnnounce self-root ancestry + overlay-advert retry hygiene - ([#82](https://github.com/jmcorgan/fips/pull/82)), - NAT-traversal MTU inheritance - ([#83](https://github.com/jmcorgan/fips/pull/83)). -- [@dskvr](https://github.com/dskvr): initial Arch Linux AUR - packaging ([#21](https://github.com/jmcorgan/fips/pull/21)) and - the AUR publish workflow. -- [@SatsAndSports](https://github.com/SatsAndSports): rekey - message-1 admit fix on non-accepting transports - ([#49](https://github.com/jmcorgan/fips/pull/49)), - TreeAnnounce semantic validation, gateway test image fix - ([#69](https://github.com/jmcorgan/fips/pull/69)). -- [@andrewheadricke](https://github.com/andrewheadricke): MIPS - atomic-ABI portability via `portable_atomic` - ([#62](https://github.com/jmcorgan/fips/pull/62)). -- [@sh1ftred](https://github.com/sh1ftred): Arch packaging namcap - fixes ([#63](https://github.com/jmcorgan/fips/pull/63)). -- [@oleksky](https://github.com/oleksky): macOS WireGuard companion - collaboration on [#51](https://github.com/jmcorgan/fips/pull/51). - -**Issue reports that drove fixes in this release**: - -- [@deavmi](https://github.com/deavmi): MIPS daemon build support - ([#26](https://github.com/jmcorgan/fips/issues/26)). -- [@Sebastix](https://github.com/Sebastix): fipsctl/fipstop - control-socket path detection - ([#30](https://github.com/jmcorgan/fips/issues/30)). -- [@SwapMarket](https://github.com/SwapMarket): auto-connect - reconnect after graceful disconnect - ([#60](https://github.com/jmcorgan/fips/issues/60)) and - fipsctl mesh-address rejection - ([#61](https://github.com/jmcorgan/fips/issues/61)). +- [@jcorgan](https://github.com/jmcorgan): release shepherd, high-level + design, control read plane, rekey hardening, admission, bug fixes, + testing, packaging, PR coordination, and issue resolution. +- [@mmalmi](https://github.com/mmalmi): opt-in mDNS LAN discovery and + data-plane performance work. +- [@Origami74](https://github.com/Origami74): macOS packaging and + website coordination. +- [@dskvr](https://github.com/dskvr): AUR packaging. +- [@oleksky](https://github.com/oleksky): Nym mixnet transport and the + single-container mixnet demo. diff --git a/docs/reference/cli-fips.md b/docs/reference/cli-fips.md index 40b5844..0d0ff4e 100644 --- a/docs/reference/cli-fips.md +++ b/docs/reference/cli-fips.md @@ -28,7 +28,7 @@ controlled through the standard service control manager. | Flag | Argument | Description | | ---- | -------- | ----------- | | `-c`, `--config` | `FILE` | Use `FILE` as the configuration. Skips the default search paths. | -| `-V` | — | Print the short version (e.g. `0.3.0-dev (rev abcdef1)`). | +| `-V` | — | Print the short version (e.g. `0.4.0 (rev abcdef1)`). | | `--version` | — | Print the long version: short version plus build target triple. | | `-h`, `--help` | — | Print usage and exit. | | `--install-service` | — | (Windows only) Install `fips` as a Windows service. Requires Administrator. | diff --git a/docs/reference/cli-fipstop.md b/docs/reference/cli-fipstop.md index 6b759ef..226a82c 100644 --- a/docs/reference/cli-fipstop.md +++ b/docs/reference/cli-fipstop.md @@ -15,8 +15,11 @@ socket, polls a small set of `show_*` queries on a timer, and renders the state in a tabbed full-screen UI. A separate poll runs against the gateway control socket when the Gateway tab is active. -`fipstop` is read-only — it cannot mutate daemon state. Use -[`fipsctl`](cli-fipsctl.md) for `connect` / `disconnect` and friends. +`fipstop` is almost entirely read-only: the only state-mutating action +it offers is disconnecting a peer (`Del` on a selected Peers row, with +a confirmation prompt — see [Keybindings](#keybindings)). For +`connect` and other mutating commands, use +[`fipsctl`](cli-fipsctl.md). ## Options @@ -86,6 +89,11 @@ empty list and the panel hides. ## Keybindings +Press `?` at any time for an in-app help overlay. The overlay and the +status-bar hint footer both read from a single keybinding registry +keyed by `(tab, mode)`, so the always-visible hints describe exactly +the keys the current context accepts. + ### Global | Key | Action | @@ -94,7 +102,8 @@ empty list and the panel hides. | `Tab` | Next tab. | | `Shift-Tab` | Previous tab. | | `g` | Jump to the Graphs tab. | -| `Esc` | Close detail view (if open). | +| `?` | Toggle the help overlay. | +| `Esc` | Close an open detail view; otherwise deselect the active table row. | ### Table tabs (Peers, Sessions, Transports, Gateway) @@ -102,6 +111,13 @@ empty list and the panel hides. | --- | ------ | | `Up`, `Down` | Move row selection. | | `Enter` | Open detail view for the selected row. | +| `Esc` | Deselect the row (return to the tab's overview state). | + +### Peers tab (extra) + +| Key | Action | +| --- | ------ | +| `Del` | Disconnect the selected peer. Opens a `Y`/`N` confirmation modal first; this is the only state-mutating action in `fipstop`. | ### Transports tab (extra) @@ -112,16 +128,43 @@ empty list and the panel hides. | `e` | Expand all transports. | | `c` | Collapse all transports. | +### Multi-pane scrolling tabs (Tree, Filters, Routing) + +Each lays out stacked panes that scroll independently. + +| Key | Action | +| --- | ------ | +| `f` | Move focus to the next pane. | +| `Up`, `Down` | Scroll the focused pane by one row. | +| `PageUp`, `PageDown` | Scroll the focused pane by ten rows. | +| `Home`, `End` | Jump to the top / bottom of the focused pane. | + +### Performance tab (extra) + +The Performance tab lays out two panes (Link MMP, Session MMP). + +| Key | Action | +| --- | ------ | +| `f` | Move focus between the Link and Session MMP panes. | +| `Up`, `Down` | Scroll the focused pane. | +| `PageUp`, `PageDown` | Scroll the focused pane by ten rows. | +| `Home`, `End` | Jump to the top / bottom of the focused pane. | +| `s` | Cycle the sort column of the focused pane. | +| `Shift-S` | Toggle the sort direction of the focused pane. | + ### Graphs tab (extra) | Key | Action | | --- | ------ | -| `Up`, `Down` | Scroll within the stacked plots. | +| `Up`, `Down` | Scroll the stacked plots; in `MetricByPeer` mode, move the by-peer selection (and follow it when the by-peer detail is open). | | `Right`, `Space` | Next time window. Cycles `1m / 1s` → `10m / 1s` → `1h / 1s` → `24h / 1m`. | | `Left` | Previous time window. | +| `Enter` | In `MetricByPeer` mode, expand the selected peer summary into a full-pane plot. | | `m` | Cycle view mode: `Node` (stacked node metrics) → `MetricByPeer` (one per-peer metric across all peers) → `PeerByMetric` (all per-peer metrics for one peer). | | `n` | Next selector (next per-peer metric in MetricByPeer; next peer in PeerByMetric). | | `Shift-N` | Previous selector. | +| `s` | Cycle the sort column of the by-peer summary list. | +| `Shift-S` | Toggle the sort direction of the by-peer summary list. | ## Exit Codes diff --git a/docs/reference/control-socket.md b/docs/reference/control-socket.md index 68c2007..1d41a3e 100644 --- a/docs/reference/control-socket.md +++ b/docs/reference/control-socket.md @@ -100,22 +100,22 @@ table below lists every command currently registered. | Command | Params | `data` shape (top-level keys) | | ------- | ------ | ----------------------------- | -| `show_status` | — | `version`, `npub`, `node_addr`, `ipv6_addr`, `state`, `is_leaf_only`, `peer_count`, `session_count`, `link_count`, `transport_count`, `connection_count`, `tun_state`, `tun_name`, `effective_ipv6_mtu`, `control_socket`, `pid`, `exe_path`, `uptime_secs`, `estimated_mesh_size`, `forwarding`, `sparklines`. | +| `show_status` | — | `version`, `npub`, `node_addr`, `ipv6_addr`, `state`, `is_leaf_only`, `is_root` (bool — this node is the spanning-tree root), `root` (hex node-addr of the current tree root), `persistent` (bool — identity is persisted, i.e. `persistent` set or an `nsec` configured), `peer_count`, `session_count`, `link_count`, `transport_count`, `connection_count`, `transport_peer_counts` (object mapping transport-type name to its connected-peer count; configured transports appear with `0`), `tun_state`, `tun_name`, `effective_ipv6_mtu`, `control_socket`, `pid`, `exe_path`, `uptime_secs`, `estimated_mesh_size`, `forwarding`, `sparklines`. | | `show_acl` | — | `allow_file`, `deny_file`, `enforcement_active`, `effective_mode`, `default_decision`, `allow_all`, `deny_all`, `allow_file_entries`, `deny_file_entries`, `allow_entries`, `deny_entries`. | -| `show_peers` | — | `peers[]` — per-peer object: `node_addr`, `npub`, `display_name`, `ipv6_addr`, `connectivity`, `link_id`, `direction`, `transport_addr`, `transport_type`, `is_parent`, `is_child`, `tree_depth`, `stats`, `noise`, `current_k_bit`, `mmp`, plus optional `nostr_traversal`, `rekey_in_progress`, `rekey_draining`. | +| `show_peers` | — | `peers[]` — per-peer object: `node_addr`, `npub`, `display_name`, `ipv6_addr`, `connectivity`, `link_id`, `direction`, `transport_addr`, `transport_type`, `is_parent`, `is_child`, `tree_depth`, `effective_depth` (`tree_depth + link_cost` — the metric `evaluate_parent` ranks on; `null` when the peer has no coords, or is unmeasured while another peer has an SRTT sample, per the cold-start gate), `stats`, `noise`, `current_k_bit`, `mmp`, plus optional `nostr_traversal`, `rekey_in_progress`, `rekey_draining`. | | `show_links` | — | `links[]` — `link_id`, `transport_id`, `remote_addr`, `direction`, `state`, `created_at_ms`, `stats`. | -| `show_tree` | — | `my_node_addr`, `root`, `is_root`, `depth`, `my_coords[]`, `parent`, `parent_display_name`, `declaration_sequence`, `declaration_signed`, `peer_tree_count`, `peers[]`, `stats`. | +| `show_tree` | — | `my_node_addr`, `root`, `root_npub` (bech32 npub of the current tree root), `is_root`, `depth`, `my_coords[]`, `parent`, `parent_display_name`, `declaration_sequence`, `declaration_signed`, `peer_tree_count`, `peers[]`, `stats`. | | `show_sessions` | — | `sessions[]` — `remote_addr`, `npub`, `display_name`, `state` (`established`, `initiating`, `awaiting_msg3`, `unknown`), `is_initiator`, `last_activity_ms`, `stats`, optional `mmp`, `current_k_bit`, `is_draining`. | -| `show_bloom` | — | `own_node_addr`, `is_leaf_only`, `sequence`, `leaf_dependent_count`, `leaf_dependents[]`, `peer_filters[]`, `stats`. | +| `show_bloom` | — | `own_node_addr`, `is_leaf_only`, `sequence`, `leaf_dependent_count`, `leaf_dependents[]`, `peer_filters[]`, `uptree_fill_ratio` (fill ratio of the last filter actually sent to the tree parent), `uptree_estimated_count` (cardinality estimate of that uptree filter — this node's whole subtree under split-horizon, not the mesh; both are `null` for a root node or before the first announce), `stats`. | | `show_mmp` | — | `peers[]` (link-layer per peer), `sessions[]` (session-layer per session). Each entry includes loss/RTT/ETX/goodput, smoothed values, trends. | | `show_cache` | — | `count`, `max_entries`, `fill_ratio`, `default_ttl_ms`, `expired`, `avg_age_ms`, `entries[]` — per-destination coords, depth, age, last-used, optional `path_mtu`. | | `show_connections` | — | `connections[]` — pending handshakes: `link_id`, `direction`, `handshake_state`, `started_at_ms`, `idle_ms`, `resend_count`, optional `expected_peer`. | | `show_transports` | — | `transports[]` — `transport_id`, `type`, `state`, `mtu`, `name`, `local_addr`, optional `tor_mode`, `onion_address`, `tor_monitoring`, `stats`. | -| `show_routing` | — | `coord_cache_entries`, `identity_cache_entries`, `pending_lookups[]`, `pending_tun_destinations`, `pending_tun_packets`, `recent_requests`, `retries[]`, `forwarding`, `discovery`, `error_signals`, `congestion`. | +| `show_routing` | — | `coord_cache_entries`, `identity_cache_entries`, `pending_lookups[]`, `pending_tun_destinations`, `pending_tun_packets`, `recent_requests`, `retries[]`, `forwarding`, `discovery` (request/response sub-counters; includes `req_deduplicated` — requests suppressed as recent duplicates — and `req_dedup_cache_full` — requests admitted because the dedup cache was full), `error_signals`, `congestion`. | | `show_identity_cache` | — | `entries[]`, `count`, `max_entries`. Each entry: `node_addr`, `npub`, `display_name`, `ipv6_addr`, `last_seen_ms`, `age_ms`. | | `show_listening_sockets` | — | `fips0_addr`, `firewall_active` (bool — `inet fips` table loaded), `sockets[]`. Each entry: `proto` (`tcp` / `udp`), `local_addr` (`::` or the node's fd00::/8 address), `port`, `pid` (nullable), `process` (nullable), `wildcard_bind` (bool — `local_addr == ::`), `filter` (`accept` / `drop` / `unknown` / `no_firewall`). Linux-only; returns an empty `sockets[]` on other platforms. | | `show_stats_list` | — | `metrics[]` (each with `name`, `unit`, `scope`), `fast_ring_seconds`, `slow_ring_minutes`, `peer_retention_seconds`. | -| `show_metrics` | — | Flat snapshot of every counter family in the metrics registry: `forwarding`, `discovery`, `tree`, `bloom`, `congestion`, `errors`. Each value is that family's counter snapshot object. Counter-only — gauges/histograms that need the live node are excluded. Served off the main loop. | +| `show_metrics` | — | Flat snapshot of every counter family in the metrics registry: `forwarding`, `discovery`, `tree`, `bloom`, `congestion`, `errors`. Each value is that family's counter snapshot object. Counter-only — gauges/histograms that need the live node are excluded. Served off the main loop. Silent-rejection sites classify their reason as a typed `RejectReason` and increment the matching per-family counter exposed here — see [Rejection reasons](#rejection-reasons). | | `show_stats_history` | `metric` (req), `peer` (req for per-peer metrics), `window` (`s` / `m` / `h`, default `10m`), `granularity` (`1s` / `1m`, default `1s`) | A single `Series`: `metric`, `unit`, `granularity_seconds`, `values[]`. | | `show_stats_all_history` | `peer` (optional npub), `window`, `granularity` | `granularity_seconds`, `window_seconds`, `peer`, `series[]` (one per metric). | | `show_stats_peers` | — | `peers[]`, `count`. Each entry: `npub`, `node_addr`, `display_name`, `is_active`, `first_seen_secs_ago`, `last_contact_secs_ago`. | @@ -125,6 +125,30 @@ The schema of each query response is pinned by snapshot tests in `src/control/snapshots/`; intentional schema changes regenerate those fixtures. +### Rejection reasons + +Silent-rejection paths across the node classify why a message was +dropped via a typed `RejectReason` rather than only logging it, so the +*what* of a rejection is visible in the counter snapshots above. The +top-level reason set has eight families, mirroring the protocol-layer / +subsystem split of the metrics: + +- **Tree** — spanning-tree `TreeAnnounce` processing rejections. +- **Bloom** — bloom-filter `FilterAnnounce` processing rejections. +- **Discovery** — discovery request / response processing rejections. +- **Handshake** — Noise handshake state-machine rejections. +- **Session** — FSP session state-machine rejections. +- **Mmp** — MMP link-layer rejections. +- **Forwarding** — forwarding-path rejections (no-route, TTL, MTU). +- **Transport** — transport-layer rejections (admission caps, framing). + +Each rejection increments the corresponding counter in its family's +stats, surfaced through `show_metrics` (the `tree`, `bloom`, +`discovery`, and `forwarding` families carry their own counters; the +`errors` family and the remaining subsystem counters carry the rest). +The full per-family variant list lives in `src/node/reject.rs`; it is +not reproduced here to avoid duplicating the source. + ### Mutating commands | Command | Required params | Behaviour | diff --git a/docs/releases/release-notes-v0.4.0.md b/docs/releases/release-notes-v0.4.0.md new file mode 100644 index 0000000..0d62767 --- /dev/null +++ b/docs/releases/release-notes-v0.4.0.md @@ -0,0 +1,279 @@ +# FIPS v0.4.0 + +**Released**: 2026-06-DD (provisional) + +v0.4.0 is the throughput-and-observability release on the v0.3.x wire +format. It adds two new ways for nodes to find and reach each other (the +Nym mixnet transport and opt-in mDNS LAN discovery), overhauls the data +plane for higher single-node throughput and lower per-packet CPU, moves +the entire operator read surface off the data-plane hot path so +observability stays responsive under load, ships a reworked `fipstop` +TUI, and hardens FMP and FSP rekey to be hitless under packet loss in +both directions. It also folds in the accumulated mesh-convergence, +admission-control, and packaging fixes from the maintenance line. + +v0.4.0 is wire-compatible with v0.3.0. Mixed meshes interoperate; there +is no flag-day upgrade. A deployed v0.3.0 node and an upgraded v0.4.0 +node peer, rekey, and route normally, so you can roll the upgrade out +across a mesh in any order. + +## At a glance + +- New outbound Nym mixnet transport with a single-container demo and a + new mixnet-relay example. +- Opt-in mDNS / DNS-SD discovery on the local link. +- Data-plane overhaul: off-task encrypt and decrypt worker pools, GSO, + connected-UDP send path, copy-avoidance on receive, batched macOS + receive. +- The full `show_*` read surface now serves off the receive loop, so + `fipsctl` and `fipstop` stay responsive on loaded nodes; a new + counter-only `show_metrics` query enables a Prometheus scraper at no + hot-path cost. +- Reworked `fipstop` TUI on a machine-verified render-snapshot base. +- Rekey is now hitless under loss and reordering in both directions. + +## What's new + +### Nym mixnet transport + +FIPS can now peer over the [Nym](https://nymtech.net/) mixnet for +metadata-resistant connectivity. The new `transports.nym` transport +makes outbound connections through a `nym-socks5-client` SOCKS5 proxy +that you run alongside the daemon (for example as a service running +alongside the fips daemon, or as a sidecar container). The transport +waits at startup for the nym-socks5-client to become ready before giving +up. + +This is a privacy and anonymity deployment mode chosen for its own +properties. It mixes your FIPS traffic into the Nym cover-traffic +network so that link-level observers cannot correlate which mesh peers +are talking. A new `examples/sidecar-nostr-mixnet-relay/` demonstrates a +FIPS-reachable Nostr relay peered across the mixnet end to end, and a +single-container demo ships with the transport. + +Enable it by adding a `transports.nym` instance and pointing it at your +running nym-socks5-client. See the transports reference for the field +set. + +### mDNS LAN discovery + +Nodes on a shared local link can now find each other with zero address +configuration. The opt-in `node.discovery.lan` path runs an mDNS / +DNS-SD responder and browser: each node advertises a FIPS service record +on the link and adopts the peers it discovers. This complements the +existing Nostr-mediated overlay discovery for the common case where the +peers are simply on the same LAN. + +Turn it on with `node.discovery.lan.enabled: true`. `service_type` and +`scope` tune the advertised service record and which interfaces +participate. Discovery on the local link needs no relay and no STUN. + +### Data-plane throughput overhaul + +The receive and send paths were reworked for higher single-node +throughput and lower per-packet CPU, building on the v0.3.0 +crypto-backend swap: + +- **Off-task encrypt and decrypt.** Per-peer encrypt and decrypt now run + on dedicated worker tasks rather than inline on the receive loop, so a + single busy peer no longer serializes the whole node's crypto. +- **GSO and connected-UDP send.** The Linux send path uses generic + segmentation offload and a connected-UDP socket where available, + cutting syscall overhead on bulk flows. +- **Copy-avoidance on receive.** The receive hot path avoids buffer + copies it previously made per packet. +- **Batched macOS receive.** macOS gains a `recvmsg_x` batched receive, + mirroring the Linux `recvmmsg` batching from v0.3.0. +- **Shared immutable-state context and an atomic metric registry.** + Immutable per-node state moved into a single shared context, and + counters live in an atomic metric registry that the new `show_metrics` + query reads without touching the hot path. + +These are all internal to the data plane and require no operator action. + +### Observability off the hot path + +Every read-only control query now renders from a snapshot published once +per tick into a lock-free `ArcSwap`, served from the control accept task +instead of round-tripping the data-plane receive loop. This covers +`show_status`, `show_stats_*`, `show_peers`, `show_sessions`, +`show_links`, `show_connections`, `show_transports`, `show_mmp`, +`show_tree`, `show_bloom`, `show_cache`, `show_routing`, +`show_identity_cache`, `show_acl`, `show_listening_sockets`, and the new +`show_metrics`. Only the mutating `connect` and `disconnect` commands +still reach the loop. + +The practical effect: on a loaded node where the receive loop was busy, +`fipsctl` and `fipstop` queries previously stalled or timed out (the +five-second query pattern operators saw). They now answer promptly +regardless of data-plane load. Per-entity snapshots reuse unchanged rows +by pointer, so the per-tick publish cost stays bounded as peer and +session counts grow. + +A new **`show_metrics`** query (surfaced as `fipsctl stats metrics`) +returns a counter-only snapshot of every metric family. It is the +enabler for a Prometheus scraper that pulls node counters at no hot-path +cost. + +### Reworked fipstop TUI + +`fipstop` gets a rendering, navigation, and read-surface overhaul on a +machine-verified base: a render-snapshot harness asserts the exact text +grid and per-cell style of every view against canned control-socket +output. New daemon-resolved fields surface through the snapshots, +including effective persistence, root and is-root state, a +per-transport-type peer-count map, per-peer effective depth, the root +npub, and the last-sent uptree filter fill ratio with the subtree size +estimate. + +A separate fix clears a garbled-screen problem on startup and stray +bytes on quit, most visible over SSH and inside tmux: startup now forces +a full repaint before the first draw, and quit stops and joins the +stdin-poll thread before restoring the terminal, so post-raw-mode +keystrokes no longer echo onto the restored screen. + +### Rekey reliability + +FMP and FSP session rekey are now hitless under packet loss and +reordering in both directions: + +- Inbound frames are authenticated against the pending session before + the K-bit cutover promotes it, so a spoofed or stale frame cannot + derail a rekey in progress. +- Rekey message-1 retransmission is bounded, and the link-dead heartbeat + is rekey-aware so an in-flight rekey is not mistaken for a dead link. +- FSP session rekey holds connectivity across the rekey window under + loss and reordering. +- Dual-initiation races (both peers starting a rekey at once on a + high-latency link) are desynchronized with symmetric jitter so the two + sides converge on one session rather than fighting. +- An exhausted retransmission-budget abort, an expected and self-limiting + outcome on lossy or high-latency links, is logged at debug rather than + warn. + +The net operator takeaway: rekey completes cleanly without dropping +traffic, even on lossy or high-latency links, and the log no longer +cries wolf when a rekey gives up and retries. + +## Behavior changes worth flagging + +These affect operators on upgrade. + +- **Bloom filter antipoison cap raised.** `node.bloom.max_inbound_fpr` + moves from 0.05 to 0.10, accepting filters with a higher derived + false-positive rate before rejecting them. This reduces spurious + filter rejections on larger meshes while keeping the antipoison + protection in place. +- **TCP inbound cap honors `max_connections`.** The TCP inbound accept + ceiling now resolves from explicit per-transport + `max_inbound_connections`, then node-wide + `node.limits.max_connections`, then the built-in default of 256. + Previously the TCP inbound ceiling was hardwired to 256 and ignored + `max_connections`, so raising it had no effect on inbound TCP. +- **Static host aliases hot-reload.** `/etc/fips/hosts` now reloads on + mtime change once per tick rather than only at startup, so display + names in `fipsctl` and `fipstop` reflect edits without a daemon + restart. The peer ACL reloads through the same lock-free snapshot + mechanism. +- **Quieter logs on busy public-mesh nodes.** Routine per-peer + connection-lifecycle and capacity-cap events, no-route session-datagram + drops, and exhausted rekey-budget aborts are demoted to debug, so + genuinely notable info and warn lines are no longer drowned out. +- **More visible drops.** Receive-path silent rejections now flow + through typed reject-reason counters, and discovery counts requests + dropped when the dedup cache is full (`req_dedup_cache_full`, visible + via `show_routing`). Drops that were previously silent are now + countable. +- **Tor connect-refused accounting.** The Tor transport increments its + `connect_refused` statistic (the "Refused" line in `fipstop`) on an + actively-refused SOCKS5 connect, instead of recording every connect + failure as a generic SOCKS5 error. + +## Notable bug fixes + +The CHANGELOG has the exhaustive list. This is the operator-relevant +subset of fixes for behavior that shipped in v0.3.0. + +- **Symmetric peer teardown on manual disconnect.** A manual + `fipsctl disconnect` now sends the peer a scoped Disconnect so both + ends tear down and re-handshake cleanly. Previously a manual + disconnect tore down only the local side, leaving the peer with a + stale session that was never re-adopted as a child and whose bloom + filter was never re-recorded. +- **Gateway holds long-lived and DNS-cached mappings.** `fips-gateway` + no longer drops a virtual-IP mapping while traffic is still flowing. + The mapping TTL clock previously advanced only on DNS re-query, so a + busy long-lived or DNS-cached client could have its mapping reclaimed + mid-flow. The tick now refreshes the mapping whenever conntrack reports + active sessions and recovers a draining mapping to active when traffic + resumes; only genuinely idle mappings drain. +- **Accurate mesh-size estimate under filter overlap.** The mesh-size + estimator now estimates the cardinality of the OR-union of self plus + every connected peer's inbound filter, instead of summing per-filter + cardinalities of tree peers. Summing assumed the filters were disjoint, + so a stale or oversized parent filter or a routing loop inflated the + reported mesh size and a tree rebalance flapped the count. OR-union + deduplicates overlap, equals the old result in the disjoint case, and + removes the estimate's dependence on tree-declaration cache freshness. +- **Single-uplink node reattaches within a round-trip.** A node with one + tree peer, which has periodic parent re-evaluation disabled, was left + self-rooted and unreachable if its one-shot attaching TreeAnnounce was + lost, until the next periodic re-broadcast. Tree-position exchange is + now self-healing on the receive path: a node that hears an announce + advertising a strictly worse root echoes its own declaration back, + provoking the better-rooted peer to re-push its real position + immediately. + +## Upgrade notes + +Operator-actionable items moving from v0.3.0 to v0.4.0: + +- **Wire-compatible, no flag day.** v0.4.0 peers with v0.3.0. Upgrade + nodes in any order. During a rolling upgrade you may see some log lines + on the upgraded side as it interacts with not-yet-upgraded peers; + behavior is correct, log noise only. +- **Bloom antipoison cap default changed.** `node.bloom.max_inbound_fpr` + now defaults to 0.10 (was 0.05). If you set this explicitly, review + whether you still want the old value. +- **New optional config surfaces.** `transports.nym` (outbound Nym + mixnet) and `node.discovery.lan` (mDNS LAN discovery) are both opt-in + and off by default. Adding them is the only way to turn the new paths + on. +- **TCP inbound cap.** If you relied on the old hardwired 256 inbound-TCP + ceiling, note it now honors `max_inbound_connections` then + `node.limits.max_connections` then 256. +- **New observability query.** `fipsctl stats metrics` (the + `show_metrics` control query) returns a counter-only snapshot suitable + for a scraper. + +## Getting v0.4.0 + +- **Linux x86_64 / aarch64**: `.deb` and tarball at the + [v0.4.0 release page](https://github.com/jmcorgan/fips/releases/tag/v0.4.0). +- **Arch Linux**: `fips` from the AUR. +- **macOS**: `.pkg` at the v0.4.0 release page. +- **Windows**: ZIP at the v0.4.0 release page. +- **OpenWrt**: `.ipk` at the v0.4.0 release page. +- **From source**: `cargo build --release` from a checkout of the v0.4.0 + tag (Rust 1.94.1 per `rust-toolchain.toml`; `libclang-dev` is a + required Linux build prerequisite). + +The full per-commit changelog lives in +[`CHANGELOG.md`](../../CHANGELOG.md). Issues and discussion at +[github.com/jmcorgan/fips](https://github.com/jmcorgan/fips). + +## Contributors + +Thanks to everyone who contributed code, packaging work, bug reports, or +reviews to this release. + +- [@jcorgan](https://github.com/jmcorgan): release shepherd, high-level + design, control read plane, rekey hardening, admission, bug fixes, + testing, packaging, PR coordination, and issue resolution. +- [@mmalmi](https://github.com/mmalmi): opt-in mDNS LAN discovery and + data-plane performance work. +- [@Origami74](https://github.com/Origami74): macOS packaging and + website coordination. +- [@dskvr](https://github.com/dskvr): AUR packaging. +- [@oleksky](https://github.com/oleksky): Nym mixnet transport and the + single-container mixnet demo. diff --git a/packaging/common/fips.yaml b/packaging/common/fips.yaml index 3efa23b..a9b2dc8 100644 --- a/packaging/common/fips.yaml +++ b/packaging/common/fips.yaml @@ -33,6 +33,23 @@ node: # - "stun:stun.l.google.com:19302" # - "stun:stun.cloudflare.com:3478" # - "stun:global.stun.twilio.com:3478" + # + # Optional mDNS-based LAN discovery for sub-second same-LAN pairing. + # Opt-in (default false): default-off avoids a per-LAN identity + # broadcast on nodes that have deliberately disabled other discovery + # channels, and avoids any multicast surprise on upgrade. Requires an + # operational UDP transport (the advertised port is the one peers dial). + # lan: + # enabled: false + # # Optional application/network scope carried in the LAN-only TXT + # # record. Browsers that set a scope ignore adverts for other scopes. + # # Kept separate from the Nostr discovery `app` tag so relay-visible + # # adverts can stay generic while LAN discovery stays per-private-network. + # # scope: "lab-floor-3" + # # Advanced: overrides the mDNS service type. Leave unset in normal + # # use — only needed to run multiple isolated services on one + # # interface (e.g. test isolation on loopback). + # # service_type: "_fips._udp.local." tun: enabled: true @@ -86,6 +103,18 @@ transports: # auto_connect: true # accept_connections: true + # Nym transport — outbound-only connections through the Nym mixnet for + # sender/receiver privacy. This is a privacy posture, not a NAT-traversal + # or failover transport: it only dials out (no inbound listener) and is + # chosen for its anonymity properties. Requires a nym-socks5-client + # running separately as its own process; FIPS dials it over SOCKS5. + # nym: + # socks5_addr: "127.0.0.1:1080" # nym-socks5-client SOCKS5 address + # connect_timeout_ms: 300000 # outbound connect timeout (300s); + # # mixnet round-trips can take minutes + # mtu: 1400 # per-connection MTU + # startup_timeout_secs: 120 # wait for nym-socks5-client readiness + # Outbound LAN gateway. Allows non-FIPS hosts on the LAN to reach # mesh destinations via DNS-allocated virtual IPs and kernel NAT. # Requires: IPv6 forwarding enabled, fips daemon running with DNS.