From c2ba0e3fcbdfe53c457f4fa7e350d2373105258c Mon Sep 17 00:00:00 2001 From: Johnathan Corgan Date: Thu, 29 Jan 2026 16:36:46 +0000 Subject: [PATCH] Add FIPS software architecture document Create comprehensive architecture document covering: - Core entities: Node, Transport, Link, Peer hierarchy - Transport vs Link distinction (interface vs connection) - State machines for Transport, Link, and Peer lifecycles - Reference transport types: UDP/IP, Ethernet, WiFi, Tor - Spanning tree and Bloom filter state requirements - Self-healing protocol design (no ack/retry needed) - Leaf-only operation for constrained devices - Comprehensive sysctl-style configuration reference Also add architecture review document capturing identified issues to resolve before implementation begins. --- docs/design/fips-architecture-review.md | 285 ++++++++ docs/design/fips-architecture.md | 841 ++++++++++++++++++++++++ 2 files changed, 1126 insertions(+) create mode 100644 docs/design/fips-architecture-review.md create mode 100644 docs/design/fips-architecture.md diff --git a/docs/design/fips-architecture-review.md b/docs/design/fips-architecture-review.md new file mode 100644 index 0000000..aae057d --- /dev/null +++ b/docs/design/fips-architecture-review.md @@ -0,0 +1,285 @@ +# FIPS Architecture Document Review + +**Date**: 2025-01-29 +**Document reviewed**: fips-architecture.md + +This file captures critique and open issues identified during architecture review. +Items should be addressed before implementation begins or marked as intentionally +deferred. + +--- + +## High Priority (Implementation Blockers) + +### 1. Coordinate Ordering Inconsistency + +**Issue**: Architecture says `[self, parent, ..., root]` but spanning-tree-dynamics.md +sometimes uses `[root, ..., self]` ordering. + +**Location**: fips-architecture.md line 165, spanning-tree-dynamics.md line 346-349 + +**Resolution needed**: Standardize on one ordering across all documents. + +**Status**: OPEN + +--- + +### 2. Authentication Protocol Not Referenced + +**Issue**: Architecture references "FIPS auth handshake" but doesn't define it or +reference where it's defined. + +**Location**: fips-architecture.md line 285 + +**Resolution needed**: Add reference to fips-design.md lines 115-205, or include +summary in architecture document. + +**Status**: OPEN + +--- + +### 3. Concurrency Model Unspecified + +**Issue**: Document doesn't specify whether implementation should be: +- Single-threaded async +- Multi-threaded with message passing +- How state machines are driven (polling, callbacks, async/await) +- Whether transports run in separate threads/tasks + +**Location**: Entire document + +**Resolution needed**: Add "Concurrency Model" section specifying the expected +runtime architecture. + +**Status**: OPEN + +--- + +### 4. Keepalive Message Format Unspecified + +**Issue**: Configuration specifies `peer.keepalive.interval` but no message type +is defined for keepalives. RTT measurement mechanism is unclear. + +**Location**: fips-architecture.md line 762 + +**Resolution needed**: Clarify whether Dummy (0x00) message from fips-design.md +is used, and how RTT is measured (request/response probing or passive observation). + +**Status**: OPEN + +--- + +### 5. Cache Naming Confusion + +**Issue**: Multiple cache references that may or may not be the same thing: +- `coord_cache: CoordCache` on Node (line 35) +- `coord_cache: HashMap` on TreeState (line 167) +- `discovery.cache.max_entries` configuration (line 745) +- `session.cache.max_entries` configuration (line 751) + +**Resolution needed**: Clarify whether these are the same cache or different caches. +If different, explain the distinction. + +**Status**: OPEN + +--- + +## Medium Priority (Design Clarity) + +### 6. Peer vs Link Relationship + +**Issue**: Document states "one-to-one mapping between peers and links" (line 148) +but Links can exist before authentication completes, meaning Links exist without +Peers temporarily. + +**Resolution needed**: Clarify the relationship lifecycle: +- Link created on connection (before auth) +- Peer created on successful auth +- Peer always references exactly one Link +- Link can exist without Peer (during auth or after auth failure) + +**Status**: OPEN + +--- + +### 7. Error Handling Strategy Absent + +**Issue**: No systematic error handling definitions: +- Signature verification failure handling +- Malformed packet handling +- Error codes and response formats +- Logging/reporting strategy + +**Resolution needed**: Add "Error Handling" section or reference to protocol spec. + +**Status**: OPEN + +--- + +### 8. Memory Bounds Missing + +**Issue**: Configuration specifies cache sizes but no limits for: +- Maximum peers +- Maximum transports +- Maximum pending operations/queues +- Overall memory budget + +**Resolution needed**: Add resource limit configuration or document that these +are implementation-defined. + +**Status**: OPEN + +--- + +### 9. Timer Management at Scale + +**Issue**: Multiple timers per peer (auth timeout, keepalive, reconnect delay, +filter debounce). For 100 peers, this could be hundreds of timers. + +**Resolution needed**: Note that timer wheel or hierarchical timing wheel may +be needed for efficient implementation at scale. + +**Status**: OPEN + +--- + +### 10. Initialization/Shutdown Sequences + +**Issue**: No startup sequence defined: +- Transport start order +- TUN interface initialization relative to peering +- Behavior when configured peers are unreachable +- When node is "ready" to route + +No shutdown procedure defined: +- Should nodes announce departure? +- Session termination +- TUN interface cleanup + +**Resolution needed**: Add "Initialization" and "Shutdown" sections. + +**Status**: OPEN + +--- + +## Lower Priority (Polish) + +### 11. Configuration Validation Rules + +**Issue**: No validation rules specified: +- What if `filter.size` is not a power of 2? +- What if `peer.keepalive.timeout` < `peer.keepalive.interval`? +- What if `timeout.adaptive.min` > `timeout.adaptive.max`? + +**Status**: OPEN + +--- + +### 12. Default Value Rationales + +**Issue**: Several defaults lack rationale: +- `filter.scope = 2`: Why 2? +- `tree.parent.hold_time = 10s`: Why 10 seconds? +- `filter.stale.threshold = 300s`: Based on what analysis? + +**Status**: OPEN + +--- + +### 13. Terminology Inconsistency Across Documents + +**Issue**: "Link" and "Transport" meanings differ between documents: +- fips-links.md uses "link" to mean underlying transport protocol +- fips-architecture.md uses "Link" to mean connection over Transport +- fips-design.md has `FipsLink` trait which is really a transport interface + +**Resolution needed**: Standardize terminology or add glossary. + +**Status**: OPEN + +--- + +### 14. Gateway/Subnet Routing Details Incomplete + +**Issue**: Document mentions `leaf_dependents` but doesn't explain: +- How a node becomes a gateway for a subnet +- How gateway prefixes are advertised in Bloom filters +- Configuration for gateway mode + +**Status**: OPEN + +--- + +### 15. DiscoveredPeer Hint Field Mismatch + +**Issue**: Transport trait shows `discover() -> Result>` but +event shows `DiscoveredPeer { transport_id, addr, hint: Option }`. +The discover() return type doesn't show the hint field. + +**Resolution needed**: Align trait signature with event structure. + +**Status**: OPEN + +--- + +## Edge Cases Not Addressed + +### Root Node Failure +- How long until new root is elected? +- How are in-flight sessions affected? +- Any proactive root backup mechanism? + +### Rapid Peer Churn +- Debounce (500ms) may not be sufficient +- Could lead to Bloom filter oscillation or announcement storms + +### Network Partition Healing +- What happens to sessions that existed in both partitions? +- How do routers with stale cache entries recover? + +### Transport Startup Timing +- If Tor takes 180s but UDP is instant, what's node status during window? +- Are Tor-only peers unreachable during Tor bootstrap? + +### MTU Mismatch Across Path +- Path includes links with different MTUs (Ethernet 1500 vs LoRa 222) +- No path MTU discovery or fragmentation strategy defined + +### Clock Skew +- TreeAnnounce timestamps with significant clock skew +- 5-minute tolerance in auth (fips-design.md) should be reflected here + +### Resource Exhaustion Attacks +- Memory exhaustion via many fake peers +- CPU exhaustion via signature verification flooding +- Bandwidth exhaustion via Bloom filter spam + +### Conflicting Peer Configurations +- Two configured peers with same npub but different addresses +- Discovered peer conflicts with configured peer + +### TUN Interface Unavailable +- TUN creation fails (permissions, kernel module) +- Can node run in "relay-only" mode without TUN? + +--- + +## Resolution Tracking + +| # | Issue | Priority | Status | Resolution | +|---|-------|----------|--------|------------| +| 1 | Coordinate ordering | High | OPEN | | +| 2 | Auth protocol reference | High | OPEN | | +| 3 | Concurrency model | High | OPEN | | +| 4 | Keepalive format | High | OPEN | | +| 5 | Cache naming | High | OPEN | | +| 6 | Peer/Link relationship | Medium | OPEN | | +| 7 | Error handling | Medium | OPEN | | +| 8 | Memory bounds | Medium | OPEN | | +| 9 | Timer management | Medium | OPEN | | +| 10 | Init/shutdown | Medium | OPEN | | +| 11 | Config validation | Low | OPEN | | +| 12 | Default rationales | Low | OPEN | | +| 13 | Terminology | Low | OPEN | | +| 14 | Gateway details | Low | OPEN | | +| 15 | DiscoveredPeer hint | Low | OPEN | | diff --git a/docs/design/fips-architecture.md b/docs/design/fips-architecture.md new file mode 100644 index 0000000..25c073e --- /dev/null +++ b/docs/design/fips-architecture.md @@ -0,0 +1,841 @@ +# FIPS Software Architecture + +**Status**: Design Draft + +This document describes the software architecture for a FIPS node implementation, +covering core entities, state machines, transport abstractions, and configuration. + +--- + +## Overview + +FIPS is a Layer 3 mesh routing protocol that provides IPv6 connectivity over +heterogeneous link types. A FIPS node exposes a TUN interface to local applications, +routes packets via a spanning tree topology, and uses Bloom filters for efficient +reachability lookup. + +The architecture is event-driven with multiple focused state machines rather than +a single monolithic event handler. Control protocols are designed for eventual +consistency, tolerating packet loss without requiring acknowledgment/retry machinery. + +--- + +## Core Entities + +### Node + +The top-level entity representing a running FIPS instance. + +``` +Node +├── identity: Identity // cryptographic identity (npub/nsec) +├── config: Config // loaded configuration +├── tun: TunInterface // IPv6 interface to local applications +├── tree_state: TreeState // local view of spanning tree +├── coord_cache: CoordCache // address → coordinates for routing +├── transports: HashMap +├── links: HashMap +└── peers: HashMap +``` + +### Identity + +Cryptographic identity using Nostr keys (secp256k1). + +``` +Identity +├── npub: PublicKey // public key (bech32: npub1...) +├── nsec: SecretKey // secret key (bech32: nsec1...) +├── node_id: NodeId // SHA-256(npub), 32 bytes +└── address: FipsAddress // IPv6 ULA derived from node_id (fd::/8) +``` + +`NodeId` is the routing identifier, derived deterministically from `npub`. +Transport addresses and FIPS identity are fully decoupled. + +### Transport + +A physical or logical interface over which links can be established. + +``` +Transport (trait) +├── transport_id: TransportId +├── transport_type: TransportType +├── config: TransportConfig +├── state: TransportState +├── mtu: u16 +│ +├── start() -> Result<()> +├── stop() -> Result<()> +├── send(addr: &TransportAddr, data: &[u8]) -> Result<()> +├── recv() -> Result<(TransportAddr, Vec)> +└── discover() -> Result> +``` + +**Transport metadata (static per type):** + +``` +TransportType +├── name: &'static str // "udp", "ethernet", "wifi", "tor" +├── connection_oriented: bool // requires link establishment? +└── reliable: bool // delivery guaranteed by transport? +``` + +Transports handle framing, fragmentation, and any transport-layer encryption +internally. The FIPS routing layer sees only FIPS packets. + +### Link + +A communication channel to a specific remote endpoint over a transport. + +``` +Link +├── link_id: LinkId +├── transport_id: TransportId +├── remote_addr: TransportAddr // opaque, transport-specific +├── direction: Inbound | Outbound +├── state: LinkState // trivial for connectionless, real for Tor +├── base_rtt: Duration // hint from transport type +└── io: ... // connection handles for connection-oriented +``` + +For connectionless transports (UDP, Ethernet, WiFi), links are lightweight—just +`(transport_id, remote_addr)` with implicit "always connected" state. + +For connection-oriented transports (Tor), links track real connection state and +hold I/O handles. + +**Link statistics (measured):** + +``` +LinkStats +├── packets_sent: u64 +├── packets_recv: u64 +├── bytes_sent: u64 +├── bytes_recv: u64 +├── last_recv: Timestamp +├── rtt_estimate: Duration // measured from probes +├── loss_rate: f32 // observed (meaningful for unreliable) +└── throughput_estimate: u64 // bytes/sec observed +``` + +### Peer + +An authenticated remote FIPS node, reachable via a link. + +``` +Peer +├── node_id: NodeId // routing identity +├── npub: PublicKey // cryptographic identity +├── link_id: LinkId // which link reaches this peer +├── state: PeerState // lifecycle state +│ +│ // Spanning tree +├── declaration: ParentDeclaration // their latest +├── ancestry: Vec // their path to root +│ +│ // Bloom filter (inbound—what's reachable through them) +├── inbound_filter: BloomFilter +├── filter_sequence: u64 +├── filter_ttl: u8 +├── filter_received_at: Timestamp +├── pending_filter_update: bool // we owe them an update +│ +│ // Statistics +└── link_stats: LinkStats +``` + +There is a one-to-one mapping between peers and links. If the same remote node +is reachable via multiple transports, that would be multiple Peer entries (though +for initial implementation, we assume single transport per peer). + +--- + +## Spanning Tree State + +### Per-Node State + +``` +TreeState +├── my_declaration: ParentDeclaration +│ ├── node_id: NodeId +│ ├── parent_id: NodeId // self if root candidate +│ ├── sequence: u64 // monotonic +│ └── signature: Signature +├── my_coords: Vec // [self, parent, grandparent, ..., root] +├── root: NodeId // elected root (smallest reachable node_id) +└── coord_cache: HashMap // for data packet routing +``` + +### Per-Peer State + +From each peer, we receive and store: +- Their `ParentDeclaration` +- Their `ancestry` (path from peer to root) + +This provides their tree coordinates for routing decisions. + +### Bounded State + +Each node's TreeState contains O(P × D) entries, not O(N): +- P = direct peer count +- D = tree depth + +A node knows only: +1. Its own parent declaration +2. Direct peers' parent declarations +3. Ancestry chains from each peer to root + +Nodes do NOT know about other subtrees—only paths toward root. + +--- + +## Bloom Filter State + +### Per-Node State + +``` +BloomState +├── own_node_id: NodeId // always included in outgoing filters +├── leaf_dependents: HashSet // leaf-only nodes we speak for +├── is_leaf_only: bool // if true, no filter processing +└── update_debounce: Duration // rate limit outgoing updates +``` + +### Per-Peer State + +Stored on Peer: +- `inbound_filter`: what they advertise to us (4KB Bloom filter) +- `filter_sequence`: freshness/dedup +- `filter_ttl`: remaining propagation hops +- `filter_received_at`: for staleness detection + +### Computed (On-Demand) + +Outgoing filter to peer Q is computed, not stored: + +``` +outbound_filter(Q) = + own_node_id + ∪ leaf_dependents + ∪ { entries from peer[P].inbound_filter for all P ≠ Q where filter_ttl > 0 } +``` + +TTL is decremented on contributed entries. Recomputation is cheap (4KB filter, +7 hashes) so on-demand is preferred over cache invalidation complexity. + +--- + +## State Machines + +### Transport Lifecycle + +``` + Configured ──► Starting ──► Up ──► Down + │ │ │ + v v │ + Failed ◄────────────┘ +``` + +- `Configured`: in config, not started +- `Starting`: initialization in progress (instant for UDP, slow for Tor) +- `Up`: ready for links +- `Down`: was up, now unavailable +- `Failed`: couldn't start + +**Events:** +- `Start` (from config policy or API) +- `Started` / `StartFailed` +- `Shutdown` +- `TransportError` + +**Cascading:** Transport down → all links over it disconnect → all peers on +those links disconnect. + +### Link Lifecycle + +**Connectionless transports (UDP, Ethernet, WiFi):** + +Links are always implicitly "active"—no state machine needed. Link exists when +we have `(transport_id, remote_addr)`. + +**Connection-oriented transports (Tor):** + +``` +Outbound: + (connect requested) ──► Connecting ──► Connected ──► Disconnected + │ ▲ + v │ + Failed ─────────────────────────┘ + +Inbound: + (transport accepts) ──► Connected ──► Disconnected +``` + +- `Connecting`: establishing connection (circuit for Tor) +- `Connected`: ready for FIPS traffic +- `Disconnected`: was connected, now gone +- `Failed`: connection attempt failed + +### Peer Lifecycle + +``` + Discovered ──► Connecting ──► Authenticating ──► Active ──► Disconnected + │ │ │ ▲ + │ v v │ + └──────── [timeout/fail] ──────────────────────────────────┘ +``` + +- `Discovered`: known via discovery or config, no link yet +- `Connecting`: link establishment in progress (connection-oriented only) +- `Authenticating`: FIPS auth handshake in progress +- `Active`: fully integrated (has declaration, ancestry, filter) +- `Disconnected`: was active, now gone + +**Events:** + +``` +PeerEvent +├── Discovered { link_id, transport_addr, hint: Option } +├── LinkConnected +├── LinkFailed { reason } +├── AuthChallengeReceived { challenge } +├── AuthResponseReceived { response } +├── AuthSuccess +├── AuthFailed { reason } +├── TreeAnnounceReceived { declaration, ancestry } +├── FilterAnnounceReceived { filter, sequence, ttl } +├── Timeout { kind: TimeoutKind } +├── PacketReceived { ... } +└── LinkDisconnected { reason } +``` + +--- + +## Reference Transport Types + +### UDP/IP + +``` +UdpTransport +├── bind_addr: SocketAddr // e.g., 0.0.0.0:4000 +├── socket: UdpSocket +└── state: TransportState + +TransportAddr = SocketAddr // IP:port +``` + +| Property | Value | +|----------|-------| +| Connection-oriented | No | +| Reliable | No | +| MTU | 1280-1472 | +| Latency | Low (1-500ms) | +| Scope | Internet | +| Discovery | DNS-SD, Nostr, static config | +| Privileges | None | +| NAT | Hole punching possible | + +### Ethernet + +``` +EthernetTransport +├── interface: String // "eth0" +├── socket: RawSocket // AF_PACKET +├── local_mac: MacAddr +├── ethertype: u16 // FIPS ethertype +└── state: TransportState + +TransportAddr = MacAddr // 6 bytes +``` + +| Property | Value | +|----------|-------| +| Connection-oriented | No | +| Reliable | No | +| MTU | 1500 | +| Latency | <1ms | +| Scope | Local segment | +| Discovery | Multicast | +| Privileges | CAP_NET_RAW | + +### WiFi + +``` +WifiTransport +├── interface: String // "wlan0" +├── socket: RawSocket +├── local_mac: MacAddr +├── mode: Infrastructure | AdHoc | Direct +└── state: TransportState + +TransportAddr = MacAddr // same as Ethernet +``` + +| Property | Value | +|----------|-------| +| Connection-oriented | No | +| Reliable | No | +| MTU | 1500 | +| Latency | 1-10ms | +| Scope | Local segment (or Direct group) | +| Discovery | Multicast, P2P service discovery | +| Privileges | CAP_NET_RAW | + +Infrastructure and Ad-hoc modes behave like Ethernet. WiFi Direct has its own +service discovery mechanism. + +### Tor Onion + +``` +TorTransport +├── tor_client: TorClient // arti or external daemon +├── onion_service: Option +├── local_onion_addr: Option +└── state: TransportState + +TransportAddr = OnionAddr // "abc...xyz.onion:port" +``` + +| Property | Value | +|----------|-------| +| Connection-oriented | Yes | +| Reliable | Yes (stream) | +| MTU | Stream (framed) | +| Latency | 500ms-5s | +| Scope | Internet (anonymous) | +| Discovery | Nostr, static config | +| Privileges | None | +| Transport startup | Slow (30s-2min for Tor bootstrap) | + +Tor links require framing (length-prefix) over the stream. The .onion address +is independent of FIPS npub—identity verified via FIPS auth after connecting. + +### Transport Comparison + +| Aspect | UDP | Ethernet | WiFi | Tor | +|--------|-----|----------|------|-----| +| Connection | No | No | No | Yes | +| Link state machine | Trivial | Trivial | Trivial | Real | +| Address type | IP:port | MAC | MAC | .onion:port | +| Startup time | Instant | Instant | Instant | 30s-2min | +| Base RTT hint | 50ms | 1ms | 5ms | 2s | +| Framing | Datagram | Datagram | Datagram | Length-prefix | + +--- + +## Event-Driven Architecture + +The system uses multiple focused state machines rather than one giant event +handler: + +1. **Transport state machines** — one per transport instance +2. **Link state machines** — one per link (meaningful for connection-oriented) +3. **Peer state machines** — one per peer +4. **Spanning tree module** — reacts to peer events, emits announcements +5. **Bloom filter module** — reacts to peer/filter events, emits updates + +**Event flow:** + +``` +Transport + │ + ├──► DiscoveredPeer { transport_id, addr, hint } + ├──► InboundConnection { transport_id, addr, io } (connection-oriented) + └──► PacketReceived { transport_id, addr, data } + │ + v + Link + │ + ├──► LinkConnected { link_id } + ├──► LinkDisconnected { link_id, reason } + └──► FipsPacketReceived { link_id, packet } + │ + v + Peer + │ + ├──► AuthSuccess { peer_id } + ├──► TreeAnnounceReceived { peer_id, decl, ancestry } + └──► FilterAnnounceReceived { peer_id, filter, seq, ttl } + │ + v + TreeState / BloomState +``` + +Timers drive keepalives, timeouts, and periodic refresh (debounced announcements). + +--- + +## Protocol Self-Healing Design + +Control protocols tolerate packet loss without ack/retry machinery: + +### TreeAnnounce + +- Monotonic sequence numbers (receiver keeps highest, ignores stale/dup) +- Full state (declaration + ancestry), not deltas +- Periodic refresh ensures convergence +- Lost announcement? Next one carries same or newer state + +### FilterAnnounce + +- Full filter replacement with sequence number +- Periodic refresh +- Debounced on rapid changes +- Lost announcement? Peer has stale filter until next update + +### LookupRequest/LookupResponse + +- Request-response pattern +- Lost request/response → sender times out, retries at application level + +### SessionSetup/SessionAck + +- Same as lookup—sender retries on timeout +- Lost setup → first data packet fails → triggers re-establishment + +This gossip-style eventual consistency is simpler and avoids the complexity of +per-message reliability over unreliable links. + +--- + +## Leaf-Only Operation + +Leaf-only mode enables constrained devices (sensors, battery-powered nodes, mobile +devices) to participate in FIPS without the overhead of full mesh routing. + +### Architectural Subset + +A leaf-only node uses a minimal subset of the full architecture: + +``` +LeafOnlyNode +├── identity: Identity // full (npub, nsec, node_id, address) +├── config: Config // simplified +├── tun: TunInterface // full (provides IPv6 to local apps) +├── transport: Transport // one +├── link: Link // one +└── upstream_peer: Peer // one (simplified) +``` + +### What's Required + +| Component | Usage | +|---------------------|------------------------------------------| +| Identity | Full—same cryptographic identity model | +| TUN interface | Full—provides IPv6 to local applications | +| Transport | One instance (to reach upstream peer) | +| Link | One instance (to upstream peer) | +| Peer | One instance (upstream), with auth only | +| Peer authentication | Full—must prove identity to upstream | +| Packet send/receive | Full | + +### What's Not Required + +| Component | Reason | +|--------------------|--------------------------------------------------| +| TreeState | No tree participation; upstream handles routing | +| ParentDeclaration | Doesn't announce position to network | +| Bloom filters | Upstream peer handles reachability | +| Filter computation | N/A | +| CoordCache | Doesn't route for others | +| Discovery protocol | Upstream peer handles lookups | +| Multiple peers | Single upstream by design | +| Session caching | Tunnels everything to upstream | +| Transit routing | Never forwards for others | + +### Simplified Peer Structure + +The upstream peer entry for a leaf-only node: + +``` +UpstreamPeer (leaf-only) +├── node_id: NodeId +├── npub: PublicKey +├── link_id: LinkId +├── state: PeerState // auth lifecycle only +└── link_stats: LinkStats // for keepalive/timeout + +// NOT present: +// - declaration, ancestry (no tree participation) +// - inbound_filter, filter_* (no Bloom filters) +``` + +### Routing Behavior + +``` +Outbound (local app → network): + TUN → upstream peer (unconditionally) + +Inbound (network → local app): + upstream peer → TUN (if dest == self) + upstream peer → DROP (if dest ≠ self, never transit) +``` + +The leaf-only node doesn't make routing decisions—it tunnels everything to/from +its upstream peer. + +### Upstream Peer Responsibilities + +The upstream peer (a full participant) handles: + +- Including leaf-only node in its Bloom filter +- Responding to LookupRequests for the leaf-only node +- Forwarding packets to/from the leaf-only node +- The leaf-only node appears as an entry in the upstream's `leaf_dependents` set + +### State Machine (Simplified) + +Only peer lifecycle matters: + +``` + Configured ──► Connecting ──► Authenticating ──► Active ──► Disconnected + │ │ │ + v v │ + [fail] ─────────────────────────────────────────┘ +``` + +No TreeAnnounce or FilterAnnounce processing in the Active state. + +### Configuration (Leaf-Only Subset) + +``` +# Required +node.identity.nsec # or auto-generated +node.leaf_only = true +node.tun.device +node.tun.mtu + +# One transport +transport.udp.enabled = true # or ethernet, wifi, tor +transport.udp.bind_addr + +# One peer (the upstream) +peers[0].npub # required: upstream identity +peers[0].addresses[0].type # transport type +peers[0].addresses[0].addr # how to reach them +peers[0].connect_policy = auto_connect + +# Timeouts +peer.auth.timeout +peer.keepalive.interval +peer.keepalive.timeout +peer.reconnect.* +``` + +Parameters NOT relevant to leaf-only operation: + +``` +tree.* # no tree participation +filter.* # no Bloom filters +discovery.* # upstream handles +session.* # no session caching +transport.*.discovery.* # single configured peer +transport.*.auto_connect # single configured peer +``` + +### Resource Comparison + +| Resource | Full Participant | Leaf-Only | +|---------------------|---------------------|-----------| +| RAM (Bloom filters) | d × 4KB (d = peers) | 0 | +| RAM (coord cache) | 10K-100K entries | 0 | +| RAM (tree state) | O(P × D) entries | 0 | +| Bandwidth (idle) | < 1 KB/sec | Near zero | +| CPU (filter ops) | Moderate | None | +| Peers | Multiple | One | + +### Use Cases + +- **IoT sensors**: Send telemetry, receive commands +- **Mobile devices**: Battery/bandwidth constraints +- **Privacy-conscious**: Don't see others' traffic +- **Monitoring nodes**: Observe network, don't route +- **Embedded systems**: Limited RAM/CPU + +--- + +## Configuration + +### Peer Configuration + +Peers are configured separately from transports: + +``` +PeerConfig +├── npub: PublicKey // required: who is this +├── alias: Option // human-readable label +├── addresses: Vec // how to reach them +└── connect_policy: ConnectPolicy + +PeerAddress +├── transport_type: TransportType // "udp", "ethernet", "tor", etc. +├── addr: String // transport-specific, parsed by driver +└── priority: u8 // preference order + +ConnectPolicy +├── AutoConnect // connect on startup +├── OnDemand // connect when traffic needs routing +└── Manual // wait for explicit API call +``` + +### Transport Configuration + +``` +TransportConfig +├── transport_type: TransportType +├── driver_config: DriverConfig // type-specific +├── start_policy: StartPolicy +├── discovery_enabled: bool +└── auto_connect: bool // auto-connect to discovered peers +``` + +### Discovery + +Discovery is per-transport: +- Transports emit `DiscoveredPeer { addr, hint }` events +- Node matches against known peer configs or creates "unknown peer" entries +- Policy (`auto_connect`, per-peer `connect_policy`) determines action + +--- + +## Configuration Reference (sysctl-style) + +### Node Identity + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `node.identity.nsec` | string | (generated) | Secret key (nsec1... or hex) | + +### TUN Interface + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `node.tun.device` | string | "fips0" | TUN device name | +| `node.tun.mtu` | u16 | 1280 | TUN interface MTU | + +### Spanning Tree + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `tree.announce.interval` | duration | 30s | Periodic TreeAnnounce refresh | +| `tree.announce.on_change` | bool | true | Immediate announce on parent change | +| `tree.parent.hold_time` | duration | 10s | Min time before switching parent | +| `tree.parent.hysteresis` | f32 | 0.1 | Cost improvement threshold to switch | + +### Bloom Filters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `filter.size` | u32 | 32768 | Filter size in bits (4KB) | +| `filter.hash_count` | u8 | 7 | Number of hash functions | +| `filter.scope` | u8 | 2 | TTL for filter propagation (K) | +| `filter.refresh.interval` | duration | 60s | Periodic FilterAnnounce refresh | +| `filter.update.debounce` | duration | 500ms | Min interval between updates | +| `filter.stale.threshold` | duration | 300s | Consider peer's filter stale | + +### Discovery Protocol + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `discovery.lookup.ttl` | u8 | 8 | Initial TTL for LookupRequest | +| `discovery.lookup.timeout` | duration | 10s | Timeout waiting for response | +| `discovery.lookup.retry_count` | u8 | 3 | Retries before giving up | +| `discovery.cache.max_entries` | u32 | 10000 | Route cache size | +| `discovery.cache.ttl` | duration | 300s | Cached coordinates expiry | + +### Session Management + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `session.setup.timeout` | duration | 5s | SessionSetup ack timeout | +| `session.cache.max_entries` | u32 | 50000 | Coord cache size (per router) | +| `session.cache.ttl` | duration | 300s | Cached coordinates expiry | +| `session.refresh.interval` | duration | 240s | Proactive session refresh | + +### Peer Defaults + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `peer.auth.timeout` | duration | 10s | Auth handshake timeout | +| `peer.auth.timeout_tor` | duration | 60s | Auth timeout for Tor links | +| `peer.keepalive.interval` | duration | 30s | Keepalive probe interval | +| `peer.keepalive.timeout` | duration | 90s | Declare peer dead after silence | +| `peer.reconnect.policy` | enum | backoff | none, immediate, backoff | +| `peer.reconnect.delay_initial` | duration | 1s | Initial reconnect delay | +| `peer.reconnect.delay_max` | duration | 300s | Maximum reconnect delay | +| `peer.reconnect.max_attempts` | u32 | 0 | 0 = unlimited | + +### Transport: UDP + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `transport.udp.enabled` | bool | true | Enable UDP transport | +| `transport.udp.bind_addr` | string | "0.0.0.0:4000" | Bind address | +| `transport.udp.discovery.enabled` | bool | true | Enable discovery | +| `transport.udp.discovery.dns_sd` | bool | false | Use DNS-SD discovery | +| `transport.udp.discovery.nostr_relays` | list | [] | Relays for peer discovery | +| `transport.udp.auto_connect` | bool | true | Connect to discovered peers | +| `transport.udp.base_rtt` | duration | 50ms | RTT hint for timeouts | + +### Transport: Ethernet + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `transport.ethernet.enabled` | bool | false | Enable Ethernet transport | +| `transport.ethernet.interface` | string | "eth0" | Interface name | +| `transport.ethernet.ethertype` | u16 | 0x88b5 | FIPS EtherType | +| `transport.ethernet.discovery.enabled` | bool | true | Enable multicast discovery | +| `transport.ethernet.discovery.multicast_addr` | string | "33:33:00:00:ff:05" | Discovery multicast MAC | +| `transport.ethernet.auto_connect` | bool | true | Connect to discovered peers | +| `transport.ethernet.base_rtt` | duration | 1ms | RTT hint for timeouts | + +### Transport: WiFi + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `transport.wifi.enabled` | bool | false | Enable WiFi transport | +| `transport.wifi.interface` | string | "wlan0" | Interface name | +| `transport.wifi.mode` | enum | infrastructure | infrastructure, adhoc, direct | +| `transport.wifi.discovery.enabled` | bool | true | Enable discovery | +| `transport.wifi.auto_connect` | bool | true | Connect to discovered peers | +| `transport.wifi.base_rtt` | duration | 5ms | RTT hint for timeouts | + +### Transport: Tor + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `transport.tor.enabled` | bool | false | Enable Tor transport | +| `transport.tor.mode` | enum | embedded | embedded, external | +| `transport.tor.control_port` | string | "127.0.0.1:9051" | External daemon control | +| `transport.tor.onion_service.enabled` | bool | true | Publish onion service | +| `transport.tor.onion_service.port` | u16 | 4000 | Onion service port | +| `transport.tor.discovery.enabled` | bool | true | Enable discovery | +| `transport.tor.discovery.nostr_relays` | list | [] | Relays for peer discovery | +| `transport.tor.auto_connect` | bool | false | Connect to discovered peers | +| `transport.tor.base_rtt` | duration | 2s | RTT hint for timeouts | +| `transport.tor.startup_timeout` | duration | 180s | Tor bootstrap timeout | + +### Adaptive Timeouts + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `timeout.adaptive.enabled` | bool | true | Use measured RTT for timeouts | +| `timeout.adaptive.rtt_multiplier` | f32 | 3.0 | Timeout = RTT × multiplier | +| `timeout.adaptive.min` | duration | 100ms | Minimum timeout | +| `timeout.adaptive.max` | duration | 60s | Maximum timeout | + +### Leaf-Only Mode + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `node.leaf_only` | bool | false | Operate as leaf-only node | + +--- + +## References + +- [fips-design.md](fips-design.md) — Overall FIPS protocol design +- [fips-links.md](fips-links.md) — Link layer requirements +- [fips-routing.md](fips-routing.md) — Routing, Bloom filters, discovery +- [spanning-tree-dynamics.md](spanning-tree-dynamics.md) — Tree protocol dynamics