Files
fips/docs/tutorials/join-the-test-mesh.md
Johnathan Corgan 5abf9a9325 docs: four-section /docs/ restructure with new-user content, accuracy pass, and gateway feature-set rewrite
Restructures /docs/ by reader purpose (tutorials, how-to,
reference, design), adds the new-user-progression and
operator-recipe content the prior layout lacked, runs an
accuracy pass against current source across the pre-existing
design docs, and rewrites the gateway feature-set documentation
end-to-end around its actual operational profile (a niche
feature designed for systems already serving DHCP/DNS to a
LAN, with two independent halves — outbound LAN→mesh, inbound
mesh→LAN — sharing one nftables table, one binary, and one
control socket). Top-level README and getting-started rewritten
around two equally-weighted deployment modes (overlay on
existing IP networks; ground-up over non-IP transports).

## Additions

- 11 new tutorials in docs/tutorials/: an 8-step new-user
  progression from single-daemon test-mesh peering through
  to a ground-up two-device mesh, an IPv6-adapter side-trip
  walkthrough, an Advanced Tutorials index, and a hand-held
  OpenWrt walk-through for fips-gateway deployment that
  exercises both halves of the feature.
- 12 new how-tos in docs/how-to/: firewall activation,
  Nostr discovery (resolve / advertise / open across five
  scenarios), Tor onion (directory + control_port modes),
  UDP buffer tuning, unprivileged-user setup, persistent
  identity, host aliases, Bluetooth LE peering, MTU
  diagnostics, manual Linux-host gateway deployment (covers
  both halves), gateway troubleshooting (organised by half),
  and a section index.
- 9 new reference docs in docs/reference/: configuration,
  wire formats, control-socket protocol, four CLI references
  (fips, fipsctl, fipstop, fips-gateway), security posture
  matrix, and Nostr events catalog. Configuration and
  wire-formats are renamed-and-extended from prior design/
  versions; the other seven are net-new.
- 6 new design docs: fips-concepts, fips-architecture, and
  fips-prior-work split out of the deleted fips-intro.md;
  consolidated fips-mmp and fips-mtu aggregations; and a
  new generic port-advertisement-and-nat-traversal doc
  (Nostr-signaled port advertisement plus UDP NAT-traversal
  protocol, FIPS as an example implementation, suitable for
  eventual NIP submission).
- Top-level docs/getting-started.md walking through the
  binary-installer-only Install story.
- packaging/common/hosts pre-populated with the eight public
  test-mesh nodes so shortnames resolve out of the box on
  every fresh install.

## Changes

- 23 wire-format diagrams relocated to reference/diagrams/
  alongside the wire-formats move.
- 4 design diagrams corrected against source code
  (fips-protocol-stack, fips-identity-derivation,
  fips-coordinate-discovery, fips-routing-decision).
- 10 pre-existing design docs reconciled with current
  source. Numeric corrections: stale link-MMP report bounds
  (now [1s, 5s] with 200 ms cold-start floor); UDP default
  MTU (now 1280, IPv6 minimum); node_addr formula
  (SHA-256(pubkey)[..16]); Noise patterns (IK at link, XK
  at session); peer-ACL semantics (strict allowlist requires
  ALL in peers.deny); daemon DNS upstream ([::1]:5354);
  on-the-wire bloom-filter size (1,071 bytes); obsolete
  Cargo-feature references (PR #79 dropped them) removed.
- Transport framing tightened across the docs: TCP is for
  UDP-filtered networks (not NAT traversal); Tor is a
  deployment mode (not failover); WebSocket dropped (not a
  shipped FIPS transport); WiFi promoted to Implemented via
  Ethernet in infrastructure mode; classic-Bluetooth row
  removed (BLE is the only Bluetooth-mode transport).
- docs/design/fips-gateway.md rewritten end-to-end to lead
  with the niche-feature framing and the two-halves
  structure. Title moved from "FIPS Outbound LAN Gateway"
  to "FIPS Gateway"; architecture section describes the
  common machinery (the fips-gateway service, the nftables
  table, the control socket) before splitting into separate
  "Outbound Half" and "Inbound Half" sections of equal
  weight; security considerations split per-half; no Future
  Work section (speculative directions live in the project
  tracker, not in protocol design docs). Inbound port
  forwarding is a first-class half rather than a buried
  "Implemented Extensions" subsection.
- Gateway terminology unified across all gateway docs as a
  separate Linux service running alongside the fips daemon
  (its own systemd unit / OpenWrt init script). Container-
  pattern terms (sidecar) are reserved for the
  Docker/Kubernetes sidecar deployment examples — the
  testing/sidecar/ tree, examples/k8s-sidecar/,
  examples/sidecar-nostr-relay/,
  examples/wireguard-sidecar-macos/, and the related
  CHANGELOG / top-level README entries — where the term
  carries its standard container meaning.
- Net-new design body content: rekey section in
  fips-mesh-layer (Noise IK msg1/msg2 over the established
  link, K-bit cutover, drain window, smaller-NodeAddr-wins
  tie-breaker on dual-init); Mesh Size Estimation and
  Antipoison FPR Cap sections in fips-bloom-filters;
  Mesh-Interface Query Filter subsection in
  fips-ipv6-adapter; failure-suppression knobs and clock-
  skew tolerance in fips-nostr-discovery; loop-rejection
  and mid-chain ancestor swap added to spanning-tree
  propagation / stability rules; Priority Chain in
  fips-mesh-operation renumbered to match the
  routing-decision diagram.
- Top-level README: dropped the stale nostr-discovery
  cargo-feature parenthetical. docs/README.md and the four
  section READMEs (tutorials, how-to, reference, design)
  refreshed for the new structure; index rows reflect both
  halves of the gateway feature and the new fips-gateway
  CLI reference.
- Cargo.toml [package.metadata.deb] assets path updated for
  the fips-security.md move; .gitignore /reference/ rule
  anchored to repo root so docs/reference/ is trackable.
- packaging/openwrt-ipk/files/etc/fips/fips.yaml
  configuration-doc URL updated to the new
  docs/reference/configuration.md location.

## Deletions

- docs/design/fips-intro.md (split into the three new intro
  design docs).
- docs/design/document-relationships.svg (orphan, no longer
  referenced).
- docs/proposals/ tree removed; the only proposal it
  contained (the Nostr UDP hole-punch protocol) was
  rewritten as the new generic
  design/port-advertisement-and-nat-traversal.md.
2026-05-08 03:02:12 +00:00

12 KiB

Join the FIPS Test Mesh

In this tutorial you will connect your FIPS daemon to a public test peer over UDP, watch the link come up, and reach the peer's mesh address from your machine. By the end you will have seen one complete end-to-end flow — config, handshake, live link, traffic — for a real peer somewhere out on the public internet.

The whole exercise should take about ten minutes. If you have already worked through getting-started.md and have the fips daemon running on your host, you have everything you need.

What you'll build

   ┌────────────────────┐         UDP/IPv4         ┌──────────────────────┐
   │   your fips node   │ ──────────────────────── │      test-us01       │
   │   ephemeral npub   │  test-us01.fips.network  │  npub1qmc3...zel98   │
   │   fips0  fd97:..:Y │           :2121          │   fips0  fd97:..:T   │
   └────────────────────┘                          └──────────────────────┘

Your daemon will peer with one of the public test nodes the project maintains. test-us01 has a stable DNS name, listens on UDP/2121, and is reachable from any network that permits arbitrary outbound UDP.

Peer vs. node. In FIPS terminology, a peer is a node you have a direct link to — same Noise IK handshake, same transport socket. A node is any participant on the mesh, whether you peer with it directly or reach it through one or more hops via your peer's connections. Peering is a local configuration choice; reachability is mesh-wide. One good peer connects you to everyone the rest of the mesh connects to.

After the link to test-us01 establishes, your daemon's fips0 adapter can reach test-us01 itself and — through test-us01's connections — any other node on the test mesh, exactly as if you had a direct connection to each of them.

About the test mesh. The project maintains a small roster of public test nodes (test-us01 through test-uk01) intended for new-user on-ramps and integration testing. They accept inbound peering from arbitrary npubs without prior coordination. A future reference doc will list the full roster; for this tutorial you only need test-us01 as your peer, and test-us02 later on as a second mesh destination to demonstrate forwarding.

Step 1: Confirm the daemon is running

sudo systemctl status fips

Expect active (running). If it is not running, the getting-started guide covers installation and service management. While you're checking, note your daemon's current npub:

sudo fipsctl show status

Look for the npub field. With the default ephemeral-identity config, this regenerates on every restart — that is fine for the tutorial. test-us01 admits any inbound npub.

Step 2: Add a static peer to the daemon config

Edit /etc/fips/fips.yaml. Find the line that reads peers: [] and replace it with:

peers:
  - npub: "npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98"
    alias: "test-us01"
    addresses:
      - transport: udp
        addr: "test-us01.fips.network:2121"
    connect_policy: auto_connect

What each field does:

  • npub — the canonical Nostr public key of test-us01. This is who your daemon will mutually authenticate with over Noise IK.
  • alias — a short name your daemon will use when referring to this peer in logs and fipsctl show peers output. Optional.
  • addresses — one or more transport endpoints. UDP on the published hostname and port is the most direct path.
  • connect_policy: auto_connect — your daemon initiates an outbound connection rather than waiting for the peer to reach in.

Step 3: Restart the daemon

sudo systemctl restart fips

Watch the daemon's journal as it comes back up and dials the peer:

sudo journalctl -u fips -f

Within a few seconds you should see lines mentioning:

  • An outbound connection attempt to test-us01 or test-us01.fips.network:2121
  • A handshake completion (a "Noise IK link handshake complete" style line, or "peer authenticated" with the test-us01 npub)
  • An MMP / link metrics entry naming test-us01

If the handshake does not complete within roughly 30 seconds, jump to Troubleshooting below.

sudo fipsctl show peers

Expect one entry whose alias is test-us01. Useful fields:

  • connectivity — should be active / authenticated.
  • transport_addr — the resolved UDP endpoint your daemon is using to reach test-us01.
  • transport_typeudp.
  • mmp.srtt_ms — appears once the first MMP report has been exchanged. This is your round-trip time to test-us01.

The transport view confirms your UDP listener and the peer mapping:

sudo fipsctl show transports

Step 5: Ping your peer

test-us01's mesh address derives from its npub. Address it as <npub>.fips and your daemon's local DNS responder will translate that to its fd97:... mesh address.

First see the resolved address:

dig npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips AAAA +short

You should see one fd97:... line.

Now ping it:

ping6 -c 4 npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips

Expect four replies. The first packet may take noticeably longer than subsequent ones — that round trip includes destination discovery, FSP session establishment, and the proactive path-MTU probe. After that, the RTT settles to a steady value reflecting the path between your host and test-us01.

This confirms the direct link works. So far, though, you have only reached the peer you configured. The next step demonstrates the mesh-wide reach that peering buys you.

Step 6: Reach a different node through the mesh

test-us02 is another public test node. You did not add it to your peers: block — your daemon has no direct link to it. But because test-us01 participates in the same mesh and has its own connections to other nodes, your daemon can reach test-us02 through test-us01 without any additional configuration.

ping6 -c 4 npub10yffd020a4ag8zcy75f9pruq3rnghvvhd5hphl9s62zgp35s560qrksp9u.fips

Same form, different npub. Expect replies. The packets travel from your daemon to test-us01 over the direct UDP link, then onward through test-us01 (and possibly other test-mesh nodes) to reach test-us02's fips0 adapter. Replies retrace the path.

This is the central FIPS guarantee: peering is local, but reachability is mesh-wide. You only need one good peer to talk to everyone else they (transitively) talk to.

If the test-us02 ping fails while the test-us01 ping succeeded, the test mesh's routing between those two nodes is momentarily unhealthy — try again in a minute, or pick a different test node from the roster. The link to your peer is unaffected.

What you've learned

You now have a single FIPS node connected to one peer in the public test mesh, with reach to every node that mesh routes you to. You have seen:

  • Identity. Your daemon's ephemeral keypair authenticated to test-us01 over Noise IK without either side trusting anyone in advance.
  • Transports. A UDP socket on your host carries authenticated, encrypted mesh frames to your peer. No central server, no VPN concentrator.
  • Peering vs. reachability. You configured one peer (test-us01) and got reach to a second node (test-us02) for free, through the mesh. The same shape extends to every other node test-us01 can reach.
  • Naming. The local .fips resolver translated npub-form hostnames into their fd97:... mesh addresses with no external DNS traffic.
  • End-to-end. ICMPv6 traffic over the FIPS data plane reached both destinations and came back, end-to-end encrypted along every link layer in the path.

By the way: shortnames. Those long npub1...fips destinations are the canonical addresses, but the installer ships an /etc/fips/hosts file with shortname entries for the public test mesh, so test-us01.fips and test-us02.fips resolve to the same addresses without typing 80 characters of bech32. You can add your own entries too. See ../how-to/host-aliases.md. The rest of the tutorials use shortnames where they're available.

Troubleshooting

If the handshake does not complete:

  • Outbound UDP may be blocked. Some networks filter arbitrary outbound UDP or block return traffic. From a UDP-filtered network you cannot reach peers that only publish UDP endpoints — your reachable peers are limited to those that accept incoming TCP (outbound TCP is typically allowed even on networks that block UDP). The test-mesh nodes publish a TCP endpoint on port 443 for exactly this case; replace the udp entry in the peer's addresses: block with the TCP equivalent:

    addresses:
      - transport: tcp
        addr: "test-us01.fips.network:443"
    

    Restart the daemon and re-check fipsctl show peers. The link will be slower than UDP but is the supported transport for restrictive egress environments.

  • Confirm the testnode is reachable at the IP layer. Run dig +short test-us01.fips.network to confirm DNS, then nc -uvz test-us01.fips.network 2121 to confirm UDP reachability.

  • Confirm your config parsed. sudo journalctl -u fips -n 50 near the daemon-start time will show config-load lines and any parse errors.

  • Time skew. A heavily skewed system clock can make signature validation fail. timedatectl status should show the system clock as synchronized.

What's next

These are the natural follow-on tutorials in the new-user progression. Some are still being written and will appear alongside this one in the tutorials/ directory.

  • Make your node's identity persistent. persistent-identity walks through pinning the daemon to a stable Nostr keypair so your npub does not change across restarts — the prerequisite for other operators adding you to their peers: blocks.

  • Resolve peers via Nostr. resolve-peers-via-nostr is the smallest useful step toward Nostr-mediated discovery: configure a peer by npub alone and let the daemon look up the current endpoint from public relays. The first of three tutorials covering Nostr discovery; the others — advertise-your-node and open-discovery — round out the publish and ambient-consume sides.

  • Trace a connection end-to-end. ipv6-adapter-walkthrough walks the data path from a .fips DNS query through session setup to the far-side TUN adapter, using fipstop and fipsctl to observe each step.

  • Reach services on other mesh nodes. reach-mesh-services generalizes the ping6 you just ran to any IPv6-capable tool — nc, traceroute6, curl, ssh — addressed by <npub>.fips. The point is that the FIPS data plane is just IPv6; applications don't need to know they're on a mesh.

  • Host a service of your own. host-a-service walks through bringing up a small HTTP server bound to fips0 so mesh nodes can reach it, with a deliberate exposure decision (mesh-only vs every interface), the mesh firewall, and a brief signpost to the separate, unrelated peer ACL (which controls who may peer with your node, not what they can reach on your fips0).

  • ground-up-mesh — Bring up two devices on a shared physical link — Ethernet, WiFi, or Bluetooth — with no pre-existing IP infrastructure. The second deployment mode of FIPS, coexisting on the same daemon as the overlay peer to test-us01 you just configured.

For "what just happened, in detail":