laantungir/fips
1
0
Fork 0
mirror of https://github.com/jmcorgan/fips.git synced 2026-07-22 15:52:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: laantungir:v0.4.0
Branches Tags
laantungir:master laantungir:next laantungir:feat/openwrt-access-ssid laantungir:maint laantungir:feat/openwrt-packaging laantungir:feat/platform-peer-queue laantungir:ble-v2
laantungir:v0.4.1 laantungir:v0.4.0 laantungir:v0.4.0-rc2 laantungir:v0.4.0-rc1 laantungir:v0.3.0 laantungir:v0.2.1-rel laantungir:v0.2.1 laantungir:v0.2.0-rel laantungir:untagged-837413e700b3aaa62052 laantungir:v0.1.0
...
compare: laantungir:v0.4.1
Branches Tags
laantungir:next laantungir:feat/openwrt-access-ssid laantungir:master laantungir:maint laantungir:feat/openwrt-packaging laantungir:feat/platform-peer-queue laantungir:ble-v2
laantungir:v0.4.1 laantungir:v0.4.0 laantungir:v0.4.0-rc2 laantungir:v0.4.0-rc1 laantungir:v0.3.0 laantungir:v0.2.1-rel laantungir:v0.2.1 laantungir:v0.2.0-rel laantungir:untagged-837413e700b3aaa62052 laantungir:v0.1.0

18 Commits
v0.4.0 ... v0.4.1

Author SHA1 Message Date
Johnathan Corgan
15db6471db Set version to 0.4.1 and correct the v0.4.0 release date 
Drop the -dev suffix from the package version and refresh the lockfile.
The version is single-sourced from CARGO_PKG_VERSION, so build.rs and
src/version.rs need no change.

Also fix the archived v0.4.0 release notes, which still read
"2026-06-21 (provisional)" while the changelog records the actual
release date of 2026-06-27. The date was confirmed in the changelog at
the time but not in the two release-notes files that carry it, so the
shipped notes have been showing a wrong date and the word provisional
since that release. The root mirror is regenerated for v0.4.1, so only
the archived copy needed correcting.
 2026-07-19 17:59:37 +00:00
Johnathan Corgan
146d19a8d8 Add v0.4.1 changelog entry and release notes 
Back-fill the changelog for the six user-facing changes on this line
since v0.4.0: the antipoison FPR cap default, the bloom probe and
secp256k1 context performance work, the coordinate-cache and path-MTU
fixes, and the removed parent_switched counter. The Unreleased section
was empty, so all six entries are new.

Write the v0.4.1 release notes and mirror them to the root copy. The
notes lead with the FPR cap change and state plainly that this is the
second raise of that default in two releases, that it buys headroom
rather than fixing the underlying fixed-filter constraint, and that the
structural remedy is the v2 filter work. They also flag the rolling
upgrade window, where an upgraded and a not-yet-upgraded node can
disagree about mesh size, and point consumers of parent_switched at
parent_switches.

Bump the README status badge and the roadmap prose together so the two
agree.
 2026-07-19 17:21:57 +00:00
Johnathan Corgan
bda327b5f5 Raise the bloom antipoison FPR cap default to 0.20 
The inbound FilterAnnounce cap at 0.10 rejects aggregates that are
legitimately near their operating ceiling, before the network reaches
the fixed-filter capacity limit. Raise the default to 0.20, which
corresponds to fill 0.7248 at k=5, about 2,114 entries on the 1 KB
filter (Swamidass-Baldi).

Also remove the duplicate default definitions in BloomConfig. Each
default was written twice, once in impl Default and once in the serde
default function, with nothing enforcing that they agree, so a config
file that omits the key took a different path from one that sets it.
impl Default now delegates to the serde default functions, leaving a
single source of truth.
 2026-07-19 17:18:49 +00:00
Johnathan Corgan
78377208af Fix two sidecar container references that skipped the per-run suffix 
The isolation loop and the failure-log dump both built container names
without the run suffix, so under a suffixed run they addressed containers
that do not exist.

The consequence was worse than the visible failure. Two of the three
checks in that loop assert a ping FAILS, and a ping into a non-existent
container fails for the wrong reason - so both passed vacuously, and the
security assertion the loop exists for was silently a no-op. Only the
third check, which expects a ping to succeed, noticed anything was wrong.

Add a guard that fails loudly when the container is absent, so a future
naming slip cannot quietly turn these back into no-ops rather than
surfacing as one confusing failure among two false passes.

These sites were missed because the suite passes when run by hand: with no
suffix set the unsuffixed names are correct, and the defect only appears
under the automation that sets one. Verified this time with the suffix
set, which is the condition that matters: 18 of 18, container names
resolving to real containers.
 2026-07-19 07:40:07 +00:00
Johnathan Corgan
26a579b1c9 Claim a free address range for the sidecar network instead of hardcoding one 
Making the sidecar network name unique per run exposed a latent problem
rather than causing a new one. With a fixed name, compose found any
existing network and reused it, which quietly masked the fact that the
address range was hardcoded. With a unique name there is nothing to
reuse, so the range is requested every time and the run fails outright if
anything already holds it.

Claim the range instead of assuming it. The suite now tries to create its
network on a candidate range and, if the daemon reports an overlap, moves
to the next candidate. The daemon's own address pool becomes the arbiter,
so two concurrent runs cannot land on the same range at all - as opposed
to a range derived from the run identifier, which only makes a collision
unlikely, and a collision is the precise failure being avoided.

The node addresses are derived from whichever range is claimed rather than
written alongside it, including the gateway address the isolation check
probes. Moved to a range that nothing in the tree claims and that sits
outside the daemon's default pool.

Only an overlap is worth advancing on. Any other creation failure is real
and is reported with the daemon's own message, rather than being retried
sixty-four times and buried - a lesson from a failure elsewhere in the
harness this week that was undiagnosable because its error output was
discarded.

All three nodes now attach to the pre-created network as external, so
teardown removes it explicitly.

Verified: three concurrent claims take three distinct ranges; a
non-overlap error fails fast rather than looping; the suite passes 18/18;
and the claimed gateway address is confirmed pingable when not firewalled,
so the isolation assertions still mean something.
 2026-07-19 07:17:19 +00:00
Johnathan Corgan
3ebb14eda4 Make the admission-cap final peer_count read tolerate a busy daemon 
The suite red-ed with an empty final peer_count while every substantive
assertion passed: both denied peers showed inbound handshake attempts and
zero outbound responses, which is the gate this suite exists to verify.

Two pre-existing defects combined. The final count was sampled once, in
the same second the load driver issues its last round of peer restarts, so
the read can land on a daemon busy with those restarts and return nothing.
And the fallback meant to cover that could never fire, because the shell
applies it to the last stage of the pipeline, which succeeds on empty
input - so an unanswered query was indistinguishable from a genuine zero,
and reported as a cap violation.

Poll the final read for a short window instead, and give the two cases
distinct output. A real regression still fails, just after the retry
window, since the loop exits early only on the expected value. Extract the
read the two phases share rather than leaving them to drift.
 2026-07-19 06:46:50 +00:00
Johnathan Corgan
e4a854f6b0 Stop concurrent local CI runs from clobbering each other's containers 
Two runs on one host destroyed each other's containers, producing
mid-test "No such container" failures that look like real defects. The
automated builder runs a full local CI on the same box every few minutes,
so the machine is contended almost always and this has red-ed both a hand
run and an automated gate.

Two independent causes are fixed here. Container names were hardcoded, and
docker names are global rather than scoped by compose project, so two runs
collided on the same name; every name now takes an optional suffix that
the harness sets from the run id. And the cleanup sweep matched a label
shared by every run, so one run's teardown force-removed another's
containers; resources now also carry a per-run label and the sweep can be
narrowed to it.

A third hazard turned up that was not in the original report: the sidecar
suite passes explicit compose project names, which override the run-scoped
project and put it outside the shared prefix entirely. Its project names,
network, and derived container references are now scoped too.

Both are default-off. With the suffix unset, names render exactly as they
do today and a bare compose invocation is unchanged, which is what keeps
the hosted CI and the documentation correct. A cleanup run with no run id
still reaps everything, which is what a manual "clear the box" wants.

The literal-name sweep was not sufficient: six scripts build container
names dynamically from node labels, and two suites create their own
containers outside compose. Those are handled at their construction sites.

Verified: syntax check on all modified scripts; compose validation on
every modified file with the suffix both set and unset; and a synthetic
two-run reproduction that shows the old cleanup destroying a bystander run
and the new one leaving it alone.

Known gap: subnets are still hardcoded, so two concurrent full runs will
still collide on address-pool overlap. That fix reaches into topology
configs, chaos scenarios, diagrams and production source, so it is left
for its own change rather than half-done here.
 2026-07-19 06:41:25 +00:00
Johnathan Corgan
7a97599921 testing: floor the rekey second-cutover wait at the log-analysis threshold 
The rekey suites' Phase 4 polled until the initiator-cutover count
reached its value at phase entry plus two, while Phase 6 asserts an
absolute minimum of four. On an unloaded host the ping phases can
outrun the jittered first rekey cycle, so Phase 4 can begin with a
single cutover on the books, release at three, and leave Phase 6 to
lose a sub-second race against the remaining first-cycle cutovers.
A rekey-outbound-only run failed exactly this way: five counted
cutover lines by t=32.6s of container life, with the Phase-6 count
snapshot at t~32.4s seeing three.

Floor the Phase-4 wait target at four so a released wait guarantees
the Phase-6 assertion — the docker-logs count is monotone over a
container's lifetime. A genuine rekey stall still times out the
bounded wait and fails Phase 6 with the real count.

Also drop the second-cycle framing from the comment, the phase
banner, and the assertion description: with rekey timers resetting at
each cutover a second cycle cannot begin inside this test's window,
so the counted events are first-cycle cutovers spread across the
topology's links, emitted once or twice per completed link rekey
depending on which side's promotion path logs them.
 2026-07-19 04:12:17 +00:00
Johnathan Corgan
fbb4fb8879 testing: wait for both second-cycle rekey cutovers before asserting the count 
Phase 4 of the rekey suite waited for only one more initiator cutover
(guaranteeing three) while Phase 6 asserts at least four. The fourth
cutover then had to land in the brief window between the wait returning
and the Phase 6 log snapshot, so host load could push it past the window
and fail the run even though every cutover completed correctly. Wait for
two more cutovers, the full second rekey cycle, matching the Phase 6
threshold, so the asserted count is guaranteed before it is checked.
 2026-07-16 00:42:04 +00:00
Johnathan Corgan
567e6a535e identity: reuse one shared secp256k1 context 
Every sign/verify/key-derive site built a fresh context via
Secp256k1::new(), which allocates a Secp256k1<All> and runs
randomization/blinding table setup on each call. Introduce one
crate-wide LazyLock<Secp256k1<All>> and reuse it across the local, peer,
and auth sites (and their tests). Behavior-neutral: identical secp256k1
API calls, only the context lifetime changes, and the shared All context
still performs the standard construction-time blinding.
 2026-07-12 16:33:53 +00:00
Johnathan Corgan
6011d233c1 discovery: keep tighter path_mtu when applying a LookupResponse 
An originator handling a LookupResponse unconditionally overwrote the
cached path_mtu_lookup entry, so a looser (larger) estimate in a later
response could clobber a tighter value already learned from a reactive
MtuExceeded or PathMtuNotification. Read-and-compare before writing and
keep the minimum, so a looser discovery estimate no longer loosens the
clamp. Add a regression test.
 2026-07-12 16:29:01 +00:00
Johnathan Corgan
cb5a32693e tree: remove redundant parent_switched metric counter 
parent_switched was incremented on the line immediately before
parent_switches at every site and never independently, so the two
counters were always identical. Drop parent_switched from TreeMetrics,
its snapshot, TreeStatsSnapshot, the show_tree fixture, and the fipstop
render, keeping parent_switches as the sole counter.
 2026-07-12 15:53:03 +00:00
Johnathan Corgan
81e4207631 bloom: compute the SHA-256 digest once per double-hash probe 
BloomFilter derived all k hash functions from one SHA-256 digest but
recomputed that digest inside the per-function loop, so every insert and
contains ran SHA-256 hash_count times (5x at the default) over the same
bytes. Hoist the digest out of the loop: base_hashes() computes it once
and returns (h1, h2), and bit_index() derives each of the k indices with
the same (h1 + k*h2) mod m arithmetic. Bit-for-bit identical output; this
is the hottest path in packet forwarding and mesh-size estimation.

Adds a test pinning the bit indices against the double-hashing formula
recomputed independently, proving the refactor is behavior-neutral.
 2026-07-11 01:22:07 +00:00
Johnathan Corgan
1d277e67c7 fix(tree): invalidate coord cache on parent-lost via peer removal 
handle_peer_removal_tree_cleanup reparents or self-roots the node when
its parent link drops, but omitted the coordinate-cache invalidation that
every other position-change path performs. Cached entries for downstream
destinations kept the node's now-stale coordinate prefix, and because
find_next_hop refreshes the TTL on every routing access, an actively
routed stale entry never self-expired — corrected only by a fresh insert.

Mirror the loop-detection branch: inside the parent-loss changed block,
invalidate both classes — invalidate_via_node (reparent) and
invalidate_other_roots (self-root). Add regression tests for a parent-link
removal that reparents (via-node entry dropped, same-root sibling
preserved) and one that self-roots (both via-node and stale old-root
entries dropped).
 2026-07-02 22:03:36 +00:00
Johnathan Corgan
2491091868 docs(testing): document ci-local.sh run isolation, preemption, and cleanup 
Add a "Running CI locally" section to testing/README.md covering the
ci-local.sh orchestrator: the per-run FIPS_CI_RUN_ID override and how it
scopes compose projects, image tags, and per-chaos-child subnets so
simultaneous runs on one host never collide; the cancellation exit codes
(130/143 vs 0/1) and how a preempting worker interprets them; and the
label-based cleanup via --reap / ci-cleanup.sh.
 2026-06-29 16:47:37 +00:00
Johnathan Corgan
965de26239 testing: make ci-local.sh preemption-safe with per-run docker isolation 
A CI worker may preempt an in-flight ci-local.sh run (SIGTERM, then SIGKILL
after a grace period) to restart on a newer commit. For that kill to be safe,
the script must clean up after itself and never let a dying run collide with
its restart. It previously had no signal handling, shared the default compose
project name across runs, and tore down each suite only at the suite end.

- Derive a per-run id (honoring FIPS_CI_RUN_ID, else short-sha+random) and
  namespace every docker resource to it: a fipsci_<run>_<suite> compose project
  per suite and per parallel chaos child, and per-run image tags
  (fips-test:<run>, fips-test-app:<run>) retagged to :latest only after both
  builds succeed so :latest never points at a half-built image.
- Install a bounded, idempotent teardown trap on SIGTERM/SIGINT (+ EXIT): reap
  parallel chaos children, then force-remove this run's docker resources via
  the new ci-cleanup.sh, wrapped in timeout so a stuck down cannot wedge it.
- Exit 143 (SIGTERM) / 130 (SIGINT), distinct from 0 (pass) / 1 (failed), so a
  preempting worker tells a cancelled run from a real failure.
- Add ci-cleanup.sh (also ci-local.sh --reap): force-removes leftover CI
  resources by the com.corganlabs.fips-ci=1 label and the fipsci_ project
  prefix, robust to however a prior run died.
- Label every per-suite docker resource so the label sweep reaps it after a
  SIGKILL regardless of network name: direct docker run/network resources, the
  sidecar compose services, and every per-suite compose network (acl-allowlist,
  boringtun, firewall, nat, static, both tor suites, and the chaos generator
  template). Parametrize the static/sidecar compose image refs so the per-run
  tags are honored.
- Give each parallel chaos child a unique /24 from 10.30.x (a new --subnet
  override on the sim CLI, assigned per-child in ci-local.sh) so parallel
  children never collide on a shared docker subnet, and a chaos net can never
  span a fixed-subnet suite (sidecar/static in 172.20.x). 10.30.x sits outside
  docker's default-address-pool range, so an auto-assigned net cannot land on
  it either; node IPs derive from the subnet, so no scenario config changes.
 2026-06-29 14:23:46 +00:00
Johnathan Corgan
ab915d0479 testing: convert remaining fixed-sleep settles to progress-aware polling 
Replace the fixed post-first-rekey and post-second-rekey settle sleeps in
the rekey integration test, and the fixed-deadline baseline wait in the
interop test, with the deterministic wait_until_connected progress-aware
polling helper already used for the initial baseline convergence.

Each converted site fails fast with structured diagnostics when the mesh
is stuck and extends its deadline only while pairwise reachability is
still climbing, removing the wall-clock settle windows that produced
intermittent connectivity failures after a rekey.
 2026-06-29 00:40:06 +00:00
Johnathan Corgan
8f30924fc7 Open 0.4.1-dev cycle on maint after v0.4.0 release 
Fast-forward maint to the v0.4.0 release and open the 0.4.x patch
series: bump the version to 0.4.1-dev and update the status badge.
 2026-06-27 18:29:34 +00:00
60 changed files with 1490 additions and 643 deletions
Expand all files Collapse all files

54
CHANGELOG.md
View File

@@ -13,6 +13,60 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
## [0.4.1] - 2026-07-19
### Changed
- `node.bloom.max_inbound_fpr` default raised from `0.10` to `0.20`. The
cap rejects inbound `FilterAnnounce` whose FPR (`fill^k`) exceeds it. On
the fixed 1 KB / k=5 filter, `0.10` corresponds to fill 0.631 (~1,630
reachable entries), and the busiest nodes' aggregates had again begun to
reach it as the mesh grew. `0.20` (fill 0.7248, ~2,114 entries) restores
headroom without materially weakening the antipoison gate: a saturated or
poisoned filter is ~100% FPR and still rejected. This is the second raise
of this cap in two releases; the fixed 1 KB filter is the underlying
constraint, and the structural remedy is the v2 filter work rather than a
further raise. A node running this default accepts announcements that a
v0.4.0 node drops, so during a rolling upgrade the two versions can
disagree about mesh size.
- Bloom filter probing computes its SHA-256 digest once per operation
rather than once per hash function. All k indices were already derived
from a single digest, but the digest was recomputed inside the
per-function loop, so every insert and membership test hashed the same
bytes `hash_count` times (5x at the default). Output is bit-for-bit
identical; this is the hottest path in packet forwarding and mesh-size
estimation.
- Identity operations reuse one shared `secp256k1` context instead of
constructing a fresh one at every sign, verify, and key-derive site.
Each construction allocated a context and ran randomization and blinding
table setup. Behavior is unchanged: the same API calls are made, only the
context lifetime differs, and the shared context still performs the
standard construction-time blinding.
### Fixed
- Spanning tree: the coordinate cache is now invalidated when the parent
link is lost through peer removal. That path reparents or self-roots the
node but omitted the invalidation every other position-change path
performs, so cached entries for downstream destinations kept the node's
now-stale coordinate prefix. Because routing access refreshes an entry's
TTL, an actively routed stale entry never self-expired and was corrected
only by a fresh insert.
- Discovery: applying a `LookupResponse` now keeps the tighter of the
cached and received `path_mtu` rather than overwriting unconditionally.
A looser estimate arriving in a later response could clobber a tighter
value already learned from a reactive `MtuExceeded` or
`PathMtuNotification`, loosening a clamp that had been correctly
tightened.
### Removed
- The `parent_switched` spanning-tree metric counter. It was incremented on
the line immediately before `parent_switches` at every site and never
independently, so the two were always identical. `parent_switches`
remains as the sole counter. Consumers reading `parent_switched` from the
control socket or `fipstop` should use `parent_switches`.
## [0.4.0] - 2026-06-27
### Added

2
Cargo.lock generated
View File

@@ -1074,7 +1074,7 @@ checksum = "9844ddc3a6e533d62bba727eb6c28b5d360921d5175e9ff0f1e621a5c590a4d5"
[[package]]
name = "fips"
version = "0.4.0"
version = "0.4.1"
dependencies = [
"arc-swap",
"bech32",

2
Cargo.toml
View File

@@ -1,6 +1,6 @@
[package]
name = "fips"
version = "0.4.0"
version = "0.4.1"
edition = "2024"
description = "A distributed, decentralized network routing protocol for mesh nodes connecting over arbitrary transports"
license = "MIT"

14
README.md
View File

@@ -3,7 +3,7 @@
![banner](docs/logos/fips_banner.png)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-1.85%2B-orange.svg)](https://www.rust-lang.org/)
[![Status](https://img.shields.io/badge/status-v0.4.0-green.svg)](#status--roadmap)
[![Status](https://img.shields.io/badge/status-v0.4.1-green.svg)](#status--roadmap)
A self-organizing encrypted mesh network built on Nostr identities,
capable of operating over arbitrary transports without central
@@ -210,12 +210,14 @@ testing/ Docker-based integration test harnesses + chaos simulation
## Status & roadmap
FIPS is at **v0.4.0**. The core protocol works end-to-end over
FIPS is at **v0.4.1** on the `maint` branch.
[v0.4.1](https://github.com/jmcorgan/fips/releases/tag/v0.4.1) has
shipped; this line carries patch-level fixes for the 0.4.x series. The
core protocol works end-to-end over
UDP, TCP, Ethernet, Tor, Nym, and Bluetooth on a global, public test
mesh of thousands of nodes. v0.4.0 builds on the v0.3.0 testing-and-polishing
track, adding the Nym mixnet transport and mDNS LAN discovery
alongside the existing Nostr-mediated peer discovery, UDP NAT
traversal, peer ACL, and packaging hardening. New wire-format work
mesh of thousands of nodes. v0.4.0 added the Nym mixnet transport and
mDNS LAN discovery alongside the existing Nostr-mediated peer discovery,
UDP NAT traversal, peer ACL, and packaging hardening. New wire-format work
continues to be staged on the `next` branch for the subsequent
release line.

393
RELEASE-NOTES.md
View File

@@ -1,302 +1,134 @@
# FIPS v0.4.0
# FIPS v0.4.1
**Released**: 2026-06-21 (provisional)
**Released**: 2026-07-19
v0.4.0 is the throughput-and-observability release on the v0.3.x wire
format. It adds two new ways for nodes to find and reach each other (the
Nym mixnet transport and opt-in mDNS LAN discovery), overhauls the data
plane for higher single-node throughput and lower per-packet CPU, moves
the entire operator read surface off the data-plane hot path so
observability stays responsive under load, ships a reworked `fipstop`
TUI, and hardens FMP and FSP rekey to be hitless under packet loss in
both directions. It also folds in the accumulated mesh-convergence,
admission-control, and packaging fixes from the maintenance line.
v0.4.1 is a maintenance release on the v0.4.x line. It raises the default
antipoison cap on inbound bloom filter announcements, removes a redundant
spanning-tree metric counter, fixes two convergence and path-MTU bugs, and
cuts per-packet CPU in the bloom and identity paths. There is no wire
format change and no new feature surface.
v0.4.0 is wire-compatible with v0.3.0. Mixed meshes interoperate; there
is no flag-day upgrade. A deployed v0.3.0 node and an upgraded v0.4.0
node peer, rekey, and route normally, so you can roll the upgrade out
across a mesh in any order.
v0.4.1 is wire-compatible with v0.4.0. Nodes can be upgraded one at a time
with no coordinated restart, though one behavior change below is worth
reading before you start a rolling upgrade.
## At a glance
- New outbound Nym mixnet transport with a single-container demo and a
new mixnet-relay example.
- Opt-in mDNS / DNS-SD discovery on the local link.
- Data-plane overhaul: off-task encrypt and decrypt worker pools, GSO,
connected-UDP send path, copy-avoidance on receive, batched macOS
receive.
- The full `show_*` read surface now serves off the receive loop, so
`fipsctl` and `fipstop` stay responsive on loaded nodes; a new
counter-only `show_metrics` query enables a Prometheus scraper at no
hot-path cost.
- Reworked `fipstop` TUI on a machine-verified render-snapshot base.
- Rekey is now hitless under loss and reordering in both directions.
- New packaging targets: an OpenWrt `.apk` for OpenWrt 25+ and a Nix
flake for reproducible from-source builds on Nix/NixOS.
- Six route-class transit counters partition forwarded traffic by its
tree relationship to the next hop, visible via `show_routing` and
`show_status`.
## What's new
### Nym mixnet transport
FIPS can now peer over the [Nym](https://nymtech.net/) mixnet for
metadata-resistant connectivity. The new `transports.nym` transport
makes outbound connections through a `nym-socks5-client` SOCKS5 proxy
that you run alongside the daemon (for example as a service running
alongside the fips daemon, or as a sidecar container). The transport
waits at startup for the nym-socks5-client to become ready before giving
up.
This is a privacy and anonymity deployment mode chosen for its own
properties. It mixes your FIPS traffic into the Nym cover-traffic
network so that link-level observers cannot correlate which mesh peers
are talking. A new `examples/sidecar-nostr-mixnet-relay/` demonstrates a
FIPS-reachable Nostr relay peered across the mixnet end to end, and a
single-container demo ships with the transport.
Enable it by adding a `transports.nym` instance and pointing it at your
running nym-socks5-client. See the transports reference for the field
set.
### mDNS LAN discovery
Nodes on a shared local link can now find each other with zero address
configuration. The opt-in `node.discovery.lan` path runs an mDNS /
DNS-SD responder and browser: each node advertises a FIPS service record
on the link and adopts the peers it discovers. This complements the
existing Nostr-mediated overlay discovery for the common case where the
peers are simply on the same LAN.
Turn it on with `node.discovery.lan.enabled: true`. `service_type` and
`scope` tune the advertised service record and which interfaces
participate. Discovery on the local link needs no relay and no STUN.
### Data-plane throughput overhaul
The receive and send paths were reworked for higher single-node
throughput and lower per-packet CPU, building on the v0.3.0
crypto-backend swap:
- **Off-task encrypt and decrypt.** Per-peer encrypt and decrypt now run
on dedicated worker tasks rather than inline on the receive loop, so a
single busy peer no longer serializes the whole node's crypto.
- **GSO and connected-UDP send.** The Linux send path uses generic
segmentation offload and a connected-UDP socket where available,
cutting syscall overhead on bulk flows.
- **Copy-avoidance on receive.** The receive hot path avoids buffer
copies it previously made per packet.
- **Batched macOS receive.** macOS gains a `recvmsg_x` batched receive,
mirroring the Linux `recvmmsg` batching from v0.3.0.
- **Shared immutable-state context and an atomic metric registry.**
Immutable per-node state moved into a single shared context, and
counters live in an atomic metric registry that the new `show_metrics`
query reads without touching the hot path.
These are all internal to the data plane and require no operator action.
### Observability off the hot path
Every read-only control query now renders from a snapshot published once
per tick into a lock-free `ArcSwap`, served from the control accept task
instead of round-tripping the data-plane receive loop. This covers
`show_status`, `show_stats_*`, `show_peers`, `show_sessions`,
`show_links`, `show_connections`, `show_transports`, `show_mmp`,
`show_tree`, `show_bloom`, `show_cache`, `show_routing`,
`show_identity_cache`, `show_acl`, `show_listening_sockets`, and the new
`show_metrics`. Only the mutating `connect` and `disconnect` commands
still reach the loop.
The practical effect: on a loaded node where the receive loop was busy,
`fipsctl` and `fipstop` queries previously stalled or timed out (the
five-second query pattern operators saw). They now answer promptly
regardless of data-plane load. Per-entity snapshots reuse unchanged rows
by pointer, so the per-tick publish cost stays bounded as peer and
session counts grow.
A new **`show_metrics`** query (surfaced as `fipsctl stats metrics`)
returns a counter-only snapshot of every metric family. It is the
enabler for a Prometheus scraper that pulls node counters at no hot-path
cost.
Six **route-class transit counters** partition transit-forwarded packets
by their tree relationship to the chosen next hop — tree-up, tree-down,
tree-down-cross, cross-link descend, cross-link ascend, and direct-peer
— and the six classes sum to `forwarded_packets`. They surface through
`show_routing` and `show_status`, and the `fipstop` routing tab is
reorganized so its two columns separate own/endpoint traffic from
forwarded/transit traffic with the tree-down-cross line visually flagged.
### Reworked fipstop TUI
`fipstop` gets a rendering, navigation, and read-surface overhaul on a
machine-verified base: a render-snapshot harness asserts the exact text
grid and per-cell style of every view against canned control-socket
output. New daemon-resolved fields surface through the snapshots,
including effective persistence, root and is-root state, a
per-transport-type peer-count map, per-peer effective depth, the root
npub, and the last-sent uptree filter fill ratio with the subtree size
estimate.
A separate fix clears a garbled-screen problem on startup and stray
bytes on quit, most visible over SSH and inside tmux: startup now forces
a full repaint before the first draw, and quit stops and joins the
stdin-poll thread before restoring the terminal, so post-raw-mode
keystrokes no longer echo onto the restored screen.
### Rekey reliability
FMP and FSP session rekey are now hitless under packet loss and
reordering in both directions:
- Inbound frames are authenticated against the pending session before
the K-bit cutover promotes it, so a spoofed or stale frame cannot
derail a rekey in progress.
- Rekey message-1 retransmission is bounded, and the link-dead heartbeat
is rekey-aware so an in-flight rekey is not mistaken for a dead link.
- FSP session rekey holds connectivity across the rekey window under
loss and reordering.
- Dual-initiation races (both peers starting a rekey at once on a
high-latency link) are desynchronized with symmetric jitter so the two
sides converge on one session rather than fighting.
- An exhausted retransmission-budget abort, an expected and self-limiting
outcome on lossy or high-latency links, is logged at debug rather than
warn.
The net operator takeaway: rekey completes cleanly without dropping
traffic, even on lossy or high-latency links, and the log no longer
cries wolf when a rekey gives up and retries.
### New packaging targets
- **OpenWrt `.apk`.** A new `.apk` package targets OpenWrt 25+, where
apk-tools is the mandatory package manager; the existing `.ipk`
continues to cover OpenWrt 24.x and earlier. It is built SDK-free,
reusing the `.ipk` cross-compile and installed-filesystem payload, and
releases publish `.apk` artifacts and checksums alongside `.ipk`. Like
the `.ipk`, the package is unsigned and installed with
`apk add --allow-untrusted`.
- **Nix flake.** A `flake.nix` at the project root builds all four
binaries (`fips`, `fipsctl`, `fips-gateway`, `fipstop`) from source on
Nix/NixOS, pinning the exact toolchain and wiring the native build
dependencies so no host setup is needed beyond Nix with flakes
enabled. It exposes `nix build`, `nix run`, a `nix develop` dev shell,
and `nix flake check`, with `flake.lock` committed for reproducibility.
- `node.bloom.max_inbound_fpr` default moves from `0.10` to `0.20`.
- The `parent_switched` metric counter is gone. Use `parent_switches`.
- Spanning tree no longer serves stale coordinates after a parent link is
lost through peer removal.
- Discovery no longer loosens a path MTU clamp it had correctly tightened.
- Bloom probing and identity operations do measurably less work per call,
with identical results.
## Behavior changes worth flagging
These affect operators on upgrade.
### The inbound filter FPR cap default doubles again
- **Bloom filter antipoison cap raised.** `node.bloom.max_inbound_fpr`
moves from 0.05 to 0.10, accepting filters with a higher derived
false-positive rate before rejecting them. This reduces spurious
filter rejections on larger meshes while keeping the antipoison
protection in place.
- **TCP inbound cap honors `max_connections`.** The TCP inbound accept
ceiling now resolves from explicit per-transport
`max_inbound_connections`, then node-wide
`node.limits.max_connections`, then the built-in default of 256.
Previously the TCP inbound ceiling was hardwired to 256 and ignored
`max_connections`, so raising it had no effect on inbound TCP.
- **Static host aliases hot-reload.** `/etc/fips/hosts` now reloads on
mtime change once per tick rather than only at startup, so display
names in `fipsctl` and `fipstop` reflect edits without a daemon
restart. The peer ACL reloads through the same lock-free snapshot
mechanism.
- **Quieter logs on busy public-mesh nodes.** Routine per-peer
connection-lifecycle and capacity-cap events, no-route session-datagram
drops, and exhausted rekey-budget aborts are demoted to debug, so
genuinely notable info and warn lines are no longer drowned out.
- **More visible drops.** Receive-path silent rejections now flow
through typed reject-reason counters, and discovery counts requests
dropped when the dedup cache is full (`req_dedup_cache_full`, visible
via `show_routing`). Drops that were previously silent are now
countable.
- **Tor connect-refused accounting.** The Tor transport increments its
`connect_refused` statistic (the "Refused" line in `fipstop`) on an
actively-refused SOCKS5 connect, instead of recording every connect
failure as a generic SOCKS5 error.
`node.bloom.max_inbound_fpr` goes from `0.10` to `0.20`. The cap rejects
inbound `FilterAnnounce` frames whose advertised false positive rate
exceeds it. On the fixed 1 KB, k=5 filter, `0.10` corresponds to a fill of
0.631 and roughly 1,630 reachable entries, and the busiest nodes'
aggregates had started reaching that ceiling as the mesh grew. `0.20`
corresponds to a fill of 0.7248 and roughly 2,114 entries.
Be aware that this is the second time in two releases that this default
has doubled, for the same reason both times. That is worth stating plainly
rather than repeating the previous release's framing: raising the cap buys
headroom, it does not fix anything. The real constraint is the fixed 1 KB
filter size, which is a protocol constant. The structural remedy is the v2
filter work, where filter capacity scales with the mesh instead of being
pinned. This release is an interim step to keep legitimate aggregates from
being rejected until that lands. It is not the start of a pattern of
raising the cap once per release, and if you are sizing capacity planning
around this number, plan against the v2 work rather than against a third
raise.
The antipoison property the cap exists for is preserved. A saturated or
deliberately poisoned filter still presents an FPR near 100% and is still
rejected.
**This matters during a rolling upgrade.** A v0.4.1 node accepts a
`FilterAnnounce` with a derived FPR between 0.10 and 0.20; a v0.4.0 node
drops the same frame, and the drop is silent on the wire with no NACK. The
cap also gates the mesh size estimator, which declines to produce a value
when any contributing filter is over the cap. So while a mesh is partly
upgraded, upgraded and not-yet-upgraded nodes can legitimately report
different mesh sizes, or one can report a size while the other reports
unknown. This resolves once every node is on v0.4.1. If you want to avoid
the window entirely, set `node.bloom.max_inbound_fpr: 0.10` explicitly in
your config before upgrading and remove it after the last node is done.
### The `parent_switched` counter is removed
`parent_switched` was incremented on the line immediately before
`parent_switches` at every site and never independently, so the two
counters always held the same value. `parent_switched` is now gone from
the tree metrics, the control socket snapshot, and the `fipstop` tree
view. `parent_switches` remains and is unchanged.
If you scrape the control socket, or have dashboards or alerts referencing
`parent_switched`, point them at `parent_switches`. Anything still asking
for `parent_switched` will find nothing rather than a zero.
## Notable bug fixes
The CHANGELOG has the exhaustive list. This is the operator-relevant
subset of fixes for behavior that shipped in v0.3.0.
### Stale coordinates after losing a parent through peer removal
- **Symmetric peer teardown on manual disconnect.** A manual
`fipsctl disconnect` now sends the peer a scoped Disconnect so both
ends tear down and re-handshake cleanly. Previously a manual
disconnect tore down only the local side, leaving the peer with a
stale session that was never re-adopted as a child and whose bloom
filter was never re-recorded.
- **Gateway holds long-lived and DNS-cached mappings.** `fips-gateway`
no longer drops a virtual-IP mapping while traffic is still flowing.
The mapping TTL clock previously advanced only on DNS re-query, so a
busy long-lived or DNS-cached client could have its mapping reclaimed
mid-flow. The tick now refreshes the mapping whenever conntrack reports
active sessions and recovers a draining mapping to active when traffic
resumes; only genuinely idle mappings drain.
- **Accurate mesh-size estimate under filter overlap.** The mesh-size
estimator now estimates the cardinality of the OR-union of self plus
every connected peer's inbound filter, instead of summing per-filter
cardinalities of tree peers. Summing assumed the filters were disjoint,
so a stale or oversized parent filter or a routing loop inflated the
reported mesh size and a tree rebalance flapped the count. OR-union
deduplicates overlap, equals the old result in the disjoint case, and
removes the estimate's dependence on tree-declaration cache freshness.
- **Single-uplink node reattaches within a round-trip.** A node with one
tree peer, which has periodic parent re-evaluation disabled, was left
self-rooted and unreachable if its one-shot attaching TreeAnnounce was
lost, until the next periodic re-broadcast. Tree-position exchange is
now self-healing on the receive path: a node that hears an announce
advertising a strictly worse root echoes its own declaration back,
provoking the better-rooted peer to re-push its real position
immediately.
- **macOS self-connections work end to end (#117).** Traffic a macOS
node sends to its own `<npub>.fips` address is now delivered locally
for full TCP/UDP, not just `ping6`. The point-to-point `utun` egresses
self-addressed packets into the daemon with an unfinished transport
checksum (macOS offloads it on the `lo0` loopback route), so
re-injecting them verbatim made the local stack drop every segment the
MSS-clamp rewrite did not happen to fix and self-connections
half-opened and hung. The hairpin path now recomputes the TCP/UDP
checksum before re-injection. Linux was unaffected.
When a node's parent link dropped via peer removal, the node correctly
reparented or self-rooted, but skipped the coordinate cache invalidation
that every other position-change path performs. Cached entries for
downstream destinations kept the node's old coordinate prefix. This did
not self-correct the way a stale cache entry normally would: routing
access refreshes an entry's TTL, so an entry that was actively being
routed through never expired, and was only fixed by an unrelated fresh
insert. Both invalidation classes now run on this path, matching the
loop-detection branch.
### Discovery could loosen a tightened path MTU clamp
An originator handling a `LookupResponse` overwrote its cached path MTU
unconditionally. If a reactive `MtuExceeded` or `PathMtuNotification` had
already taught it a tighter value, a later, looser discovery estimate
would clobber that and re-loosen the clamp, risking a return to dropped
oversized packets. The cached and received values are now compared and the
tighter one is kept.
## Upgrade notes
Operator-actionable items moving from v0.3.0 to v0.4.0:
This is a drop-in upgrade from v0.4.0 with no wire format change, no
config migration, and no coordinated restart. Upgrade nodes in whatever
order you like.
- **Wire-compatible, no flag day.** v0.4.0 peers with v0.3.0. Upgrade
nodes in any order. During a rolling upgrade you may see some log lines
on the upgraded side as it interacts with not-yet-upgraded peers;
behavior is correct, log noise only.
- **Bloom antipoison cap default changed.** `node.bloom.max_inbound_fpr`
now defaults to 0.10 (was 0.05). If you set this explicitly, review
whether you still want the old value.
- **New optional config surfaces.** `transports.nym` (outbound Nym
mixnet) and `node.discovery.lan` (mDNS LAN discovery) are both opt-in
and off by default. Adding them is the only way to turn the new paths
on.
- **TCP inbound cap.** If you relied on the old hardwired 256 inbound-TCP
ceiling, note it now honors `max_inbound_connections` then
`node.limits.max_connections` then 256.
- **New observability query.** `fipsctl stats metrics` (the
`show_metrics` control query) returns a counter-only snapshot suitable
for a scraper.
Two things to do rather than assume:
## Getting v0.4.0
1. If you monitor `parent_switched`, move to `parent_switches` before
upgrading, or your dashboards will go blank rather than error.
2. During the rolling window, expect upgraded and not-yet-upgraded nodes
to potentially disagree about mesh size, per the FPR cap section above.
This is expected and self-resolves. Do not chase it as a bug unless it
persists after every node reports `0.4.1`.
If you have pinned `node.bloom.max_inbound_fpr` explicitly in your config,
your setting is honored and nothing changes for you. The change only
affects nodes taking the default.
Downgrading to v0.4.0 is supported and needs no special handling.
## Getting v0.4.1
- **Linux x86_64 / aarch64**: `.deb` and tarball at the
[v0.4.0 release page](https://github.com/jmcorgan/fips/releases/tag/v0.4.0).
[v0.4.1 release page](https://github.com/jmcorgan/fips/releases/tag/v0.4.1).
- **Arch Linux**: `fips` from the AUR.
- **macOS**: `.pkg` at the v0.4.0 release page.
- **Windows**: ZIP at the v0.4.0 release page.
- **macOS**: `.pkg` at the v0.4.1 release page.
- **Windows**: ZIP at the v0.4.1 release page.
- **OpenWrt**: `.ipk` (OpenWrt 24.x and earlier) or `.apk` (OpenWrt 25+)
at the v0.4.0 release page.
- **From source**: `cargo build --release` from a checkout of the v0.4.0
at the v0.4.1 release page.
- **From source**: `cargo build --release` from a checkout of the v0.4.1
tag (Rust 1.94.1 per `rust-toolchain.toml`; `libclang-dev` is a
required Linux build prerequisite).
- **Nix / NixOS**: `nix build .#fips` from a checkout of the v0.4.0 tag
- **Nix / NixOS**: `nix build .#fips` from a checkout of the v0.4.1 tag
builds the binaries from source with the pinned toolchain and no manual
prerequisites (see the Nix section of `packaging/README.md`).
@@ -309,13 +141,6 @@ The full per-commit changelog lives in
Thanks to everyone who contributed code, packaging work, bug reports, or
reviews to this release.
- [@jcorgan](https://github.com/jmcorgan): release shepherd, high-level
design, control read plane, rekey hardening, admission, bug fixes,
testing, packaging, PR coordination, and issue resolution.
- [@mmalmi](https://github.com/mmalmi): opt-in mDNS LAN discovery and
data-plane performance work.
- [@Origami74](https://github.com/Origami74): macOS packaging and
website coordination.
- [@dskvr](https://github.com/dskvr): AUR packaging.
- [@oleksky](https://github.com/oleksky): Nym mixnet transport and the
single-container mixnet demo.
- [@jcorgan](https://github.com/jmcorgan): release shepherd, spanning-tree
and discovery fixes, bloom and identity performance work, antipoison cap
change, and testing.

4
docs/design/fips-bloom-filters.md
View File

@@ -360,14 +360,14 @@ control socket and `fipstop` dashboard. (See `compute_mesh_size()` in
The estimator refuses to produce a value when any contributing filter
is above the antipoison FPR cap (`node.bloom.max_inbound_fpr`,
default `0.10`); a partial aggregate would silently underestimate.
default `0.20`); a partial aggregate would silently underestimate.
Consumers handle the resulting `None` by displaying an "unknown"
state rather than a misleading number.
## Antipoison: Inbound FPR Cap
Inbound `FilterAnnounce` payloads are checked against
`node.bloom.max_inbound_fpr` (default `0.10`). Filters whose
`node.bloom.max_inbound_fpr` (default `0.20`). Filters whose
estimated false positive rate exceeds the cap are dropped silently
(no NACK on the wire) — they would otherwise inflate downstream
candidate evaluation cost without contributing useful discrimination.

4
docs/reference/configuration.md
View File

@@ -277,7 +277,7 @@ Controls tree construction and parent selection.
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `node.bloom.update_debounce_ms` | u64 | `500` | Debounce interval for filter update propagation |
| `node.bloom.max_inbound_fpr` | f64 | `0.10` | Antipoison cap: reject inbound `FilterAnnounce` frames whose advertised false-positive rate exceeds this value. Valid range `(0.0, 1.0)`. The default `0.10` corresponds to fill 0.631 at k=5 (≈1,630 entries on the 1 KB filter); a saturated/poisoned filter is still ~100% FPR and rejected |
| `node.bloom.max_inbound_fpr` | f64 | `0.20` | Antipoison cap: reject inbound `FilterAnnounce` frames whose advertised false-positive rate exceeds this value. Valid range `(0.0, 1.0)`. The default `0.20` corresponds to fill 0.7248 at k=5 (≈2,114 entries on the 1 KB filter); a saturated/poisoned filter is still ~100% FPR and rejected |
Bloom filter size (1 KB), hash count (5), and size classes are protocol
constants and not configurable.
@@ -944,7 +944,7 @@ node:
flap_dampening_secs: 120 # extended hold-down on flap
bloom:
update_debounce_ms: 500
max_inbound_fpr: 0.10 # antipoison cap on inbound FilterAnnounce FPR
max_inbound_fpr: 0.20 # antipoison cap on inbound FilterAnnounce FPR
session:
default_ttl: 64
pending_packets_per_dest: 16

2
docs/releases/release-notes-v0.4.0.md
View File

@@ -1,6 +1,6 @@
# FIPS v0.4.0
**Released**: 2026-06-21 (provisional)
**Released**: 2026-06-27
v0.4.0 is the throughput-and-observability release on the v0.3.x wire
format. It adds two new ways for nodes to find and reach each other (the

146
docs/releases/release-notes-v0.4.1.md Normal file
View File

@@ -0,0 +1,146 @@
# FIPS v0.4.1
**Released**: 2026-07-19
v0.4.1 is a maintenance release on the v0.4.x line. It raises the default
antipoison cap on inbound bloom filter announcements, removes a redundant
spanning-tree metric counter, fixes two convergence and path-MTU bugs, and
cuts per-packet CPU in the bloom and identity paths. There is no wire
format change and no new feature surface.
v0.4.1 is wire-compatible with v0.4.0. Nodes can be upgraded one at a time
with no coordinated restart, though one behavior change below is worth
reading before you start a rolling upgrade.
## At a glance
- `node.bloom.max_inbound_fpr` default moves from `0.10` to `0.20`.
- The `parent_switched` metric counter is gone. Use `parent_switches`.
- Spanning tree no longer serves stale coordinates after a parent link is
lost through peer removal.
- Discovery no longer loosens a path MTU clamp it had correctly tightened.
- Bloom probing and identity operations do measurably less work per call,
with identical results.
## Behavior changes worth flagging
### The inbound filter FPR cap default doubles again
`node.bloom.max_inbound_fpr` goes from `0.10` to `0.20`. The cap rejects
inbound `FilterAnnounce` frames whose advertised false positive rate
exceeds it. On the fixed 1 KB, k=5 filter, `0.10` corresponds to a fill of
0.631 and roughly 1,630 reachable entries, and the busiest nodes'
aggregates had started reaching that ceiling as the mesh grew. `0.20`
corresponds to a fill of 0.7248 and roughly 2,114 entries.
Be aware that this is the second time in two releases that this default
has doubled, for the same reason both times. That is worth stating plainly
rather than repeating the previous release's framing: raising the cap buys
headroom, it does not fix anything. The real constraint is the fixed 1 KB
filter size, which is a protocol constant. The structural remedy is the v2
filter work, where filter capacity scales with the mesh instead of being
pinned. This release is an interim step to keep legitimate aggregates from
being rejected until that lands. It is not the start of a pattern of
raising the cap once per release, and if you are sizing capacity planning
around this number, plan against the v2 work rather than against a third
raise.
The antipoison property the cap exists for is preserved. A saturated or
deliberately poisoned filter still presents an FPR near 100% and is still
rejected.
**This matters during a rolling upgrade.** A v0.4.1 node accepts a
`FilterAnnounce` with a derived FPR between 0.10 and 0.20; a v0.4.0 node
drops the same frame, and the drop is silent on the wire with no NACK. The
cap also gates the mesh size estimator, which declines to produce a value
when any contributing filter is over the cap. So while a mesh is partly
upgraded, upgraded and not-yet-upgraded nodes can legitimately report
different mesh sizes, or one can report a size while the other reports
unknown. This resolves once every node is on v0.4.1. If you want to avoid
the window entirely, set `node.bloom.max_inbound_fpr: 0.10` explicitly in
your config before upgrading and remove it after the last node is done.
### The `parent_switched` counter is removed
`parent_switched` was incremented on the line immediately before
`parent_switches` at every site and never independently, so the two
counters always held the same value. `parent_switched` is now gone from
the tree metrics, the control socket snapshot, and the `fipstop` tree
view. `parent_switches` remains and is unchanged.
If you scrape the control socket, or have dashboards or alerts referencing
`parent_switched`, point them at `parent_switches`. Anything still asking
for `parent_switched` will find nothing rather than a zero.
## Notable bug fixes
### Stale coordinates after losing a parent through peer removal
When a node's parent link dropped via peer removal, the node correctly
reparented or self-rooted, but skipped the coordinate cache invalidation
that every other position-change path performs. Cached entries for
downstream destinations kept the node's old coordinate prefix. This did
not self-correct the way a stale cache entry normally would: routing
access refreshes an entry's TTL, so an entry that was actively being
routed through never expired, and was only fixed by an unrelated fresh
insert. Both invalidation classes now run on this path, matching the
loop-detection branch.
### Discovery could loosen a tightened path MTU clamp
An originator handling a `LookupResponse` overwrote its cached path MTU
unconditionally. If a reactive `MtuExceeded` or `PathMtuNotification` had
already taught it a tighter value, a later, looser discovery estimate
would clobber that and re-loosen the clamp, risking a return to dropped
oversized packets. The cached and received values are now compared and the
tighter one is kept.
## Upgrade notes
This is a drop-in upgrade from v0.4.0 with no wire format change, no
config migration, and no coordinated restart. Upgrade nodes in whatever
order you like.
Two things to do rather than assume:
1. If you monitor `parent_switched`, move to `parent_switches` before
upgrading, or your dashboards will go blank rather than error.
2. During the rolling window, expect upgraded and not-yet-upgraded nodes
to potentially disagree about mesh size, per the FPR cap section above.
This is expected and self-resolves. Do not chase it as a bug unless it
persists after every node reports `0.4.1`.
If you have pinned `node.bloom.max_inbound_fpr` explicitly in your config,
your setting is honored and nothing changes for you. The change only
affects nodes taking the default.
Downgrading to v0.4.0 is supported and needs no special handling.
## Getting v0.4.1
- **Linux x86_64 / aarch64**: `.deb` and tarball at the
[v0.4.1 release page](https://github.com/jmcorgan/fips/releases/tag/v0.4.1).
- **Arch Linux**: `fips` from the AUR.
- **macOS**: `.pkg` at the v0.4.1 release page.
- **Windows**: ZIP at the v0.4.1 release page.
- **OpenWrt**: `.ipk` (OpenWrt 24.x and earlier) or `.apk` (OpenWrt 25+)
at the v0.4.1 release page.
- **From source**: `cargo build --release` from a checkout of the v0.4.1
tag (Rust 1.94.1 per `rust-toolchain.toml`; `libclang-dev` is a
required Linux build prerequisite).
- **Nix / NixOS**: `nix build .#fips` from a checkout of the v0.4.1 tag
builds the binaries from source with the pinned toolchain and no manual
prerequisites (see the Nix section of `packaging/README.md`).
The full per-commit changelog lives in
[`CHANGELOG.md`](../../CHANGELOG.md). Issues and discussion at
[github.com/jmcorgan/fips](https://github.com/jmcorgan/fips).
## Contributors
Thanks to everyone who contributed code, packaging work, bug reports, or
reviews to this release.
- [@jcorgan](https://github.com/jmcorgan): release shepherd, spanning-tree
and discovery fixes, bloom and identity performance work, antipoison cap
change, and testing.

4
src/bin/fipstop/ui/tree.rs
View File

@@ -174,10 +174,6 @@ fn draw_stats(frame: &mut Frame, data: &serde_json::Value, scroll: u16, focused:
&helpers::nested_u64(data, "stats", "sig_failed"),
),
helpers::kv_line("Stale", &helpers::nested_u64(data, "stats", "stale")),
helpers::kv_line(
"Parent Switched",
&helpers::nested_u64(data, "stats", "parent_switched"),
),
helpers::kv_line(
"Loop Detected",
&helpers::nested_u64(data, "stats", "loop_detected"),

28
src/bloom/filter.rs
View File

@@ -69,16 +69,18 @@ impl BloomFilter {
/// Insert a NodeAddr into the filter.
pub fn insert(&mut self, node_addr: &NodeAddr) {
let (h1, h2) = Self::base_hashes(node_addr.as_bytes());
for i in 0..self.hash_count {
let bit_index = self.hash(node_addr.as_bytes(), i);
let bit_index = self.bit_index(h1, h2, i);
self.set_bit(bit_index);
}
}
/// Insert raw bytes into the filter.
pub fn insert_bytes(&mut self, data: &[u8]) {
let (h1, h2) = Self::base_hashes(data);
for i in 0..self.hash_count {
let bit_index = self.hash(data, i);
let bit_index = self.bit_index(h1, h2, i);
self.set_bit(bit_index);
}
}
@@ -93,8 +95,9 @@ impl BloomFilter {
/// Check if the filter might contain raw bytes.
pub fn contains_bytes(&self, data: &[u8]) -> bool {
let (h1, h2) = Self::base_hashes(data);
for i in 0..self.hash_count {
let bit_index = self.hash(data, i);
let bit_index = self.bit_index(h1, h2, i);
if !self.get_bit(bit_index) {
return false;
}
@@ -196,21 +199,28 @@ impl BloomFilter {
self.hash_count
}
/// Compute a hash index for the given data and hash function number.
/// Compute the two base hashes for `data` with a single SHA-256 digest.
///
/// Uses double hashing: h(x,i) = (h1(x) + i*h2(x)) mod m
fn hash(&self, data: &[u8], k: u8) -> usize {
// Use first 16 bytes of SHA-256 for h1 and h2
/// Double hashing derives the k hash functions from two base hashes:
/// h(x,i) = (h1(x) + i*h2(x)) mod m. Computing the digest once here and
/// reusing `(h1, h2)` across all k functions avoids re-hashing per k.
fn base_hashes(data: &[u8]) -> (u64, u64) {
// Use first 16 bytes of SHA-256 for h1 and h2.
use sha2::{Digest, Sha256};
let mut hasher = Sha256::new();
hasher.update(data);
let hash = hasher.finalize();
// h1 from first 8 bytes
// h1 from first 8 bytes, h2 from next 8 bytes (little-endian).
let h1 = u64::from_le_bytes(hash[0..8].try_into().unwrap());
// h2 from next 8 bytes
let h2 = u64::from_le_bytes(hash[8..16].try_into().unwrap());
(h1, h2)
}
/// Derive the bit index for hash function `k` from the base hashes.
///
/// Uses double hashing: h(x,k) = (h1(x) + k*h2(x)) mod m.
fn bit_index(&self, h1: u64, h2: u64, k: u8) -> usize {
let combined = h1.wrapping_add((k as u64).wrapping_mul(h2));
(combined as usize) % self.num_bits
}

78
src/bloom/tests.rs
View File

@@ -228,6 +228,84 @@ fn test_bloom_filter_insert_bytes_contains_bytes() {
assert!(filter.contains_bytes(data2));
}
#[test]
fn test_bloom_filter_bit_indices_match_double_hashing_formula() {
use sha2::{Digest, Sha256};
// Independently recompute the documented double-hashing bit indices:
// one SHA-256 digest of the input, h1 = bytes[0..8] LE, h2 = bytes[8..16]
// LE, then for k in 0..hash_count: (h1 + k*h2) mod num_bits. This pins
// bit-identical behavior regardless of the internal implementation.
fn expected_indices(data: &[u8], num_bits: usize, hash_count: u8) -> Vec<usize> {
let digest = Sha256::digest(data);
let h1 = u64::from_le_bytes(digest[0..8].try_into().unwrap());
let h2 = u64::from_le_bytes(digest[8..16].try_into().unwrap());
(0..hash_count)
.map(|k| {
let combined = h1.wrapping_add((k as u64).wrapping_mul(h2));
(combined as usize) % num_bits
})
.collect()
}
fn bit_is_set(filter: &BloomFilter, index: usize) -> bool {
let byte = filter.as_bytes()[index / 8];
(byte >> (index % 8)) & 1 == 1
}
let configs = [(1024usize, 5u8), (8192usize, 7u8)];
let inputs: [&[u8]; 4] = [b"", b"alpha", b"the quick brown fox", &[0u8, 1, 2, 3, 255]];
for (num_bits, hash_count) in configs {
for data in inputs {
let mut filter = BloomFilter::with_params(num_bits, hash_count).unwrap();
let expected = expected_indices(data, num_bits, hash_count);
filter.insert_bytes(data);
// Every expected bit is set.
for &idx in &expected {
assert!(
bit_is_set(&filter, idx),
"expected bit {} set for input {:?} (num_bits={}, k={})",
idx,
data,
num_bits,
hash_count
);
}
// No unexpected bits are set: the set-bit count never exceeds the
// number of distinct expected indices.
use std::collections::HashSet;
let distinct: HashSet<usize> = expected.iter().copied().collect();
assert_eq!(
filter.count_ones(),
distinct.len(),
"unexpected bits set for input {:?}",
data
);
// contains reports the inserted item as present.
assert!(filter.contains_bytes(data));
}
}
// NodeAddr path uses the same formula over its byte view.
let node = make_node_addr(7);
let mut filter = BloomFilter::with_params(1024, 5).unwrap();
let expected = expected_indices(node.as_bytes(), 1024, 5);
filter.insert(&node);
for &idx in &expected {
assert!(bit_is_set(&filter, idx));
}
assert!(filter.contains(&node));
// Spot-check a definitely-absent item is reported absent.
let absent = make_node_addr(200);
assert!(!filter.contains(&absent));
}
#[test]
fn test_bloom_filter_estimated_count_saturated() {
// Create a small filter with all bits set

10
src/config/node.rs
View File

@@ -635,8 +635,8 @@ pub struct BloomConfig {
pub update_debounce_ms: u64,
/// Antipoison cap: reject inbound FilterAnnounce whose FPR exceeds
/// this value (`node.bloom.max_inbound_fpr`). Valid range `(0.0, 1.0)`.
/// Default `0.10` ≈ fill 0.631 at k=5 ≈ ~1,630 entries on the 1 KB
/// filter (Swamidass–Baldi). Raised from 0.05 so aggregates that are
/// Default `0.20` ≈ fill 0.7248 at k=5 ≈ ~2,114 entries on the 1 KB
/// filter (Swamidass–Baldi). Raised from 0.10 so aggregates that are
/// legitimately near their operating ceiling are not rejected before
/// the network reaches the fixed-filter capacity limit; conceptually
/// distinct from future autoscaling hysteresis setpoints — same unit,
@@ -648,8 +648,8 @@ pub struct BloomConfig {
impl Default for BloomConfig {
fn default() -> Self {
Self {
update_debounce_ms: 500,
max_inbound_fpr: 0.10,
update_debounce_ms: Self::default_update_debounce_ms(),
max_inbound_fpr: Self::default_max_inbound_fpr(),
}
}
}
@@ -659,7 +659,7 @@ impl BloomConfig {
500
}
fn default_max_inbound_fpr() -> f64 {
0.10
0.20
}
}

1
src/control/snapshots/show_tree.json
View File

@@ -24,7 +24,6 @@
"loop_detected": 0,
"outbound_sign_failed": 0,
"parent_losses": 0,
"parent_switched": 0,
"parent_switches": 0,
"rate_limited": 0,
"received": 0,

6
src/identity/auth.rs
View File

@@ -1,7 +1,7 @@
//! Authentication challenge-response protocol.
use rand::Rng;
use secp256k1::{Secp256k1, XOnlyPublicKey};
use secp256k1::XOnlyPublicKey;
use sha2::{Digest, Sha256};
use super::{IdentityError, NodeAddr};
@@ -34,9 +34,9 @@ impl AuthChallenge {
/// Verify a response to this challenge.
pub fn verify(&self, response: &AuthResponse) -> Result<NodeAddr, IdentityError> {
let digest = auth_challenge_digest(&self.0, response.timestamp);
let secp = Secp256k1::new();
secp.verify_schnorr(&response.signature, &digest, &response.pubkey)
super::SECP
.verify_schnorr(&response.signature, &digest, &response.pubkey)
.map_err(|_| IdentityError::SignatureVerificationFailed)?;
Ok(NodeAddr::from_pubkey(&response.pubkey))

11
src/identity/local.rs
View File

@@ -1,6 +1,6 @@
//! Local node identity with signing capability.
use secp256k1::{Keypair, PublicKey, Secp256k1, SecretKey, XOnlyPublicKey};
use secp256k1::{Keypair, PublicKey, SecretKey, XOnlyPublicKey};
use std::fmt;
use super::auth::{AuthResponse, auth_challenge_digest};
@@ -42,8 +42,7 @@ impl Identity {
/// Create an identity from a secret key.
pub fn from_secret_key(secret_key: SecretKey) -> Self {
let secp = Secp256k1::new();
let keypair = Keypair::from_secret_key(&secp, &secret_key);
let keypair = Keypair::from_secret_key(&super::SECP, &secret_key);
Self::from_keypair(keypair)
}
@@ -93,9 +92,8 @@ impl Identity {
/// Sign arbitrary data with this identity's secret key.
pub fn sign(&self, data: &[u8]) -> secp256k1::schnorr::Signature {
let secp = Secp256k1::new();
let digest = sha256(data);
secp.sign_schnorr(&digest, &self.keypair)
super::SECP.sign_schnorr(&digest, &self.keypair)
}
/// Create an authentication response for a challenge.
@@ -103,8 +101,7 @@ impl Identity {
/// The response signs: SHA256("fips-auth-v1" || challenge || timestamp)
pub fn sign_challenge(&self, challenge: &[u8; 32], timestamp: u64) -> AuthResponse {
let digest = auth_challenge_digest(challenge, timestamp);
let secp = Secp256k1::new();
let signature = secp.sign_schnorr(&digest, &self.keypair);
let signature = super::SECP.sign_schnorr(&digest, &self.keypair);
AuthResponse {
pubkey: self.pubkey(),
timestamp,

12
src/identity/mod.rs
View File

@@ -11,6 +11,9 @@ mod local;
mod node_addr;
mod peer;
use std::sync::LazyLock;
use secp256k1::{All, Secp256k1};
use sha2::{Digest, Sha256};
use thiserror::Error;
@@ -21,6 +24,15 @@ pub use local::Identity;
pub use node_addr::NodeAddr;
pub use peer::PeerIdentity;
/// Shared secp256k1 context reused across all identity operations.
///
/// `Secp256k1::new()` allocates a `Secp256k1<All>` and runs randomization /
/// blinding table setup; it is designed to be created once and reused rather
/// than rebuilt per sign / verify / key-derive call. This single `All` context
/// serves both signing and verification across the identity module and still
/// performs the standard construction-time blinding.
pub(crate) static SECP: LazyLock<Secp256k1<All>> = LazyLock::new(Secp256k1::new);
/// FIPS address prefix (IPv6 ULA range).
pub const FIPS_ADDRESS_PREFIX: u8 = 0xfd;

6
src/identity/peer.rs
View File

@@ -1,6 +1,6 @@
//! Remote peer identity (public key only, no signing capability).
use secp256k1::{Parity, PublicKey, Secp256k1, XOnlyPublicKey};
use secp256k1::{Parity, PublicKey, XOnlyPublicKey};
use std::fmt;
use super::encoding::{decode_npub, encode_npub};
@@ -107,9 +107,9 @@ impl PeerIdentity {
/// Verify a signature from this peer.
pub fn verify(&self, data: &[u8], signature: &secp256k1::schnorr::Signature) -> bool {
let secp = Secp256k1::new();
let digest = sha256(data);
secp.verify_schnorr(signature, &digest, &self.pubkey)
super::SECP
.verify_schnorr(signature, &digest, &self.pubkey)
.is_ok()
}
}

9
src/identity/tests.rs
View File

@@ -1,7 +1,7 @@
use std::collections::HashSet;
use std::net::Ipv6Addr;
use secp256k1::{Keypair, Secp256k1, SecretKey};
use secp256k1::{Keypair, SecretKey};
use super::*;
@@ -161,10 +161,10 @@ fn test_identity_sign() {
let sig = identity.sign(data);
// Verify the signature manually
let secp = secp256k1::Secp256k1::new();
let digest = super::sha256(data);
assert!(
secp.verify_schnorr(&sig, &digest, &identity.pubkey())
super::SECP
.verify_schnorr(&sig, &digest, &identity.pubkey())
.is_ok()
);
}
@@ -580,13 +580,12 @@ fn test_peer_identity_pubkey_full_even_parity_fallback() {
#[test]
fn test_peer_identity_pubkey_full_preserved_parity() {
// Create two identities and find one with odd parity to make this test meaningful
let secp = Secp256k1::new();
let secret_bytes: [u8; 32] = [
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e,
0x1f, 0x20,
];
let keypair = Keypair::from_seckey_slice(&secp, &secret_bytes).unwrap();
let keypair = Keypair::from_seckey_slice(&super::SECP, &secret_bytes).unwrap();
let full_pubkey = keypair.public_key();
let peer = PeerIdentity::from_pubkey_full(full_pubkey);

37
src/node/handlers/discovery.rs
View File

@@ -240,17 +240,32 @@ impl Node {
// map used by the TUN reader/writer at TCP MSS clamp time.
let fips_addr = crate::FipsAddress::from_node_addr(&target);
match self.path_mtu_lookup.write() {
Ok(mut map) => {
let prior = map.insert(fips_addr, path_mtu);
debug!(
target = %self.peer_display_name(&target),
fips_addr = %fips_addr,
path_mtu = path_mtu,
prior = ?prior,
map_len = map.len(),
"Wrote path_mtu_lookup from discovery LookupResponse"
);
}
Ok(mut map) => match map.get(&fips_addr).copied() {
Some(existing) if existing <= path_mtu => {
// Keep the tighter learned value; never loosen the
// clamp. A reactive MtuExceeded or PathMtuNotification
// tighten takes precedence over a looser discovery
// estimate (cross-carrier keep-tighter).
debug!(
target = %self.peer_display_name(&target),
fips_addr = %fips_addr,
path_mtu = path_mtu,
existing = existing,
"LookupResponse: keeping tighter existing path_mtu_lookup value"
);
}
other => {
map.insert(fips_addr, path_mtu);
debug!(
target = %self.peer_display_name(&target),
fips_addr = %fips_addr,
path_mtu = path_mtu,
prior = ?other,
map_len = map.len(),
"Wrote path_mtu_lookup from discovery LookupResponse"
);
}
},
Err(e) => {
warn!(
target = %self.peer_display_name(&target),

2
src/node/handlers/mmp.rs
View File

@@ -173,7 +173,6 @@ impl Node {
self.coord_cache
.invalidate_via_node(our_identity.node_addr());
self.reset_discovery_backoff();
self.metrics().tree.parent_switched.inc();
self.metrics().tree.parent_switches.inc();
info!(
new_parent = %self.peer_display_name(&new_parent),
@@ -205,7 +204,6 @@ impl Node {
self.coord_cache
.invalidate_other_roots(our_identity.node_addr());
self.reset_discovery_backoff();
self.metrics().tree.parent_switched.inc();
self.metrics().tree.parent_switches.inc();
info!(
new_root = %self.tree_state.root(),

2
src/node/metrics.rs
View File

@@ -317,7 +317,6 @@ pub struct TreeMetrics {
pub stale: Counter,
pub ancestry_invalid: Counter,
pub accepted: Counter,
pub parent_switched: Counter,
pub loop_detected: Counter,
pub ancestry_changed: Counter,
pub sent: Counter,
@@ -351,7 +350,6 @@ impl TreeMetrics {
stale: self.stale.get(),
ancestry_invalid: self.ancestry_invalid.get(),
accepted: self.accepted.get(),
parent_switched: self.parent_switched.get(),
loop_detected: self.loop_detected.get(),
ancestry_changed: self.ancestry_changed.get(),
sent: self.sent.get(),

1
src/node/stats.rs
View File

@@ -273,7 +273,6 @@ pub struct TreeStatsSnapshot {
pub stale: u64,
pub ancestry_invalid: u64,
pub accepted: u64,
pub parent_switched: u64,
pub loop_detected: u64,
pub ancestry_changed: u64,
pub sent: u64,

39
src/node/tests/discovery.rs
View File

@@ -912,6 +912,45 @@ async fn test_originator_stores_path_mtu_in_cache() {
);
}
#[tokio::test]
async fn test_originator_lookup_response_keeps_tighter_path_mtu_lookup() {
// Regression: a LookupResponse carrying a looser (larger) path_mtu must
// NOT clobber a tighter (smaller) value already in path_mtu_lookup that a
// reactive MtuExceeded or PathMtuNotification learned. Cross-carrier
// keep-tighter: the clamp must never loosen.
let mut node = make_node();
let from = make_node_addr(0xAA);
let target_identity = Identity::generate();
let target = *target_identity.node_addr();
let root = make_node_addr(0xF0);
let coords = TreeCoordinate::from_addrs(vec![target, root]).unwrap();
node.register_identity(target, target_identity.pubkey_full());
// Pre-seed a tighter value, as if a reactive signal already narrowed it.
let target_fips = crate::FipsAddress::from_node_addr(&target);
node.path_mtu_lookup_insert(target_fips, 1280);
let proof_data = LookupResponse::proof_bytes(800, &target, &coords);
let proof = target_identity.sign(&proof_data);
let mut response = LookupResponse::new(800, target, coords.clone(), proof);
// Looser discovery estimate that must be rejected in favor of the tighter
// existing entry.
response.path_mtu = 1500;
let payload = &response.encode()[1..];
node.handle_lookup_response(&from, payload).await;
assert_eq!(
node.path_mtu_lookup_get(&target_fips),
Some(1280),
"LookupResponse must not loosen a tighter existing path_mtu_lookup value"
);
}
// ============================================================================
// Open-Discovery Sweep — cache-injection unit test
// ============================================================================

122
src/node/tests/routing.rs
View File

@@ -1333,3 +1333,125 @@ fn test_route_class_partition_sums_to_forwarded() {
assert_eq!(snap.route_crosslink_ascend, 3);
assert_eq!(snap.route_direct_peer, 1);
}
// === Coord-cache invalidation on parent loss ===
//
// Parent-lost-via-peer-removal is a genuine position change and must
// surgically invalidate the coordinate cache like every other such path
// (reparent → invalidate_via_node; self-root → invalidate_other_roots).
// `make_node_addr(0)` is the network minimum, so the node's random identity
// addr is always greater than it — the reparent/child geometry is deterministic.
#[test]
fn test_parent_loss_reparent_invalidates_coord_cache() {
let mut node = make_node();
let my_addr = *node.node_addr();
let root = make_node_addr(0);
let parent = make_node_addr(1);
let alt = make_node_addr(2);
// Current parent and an alternative, both rooted at `root`.
node.tree_state_mut().update_peer(
ParentDeclaration::new(parent, root, 1, 1000),
TreeCoordinate::from_addrs(vec![parent, root]).unwrap(),
);
node.tree_state_mut().update_peer(
ParentDeclaration::new(alt, root, 1, 1000),
TreeCoordinate::from_addrs(vec![alt, root]).unwrap(),
);
// Adopt `parent`; our coords become [my_addr, parent, root], root = `root`.
node.tree_state_mut().set_parent(parent, 1, 1000);
node.tree_state_mut().recompute_coords();
assert!(!node.tree_state().is_root());
assert_eq!(node.tree_state().root(), &root);
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
// via-node class: a downstream destination that routes through us.
let downstream = make_node_addr(10);
node.coord_cache_mut().insert(
downstream,
TreeCoordinate::from_addrs(vec![downstream, my_addr, root]).unwrap(),
now_ms,
);
// survivor: same root, does not route through us.
let sibling_dest = make_node_addr(11);
node.coord_cache_mut().insert(
sibling_dest,
TreeCoordinate::from_addrs(vec![sibling_dest, alt, root]).unwrap(),
now_ms,
);
// Parent link drops; node reparents onto `alt` (still rooted at `root`).
let changed = node.handle_peer_removal_tree_cleanup(&parent);
assert!(changed);
assert_eq!(node.tree_state().my_declaration().parent_id(), &alt);
assert_eq!(node.tree_state().root(), &root);
assert!(
!node.coord_cache().contains(&downstream, now_ms),
"entry routing through us must be invalidated after reparent"
);
assert!(
node.coord_cache().contains(&sibling_dest, now_ms),
"same-root entry not routing through us must survive (surgical, not a flush)"
);
}
#[test]
fn test_parent_loss_selfroot_invalidates_coord_cache() {
let mut node = make_node();
let my_addr = *node.node_addr();
let old_root = make_node_addr(0);
let parent = make_node_addr(1);
// Adopt `parent` (rooted at `old_root`); no alternative peers exist, so a
// parent loss self-roots the node.
node.tree_state_mut().update_peer(
ParentDeclaration::new(parent, old_root, 1, 1000),
TreeCoordinate::from_addrs(vec![parent, old_root]).unwrap(),
);
node.tree_state_mut().set_parent(parent, 1, 1000);
node.tree_state_mut().recompute_coords();
assert!(!node.tree_state().is_root());
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
// via-node class: routes through us.
let downstream = make_node_addr(10);
node.coord_cache_mut().insert(
downstream,
TreeCoordinate::from_addrs(vec![downstream, my_addr, old_root]).unwrap(),
now_ms,
);
// other-roots class: on the old root, does not route through us.
let foreign = make_node_addr(11);
node.coord_cache_mut().insert(
foreign,
TreeCoordinate::from_addrs(vec![foreign, parent, old_root]).unwrap(),
now_ms,
);
// Parent link drops; no alternative parent → node self-roots.
let changed = node.handle_peer_removal_tree_cleanup(&parent);
assert!(changed);
assert!(node.tree_state().is_root());
assert_eq!(node.tree_state().root(), &my_addr);
assert!(
!node.coord_cache().contains(&downstream, now_ms),
"via-node entry must be invalidated after self-root"
);
assert!(
!node.coord_cache().contains(&foreign, now_ms),
"stale old-root entry must be invalidated after self-root"
);
}

14
src/node/tree.rs
View File

@@ -300,7 +300,6 @@ impl Node {
.invalidate_via_node(our_identity.node_addr());
self.reset_discovery_backoff();
self.metrics().tree.parent_switched.inc();
self.metrics().tree.parent_switches.inc();
info!(
@@ -338,7 +337,6 @@ impl Node {
self.coord_cache
.invalidate_other_roots(our_identity.node_addr());
self.reset_discovery_backoff();
self.metrics().tree.parent_switched.inc();
self.metrics().tree.parent_switches.inc();
info!(
new_root = %self.tree_state.root(),
@@ -524,7 +522,6 @@ impl Node {
.invalidate_via_node(our_identity.node_addr());
self.reset_discovery_backoff();
self.metrics().tree.parent_switched.inc();
self.metrics().tree.parent_switches.inc();
info!(
@@ -560,7 +557,6 @@ impl Node {
self.coord_cache
.invalidate_other_roots(our_identity.node_addr());
self.reset_discovery_backoff();
self.metrics().tree.parent_switched.inc();
self.metrics().tree.parent_switches.inc();
info!(
new_root = %self.tree_state.root(),
@@ -620,6 +616,16 @@ impl Node {
.tree
.record_reject(TreeReject::OutboundSignFailed);
}
// handle_parent_lost may promote to root OR find new parent;
// cover both invalidation classes (same as the loop-detection
// branch above). Without this, cached downstream entries keep
// our now-stale coordinate prefix until TTL — and get_and_touch
// refreshes the TTL on every routing access, so an actively
// routed stale entry never self-expires.
self.coord_cache
.invalidate_via_node(our_identity.node_addr());
self.coord_cache
.invalidate_other_roots(self.tree_state.root());
info!(
new_root = %self.tree_state.root(),
is_root = self.tree_state.is_root(),

72
testing/README.md
View File

@@ -72,3 +72,75 @@ and optional trace-level RUST_LOG, capturing per-rep diagnostics and a
mechanism-match summary across the run. Used for statistical reliability
characterization of known flake classes under calibrated stress, not as
a per-commit gate; not part of `ci-local.sh`.
## Running CI locally (`ci-local.sh`)
[`ci-local.sh`](ci-local.sh) runs the full local CI pipeline — build,
clippy, unit tests, and the integration suites (including the chaos
scenarios) — mirroring the GitHub `ci.yml` integration matrix. Run
`./ci-local.sh --help` for the full option list and `--list` for the
available suites. `--check-parity` verifies the local suite set matches
the GitHub matrix (see [check-ci-parity.sh](check-ci-parity.sh)).
### Per-run isolation and the `FIPS_CI_RUN_ID` override
Every invocation derives a **run id** and scopes all of its Docker
resources to it, so two simultaneous runs on the same host (for example,
one per git worktree, or an operator testing by hand while CI is in
flight) never collide:
- **Compose projects** are named `fipsci_<run-id>_<suite>`, so
container, network, and volume names are all prefixed per run.
- **Build images** are tagged `fips-test:<run-id>` and
`fips-test-app:<run-id>` (exported as `FIPS_TEST_IMAGE` /
`FIPS_TEST_APP_IMAGE` for the compose consumers).
- Each parallel chaos child gets a unique, non-overlapping `/24` in
`10.30.x` (via the sim `--subnet` override). `10.30.x` sits outside
Docker's default address pool and the fixed-subnet suites' `172.x`
ranges, so neither a sibling chaos child nor an auto-assigned network
can swallow a pinned subnet.
By default the run id is `<short-git-sha>-<random>` — the SHA portion
records *what code* a container is testing, the random suffix keeps
simultaneous runs of the same SHA disjoint. Override it for a
reproducible, attach-by-name debug session:
```sh
FIPS_CI_RUN_ID=mydebug ./ci-local.sh --only static-mesh
# containers are named fipsci_mydebug_static_fips-node-a, etc.
```
### Preemption-safety and exit codes
`ci-local.sh` is safe to cancel mid-run. A signal trap tears down *every*
compose project the run started (not just the current suite) and reaps
any in-flight parallel chaos children, bounded by a `timeout` so a stuck
`compose down` cannot wedge the trap. Exit codes distinguish a cancelled
run from a failing one:
| Code | Meaning |
| ---- | ------- |
| `0` | all stages passed |
| `1` | one or more stages failed |
| `130` | interrupted by SIGINT — cancelled, not a failure |
| `143` | terminated by SIGTERM — cancelled, not a failure |
A preempting CI worker (the push-triggered, CI-gated build pipeline that
kills an in-flight run when a newer same-branch tip arrives) maps
`130`/`143` → *cancelled* (discard, do not record a failing commit), `0`
→ green, any other non-zero → red.
### Cleaning up leftover resources
Every CI-created container, network, and volume carries the label
`com.corganlabs.fips-ci=1`. If a run is hard-killed (SIGKILL, OOM, crash)
and leaves resources behind, reap them with:
```sh
./ci-local.sh --reap # or: ./ci-cleanup.sh
```
[`ci-cleanup.sh`](ci-cleanup.sh) force-removes everything bearing the CI
label or a `fipsci_` compose-project prefix; it is safe to run when there
is nothing to reap and safe to run repeatedly. Pass `--project-prefix` to
scope the sweep to a single run.

14
testing/acl-allowlist/docker-compose.yml
View File

@@ -1,6 +1,8 @@
networks:
acl-net:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
ipam:
config:
- subnet: 172.31.0.0/24
@@ -26,7 +28,7 @@ x-fips-common: &fips-common
services:
service-a:
<<: *fips-common
container_name: fips-acl-container-a
container_name: fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-}
hostname: host-a
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -41,7 +43,7 @@ services:
service-b:
<<: *fips-common
container_name: fips-acl-container-b
container_name: fips-acl-container-b${FIPS_CI_NAME_SUFFIX:-}
hostname: host-b
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -56,7 +58,7 @@ services:
service-c:
<<: *fips-common
container_name: fips-acl-container-c
container_name: fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-}
hostname: host-c
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -71,7 +73,7 @@ services:
service-d:
<<: *fips-common
container_name: fips-acl-container-d
container_name: fips-acl-container-d${FIPS_CI_NAME_SUFFIX:-}
hostname: host-d
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -86,7 +88,7 @@ services:
service-e:
<<: *fips-common
container_name: fips-acl-container-e
container_name: fips-acl-container-e${FIPS_CI_NAME_SUFFIX:-}
hostname: host-e
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -99,7 +101,7 @@ services:
service-f:
<<: *fips-common
container_name: fips-acl-container-f
container_name: fips-acl-container-f${FIPS_CI_NAME_SUFFIX:-}
hostname: host-f
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro

40
testing/acl-allowlist/test.sh
View File

@@ -132,31 +132,31 @@ docker compose -f "$COMPOSE_FILE" down >/dev/null 2>&1 || true
docker compose -f "$COMPOSE_FILE" up -d --build
log "Waiting for expected peer convergence"
wait_for_peers_exact fips-acl-container-a 3 40
wait_for_peers_exact fips-acl-container-b 1 40
wait_for_peers_exact fips-acl-container-c 0 5
wait_for_peers_exact fips-acl-container-d 0 5
wait_for_peers_exact fips-acl-container-e 1 40
wait_for_peers_exact fips-acl-container-f 1 40
wait_for_peers_exact fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} 3 40
wait_for_peers_exact fips-acl-container-b${FIPS_CI_NAME_SUFFIX:-} 1 40
wait_for_peers_exact fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-} 0 5
wait_for_peers_exact fips-acl-container-d${FIPS_CI_NAME_SUFFIX:-} 0 5
wait_for_peers_exact fips-acl-container-e${FIPS_CI_NAME_SUFFIX:-} 1 40
wait_for_peers_exact fips-acl-container-f${FIPS_CI_NAME_SUFFIX:-} 1 40
log "Verifying peer sets"
assert_peer_set fips-acl-container-a "npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
assert_peer_set fips-acl-container-b "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
assert_peer_set fips-acl-container-c ""
assert_peer_set fips-acl-container-d ""
assert_peer_set fips-acl-container-e "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
assert_peer_set fips-acl-container-f "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
assert_peer_set fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
assert_peer_set fips-acl-container-b${FIPS_CI_NAME_SUFFIX:-} "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
assert_peer_set fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-} ""
assert_peer_set fips-acl-container-d${FIPS_CI_NAME_SUFFIX:-} ""
assert_peer_set fips-acl-container-e${FIPS_CI_NAME_SUFFIX:-} "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
assert_peer_set fips-acl-container-f${FIPS_CI_NAME_SUFFIX:-} "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
log "Checking alias-based ACL resolution"
assert_acl_field fips-acl-container-a allow_file_entries "node-a node-b node-e node-f"
assert_acl_field fips-acl-container-a allow_entries "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
assert_acl_field fips-acl-container-c allow_file_entries "node-a node-b node-c node-d node-e node-f"
assert_acl_field fips-acl-container-c allow_entries "npub1cld9yay0u24davpu6c35l4vldrhzvaq66pcqtg9a0j2cnjrn9rtsxx2pe6 npub1n9lpnv0592cc2ps6nm0ca3qls642vx7yjsv35rkxqzj2vgds52sqgpverl npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
assert_acl_field fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} allow_file_entries "node-a node-b node-e node-f"
assert_acl_field fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} allow_entries "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
assert_acl_field fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-} allow_file_entries "node-a node-b node-c node-d node-e node-f"
assert_acl_field fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-} allow_entries "npub1cld9yay0u24davpu6c35l4vldrhzvaq66pcqtg9a0j2cnjrn9rtsxx2pe6 npub1n9lpnv0592cc2ps6nm0ca3qls642vx7yjsv35rkxqzj2vgds52sqgpverl npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
log "Checking ACL rejection logs"
assert_log_contains fips-acl-container-a "npub1cld9yay0u24davpu6c35l4vldrhzvaq66pcqtg9a0j2cnjrn9rtsxx2pe6"
assert_log_contains fips-acl-container-a "npub1n9lpnv0592cc2ps6nm0ca3qls642vx7yjsv35rkxqzj2vgds52sqgpverl"
assert_log_contains fips-acl-container-a "context=inbound_handshake"
assert_log_contains fips-acl-container-a "decision=denylist match"
assert_log_contains fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "npub1cld9yay0u24davpu6c35l4vldrhzvaq66pcqtg9a0j2cnjrn9rtsxx2pe6"
assert_log_contains fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "npub1n9lpnv0592cc2ps6nm0ca3qls642vx7yjsv35rkxqzj2vgds52sqgpverl"
assert_log_contains fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "context=inbound_handshake"
assert_log_contains fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "decision=denylist match"
log "ACL allowlist integration test passed"

6
testing/boringtun/docker-compose.yml
View File

@@ -6,6 +6,8 @@
networks:
bt-net:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
ipam:
config:
- subnet: 172.99.0.0/24
@@ -25,7 +27,7 @@ x-boringtun-common: &boringtun-common
services:
alice:
<<: *boringtun-common
container_name: bt-alice
container_name: bt-alice${FIPS_CI_NAME_SUFFIX:-}
hostname: alice
environment:
- ROLE=alice
@@ -36,7 +38,7 @@ services:
bob:
<<: *boringtun-common
container_name: bt-bob
container_name: bt-bob${FIPS_CI_NAME_SUFFIX:-}
hostname: bob
environment:
- ROLE=bob

4
testing/boringtun/scripts/bench.sh
View File

@@ -10,14 +10,14 @@ PARALLEL="${PARALLEL:-1}"
echo "=== boringtun iperf3 throughput (single TCP stream, ${DURATION}s) ==="
# Run iperf3 server on alice (background), client on bob.
docker exec -d bt-alice iperf3 -s -1 -B 10.99.0.1 -p 5201
docker exec -d bt-alice${FIPS_CI_NAME_SUFFIX:-} iperf3 -s -1 -B 10.99.0.1 -p 5201
sleep 1
# wait for tun handshake to settle (boringtun + WG keepalive)
sleep 2
# Client: bob → alice over WG (10.99.0.1)
OUT=$(docker exec bt-bob iperf3 -c 10.99.0.1 -p 5201 -t "$DURATION" -P "$PARALLEL" -J)
OUT=$(docker exec bt-bob${FIPS_CI_NAME_SUFFIX:-} iperf3 -c 10.99.0.1 -p 5201 -t "$DURATION" -P "$PARALLEL" -J)
# Pull SUM bps.
MBPS=$(echo "$OUT" | python3 -c "import json,sys; d=json.load(sys.stdin); print(f\"{d['end']['sum_received']['bits_per_second'] / 1_000_000:.2f}\")")

4
testing/chaos/scripts/chaos.sh
View File

@@ -65,6 +65,7 @@ VERBOSE=""
SEED=""
DURATION=""
NODES=""
SUBNET=""
while [ $# -gt 0 ]; do
case "$1" in
@@ -72,6 +73,7 @@ while [ $# -gt 0 ]; do
--seed) SEED="$2"; shift 2 ;;
--duration) DURATION="$2"; shift 2 ;;
--nodes) NODES="$2"; shift 2 ;;
--subnet) SUBNET="$2"; shift 2 ;;
--list) list_scenarios ;;
-*) echo "Error: Unknown option '$1'" >&2; usage ;;
*)
@@ -135,6 +137,7 @@ PYTHON_ARGS=("$SCENARIO_FILE")
[ -n "$VERBOSE" ] && PYTHON_ARGS+=("$VERBOSE")
[ -n "$SEED" ] && PYTHON_ARGS+=("--seed" "$SEED")
[ -n "$DURATION" ] && PYTHON_ARGS+=("--duration" "$DURATION")
[ -n "$SUBNET" ] && PYTHON_ARGS+=("--subnet" "$SUBNET")
echo "=== FIPS Stochastic Simulation ==="
echo ""
@@ -143,6 +146,7 @@ echo " File: $SCENARIO_FILE"
[ -n "$SEED" ] && echo " Seed: $SEED (override)"
[ -n "$DURATION" ] && echo " Duration: ${DURATION}s (override)"
[ -n "$NODES" ] && echo " Nodes: $NODES (override)"
[ -n "$SUBNET" ] && echo " Subnet: $SUBNET (override)"
echo ""
# If --nodes is specified, create a patched copy of the scenario file

8
testing/chaos/sim/__main__.py
View File

@@ -25,6 +25,12 @@ def main():
"--duration", type=int, default=None,
help="Override scenario duration in seconds",
)
parser.add_argument(
"--subnet", type=str, default=None,
help="Override topology subnet CIDR (e.g. 10.30.0.0/24); node IPs "
"derive from it. Used by CI to give each parallel run a "
"non-overlapping network.",
)
args = parser.parse_args()
level = logging.DEBUG if args.verbose else logging.INFO
@@ -48,6 +54,8 @@ def main():
print("Error: --duration must be >= 1", file=sys.stderr)
sys.exit(1)
scenario.duration_secs = args.duration
if args.subnet is not None:
scenario.topology.subnet = args.subnet
runner = SimRunner(scenario)
result = runner.run()

2
testing/chaos/sim/compose.py
View File

@@ -20,6 +20,8 @@ _COMPOSE_TEMPLATE = Template(
networks:
fips-net:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
ipam:
config:
- subnet: {{ subnet }}

124
testing/ci-cleanup.sh Executable file
View File

@@ -0,0 +1,124 @@
#!/bin/bash
# Reap FIPS CI docker resources: containers, networks, volumes, images.
#
# Force-removes everything created by ci-local.sh that is still around —
# whether a run finished cleanly, was preempted (SIGTERM/SIGKILL), OOM-killed,
# or crashed. Two complementary selectors make this robust no matter how a
# prior run died:
#
# 1. The CI label com.corganlabs.fips-ci=1 (attached to every direct
# `docker run`/network/volume ci-local drives). Every run additionally
# stamps com.corganlabs.fips-ci.run=<run-id> on the same resources.
# 2. The compose project-name prefix fipsci_ (every compose project ci-local
# starts is named fipsci_<run-id>_<suite>, so its containers/networks/
# volumes all carry com.docker.compose.project=fipsci_... and are named
# with that prefix).
#
# The generic CI label is shared by every run on the host, so an unscoped label
# sweep would tear down a CONCURRENT run's resources. Pass --run-id to narrow
# the label sweep to one run; a run's own teardown always does. Without it the
# label sweep stays broad, which is what a manual "reap everything" wants.
#
# Usage:
# ci-cleanup.sh Reap ALL fips-ci resources (any run)
# ci-cleanup.sh --project-prefix P Restrict the compose-project sweep to
# names starting with P (scopes the reap
# to a single run)
# ci-cleanup.sh --run-id ID Restrict the label sweep to the run
# labelled ID (leaves other runs alone)
# ci-cleanup.sh --label L Override the CI label (default above)
# ci-cleanup.sh --images "a b,c" Also `docker rmi -f` these image tags
# (space- or comma-separated)
#
# Safe to run when there is nothing to reap, and safe to run repeatedly.
# Also reachable as `ci-local.sh --reap`.
set -uo pipefail
LABEL="com.corganlabs.fips-ci=1"
RUN_LABEL_KEY="com.corganlabs.fips-ci.run"
PROJECT_PREFIX="fipsci_" # broad default: every CI run
RUN_ID="" # broad default: every CI run
IMAGES=""
while [[ $# -gt 0 ]]; do
case "$1" in
--label) LABEL="$2"; shift 2 ;;
--project-prefix) PROJECT_PREFIX="$2"; shift 2 ;;
--run-id) RUN_ID="$2"; shift 2 ;;
--images) IMAGES="$2"; shift 2 ;;
-h|--help) sed -n '2,/^set /{ /^set /d; s/^# \?//; p }' "$0"; exit 0 ;;
*) echo "Unknown option: $1" >&2; exit 2 ;;
esac
done
if ! command -v docker >/dev/null 2>&1; then
# No docker, nothing to reap.
exit 0
fi
if ! docker info >/dev/null 2>&1; then
# Daemon unreachable; treat as nothing to reap rather than wedging a caller.
exit 0
fi
# Each docker mutation is wrapped in `timeout` so a stuck daemon/resource can
# never wedge a caller (ci-local's signal trap relies on this being bounded).
TMO=30
# Selector for the label sweep. With --run-id it matches only the named run, so
# a concurrent run's resources are left alone; without it, every CI run.
if [[ -n "$RUN_ID" ]]; then
SWEEP_LABEL="${RUN_LABEL_KEY}=${RUN_ID}"
else
SWEEP_LABEL="$LABEL"
fi
# Distinct compose project names (read off container labels) that start with
# the configured prefix.
ci_projects() {
docker ps -a --format '{{.Label "com.docker.compose.project"}}' 2>/dev/null \
| grep -E "^${PROJECT_PREFIX}" | sort -u
}
reap_containers() {
# By CI label.
docker ps -aq --filter "label=${SWEEP_LABEL}" 2>/dev/null \
| xargs -r timeout "$TMO" docker rm -f >/dev/null 2>&1 || true
# By compose project (carried even when container_name is explicit).
local p
for p in $(ci_projects); do
docker ps -aq --filter "label=com.docker.compose.project=${p}" 2>/dev/null \
| xargs -r timeout "$TMO" docker rm -f >/dev/null 2>&1 || true
done
}
reap_networks() {
docker network ls -q --filter "label=${SWEEP_LABEL}" 2>/dev/null \
| xargs -r timeout "$TMO" docker network rm >/dev/null 2>&1 || true
# Compose networks are named <project>_<net> → match by name prefix so
# orphaned networks (whose containers are already gone) are still caught.
docker network ls --format '{{.Name}}' 2>/dev/null | grep -E "^${PROJECT_PREFIX}" \
| xargs -r timeout "$TMO" docker network rm >/dev/null 2>&1 || true
}
reap_volumes() {
docker volume ls -q --filter "label=${SWEEP_LABEL}" 2>/dev/null \
| xargs -r timeout "$TMO" docker volume rm >/dev/null 2>&1 || true
docker volume ls --format '{{.Name}}' 2>/dev/null | grep -E "^${PROJECT_PREFIX}" \
| xargs -r timeout "$TMO" docker volume rm >/dev/null 2>&1 || true
}
reap_images() {
[[ -z "$IMAGES" ]] && return 0
local imgs
read -ra imgs <<< "${IMAGES//,/ }"
[[ ${#imgs[@]} -eq 0 ]] && return 0
timeout "$TMO" docker rmi -f "${imgs[@]}" >/dev/null 2>&1 || true
}
# Order matters: containers reference networks/volumes, so drop them first.
reap_containers
reap_networks
reap_volumes
reap_images
exit 0

171
testing/ci-local.sh
View File

@@ -14,6 +14,9 @@
# --list List available integration suites
# --check-parity Verify this suite set matches ci.yml's integration
# matrix (see testing/check-ci-parity.sh), then exit
# --reap Force-remove all leftover FIPS CI docker resources
# (containers/networks/volumes carrying the CI label or a
# fipsci_ compose project), then exit. See ci-cleanup.sh.
# -h, --help Show this help
#
# Integration suites (default coverage):
@@ -32,8 +35,13 @@
# tor-socks5, tor-directory
#
# Exit codes:
# 0 — all stages passed
# 1 — one or more stages failed
# 0 — all stages passed
# 1 — one or more stages failed
# 130 — interrupted by SIGINT (128 + 2; run was cancelled, not a failure)
# 143 — terminated by SIGTERM (128 + 15; run was cancelled, not a failure)
#
# A preempting CI worker maps 130/143 → "cancelled" (discard, do not record a
# failing commit), 0 → green, any other non-zero → red.
#
# ── CI parity invariant ─────────────────────────────────────────────────────
# This local default suite set and the GitHub integration matrix
@@ -193,6 +201,7 @@ while [[ $# -gt 0 ]]; do
-j|--jobs) PARALLEL_JOBS="$2"; shift 2 ;;
--list) list_suites ;;
--check-parity) exec "$SCRIPT_DIR/check-ci-parity.sh" ;;
--reap) exec "$SCRIPT_DIR/ci-cleanup.sh" ;;
-h|--help) usage ;;
*) echo "Unknown option: $1"; usage ;;
esac
@@ -214,6 +223,115 @@ record() {
fi
}
# ── Per-run isolation + signal-safe teardown ───────────────────────────────
#
# This script may be preempted (a CI worker sends SIGTERM, waits ~30s, then
# SIGKILL) so it can restart on a newer tip. To make that safe:
# * every docker resource is namespaced to THIS run (compose project prefix
# + per-run image tags) so a restart never collides with a dying run;
# * a trap tears down everything this run created on signal/exit, bounded by
# `timeout` so a stuck `down` cannot wedge the trap (SIGKILL is the backstop).
# Derive a run id: honor the worker's $FIPS_CI_RUN_ID, else <short-sha>-<rand>,
# else $$-<timestamp>. Sanitize to a valid compose project / image-tag token.
if [[ -n "${FIPS_CI_RUN_ID:-}" ]]; then
CI_RUN_ID="$FIPS_CI_RUN_ID"
else
_ci_sha="$(git -C "$PROJECT_ROOT" rev-parse --short HEAD 2>/dev/null || true)"
if [[ -n "$_ci_sha" ]]; then
CI_RUN_ID="${_ci_sha}-${RANDOM}${RANDOM}"
else
CI_RUN_ID="$$-$(date +%s)"
fi
fi
CI_RUN_ID="$(printf '%s' "$CI_RUN_ID" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9_-' '-')"
# A docker image tag and a compose project name must both START with an
# alphanumeric, so strip any leading -/_ left by sanitization.
CI_RUN_ID="$(printf '%s' "$CI_RUN_ID" | sed -E 's/^[^a-z0-9]+//')"
[[ -z "$CI_RUN_ID" ]] && CI_RUN_ID="r$$"
CI_PROJECT_PREFIX="fipsci_${CI_RUN_ID}"
CI_IMAGE_TEST="fips-test:${CI_RUN_ID}"
CI_IMAGE_APP="fips-test-app:${CI_RUN_ID}"
CI_LABEL="com.corganlabs.fips-ci=1"
# Run-scoped companion to CI_LABEL. The generic label is shared by every run on
# the host, so teardown filters on this one instead — otherwise this run's reap
# would force-remove a concurrent run's containers out from under it.
CI_LABEL_RUN="com.corganlabs.fips-ci.run=${CI_RUN_ID}"
# Exported so child suite scripts + their compose/`docker run` invocations
# inherit the run identity. Compose files read FIPS_TEST_IMAGE/FIPS_TEST_APP_IMAGE
# (default :latest when unset, preserving manual `docker compose` use).
export FIPS_CI_RUN_ID="$CI_RUN_ID"
export FIPS_TEST_IMAGE="$CI_IMAGE_TEST"
export FIPS_TEST_APP_IMAGE="$CI_IMAGE_APP"
# Docker container names are GLOBAL — a compose project name does not scope
# them — so the suite compose files append this suffix to every explicit
# container_name, and the suite scripts append it wherever they address a
# container by name. Empty when unset, so a bare `docker compose up` outside
# this harness still produces today's plain names.
export FIPS_CI_NAME_SUFFIX="-${CI_RUN_ID}"
# Per-suite compose project name: ${prefix}_<suite-or-compose-basename>. Keeps
# today's intra-run distinctness (one project per compose file / chaos child)
# while adding the cross-run prefix that scopes the reap.
ci_project() { printf '%s_%s' "$CI_PROJECT_PREFIX" "$1"; }
# PIDs of in-flight parallel chaos children (subshells). The trap signals these.
CI_CHAOS_PIDS=()
CI_CLEANED=0
# Best-effort, BOUNDED teardown of every docker resource THIS run may have
# created. Idempotent (guarded), so the signal and EXIT paths don't double-run.
ci_teardown() {
[[ $CI_CLEANED -eq 1 ]] && return 0
CI_CLEANED=1
# 1. Propagate to parallel chaos children and reap them (bounded).
if [[ ${#CI_CHAOS_PIDS[@]} -gt 0 ]]; then
kill -TERM "${CI_CHAOS_PIDS[@]}" 2>/dev/null || true
local _end=$(( SECONDS + 10 )) _p
for _p in "${CI_CHAOS_PIDS[@]}"; do
while kill -0 "$_p" 2>/dev/null && (( SECONDS < _end )); do
sleep 0.3
done
done
kill -KILL "${CI_CHAOS_PIDS[@]}" 2>/dev/null || true
wait "${CI_CHAOS_PIDS[@]}" 2>/dev/null || true
fi
# 2. Remove all compose projects + direct-run resources + per-run images
# for this run. ci-cleanup.sh wraps each docker op in `timeout`; bound
# the whole sweep too so the trap can never wedge.
timeout 150 bash "$SCRIPT_DIR/ci-cleanup.sh" \
--label "$CI_LABEL" \
--run-id "$CI_RUN_ID" \
--project-prefix "$CI_PROJECT_PREFIX" \
--images "$CI_IMAGE_TEST $CI_IMAGE_APP" >/dev/null 2>&1 || true
}
on_signal() {
local sig="$1"
# Block re-entry and stop the EXIT trap from overriding the signal code.
trap '' TERM INT EXIT
echo "" >&2
fail "Received SIG$sig — cancelling run, tearing down ${CI_PROJECT_PREFIX}"
ci_teardown
# 128 + signal number: distinct from 0 (green) / 1 (stage failed).
if [[ "$sig" == "TERM" ]]; then exit 143; else exit 130; fi
}
on_exit() {
local code=$?
trap '' TERM INT EXIT
ci_teardown
exit "$code"
}
trap 'on_signal TERM' TERM
trap 'on_signal INT' INT
trap 'on_exit' EXIT
# ── Stage 1: Build ─────────────────────────────────────────────────────────
run_build() {
@@ -317,6 +435,7 @@ run_static() {
local topology="$1"
local compose="testing/static/docker-compose.yml"
local rc=0
export COMPOSE_PROJECT_NAME="$(ci_project static)"
info "[$topology] Generating configs"
bash testing/static/scripts/generate-configs.sh "$topology" || { record "static-$topology" 1; return; }
@@ -341,6 +460,7 @@ run_static() {
run_rekey() {
local compose="testing/static/docker-compose.yml"
local rc=0
export COMPOSE_PROJECT_NAME="$(ci_project static)"
info "[rekey] Generating configs"
bash testing/static/scripts/generate-configs.sh rekey || { record "rekey" 1; return; }
@@ -369,6 +489,7 @@ run_rekey() {
run_admission_cap() {
local compose="testing/static/docker-compose.yml"
local rc=0
export COMPOSE_PROJECT_NAME="$(ci_project static)"
info "[admission-cap] Generating configs"
bash testing/static/scripts/generate-configs.sh mesh || { record "admission-cap" 1; return; }
@@ -395,6 +516,9 @@ run_chaos() {
local name="$1"
shift
local rc=0
# Distinct project per scenario (chaos children run in parallel). When
# invoked from a background subshell this export is local to that child.
export COMPOSE_PROJECT_NAME="$(ci_project "chaos-$name")"
info "[chaos/$name] Running simulation"
if bash testing/chaos/scripts/chaos.sh "$@" 2>&1; then
@@ -410,6 +534,7 @@ run_chaos() {
run_gateway() {
local compose="testing/static/docker-compose.yml"
local rc=0
export COMPOSE_PROJECT_NAME="$(ci_project static)"
info "[gateway] Generating configs"
bash testing/static/scripts/generate-configs.sh gateway gateway-test || { record "gateway" 1; return; }
@@ -434,6 +559,7 @@ run_gateway() {
# Run sidecar test
run_sidecar() {
local rc=0
export COMPOSE_PROJECT_NAME="$(ci_project sidecar)"
info "[sidecar] Running integration test"
if bash testing/sidecar/scripts/test-sidecar.sh --skip-build 2>&1; then
@@ -450,6 +576,7 @@ run_sidecar() {
run_rekey_accept_off() {
local compose="testing/static/docker-compose.yml"
local rc=0
export COMPOSE_PROJECT_NAME="$(ci_project static)"
info "[rekey-accept-off] Generating configs"
bash testing/static/scripts/generate-configs.sh rekey-accept-off || \
@@ -484,6 +611,7 @@ run_rekey_accept_off() {
run_rekey_outbound_only() {
local compose="testing/static/docker-compose.yml"
local rc=0
export COMPOSE_PROJECT_NAME="$(ci_project static)"
info "[rekey-outbound-only] Generating configs"
bash testing/static/scripts/generate-configs.sh rekey-outbound-only || \
@@ -512,6 +640,7 @@ run_rekey_outbound_only() {
# Run ACL allowlist integration test
run_acl_allowlist() {
export COMPOSE_PROJECT_NAME="$(ci_project acl)"
info "[acl-allowlist] Running integration test"
if bash testing/acl-allowlist/test.sh --skip-build 2>&1; then
record "acl-allowlist" 0
@@ -522,6 +651,7 @@ run_acl_allowlist() {
# Run firewall baseline integration test
run_firewall() {
export COMPOSE_PROJECT_NAME="$(ci_project firewall)"
info "[firewall] Running integration test"
if bash testing/firewall/test.sh --skip-build 2>&1; then
record "firewall" 0
@@ -533,6 +663,7 @@ run_firewall() {
# Run a NAT scenario (cone, symmetric, lan)
run_nat() {
local scenario="$1"
export COMPOSE_PROJECT_NAME="$(ci_project nat)"
info "[nat-$scenario] Running NAT lab"
if bash testing/nat/scripts/nat-test.sh "$scenario" 2>&1; then
record "nat-$scenario" 0
@@ -546,6 +677,7 @@ run_nat() {
# (A→B publish/consume), Phase 2 (B→A reverse), and Phase 3 (malformed
# advert injected directly to the relay; consumer-liveness assertion).
run_nostr_publish_consume() {
export COMPOSE_PROJECT_NAME="$(ci_project nat)"
info "[nostr-publish-consume] Running Nostr publish/consume test"
if bash testing/nat/scripts/nostr-relay-test.sh 2>&1; then
record "nostr-publish-consume" 0
@@ -560,6 +692,7 @@ run_nostr_publish_consume() {
# kill. Asserts the daemon detects each fault, recovers from delay, and
# never panics.
run_stun_faults() {
export COMPOSE_PROJECT_NAME="$(ci_project nat)"
info "[stun-faults] Running STUN fault-injection test"
if bash testing/nat/scripts/stun-faults-test.sh 2>&1; then
record "stun-faults" 0
@@ -590,6 +723,7 @@ run_deb_install() {
# Run Tor SOCKS5 outbound test (live Tor network)
run_tor_socks5() {
export COMPOSE_PROJECT_NAME="$(ci_project tor-socks5)"
info "[tor-socks5] Running Tor SOCKS5 outbound test (live Tor)"
if bash testing/tor/socks5-outbound/scripts/tor-test.sh 2>&1; then
record "tor-socks5" 0
@@ -600,6 +734,7 @@ run_tor_socks5() {
# Run Tor directory-mode test (live Tor network)
run_tor_directory() {
export COMPOSE_PROJECT_NAME="$(ci_project tor-directory)"
info "[tor-directory] Running Tor directory-mode test (live Tor)"
if bash testing/tor/directory-mode/scripts/directory-test.sh 2>&1; then
record "tor-directory" 0
@@ -616,10 +751,20 @@ run_integration() {
info "Installing release binaries"
install_binaries testing/docker
# Build unified test image once (used by all harnesses)
info "Building fips-test Docker image"
docker build -t fips-test:latest testing/docker --quiet || { record "docker-build" 1; return; }
docker build -t fips-test-app:latest -f testing/docker/Dockerfile.app testing/docker --quiet || { record "docker-build-app" 1; return; }
# Build unified test image once (used by all harnesses). Tag per-run
# (fips-test:${run}) so a build killed mid-flight never wedges the next
# run's rebuild, and concurrent runs never clobber each other's image.
# Then retag :latest for the compose files / harness scripts that still
# reference fips-test:latest directly; the retag happens only after BOTH
# builds succeed, so :latest never points at a half-built image.
info "Building $CI_IMAGE_TEST Docker image"
docker build -t "$CI_IMAGE_TEST" --label "$CI_LABEL" --label "$CI_LABEL_RUN" testing/docker --quiet \
|| { record "docker-build" 1; return; }
docker build -t "$CI_IMAGE_APP" --label "$CI_LABEL" --label "$CI_LABEL_RUN" \
-f testing/docker/Dockerfile.app testing/docker --quiet \
|| { record "docker-build-app" 1; return; }
docker tag "$CI_IMAGE_TEST" fips-test:latest
docker tag "$CI_IMAGE_APP" fips-test-app:latest
# Single suite mode
if [[ -n "$ONLY_SUITE" ]]; then
@@ -673,6 +818,7 @@ run_integration() {
local pids=()
local suite_names=()
local running=0
local chaos_idx=0
for entry in "${CHAOS_SUITES[@]}"; do
# Parse: "display-name scenario [flags...]"
@@ -680,6 +826,15 @@ run_integration() {
local name="${parts[0]}"
local args=("${parts[@]:1}")
# Give each chaos child a unique, non-overlapping /24 in 10.30.x so
# parallel children never collide with each other, and so a chaos
# net can never swallow a fixed-subnet suite (sidecar/static/nat in
# 172.x). 10.30.x sits outside docker's default-address-pool range
# (172.17-31 / 192.168), so auto-assigned nets can't land on it
# either. Node IPs derive from this subnet inside the sim.
args+=("--subnet" "10.30.${chaos_idx}.0/24")
chaos_idx=$((chaos_idx + 1))
# Throttle: wait for a slot
while [[ $running -ge $PARALLEL_JOBS ]]; do
wait -n -p done_pid 2>/dev/null || true
@@ -693,6 +848,7 @@ run_integration() {
run_chaos "$name" "${args[@]}" >"$logfile" 2>&1
) &
pids+=($!)
CI_CHAOS_PIDS+=("$!")
suite_names+=("$name:$logfile")
running=$((running + 1))
done
@@ -715,6 +871,9 @@ run_integration() {
fi
rm -f "$logfile"
done
# All chaos children have been waited on; clear so a later signal does
# not try to kill already-reaped PIDs.
CI_CHAOS_PIDS=()
fi
# Sidecar

3
testing/deb-install/test.sh
View File

@@ -65,6 +65,7 @@ start_systemd_container_with_tun() {
local name="$1" image="$2"
cleanup_container "$name"
docker run -d --name "$name" \
--label com.corganlabs.fips-ci=1 \
--privileged \
--cgroupns=host \
--device /dev/net/tun \
@@ -186,7 +187,7 @@ _run_deb_install_scenario() {
local distro_label="$1"
local base_image="$2"
local name="fips-deb-test-${distro_label}"
local name="fips-deb-test-${distro_label}${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-deb-test:${distro_label}"
log ".deb install: ${base_image}"

20
testing/dns-resolver/test.sh
View File

@@ -67,6 +67,7 @@ start_systemd_container() {
local name="$1" image="$2"
cleanup_container "$name"
docker run -d --name "$name" \
--label com.corganlabs.fips-ci=1 \
--privileged \
--cgroupns=host \
-v /sys/fs/cgroup:/sys/fs/cgroup:rw \
@@ -79,6 +80,7 @@ start_systemd_container_with_tun() {
local name="$1" image="$2"
cleanup_container "$name"
docker run -d --name "$name" \
--label com.corganlabs.fips-ci=1 \
--privileged \
--cgroupns=host \
--device /dev/net/tun \
@@ -282,7 +284,7 @@ DOCKERFILE
# ─────────────────────────────────────────────────────────────────────
test_debian12_resolved() {
local name="fips-dns-test-deb12-resolved"
local name="fips-dns-test-deb12-resolved${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-dns-test:debian12-resolved"
log "Debian 12 + systemd-resolved (expects global-drop-in)"
@@ -310,7 +312,7 @@ DOCKERFILE
}
test_debian13_resolved() {
local name="fips-dns-test-deb13-resolved"
local name="fips-dns-test-deb13-resolved${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-dns-test:debian13-resolved"
log "Debian 13 (trixie) + systemd-resolved (expects global-drop-in)"
@@ -338,7 +340,7 @@ DOCKERFILE
}
test_ubuntu22_resolved() {
local name="fips-dns-test-u22-resolved"
local name="fips-dns-test-u22-resolved${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-dns-test:ubuntu22-resolved"
log "Ubuntu 22.04 + systemd-resolved (expects global-drop-in)"
@@ -368,7 +370,7 @@ DOCKERFILE
}
test_ubuntu24_resolved() {
local name="fips-dns-test-u24-resolved"
local name="fips-dns-test-u24-resolved${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-dns-test:ubuntu24-resolved"
log "Ubuntu 24.04 + systemd-resolved (expects global-drop-in)"
@@ -396,7 +398,7 @@ DOCKERFILE
}
test_ubuntu26_resolved() {
local name="fips-dns-test-u26-resolved"
local name="fips-dns-test-u26-resolved${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-dns-test:ubuntu26-resolved"
log "Ubuntu 26.04 + systemd-resolved (expects global-drop-in)"
@@ -424,7 +426,7 @@ DOCKERFILE
}
test_dnsmasq() {
local name="fips-dns-test-dnsmasq"
local name="fips-dns-test-dnsmasq${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-dns-test:dnsmasq"
log "Debian 12 + dnsmasq standalone"
@@ -490,7 +492,7 @@ DOCKERFILE
}
test_nm_dnsmasq() {
local name="fips-dns-test-nm-dnsmasq"
local name="fips-dns-test-nm-dnsmasq${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-dns-test:nm-dnsmasq"
log "Fedora + NetworkManager + dnsmasq plugin"
@@ -553,7 +555,7 @@ DOCKERFILE
}
test_no_resolver() {
local name="fips-dns-test-none"
local name="fips-dns-test-none${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-dns-test:none"
log "Debian 12 bare (no resolver)"
@@ -623,7 +625,7 @@ _run_e2e_scenario() {
local base_image="$2"
local apt_packages="$3"
local name="fips-dns-test-e2e-${distro_label}"
local name="fips-dns-test-e2e-${distro_label}${FIPS_CI_NAME_SUFFIX:-}"
local image="fips-dns-test:e2e-${distro_label}"
log "End-to-end: ${base_image} + systemd-resolved + real fips + fips-gateway + dig"

6
testing/firewall/docker-compose.yml
View File

@@ -1,6 +1,8 @@
networks:
fw-net:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
ipam:
config:
- subnet: 172.32.0.0/24
@@ -25,7 +27,7 @@ x-fips-common: &fips-common
services:
service-a:
<<: *fips-common
container_name: fips-fw-container-a
container_name: fips-fw-container-a${FIPS_CI_NAME_SUFFIX:-}
hostname: host-a
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -38,7 +40,7 @@ services:
service-b:
<<: *fips-common
container_name: fips-fw-container-b
container_name: fips-fw-container-b${FIPS_CI_NAME_SUFFIX:-}
hostname: host-b
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro

4
testing/firewall/test.sh
View File

@@ -22,8 +22,8 @@ TESTING_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
COMPOSE_FILE="$SCRIPT_DIR/docker-compose.yml"
GENERATE_CONFIGS="$SCRIPT_DIR/generate-configs.sh"
CONTAINER_A="fips-fw-container-a"
CONTAINER_B="fips-fw-container-b"
CONTAINER_A="fips-fw-container-a${FIPS_CI_NAME_SUFFIX:-}"
CONTAINER_B="fips-fw-container-b${FIPS_CI_NAME_SUFFIX:-}"
NPUB_A="npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
NPUB_B="npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le"

44
testing/interop/interop-test.sh
View File

@@ -162,6 +162,13 @@ REKEY_SETTLE=12 # FSP-cutover settle budget (Phase 6)
# strict assertion sweep runs. A genuinely stuck pair still fails — the
# poll times out and the recording sweep captures it.
POST_REKEY_TIMEOUT=45
# Progress-aware stall budget for the convergence detector
# (wait_for_full_baseline → wait_until_connected). If no additional pair
# becomes reachable for this long while more than the near-converged
# slack of pairs is still down, the detector gives up early instead of
# burning the whole convergence/post-rekey window; any progress resets
# the clock, so a slow-but-converging mesh under netem keeps polling.
RECONVERGE_STALL=15
LOG_POLL_INTERVAL=2
# Data-plane continuity stream (control-differential). Streams run a
@@ -434,25 +441,27 @@ ping_all_pairs() {
done
}
# Convergence detector probe: one full all-pairs ping sweep in the
# "convergence" context (which ping_all_pairs deliberately does NOT
# record as a failure), setting PASSED/FAILED for wait_until_connected.
_baseline_probe() {
ping_all_pairs quiet 1 "convergence"
}
# Poll until every directed pair pings clean, or until timeout. This is a
# convergence DETECTOR — used at establishment (Phase 1) and after each
# rekey (Phases 3/5). It pings with the "convergence" context, which
# ping_all_pairs deliberately does not record as a failure; the caller
# runs a separate strict assertion sweep afterwards.
#
# Delegates to the shared progress-aware wait_until_connected so the
# deadline extends while more pairs are still coming up and gives up fast
# on a genuine stall, instead of the prior fixed-deadline poll that could
# false-time-out under heavy CI contention even while still converging.
# Returns 0 once every pair is reachable, 1 on stall/timeout.
wait_for_full_baseline() {
local timeout="$1"
local start=$SECONDS
local best_passed=0 best_failed="$NUM_DIRECTED"
while (( SECONDS - start < timeout )); do
ping_all_pairs quiet 1 "convergence"
if [ "$PASSED" -gt "$best_passed" ]; then
best_passed="$PASSED"; best_failed="$FAILED"
fi
[ "$FAILED" -eq 0 ] && return 0
sleep 1
done
PASSED="$best_passed"; FAILED="$best_failed"
return 1
wait_until_connected _baseline_probe "$timeout" "$RECONVERGE_STALL"
}
phase_result() {
@@ -759,8 +768,15 @@ echo ""
# ── Phase 4: second rekey cycle ──────────────────────────────────────
echo "Phase 4: Second rekey cycle (waiting ${SECOND_REKEY_WAIT}s)"
sleep "$SECOND_REKEY_WAIT"
echo "Phase 4: Second rekey cycle (waiting up to ${SECOND_REKEY_WAIT}s for the next cutover)"
# Poll for the next FMP cutover beyond what Phases 2/3 already saw, using
# the same pre/post cutover-count delta convention as the control window
# (Phase 1b), instead of a blind sleep. Bounded by SECOND_REKEY_WAIT so a
# stalled rekey falls through to the strict Phase 5/6 assertions.
fmp_cutovers_before="$(count_log_pattern 'Rekey cutover complete \(initiator\), K-bit flipped')"
wait_for_log_pattern_count \
"Rekey cutover complete \(initiator\), K-bit flipped" \
"$((fmp_cutovers_before + 1))" "$SECOND_REKEY_WAIT" || true
echo ""
echo "Phase 5: Post-second-rekey connectivity (reconverge within ${POST_REKEY_TIMEOUT}s)"

34
testing/nat/docker-compose.yml
View File

@@ -1,11 +1,15 @@
networks:
wan:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
ipam:
config:
- subnet: 172.31.254.0/24
shared-lan:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
ipam:
config:
- subnet: 172.31.10.0/24
@@ -30,7 +34,7 @@ services:
build:
context: ../..
dockerfile: examples/sidecar-nostr-relay/Dockerfile.app
container_name: fips-nat-relay
container_name: fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}
restart: "no"
volumes:
- relay-data:/usr/src/app/strfry-db
@@ -45,7 +49,7 @@ services:
stun:
build:
context: ./stun
container_name: fips-nat-stun
container_name: fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}
restart: "no"
networks:
wan:
@@ -57,7 +61,7 @@ services:
build:
context: ./router
profiles: ["cone", "symmetric"]
container_name: fips-nat-router-a
container_name: fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-}
cap_add:
- NET_ADMIN
sysctls:
@@ -80,7 +84,7 @@ services:
build:
context: ./router
profiles: ["cone", "symmetric"]
container_name: fips-nat-router-b
container_name: fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-}
cap_add:
- NET_ADMIN
sysctls:
@@ -102,7 +106,7 @@ services:
cone-a:
<<: *fips-common
profiles: ["cone"]
container_name: fips-nat-cone-a
container_name: fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-cone-a
depends_on:
- nat-a
@@ -128,7 +132,7 @@ services:
cone-b:
<<: *fips-common
profiles: ["cone"]
container_name: fips-nat-cone-b
container_name: fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-cone-b
depends_on:
- nat-b
@@ -154,7 +158,7 @@ services:
symmetric-a:
<<: *fips-common
profiles: ["symmetric"]
container_name: fips-nat-symmetric-a
container_name: fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-symmetric-a
depends_on:
- nat-a
@@ -180,7 +184,7 @@ services:
symmetric-b:
<<: *fips-common
profiles: ["symmetric"]
container_name: fips-nat-symmetric-b
container_name: fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-symmetric-b
depends_on:
- nat-b
@@ -206,7 +210,7 @@ services:
lan-a:
<<: *fips-common
profiles: ["lan"]
container_name: fips-nat-lan-a
container_name: fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-lan-a
depends_on:
- relay
@@ -221,7 +225,7 @@ services:
lan-b:
<<: *fips-common
profiles: ["lan"]
container_name: fips-nat-lan-b
container_name: fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-lan-b
depends_on:
- relay
@@ -243,7 +247,7 @@ services:
nostr-pub-a:
<<: *fips-common
profiles: ["nostr-publish-consume"]
container_name: fips-nat-nostr-pub-a
container_name: fips-nat-nostr-pub-a${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-nostr-pub-a
depends_on:
- relay
@@ -258,7 +262,7 @@ services:
nostr-pub-b:
<<: *fips-common
profiles: ["nostr-publish-consume"]
container_name: fips-nat-nostr-pub-b
container_name: fips-nat-nostr-pub-b${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-nostr-pub-b
depends_on:
- relay
@@ -287,7 +291,7 @@ services:
stun-fault-node:
<<: *fips-common
profiles: ["stun-faults"]
container_name: fips-nat-stun-fault-node
container_name: fips-nat-stun-fault-node${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-stun-fault-node
depends_on:
- relay
@@ -308,7 +312,7 @@ services:
stun-fault-peer:
<<: *fips-common
profiles: ["stun-faults"]
container_name: fips-nat-stun-fault-peer
container_name: fips-nat-stun-fault-peer${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-nat-stun-fault-peer
depends_on:
- relay
@@ -323,7 +327,7 @@ services:
stun-fault-shim:
image: fips-test:latest
profiles: ["stun-faults"]
container_name: fips-nat-stun-fault-shim
container_name: fips-nat-stun-fault-shim${FIPS_CI_NAME_SUFFIX:-}
depends_on:
- stun-fault-node
cap_add:

100
testing/nat/scripts/nat-test.sh
View File

@@ -36,7 +36,7 @@ cleanup() {
}
helper_tcpdump_image() {
docker inspect -f '{{.Config.Image}}' fips-nat-router-a 2>/dev/null || echo nat-nat-a
docker inspect -f '{{.Config.Image}}' fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-} 2>/dev/null || echo nat-nat-a
}
dump_container_state() {
@@ -202,7 +202,7 @@ dump_stun_udp_probe() {
local capture_file
capture_file="$(mktemp)"
docker run --rm --net=container:fips-nat-stun --cap-add NET_ADMIN --cap-add NET_RAW \
docker run --rm --label com.corganlabs.fips-ci=1 --label "com.corganlabs.fips-ci.run=${FIPS_CI_RUN_ID:-manual}" --net=container:fips-nat-stun${FIPS_CI_NAME_SUFFIX:-} --cap-add NET_ADMIN --cap-add NET_RAW \
--entrypoint sh "$helper_image" \
-lc "timeout 8 tcpdump -ni any 'udp and not port 53' -c 80" \
>"$capture_file" 2>&1 &
@@ -225,38 +225,38 @@ dump_stun_udp_probe() {
dump_cone_diagnostics() {
echo ""
echo "=== cone diagnostics ==="
dump_fips_state fips-nat-cone-a 172.31.254.30 7777 172.31.254.40 3478
dump_node_udp_probe fips-nat-cone-a
dump_fips_state fips-nat-cone-b 172.31.254.30 7777 172.31.254.40 3478
dump_node_udp_probe fips-nat-cone-b
dump_container_state fips-nat-router-a
dump_router_udp_probe fips-nat-router-a fips-nat-cone-a
dump_container_state fips-nat-router-b
dump_router_udp_probe fips-nat-router-b fips-nat-cone-b
dump_container_state fips-nat-relay
dump_stun_udp_probe fips-nat-cone-a
dump_stun_udp_probe fips-nat-cone-b
dump_container_state fips-nat-stun
dump_fips_state fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} 172.31.254.30 7777 172.31.254.40 3478
dump_node_udp_probe fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}
dump_fips_state fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} 172.31.254.30 7777 172.31.254.40 3478
dump_node_udp_probe fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}
dump_container_state fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-}
dump_router_udp_probe fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-} fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}
dump_container_state fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-}
dump_router_udp_probe fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-} fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}
dump_container_state fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}
dump_stun_udp_probe fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}
dump_stun_udp_probe fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}
dump_container_state fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}
}
dump_symmetric_diagnostics() {
echo ""
echo "=== symmetric diagnostics ==="
dump_fips_state fips-nat-symmetric-a 172.31.254.30 7777 172.31.254.40 3478
dump_fips_state fips-nat-symmetric-b 172.31.254.30 7777 172.31.254.40 3478
dump_container_state fips-nat-router-a
dump_container_state fips-nat-router-b
dump_container_state fips-nat-relay
dump_container_state fips-nat-stun
dump_fips_state fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} 172.31.254.30 7777 172.31.254.40 3478
dump_fips_state fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} 172.31.254.30 7777 172.31.254.40 3478
dump_container_state fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-}
dump_container_state fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-}
dump_container_state fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}
dump_container_state fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}
}
dump_lan_diagnostics() {
echo ""
echo "=== lan diagnostics ==="
dump_fips_state fips-nat-lan-a 172.31.10.30 7777 172.31.10.40 3478
dump_fips_state fips-nat-lan-b 172.31.10.30 7777 172.31.10.40 3478
dump_container_state fips-nat-relay
dump_container_state fips-nat-stun
dump_fips_state fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} 172.31.10.30 7777 172.31.10.40 3478
dump_fips_state fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} 172.31.10.30 7777 172.31.10.40 3478
dump_container_state fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}
dump_container_state fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}
}
trap 'echo ""; echo "NAT test interrupted"; cleanup; exit 130' INT TERM
@@ -334,22 +334,22 @@ run_cone() {
"$GENERATE_SCRIPT" cone
"${COMPOSE[@]}" --profile cone up -d --build --force-recreate
"$TOPOLOGY_SCRIPT" cone
wait_for_peers fips-nat-cone-a 1 45 || {
wait_for_peers fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} 1 45 || {
dump_cone_diagnostics
return 1
}
wait_for_peers fips-nat-cone-b 1 45 || {
wait_for_peers fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} 1 45 || {
dump_cone_diagnostics
return 1
}
assert_peer_path fips-nat-cone-a udp 172.31.254.
assert_peer_path fips-nat-cone-b udp 172.31.254.
assert_link_path fips-nat-cone-a 172.31.254.
assert_link_path fips-nat-cone-b 172.31.254.
assert_peer_path fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} udp 172.31.254.
assert_peer_path fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} udp 172.31.254.
assert_link_path fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} 172.31.254.
assert_link_path fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} 172.31.254.
# shellcheck disable=SC1090
source "$NAT_DIR/generated-configs/cone/npubs.env"
ping_peer fips-nat-cone-a "$NPUB_B"
ping_peer fips-nat-cone-b "$NPUB_A"
ping_peer fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B"
ping_peer fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A"
cleanup
}
@@ -359,24 +359,24 @@ run_symmetric() {
NAT_MODE_A=symmetric NAT_MODE_B=symmetric "$GENERATE_SCRIPT" symmetric
NAT_MODE_A=symmetric NAT_MODE_B=symmetric "${COMPOSE[@]}" --profile symmetric up -d --build --force-recreate
"$TOPOLOGY_SCRIPT" symmetric
wait_for_peers fips-nat-symmetric-a 1 60 || {
wait_for_peers fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} 1 60 || {
dump_symmetric_diagnostics
return 1
}
wait_for_peers fips-nat-symmetric-b 1 60 || {
wait_for_peers fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} 1 60 || {
dump_symmetric_diagnostics
return 1
}
assert_peer_path fips-nat-symmetric-a tcp 172.31.254.11:
assert_peer_path fips-nat-symmetric-b tcp 172.31.254.10:
assert_link_path fips-nat-symmetric-a 172.31.254.11:
assert_link_path fips-nat-symmetric-b 172.31.254.10:
require_bootstrap_activity fips-nat-symmetric-a
require_bootstrap_activity fips-nat-symmetric-b
assert_peer_path fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} tcp 172.31.254.11:
assert_peer_path fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} tcp 172.31.254.10:
assert_link_path fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} 172.31.254.11:
assert_link_path fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} 172.31.254.10:
require_bootstrap_activity fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-}
require_bootstrap_activity fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-}
# shellcheck disable=SC1090
source "$NAT_DIR/generated-configs/symmetric/npubs.env"
ping_peer fips-nat-symmetric-a "$NPUB_B"
ping_peer fips-nat-symmetric-b "$NPUB_A"
ping_peer fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B"
ping_peer fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A"
cleanup
}
@@ -385,22 +385,22 @@ run_lan() {
cleanup
"$GENERATE_SCRIPT" lan
"${COMPOSE[@]}" --profile lan up -d --build --force-recreate
wait_for_peers fips-nat-lan-a 1 45 || {
wait_for_peers fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} 1 45 || {
dump_lan_diagnostics
return 1
}
wait_for_peers fips-nat-lan-b 1 45 || {
wait_for_peers fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} 1 45 || {
dump_lan_diagnostics
return 1
}
assert_peer_path fips-nat-lan-a udp 172.31.10.
assert_peer_path fips-nat-lan-b udp 172.31.10.
assert_link_path fips-nat-lan-a 172.31.10.
assert_link_path fips-nat-lan-b 172.31.10.
assert_peer_path fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} udp 172.31.10.
assert_peer_path fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} udp 172.31.10.
assert_link_path fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} 172.31.10.
assert_link_path fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} 172.31.10.
# shellcheck disable=SC1090
source "$NAT_DIR/generated-configs/lan/npubs.env"
ping_peer fips-nat-lan-a "$NPUB_B"
ping_peer fips-nat-lan-b "$NPUB_A"
ping_peer fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B"
ping_peer fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A"
# Skip the final teardown when the mesh-lab harness wraps this
# script: it needs to docker-logs the containers before teardown,
# and will run its own cleanup after capture. Failure paths above

6
testing/nat/scripts/nostr-relay-test.sh
View File

@@ -25,11 +25,11 @@ WAIT_LIB="$ROOT_DIR/testing/lib/wait-converge.sh"
PROFILE="nostr-publish-consume"
SCENARIO="$PROFILE"
COMPOSE=(docker compose -f "$NAT_DIR/docker-compose.yml")
NODE_A="fips-nat-nostr-pub-a"
NODE_B="fips-nat-nostr-pub-b"
NODE_A="fips-nat-nostr-pub-a${FIPS_CI_NAME_SUFFIX:-}"
NODE_B="fips-nat-nostr-pub-b${FIPS_CI_NAME_SUFFIX:-}"
RELAY_HOST="172.31.10.30"
RELAY_PORT=7777
RELAY_CONTAINER="fips-nat-relay"
RELAY_CONTAINER="fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}"
# shellcheck disable=SC1090
source "$WAIT_LIB"

13
testing/nat/scripts/setup-topology.sh
View File

@@ -9,12 +9,12 @@ SCENARIO="${1:?usage: setup-topology.sh <cone|symmetric>}"
case "$SCENARIO" in
cone)
node_a="fips-nat-cone-a"
node_b="fips-nat-cone-b"
node_a="fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}"
node_b="fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}"
;;
symmetric)
node_a="fips-nat-symmetric-a"
node_b="fips-nat-symmetric-b"
node_a="fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-}"
node_b="fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-}"
;;
*)
echo "Unsupported topology scenario: $SCENARIO" >&2
@@ -22,8 +22,8 @@ case "$SCENARIO" in
;;
esac
router_a="fips-nat-router-a"
router_b="fips-nat-router-b"
router_a="fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-}"
router_b="fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-}"
helper_image() {
if [ -n "${IP_HELPER_IMAGE:-}" ]; then
@@ -57,6 +57,7 @@ run_host_ip() {
local image="$1"
shift
docker run --rm \
--label com.corganlabs.fips-ci=1 \
--privileged \
--net=host \
--pid=host \

8
testing/nat/scripts/stun-faults-test.sh
View File

@@ -33,10 +33,10 @@ GENERATE_SCRIPT="$SCRIPT_DIR/generate-configs.sh"
PROFILE="stun-faults"
SCENARIO="$PROFILE"
COMPOSE=(docker compose -f "$NAT_DIR/docker-compose.yml")
NODE="fips-nat-stun-fault-node"
PEER="fips-nat-stun-fault-peer"
SHIM="fips-nat-stun-fault-shim"
STUN_CONTAINER="fips-nat-stun"
NODE="fips-nat-stun-fault-node${FIPS_CI_NAME_SUFFIX:-}"
PEER="fips-nat-stun-fault-peer${FIPS_CI_NAME_SUFFIX:-}"
SHIM="fips-nat-stun-fault-shim${FIPS_CI_NAME_SUFFIX:-}"
STUN_CONTAINER="fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}"
STUN_HOST="172.31.10.40"
STUN_PORT=3478
DEV="eth0"

10
testing/sidecar/docker-compose.yml
View File

@@ -2,6 +2,12 @@ networks:
fips-net:
name: ${FIPS_NETWORK:-fips-sidecar-net}
driver: bridge
# Reap label: this harness uses fixed -p sidecar-{a,b,c} project names
# (the test execs containers by those derived names), so its resources are
# NOT covered by ci-local's fipsci_ project prefix. Label them directly so
# ci-cleanup.sh can still reap them after a preemption/SIGKILL.
labels:
- "com.corganlabs.fips-ci=1"
ipam:
config:
- subnet: ${FIPS_SUBNET:-172.20.1.0/24}
@@ -10,6 +16,8 @@ services:
fips:
image: fips-test:latest
hostname: fips-sidecar
labels:
- "com.corganlabs.fips-ci=1"
cap_add:
- NET_ADMIN
devices:
@@ -33,6 +41,8 @@ services:
app:
image: fips-test-app:latest
labels:
- "com.corganlabs.fips-ci=1"
network_mode: "service:fips"
depends_on:
- fips

123
testing/sidecar/scripts/test-sidecar.sh
View File

@@ -25,11 +25,28 @@ NODE_B_NPUB="npub15h7z0ljzudqe9pgwx99cjsz2c0ennuyvkcc8zvtk3lg97xwzex9ska6g4y"
NODE_C_NSEC="15148ed0131f7da43fd13e369dfedede14fb64698f3756636b569c3a3e87438f"
NODE_C_NPUB="npub1zhezcykd0e34z4fxtranl45jaasgnlxv0kjqwlq2v56ggssn0w4qelcrvr"
NETWORK_NAME="fips-sidecar-test"
SUBNET="172.20.2.0/24"
NODE_A_IP="172.20.2.10"
NODE_B_IP="172.20.2.11"
NODE_C_IP="172.20.2.12"
NETWORK_NAME="fips-sidecar-test${FIPS_CI_NAME_SUFFIX:-}"
# Compose project names. These are explicit (they override COMPOSE_PROJECT_NAME),
# so they carry the run suffix themselves — otherwise concurrent runs would share
# one project and tear down each other's containers.
PROJ_A="sidecar-a${FIPS_CI_NAME_SUFFIX:-}"
PROJ_B="sidecar-b${FIPS_CI_NAME_SUFFIX:-}"
PROJ_C="sidecar-c${FIPS_CI_NAME_SUFFIX:-}"
# Network address range — claimed at startup by alloc_network below rather than
# hardcoded, so two concurrent runs cannot request the same range.
#
# 10.40.0.0/16 is unclaimed in-tree: the suites use 172.20-172.32 and 172.99,
# the chaos children 10.30.x, boringtun's inner network 10.99.x, and docker's
# own default pool is 172.17-31 plus 192.168.
NET_BASE="10.40"
NET_CANDIDATES=64
SUBNET=""
GATEWAY_IP=""
NODE_A_IP=""
NODE_B_IP=""
NODE_C_IP=""
CONVERGE_TIMEOUT=30
PASSED=0
@@ -45,13 +62,50 @@ log() { echo "=== $*"; }
pass() { echo " PASS: $*"; PASSED=$((PASSED + 1)); }
fail() { echo " FAIL: $*"; FAILED=$((FAILED + 1)); }
# Claim a free /24 for this run's network.
#
# Claim-and-advance rather than deriving an offset from the run id: we attempt
# creation on a candidate range and, if docker reports the range is taken, move
# to the next. Docker's own address pool is then the arbiter, which makes an
# overlap between two concurrent runs *impossible*. A hashed offset would only
# make it unlikely, and an overlap is precisely the failure being avoided here.
#
# Deliberately does NOT discard stderr: only an address-pool conflict is worth
# advancing on. Any other failure (name already taken, daemon error) is real,
# and retrying it 64 times would bury the reason.
alloc_network() {
local i err
for (( i = 0; i < NET_CANDIDATES; i++ )); do
SUBNET="${NET_BASE}.${i}.0/24"
if err=$(docker network create \
--subnet "$SUBNET" \
--label com.corganlabs.fips-ci=1 \
--label "com.corganlabs.fips-ci.run=${FIPS_CI_RUN_ID:-manual}" \
"$NETWORK_NAME" 2>&1); then
GATEWAY_IP="${NET_BASE}.${i}.1"
NODE_A_IP="${NET_BASE}.${i}.10"
NODE_B_IP="${NET_BASE}.${i}.11"
NODE_C_IP="${NET_BASE}.${i}.12"
log "Claimed network $NETWORK_NAME on $SUBNET"
return 0
fi
case "$err" in
*"Pool overlaps"*|*"pool overlaps"*) continue ;;
*) echo " FAIL: docker network create: $err" >&2; return 1 ;;
esac
done
echo " FAIL: no free /24 in ${NET_BASE}.0.0/16 after ${NET_CANDIDATES} attempts" >&2
return 1
}
cleanup() {
log "Cleaning up..."
# Tear down B and C first (they reference A's network as external)
docker compose $COMPOSE_EXT -p sidecar-c down --volumes --remove-orphans 2>/dev/null || true
docker compose $COMPOSE_EXT -p sidecar-b down --volumes --remove-orphans 2>/dev/null || true
# Tear down A last (it owns the network)
docker compose $COMPOSE_BASE -p sidecar-a down --volumes --remove-orphans 2>/dev/null || true
# Tear down C and B first, then A. All three attach to the network as
# external, so none of them removes it — we do that last, by name.
docker compose $COMPOSE_EXT -p "$PROJ_C" down --volumes --remove-orphans 2>/dev/null || true
docker compose $COMPOSE_EXT -p "$PROJ_B" down --volumes --remove-orphans 2>/dev/null || true
docker compose $COMPOSE_EXT -p "$PROJ_A" down --volumes --remove-orphans 2>/dev/null || true
docker network rm "$NETWORK_NAME" >/dev/null 2>&1 || true
}
# Always clean up on exit
@@ -73,7 +127,9 @@ fi
# node-b: peers with A (middle node, transit router)
# node-c: peers with B (end node)
log "Starting node-a (no peers, creates network)..."
alloc_network || exit 1
log "Starting node-a (no peers, joins the claimed network)..."
# node-a is the root: explicitly clear FIPS_PEER_* so it does not inherit the
# external peer default from .env (which points at a real public mesh node).
# Without this, node-a auto-connects to the live mesh and the chain attaches
@@ -84,7 +140,7 @@ FIPS_PEER_ADDR="" \
FIPS_NETWORK="$NETWORK_NAME" \
FIPS_SUBNET="$SUBNET" \
FIPS_IPV4="$NODE_A_IP" \
docker compose $COMPOSE_BASE -p sidecar-a up -d
docker compose $COMPOSE_EXT -p "$PROJ_A" up -d
log "Starting node-b (peers with node-a, joins external network)..."
FIPS_NSEC="$NODE_B_NSEC" \
@@ -94,7 +150,7 @@ FIPS_PEER_ALIAS="node-a" \
FIPS_NETWORK="$NETWORK_NAME" \
FIPS_SUBNET="$SUBNET" \
FIPS_IPV4="$NODE_B_IP" \
docker compose $COMPOSE_EXT -p sidecar-b up -d
docker compose $COMPOSE_EXT -p "$PROJ_B" up -d
log "Starting node-c (peers with node-b, joins external network)..."
FIPS_NSEC="$NODE_C_NSEC" \
@@ -104,7 +160,7 @@ FIPS_PEER_ALIAS="node-b" \
FIPS_NETWORK="$NETWORK_NAME" \
FIPS_SUBNET="$SUBNET" \
FIPS_IPV4="$NODE_C_IP" \
docker compose $COMPOSE_EXT -p sidecar-c up -d
docker compose $COMPOSE_EXT -p "$PROJ_C" up -d
# ── Wait for convergence ──────────────────────────────────────────────────
@@ -113,7 +169,7 @@ log "Waiting for link establishment (up to ${CONVERGE_TIMEOUT}s)..."
converged=false
for i in $(seq 1 "$CONVERGE_TIMEOUT"); do
# node-b should have 2 links (A and C)
link_count=$(docker exec sidecar-b-fips-1 fipsctl show links 2>/dev/null \
link_count=$(docker exec "${PROJ_B}-fips-1" fipsctl show links 2>/dev/null \
| grep -c '"state": "connected"' || true)
if [ "$link_count" -ge 2 ]; then
converged=true
@@ -127,11 +183,11 @@ if [ "$converged" = true ]; then
else
log "TIMEOUT: links did not converge in ${CONVERGE_TIMEOUT}s"
log "node-a links:"
docker exec sidecar-a-fips-1 fipsctl show links 2>&1 || true
docker exec "${PROJ_A}-fips-1" fipsctl show links 2>&1 || true
log "node-b links:"
docker exec sidecar-b-fips-1 fipsctl show links 2>&1 || true
docker exec "${PROJ_B}-fips-1" fipsctl show links 2>&1 || true
log "node-c links:"
docker exec sidecar-c-fips-1 fipsctl show links 2>&1 || true
docker exec "${PROJ_C}-fips-1" fipsctl show links 2>&1 || true
exit 1
fi
@@ -141,9 +197,9 @@ fi
# chain. The directed pings below remain the actual assertions.
_sidecar_converged() {
PASSED=0; FAILED=0
if docker exec sidecar-b-app-1 ping6 -c1 -W2 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
if docker exec sidecar-c-app-1 ping6 -c1 -W2 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
if docker exec sidecar-a-app-1 ping6 -c1 -W2 "${NODE_C_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
if docker exec "${PROJ_B}-app-1" ping6 -c1 -W2 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
if docker exec "${PROJ_C}-app-1" ping6 -c1 -W2 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
if docker exec "${PROJ_A}-app-1" ping6 -c1 -W2 "${NODE_C_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
}
wait_until_connected _sidecar_converged "$CONVERGE_TIMEOUT" 10 || true
@@ -151,11 +207,11 @@ wait_until_connected _sidecar_converged "$CONVERGE_TIMEOUT" 10 || true
log "Verifying link counts..."
a_links=$(docker exec sidecar-a-fips-1 fipsctl show links 2>/dev/null \
a_links=$(docker exec "${PROJ_A}-fips-1" fipsctl show links 2>/dev/null \
| grep -c '"state": "connected"' || true)
b_links=$(docker exec sidecar-b-fips-1 fipsctl show links 2>/dev/null \
b_links=$(docker exec "${PROJ_B}-fips-1" fipsctl show links 2>/dev/null \
| grep -c '"state": "connected"' || true)
c_links=$(docker exec sidecar-c-fips-1 fipsctl show links 2>/dev/null \
c_links=$(docker exec "${PROJ_C}-fips-1" fipsctl show links 2>/dev/null \
| grep -c '"state": "connected"' || true)
[ "$a_links" -ge 1 ] && pass "node-a has $a_links link(s)" || fail "node-a has $a_links links (expected >= 1)"
@@ -166,7 +222,7 @@ c_links=$(docker exec sidecar-c-fips-1 fipsctl show links 2>/dev/null \
log "Testing direct connectivity (B app → A via fips0)..."
if docker exec sidecar-b-app-1 ping6 -c2 -W5 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then
if docker exec "${PROJ_B}-app-1" ping6 -c2 -W5 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then
pass "node-b app can ping node-a via fips0"
else
fail "node-b app cannot ping node-a via fips0"
@@ -176,7 +232,7 @@ fi
log "Testing multi-hop connectivity (C app → A via fips0, through B)..."
if docker exec sidecar-c-app-1 ping6 -c2 -W10 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then
if docker exec "${PROJ_C}-app-1" ping6 -c2 -W10 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then
pass "node-c app can ping node-a via fips0 (multi-hop through B)"
else
fail "node-c app cannot ping node-a via fips0 (multi-hop through B)"
@@ -186,7 +242,7 @@ fi
log "Testing reverse multi-hop (A app → C via fips0, through B)..."
if docker exec sidecar-a-app-1 ping6 -c2 -W10 "${NODE_C_NPUB}.fips" >/dev/null 2>&1; then
if docker exec "${PROJ_A}-app-1" ping6 -c2 -W10 "${NODE_C_NPUB}.fips" >/dev/null 2>&1; then
pass "node-a app can ping node-c via fips0 (multi-hop through B)"
else
fail "node-a app cannot ping node-c via fips0 (multi-hop through B)"
@@ -200,7 +256,14 @@ fi
log "Verifying network isolation on app containers..."
for node in a b c; do
container="sidecar-${node}-app-1"
container="sidecar-${node}${FIPS_CI_NAME_SUFFIX:-}-app-1"
# Fail loudly if the container is missing. Without this the two isolation
# assertions below pass vacuously: they assert a ping FAILS, and a ping into
# a non-existent container fails for the wrong reason. Only the loopback
# check, which expects success, would notice — so a naming slip here would
# silently turn the security assertions into no-ops.
docker inspect "$container" >/dev/null 2>&1 \
|| { fail "$container does not exist — isolation assertions cannot be trusted"; continue; }
# Pick a peer IP that isn't this node's own address
case $node in
a) peer_ip="$NODE_B_IP" ;;
@@ -210,7 +273,7 @@ for node in a b c; do
log " Checking $container..."
# IPv4 gateway should be unreachable (iptables DROP on eth0)
if docker exec "$container" ping -c1 -W2 172.20.2.1 >/dev/null 2>&1; then
if docker exec "$container" ping -c1 -W2 "$GATEWAY_IP" >/dev/null 2>&1; then
fail "$container can reach IPv4 gateway (isolation broken!)"
else
pass "$container cannot reach IPv4 gateway (IPv4 blocked)"
@@ -240,7 +303,7 @@ if [ "$FAILED" -gt 0 ]; then
log "Dumping logs for failed run..."
for node in a b c; do
echo "--- sidecar-${node} logs ---"
docker logs "sidecar-${node}-fips-1" 2>&1 | tail -30
docker logs "sidecar-${node}${FIPS_CI_NAME_SUFFIX:-}-fips-1" 2>&1 | tail -30
echo ""
done
exit 1

84
testing/static/docker-compose.yml
View File

@@ -1,11 +1,15 @@
networks:
fips-net:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
ipam:
config:
- subnet: 172.20.0.0/24
gateway-lan:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
enable_ipv6: true
ipam:
config:
@@ -37,7 +41,7 @@ services:
node-a:
<<: *fips-common
profiles: ["mesh"]
container_name: fips-node-a
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
hostname: node-a
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -49,7 +53,7 @@ services:
node-b:
<<: *fips-common
profiles: ["mesh"]
container_name: fips-node-b
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
hostname: node-b
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -61,7 +65,7 @@ services:
node-c:
<<: *fips-common
profiles: ["mesh"]
container_name: fips-node-c
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
hostname: node-c
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -73,7 +77,7 @@ services:
node-d:
<<: *fips-common
profiles: ["mesh"]
container_name: fips-node-d
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
hostname: node-d
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -85,7 +89,7 @@ services:
node-e:
<<: *fips-common
profiles: ["mesh"]
container_name: fips-node-e
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
hostname: node-e
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -98,7 +102,7 @@ services:
pub-a:
<<: *fips-common
profiles: ["mesh-public"]
container_name: fips-node-a
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
hostname: node-a
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -110,7 +114,7 @@ services:
pub-b:
<<: *fips-common
profiles: ["mesh-public"]
container_name: fips-node-b
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
hostname: node-b
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -122,7 +126,7 @@ services:
pub-c:
<<: *fips-common
profiles: ["mesh-public"]
container_name: fips-node-c
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
hostname: node-c
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -134,7 +138,7 @@ services:
pub-d:
<<: *fips-common
profiles: ["mesh-public"]
container_name: fips-node-d
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
hostname: node-d
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -146,7 +150,7 @@ services:
pub-e:
<<: *fips-common
profiles: ["mesh-public"]
container_name: fips-node-e
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
hostname: node-e
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -159,7 +163,7 @@ services:
chain-a:
<<: *fips-common
profiles: ["chain"]
container_name: fips-node-a
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
hostname: node-a
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -171,7 +175,7 @@ services:
chain-b:
<<: *fips-common
profiles: ["chain"]
container_name: fips-node-b
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
hostname: node-b
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -183,7 +187,7 @@ services:
chain-c:
<<: *fips-common
profiles: ["chain"]
container_name: fips-node-c
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
hostname: node-c
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -195,7 +199,7 @@ services:
chain-d:
<<: *fips-common
profiles: ["chain"]
container_name: fips-node-d
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
hostname: node-d
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -207,7 +211,7 @@ services:
chain-e:
<<: *fips-common
profiles: ["chain"]
container_name: fips-node-e
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
hostname: node-e
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -220,7 +224,7 @@ services:
rekey-a:
<<: *fips-common
profiles: ["rekey"]
container_name: fips-node-a
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
hostname: node-a
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug
@@ -234,7 +238,7 @@ services:
rekey-b:
<<: *fips-common
profiles: ["rekey"]
container_name: fips-node-b
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
hostname: node-b
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug
@@ -248,7 +252,7 @@ services:
rekey-c:
<<: *fips-common
profiles: ["rekey"]
container_name: fips-node-c
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
hostname: node-c
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug
@@ -262,7 +266,7 @@ services:
rekey-d:
<<: *fips-common
profiles: ["rekey"]
container_name: fips-node-d
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
hostname: node-d
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug
@@ -276,7 +280,7 @@ services:
rekey-e:
<<: *fips-common
profiles: ["rekey"]
container_name: fips-node-e
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
hostname: node-e
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug
@@ -295,7 +299,7 @@ services:
rekey-accept-off-a:
<<: *fips-common
profiles: ["rekey-accept-off"]
container_name: fips-node-a
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
hostname: node-a
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -309,7 +313,7 @@ services:
rekey-accept-off-b:
<<: *fips-common
profiles: ["rekey-accept-off"]
container_name: fips-node-b
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
hostname: node-b
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -323,7 +327,7 @@ services:
rekey-accept-off-c:
<<: *fips-common
profiles: ["rekey-accept-off"]
container_name: fips-node-c
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
hostname: node-c
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -337,7 +341,7 @@ services:
rekey-accept-off-d:
<<: *fips-common
profiles: ["rekey-accept-off"]
container_name: fips-node-d
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
hostname: node-d
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -351,7 +355,7 @@ services:
rekey-accept-off-e:
<<: *fips-common
profiles: ["rekey-accept-off"]
container_name: fips-node-e
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
hostname: node-e
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -375,7 +379,7 @@ services:
rekey-outbound-only-a:
<<: *fips-common
profiles: ["rekey-outbound-only"]
container_name: fips-node-a
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
hostname: node-a
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -389,7 +393,7 @@ services:
rekey-outbound-only-b:
<<: *fips-common
profiles: ["rekey-outbound-only"]
container_name: fips-node-b
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
hostname: node-b
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -403,7 +407,7 @@ services:
rekey-outbound-only-c:
<<: *fips-common
profiles: ["rekey-outbound-only"]
container_name: fips-node-c
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
hostname: node-c
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -417,7 +421,7 @@ services:
rekey-outbound-only-d:
<<: *fips-common
profiles: ["rekey-outbound-only"]
container_name: fips-node-d
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
hostname: node-d
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -431,7 +435,7 @@ services:
rekey-outbound-only-e:
<<: *fips-common
profiles: ["rekey-outbound-only"]
container_name: fips-node-e
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
hostname: node-e
environment:
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
@@ -446,7 +450,7 @@ services:
tcp-a:
<<: *fips-common
profiles: ["tcp-chain"]
container_name: fips-node-a
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
hostname: node-a
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -458,7 +462,7 @@ services:
tcp-b:
<<: *fips-common
profiles: ["tcp-chain"]
container_name: fips-node-b
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
hostname: node-b
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -470,7 +474,7 @@ services:
tcp-c:
<<: *fips-common
profiles: ["tcp-chain"]
container_name: fips-node-c
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
hostname: node-c
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -483,7 +487,7 @@ services:
gw-gateway:
<<: *fips-common
profiles: ["gateway"]
container_name: fips-gw-gateway
container_name: fips-gw-gateway${FIPS_CI_NAME_SUFFIX:-}
hostname: gw-gateway
# Privileged required: gateway must enable IPv6 on eth1 (second network,
# attached after container start) and manage nftables NAT rules.
@@ -509,7 +513,7 @@ services:
gw-server:
<<: *fips-common
profiles: ["gateway"]
container_name: fips-gw-server
container_name: fips-gw-server${FIPS_CI_NAME_SUFFIX:-}
hostname: gw-server
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -524,7 +528,7 @@ services:
gw-server-2:
<<: *fips-common
profiles: ["gateway"]
container_name: fips-gw-server-2
container_name: fips-gw-server-2${FIPS_CI_NAME_SUFFIX:-}
hostname: gw-server-2
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
@@ -534,9 +538,9 @@ services:
ipv4_address: 172.20.0.12
gw-client:
image: fips-test-app:latest
image: ${FIPS_TEST_APP_IMAGE:-fips-test-app:latest}
profiles: ["gateway"]
container_name: fips-gw-client
container_name: fips-gw-client${FIPS_CI_NAME_SUFFIX:-}
hostname: gw-client
cap_add:
- NET_ADMIN
@@ -556,9 +560,9 @@ services:
# Same image and gateway-lan attachment as
# gw-client; the gateway must allocate a distinct virtual IP for it.
gw-client-2:
image: fips-test-app:latest
image: ${FIPS_TEST_APP_IMAGE:-fips-test-app:latest}
profiles: ["gateway"]
container_name: fips-gw-client-2
container_name: fips-gw-client-2${FIPS_CI_NAME_SUFFIX:-}
hostname: gw-client-2
cap_add:
- NET_ADMIN

42
testing/static/scripts/admission-cap-test.sh
View File

@@ -87,13 +87,23 @@ CAP_IP=$(node_ip "$CAP_NODE")
[ -n "$CAP_IP" ] || fail "could not resolve docker_ip for node-$CAP_NODE in $TOPO_FILE"
info "cap'd node: node-$CAP_NODE (ip $CAP_IP, max_peers=$MAX_PEERS)"
# Read the cap'd node's peer_count, or the empty string if it did not answer.
#
# Empty is deliberately distinct from a real 0. An `|| echo 0` fallback cannot
# do that job here: `||` binds to the last stage of the pipeline, which succeeds
# on empty input, so the fallback never fires and an unreachable container would
# be reported as a legitimate zero.
read_peer_count() {
docker exec "fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show status 2>/dev/null \
| grep -m1 peer_count | sed 's/.*: *//' | tr -d ','
}
# ── Phase 1: wait for convergence ────────────────────────────────────
info "phase 1: wait for node-$CAP_NODE peer_count to reach $MAX_PEERS (90s timeout)"
deadline=$(($(date +%s) + 90))
pc=0
while [ "$(date +%s)" -lt "$deadline" ]; do
pc=$(docker exec fips-node-$CAP_NODE fipsctl show status 2>/dev/null \
| grep -m1 peer_count | sed 's/.*: *//' | tr -d ',' || echo 0)
pc=$(read_peer_count)
[ "$pc" = "$MAX_PEERS" ] && break
sleep 2
done
@@ -102,7 +112,7 @@ done
info "node-$CAP_NODE converged: peer_count=$pc"
# Identify admitted vs denied peers among configured peers
ADMITTED_NPUBS=$(docker exec fips-node-$CAP_NODE fipsctl show peers 2>/dev/null \
ADMITTED_NPUBS=$(docker exec "fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show peers 2>/dev/null \
| grep -oE 'npub1[a-z0-9]+' | sort -u || true)
DENIED=""
ADMITTED=""
@@ -130,8 +140,8 @@ info "denied (sustained-retry): ${DENIED:-<none>}"
# with restarts every 15s we get ~30-50 firings across both denied peers.
info "phase 2: capture UDP/2121 on node-$CAP_NODE for ${CAPTURE_SECS}s, with denied-peer restart loop"
CAP_FILE=$(mktemp /tmp/admission-cap-pcap.XXXXXX.txt)
HELPER_IMAGE=$(docker inspect -f '{{.Config.Image}}' fips-node-$CAP_NODE 2>/dev/null)
[ -n "$HELPER_IMAGE" ] || fail "could not resolve helper image from fips-node-$CAP_NODE"
HELPER_IMAGE=$(docker inspect -f '{{.Config.Image}}' "fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}" 2>/dev/null)
[ -n "$HELPER_IMAGE" ] || fail "could not resolve helper image from fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}"
# Background: cycle denied peers to reset their backoff and drive load.
(
@@ -140,7 +150,7 @@ HELPER_IMAGE=$(docker inspect -f '{{.Config.Image}}' fips-node-$CAP_NODE 2>/dev/
sleep 15
elapsed=$((elapsed + 15))
for n in $DENIED; do
docker restart "fips-node-$n" >/dev/null 2>&1 &
docker restart "fips-node-${n}${FIPS_CI_NAME_SUFFIX:-}" >/dev/null 2>&1 &
done
wait
info " [load-driver] restarted denied peers ($DENIED) at t+${elapsed}s"
@@ -149,7 +159,7 @@ HELPER_IMAGE=$(docker inspect -f '{{.Config.Image}}' fips-node-$CAP_NODE 2>/dev/
LOAD_PID=$!
# Foreground: tcpdump capture for CAPTURE_SECS
docker run --rm --net=container:fips-node-$CAP_NODE \
docker run --rm --label com.corganlabs.fips-ci=1 --label "com.corganlabs.fips-ci.run=${FIPS_CI_RUN_ID:-manual}" --net=container:"fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}" \
--cap-add NET_ADMIN --cap-add NET_RAW \
--entrypoint sh "$HELPER_IMAGE" \
-c "timeout $CAPTURE_SECS tcpdump -nn -i any 'udp port 2121' -l 2>&1 || true" \
@@ -186,9 +196,21 @@ for n in $DENIED; do
done
# ── Phase 4: cap'd node still at exactly max_peers ───────────────────
pc_final=$(docker exec fips-node-$CAP_NODE fipsctl show status 2>/dev/null \
| grep -m1 peer_count | sed 's/.*: *//' | tr -d ',' || echo 0)
info "node-$CAP_NODE final peer_count=$pc_final (expected $MAX_PEERS)"
#
# Polled rather than sampled once. The load driver above restarts the denied
# peers every 15s and its final restart lands in the same second this runs, so a
# single read can hit a daemon busy with those restarts and come back empty --
# which is a harness race, not a cap regression. Retry briefly before believing
# the answer. A genuine regression still fails, just after the retry window,
# because the loop exits early only on the expected value.
pc_final=""
deadline=$(($(date +%s) + 30))
while [ "$(date +%s)" -lt "$deadline" ]; do
pc_final=$(read_peer_count)
[ "$pc_final" = "$MAX_PEERS" ] && break
sleep 2
done
info "node-$CAP_NODE final peer_count=${pc_final:-<no answer>} (expected $MAX_PEERS)"
[ "$pc_final" = "$MAX_PEERS" ] || OVERALL=1
if [ "$OVERALL" -eq 0 ]; then

8
testing/static/scripts/bench-multirun.sh
View File

@@ -47,7 +47,7 @@ source "$ENV_FILE"
iperf_once() {
local client="$1" dest_npub="$2"
local out
if ! out=$(docker exec "fips-$client" iperf3 -c "${dest_npub}.fips" \
if ! out=$(docker exec "fips-${client}${FIPS_CI_NAME_SUFFIX:-}" iperf3 -c "${dest_npub}.fips" \
-t "$DURATION" -P "$PARALLEL" -J 2>&1); then
echo FAIL
return
@@ -105,7 +105,7 @@ PYEOF
ping_path() {
local client="$1" dest_npub="$2"
local out
if ! out=$(docker exec "fips-$client" \
if ! out=$(docker exec "fips-${client}${FIPS_CI_NAME_SUFFIX:-}" \
ping -c "$PING_COUNT" -i 0.2 -w "$((PING_COUNT * 2))" \
-q "${dest_npub}.fips" 2>&1); then
echo FAIL
@@ -203,7 +203,7 @@ CONVERGE_SECS="${FIPS_BENCH_CONVERGE_SECS:-15}"
peer_is_direct() {
local client="$1" dest_npub="$2"
docker exec "fips-$client" fipsctl show peers 2>/dev/null \
docker exec "fips-${client}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show peers 2>/dev/null \
| python3 -c '
import json, sys
target = sys.argv[1]
@@ -225,7 +225,7 @@ except Exception:
# multihop alternative).
peer_bytes_sent_snapshot() {
local client="$1"
docker exec "fips-$client" fipsctl show peers 2>/dev/null \
docker exec "fips-${client}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show peers 2>/dev/null \
| python3 -c '
import json, sys
try:

10
testing/static/scripts/gateway-test.sh
View File

@@ -20,11 +20,11 @@ source "$SCRIPT_DIR/../../lib/wait-converge.sh"
GENERATED_DIR="$SCRIPT_DIR/../generated-configs"
ENV_FILE="$GENERATED_DIR/npubs.env"
GATEWAY="fips-gw-gateway"
SERVER="fips-gw-server"
SERVER2="fips-gw-server-2"
CLIENT="fips-gw-client"
CLIENT2="fips-gw-client-2"
GATEWAY="fips-gw-gateway${FIPS_CI_NAME_SUFFIX:-}"
SERVER="fips-gw-server${FIPS_CI_NAME_SUFFIX:-}"
SERVER2="fips-gw-server-2${FIPS_CI_NAME_SUFFIX:-}"
CLIENT="fips-gw-client${FIPS_CI_NAME_SUFFIX:-}"
CLIENT2="fips-gw-client-2${FIPS_CI_NAME_SUFFIX:-}"
# ── inject-config subcommand ─────────────────────────────────────────────

4
testing/static/scripts/iperf-test.sh
View File

@@ -49,7 +49,7 @@ iperf_test() {
if [ "$LIVE_OUTPUT" = true ]; then
# Show live output
echo "Running iperf3 test (live output):"
if docker exec "fips-$client_node" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL"; then
if docker exec "fips-${client_node}${FIPS_CI_NAME_SUFFIX:-}" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL"; then
PASSED=$((PASSED + 1))
else
echo "FAIL"
@@ -59,7 +59,7 @@ iperf_test() {
# Capture and summarize output
echo -n "Running iperf3 test... "
local output
if output=$(docker exec "fips-$client_node" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL" 2>&1); then
if output=$(docker exec "fips-${client_node}${FIPS_CI_NAME_SUFFIX:-}" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL" 2>&1); then
# Check if we got valid results
if echo "$output" | grep -q "sender"; then
# Extract and display results (get SUM line for aggregate bandwidth)

6
testing/static/scripts/netem.sh
View File

@@ -180,7 +180,7 @@ do_apply() {
echo ""
for node in $NODES; do
local container="fips-node-$node"
local container="fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}"
echo -n " $container ... "
if ! container_running "$container"; then
echo "SKIP (not running)"
@@ -199,7 +199,7 @@ do_remove() {
echo ""
for node in $NODES; do
local container="fips-node-$node"
local container="fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}"
echo -n " $container ... "
if ! container_running "$container"; then
echo "SKIP (not running)"
@@ -219,7 +219,7 @@ do_status() {
echo ""
for node in $NODES; do
local container="fips-node-$node"
local container="fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}"
echo " $container:"
if ! container_running "$container"; then
echo " (not running)"

24
testing/static/scripts/ping-test.sh
View File

@@ -37,7 +37,7 @@ ping_test() {
echo -n " $label ... "
local output
if output=$(docker exec "fips-$from" ping6 -c "$COUNT" -W "$TIMEOUT" "${to_npub}.fips" 2>&1); then
if output=$(docker exec "fips-${from}${FIPS_CI_NAME_SUFFIX:-}" ping6 -c "$COUNT" -W "$TIMEOUT" "${to_npub}.fips" 2>&1); then
# Extract round-trip time from ping output
local rtt
rtt=$(echo "$output" | grep -oE 'time=[0-9.]+' | cut -d= -f2)
@@ -61,7 +61,7 @@ ping_all_quiet() {
for ((i=0; i<n; i++)); do
for ((j=0; j<n; j++)); do
[ "$i" -eq "$j" ] && continue
if docker exec "fips-node-${LABELS[$i],,}" \
if docker exec "fips-node-${LABELS[$i],,}${FIPS_CI_NAME_SUFFIX:-}" \
ping6 -c 1 -W 1 "${NPUBS[$j]}.fips" >/dev/null 2>&1; then
PASSED=$((PASSED + 1))
else
@@ -78,18 +78,18 @@ echo ""
echo "Waiting for mesh convergence..."
if [ "$PROFILE" = "chain" ]; then
# Chain: A-B-C-D-E, each interior node has 2 peers, endpoints have 1
wait_for_peers fips-node-a 1 20 || true
wait_for_peers fips-node-b 2 20 || true
wait_for_peers fips-node-c 2 20 || true
wait_for_peers fips-node-d 2 20 || true
wait_for_peers fips-node-e 1 20 || true
wait_for_peers fips-node-a${FIPS_CI_NAME_SUFFIX:-} 1 20 || true
wait_for_peers fips-node-b${FIPS_CI_NAME_SUFFIX:-} 2 20 || true
wait_for_peers fips-node-c${FIPS_CI_NAME_SUFFIX:-} 2 20 || true
wait_for_peers fips-node-d${FIPS_CI_NAME_SUFFIX:-} 2 20 || true
wait_for_peers fips-node-e${FIPS_CI_NAME_SUFFIX:-} 1 20 || true
elif [ "$PROFILE" = "mesh" ] || [ "$PROFILE" = "mesh-public" ]; then
# Mesh: check all nodes reach their configured peer counts
wait_for_peers fips-node-a 2 20 || true
wait_for_peers fips-node-b 1 20 || true
wait_for_peers fips-node-c 3 20 || true
wait_for_peers fips-node-d 3 20 || true
wait_for_peers fips-node-e 3 20 || true
wait_for_peers fips-node-a${FIPS_CI_NAME_SUFFIX:-} 2 20 || true
wait_for_peers fips-node-b${FIPS_CI_NAME_SUFFIX:-} 1 20 || true
wait_for_peers fips-node-c${FIPS_CI_NAME_SUFFIX:-} 3 20 || true
wait_for_peers fips-node-d${FIPS_CI_NAME_SUFFIX:-} 3 20 || true
wait_for_peers fips-node-e${FIPS_CI_NAME_SUFFIX:-} 3 20 || true
fi
# Wait for full pairwise connectivity, progress-aware: the actual pings
# are the convergence signal, the deadline extends while more pairs come

86
testing/static/scripts/rekey-test.sh
View File

@@ -158,9 +158,20 @@ REKEY_SETTLE=12 # > DRAIN_WINDOW_SECS (10) so post-rekey samples are off
# fully converged. Keep this bounded to preserve a meaningful scheduling check
# while still allowing for log visibility at the timeout edge.
FIRST_REKEY_TIMEOUT=$((REKEY_AFTER_SECS + 15))
SECOND_REKEY_WAIT=40 # wait for second cycle
SECOND_REKEY_WAIT=40 # upper bound for observing the second rekey cutover
LOG_EVENT_POLL_INTERVAL=1
# Post-rekey data-plane reconvergence is polled, not fixed-slept.
# wait_until_connected drives the same all-pairs ping sweep the strict
# per-phase assert depends on: it fails fast when reachability stops
# improving (no new reachable pair for RECONVERGE_STALL seconds while
# more than the near-converged slack of pairs is still down) and extends
# its deadline up to RECONVERGE_TIMEOUT only while pairs keep coming up,
# so a slow-but-converging post-rekey mesh under CI load passes instead
# of false-failing on a fixed settle window.
RECONVERGE_TIMEOUT=45
RECONVERGE_STALL=10
TIMEOUT=5
CONVERGENCE_PING_TIMEOUT=1
# Strict-ping retry policy for the per-phase ping_all asserts. Under 1%
@@ -212,7 +223,7 @@ ping_one() {
if (( attempt > 1 )); then
sleep "$PING_RETRY_DELAY"
fi
if output=$(docker exec "fips-$from" ping6 -c 1 -W "$ping_timeout" "${to_npub}.fips" 2>&1); then
if output=$(docker exec "fips-${from}${FIPS_CI_NAME_SUFFIX:-}" ping6 -c 1 -W "$ping_timeout" "${to_npub}.fips" 2>&1); then
rtt=$(echo "$output" | grep -oE 'time=[0-9.]+' | cut -d= -f2)
if [ -z "$quiet" ]; then
if (( attempt == 1 )); then
@@ -278,7 +289,7 @@ count_log_pattern() {
local pattern="$1"
local total=0
for node in $NODES; do
local count=$(docker logs "fips-node-$node" 2>&1 | grep -c "$pattern" || true)
local count=$(docker logs "fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 | grep -c "$pattern" || true)
total=$((total + count))
done
echo "$total"
@@ -341,7 +352,7 @@ dump_peer_connectivity() {
echo "=== Peer connectivity snapshot ==="
for node in $NODES; do
echo "--- node-$node ---"
docker exec "fips-node-$node" fipsctl show peers 2>/dev/null || true
docker exec "fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show peers 2>/dev/null || true
echo ""
done
}
@@ -392,20 +403,59 @@ assert_min_count "Rekey cutover complete (initiator), K-bit flipped" 1 \
phase_result "FMP rekey events"
echo ""
# Verify connectivity after first rekey (strict — no failures allowed)
echo "Phase 3: Post-rekey connectivity (settling ${REKEY_SETTLE}s)"
sleep "$REKEY_SETTLE"
# Verify connectivity after first rekey (strict — no failures allowed).
# Wait for the data plane to reconverge with a progress-aware deadline
# (the same all-pairs ping sweep the strict assert below depends on)
# instead of a fixed settle: fail fast on a genuinely stuck pair, extend
# only while reachability is still climbing. The strict ping_all is the
# actual assertion, run only after the reconverge poll.
echo "Phase 3: Post-rekey connectivity (reconverging up to ${RECONVERGE_TIMEOUT}s)"
wait_until_connected _baseline_ping "$RECONVERGE_TIMEOUT" "$RECONVERGE_STALL" || true
ping_all "" "$TIMEOUT" "$MAX_PING_ATTEMPTS"
phase_result "Post-first-rekey (all 20 pairs)"
echo ""
# ── Phase 4: Wait for second rekey cycle ──────────────────────────────
echo "Phase 4: Second rekey cycle (waiting ${SECOND_REKEY_WAIT}s)"
sleep "$SECOND_REKEY_WAIT"
# ── Phase 4: Wait for further rekey cutovers ──────────────────────────
# Poll for the FMP rekey cutovers instead of blind-sleeping: capture the
# cutover count reached so far, then wait until two more land AND the
# Phase 6 absolute threshold (>= 4) is covered. The relative "+2" alone
# is not enough: it inherits whatever count Phase 4 starts from, and on
# an unloaded host the ping phases can outrun the jittered first rekey
# cycle, entering here with a single cutover on the books — then "+2"
# releases at three, the remaining first-cycle cutovers land moments
# after the Phase 6 snapshot, and the >= 4 assertion fails even though
# every cutover completed on schedule. Flooring the wait target at the
# Phase 6 threshold makes this wait's guarantee cover that assertion:
# the log count is monotone over a container's lifetime, so a released
# wait means the later count cannot lose the timing race.
#
# What the count measures: the counted line is emitted by the cadence
# path on whichever side(s) flip first, so a completed link rekey yields
# one or two lines (a side promoted by receiving a flipped-K frame logs
# only a DEBUG on a target the test's log filter suppresses). With rekey
# timers resetting at each cutover, the events landing inside this
# test's window are first-cycle cutovers spread across the topology's
# links, not a second cycle on one link.
# Bounded by SECOND_REKEY_WAIT so a stalled rekey still falls through to
# the strict Phase 5/6 assertions rather than hanging.
echo "Phase 4: Further rekey cutovers (waiting up to ${SECOND_REKEY_WAIT}s)"
fmp_cutovers_before=$(count_log_pattern "Rekey cutover complete (initiator), K-bit flipped")
fmp_cutover_target=$((fmp_cutovers_before + 2))
if [ "$fmp_cutover_target" -lt 4 ]; then
fmp_cutover_target=4
fi
wait_for_log_pattern_count \
"Rekey cutover complete (initiator), K-bit flipped" \
"$fmp_cutover_target" "$SECOND_REKEY_WAIT" || true
# Verify connectivity after second rekey (back-to-back)
echo "Phase 5: Post-second-rekey connectivity (settling ${REKEY_SETTLE}s)"
sleep "$REKEY_SETTLE"
# Verify connectivity after second rekey (back-to-back). This is the
# site of the recurring post-second-rekey straggler-pair flake: wait for
# the data plane to reconverge with a progress-aware deadline (the same
# all-pairs ping sweep the strict assert below depends on) rather than a
# fixed settle window — fail fast if reachability stops improving, extend
# only while pairs are still coming up.
echo "Phase 5: Post-second-rekey connectivity (reconverging up to ${RECONVERGE_TIMEOUT}s)"
wait_until_connected _baseline_ping "$RECONVERGE_TIMEOUT" "$RECONVERGE_STALL" || true
ping_all "" "$TIMEOUT" "$MAX_PING_ATTEMPTS"
phase_result "Post-second-rekey (all 20 pairs)"
echo ""
@@ -428,7 +478,7 @@ wait_for_log_pattern_count "Peer FSP new-epoch frame authenticated" 1 "$REKEY_SE
# Positive checks: rekey machinery worked
assert_min_count "Rekey cutover complete (initiator), K-bit flipped" 4 \
"FMP rekey initiator cutovers (>= 2 cycles)"
"FMP rekey initiator cutovers"
# FSP rekey checks (sessions between non-adjacent nodes)
assert_min_count "FSP rekey cutover complete (initiator)" 1 \
@@ -454,7 +504,7 @@ assert_zero_count "Session AEAD decryption failed" \
if [ -n "$REKEY_ACCEPT_OFF_NODES" ]; then
DUAL_INIT_THRESHOLD=10
for off_node in ${REKEY_ACCEPT_OFF_NODES//,/ }; do
count=$(docker logs "fips-node-$off_node" 2>&1 \
count=$(docker logs "fips-node-${off_node}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 \
| grep -cE "Dual rekey initiation: we win" || true)
if [ "${count:-0}" -le "$DUAL_INIT_THRESHOLD" ]; then
echo " PASS: node-$off_node dual-init drops below threshold ($count <= $DUAL_INIT_THRESHOLD)"
@@ -477,7 +527,7 @@ fi
if [ -n "$REKEY_OUTBOUND_ONLY_NODES" ]; then
DUAL_INIT_THRESHOLD=10
for n in $NODES; do
count=$(docker logs "fips-node-$n" 2>&1 \
count=$(docker logs "fips-node-${n}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 \
| grep -cE "Dual rekey initiation: we win" || true)
if [ "${count:-0}" -le "$DUAL_INIT_THRESHOLD" ]; then
echo " PASS: node-$n dual-init drops below threshold ($count <= $DUAL_INIT_THRESHOLD)"
@@ -512,7 +562,7 @@ else
echo "=== Node logs (rekey-related, head -200) ==="
for node in $NODES; do
echo "--- node-$node ---"
docker logs "fips-node-$node" 2>&1 | \
docker logs "fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 | \
grep -E "(rekey|Rekey|cross|Cross|teardown|ERROR|PANIC|K-bit|no route|next hop|TTL exhausted|MTU exceeded|Congestion|decrypt|Decrypt|AEAD|Notify|drain|Drain|promot)" | \
head -200
echo ""
@@ -521,7 +571,7 @@ else
echo "=== Node logs (last 80 lines, unfiltered) ==="
for node in $NODES; do
echo "--- node-$node ---"
docker logs "fips-node-$node" 2>&1 | tail -80
docker logs "fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 | tail -80
echo ""
done
exit 1

6
testing/tor/directory-mode/docker-compose.yml
View File

@@ -11,6 +11,8 @@
networks:
dir-test:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
services:
fips-a:
@@ -22,7 +24,7 @@ services:
sysctls:
- net.ipv6.conf.all.disable_ipv6=0
restart: "no"
container_name: fips-dir-a
container_name: fips-dir-a${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-dir-a
volumes:
- ./configs/node-a.yaml:/etc/fips/fips.yaml:ro
@@ -42,7 +44,7 @@ services:
sysctls:
- net.ipv6.conf.all.disable_ipv6=0
restart: "no"
container_name: fips-dir-b
container_name: fips-dir-b${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-dir-b
volumes:
- ./configs/node-b.yaml:/etc/fips/fips.yaml:ro

18
testing/tor/directory-mode/scripts/directory-test.sh
View File

@@ -95,7 +95,7 @@ elapsed=0
while [ "$elapsed" -lt "$MAX_WAIT_ONION" ]; do
# Extract .onion address from structured log: onion_address=<addr>.onion
# Strip ANSI color codes before matching (tracing emits them by default)
ONION_ADDR=$(docker logs fips-dir-a 2>&1 \
ONION_ADDR=$(docker logs fips-dir-a${FIPS_CI_NAME_SUFFIX:-} 2>&1 \
| sed 's/\x1b\[[0-9;]*m//g' \
| grep -oE 'onion_address=[a-z2-7]{56}\.onion' \
| head -1 \
@@ -115,7 +115,7 @@ if [ -z "$ONION_ADDR" ]; then
echo " FAIL: Onion service not created within ${MAX_WAIT_ONION}s"
echo ""
echo "Node A logs (last 30 lines):"
docker logs fips-dir-a 2>&1 | tail -30
docker logs fips-dir-a${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
docker compose down
exit 1
fi
@@ -143,8 +143,8 @@ peers_a=0
peers_b=0
elapsed=0
while [ "$elapsed" -lt "$MAX_WAIT_PEER" ]; do
peers_a=$(count_connected_peers fips-dir-a)
peers_b=$(count_connected_peers fips-dir-b)
peers_a=$(count_connected_peers fips-dir-a${FIPS_CI_NAME_SUFFIX:-})
peers_b=$(count_connected_peers fips-dir-b${FIPS_CI_NAME_SUFFIX:-})
if [ "$peers_a" -ge 1 ] && [ "$peers_b" -ge 1 ]; then
echo " Both nodes connected after ${elapsed}s (A: ${peers_a}, B: ${peers_b})"
@@ -159,10 +159,10 @@ if [ "$peers_a" -lt 1 ] || [ "$peers_b" -lt 1 ]; then
echo " FAIL: Peers not established within ${MAX_WAIT_PEER}s"
echo ""
echo "Node A logs (last 30 lines):"
docker logs fips-dir-a 2>&1 | tail -30
docker logs fips-dir-a${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
echo ""
echo "Node B logs (last 30 lines):"
docker logs fips-dir-b 2>&1 | tail -30
docker logs fips-dir-b${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
docker compose down
exit 1
fi
@@ -229,15 +229,15 @@ print(f'{sum(trimmed)/len(trimmed):.1f}')
echo ""
echo " Ping via Tor onion service (directory mode, Sandbox 1):"
ping_series fips-dir-a "$NPUB_B" "A → B"
ping_series fips-dir-b "$NPUB_A" "B → A"
ping_series fips-dir-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B" "A → B"
ping_series fips-dir-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A" "B → A"
echo ""
# ── Phase 6: Log analysis ────────────────────────────────────────
echo "Phase 6: Log analysis"
for node in fips-dir-a fips-dir-b; do
for node in fips-dir-a${FIPS_CI_NAME_SUFFIX:-} fips-dir-b${FIPS_CI_NAME_SUFFIX:-}; do
panics=$(docker logs "$node" 2>&1 | grep -ci "panic" || true)
errors=$(docker logs "$node" 2>&1 | grep -ci "error" || true)
onion=$(docker logs "$node" 2>&1 | grep -ci "onion" || true)

8
testing/tor/socks5-outbound/docker-compose.yml
View File

@@ -10,11 +10,13 @@
networks:
tor-test:
driver: bridge
labels:
- "com.corganlabs.fips-ci=1"
services:
tor-daemon:
image: osminogin/tor-simple:latest
container_name: tor-daemon
container_name: tor-daemon${FIPS_CI_NAME_SUFFIX:-}
restart: "no"
volumes:
- ../common/torrc:/etc/tor/torrc:ro
@@ -30,7 +32,7 @@ services:
sysctls:
- net.ipv6.conf.all.disable_ipv6=0
restart: "no"
container_name: fips-tor-a
container_name: fips-tor-a${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-tor-a
depends_on:
- tor-daemon
@@ -51,7 +53,7 @@ services:
sysctls:
- net.ipv6.conf.all.disable_ipv6=0
restart: "no"
container_name: fips-tor-b
container_name: fips-tor-b${FIPS_CI_NAME_SUFFIX:-}
hostname: fips-tor-b
depends_on:
- tor-daemon

18
testing/tor/socks5-outbound/scripts/tor-test.sh
View File

@@ -87,7 +87,7 @@ echo ""
echo "Phase 2: Waiting for Tor daemon to bootstrap (up to ${MAX_WAIT_TOR}s)..."
elapsed=0
while [ "$elapsed" -lt "$MAX_WAIT_TOR" ]; do
if docker logs tor-daemon 2>&1 | grep -q "Bootstrapped 100%"; then
if docker logs tor-daemon${FIPS_CI_NAME_SUFFIX:-} 2>&1 | grep -q "Bootstrapped 100%"; then
echo " Tor bootstrapped after ${elapsed}s"
break
fi
@@ -100,7 +100,7 @@ if [ "$elapsed" -ge "$MAX_WAIT_TOR" ]; then
echo " FAIL: Tor daemon did not bootstrap within ${MAX_WAIT_TOR}s"
echo ""
echo "Tor daemon logs:"
docker logs tor-daemon 2>&1 | tail -20
docker logs tor-daemon${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -20
docker compose down
exit 1
fi
@@ -114,8 +114,8 @@ peers_a=0
peers_b=0
elapsed=0
while [ "$elapsed" -lt "$MAX_WAIT_PEER" ]; do
peers_a=$(count_connected_peers fips-tor-a)
peers_b=$(count_connected_peers fips-tor-b)
peers_a=$(count_connected_peers fips-tor-a${FIPS_CI_NAME_SUFFIX:-})
peers_b=$(count_connected_peers fips-tor-b${FIPS_CI_NAME_SUFFIX:-})
if [ "$peers_a" -ge 1 ] && [ "$peers_b" -ge 1 ]; then
echo " Both nodes have connected peers after ${elapsed}s (A: ${peers_a}, B: ${peers_b})"
@@ -130,10 +130,10 @@ if [ "$peers_a" -lt 1 ] || [ "$peers_b" -lt 1 ]; then
echo " FAIL: Peers not established within ${MAX_WAIT_PEER}s"
echo ""
echo "Node A logs (last 30 lines):"
docker logs fips-tor-a 2>&1 | tail -30
docker logs fips-tor-a${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
echo ""
echo "Node B logs (last 30 lines):"
docker logs fips-tor-b 2>&1 | tail -30
docker logs fips-tor-b${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
docker compose down
exit 1
fi
@@ -200,15 +200,15 @@ print(f'{sum(trimmed)/len(trimmed):.1f}')
echo ""
echo " Ping via Tor (routed through test-us01):"
ping_series fips-tor-a "$NPUB_B" "A → B"
ping_series fips-tor-b "$NPUB_A" "B → A"
ping_series fips-tor-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B" "A → B"
ping_series fips-tor-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A" "B → A"
echo ""
# ── Phase 5: Log analysis ────────────────────────────────────────
echo "Phase 5: Log analysis"
for node in fips-tor-a fips-tor-b; do
for node in fips-tor-a${FIPS_CI_NAME_SUFFIX:-} fips-tor-b${FIPS_CI_NAME_SUFFIX:-}; do
panics=$(docker logs "$node" 2>&1 | grep -ci "panic" || true)
errors=$(docker logs "$node" 2>&1 | grep -ci "error" || true)
socks5=$(docker logs "$node" 2>&1 | grep -ci "socks5\|socks" || true)