mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-22 15:52:20 +00:00
Compare commits
9 Commits
feat/openw
...
v0.4.1
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
15db6471db | ||
|
|
146d19a8d8 | ||
|
|
bda327b5f5 | ||
|
|
78377208af | ||
|
|
26a579b1c9 | ||
|
|
3ebb14eda4 | ||
|
|
e4a854f6b0 | ||
|
|
7a97599921 | ||
|
|
fbb4fb8879 |
54
CHANGELOG.md
54
CHANGELOG.md
@@ -13,6 +13,60 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
## [0.4.1] - 2026-07-19
|
||||
|
||||
### Changed
|
||||
|
||||
- `node.bloom.max_inbound_fpr` default raised from `0.10` to `0.20`. The
|
||||
cap rejects inbound `FilterAnnounce` whose FPR (`fill^k`) exceeds it. On
|
||||
the fixed 1 KB / k=5 filter, `0.10` corresponds to fill 0.631 (~1,630
|
||||
reachable entries), and the busiest nodes' aggregates had again begun to
|
||||
reach it as the mesh grew. `0.20` (fill 0.7248, ~2,114 entries) restores
|
||||
headroom without materially weakening the antipoison gate: a saturated or
|
||||
poisoned filter is ~100% FPR and still rejected. This is the second raise
|
||||
of this cap in two releases; the fixed 1 KB filter is the underlying
|
||||
constraint, and the structural remedy is the v2 filter work rather than a
|
||||
further raise. A node running this default accepts announcements that a
|
||||
v0.4.0 node drops, so during a rolling upgrade the two versions can
|
||||
disagree about mesh size.
|
||||
- Bloom filter probing computes its SHA-256 digest once per operation
|
||||
rather than once per hash function. All k indices were already derived
|
||||
from a single digest, but the digest was recomputed inside the
|
||||
per-function loop, so every insert and membership test hashed the same
|
||||
bytes `hash_count` times (5x at the default). Output is bit-for-bit
|
||||
identical; this is the hottest path in packet forwarding and mesh-size
|
||||
estimation.
|
||||
- Identity operations reuse one shared `secp256k1` context instead of
|
||||
constructing a fresh one at every sign, verify, and key-derive site.
|
||||
Each construction allocated a context and ran randomization and blinding
|
||||
table setup. Behavior is unchanged: the same API calls are made, only the
|
||||
context lifetime differs, and the shared context still performs the
|
||||
standard construction-time blinding.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Spanning tree: the coordinate cache is now invalidated when the parent
|
||||
link is lost through peer removal. That path reparents or self-roots the
|
||||
node but omitted the invalidation every other position-change path
|
||||
performs, so cached entries for downstream destinations kept the node's
|
||||
now-stale coordinate prefix. Because routing access refreshes an entry's
|
||||
TTL, an actively routed stale entry never self-expired and was corrected
|
||||
only by a fresh insert.
|
||||
- Discovery: applying a `LookupResponse` now keeps the tighter of the
|
||||
cached and received `path_mtu` rather than overwriting unconditionally.
|
||||
A looser estimate arriving in a later response could clobber a tighter
|
||||
value already learned from a reactive `MtuExceeded` or
|
||||
`PathMtuNotification`, loosening a clamp that had been correctly
|
||||
tightened.
|
||||
|
||||
### Removed
|
||||
|
||||
- The `parent_switched` spanning-tree metric counter. It was incremented on
|
||||
the line immediately before `parent_switches` at every site and never
|
||||
independently, so the two were always identical. `parent_switches`
|
||||
remains as the sole counter. Consumers reading `parent_switched` from the
|
||||
control socket or `fipstop` should use `parent_switches`.
|
||||
|
||||
## [0.4.0] - 2026-06-27
|
||||
|
||||
### Added
|
||||
|
||||
2
Cargo.lock
generated
2
Cargo.lock
generated
@@ -1074,7 +1074,7 @@ checksum = "9844ddc3a6e533d62bba727eb6c28b5d360921d5175e9ff0f1e621a5c590a4d5"
|
||||
|
||||
[[package]]
|
||||
name = "fips"
|
||||
version = "0.4.1-dev"
|
||||
version = "0.4.1"
|
||||
dependencies = [
|
||||
"arc-swap",
|
||||
"bech32",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "fips"
|
||||
version = "0.4.1-dev"
|
||||
version = "0.4.1"
|
||||
edition = "2024"
|
||||
description = "A distributed, decentralized network routing protocol for mesh nodes connecting over arbitrary transports"
|
||||
license = "MIT"
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||

|
||||
[](LICENSE)
|
||||
[](https://www.rust-lang.org/)
|
||||
[](#status--roadmap)
|
||||
[](#status--roadmap)
|
||||
|
||||
A self-organizing encrypted mesh network built on Nostr identities,
|
||||
capable of operating over arbitrary transports without central
|
||||
@@ -210,8 +210,8 @@ testing/ Docker-based integration test harnesses + chaos simulation
|
||||
|
||||
## Status & roadmap
|
||||
|
||||
FIPS is at **v0.4.1-dev** on the `maint` branch.
|
||||
[v0.4.0](https://github.com/jmcorgan/fips/releases/tag/v0.4.0) has
|
||||
FIPS is at **v0.4.1** on the `maint` branch.
|
||||
[v0.4.1](https://github.com/jmcorgan/fips/releases/tag/v0.4.1) has
|
||||
shipped; this line carries patch-level fixes for the 0.4.x series. The
|
||||
core protocol works end-to-end over
|
||||
UDP, TCP, Ethernet, Tor, Nym, and Bluetooth on a global, public test
|
||||
|
||||
393
RELEASE-NOTES.md
393
RELEASE-NOTES.md
@@ -1,302 +1,134 @@
|
||||
# FIPS v0.4.0
|
||||
# FIPS v0.4.1
|
||||
|
||||
**Released**: 2026-06-21 (provisional)
|
||||
**Released**: 2026-07-19
|
||||
|
||||
v0.4.0 is the throughput-and-observability release on the v0.3.x wire
|
||||
format. It adds two new ways for nodes to find and reach each other (the
|
||||
Nym mixnet transport and opt-in mDNS LAN discovery), overhauls the data
|
||||
plane for higher single-node throughput and lower per-packet CPU, moves
|
||||
the entire operator read surface off the data-plane hot path so
|
||||
observability stays responsive under load, ships a reworked `fipstop`
|
||||
TUI, and hardens FMP and FSP rekey to be hitless under packet loss in
|
||||
both directions. It also folds in the accumulated mesh-convergence,
|
||||
admission-control, and packaging fixes from the maintenance line.
|
||||
v0.4.1 is a maintenance release on the v0.4.x line. It raises the default
|
||||
antipoison cap on inbound bloom filter announcements, removes a redundant
|
||||
spanning-tree metric counter, fixes two convergence and path-MTU bugs, and
|
||||
cuts per-packet CPU in the bloom and identity paths. There is no wire
|
||||
format change and no new feature surface.
|
||||
|
||||
v0.4.0 is wire-compatible with v0.3.0. Mixed meshes interoperate; there
|
||||
is no flag-day upgrade. A deployed v0.3.0 node and an upgraded v0.4.0
|
||||
node peer, rekey, and route normally, so you can roll the upgrade out
|
||||
across a mesh in any order.
|
||||
v0.4.1 is wire-compatible with v0.4.0. Nodes can be upgraded one at a time
|
||||
with no coordinated restart, though one behavior change below is worth
|
||||
reading before you start a rolling upgrade.
|
||||
|
||||
## At a glance
|
||||
|
||||
- New outbound Nym mixnet transport with a single-container demo and a
|
||||
new mixnet-relay example.
|
||||
- Opt-in mDNS / DNS-SD discovery on the local link.
|
||||
- Data-plane overhaul: off-task encrypt and decrypt worker pools, GSO,
|
||||
connected-UDP send path, copy-avoidance on receive, batched macOS
|
||||
receive.
|
||||
- The full `show_*` read surface now serves off the receive loop, so
|
||||
`fipsctl` and `fipstop` stay responsive on loaded nodes; a new
|
||||
counter-only `show_metrics` query enables a Prometheus scraper at no
|
||||
hot-path cost.
|
||||
- Reworked `fipstop` TUI on a machine-verified render-snapshot base.
|
||||
- Rekey is now hitless under loss and reordering in both directions.
|
||||
- New packaging targets: an OpenWrt `.apk` for OpenWrt 25+ and a Nix
|
||||
flake for reproducible from-source builds on Nix/NixOS.
|
||||
- Six route-class transit counters partition forwarded traffic by its
|
||||
tree relationship to the next hop, visible via `show_routing` and
|
||||
`show_status`.
|
||||
|
||||
## What's new
|
||||
|
||||
### Nym mixnet transport
|
||||
|
||||
FIPS can now peer over the [Nym](https://nymtech.net/) mixnet for
|
||||
metadata-resistant connectivity. The new `transports.nym` transport
|
||||
makes outbound connections through a `nym-socks5-client` SOCKS5 proxy
|
||||
that you run alongside the daemon (for example as a service running
|
||||
alongside the fips daemon, or as a sidecar container). The transport
|
||||
waits at startup for the nym-socks5-client to become ready before giving
|
||||
up.
|
||||
|
||||
This is a privacy and anonymity deployment mode chosen for its own
|
||||
properties. It mixes your FIPS traffic into the Nym cover-traffic
|
||||
network so that link-level observers cannot correlate which mesh peers
|
||||
are talking. A new `examples/sidecar-nostr-mixnet-relay/` demonstrates a
|
||||
FIPS-reachable Nostr relay peered across the mixnet end to end, and a
|
||||
single-container demo ships with the transport.
|
||||
|
||||
Enable it by adding a `transports.nym` instance and pointing it at your
|
||||
running nym-socks5-client. See the transports reference for the field
|
||||
set.
|
||||
|
||||
### mDNS LAN discovery
|
||||
|
||||
Nodes on a shared local link can now find each other with zero address
|
||||
configuration. The opt-in `node.discovery.lan` path runs an mDNS /
|
||||
DNS-SD responder and browser: each node advertises a FIPS service record
|
||||
on the link and adopts the peers it discovers. This complements the
|
||||
existing Nostr-mediated overlay discovery for the common case where the
|
||||
peers are simply on the same LAN.
|
||||
|
||||
Turn it on with `node.discovery.lan.enabled: true`. `service_type` and
|
||||
`scope` tune the advertised service record and which interfaces
|
||||
participate. Discovery on the local link needs no relay and no STUN.
|
||||
|
||||
### Data-plane throughput overhaul
|
||||
|
||||
The receive and send paths were reworked for higher single-node
|
||||
throughput and lower per-packet CPU, building on the v0.3.0
|
||||
crypto-backend swap:
|
||||
|
||||
- **Off-task encrypt and decrypt.** Per-peer encrypt and decrypt now run
|
||||
on dedicated worker tasks rather than inline on the receive loop, so a
|
||||
single busy peer no longer serializes the whole node's crypto.
|
||||
- **GSO and connected-UDP send.** The Linux send path uses generic
|
||||
segmentation offload and a connected-UDP socket where available,
|
||||
cutting syscall overhead on bulk flows.
|
||||
- **Copy-avoidance on receive.** The receive hot path avoids buffer
|
||||
copies it previously made per packet.
|
||||
- **Batched macOS receive.** macOS gains a `recvmsg_x` batched receive,
|
||||
mirroring the Linux `recvmmsg` batching from v0.3.0.
|
||||
- **Shared immutable-state context and an atomic metric registry.**
|
||||
Immutable per-node state moved into a single shared context, and
|
||||
counters live in an atomic metric registry that the new `show_metrics`
|
||||
query reads without touching the hot path.
|
||||
|
||||
These are all internal to the data plane and require no operator action.
|
||||
|
||||
### Observability off the hot path
|
||||
|
||||
Every read-only control query now renders from a snapshot published once
|
||||
per tick into a lock-free `ArcSwap`, served from the control accept task
|
||||
instead of round-tripping the data-plane receive loop. This covers
|
||||
`show_status`, `show_stats_*`, `show_peers`, `show_sessions`,
|
||||
`show_links`, `show_connections`, `show_transports`, `show_mmp`,
|
||||
`show_tree`, `show_bloom`, `show_cache`, `show_routing`,
|
||||
`show_identity_cache`, `show_acl`, `show_listening_sockets`, and the new
|
||||
`show_metrics`. Only the mutating `connect` and `disconnect` commands
|
||||
still reach the loop.
|
||||
|
||||
The practical effect: on a loaded node where the receive loop was busy,
|
||||
`fipsctl` and `fipstop` queries previously stalled or timed out (the
|
||||
five-second query pattern operators saw). They now answer promptly
|
||||
regardless of data-plane load. Per-entity snapshots reuse unchanged rows
|
||||
by pointer, so the per-tick publish cost stays bounded as peer and
|
||||
session counts grow.
|
||||
|
||||
A new **`show_metrics`** query (surfaced as `fipsctl stats metrics`)
|
||||
returns a counter-only snapshot of every metric family. It is the
|
||||
enabler for a Prometheus scraper that pulls node counters at no hot-path
|
||||
cost.
|
||||
|
||||
Six **route-class transit counters** partition transit-forwarded packets
|
||||
by their tree relationship to the chosen next hop — tree-up, tree-down,
|
||||
tree-down-cross, cross-link descend, cross-link ascend, and direct-peer
|
||||
— and the six classes sum to `forwarded_packets`. They surface through
|
||||
`show_routing` and `show_status`, and the `fipstop` routing tab is
|
||||
reorganized so its two columns separate own/endpoint traffic from
|
||||
forwarded/transit traffic with the tree-down-cross line visually flagged.
|
||||
|
||||
### Reworked fipstop TUI
|
||||
|
||||
`fipstop` gets a rendering, navigation, and read-surface overhaul on a
|
||||
machine-verified base: a render-snapshot harness asserts the exact text
|
||||
grid and per-cell style of every view against canned control-socket
|
||||
output. New daemon-resolved fields surface through the snapshots,
|
||||
including effective persistence, root and is-root state, a
|
||||
per-transport-type peer-count map, per-peer effective depth, the root
|
||||
npub, and the last-sent uptree filter fill ratio with the subtree size
|
||||
estimate.
|
||||
|
||||
A separate fix clears a garbled-screen problem on startup and stray
|
||||
bytes on quit, most visible over SSH and inside tmux: startup now forces
|
||||
a full repaint before the first draw, and quit stops and joins the
|
||||
stdin-poll thread before restoring the terminal, so post-raw-mode
|
||||
keystrokes no longer echo onto the restored screen.
|
||||
|
||||
### Rekey reliability
|
||||
|
||||
FMP and FSP session rekey are now hitless under packet loss and
|
||||
reordering in both directions:
|
||||
|
||||
- Inbound frames are authenticated against the pending session before
|
||||
the K-bit cutover promotes it, so a spoofed or stale frame cannot
|
||||
derail a rekey in progress.
|
||||
- Rekey message-1 retransmission is bounded, and the link-dead heartbeat
|
||||
is rekey-aware so an in-flight rekey is not mistaken for a dead link.
|
||||
- FSP session rekey holds connectivity across the rekey window under
|
||||
loss and reordering.
|
||||
- Dual-initiation races (both peers starting a rekey at once on a
|
||||
high-latency link) are desynchronized with symmetric jitter so the two
|
||||
sides converge on one session rather than fighting.
|
||||
- An exhausted retransmission-budget abort, an expected and self-limiting
|
||||
outcome on lossy or high-latency links, is logged at debug rather than
|
||||
warn.
|
||||
|
||||
The net operator takeaway: rekey completes cleanly without dropping
|
||||
traffic, even on lossy or high-latency links, and the log no longer
|
||||
cries wolf when a rekey gives up and retries.
|
||||
|
||||
### New packaging targets
|
||||
|
||||
- **OpenWrt `.apk`.** A new `.apk` package targets OpenWrt 25+, where
|
||||
apk-tools is the mandatory package manager; the existing `.ipk`
|
||||
continues to cover OpenWrt 24.x and earlier. It is built SDK-free,
|
||||
reusing the `.ipk` cross-compile and installed-filesystem payload, and
|
||||
releases publish `.apk` artifacts and checksums alongside `.ipk`. Like
|
||||
the `.ipk`, the package is unsigned and installed with
|
||||
`apk add --allow-untrusted`.
|
||||
- **Nix flake.** A `flake.nix` at the project root builds all four
|
||||
binaries (`fips`, `fipsctl`, `fips-gateway`, `fipstop`) from source on
|
||||
Nix/NixOS, pinning the exact toolchain and wiring the native build
|
||||
dependencies so no host setup is needed beyond Nix with flakes
|
||||
enabled. It exposes `nix build`, `nix run`, a `nix develop` dev shell,
|
||||
and `nix flake check`, with `flake.lock` committed for reproducibility.
|
||||
- `node.bloom.max_inbound_fpr` default moves from `0.10` to `0.20`.
|
||||
- The `parent_switched` metric counter is gone. Use `parent_switches`.
|
||||
- Spanning tree no longer serves stale coordinates after a parent link is
|
||||
lost through peer removal.
|
||||
- Discovery no longer loosens a path MTU clamp it had correctly tightened.
|
||||
- Bloom probing and identity operations do measurably less work per call,
|
||||
with identical results.
|
||||
|
||||
## Behavior changes worth flagging
|
||||
|
||||
These affect operators on upgrade.
|
||||
### The inbound filter FPR cap default doubles again
|
||||
|
||||
- **Bloom filter antipoison cap raised.** `node.bloom.max_inbound_fpr`
|
||||
moves from 0.05 to 0.10, accepting filters with a higher derived
|
||||
false-positive rate before rejecting them. This reduces spurious
|
||||
filter rejections on larger meshes while keeping the antipoison
|
||||
protection in place.
|
||||
- **TCP inbound cap honors `max_connections`.** The TCP inbound accept
|
||||
ceiling now resolves from explicit per-transport
|
||||
`max_inbound_connections`, then node-wide
|
||||
`node.limits.max_connections`, then the built-in default of 256.
|
||||
Previously the TCP inbound ceiling was hardwired to 256 and ignored
|
||||
`max_connections`, so raising it had no effect on inbound TCP.
|
||||
- **Static host aliases hot-reload.** `/etc/fips/hosts` now reloads on
|
||||
mtime change once per tick rather than only at startup, so display
|
||||
names in `fipsctl` and `fipstop` reflect edits without a daemon
|
||||
restart. The peer ACL reloads through the same lock-free snapshot
|
||||
mechanism.
|
||||
- **Quieter logs on busy public-mesh nodes.** Routine per-peer
|
||||
connection-lifecycle and capacity-cap events, no-route session-datagram
|
||||
drops, and exhausted rekey-budget aborts are demoted to debug, so
|
||||
genuinely notable info and warn lines are no longer drowned out.
|
||||
- **More visible drops.** Receive-path silent rejections now flow
|
||||
through typed reject-reason counters, and discovery counts requests
|
||||
dropped when the dedup cache is full (`req_dedup_cache_full`, visible
|
||||
via `show_routing`). Drops that were previously silent are now
|
||||
countable.
|
||||
- **Tor connect-refused accounting.** The Tor transport increments its
|
||||
`connect_refused` statistic (the "Refused" line in `fipstop`) on an
|
||||
actively-refused SOCKS5 connect, instead of recording every connect
|
||||
failure as a generic SOCKS5 error.
|
||||
`node.bloom.max_inbound_fpr` goes from `0.10` to `0.20`. The cap rejects
|
||||
inbound `FilterAnnounce` frames whose advertised false positive rate
|
||||
exceeds it. On the fixed 1 KB, k=5 filter, `0.10` corresponds to a fill of
|
||||
0.631 and roughly 1,630 reachable entries, and the busiest nodes'
|
||||
aggregates had started reaching that ceiling as the mesh grew. `0.20`
|
||||
corresponds to a fill of 0.7248 and roughly 2,114 entries.
|
||||
|
||||
Be aware that this is the second time in two releases that this default
|
||||
has doubled, for the same reason both times. That is worth stating plainly
|
||||
rather than repeating the previous release's framing: raising the cap buys
|
||||
headroom, it does not fix anything. The real constraint is the fixed 1 KB
|
||||
filter size, which is a protocol constant. The structural remedy is the v2
|
||||
filter work, where filter capacity scales with the mesh instead of being
|
||||
pinned. This release is an interim step to keep legitimate aggregates from
|
||||
being rejected until that lands. It is not the start of a pattern of
|
||||
raising the cap once per release, and if you are sizing capacity planning
|
||||
around this number, plan against the v2 work rather than against a third
|
||||
raise.
|
||||
|
||||
The antipoison property the cap exists for is preserved. A saturated or
|
||||
deliberately poisoned filter still presents an FPR near 100% and is still
|
||||
rejected.
|
||||
|
||||
**This matters during a rolling upgrade.** A v0.4.1 node accepts a
|
||||
`FilterAnnounce` with a derived FPR between 0.10 and 0.20; a v0.4.0 node
|
||||
drops the same frame, and the drop is silent on the wire with no NACK. The
|
||||
cap also gates the mesh size estimator, which declines to produce a value
|
||||
when any contributing filter is over the cap. So while a mesh is partly
|
||||
upgraded, upgraded and not-yet-upgraded nodes can legitimately report
|
||||
different mesh sizes, or one can report a size while the other reports
|
||||
unknown. This resolves once every node is on v0.4.1. If you want to avoid
|
||||
the window entirely, set `node.bloom.max_inbound_fpr: 0.10` explicitly in
|
||||
your config before upgrading and remove it after the last node is done.
|
||||
|
||||
### The `parent_switched` counter is removed
|
||||
|
||||
`parent_switched` was incremented on the line immediately before
|
||||
`parent_switches` at every site and never independently, so the two
|
||||
counters always held the same value. `parent_switched` is now gone from
|
||||
the tree metrics, the control socket snapshot, and the `fipstop` tree
|
||||
view. `parent_switches` remains and is unchanged.
|
||||
|
||||
If you scrape the control socket, or have dashboards or alerts referencing
|
||||
`parent_switched`, point them at `parent_switches`. Anything still asking
|
||||
for `parent_switched` will find nothing rather than a zero.
|
||||
|
||||
## Notable bug fixes
|
||||
|
||||
The CHANGELOG has the exhaustive list. This is the operator-relevant
|
||||
subset of fixes for behavior that shipped in v0.3.0.
|
||||
### Stale coordinates after losing a parent through peer removal
|
||||
|
||||
- **Symmetric peer teardown on manual disconnect.** A manual
|
||||
`fipsctl disconnect` now sends the peer a scoped Disconnect so both
|
||||
ends tear down and re-handshake cleanly. Previously a manual
|
||||
disconnect tore down only the local side, leaving the peer with a
|
||||
stale session that was never re-adopted as a child and whose bloom
|
||||
filter was never re-recorded.
|
||||
- **Gateway holds long-lived and DNS-cached mappings.** `fips-gateway`
|
||||
no longer drops a virtual-IP mapping while traffic is still flowing.
|
||||
The mapping TTL clock previously advanced only on DNS re-query, so a
|
||||
busy long-lived or DNS-cached client could have its mapping reclaimed
|
||||
mid-flow. The tick now refreshes the mapping whenever conntrack reports
|
||||
active sessions and recovers a draining mapping to active when traffic
|
||||
resumes; only genuinely idle mappings drain.
|
||||
- **Accurate mesh-size estimate under filter overlap.** The mesh-size
|
||||
estimator now estimates the cardinality of the OR-union of self plus
|
||||
every connected peer's inbound filter, instead of summing per-filter
|
||||
cardinalities of tree peers. Summing assumed the filters were disjoint,
|
||||
so a stale or oversized parent filter or a routing loop inflated the
|
||||
reported mesh size and a tree rebalance flapped the count. OR-union
|
||||
deduplicates overlap, equals the old result in the disjoint case, and
|
||||
removes the estimate's dependence on tree-declaration cache freshness.
|
||||
- **Single-uplink node reattaches within a round-trip.** A node with one
|
||||
tree peer, which has periodic parent re-evaluation disabled, was left
|
||||
self-rooted and unreachable if its one-shot attaching TreeAnnounce was
|
||||
lost, until the next periodic re-broadcast. Tree-position exchange is
|
||||
now self-healing on the receive path: a node that hears an announce
|
||||
advertising a strictly worse root echoes its own declaration back,
|
||||
provoking the better-rooted peer to re-push its real position
|
||||
immediately.
|
||||
- **macOS self-connections work end to end (#117).** Traffic a macOS
|
||||
node sends to its own `<npub>.fips` address is now delivered locally
|
||||
for full TCP/UDP, not just `ping6`. The point-to-point `utun` egresses
|
||||
self-addressed packets into the daemon with an unfinished transport
|
||||
checksum (macOS offloads it on the `lo0` loopback route), so
|
||||
re-injecting them verbatim made the local stack drop every segment the
|
||||
MSS-clamp rewrite did not happen to fix and self-connections
|
||||
half-opened and hung. The hairpin path now recomputes the TCP/UDP
|
||||
checksum before re-injection. Linux was unaffected.
|
||||
When a node's parent link dropped via peer removal, the node correctly
|
||||
reparented or self-rooted, but skipped the coordinate cache invalidation
|
||||
that every other position-change path performs. Cached entries for
|
||||
downstream destinations kept the node's old coordinate prefix. This did
|
||||
not self-correct the way a stale cache entry normally would: routing
|
||||
access refreshes an entry's TTL, so an entry that was actively being
|
||||
routed through never expired, and was only fixed by an unrelated fresh
|
||||
insert. Both invalidation classes now run on this path, matching the
|
||||
loop-detection branch.
|
||||
|
||||
### Discovery could loosen a tightened path MTU clamp
|
||||
|
||||
An originator handling a `LookupResponse` overwrote its cached path MTU
|
||||
unconditionally. If a reactive `MtuExceeded` or `PathMtuNotification` had
|
||||
already taught it a tighter value, a later, looser discovery estimate
|
||||
would clobber that and re-loosen the clamp, risking a return to dropped
|
||||
oversized packets. The cached and received values are now compared and the
|
||||
tighter one is kept.
|
||||
|
||||
## Upgrade notes
|
||||
|
||||
Operator-actionable items moving from v0.3.0 to v0.4.0:
|
||||
This is a drop-in upgrade from v0.4.0 with no wire format change, no
|
||||
config migration, and no coordinated restart. Upgrade nodes in whatever
|
||||
order you like.
|
||||
|
||||
- **Wire-compatible, no flag day.** v0.4.0 peers with v0.3.0. Upgrade
|
||||
nodes in any order. During a rolling upgrade you may see some log lines
|
||||
on the upgraded side as it interacts with not-yet-upgraded peers;
|
||||
behavior is correct, log noise only.
|
||||
- **Bloom antipoison cap default changed.** `node.bloom.max_inbound_fpr`
|
||||
now defaults to 0.10 (was 0.05). If you set this explicitly, review
|
||||
whether you still want the old value.
|
||||
- **New optional config surfaces.** `transports.nym` (outbound Nym
|
||||
mixnet) and `node.discovery.lan` (mDNS LAN discovery) are both opt-in
|
||||
and off by default. Adding them is the only way to turn the new paths
|
||||
on.
|
||||
- **TCP inbound cap.** If you relied on the old hardwired 256 inbound-TCP
|
||||
ceiling, note it now honors `max_inbound_connections` then
|
||||
`node.limits.max_connections` then 256.
|
||||
- **New observability query.** `fipsctl stats metrics` (the
|
||||
`show_metrics` control query) returns a counter-only snapshot suitable
|
||||
for a scraper.
|
||||
Two things to do rather than assume:
|
||||
|
||||
## Getting v0.4.0
|
||||
1. If you monitor `parent_switched`, move to `parent_switches` before
|
||||
upgrading, or your dashboards will go blank rather than error.
|
||||
2. During the rolling window, expect upgraded and not-yet-upgraded nodes
|
||||
to potentially disagree about mesh size, per the FPR cap section above.
|
||||
This is expected and self-resolves. Do not chase it as a bug unless it
|
||||
persists after every node reports `0.4.1`.
|
||||
|
||||
If you have pinned `node.bloom.max_inbound_fpr` explicitly in your config,
|
||||
your setting is honored and nothing changes for you. The change only
|
||||
affects nodes taking the default.
|
||||
|
||||
Downgrading to v0.4.0 is supported and needs no special handling.
|
||||
|
||||
## Getting v0.4.1
|
||||
|
||||
- **Linux x86_64 / aarch64**: `.deb` and tarball at the
|
||||
[v0.4.0 release page](https://github.com/jmcorgan/fips/releases/tag/v0.4.0).
|
||||
[v0.4.1 release page](https://github.com/jmcorgan/fips/releases/tag/v0.4.1).
|
||||
- **Arch Linux**: `fips` from the AUR.
|
||||
- **macOS**: `.pkg` at the v0.4.0 release page.
|
||||
- **Windows**: ZIP at the v0.4.0 release page.
|
||||
- **macOS**: `.pkg` at the v0.4.1 release page.
|
||||
- **Windows**: ZIP at the v0.4.1 release page.
|
||||
- **OpenWrt**: `.ipk` (OpenWrt 24.x and earlier) or `.apk` (OpenWrt 25+)
|
||||
at the v0.4.0 release page.
|
||||
- **From source**: `cargo build --release` from a checkout of the v0.4.0
|
||||
at the v0.4.1 release page.
|
||||
- **From source**: `cargo build --release` from a checkout of the v0.4.1
|
||||
tag (Rust 1.94.1 per `rust-toolchain.toml`; `libclang-dev` is a
|
||||
required Linux build prerequisite).
|
||||
- **Nix / NixOS**: `nix build .#fips` from a checkout of the v0.4.0 tag
|
||||
- **Nix / NixOS**: `nix build .#fips` from a checkout of the v0.4.1 tag
|
||||
builds the binaries from source with the pinned toolchain and no manual
|
||||
prerequisites (see the Nix section of `packaging/README.md`).
|
||||
|
||||
@@ -309,13 +141,6 @@ The full per-commit changelog lives in
|
||||
Thanks to everyone who contributed code, packaging work, bug reports, or
|
||||
reviews to this release.
|
||||
|
||||
- [@jcorgan](https://github.com/jmcorgan): release shepherd, high-level
|
||||
design, control read plane, rekey hardening, admission, bug fixes,
|
||||
testing, packaging, PR coordination, and issue resolution.
|
||||
- [@mmalmi](https://github.com/mmalmi): opt-in mDNS LAN discovery and
|
||||
data-plane performance work.
|
||||
- [@Origami74](https://github.com/Origami74): macOS packaging and
|
||||
website coordination.
|
||||
- [@dskvr](https://github.com/dskvr): AUR packaging.
|
||||
- [@oleksky](https://github.com/oleksky): Nym mixnet transport and the
|
||||
single-container mixnet demo.
|
||||
- [@jcorgan](https://github.com/jmcorgan): release shepherd, spanning-tree
|
||||
and discovery fixes, bloom and identity performance work, antipoison cap
|
||||
change, and testing.
|
||||
|
||||
@@ -360,14 +360,14 @@ control socket and `fipstop` dashboard. (See `compute_mesh_size()` in
|
||||
|
||||
The estimator refuses to produce a value when any contributing filter
|
||||
is above the antipoison FPR cap (`node.bloom.max_inbound_fpr`,
|
||||
default `0.10`); a partial aggregate would silently underestimate.
|
||||
default `0.20`); a partial aggregate would silently underestimate.
|
||||
Consumers handle the resulting `None` by displaying an "unknown"
|
||||
state rather than a misleading number.
|
||||
|
||||
## Antipoison: Inbound FPR Cap
|
||||
|
||||
Inbound `FilterAnnounce` payloads are checked against
|
||||
`node.bloom.max_inbound_fpr` (default `0.10`). Filters whose
|
||||
`node.bloom.max_inbound_fpr` (default `0.20`). Filters whose
|
||||
estimated false positive rate exceeds the cap are dropped silently
|
||||
(no NACK on the wire) — they would otherwise inflate downstream
|
||||
candidate evaluation cost without contributing useful discrimination.
|
||||
|
||||
@@ -277,7 +277,7 @@ Controls tree construction and parent selection.
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `node.bloom.update_debounce_ms` | u64 | `500` | Debounce interval for filter update propagation |
|
||||
| `node.bloom.max_inbound_fpr` | f64 | `0.10` | Antipoison cap: reject inbound `FilterAnnounce` frames whose advertised false-positive rate exceeds this value. Valid range `(0.0, 1.0)`. The default `0.10` corresponds to fill 0.631 at k=5 (≈1,630 entries on the 1 KB filter); a saturated/poisoned filter is still ~100% FPR and rejected |
|
||||
| `node.bloom.max_inbound_fpr` | f64 | `0.20` | Antipoison cap: reject inbound `FilterAnnounce` frames whose advertised false-positive rate exceeds this value. Valid range `(0.0, 1.0)`. The default `0.20` corresponds to fill 0.7248 at k=5 (≈2,114 entries on the 1 KB filter); a saturated/poisoned filter is still ~100% FPR and rejected |
|
||||
|
||||
Bloom filter size (1 KB), hash count (5), and size classes are protocol
|
||||
constants and not configurable.
|
||||
@@ -944,7 +944,7 @@ node:
|
||||
flap_dampening_secs: 120 # extended hold-down on flap
|
||||
bloom:
|
||||
update_debounce_ms: 500
|
||||
max_inbound_fpr: 0.10 # antipoison cap on inbound FilterAnnounce FPR
|
||||
max_inbound_fpr: 0.20 # antipoison cap on inbound FilterAnnounce FPR
|
||||
session:
|
||||
default_ttl: 64
|
||||
pending_packets_per_dest: 16
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# FIPS v0.4.0
|
||||
|
||||
**Released**: 2026-06-21 (provisional)
|
||||
**Released**: 2026-06-27
|
||||
|
||||
v0.4.0 is the throughput-and-observability release on the v0.3.x wire
|
||||
format. It adds two new ways for nodes to find and reach each other (the
|
||||
|
||||
146
docs/releases/release-notes-v0.4.1.md
Normal file
146
docs/releases/release-notes-v0.4.1.md
Normal file
@@ -0,0 +1,146 @@
|
||||
# FIPS v0.4.1
|
||||
|
||||
**Released**: 2026-07-19
|
||||
|
||||
v0.4.1 is a maintenance release on the v0.4.x line. It raises the default
|
||||
antipoison cap on inbound bloom filter announcements, removes a redundant
|
||||
spanning-tree metric counter, fixes two convergence and path-MTU bugs, and
|
||||
cuts per-packet CPU in the bloom and identity paths. There is no wire
|
||||
format change and no new feature surface.
|
||||
|
||||
v0.4.1 is wire-compatible with v0.4.0. Nodes can be upgraded one at a time
|
||||
with no coordinated restart, though one behavior change below is worth
|
||||
reading before you start a rolling upgrade.
|
||||
|
||||
## At a glance
|
||||
|
||||
- `node.bloom.max_inbound_fpr` default moves from `0.10` to `0.20`.
|
||||
- The `parent_switched` metric counter is gone. Use `parent_switches`.
|
||||
- Spanning tree no longer serves stale coordinates after a parent link is
|
||||
lost through peer removal.
|
||||
- Discovery no longer loosens a path MTU clamp it had correctly tightened.
|
||||
- Bloom probing and identity operations do measurably less work per call,
|
||||
with identical results.
|
||||
|
||||
## Behavior changes worth flagging
|
||||
|
||||
### The inbound filter FPR cap default doubles again
|
||||
|
||||
`node.bloom.max_inbound_fpr` goes from `0.10` to `0.20`. The cap rejects
|
||||
inbound `FilterAnnounce` frames whose advertised false positive rate
|
||||
exceeds it. On the fixed 1 KB, k=5 filter, `0.10` corresponds to a fill of
|
||||
0.631 and roughly 1,630 reachable entries, and the busiest nodes'
|
||||
aggregates had started reaching that ceiling as the mesh grew. `0.20`
|
||||
corresponds to a fill of 0.7248 and roughly 2,114 entries.
|
||||
|
||||
Be aware that this is the second time in two releases that this default
|
||||
has doubled, for the same reason both times. That is worth stating plainly
|
||||
rather than repeating the previous release's framing: raising the cap buys
|
||||
headroom, it does not fix anything. The real constraint is the fixed 1 KB
|
||||
filter size, which is a protocol constant. The structural remedy is the v2
|
||||
filter work, where filter capacity scales with the mesh instead of being
|
||||
pinned. This release is an interim step to keep legitimate aggregates from
|
||||
being rejected until that lands. It is not the start of a pattern of
|
||||
raising the cap once per release, and if you are sizing capacity planning
|
||||
around this number, plan against the v2 work rather than against a third
|
||||
raise.
|
||||
|
||||
The antipoison property the cap exists for is preserved. A saturated or
|
||||
deliberately poisoned filter still presents an FPR near 100% and is still
|
||||
rejected.
|
||||
|
||||
**This matters during a rolling upgrade.** A v0.4.1 node accepts a
|
||||
`FilterAnnounce` with a derived FPR between 0.10 and 0.20; a v0.4.0 node
|
||||
drops the same frame, and the drop is silent on the wire with no NACK. The
|
||||
cap also gates the mesh size estimator, which declines to produce a value
|
||||
when any contributing filter is over the cap. So while a mesh is partly
|
||||
upgraded, upgraded and not-yet-upgraded nodes can legitimately report
|
||||
different mesh sizes, or one can report a size while the other reports
|
||||
unknown. This resolves once every node is on v0.4.1. If you want to avoid
|
||||
the window entirely, set `node.bloom.max_inbound_fpr: 0.10` explicitly in
|
||||
your config before upgrading and remove it after the last node is done.
|
||||
|
||||
### The `parent_switched` counter is removed
|
||||
|
||||
`parent_switched` was incremented on the line immediately before
|
||||
`parent_switches` at every site and never independently, so the two
|
||||
counters always held the same value. `parent_switched` is now gone from
|
||||
the tree metrics, the control socket snapshot, and the `fipstop` tree
|
||||
view. `parent_switches` remains and is unchanged.
|
||||
|
||||
If you scrape the control socket, or have dashboards or alerts referencing
|
||||
`parent_switched`, point them at `parent_switches`. Anything still asking
|
||||
for `parent_switched` will find nothing rather than a zero.
|
||||
|
||||
## Notable bug fixes
|
||||
|
||||
### Stale coordinates after losing a parent through peer removal
|
||||
|
||||
When a node's parent link dropped via peer removal, the node correctly
|
||||
reparented or self-rooted, but skipped the coordinate cache invalidation
|
||||
that every other position-change path performs. Cached entries for
|
||||
downstream destinations kept the node's old coordinate prefix. This did
|
||||
not self-correct the way a stale cache entry normally would: routing
|
||||
access refreshes an entry's TTL, so an entry that was actively being
|
||||
routed through never expired, and was only fixed by an unrelated fresh
|
||||
insert. Both invalidation classes now run on this path, matching the
|
||||
loop-detection branch.
|
||||
|
||||
### Discovery could loosen a tightened path MTU clamp
|
||||
|
||||
An originator handling a `LookupResponse` overwrote its cached path MTU
|
||||
unconditionally. If a reactive `MtuExceeded` or `PathMtuNotification` had
|
||||
already taught it a tighter value, a later, looser discovery estimate
|
||||
would clobber that and re-loosen the clamp, risking a return to dropped
|
||||
oversized packets. The cached and received values are now compared and the
|
||||
tighter one is kept.
|
||||
|
||||
## Upgrade notes
|
||||
|
||||
This is a drop-in upgrade from v0.4.0 with no wire format change, no
|
||||
config migration, and no coordinated restart. Upgrade nodes in whatever
|
||||
order you like.
|
||||
|
||||
Two things to do rather than assume:
|
||||
|
||||
1. If you monitor `parent_switched`, move to `parent_switches` before
|
||||
upgrading, or your dashboards will go blank rather than error.
|
||||
2. During the rolling window, expect upgraded and not-yet-upgraded nodes
|
||||
to potentially disagree about mesh size, per the FPR cap section above.
|
||||
This is expected and self-resolves. Do not chase it as a bug unless it
|
||||
persists after every node reports `0.4.1`.
|
||||
|
||||
If you have pinned `node.bloom.max_inbound_fpr` explicitly in your config,
|
||||
your setting is honored and nothing changes for you. The change only
|
||||
affects nodes taking the default.
|
||||
|
||||
Downgrading to v0.4.0 is supported and needs no special handling.
|
||||
|
||||
## Getting v0.4.1
|
||||
|
||||
- **Linux x86_64 / aarch64**: `.deb` and tarball at the
|
||||
[v0.4.1 release page](https://github.com/jmcorgan/fips/releases/tag/v0.4.1).
|
||||
- **Arch Linux**: `fips` from the AUR.
|
||||
- **macOS**: `.pkg` at the v0.4.1 release page.
|
||||
- **Windows**: ZIP at the v0.4.1 release page.
|
||||
- **OpenWrt**: `.ipk` (OpenWrt 24.x and earlier) or `.apk` (OpenWrt 25+)
|
||||
at the v0.4.1 release page.
|
||||
- **From source**: `cargo build --release` from a checkout of the v0.4.1
|
||||
tag (Rust 1.94.1 per `rust-toolchain.toml`; `libclang-dev` is a
|
||||
required Linux build prerequisite).
|
||||
- **Nix / NixOS**: `nix build .#fips` from a checkout of the v0.4.1 tag
|
||||
builds the binaries from source with the pinned toolchain and no manual
|
||||
prerequisites (see the Nix section of `packaging/README.md`).
|
||||
|
||||
The full per-commit changelog lives in
|
||||
[`CHANGELOG.md`](../../CHANGELOG.md). Issues and discussion at
|
||||
[github.com/jmcorgan/fips](https://github.com/jmcorgan/fips).
|
||||
|
||||
## Contributors
|
||||
|
||||
Thanks to everyone who contributed code, packaging work, bug reports, or
|
||||
reviews to this release.
|
||||
|
||||
- [@jcorgan](https://github.com/jmcorgan): release shepherd, spanning-tree
|
||||
and discovery fixes, bloom and identity performance work, antipoison cap
|
||||
change, and testing.
|
||||
@@ -635,8 +635,8 @@ pub struct BloomConfig {
|
||||
pub update_debounce_ms: u64,
|
||||
/// Antipoison cap: reject inbound FilterAnnounce whose FPR exceeds
|
||||
/// this value (`node.bloom.max_inbound_fpr`). Valid range `(0.0, 1.0)`.
|
||||
/// Default `0.10` ≈ fill 0.631 at k=5 ≈ ~1,630 entries on the 1 KB
|
||||
/// filter (Swamidass–Baldi). Raised from 0.05 so aggregates that are
|
||||
/// Default `0.20` ≈ fill 0.7248 at k=5 ≈ ~2,114 entries on the 1 KB
|
||||
/// filter (Swamidass–Baldi). Raised from 0.10 so aggregates that are
|
||||
/// legitimately near their operating ceiling are not rejected before
|
||||
/// the network reaches the fixed-filter capacity limit; conceptually
|
||||
/// distinct from future autoscaling hysteresis setpoints — same unit,
|
||||
@@ -648,8 +648,8 @@ pub struct BloomConfig {
|
||||
impl Default for BloomConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
update_debounce_ms: 500,
|
||||
max_inbound_fpr: 0.10,
|
||||
update_debounce_ms: Self::default_update_debounce_ms(),
|
||||
max_inbound_fpr: Self::default_max_inbound_fpr(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -659,7 +659,7 @@ impl BloomConfig {
|
||||
500
|
||||
}
|
||||
fn default_max_inbound_fpr() -> f64 {
|
||||
0.10
|
||||
0.20
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ x-fips-common: &fips-common
|
||||
services:
|
||||
service-a:
|
||||
<<: *fips-common
|
||||
container_name: fips-acl-container-a
|
||||
container_name: fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: host-a
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -43,7 +43,7 @@ services:
|
||||
|
||||
service-b:
|
||||
<<: *fips-common
|
||||
container_name: fips-acl-container-b
|
||||
container_name: fips-acl-container-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: host-b
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -58,7 +58,7 @@ services:
|
||||
|
||||
service-c:
|
||||
<<: *fips-common
|
||||
container_name: fips-acl-container-c
|
||||
container_name: fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: host-c
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -73,7 +73,7 @@ services:
|
||||
|
||||
service-d:
|
||||
<<: *fips-common
|
||||
container_name: fips-acl-container-d
|
||||
container_name: fips-acl-container-d${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: host-d
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -88,7 +88,7 @@ services:
|
||||
|
||||
service-e:
|
||||
<<: *fips-common
|
||||
container_name: fips-acl-container-e
|
||||
container_name: fips-acl-container-e${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: host-e
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -101,7 +101,7 @@ services:
|
||||
|
||||
service-f:
|
||||
<<: *fips-common
|
||||
container_name: fips-acl-container-f
|
||||
container_name: fips-acl-container-f${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: host-f
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
|
||||
@@ -132,31 +132,31 @@ docker compose -f "$COMPOSE_FILE" down >/dev/null 2>&1 || true
|
||||
docker compose -f "$COMPOSE_FILE" up -d --build
|
||||
|
||||
log "Waiting for expected peer convergence"
|
||||
wait_for_peers_exact fips-acl-container-a 3 40
|
||||
wait_for_peers_exact fips-acl-container-b 1 40
|
||||
wait_for_peers_exact fips-acl-container-c 0 5
|
||||
wait_for_peers_exact fips-acl-container-d 0 5
|
||||
wait_for_peers_exact fips-acl-container-e 1 40
|
||||
wait_for_peers_exact fips-acl-container-f 1 40
|
||||
wait_for_peers_exact fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} 3 40
|
||||
wait_for_peers_exact fips-acl-container-b${FIPS_CI_NAME_SUFFIX:-} 1 40
|
||||
wait_for_peers_exact fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-} 0 5
|
||||
wait_for_peers_exact fips-acl-container-d${FIPS_CI_NAME_SUFFIX:-} 0 5
|
||||
wait_for_peers_exact fips-acl-container-e${FIPS_CI_NAME_SUFFIX:-} 1 40
|
||||
wait_for_peers_exact fips-acl-container-f${FIPS_CI_NAME_SUFFIX:-} 1 40
|
||||
|
||||
log "Verifying peer sets"
|
||||
assert_peer_set fips-acl-container-a "npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
|
||||
assert_peer_set fips-acl-container-b "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
|
||||
assert_peer_set fips-acl-container-c ""
|
||||
assert_peer_set fips-acl-container-d ""
|
||||
assert_peer_set fips-acl-container-e "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
|
||||
assert_peer_set fips-acl-container-f "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
|
||||
assert_peer_set fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
|
||||
assert_peer_set fips-acl-container-b${FIPS_CI_NAME_SUFFIX:-} "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
|
||||
assert_peer_set fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-} ""
|
||||
assert_peer_set fips-acl-container-d${FIPS_CI_NAME_SUFFIX:-} ""
|
||||
assert_peer_set fips-acl-container-e${FIPS_CI_NAME_SUFFIX:-} "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
|
||||
assert_peer_set fips-acl-container-f${FIPS_CI_NAME_SUFFIX:-} "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
|
||||
|
||||
log "Checking alias-based ACL resolution"
|
||||
assert_acl_field fips-acl-container-a allow_file_entries "node-a node-b node-e node-f"
|
||||
assert_acl_field fips-acl-container-a allow_entries "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
|
||||
assert_acl_field fips-acl-container-c allow_file_entries "node-a node-b node-c node-d node-e node-f"
|
||||
assert_acl_field fips-acl-container-c allow_entries "npub1cld9yay0u24davpu6c35l4vldrhzvaq66pcqtg9a0j2cnjrn9rtsxx2pe6 npub1n9lpnv0592cc2ps6nm0ca3qls642vx7yjsv35rkxqzj2vgds52sqgpverl npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
|
||||
assert_acl_field fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} allow_file_entries "node-a node-b node-e node-f"
|
||||
assert_acl_field fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} allow_entries "npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
|
||||
assert_acl_field fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-} allow_file_entries "node-a node-b node-c node-d node-e node-f"
|
||||
assert_acl_field fips-acl-container-c${FIPS_CI_NAME_SUFFIX:-} allow_entries "npub1cld9yay0u24davpu6c35l4vldrhzvaq66pcqtg9a0j2cnjrn9rtsxx2pe6 npub1n9lpnv0592cc2ps6nm0ca3qls642vx7yjsv35rkxqzj2vgds52sqgpverl npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le npub1x5z9rwzzm26q9verutx4aajhf2zw2pyp34c6whhde2zduxqav40qgq36l6 npub1ytrut7gjncn2zfnhn56c0zgftf0w6p99gf6fu8j73hzw5603zglqc9av6c"
|
||||
|
||||
log "Checking ACL rejection logs"
|
||||
assert_log_contains fips-acl-container-a "npub1cld9yay0u24davpu6c35l4vldrhzvaq66pcqtg9a0j2cnjrn9rtsxx2pe6"
|
||||
assert_log_contains fips-acl-container-a "npub1n9lpnv0592cc2ps6nm0ca3qls642vx7yjsv35rkxqzj2vgds52sqgpverl"
|
||||
assert_log_contains fips-acl-container-a "context=inbound_handshake"
|
||||
assert_log_contains fips-acl-container-a "decision=denylist match"
|
||||
assert_log_contains fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "npub1cld9yay0u24davpu6c35l4vldrhzvaq66pcqtg9a0j2cnjrn9rtsxx2pe6"
|
||||
assert_log_contains fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "npub1n9lpnv0592cc2ps6nm0ca3qls642vx7yjsv35rkxqzj2vgds52sqgpverl"
|
||||
assert_log_contains fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "context=inbound_handshake"
|
||||
assert_log_contains fips-acl-container-a${FIPS_CI_NAME_SUFFIX:-} "decision=denylist match"
|
||||
|
||||
log "ACL allowlist integration test passed"
|
||||
|
||||
@@ -27,7 +27,7 @@ x-boringtun-common: &boringtun-common
|
||||
services:
|
||||
alice:
|
||||
<<: *boringtun-common
|
||||
container_name: bt-alice
|
||||
container_name: bt-alice${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: alice
|
||||
environment:
|
||||
- ROLE=alice
|
||||
@@ -38,7 +38,7 @@ services:
|
||||
|
||||
bob:
|
||||
<<: *boringtun-common
|
||||
container_name: bt-bob
|
||||
container_name: bt-bob${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: bob
|
||||
environment:
|
||||
- ROLE=bob
|
||||
|
||||
@@ -10,14 +10,14 @@ PARALLEL="${PARALLEL:-1}"
|
||||
echo "=== boringtun iperf3 throughput (single TCP stream, ${DURATION}s) ==="
|
||||
|
||||
# Run iperf3 server on alice (background), client on bob.
|
||||
docker exec -d bt-alice iperf3 -s -1 -B 10.99.0.1 -p 5201
|
||||
docker exec -d bt-alice${FIPS_CI_NAME_SUFFIX:-} iperf3 -s -1 -B 10.99.0.1 -p 5201
|
||||
sleep 1
|
||||
|
||||
# wait for tun handshake to settle (boringtun + WG keepalive)
|
||||
sleep 2
|
||||
|
||||
# Client: bob → alice over WG (10.99.0.1)
|
||||
OUT=$(docker exec bt-bob iperf3 -c 10.99.0.1 -p 5201 -t "$DURATION" -P "$PARALLEL" -J)
|
||||
OUT=$(docker exec bt-bob${FIPS_CI_NAME_SUFFIX:-} iperf3 -c 10.99.0.1 -p 5201 -t "$DURATION" -P "$PARALLEL" -J)
|
||||
|
||||
# Pull SUM bps.
|
||||
MBPS=$(echo "$OUT" | python3 -c "import json,sys; d=json.load(sys.stdin); print(f\"{d['end']['sum_received']['bits_per_second'] / 1_000_000:.2f}\")")
|
||||
|
||||
@@ -7,18 +7,25 @@
|
||||
# prior run died:
|
||||
#
|
||||
# 1. The CI label com.corganlabs.fips-ci=1 (attached to every direct
|
||||
# `docker run`/network/volume ci-local drives).
|
||||
# `docker run`/network/volume ci-local drives). Every run additionally
|
||||
# stamps com.corganlabs.fips-ci.run=<run-id> on the same resources.
|
||||
# 2. The compose project-name prefix fipsci_ (every compose project ci-local
|
||||
# starts is named fipsci_<run-id>_<suite>, so its containers/networks/
|
||||
# volumes all carry com.docker.compose.project=fipsci_... and are named
|
||||
# with that prefix).
|
||||
#
|
||||
# The generic CI label is shared by every run on the host, so an unscoped label
|
||||
# sweep would tear down a CONCURRENT run's resources. Pass --run-id to narrow
|
||||
# the label sweep to one run; a run's own teardown always does. Without it the
|
||||
# label sweep stays broad, which is what a manual "reap everything" wants.
|
||||
#
|
||||
# Usage:
|
||||
# ci-cleanup.sh Reap ALL fips-ci resources (any run)
|
||||
# ci-cleanup.sh --project-prefix P Restrict the compose-project sweep to
|
||||
# names starting with P (scopes the reap
|
||||
# to a single run; the label sweep still
|
||||
# runs)
|
||||
# to a single run)
|
||||
# ci-cleanup.sh --run-id ID Restrict the label sweep to the run
|
||||
# labelled ID (leaves other runs alone)
|
||||
# ci-cleanup.sh --label L Override the CI label (default above)
|
||||
# ci-cleanup.sh --images "a b,c" Also `docker rmi -f` these image tags
|
||||
# (space- or comma-separated)
|
||||
@@ -28,13 +35,16 @@
|
||||
set -uo pipefail
|
||||
|
||||
LABEL="com.corganlabs.fips-ci=1"
|
||||
RUN_LABEL_KEY="com.corganlabs.fips-ci.run"
|
||||
PROJECT_PREFIX="fipsci_" # broad default: every CI run
|
||||
RUN_ID="" # broad default: every CI run
|
||||
IMAGES=""
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--label) LABEL="$2"; shift 2 ;;
|
||||
--project-prefix) PROJECT_PREFIX="$2"; shift 2 ;;
|
||||
--run-id) RUN_ID="$2"; shift 2 ;;
|
||||
--images) IMAGES="$2"; shift 2 ;;
|
||||
-h|--help) sed -n '2,/^set /{ /^set /d; s/^# \?//; p }' "$0"; exit 0 ;;
|
||||
*) echo "Unknown option: $1" >&2; exit 2 ;;
|
||||
@@ -54,6 +64,14 @@ fi
|
||||
# never wedge a caller (ci-local's signal trap relies on this being bounded).
|
||||
TMO=30
|
||||
|
||||
# Selector for the label sweep. With --run-id it matches only the named run, so
|
||||
# a concurrent run's resources are left alone; without it, every CI run.
|
||||
if [[ -n "$RUN_ID" ]]; then
|
||||
SWEEP_LABEL="${RUN_LABEL_KEY}=${RUN_ID}"
|
||||
else
|
||||
SWEEP_LABEL="$LABEL"
|
||||
fi
|
||||
|
||||
# Distinct compose project names (read off container labels) that start with
|
||||
# the configured prefix.
|
||||
ci_projects() {
|
||||
@@ -63,7 +81,7 @@ ci_projects() {
|
||||
|
||||
reap_containers() {
|
||||
# By CI label.
|
||||
docker ps -aq --filter "label=${LABEL}" 2>/dev/null \
|
||||
docker ps -aq --filter "label=${SWEEP_LABEL}" 2>/dev/null \
|
||||
| xargs -r timeout "$TMO" docker rm -f >/dev/null 2>&1 || true
|
||||
# By compose project (carried even when container_name is explicit).
|
||||
local p
|
||||
@@ -74,7 +92,7 @@ reap_containers() {
|
||||
}
|
||||
|
||||
reap_networks() {
|
||||
docker network ls -q --filter "label=${LABEL}" 2>/dev/null \
|
||||
docker network ls -q --filter "label=${SWEEP_LABEL}" 2>/dev/null \
|
||||
| xargs -r timeout "$TMO" docker network rm >/dev/null 2>&1 || true
|
||||
# Compose networks are named <project>_<net> → match by name prefix so
|
||||
# orphaned networks (whose containers are already gone) are still caught.
|
||||
@@ -83,7 +101,7 @@ reap_networks() {
|
||||
}
|
||||
|
||||
reap_volumes() {
|
||||
docker volume ls -q --filter "label=${LABEL}" 2>/dev/null \
|
||||
docker volume ls -q --filter "label=${SWEEP_LABEL}" 2>/dev/null \
|
||||
| xargs -r timeout "$TMO" docker volume rm >/dev/null 2>&1 || true
|
||||
docker volume ls --format '{{.Name}}' 2>/dev/null | grep -E "^${PROJECT_PREFIX}" \
|
||||
| xargs -r timeout "$TMO" docker volume rm >/dev/null 2>&1 || true
|
||||
|
||||
@@ -254,6 +254,10 @@ CI_PROJECT_PREFIX="fipsci_${CI_RUN_ID}"
|
||||
CI_IMAGE_TEST="fips-test:${CI_RUN_ID}"
|
||||
CI_IMAGE_APP="fips-test-app:${CI_RUN_ID}"
|
||||
CI_LABEL="com.corganlabs.fips-ci=1"
|
||||
# Run-scoped companion to CI_LABEL. The generic label is shared by every run on
|
||||
# the host, so teardown filters on this one instead — otherwise this run's reap
|
||||
# would force-remove a concurrent run's containers out from under it.
|
||||
CI_LABEL_RUN="com.corganlabs.fips-ci.run=${CI_RUN_ID}"
|
||||
|
||||
# Exported so child suite scripts + their compose/`docker run` invocations
|
||||
# inherit the run identity. Compose files read FIPS_TEST_IMAGE/FIPS_TEST_APP_IMAGE
|
||||
@@ -261,6 +265,12 @@ CI_LABEL="com.corganlabs.fips-ci=1"
|
||||
export FIPS_CI_RUN_ID="$CI_RUN_ID"
|
||||
export FIPS_TEST_IMAGE="$CI_IMAGE_TEST"
|
||||
export FIPS_TEST_APP_IMAGE="$CI_IMAGE_APP"
|
||||
# Docker container names are GLOBAL — a compose project name does not scope
|
||||
# them — so the suite compose files append this suffix to every explicit
|
||||
# container_name, and the suite scripts append it wherever they address a
|
||||
# container by name. Empty when unset, so a bare `docker compose up` outside
|
||||
# this harness still produces today's plain names.
|
||||
export FIPS_CI_NAME_SUFFIX="-${CI_RUN_ID}"
|
||||
|
||||
# Per-suite compose project name: ${prefix}_<suite-or-compose-basename>. Keeps
|
||||
# today's intra-run distinctness (one project per compose file / chaos child)
|
||||
@@ -295,6 +305,7 @@ ci_teardown() {
|
||||
# the whole sweep too so the trap can never wedge.
|
||||
timeout 150 bash "$SCRIPT_DIR/ci-cleanup.sh" \
|
||||
--label "$CI_LABEL" \
|
||||
--run-id "$CI_RUN_ID" \
|
||||
--project-prefix "$CI_PROJECT_PREFIX" \
|
||||
--images "$CI_IMAGE_TEST $CI_IMAGE_APP" >/dev/null 2>&1 || true
|
||||
}
|
||||
@@ -747,9 +758,9 @@ run_integration() {
|
||||
# reference fips-test:latest directly; the retag happens only after BOTH
|
||||
# builds succeed, so :latest never points at a half-built image.
|
||||
info "Building $CI_IMAGE_TEST Docker image"
|
||||
docker build -t "$CI_IMAGE_TEST" --label "$CI_LABEL" testing/docker --quiet \
|
||||
docker build -t "$CI_IMAGE_TEST" --label "$CI_LABEL" --label "$CI_LABEL_RUN" testing/docker --quiet \
|
||||
|| { record "docker-build" 1; return; }
|
||||
docker build -t "$CI_IMAGE_APP" --label "$CI_LABEL" \
|
||||
docker build -t "$CI_IMAGE_APP" --label "$CI_LABEL" --label "$CI_LABEL_RUN" \
|
||||
-f testing/docker/Dockerfile.app testing/docker --quiet \
|
||||
|| { record "docker-build-app" 1; return; }
|
||||
docker tag "$CI_IMAGE_TEST" fips-test:latest
|
||||
|
||||
@@ -187,7 +187,7 @@ _run_deb_install_scenario() {
|
||||
local distro_label="$1"
|
||||
local base_image="$2"
|
||||
|
||||
local name="fips-deb-test-${distro_label}"
|
||||
local name="fips-deb-test-${distro_label}${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-deb-test:${distro_label}"
|
||||
log ".deb install: ${base_image}"
|
||||
|
||||
|
||||
@@ -284,7 +284,7 @@ DOCKERFILE
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
test_debian12_resolved() {
|
||||
local name="fips-dns-test-deb12-resolved"
|
||||
local name="fips-dns-test-deb12-resolved${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-dns-test:debian12-resolved"
|
||||
log "Debian 12 + systemd-resolved (expects global-drop-in)"
|
||||
|
||||
@@ -312,7 +312,7 @@ DOCKERFILE
|
||||
}
|
||||
|
||||
test_debian13_resolved() {
|
||||
local name="fips-dns-test-deb13-resolved"
|
||||
local name="fips-dns-test-deb13-resolved${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-dns-test:debian13-resolved"
|
||||
log "Debian 13 (trixie) + systemd-resolved (expects global-drop-in)"
|
||||
|
||||
@@ -340,7 +340,7 @@ DOCKERFILE
|
||||
}
|
||||
|
||||
test_ubuntu22_resolved() {
|
||||
local name="fips-dns-test-u22-resolved"
|
||||
local name="fips-dns-test-u22-resolved${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-dns-test:ubuntu22-resolved"
|
||||
log "Ubuntu 22.04 + systemd-resolved (expects global-drop-in)"
|
||||
|
||||
@@ -370,7 +370,7 @@ DOCKERFILE
|
||||
}
|
||||
|
||||
test_ubuntu24_resolved() {
|
||||
local name="fips-dns-test-u24-resolved"
|
||||
local name="fips-dns-test-u24-resolved${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-dns-test:ubuntu24-resolved"
|
||||
log "Ubuntu 24.04 + systemd-resolved (expects global-drop-in)"
|
||||
|
||||
@@ -398,7 +398,7 @@ DOCKERFILE
|
||||
}
|
||||
|
||||
test_ubuntu26_resolved() {
|
||||
local name="fips-dns-test-u26-resolved"
|
||||
local name="fips-dns-test-u26-resolved${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-dns-test:ubuntu26-resolved"
|
||||
log "Ubuntu 26.04 + systemd-resolved (expects global-drop-in)"
|
||||
|
||||
@@ -426,7 +426,7 @@ DOCKERFILE
|
||||
}
|
||||
|
||||
test_dnsmasq() {
|
||||
local name="fips-dns-test-dnsmasq"
|
||||
local name="fips-dns-test-dnsmasq${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-dns-test:dnsmasq"
|
||||
log "Debian 12 + dnsmasq standalone"
|
||||
|
||||
@@ -492,7 +492,7 @@ DOCKERFILE
|
||||
}
|
||||
|
||||
test_nm_dnsmasq() {
|
||||
local name="fips-dns-test-nm-dnsmasq"
|
||||
local name="fips-dns-test-nm-dnsmasq${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-dns-test:nm-dnsmasq"
|
||||
log "Fedora + NetworkManager + dnsmasq plugin"
|
||||
|
||||
@@ -555,7 +555,7 @@ DOCKERFILE
|
||||
}
|
||||
|
||||
test_no_resolver() {
|
||||
local name="fips-dns-test-none"
|
||||
local name="fips-dns-test-none${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-dns-test:none"
|
||||
log "Debian 12 bare (no resolver)"
|
||||
|
||||
@@ -625,7 +625,7 @@ _run_e2e_scenario() {
|
||||
local base_image="$2"
|
||||
local apt_packages="$3"
|
||||
|
||||
local name="fips-dns-test-e2e-${distro_label}"
|
||||
local name="fips-dns-test-e2e-${distro_label}${FIPS_CI_NAME_SUFFIX:-}"
|
||||
local image="fips-dns-test:e2e-${distro_label}"
|
||||
log "End-to-end: ${base_image} + systemd-resolved + real fips + fips-gateway + dig"
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ x-fips-common: &fips-common
|
||||
services:
|
||||
service-a:
|
||||
<<: *fips-common
|
||||
container_name: fips-fw-container-a
|
||||
container_name: fips-fw-container-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: host-a
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -40,7 +40,7 @@ services:
|
||||
|
||||
service-b:
|
||||
<<: *fips-common
|
||||
container_name: fips-fw-container-b
|
||||
container_name: fips-fw-container-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: host-b
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
|
||||
@@ -22,8 +22,8 @@ TESTING_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
COMPOSE_FILE="$SCRIPT_DIR/docker-compose.yml"
|
||||
GENERATE_CONFIGS="$SCRIPT_DIR/generate-configs.sh"
|
||||
|
||||
CONTAINER_A="fips-fw-container-a"
|
||||
CONTAINER_B="fips-fw-container-b"
|
||||
CONTAINER_A="fips-fw-container-a${FIPS_CI_NAME_SUFFIX:-}"
|
||||
CONTAINER_B="fips-fw-container-b${FIPS_CI_NAME_SUFFIX:-}"
|
||||
|
||||
NPUB_A="npub1sjlh2c3x9w7kjsqg2ay080n2lff2uvt325vpan33ke34rn8l5jcqawh57m"
|
||||
NPUB_B="npub1tdwa4vjrjl33pcjdpf2t4p027nl86xrx24g4d3avg4vwvayr3g8qhd84le"
|
||||
|
||||
@@ -34,7 +34,7 @@ services:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: examples/sidecar-nostr-relay/Dockerfile.app
|
||||
container_name: fips-nat-relay
|
||||
container_name: fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}
|
||||
restart: "no"
|
||||
volumes:
|
||||
- relay-data:/usr/src/app/strfry-db
|
||||
@@ -49,7 +49,7 @@ services:
|
||||
stun:
|
||||
build:
|
||||
context: ./stun
|
||||
container_name: fips-nat-stun
|
||||
container_name: fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}
|
||||
restart: "no"
|
||||
networks:
|
||||
wan:
|
||||
@@ -61,7 +61,7 @@ services:
|
||||
build:
|
||||
context: ./router
|
||||
profiles: ["cone", "symmetric"]
|
||||
container_name: fips-nat-router-a
|
||||
container_name: fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
sysctls:
|
||||
@@ -84,7 +84,7 @@ services:
|
||||
build:
|
||||
context: ./router
|
||||
profiles: ["cone", "symmetric"]
|
||||
container_name: fips-nat-router-b
|
||||
container_name: fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
sysctls:
|
||||
@@ -106,7 +106,7 @@ services:
|
||||
cone-a:
|
||||
<<: *fips-common
|
||||
profiles: ["cone"]
|
||||
container_name: fips-nat-cone-a
|
||||
container_name: fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-cone-a
|
||||
depends_on:
|
||||
- nat-a
|
||||
@@ -132,7 +132,7 @@ services:
|
||||
cone-b:
|
||||
<<: *fips-common
|
||||
profiles: ["cone"]
|
||||
container_name: fips-nat-cone-b
|
||||
container_name: fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-cone-b
|
||||
depends_on:
|
||||
- nat-b
|
||||
@@ -158,7 +158,7 @@ services:
|
||||
symmetric-a:
|
||||
<<: *fips-common
|
||||
profiles: ["symmetric"]
|
||||
container_name: fips-nat-symmetric-a
|
||||
container_name: fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-symmetric-a
|
||||
depends_on:
|
||||
- nat-a
|
||||
@@ -184,7 +184,7 @@ services:
|
||||
symmetric-b:
|
||||
<<: *fips-common
|
||||
profiles: ["symmetric"]
|
||||
container_name: fips-nat-symmetric-b
|
||||
container_name: fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-symmetric-b
|
||||
depends_on:
|
||||
- nat-b
|
||||
@@ -210,7 +210,7 @@ services:
|
||||
lan-a:
|
||||
<<: *fips-common
|
||||
profiles: ["lan"]
|
||||
container_name: fips-nat-lan-a
|
||||
container_name: fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-lan-a
|
||||
depends_on:
|
||||
- relay
|
||||
@@ -225,7 +225,7 @@ services:
|
||||
lan-b:
|
||||
<<: *fips-common
|
||||
profiles: ["lan"]
|
||||
container_name: fips-nat-lan-b
|
||||
container_name: fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-lan-b
|
||||
depends_on:
|
||||
- relay
|
||||
@@ -247,7 +247,7 @@ services:
|
||||
nostr-pub-a:
|
||||
<<: *fips-common
|
||||
profiles: ["nostr-publish-consume"]
|
||||
container_name: fips-nat-nostr-pub-a
|
||||
container_name: fips-nat-nostr-pub-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-nostr-pub-a
|
||||
depends_on:
|
||||
- relay
|
||||
@@ -262,7 +262,7 @@ services:
|
||||
nostr-pub-b:
|
||||
<<: *fips-common
|
||||
profiles: ["nostr-publish-consume"]
|
||||
container_name: fips-nat-nostr-pub-b
|
||||
container_name: fips-nat-nostr-pub-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-nostr-pub-b
|
||||
depends_on:
|
||||
- relay
|
||||
@@ -291,7 +291,7 @@ services:
|
||||
stun-fault-node:
|
||||
<<: *fips-common
|
||||
profiles: ["stun-faults"]
|
||||
container_name: fips-nat-stun-fault-node
|
||||
container_name: fips-nat-stun-fault-node${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-stun-fault-node
|
||||
depends_on:
|
||||
- relay
|
||||
@@ -312,7 +312,7 @@ services:
|
||||
stun-fault-peer:
|
||||
<<: *fips-common
|
||||
profiles: ["stun-faults"]
|
||||
container_name: fips-nat-stun-fault-peer
|
||||
container_name: fips-nat-stun-fault-peer${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-nat-stun-fault-peer
|
||||
depends_on:
|
||||
- relay
|
||||
@@ -327,7 +327,7 @@ services:
|
||||
stun-fault-shim:
|
||||
image: fips-test:latest
|
||||
profiles: ["stun-faults"]
|
||||
container_name: fips-nat-stun-fault-shim
|
||||
container_name: fips-nat-stun-fault-shim${FIPS_CI_NAME_SUFFIX:-}
|
||||
depends_on:
|
||||
- stun-fault-node
|
||||
cap_add:
|
||||
|
||||
@@ -36,7 +36,7 @@ cleanup() {
|
||||
}
|
||||
|
||||
helper_tcpdump_image() {
|
||||
docker inspect -f '{{.Config.Image}}' fips-nat-router-a 2>/dev/null || echo nat-nat-a
|
||||
docker inspect -f '{{.Config.Image}}' fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-} 2>/dev/null || echo nat-nat-a
|
||||
}
|
||||
|
||||
dump_container_state() {
|
||||
@@ -202,7 +202,7 @@ dump_stun_udp_probe() {
|
||||
|
||||
local capture_file
|
||||
capture_file="$(mktemp)"
|
||||
docker run --rm --label com.corganlabs.fips-ci=1 --net=container:fips-nat-stun --cap-add NET_ADMIN --cap-add NET_RAW \
|
||||
docker run --rm --label com.corganlabs.fips-ci=1 --label "com.corganlabs.fips-ci.run=${FIPS_CI_RUN_ID:-manual}" --net=container:fips-nat-stun${FIPS_CI_NAME_SUFFIX:-} --cap-add NET_ADMIN --cap-add NET_RAW \
|
||||
--entrypoint sh "$helper_image" \
|
||||
-lc "timeout 8 tcpdump -ni any 'udp and not port 53' -c 80" \
|
||||
>"$capture_file" 2>&1 &
|
||||
@@ -225,38 +225,38 @@ dump_stun_udp_probe() {
|
||||
dump_cone_diagnostics() {
|
||||
echo ""
|
||||
echo "=== cone diagnostics ==="
|
||||
dump_fips_state fips-nat-cone-a 172.31.254.30 7777 172.31.254.40 3478
|
||||
dump_node_udp_probe fips-nat-cone-a
|
||||
dump_fips_state fips-nat-cone-b 172.31.254.30 7777 172.31.254.40 3478
|
||||
dump_node_udp_probe fips-nat-cone-b
|
||||
dump_container_state fips-nat-router-a
|
||||
dump_router_udp_probe fips-nat-router-a fips-nat-cone-a
|
||||
dump_container_state fips-nat-router-b
|
||||
dump_router_udp_probe fips-nat-router-b fips-nat-cone-b
|
||||
dump_container_state fips-nat-relay
|
||||
dump_stun_udp_probe fips-nat-cone-a
|
||||
dump_stun_udp_probe fips-nat-cone-b
|
||||
dump_container_state fips-nat-stun
|
||||
dump_fips_state fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} 172.31.254.30 7777 172.31.254.40 3478
|
||||
dump_node_udp_probe fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_fips_state fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} 172.31.254.30 7777 172.31.254.40 3478
|
||||
dump_node_udp_probe fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_container_state fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_router_udp_probe fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-} fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_container_state fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_router_udp_probe fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-} fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_container_state fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_stun_udp_probe fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_stun_udp_probe fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_container_state fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}
|
||||
}
|
||||
|
||||
dump_symmetric_diagnostics() {
|
||||
echo ""
|
||||
echo "=== symmetric diagnostics ==="
|
||||
dump_fips_state fips-nat-symmetric-a 172.31.254.30 7777 172.31.254.40 3478
|
||||
dump_fips_state fips-nat-symmetric-b 172.31.254.30 7777 172.31.254.40 3478
|
||||
dump_container_state fips-nat-router-a
|
||||
dump_container_state fips-nat-router-b
|
||||
dump_container_state fips-nat-relay
|
||||
dump_container_state fips-nat-stun
|
||||
dump_fips_state fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} 172.31.254.30 7777 172.31.254.40 3478
|
||||
dump_fips_state fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} 172.31.254.30 7777 172.31.254.40 3478
|
||||
dump_container_state fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_container_state fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_container_state fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_container_state fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}
|
||||
}
|
||||
|
||||
dump_lan_diagnostics() {
|
||||
echo ""
|
||||
echo "=== lan diagnostics ==="
|
||||
dump_fips_state fips-nat-lan-a 172.31.10.30 7777 172.31.10.40 3478
|
||||
dump_fips_state fips-nat-lan-b 172.31.10.30 7777 172.31.10.40 3478
|
||||
dump_container_state fips-nat-relay
|
||||
dump_container_state fips-nat-stun
|
||||
dump_fips_state fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} 172.31.10.30 7777 172.31.10.40 3478
|
||||
dump_fips_state fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} 172.31.10.30 7777 172.31.10.40 3478
|
||||
dump_container_state fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}
|
||||
dump_container_state fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}
|
||||
}
|
||||
|
||||
trap 'echo ""; echo "NAT test interrupted"; cleanup; exit 130' INT TERM
|
||||
@@ -334,22 +334,22 @@ run_cone() {
|
||||
"$GENERATE_SCRIPT" cone
|
||||
"${COMPOSE[@]}" --profile cone up -d --build --force-recreate
|
||||
"$TOPOLOGY_SCRIPT" cone
|
||||
wait_for_peers fips-nat-cone-a 1 45 || {
|
||||
wait_for_peers fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} 1 45 || {
|
||||
dump_cone_diagnostics
|
||||
return 1
|
||||
}
|
||||
wait_for_peers fips-nat-cone-b 1 45 || {
|
||||
wait_for_peers fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} 1 45 || {
|
||||
dump_cone_diagnostics
|
||||
return 1
|
||||
}
|
||||
assert_peer_path fips-nat-cone-a udp 172.31.254.
|
||||
assert_peer_path fips-nat-cone-b udp 172.31.254.
|
||||
assert_link_path fips-nat-cone-a 172.31.254.
|
||||
assert_link_path fips-nat-cone-b 172.31.254.
|
||||
assert_peer_path fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} udp 172.31.254.
|
||||
assert_peer_path fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} udp 172.31.254.
|
||||
assert_link_path fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} 172.31.254.
|
||||
assert_link_path fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} 172.31.254.
|
||||
# shellcheck disable=SC1090
|
||||
source "$NAT_DIR/generated-configs/cone/npubs.env"
|
||||
ping_peer fips-nat-cone-a "$NPUB_B"
|
||||
ping_peer fips-nat-cone-b "$NPUB_A"
|
||||
ping_peer fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B"
|
||||
ping_peer fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A"
|
||||
cleanup
|
||||
}
|
||||
|
||||
@@ -359,24 +359,24 @@ run_symmetric() {
|
||||
NAT_MODE_A=symmetric NAT_MODE_B=symmetric "$GENERATE_SCRIPT" symmetric
|
||||
NAT_MODE_A=symmetric NAT_MODE_B=symmetric "${COMPOSE[@]}" --profile symmetric up -d --build --force-recreate
|
||||
"$TOPOLOGY_SCRIPT" symmetric
|
||||
wait_for_peers fips-nat-symmetric-a 1 60 || {
|
||||
wait_for_peers fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} 1 60 || {
|
||||
dump_symmetric_diagnostics
|
||||
return 1
|
||||
}
|
||||
wait_for_peers fips-nat-symmetric-b 1 60 || {
|
||||
wait_for_peers fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} 1 60 || {
|
||||
dump_symmetric_diagnostics
|
||||
return 1
|
||||
}
|
||||
assert_peer_path fips-nat-symmetric-a tcp 172.31.254.11:
|
||||
assert_peer_path fips-nat-symmetric-b tcp 172.31.254.10:
|
||||
assert_link_path fips-nat-symmetric-a 172.31.254.11:
|
||||
assert_link_path fips-nat-symmetric-b 172.31.254.10:
|
||||
require_bootstrap_activity fips-nat-symmetric-a
|
||||
require_bootstrap_activity fips-nat-symmetric-b
|
||||
assert_peer_path fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} tcp 172.31.254.11:
|
||||
assert_peer_path fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} tcp 172.31.254.10:
|
||||
assert_link_path fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} 172.31.254.11:
|
||||
assert_link_path fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} 172.31.254.10:
|
||||
require_bootstrap_activity fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
require_bootstrap_activity fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
# shellcheck disable=SC1090
|
||||
source "$NAT_DIR/generated-configs/symmetric/npubs.env"
|
||||
ping_peer fips-nat-symmetric-a "$NPUB_B"
|
||||
ping_peer fips-nat-symmetric-b "$NPUB_A"
|
||||
ping_peer fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B"
|
||||
ping_peer fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A"
|
||||
cleanup
|
||||
}
|
||||
|
||||
@@ -385,22 +385,22 @@ run_lan() {
|
||||
cleanup
|
||||
"$GENERATE_SCRIPT" lan
|
||||
"${COMPOSE[@]}" --profile lan up -d --build --force-recreate
|
||||
wait_for_peers fips-nat-lan-a 1 45 || {
|
||||
wait_for_peers fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} 1 45 || {
|
||||
dump_lan_diagnostics
|
||||
return 1
|
||||
}
|
||||
wait_for_peers fips-nat-lan-b 1 45 || {
|
||||
wait_for_peers fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} 1 45 || {
|
||||
dump_lan_diagnostics
|
||||
return 1
|
||||
}
|
||||
assert_peer_path fips-nat-lan-a udp 172.31.10.
|
||||
assert_peer_path fips-nat-lan-b udp 172.31.10.
|
||||
assert_link_path fips-nat-lan-a 172.31.10.
|
||||
assert_link_path fips-nat-lan-b 172.31.10.
|
||||
assert_peer_path fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} udp 172.31.10.
|
||||
assert_peer_path fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} udp 172.31.10.
|
||||
assert_link_path fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} 172.31.10.
|
||||
assert_link_path fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} 172.31.10.
|
||||
# shellcheck disable=SC1090
|
||||
source "$NAT_DIR/generated-configs/lan/npubs.env"
|
||||
ping_peer fips-nat-lan-a "$NPUB_B"
|
||||
ping_peer fips-nat-lan-b "$NPUB_A"
|
||||
ping_peer fips-nat-lan-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B"
|
||||
ping_peer fips-nat-lan-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A"
|
||||
# Skip the final teardown when the mesh-lab harness wraps this
|
||||
# script: it needs to docker-logs the containers before teardown,
|
||||
# and will run its own cleanup after capture. Failure paths above
|
||||
|
||||
@@ -25,11 +25,11 @@ WAIT_LIB="$ROOT_DIR/testing/lib/wait-converge.sh"
|
||||
PROFILE="nostr-publish-consume"
|
||||
SCENARIO="$PROFILE"
|
||||
COMPOSE=(docker compose -f "$NAT_DIR/docker-compose.yml")
|
||||
NODE_A="fips-nat-nostr-pub-a"
|
||||
NODE_B="fips-nat-nostr-pub-b"
|
||||
NODE_A="fips-nat-nostr-pub-a${FIPS_CI_NAME_SUFFIX:-}"
|
||||
NODE_B="fips-nat-nostr-pub-b${FIPS_CI_NAME_SUFFIX:-}"
|
||||
RELAY_HOST="172.31.10.30"
|
||||
RELAY_PORT=7777
|
||||
RELAY_CONTAINER="fips-nat-relay"
|
||||
RELAY_CONTAINER="fips-nat-relay${FIPS_CI_NAME_SUFFIX:-}"
|
||||
|
||||
# shellcheck disable=SC1090
|
||||
source "$WAIT_LIB"
|
||||
|
||||
@@ -9,12 +9,12 @@ SCENARIO="${1:?usage: setup-topology.sh <cone|symmetric>}"
|
||||
|
||||
case "$SCENARIO" in
|
||||
cone)
|
||||
node_a="fips-nat-cone-a"
|
||||
node_b="fips-nat-cone-b"
|
||||
node_a="fips-nat-cone-a${FIPS_CI_NAME_SUFFIX:-}"
|
||||
node_b="fips-nat-cone-b${FIPS_CI_NAME_SUFFIX:-}"
|
||||
;;
|
||||
symmetric)
|
||||
node_a="fips-nat-symmetric-a"
|
||||
node_b="fips-nat-symmetric-b"
|
||||
node_a="fips-nat-symmetric-a${FIPS_CI_NAME_SUFFIX:-}"
|
||||
node_b="fips-nat-symmetric-b${FIPS_CI_NAME_SUFFIX:-}"
|
||||
;;
|
||||
*)
|
||||
echo "Unsupported topology scenario: $SCENARIO" >&2
|
||||
@@ -22,8 +22,8 @@ case "$SCENARIO" in
|
||||
;;
|
||||
esac
|
||||
|
||||
router_a="fips-nat-router-a"
|
||||
router_b="fips-nat-router-b"
|
||||
router_a="fips-nat-router-a${FIPS_CI_NAME_SUFFIX:-}"
|
||||
router_b="fips-nat-router-b${FIPS_CI_NAME_SUFFIX:-}"
|
||||
|
||||
helper_image() {
|
||||
if [ -n "${IP_HELPER_IMAGE:-}" ]; then
|
||||
|
||||
@@ -33,10 +33,10 @@ GENERATE_SCRIPT="$SCRIPT_DIR/generate-configs.sh"
|
||||
PROFILE="stun-faults"
|
||||
SCENARIO="$PROFILE"
|
||||
COMPOSE=(docker compose -f "$NAT_DIR/docker-compose.yml")
|
||||
NODE="fips-nat-stun-fault-node"
|
||||
PEER="fips-nat-stun-fault-peer"
|
||||
SHIM="fips-nat-stun-fault-shim"
|
||||
STUN_CONTAINER="fips-nat-stun"
|
||||
NODE="fips-nat-stun-fault-node${FIPS_CI_NAME_SUFFIX:-}"
|
||||
PEER="fips-nat-stun-fault-peer${FIPS_CI_NAME_SUFFIX:-}"
|
||||
SHIM="fips-nat-stun-fault-shim${FIPS_CI_NAME_SUFFIX:-}"
|
||||
STUN_CONTAINER="fips-nat-stun${FIPS_CI_NAME_SUFFIX:-}"
|
||||
STUN_HOST="172.31.10.40"
|
||||
STUN_PORT=3478
|
||||
DEV="eth0"
|
||||
|
||||
@@ -25,11 +25,28 @@ NODE_B_NPUB="npub15h7z0ljzudqe9pgwx99cjsz2c0ennuyvkcc8zvtk3lg97xwzex9ska6g4y"
|
||||
NODE_C_NSEC="15148ed0131f7da43fd13e369dfedede14fb64698f3756636b569c3a3e87438f"
|
||||
NODE_C_NPUB="npub1zhezcykd0e34z4fxtranl45jaasgnlxv0kjqwlq2v56ggssn0w4qelcrvr"
|
||||
|
||||
NETWORK_NAME="fips-sidecar-test"
|
||||
SUBNET="172.20.2.0/24"
|
||||
NODE_A_IP="172.20.2.10"
|
||||
NODE_B_IP="172.20.2.11"
|
||||
NODE_C_IP="172.20.2.12"
|
||||
NETWORK_NAME="fips-sidecar-test${FIPS_CI_NAME_SUFFIX:-}"
|
||||
|
||||
# Compose project names. These are explicit (they override COMPOSE_PROJECT_NAME),
|
||||
# so they carry the run suffix themselves — otherwise concurrent runs would share
|
||||
# one project and tear down each other's containers.
|
||||
PROJ_A="sidecar-a${FIPS_CI_NAME_SUFFIX:-}"
|
||||
PROJ_B="sidecar-b${FIPS_CI_NAME_SUFFIX:-}"
|
||||
PROJ_C="sidecar-c${FIPS_CI_NAME_SUFFIX:-}"
|
||||
# Network address range — claimed at startup by alloc_network below rather than
|
||||
# hardcoded, so two concurrent runs cannot request the same range.
|
||||
#
|
||||
# 10.40.0.0/16 is unclaimed in-tree: the suites use 172.20-172.32 and 172.99,
|
||||
# the chaos children 10.30.x, boringtun's inner network 10.99.x, and docker's
|
||||
# own default pool is 172.17-31 plus 192.168.
|
||||
NET_BASE="10.40"
|
||||
NET_CANDIDATES=64
|
||||
|
||||
SUBNET=""
|
||||
GATEWAY_IP=""
|
||||
NODE_A_IP=""
|
||||
NODE_B_IP=""
|
||||
NODE_C_IP=""
|
||||
|
||||
CONVERGE_TIMEOUT=30
|
||||
PASSED=0
|
||||
@@ -45,13 +62,50 @@ log() { echo "=== $*"; }
|
||||
pass() { echo " PASS: $*"; PASSED=$((PASSED + 1)); }
|
||||
fail() { echo " FAIL: $*"; FAILED=$((FAILED + 1)); }
|
||||
|
||||
# Claim a free /24 for this run's network.
|
||||
#
|
||||
# Claim-and-advance rather than deriving an offset from the run id: we attempt
|
||||
# creation on a candidate range and, if docker reports the range is taken, move
|
||||
# to the next. Docker's own address pool is then the arbiter, which makes an
|
||||
# overlap between two concurrent runs *impossible*. A hashed offset would only
|
||||
# make it unlikely, and an overlap is precisely the failure being avoided here.
|
||||
#
|
||||
# Deliberately does NOT discard stderr: only an address-pool conflict is worth
|
||||
# advancing on. Any other failure (name already taken, daemon error) is real,
|
||||
# and retrying it 64 times would bury the reason.
|
||||
alloc_network() {
|
||||
local i err
|
||||
for (( i = 0; i < NET_CANDIDATES; i++ )); do
|
||||
SUBNET="${NET_BASE}.${i}.0/24"
|
||||
if err=$(docker network create \
|
||||
--subnet "$SUBNET" \
|
||||
--label com.corganlabs.fips-ci=1 \
|
||||
--label "com.corganlabs.fips-ci.run=${FIPS_CI_RUN_ID:-manual}" \
|
||||
"$NETWORK_NAME" 2>&1); then
|
||||
GATEWAY_IP="${NET_BASE}.${i}.1"
|
||||
NODE_A_IP="${NET_BASE}.${i}.10"
|
||||
NODE_B_IP="${NET_BASE}.${i}.11"
|
||||
NODE_C_IP="${NET_BASE}.${i}.12"
|
||||
log "Claimed network $NETWORK_NAME on $SUBNET"
|
||||
return 0
|
||||
fi
|
||||
case "$err" in
|
||||
*"Pool overlaps"*|*"pool overlaps"*) continue ;;
|
||||
*) echo " FAIL: docker network create: $err" >&2; return 1 ;;
|
||||
esac
|
||||
done
|
||||
echo " FAIL: no free /24 in ${NET_BASE}.0.0/16 after ${NET_CANDIDATES} attempts" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
cleanup() {
|
||||
log "Cleaning up..."
|
||||
# Tear down B and C first (they reference A's network as external)
|
||||
docker compose $COMPOSE_EXT -p sidecar-c down --volumes --remove-orphans 2>/dev/null || true
|
||||
docker compose $COMPOSE_EXT -p sidecar-b down --volumes --remove-orphans 2>/dev/null || true
|
||||
# Tear down A last (it owns the network)
|
||||
docker compose $COMPOSE_BASE -p sidecar-a down --volumes --remove-orphans 2>/dev/null || true
|
||||
# Tear down C and B first, then A. All three attach to the network as
|
||||
# external, so none of them removes it — we do that last, by name.
|
||||
docker compose $COMPOSE_EXT -p "$PROJ_C" down --volumes --remove-orphans 2>/dev/null || true
|
||||
docker compose $COMPOSE_EXT -p "$PROJ_B" down --volumes --remove-orphans 2>/dev/null || true
|
||||
docker compose $COMPOSE_EXT -p "$PROJ_A" down --volumes --remove-orphans 2>/dev/null || true
|
||||
docker network rm "$NETWORK_NAME" >/dev/null 2>&1 || true
|
||||
}
|
||||
|
||||
# Always clean up on exit
|
||||
@@ -73,7 +127,9 @@ fi
|
||||
# node-b: peers with A (middle node, transit router)
|
||||
# node-c: peers with B (end node)
|
||||
|
||||
log "Starting node-a (no peers, creates network)..."
|
||||
alloc_network || exit 1
|
||||
|
||||
log "Starting node-a (no peers, joins the claimed network)..."
|
||||
# node-a is the root: explicitly clear FIPS_PEER_* so it does not inherit the
|
||||
# external peer default from .env (which points at a real public mesh node).
|
||||
# Without this, node-a auto-connects to the live mesh and the chain attaches
|
||||
@@ -84,7 +140,7 @@ FIPS_PEER_ADDR="" \
|
||||
FIPS_NETWORK="$NETWORK_NAME" \
|
||||
FIPS_SUBNET="$SUBNET" \
|
||||
FIPS_IPV4="$NODE_A_IP" \
|
||||
docker compose $COMPOSE_BASE -p sidecar-a up -d
|
||||
docker compose $COMPOSE_EXT -p "$PROJ_A" up -d
|
||||
|
||||
log "Starting node-b (peers with node-a, joins external network)..."
|
||||
FIPS_NSEC="$NODE_B_NSEC" \
|
||||
@@ -94,7 +150,7 @@ FIPS_PEER_ALIAS="node-a" \
|
||||
FIPS_NETWORK="$NETWORK_NAME" \
|
||||
FIPS_SUBNET="$SUBNET" \
|
||||
FIPS_IPV4="$NODE_B_IP" \
|
||||
docker compose $COMPOSE_EXT -p sidecar-b up -d
|
||||
docker compose $COMPOSE_EXT -p "$PROJ_B" up -d
|
||||
|
||||
log "Starting node-c (peers with node-b, joins external network)..."
|
||||
FIPS_NSEC="$NODE_C_NSEC" \
|
||||
@@ -104,7 +160,7 @@ FIPS_PEER_ALIAS="node-b" \
|
||||
FIPS_NETWORK="$NETWORK_NAME" \
|
||||
FIPS_SUBNET="$SUBNET" \
|
||||
FIPS_IPV4="$NODE_C_IP" \
|
||||
docker compose $COMPOSE_EXT -p sidecar-c up -d
|
||||
docker compose $COMPOSE_EXT -p "$PROJ_C" up -d
|
||||
|
||||
# ── Wait for convergence ──────────────────────────────────────────────────
|
||||
|
||||
@@ -113,7 +169,7 @@ log "Waiting for link establishment (up to ${CONVERGE_TIMEOUT}s)..."
|
||||
converged=false
|
||||
for i in $(seq 1 "$CONVERGE_TIMEOUT"); do
|
||||
# node-b should have 2 links (A and C)
|
||||
link_count=$(docker exec sidecar-b-fips-1 fipsctl show links 2>/dev/null \
|
||||
link_count=$(docker exec "${PROJ_B}-fips-1" fipsctl show links 2>/dev/null \
|
||||
| grep -c '"state": "connected"' || true)
|
||||
if [ "$link_count" -ge 2 ]; then
|
||||
converged=true
|
||||
@@ -127,11 +183,11 @@ if [ "$converged" = true ]; then
|
||||
else
|
||||
log "TIMEOUT: links did not converge in ${CONVERGE_TIMEOUT}s"
|
||||
log "node-a links:"
|
||||
docker exec sidecar-a-fips-1 fipsctl show links 2>&1 || true
|
||||
docker exec "${PROJ_A}-fips-1" fipsctl show links 2>&1 || true
|
||||
log "node-b links:"
|
||||
docker exec sidecar-b-fips-1 fipsctl show links 2>&1 || true
|
||||
docker exec "${PROJ_B}-fips-1" fipsctl show links 2>&1 || true
|
||||
log "node-c links:"
|
||||
docker exec sidecar-c-fips-1 fipsctl show links 2>&1 || true
|
||||
docker exec "${PROJ_C}-fips-1" fipsctl show links 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -141,9 +197,9 @@ fi
|
||||
# chain. The directed pings below remain the actual assertions.
|
||||
_sidecar_converged() {
|
||||
PASSED=0; FAILED=0
|
||||
if docker exec sidecar-b-app-1 ping6 -c1 -W2 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
|
||||
if docker exec sidecar-c-app-1 ping6 -c1 -W2 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
|
||||
if docker exec sidecar-a-app-1 ping6 -c1 -W2 "${NODE_C_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
|
||||
if docker exec "${PROJ_B}-app-1" ping6 -c1 -W2 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
|
||||
if docker exec "${PROJ_C}-app-1" ping6 -c1 -W2 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
|
||||
if docker exec "${PROJ_A}-app-1" ping6 -c1 -W2 "${NODE_C_NPUB}.fips" >/dev/null 2>&1; then PASSED=$((PASSED+1)); else FAILED=$((FAILED+1)); fi
|
||||
}
|
||||
wait_until_connected _sidecar_converged "$CONVERGE_TIMEOUT" 10 || true
|
||||
|
||||
@@ -151,11 +207,11 @@ wait_until_connected _sidecar_converged "$CONVERGE_TIMEOUT" 10 || true
|
||||
|
||||
log "Verifying link counts..."
|
||||
|
||||
a_links=$(docker exec sidecar-a-fips-1 fipsctl show links 2>/dev/null \
|
||||
a_links=$(docker exec "${PROJ_A}-fips-1" fipsctl show links 2>/dev/null \
|
||||
| grep -c '"state": "connected"' || true)
|
||||
b_links=$(docker exec sidecar-b-fips-1 fipsctl show links 2>/dev/null \
|
||||
b_links=$(docker exec "${PROJ_B}-fips-1" fipsctl show links 2>/dev/null \
|
||||
| grep -c '"state": "connected"' || true)
|
||||
c_links=$(docker exec sidecar-c-fips-1 fipsctl show links 2>/dev/null \
|
||||
c_links=$(docker exec "${PROJ_C}-fips-1" fipsctl show links 2>/dev/null \
|
||||
| grep -c '"state": "connected"' || true)
|
||||
|
||||
[ "$a_links" -ge 1 ] && pass "node-a has $a_links link(s)" || fail "node-a has $a_links links (expected >= 1)"
|
||||
@@ -166,7 +222,7 @@ c_links=$(docker exec sidecar-c-fips-1 fipsctl show links 2>/dev/null \
|
||||
|
||||
log "Testing direct connectivity (B app → A via fips0)..."
|
||||
|
||||
if docker exec sidecar-b-app-1 ping6 -c2 -W5 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then
|
||||
if docker exec "${PROJ_B}-app-1" ping6 -c2 -W5 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then
|
||||
pass "node-b app can ping node-a via fips0"
|
||||
else
|
||||
fail "node-b app cannot ping node-a via fips0"
|
||||
@@ -176,7 +232,7 @@ fi
|
||||
|
||||
log "Testing multi-hop connectivity (C app → A via fips0, through B)..."
|
||||
|
||||
if docker exec sidecar-c-app-1 ping6 -c2 -W10 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then
|
||||
if docker exec "${PROJ_C}-app-1" ping6 -c2 -W10 "${NODE_A_NPUB}.fips" >/dev/null 2>&1; then
|
||||
pass "node-c app can ping node-a via fips0 (multi-hop through B)"
|
||||
else
|
||||
fail "node-c app cannot ping node-a via fips0 (multi-hop through B)"
|
||||
@@ -186,7 +242,7 @@ fi
|
||||
|
||||
log "Testing reverse multi-hop (A app → C via fips0, through B)..."
|
||||
|
||||
if docker exec sidecar-a-app-1 ping6 -c2 -W10 "${NODE_C_NPUB}.fips" >/dev/null 2>&1; then
|
||||
if docker exec "${PROJ_A}-app-1" ping6 -c2 -W10 "${NODE_C_NPUB}.fips" >/dev/null 2>&1; then
|
||||
pass "node-a app can ping node-c via fips0 (multi-hop through B)"
|
||||
else
|
||||
fail "node-a app cannot ping node-c via fips0 (multi-hop through B)"
|
||||
@@ -200,7 +256,14 @@ fi
|
||||
log "Verifying network isolation on app containers..."
|
||||
|
||||
for node in a b c; do
|
||||
container="sidecar-${node}-app-1"
|
||||
container="sidecar-${node}${FIPS_CI_NAME_SUFFIX:-}-app-1"
|
||||
# Fail loudly if the container is missing. Without this the two isolation
|
||||
# assertions below pass vacuously: they assert a ping FAILS, and a ping into
|
||||
# a non-existent container fails for the wrong reason. Only the loopback
|
||||
# check, which expects success, would notice — so a naming slip here would
|
||||
# silently turn the security assertions into no-ops.
|
||||
docker inspect "$container" >/dev/null 2>&1 \
|
||||
|| { fail "$container does not exist — isolation assertions cannot be trusted"; continue; }
|
||||
# Pick a peer IP that isn't this node's own address
|
||||
case $node in
|
||||
a) peer_ip="$NODE_B_IP" ;;
|
||||
@@ -210,7 +273,7 @@ for node in a b c; do
|
||||
log " Checking $container..."
|
||||
|
||||
# IPv4 gateway should be unreachable (iptables DROP on eth0)
|
||||
if docker exec "$container" ping -c1 -W2 172.20.2.1 >/dev/null 2>&1; then
|
||||
if docker exec "$container" ping -c1 -W2 "$GATEWAY_IP" >/dev/null 2>&1; then
|
||||
fail "$container can reach IPv4 gateway (isolation broken!)"
|
||||
else
|
||||
pass "$container cannot reach IPv4 gateway (IPv4 blocked)"
|
||||
@@ -240,7 +303,7 @@ if [ "$FAILED" -gt 0 ]; then
|
||||
log "Dumping logs for failed run..."
|
||||
for node in a b c; do
|
||||
echo "--- sidecar-${node} logs ---"
|
||||
docker logs "sidecar-${node}-fips-1" 2>&1 | tail -30
|
||||
docker logs "sidecar-${node}${FIPS_CI_NAME_SUFFIX:-}-fips-1" 2>&1 | tail -30
|
||||
echo ""
|
||||
done
|
||||
exit 1
|
||||
|
||||
@@ -41,7 +41,7 @@ services:
|
||||
node-a:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh"]
|
||||
container_name: fips-node-a
|
||||
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-a
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -53,7 +53,7 @@ services:
|
||||
node-b:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh"]
|
||||
container_name: fips-node-b
|
||||
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-b
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -65,7 +65,7 @@ services:
|
||||
node-c:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh"]
|
||||
container_name: fips-node-c
|
||||
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-c
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -77,7 +77,7 @@ services:
|
||||
node-d:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh"]
|
||||
container_name: fips-node-d
|
||||
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-d
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -89,7 +89,7 @@ services:
|
||||
node-e:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh"]
|
||||
container_name: fips-node-e
|
||||
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-e
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -102,7 +102,7 @@ services:
|
||||
pub-a:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh-public"]
|
||||
container_name: fips-node-a
|
||||
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-a
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -114,7 +114,7 @@ services:
|
||||
pub-b:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh-public"]
|
||||
container_name: fips-node-b
|
||||
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-b
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -126,7 +126,7 @@ services:
|
||||
pub-c:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh-public"]
|
||||
container_name: fips-node-c
|
||||
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-c
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -138,7 +138,7 @@ services:
|
||||
pub-d:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh-public"]
|
||||
container_name: fips-node-d
|
||||
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-d
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -150,7 +150,7 @@ services:
|
||||
pub-e:
|
||||
<<: *fips-common
|
||||
profiles: ["mesh-public"]
|
||||
container_name: fips-node-e
|
||||
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-e
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -163,7 +163,7 @@ services:
|
||||
chain-a:
|
||||
<<: *fips-common
|
||||
profiles: ["chain"]
|
||||
container_name: fips-node-a
|
||||
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-a
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -175,7 +175,7 @@ services:
|
||||
chain-b:
|
||||
<<: *fips-common
|
||||
profiles: ["chain"]
|
||||
container_name: fips-node-b
|
||||
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-b
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -187,7 +187,7 @@ services:
|
||||
chain-c:
|
||||
<<: *fips-common
|
||||
profiles: ["chain"]
|
||||
container_name: fips-node-c
|
||||
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-c
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -199,7 +199,7 @@ services:
|
||||
chain-d:
|
||||
<<: *fips-common
|
||||
profiles: ["chain"]
|
||||
container_name: fips-node-d
|
||||
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-d
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -211,7 +211,7 @@ services:
|
||||
chain-e:
|
||||
<<: *fips-common
|
||||
profiles: ["chain"]
|
||||
container_name: fips-node-e
|
||||
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-e
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -224,7 +224,7 @@ services:
|
||||
rekey-a:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey"]
|
||||
container_name: fips-node-a
|
||||
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-a
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug
|
||||
@@ -238,7 +238,7 @@ services:
|
||||
rekey-b:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey"]
|
||||
container_name: fips-node-b
|
||||
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-b
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug
|
||||
@@ -252,7 +252,7 @@ services:
|
||||
rekey-c:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey"]
|
||||
container_name: fips-node-c
|
||||
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-c
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug
|
||||
@@ -266,7 +266,7 @@ services:
|
||||
rekey-d:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey"]
|
||||
container_name: fips-node-d
|
||||
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-d
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug
|
||||
@@ -280,7 +280,7 @@ services:
|
||||
rekey-e:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey"]
|
||||
container_name: fips-node-e
|
||||
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-e
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug
|
||||
@@ -299,7 +299,7 @@ services:
|
||||
rekey-accept-off-a:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-accept-off"]
|
||||
container_name: fips-node-a
|
||||
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-a
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -313,7 +313,7 @@ services:
|
||||
rekey-accept-off-b:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-accept-off"]
|
||||
container_name: fips-node-b
|
||||
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-b
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -327,7 +327,7 @@ services:
|
||||
rekey-accept-off-c:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-accept-off"]
|
||||
container_name: fips-node-c
|
||||
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-c
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -341,7 +341,7 @@ services:
|
||||
rekey-accept-off-d:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-accept-off"]
|
||||
container_name: fips-node-d
|
||||
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-d
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -355,7 +355,7 @@ services:
|
||||
rekey-accept-off-e:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-accept-off"]
|
||||
container_name: fips-node-e
|
||||
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-e
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -379,7 +379,7 @@ services:
|
||||
rekey-outbound-only-a:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-outbound-only"]
|
||||
container_name: fips-node-a
|
||||
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-a
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -393,7 +393,7 @@ services:
|
||||
rekey-outbound-only-b:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-outbound-only"]
|
||||
container_name: fips-node-b
|
||||
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-b
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -407,7 +407,7 @@ services:
|
||||
rekey-outbound-only-c:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-outbound-only"]
|
||||
container_name: fips-node-c
|
||||
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-c
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -421,7 +421,7 @@ services:
|
||||
rekey-outbound-only-d:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-outbound-only"]
|
||||
container_name: fips-node-d
|
||||
container_name: fips-node-d${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-d
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -435,7 +435,7 @@ services:
|
||||
rekey-outbound-only-e:
|
||||
<<: *fips-common
|
||||
profiles: ["rekey-outbound-only"]
|
||||
container_name: fips-node-e
|
||||
container_name: fips-node-e${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-e
|
||||
environment:
|
||||
- RUST_LOG=info,fips::node::handlers::rekey=debug,fips::node::handlers::handshake=debug
|
||||
@@ -450,7 +450,7 @@ services:
|
||||
tcp-a:
|
||||
<<: *fips-common
|
||||
profiles: ["tcp-chain"]
|
||||
container_name: fips-node-a
|
||||
container_name: fips-node-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-a
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -462,7 +462,7 @@ services:
|
||||
tcp-b:
|
||||
<<: *fips-common
|
||||
profiles: ["tcp-chain"]
|
||||
container_name: fips-node-b
|
||||
container_name: fips-node-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-b
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -474,7 +474,7 @@ services:
|
||||
tcp-c:
|
||||
<<: *fips-common
|
||||
profiles: ["tcp-chain"]
|
||||
container_name: fips-node-c
|
||||
container_name: fips-node-c${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: node-c
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -487,7 +487,7 @@ services:
|
||||
gw-gateway:
|
||||
<<: *fips-common
|
||||
profiles: ["gateway"]
|
||||
container_name: fips-gw-gateway
|
||||
container_name: fips-gw-gateway${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: gw-gateway
|
||||
# Privileged required: gateway must enable IPv6 on eth1 (second network,
|
||||
# attached after container start) and manage nftables NAT rules.
|
||||
@@ -513,7 +513,7 @@ services:
|
||||
gw-server:
|
||||
<<: *fips-common
|
||||
profiles: ["gateway"]
|
||||
container_name: fips-gw-server
|
||||
container_name: fips-gw-server${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: gw-server
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -528,7 +528,7 @@ services:
|
||||
gw-server-2:
|
||||
<<: *fips-common
|
||||
profiles: ["gateway"]
|
||||
container_name: fips-gw-server-2
|
||||
container_name: fips-gw-server-2${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: gw-server-2
|
||||
volumes:
|
||||
- ../docker/resolv.conf:/etc/resolv.conf:ro
|
||||
@@ -540,7 +540,7 @@ services:
|
||||
gw-client:
|
||||
image: ${FIPS_TEST_APP_IMAGE:-fips-test-app:latest}
|
||||
profiles: ["gateway"]
|
||||
container_name: fips-gw-client
|
||||
container_name: fips-gw-client${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: gw-client
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
@@ -562,7 +562,7 @@ services:
|
||||
gw-client-2:
|
||||
image: ${FIPS_TEST_APP_IMAGE:-fips-test-app:latest}
|
||||
profiles: ["gateway"]
|
||||
container_name: fips-gw-client-2
|
||||
container_name: fips-gw-client-2${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: gw-client-2
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
|
||||
@@ -87,13 +87,23 @@ CAP_IP=$(node_ip "$CAP_NODE")
|
||||
[ -n "$CAP_IP" ] || fail "could not resolve docker_ip for node-$CAP_NODE in $TOPO_FILE"
|
||||
info "cap'd node: node-$CAP_NODE (ip $CAP_IP, max_peers=$MAX_PEERS)"
|
||||
|
||||
# Read the cap'd node's peer_count, or the empty string if it did not answer.
|
||||
#
|
||||
# Empty is deliberately distinct from a real 0. An `|| echo 0` fallback cannot
|
||||
# do that job here: `||` binds to the last stage of the pipeline, which succeeds
|
||||
# on empty input, so the fallback never fires and an unreachable container would
|
||||
# be reported as a legitimate zero.
|
||||
read_peer_count() {
|
||||
docker exec "fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show status 2>/dev/null \
|
||||
| grep -m1 peer_count | sed 's/.*: *//' | tr -d ','
|
||||
}
|
||||
|
||||
# ── Phase 1: wait for convergence ────────────────────────────────────
|
||||
info "phase 1: wait for node-$CAP_NODE peer_count to reach $MAX_PEERS (90s timeout)"
|
||||
deadline=$(($(date +%s) + 90))
|
||||
pc=0
|
||||
while [ "$(date +%s)" -lt "$deadline" ]; do
|
||||
pc=$(docker exec fips-node-$CAP_NODE fipsctl show status 2>/dev/null \
|
||||
| grep -m1 peer_count | sed 's/.*: *//' | tr -d ',' || echo 0)
|
||||
pc=$(read_peer_count)
|
||||
[ "$pc" = "$MAX_PEERS" ] && break
|
||||
sleep 2
|
||||
done
|
||||
@@ -102,7 +112,7 @@ done
|
||||
info "node-$CAP_NODE converged: peer_count=$pc"
|
||||
|
||||
# Identify admitted vs denied peers among configured peers
|
||||
ADMITTED_NPUBS=$(docker exec fips-node-$CAP_NODE fipsctl show peers 2>/dev/null \
|
||||
ADMITTED_NPUBS=$(docker exec "fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show peers 2>/dev/null \
|
||||
| grep -oE 'npub1[a-z0-9]+' | sort -u || true)
|
||||
DENIED=""
|
||||
ADMITTED=""
|
||||
@@ -130,8 +140,8 @@ info "denied (sustained-retry): ${DENIED:-<none>}"
|
||||
# with restarts every 15s we get ~30-50 firings across both denied peers.
|
||||
info "phase 2: capture UDP/2121 on node-$CAP_NODE for ${CAPTURE_SECS}s, with denied-peer restart loop"
|
||||
CAP_FILE=$(mktemp /tmp/admission-cap-pcap.XXXXXX.txt)
|
||||
HELPER_IMAGE=$(docker inspect -f '{{.Config.Image}}' fips-node-$CAP_NODE 2>/dev/null)
|
||||
[ -n "$HELPER_IMAGE" ] || fail "could not resolve helper image from fips-node-$CAP_NODE"
|
||||
HELPER_IMAGE=$(docker inspect -f '{{.Config.Image}}' "fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}" 2>/dev/null)
|
||||
[ -n "$HELPER_IMAGE" ] || fail "could not resolve helper image from fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}"
|
||||
|
||||
# Background: cycle denied peers to reset their backoff and drive load.
|
||||
(
|
||||
@@ -140,7 +150,7 @@ HELPER_IMAGE=$(docker inspect -f '{{.Config.Image}}' fips-node-$CAP_NODE 2>/dev/
|
||||
sleep 15
|
||||
elapsed=$((elapsed + 15))
|
||||
for n in $DENIED; do
|
||||
docker restart "fips-node-$n" >/dev/null 2>&1 &
|
||||
docker restart "fips-node-${n}${FIPS_CI_NAME_SUFFIX:-}" >/dev/null 2>&1 &
|
||||
done
|
||||
wait
|
||||
info " [load-driver] restarted denied peers ($DENIED) at t+${elapsed}s"
|
||||
@@ -149,7 +159,7 @@ HELPER_IMAGE=$(docker inspect -f '{{.Config.Image}}' fips-node-$CAP_NODE 2>/dev/
|
||||
LOAD_PID=$!
|
||||
|
||||
# Foreground: tcpdump capture for CAPTURE_SECS
|
||||
docker run --rm --label com.corganlabs.fips-ci=1 --net=container:fips-node-$CAP_NODE \
|
||||
docker run --rm --label com.corganlabs.fips-ci=1 --label "com.corganlabs.fips-ci.run=${FIPS_CI_RUN_ID:-manual}" --net=container:"fips-node-${CAP_NODE}${FIPS_CI_NAME_SUFFIX:-}" \
|
||||
--cap-add NET_ADMIN --cap-add NET_RAW \
|
||||
--entrypoint sh "$HELPER_IMAGE" \
|
||||
-c "timeout $CAPTURE_SECS tcpdump -nn -i any 'udp port 2121' -l 2>&1 || true" \
|
||||
@@ -186,9 +196,21 @@ for n in $DENIED; do
|
||||
done
|
||||
|
||||
# ── Phase 4: cap'd node still at exactly max_peers ───────────────────
|
||||
pc_final=$(docker exec fips-node-$CAP_NODE fipsctl show status 2>/dev/null \
|
||||
| grep -m1 peer_count | sed 's/.*: *//' | tr -d ',' || echo 0)
|
||||
info "node-$CAP_NODE final peer_count=$pc_final (expected $MAX_PEERS)"
|
||||
#
|
||||
# Polled rather than sampled once. The load driver above restarts the denied
|
||||
# peers every 15s and its final restart lands in the same second this runs, so a
|
||||
# single read can hit a daemon busy with those restarts and come back empty --
|
||||
# which is a harness race, not a cap regression. Retry briefly before believing
|
||||
# the answer. A genuine regression still fails, just after the retry window,
|
||||
# because the loop exits early only on the expected value.
|
||||
pc_final=""
|
||||
deadline=$(($(date +%s) + 30))
|
||||
while [ "$(date +%s)" -lt "$deadline" ]; do
|
||||
pc_final=$(read_peer_count)
|
||||
[ "$pc_final" = "$MAX_PEERS" ] && break
|
||||
sleep 2
|
||||
done
|
||||
info "node-$CAP_NODE final peer_count=${pc_final:-<no answer>} (expected $MAX_PEERS)"
|
||||
[ "$pc_final" = "$MAX_PEERS" ] || OVERALL=1
|
||||
|
||||
if [ "$OVERALL" -eq 0 ]; then
|
||||
|
||||
@@ -47,7 +47,7 @@ source "$ENV_FILE"
|
||||
iperf_once() {
|
||||
local client="$1" dest_npub="$2"
|
||||
local out
|
||||
if ! out=$(docker exec "fips-$client" iperf3 -c "${dest_npub}.fips" \
|
||||
if ! out=$(docker exec "fips-${client}${FIPS_CI_NAME_SUFFIX:-}" iperf3 -c "${dest_npub}.fips" \
|
||||
-t "$DURATION" -P "$PARALLEL" -J 2>&1); then
|
||||
echo FAIL
|
||||
return
|
||||
@@ -105,7 +105,7 @@ PYEOF
|
||||
ping_path() {
|
||||
local client="$1" dest_npub="$2"
|
||||
local out
|
||||
if ! out=$(docker exec "fips-$client" \
|
||||
if ! out=$(docker exec "fips-${client}${FIPS_CI_NAME_SUFFIX:-}" \
|
||||
ping -c "$PING_COUNT" -i 0.2 -w "$((PING_COUNT * 2))" \
|
||||
-q "${dest_npub}.fips" 2>&1); then
|
||||
echo FAIL
|
||||
@@ -203,7 +203,7 @@ CONVERGE_SECS="${FIPS_BENCH_CONVERGE_SECS:-15}"
|
||||
|
||||
peer_is_direct() {
|
||||
local client="$1" dest_npub="$2"
|
||||
docker exec "fips-$client" fipsctl show peers 2>/dev/null \
|
||||
docker exec "fips-${client}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show peers 2>/dev/null \
|
||||
| python3 -c '
|
||||
import json, sys
|
||||
target = sys.argv[1]
|
||||
@@ -225,7 +225,7 @@ except Exception:
|
||||
# multihop alternative).
|
||||
peer_bytes_sent_snapshot() {
|
||||
local client="$1"
|
||||
docker exec "fips-$client" fipsctl show peers 2>/dev/null \
|
||||
docker exec "fips-${client}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show peers 2>/dev/null \
|
||||
| python3 -c '
|
||||
import json, sys
|
||||
try:
|
||||
|
||||
@@ -20,11 +20,11 @@ source "$SCRIPT_DIR/../../lib/wait-converge.sh"
|
||||
GENERATED_DIR="$SCRIPT_DIR/../generated-configs"
|
||||
ENV_FILE="$GENERATED_DIR/npubs.env"
|
||||
|
||||
GATEWAY="fips-gw-gateway"
|
||||
SERVER="fips-gw-server"
|
||||
SERVER2="fips-gw-server-2"
|
||||
CLIENT="fips-gw-client"
|
||||
CLIENT2="fips-gw-client-2"
|
||||
GATEWAY="fips-gw-gateway${FIPS_CI_NAME_SUFFIX:-}"
|
||||
SERVER="fips-gw-server${FIPS_CI_NAME_SUFFIX:-}"
|
||||
SERVER2="fips-gw-server-2${FIPS_CI_NAME_SUFFIX:-}"
|
||||
CLIENT="fips-gw-client${FIPS_CI_NAME_SUFFIX:-}"
|
||||
CLIENT2="fips-gw-client-2${FIPS_CI_NAME_SUFFIX:-}"
|
||||
|
||||
# ── inject-config subcommand ─────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ iperf_test() {
|
||||
if [ "$LIVE_OUTPUT" = true ]; then
|
||||
# Show live output
|
||||
echo "Running iperf3 test (live output):"
|
||||
if docker exec "fips-$client_node" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL"; then
|
||||
if docker exec "fips-${client_node}${FIPS_CI_NAME_SUFFIX:-}" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL"; then
|
||||
PASSED=$((PASSED + 1))
|
||||
else
|
||||
echo "FAIL"
|
||||
@@ -59,7 +59,7 @@ iperf_test() {
|
||||
# Capture and summarize output
|
||||
echo -n "Running iperf3 test... "
|
||||
local output
|
||||
if output=$(docker exec "fips-$client_node" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL" 2>&1); then
|
||||
if output=$(docker exec "fips-${client_node}${FIPS_CI_NAME_SUFFIX:-}" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL" 2>&1); then
|
||||
# Check if we got valid results
|
||||
if echo "$output" | grep -q "sender"; then
|
||||
# Extract and display results (get SUM line for aggregate bandwidth)
|
||||
|
||||
@@ -180,7 +180,7 @@ do_apply() {
|
||||
echo ""
|
||||
|
||||
for node in $NODES; do
|
||||
local container="fips-node-$node"
|
||||
local container="fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}"
|
||||
echo -n " $container ... "
|
||||
if ! container_running "$container"; then
|
||||
echo "SKIP (not running)"
|
||||
@@ -199,7 +199,7 @@ do_remove() {
|
||||
echo ""
|
||||
|
||||
for node in $NODES; do
|
||||
local container="fips-node-$node"
|
||||
local container="fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}"
|
||||
echo -n " $container ... "
|
||||
if ! container_running "$container"; then
|
||||
echo "SKIP (not running)"
|
||||
@@ -219,7 +219,7 @@ do_status() {
|
||||
echo ""
|
||||
|
||||
for node in $NODES; do
|
||||
local container="fips-node-$node"
|
||||
local container="fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}"
|
||||
echo " $container:"
|
||||
if ! container_running "$container"; then
|
||||
echo " (not running)"
|
||||
|
||||
@@ -37,7 +37,7 @@ ping_test() {
|
||||
|
||||
echo -n " $label ... "
|
||||
local output
|
||||
if output=$(docker exec "fips-$from" ping6 -c "$COUNT" -W "$TIMEOUT" "${to_npub}.fips" 2>&1); then
|
||||
if output=$(docker exec "fips-${from}${FIPS_CI_NAME_SUFFIX:-}" ping6 -c "$COUNT" -W "$TIMEOUT" "${to_npub}.fips" 2>&1); then
|
||||
# Extract round-trip time from ping output
|
||||
local rtt
|
||||
rtt=$(echo "$output" | grep -oE 'time=[0-9.]+' | cut -d= -f2)
|
||||
@@ -61,7 +61,7 @@ ping_all_quiet() {
|
||||
for ((i=0; i<n; i++)); do
|
||||
for ((j=0; j<n; j++)); do
|
||||
[ "$i" -eq "$j" ] && continue
|
||||
if docker exec "fips-node-${LABELS[$i],,}" \
|
||||
if docker exec "fips-node-${LABELS[$i],,}${FIPS_CI_NAME_SUFFIX:-}" \
|
||||
ping6 -c 1 -W 1 "${NPUBS[$j]}.fips" >/dev/null 2>&1; then
|
||||
PASSED=$((PASSED + 1))
|
||||
else
|
||||
@@ -78,18 +78,18 @@ echo ""
|
||||
echo "Waiting for mesh convergence..."
|
||||
if [ "$PROFILE" = "chain" ]; then
|
||||
# Chain: A-B-C-D-E, each interior node has 2 peers, endpoints have 1
|
||||
wait_for_peers fips-node-a 1 20 || true
|
||||
wait_for_peers fips-node-b 2 20 || true
|
||||
wait_for_peers fips-node-c 2 20 || true
|
||||
wait_for_peers fips-node-d 2 20 || true
|
||||
wait_for_peers fips-node-e 1 20 || true
|
||||
wait_for_peers fips-node-a${FIPS_CI_NAME_SUFFIX:-} 1 20 || true
|
||||
wait_for_peers fips-node-b${FIPS_CI_NAME_SUFFIX:-} 2 20 || true
|
||||
wait_for_peers fips-node-c${FIPS_CI_NAME_SUFFIX:-} 2 20 || true
|
||||
wait_for_peers fips-node-d${FIPS_CI_NAME_SUFFIX:-} 2 20 || true
|
||||
wait_for_peers fips-node-e${FIPS_CI_NAME_SUFFIX:-} 1 20 || true
|
||||
elif [ "$PROFILE" = "mesh" ] || [ "$PROFILE" = "mesh-public" ]; then
|
||||
# Mesh: check all nodes reach their configured peer counts
|
||||
wait_for_peers fips-node-a 2 20 || true
|
||||
wait_for_peers fips-node-b 1 20 || true
|
||||
wait_for_peers fips-node-c 3 20 || true
|
||||
wait_for_peers fips-node-d 3 20 || true
|
||||
wait_for_peers fips-node-e 3 20 || true
|
||||
wait_for_peers fips-node-a${FIPS_CI_NAME_SUFFIX:-} 2 20 || true
|
||||
wait_for_peers fips-node-b${FIPS_CI_NAME_SUFFIX:-} 1 20 || true
|
||||
wait_for_peers fips-node-c${FIPS_CI_NAME_SUFFIX:-} 3 20 || true
|
||||
wait_for_peers fips-node-d${FIPS_CI_NAME_SUFFIX:-} 3 20 || true
|
||||
wait_for_peers fips-node-e${FIPS_CI_NAME_SUFFIX:-} 3 20 || true
|
||||
fi
|
||||
# Wait for full pairwise connectivity, progress-aware: the actual pings
|
||||
# are the convergence signal, the deadline extends while more pairs come
|
||||
|
||||
@@ -223,7 +223,7 @@ ping_one() {
|
||||
if (( attempt > 1 )); then
|
||||
sleep "$PING_RETRY_DELAY"
|
||||
fi
|
||||
if output=$(docker exec "fips-$from" ping6 -c 1 -W "$ping_timeout" "${to_npub}.fips" 2>&1); then
|
||||
if output=$(docker exec "fips-${from}${FIPS_CI_NAME_SUFFIX:-}" ping6 -c 1 -W "$ping_timeout" "${to_npub}.fips" 2>&1); then
|
||||
rtt=$(echo "$output" | grep -oE 'time=[0-9.]+' | cut -d= -f2)
|
||||
if [ -z "$quiet" ]; then
|
||||
if (( attempt == 1 )); then
|
||||
@@ -289,7 +289,7 @@ count_log_pattern() {
|
||||
local pattern="$1"
|
||||
local total=0
|
||||
for node in $NODES; do
|
||||
local count=$(docker logs "fips-node-$node" 2>&1 | grep -c "$pattern" || true)
|
||||
local count=$(docker logs "fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 | grep -c "$pattern" || true)
|
||||
total=$((total + count))
|
||||
done
|
||||
echo "$total"
|
||||
@@ -352,7 +352,7 @@ dump_peer_connectivity() {
|
||||
echo "=== Peer connectivity snapshot ==="
|
||||
for node in $NODES; do
|
||||
echo "--- node-$node ---"
|
||||
docker exec "fips-node-$node" fipsctl show peers 2>/dev/null || true
|
||||
docker exec "fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}" fipsctl show peers 2>/dev/null || true
|
||||
echo ""
|
||||
done
|
||||
}
|
||||
@@ -415,17 +415,38 @@ ping_all "" "$TIMEOUT" "$MAX_PING_ATTEMPTS"
|
||||
phase_result "Post-first-rekey (all 20 pairs)"
|
||||
echo ""
|
||||
|
||||
# ── Phase 4: Wait for second rekey cycle ──────────────────────────────
|
||||
# Poll for the next FMP rekey cutover instead of blind-sleeping: capture
|
||||
# the cutover count reached so far, then wait until at least one more
|
||||
# cutover lands — that increment is the second rekey cycle firing.
|
||||
# ── Phase 4: Wait for further rekey cutovers ──────────────────────────
|
||||
# Poll for the FMP rekey cutovers instead of blind-sleeping: capture the
|
||||
# cutover count reached so far, then wait until two more land AND the
|
||||
# Phase 6 absolute threshold (>= 4) is covered. The relative "+2" alone
|
||||
# is not enough: it inherits whatever count Phase 4 starts from, and on
|
||||
# an unloaded host the ping phases can outrun the jittered first rekey
|
||||
# cycle, entering here with a single cutover on the books — then "+2"
|
||||
# releases at three, the remaining first-cycle cutovers land moments
|
||||
# after the Phase 6 snapshot, and the >= 4 assertion fails even though
|
||||
# every cutover completed on schedule. Flooring the wait target at the
|
||||
# Phase 6 threshold makes this wait's guarantee cover that assertion:
|
||||
# the log count is monotone over a container's lifetime, so a released
|
||||
# wait means the later count cannot lose the timing race.
|
||||
#
|
||||
# What the count measures: the counted line is emitted by the cadence
|
||||
# path on whichever side(s) flip first, so a completed link rekey yields
|
||||
# one or two lines (a side promoted by receiving a flipped-K frame logs
|
||||
# only a DEBUG on a target the test's log filter suppresses). With rekey
|
||||
# timers resetting at each cutover, the events landing inside this
|
||||
# test's window are first-cycle cutovers spread across the topology's
|
||||
# links, not a second cycle on one link.
|
||||
# Bounded by SECOND_REKEY_WAIT so a stalled rekey still falls through to
|
||||
# the strict Phase 5/6 assertions rather than hanging.
|
||||
echo "Phase 4: Second rekey cycle (waiting up to ${SECOND_REKEY_WAIT}s for the next cutover)"
|
||||
echo "Phase 4: Further rekey cutovers (waiting up to ${SECOND_REKEY_WAIT}s)"
|
||||
fmp_cutovers_before=$(count_log_pattern "Rekey cutover complete (initiator), K-bit flipped")
|
||||
fmp_cutover_target=$((fmp_cutovers_before + 2))
|
||||
if [ "$fmp_cutover_target" -lt 4 ]; then
|
||||
fmp_cutover_target=4
|
||||
fi
|
||||
wait_for_log_pattern_count \
|
||||
"Rekey cutover complete (initiator), K-bit flipped" \
|
||||
"$((fmp_cutovers_before + 1))" "$SECOND_REKEY_WAIT" || true
|
||||
"$fmp_cutover_target" "$SECOND_REKEY_WAIT" || true
|
||||
|
||||
# Verify connectivity after second rekey (back-to-back). This is the
|
||||
# site of the recurring post-second-rekey straggler-pair flake: wait for
|
||||
@@ -457,7 +478,7 @@ wait_for_log_pattern_count "Peer FSP new-epoch frame authenticated" 1 "$REKEY_SE
|
||||
|
||||
# Positive checks: rekey machinery worked
|
||||
assert_min_count "Rekey cutover complete (initiator), K-bit flipped" 4 \
|
||||
"FMP rekey initiator cutovers (>= 2 cycles)"
|
||||
"FMP rekey initiator cutovers"
|
||||
|
||||
# FSP rekey checks (sessions between non-adjacent nodes)
|
||||
assert_min_count "FSP rekey cutover complete (initiator)" 1 \
|
||||
@@ -483,7 +504,7 @@ assert_zero_count "Session AEAD decryption failed" \
|
||||
if [ -n "$REKEY_ACCEPT_OFF_NODES" ]; then
|
||||
DUAL_INIT_THRESHOLD=10
|
||||
for off_node in ${REKEY_ACCEPT_OFF_NODES//,/ }; do
|
||||
count=$(docker logs "fips-node-$off_node" 2>&1 \
|
||||
count=$(docker logs "fips-node-${off_node}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 \
|
||||
| grep -cE "Dual rekey initiation: we win" || true)
|
||||
if [ "${count:-0}" -le "$DUAL_INIT_THRESHOLD" ]; then
|
||||
echo " PASS: node-$off_node dual-init drops below threshold ($count <= $DUAL_INIT_THRESHOLD)"
|
||||
@@ -506,7 +527,7 @@ fi
|
||||
if [ -n "$REKEY_OUTBOUND_ONLY_NODES" ]; then
|
||||
DUAL_INIT_THRESHOLD=10
|
||||
for n in $NODES; do
|
||||
count=$(docker logs "fips-node-$n" 2>&1 \
|
||||
count=$(docker logs "fips-node-${n}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 \
|
||||
| grep -cE "Dual rekey initiation: we win" || true)
|
||||
if [ "${count:-0}" -le "$DUAL_INIT_THRESHOLD" ]; then
|
||||
echo " PASS: node-$n dual-init drops below threshold ($count <= $DUAL_INIT_THRESHOLD)"
|
||||
@@ -541,7 +562,7 @@ else
|
||||
echo "=== Node logs (rekey-related, head -200) ==="
|
||||
for node in $NODES; do
|
||||
echo "--- node-$node ---"
|
||||
docker logs "fips-node-$node" 2>&1 | \
|
||||
docker logs "fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 | \
|
||||
grep -E "(rekey|Rekey|cross|Cross|teardown|ERROR|PANIC|K-bit|no route|next hop|TTL exhausted|MTU exceeded|Congestion|decrypt|Decrypt|AEAD|Notify|drain|Drain|promot)" | \
|
||||
head -200
|
||||
echo ""
|
||||
@@ -550,7 +571,7 @@ else
|
||||
echo "=== Node logs (last 80 lines, unfiltered) ==="
|
||||
for node in $NODES; do
|
||||
echo "--- node-$node ---"
|
||||
docker logs "fips-node-$node" 2>&1 | tail -80
|
||||
docker logs "fips-node-${node}${FIPS_CI_NAME_SUFFIX:-}" 2>&1 | tail -80
|
||||
echo ""
|
||||
done
|
||||
exit 1
|
||||
|
||||
@@ -24,7 +24,7 @@ services:
|
||||
sysctls:
|
||||
- net.ipv6.conf.all.disable_ipv6=0
|
||||
restart: "no"
|
||||
container_name: fips-dir-a
|
||||
container_name: fips-dir-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-dir-a
|
||||
volumes:
|
||||
- ./configs/node-a.yaml:/etc/fips/fips.yaml:ro
|
||||
@@ -44,7 +44,7 @@ services:
|
||||
sysctls:
|
||||
- net.ipv6.conf.all.disable_ipv6=0
|
||||
restart: "no"
|
||||
container_name: fips-dir-b
|
||||
container_name: fips-dir-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-dir-b
|
||||
volumes:
|
||||
- ./configs/node-b.yaml:/etc/fips/fips.yaml:ro
|
||||
|
||||
@@ -95,7 +95,7 @@ elapsed=0
|
||||
while [ "$elapsed" -lt "$MAX_WAIT_ONION" ]; do
|
||||
# Extract .onion address from structured log: onion_address=<addr>.onion
|
||||
# Strip ANSI color codes before matching (tracing emits them by default)
|
||||
ONION_ADDR=$(docker logs fips-dir-a 2>&1 \
|
||||
ONION_ADDR=$(docker logs fips-dir-a${FIPS_CI_NAME_SUFFIX:-} 2>&1 \
|
||||
| sed 's/\x1b\[[0-9;]*m//g' \
|
||||
| grep -oE 'onion_address=[a-z2-7]{56}\.onion' \
|
||||
| head -1 \
|
||||
@@ -115,7 +115,7 @@ if [ -z "$ONION_ADDR" ]; then
|
||||
echo " FAIL: Onion service not created within ${MAX_WAIT_ONION}s"
|
||||
echo ""
|
||||
echo "Node A logs (last 30 lines):"
|
||||
docker logs fips-dir-a 2>&1 | tail -30
|
||||
docker logs fips-dir-a${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
|
||||
docker compose down
|
||||
exit 1
|
||||
fi
|
||||
@@ -143,8 +143,8 @@ peers_a=0
|
||||
peers_b=0
|
||||
elapsed=0
|
||||
while [ "$elapsed" -lt "$MAX_WAIT_PEER" ]; do
|
||||
peers_a=$(count_connected_peers fips-dir-a)
|
||||
peers_b=$(count_connected_peers fips-dir-b)
|
||||
peers_a=$(count_connected_peers fips-dir-a${FIPS_CI_NAME_SUFFIX:-})
|
||||
peers_b=$(count_connected_peers fips-dir-b${FIPS_CI_NAME_SUFFIX:-})
|
||||
|
||||
if [ "$peers_a" -ge 1 ] && [ "$peers_b" -ge 1 ]; then
|
||||
echo " Both nodes connected after ${elapsed}s (A: ${peers_a}, B: ${peers_b})"
|
||||
@@ -159,10 +159,10 @@ if [ "$peers_a" -lt 1 ] || [ "$peers_b" -lt 1 ]; then
|
||||
echo " FAIL: Peers not established within ${MAX_WAIT_PEER}s"
|
||||
echo ""
|
||||
echo "Node A logs (last 30 lines):"
|
||||
docker logs fips-dir-a 2>&1 | tail -30
|
||||
docker logs fips-dir-a${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
|
||||
echo ""
|
||||
echo "Node B logs (last 30 lines):"
|
||||
docker logs fips-dir-b 2>&1 | tail -30
|
||||
docker logs fips-dir-b${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
|
||||
docker compose down
|
||||
exit 1
|
||||
fi
|
||||
@@ -229,15 +229,15 @@ print(f'{sum(trimmed)/len(trimmed):.1f}')
|
||||
|
||||
echo ""
|
||||
echo " Ping via Tor onion service (directory mode, Sandbox 1):"
|
||||
ping_series fips-dir-a "$NPUB_B" "A → B"
|
||||
ping_series fips-dir-b "$NPUB_A" "B → A"
|
||||
ping_series fips-dir-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B" "A → B"
|
||||
ping_series fips-dir-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A" "B → A"
|
||||
|
||||
echo ""
|
||||
|
||||
# ── Phase 6: Log analysis ────────────────────────────────────────
|
||||
echo "Phase 6: Log analysis"
|
||||
|
||||
for node in fips-dir-a fips-dir-b; do
|
||||
for node in fips-dir-a${FIPS_CI_NAME_SUFFIX:-} fips-dir-b${FIPS_CI_NAME_SUFFIX:-}; do
|
||||
panics=$(docker logs "$node" 2>&1 | grep -ci "panic" || true)
|
||||
errors=$(docker logs "$node" 2>&1 | grep -ci "error" || true)
|
||||
onion=$(docker logs "$node" 2>&1 | grep -ci "onion" || true)
|
||||
|
||||
@@ -16,7 +16,7 @@ networks:
|
||||
services:
|
||||
tor-daemon:
|
||||
image: osminogin/tor-simple:latest
|
||||
container_name: tor-daemon
|
||||
container_name: tor-daemon${FIPS_CI_NAME_SUFFIX:-}
|
||||
restart: "no"
|
||||
volumes:
|
||||
- ../common/torrc:/etc/tor/torrc:ro
|
||||
@@ -32,7 +32,7 @@ services:
|
||||
sysctls:
|
||||
- net.ipv6.conf.all.disable_ipv6=0
|
||||
restart: "no"
|
||||
container_name: fips-tor-a
|
||||
container_name: fips-tor-a${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-tor-a
|
||||
depends_on:
|
||||
- tor-daemon
|
||||
@@ -53,7 +53,7 @@ services:
|
||||
sysctls:
|
||||
- net.ipv6.conf.all.disable_ipv6=0
|
||||
restart: "no"
|
||||
container_name: fips-tor-b
|
||||
container_name: fips-tor-b${FIPS_CI_NAME_SUFFIX:-}
|
||||
hostname: fips-tor-b
|
||||
depends_on:
|
||||
- tor-daemon
|
||||
|
||||
@@ -87,7 +87,7 @@ echo ""
|
||||
echo "Phase 2: Waiting for Tor daemon to bootstrap (up to ${MAX_WAIT_TOR}s)..."
|
||||
elapsed=0
|
||||
while [ "$elapsed" -lt "$MAX_WAIT_TOR" ]; do
|
||||
if docker logs tor-daemon 2>&1 | grep -q "Bootstrapped 100%"; then
|
||||
if docker logs tor-daemon${FIPS_CI_NAME_SUFFIX:-} 2>&1 | grep -q "Bootstrapped 100%"; then
|
||||
echo " Tor bootstrapped after ${elapsed}s"
|
||||
break
|
||||
fi
|
||||
@@ -100,7 +100,7 @@ if [ "$elapsed" -ge "$MAX_WAIT_TOR" ]; then
|
||||
echo " FAIL: Tor daemon did not bootstrap within ${MAX_WAIT_TOR}s"
|
||||
echo ""
|
||||
echo "Tor daemon logs:"
|
||||
docker logs tor-daemon 2>&1 | tail -20
|
||||
docker logs tor-daemon${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -20
|
||||
docker compose down
|
||||
exit 1
|
||||
fi
|
||||
@@ -114,8 +114,8 @@ peers_a=0
|
||||
peers_b=0
|
||||
elapsed=0
|
||||
while [ "$elapsed" -lt "$MAX_WAIT_PEER" ]; do
|
||||
peers_a=$(count_connected_peers fips-tor-a)
|
||||
peers_b=$(count_connected_peers fips-tor-b)
|
||||
peers_a=$(count_connected_peers fips-tor-a${FIPS_CI_NAME_SUFFIX:-})
|
||||
peers_b=$(count_connected_peers fips-tor-b${FIPS_CI_NAME_SUFFIX:-})
|
||||
|
||||
if [ "$peers_a" -ge 1 ] && [ "$peers_b" -ge 1 ]; then
|
||||
echo " Both nodes have connected peers after ${elapsed}s (A: ${peers_a}, B: ${peers_b})"
|
||||
@@ -130,10 +130,10 @@ if [ "$peers_a" -lt 1 ] || [ "$peers_b" -lt 1 ]; then
|
||||
echo " FAIL: Peers not established within ${MAX_WAIT_PEER}s"
|
||||
echo ""
|
||||
echo "Node A logs (last 30 lines):"
|
||||
docker logs fips-tor-a 2>&1 | tail -30
|
||||
docker logs fips-tor-a${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
|
||||
echo ""
|
||||
echo "Node B logs (last 30 lines):"
|
||||
docker logs fips-tor-b 2>&1 | tail -30
|
||||
docker logs fips-tor-b${FIPS_CI_NAME_SUFFIX:-} 2>&1 | tail -30
|
||||
docker compose down
|
||||
exit 1
|
||||
fi
|
||||
@@ -200,15 +200,15 @@ print(f'{sum(trimmed)/len(trimmed):.1f}')
|
||||
|
||||
echo ""
|
||||
echo " Ping via Tor (routed through test-us01):"
|
||||
ping_series fips-tor-a "$NPUB_B" "A → B"
|
||||
ping_series fips-tor-b "$NPUB_A" "B → A"
|
||||
ping_series fips-tor-a${FIPS_CI_NAME_SUFFIX:-} "$NPUB_B" "A → B"
|
||||
ping_series fips-tor-b${FIPS_CI_NAME_SUFFIX:-} "$NPUB_A" "B → A"
|
||||
|
||||
echo ""
|
||||
|
||||
# ── Phase 5: Log analysis ────────────────────────────────────────
|
||||
echo "Phase 5: Log analysis"
|
||||
|
||||
for node in fips-tor-a fips-tor-b; do
|
||||
for node in fips-tor-a${FIPS_CI_NAME_SUFFIX:-} fips-tor-b${FIPS_CI_NAME_SUFFIX:-}; do
|
||||
panics=$(docker logs "$node" 2>&1 | grep -ci "panic" || true)
|
||||
errors=$(docker logs "$node" 2>&1 | grep -ci "error" || true)
|
||||
socks5=$(docker logs "$node" 2>&1 | grep -ci "socks5\|socks" || true)
|
||||
|
||||
Reference in New Issue
Block a user