mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-22 07:48:26 +00:00
Single combined commit covering five interlocking pieces of test and CI work that landed during the v0.3.0-prep cycle. ## fips-gateway robustness - src/bin/fips-gateway.rs DNS upstream probe converted from a 3-second hard-fail to a bounded retry loop (5 attempts × 1s timeout, 1s sleep between attempts; ~10s worst case). Covers the cold-boot race where the daemon's TUN is up but the DNS responder at [::1]:5354 is still binding. Each failed attempt logs at INFO. In production the binary's retry is the live recovery mechanism; with retry it recovers silently instead of relying on Restart=on-failure (~5s blip + spurious ERROR per cycle). - packaging/debian/fips-gateway.service `ExecStartPre` now waits up to 30 seconds for the daemon's `fips0` TUN to appear before exec'ing the gateway binary. Eliminates the cold-boot race where the gateway exits with `fips0 interface not found` and recovers via `Restart=on-failure`, producing a 5-second blip and a spurious error log per restart cycle. - testing/docker/entrypoint.sh gateway-mode waits up to 30s for the daemon's DNS responder to bind [::1]:5354 (probes once per second with `dig @::1 -p 5354 ... test.fips`) before exec'ing fips-gateway. Belt-and-suspenders with the binary's own retry: in CI we want deterministic startup ordering. On timeout, fall through so the binary's probe reports the definitive error. ## Test infrastructure DNS bind migration to ::1 After session 359's daemon DNS-bind default flipped from `127.0.0.1` to `::1` (the production fix for ISSUE-2026-0002), the static-test infrastructure was carrying a stale workaround that overrode the default back to IPv4 loopback. The fips-gateway integration test exposed the divergence: the gateway probes its DNS upstream at `[::1]:5354` (production default) while the daemon was binding `127.0.0.1:5354` from the template override — IPv6-explicit sockets do not accept v4-mapped traffic, so the upstream probe exhausted retries and the gateway exited. - Drop the explicit `bind_addr: "127.0.0.1"` line from every test config that emits it: testing/static/configs/node.template.yaml, testing/chaos/configs/node.template.yaml, the sidecar heredoc in testing/docker/entrypoint.sh, testing/acl-allowlist/generate-configs.sh (six per-node blocks), testing/nat/scripts/generate-configs.sh, and the four tor templates under testing/tor/. Daemon picks up its production `::1` default. - Flip the dnsmasq forwarder for `.fips` in testing/docker/Dockerfile from `127.0.0.1#5354` to `::1#5354` so dnsmasq on the shared test image continues to reach the daemon. Template and Dockerfile must move together since most static suites resolve `<npub>.fips` via the test-image dnsmasq. ## rekey-accept-off integration variant + UDP unit test - New `rekey-accept-off` topology and docker-compose profile under testing/static/. 2-node variant where node-b runs with `udp.accept_connections: false`. Pins the regression class that ISSUE-2026-0004 fixed (cross-connection winner's rekey msg1 was being filtered by the accept_connections gate, breaking rekey). - testing/static/scripts/rekey-test.sh accepts REKEY_TOPOLOGY and REKEY_ACCEPT_OFF_NODES env vars; its inject-config subcommand applies the per-node `udp.accept_connections: false` edit, and the test asserts no sustained "Dual rekey initiation" log lines. - New UDP variant of `should_admit_msg1` admit-rekey unit test in src/node/tests/handshake.rs. ## ci-local.sh full integration coverage - New runner functions and dispatcher entries for `acl-allowlist`, `nat-cone` / `nat-symmetric` / `nat-lan`, `rekey-accept-off`, `dns-resolver`, `deb-install`. Each integrates with the existing summary tracking via `record`. - New `--with-tor` flag (off by default) gates `tor-socks5-outbound` and `tor-directory-mode` runners. Tor stays opt-in because both harnesses depend on the live Tor network and would introduce a flake source unrelated to the FIPS code. - New suite arrays (`ACL_SUITES`, `NAT_SUITES`, `DNS_RESOLVER_SUITES`, `DEB_INSTALL_SUITES`, `TOR_SUITES`) drive both the default sweep and `--list` output. - `run_suite` extended to accept the new suite names for `--only` invocations. ## GitHub CI matrix expansions - `gateway` matrix entry runs testing/static/scripts/gateway-test.sh against the existing docker-compose `gateway` profile. - `rekey-accept-off` matrix entry exercises the new topology with REKEY_ACCEPT_OFF_NODES=b. - `deb-install` matrix (debian12 + ubuntu24 + ubuntu26) runs testing/deb-install/test.sh with privileged systemd containers. ~5-7 min cold cache, ~2 min warm per distro. Self-contained: builds its own .deb in a Debian 12 cargo-deb builder image; does not depend on the build job's pre-built artifact. - `dns-resolver` matrix entry runs the full 13-scenario harness (per-distro systemd resolver-backend tests + real-fips end-to-end scenarios) in a single job. Pins the production DNS bind path that ISSUE-2026-0002 lived in. ~7-12 min warm, ~12-15 min cold. Verified locally: full `bash testing/ci-local.sh` sweep passes, including 5/5 deb-install distros and all 13 dns-resolver scenarios. Tor-inclusive sweep (`--with-tor`) verified in a follow-up run.
266 lines
8.5 KiB
Bash
266 lines
8.5 KiB
Bash
#!/bin/bash
|
||
# Unified entrypoint for FIPS test containers.
|
||
#
|
||
# Mode is selected via FIPS_TEST_MODE environment variable:
|
||
# default — dnsmasq + sshd + iperf3 + http server + fips
|
||
# chaos — above + TCP ECN + ethernet interface wait
|
||
# sidecar — generate config from env + iptables isolation + fips
|
||
# tor-socks5 — dnsmasq + sshd + fips (tor daemon is separate)
|
||
# tor-directory — dnsmasq + tor + wait for .onion hostname + fips
|
||
|
||
set -e
|
||
|
||
MODE="${FIPS_TEST_MODE:-default}"
|
||
CONFIG="/etc/fips/fips.yaml"
|
||
|
||
# ── Common: dnsmasq ──────────────────────────────────────────────────────
|
||
|
||
start_dnsmasq() {
|
||
dnsmasq
|
||
}
|
||
|
||
# ── Common: background services (sshd, iperf3, http) ────────────────────
|
||
|
||
start_services() {
|
||
/usr/sbin/sshd
|
||
iperf3 -s -D
|
||
python3 -m http.server 8000 -d /root -b :: &>/dev/null &
|
||
}
|
||
|
||
# ── Chaos: TCP ECN + ethernet wait ──────────────────────────────────────
|
||
|
||
enable_ecn() {
|
||
sysctl -w net.ipv4.tcp_ecn=1 >/dev/null 2>&1 || true
|
||
}
|
||
|
||
wait_for_ethernet() {
|
||
# If config references ethernet transports, wait for interfaces to appear.
|
||
# Veth pairs are created from the host after the container starts.
|
||
local eth_ifaces=""
|
||
if grep -q 'ethernet:' "$CONFIG" 2>/dev/null; then
|
||
eth_ifaces=$(grep '^\s*interface:' "$CONFIG" \
|
||
| sed 's/.*interface:\s*//' \
|
||
| tr -d ' ' || true)
|
||
fi
|
||
|
||
if [ -n "$eth_ifaces" ]; then
|
||
echo "Waiting for Ethernet interfaces: $eth_ifaces"
|
||
local deadline=$((SECONDS + 30))
|
||
local all_found=false
|
||
while [ $SECONDS -lt $deadline ]; do
|
||
all_found=true
|
||
for iface in $eth_ifaces; do
|
||
if [ ! -e "/sys/class/net/$iface" ]; then
|
||
all_found=false
|
||
break
|
||
fi
|
||
done
|
||
if $all_found; then
|
||
echo "All Ethernet interfaces ready"
|
||
break
|
||
fi
|
||
sleep 0.2
|
||
done
|
||
if ! $all_found; then
|
||
echo "WARNING: Timed out waiting for Ethernet interfaces"
|
||
fi
|
||
fi
|
||
}
|
||
|
||
# ── Sidecar: config generation + iptables isolation ─────────────────────
|
||
|
||
generate_sidecar_config() {
|
||
FIPS_NSEC="${FIPS_NSEC:?FIPS_NSEC is required}"
|
||
FIPS_UDP_BIND="${FIPS_UDP_BIND:-0.0.0.0:2121}"
|
||
FIPS_TUN_MTU="${FIPS_TUN_MTU:-1280}"
|
||
FIPS_PEER_TRANSPORT="${FIPS_PEER_TRANSPORT:-udp}"
|
||
|
||
mkdir -p /etc/fips
|
||
|
||
local peers_section=""
|
||
if [ -n "$FIPS_PEER_NPUB" ] && [ -n "$FIPS_PEER_ADDR" ]; then
|
||
FIPS_PEER_ALIAS="${FIPS_PEER_ALIAS:-peer}"
|
||
peers_section=" - npub: \"${FIPS_PEER_NPUB}\"
|
||
alias: \"${FIPS_PEER_ALIAS}\"
|
||
addresses:
|
||
- transport: ${FIPS_PEER_TRANSPORT}
|
||
addr: \"${FIPS_PEER_ADDR}\"
|
||
connect_policy: auto_connect"
|
||
fi
|
||
|
||
cat > "$CONFIG" <<EOF
|
||
node:
|
||
identity:
|
||
nsec: "${FIPS_NSEC}"
|
||
|
||
tun:
|
||
enabled: true
|
||
name: fips0
|
||
mtu: ${FIPS_TUN_MTU}
|
||
|
||
dns:
|
||
enabled: true
|
||
|
||
transports:
|
||
udp:
|
||
bind_addr: "${FIPS_UDP_BIND}"
|
||
mtu: 1472
|
||
tcp: {}
|
||
|
||
peers:
|
||
${peers_section:- []}
|
||
EOF
|
||
|
||
echo "Generated $CONFIG"
|
||
}
|
||
|
||
apply_iptables_isolation() {
|
||
# Only FIPS transport (UDP 2121, TCP 443) may use eth0.
|
||
# All other eth0 traffic is dropped. fips0 and loopback unrestricted.
|
||
iptables -A OUTPUT -o lo -j ACCEPT
|
||
iptables -A INPUT -i lo -j ACCEPT
|
||
iptables -A OUTPUT -o eth0 -p udp --dport 2121 -j ACCEPT
|
||
iptables -A OUTPUT -o eth0 -p udp --sport 2121 -j ACCEPT
|
||
iptables -A INPUT -i eth0 -p udp --dport 2121 -j ACCEPT
|
||
iptables -A INPUT -i eth0 -p udp --sport 2121 -j ACCEPT
|
||
iptables -A OUTPUT -o eth0 -p tcp --dport 443 -j ACCEPT
|
||
iptables -A INPUT -i eth0 -p tcp --sport 443 -j ACCEPT
|
||
iptables -A OUTPUT -o eth0 -j DROP
|
||
iptables -A INPUT -i eth0 -j DROP
|
||
|
||
ip6tables -A OUTPUT -o lo -j ACCEPT
|
||
ip6tables -A INPUT -i lo -j ACCEPT
|
||
ip6tables -A OUTPUT -o fips0 -j ACCEPT
|
||
ip6tables -A INPUT -i fips0 -j ACCEPT
|
||
ip6tables -A OUTPUT -o eth0 -j DROP
|
||
ip6tables -A INPUT -i eth0 -j DROP
|
||
|
||
echo "iptables isolation rules applied"
|
||
}
|
||
|
||
# ── Tor directory mode: start tor + wait for hostname ────────────────────
|
||
|
||
start_tor_directory() {
|
||
local hidden_service_dir="/var/lib/tor/fips_onion_service"
|
||
local is_directory=false
|
||
|
||
if grep -qE '^\s+mode:\s+"directory"' "$CONFIG" 2>/dev/null; then
|
||
is_directory=true
|
||
fi
|
||
|
||
if [ "$is_directory" = true ]; then
|
||
mkdir -p "$hidden_service_dir"
|
||
chmod 700 "$hidden_service_dir"
|
||
fi
|
||
|
||
echo "Starting Tor daemon..."
|
||
tor -f /etc/tor/torrc &
|
||
|
||
if [ "$is_directory" = true ]; then
|
||
local hostname_file="${hidden_service_dir}/hostname"
|
||
echo "Waiting for Tor to create ${hostname_file}..."
|
||
for i in $(seq 1 120); do
|
||
if [ -f "$hostname_file" ]; then
|
||
echo "Tor hostname file ready after ${i}s: $(cat "$hostname_file")"
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
if [ ! -f "$hostname_file" ]; then
|
||
echo "FATAL: Tor did not create hostname file within 120s"
|
||
exit 1
|
||
fi
|
||
fi
|
||
}
|
||
|
||
# ── Mode dispatch ────────────────────────────────────────────────────────
|
||
|
||
case "$MODE" in
|
||
default)
|
||
start_dnsmasq
|
||
start_services
|
||
exec fips --config "$CONFIG"
|
||
;;
|
||
chaos)
|
||
enable_ecn
|
||
start_dnsmasq
|
||
start_services
|
||
wait_for_ethernet
|
||
exec fips --config "$CONFIG"
|
||
;;
|
||
sidecar)
|
||
generate_sidecar_config
|
||
apply_iptables_isolation
|
||
start_dnsmasq
|
||
exec fips --config "$CONFIG"
|
||
;;
|
||
tor-socks5)
|
||
start_dnsmasq
|
||
/usr/sbin/sshd
|
||
exec fips --config "$CONFIG"
|
||
;;
|
||
tor-directory)
|
||
start_dnsmasq
|
||
start_tor_directory
|
||
echo "Starting FIPS daemon..."
|
||
exec fips --config "$CONFIG"
|
||
;;
|
||
gateway)
|
||
# No dnsmasq — gateway DNS replaces it on port 53
|
||
start_services
|
||
|
||
# Extract LAN interface from config (gateway.lan_interface)
|
||
LAN_IF=$(grep 'lan_interface:' "$CONFIG" | head -1 | sed 's/.*: *//' | tr -d '"' | tr -d "'")
|
||
LAN_IF="${LAN_IF:-eth0}"
|
||
|
||
# Wait for LAN interface (Docker attaches second network after start)
|
||
for i in $(seq 1 15); do
|
||
[ -e "/sys/class/net/$LAN_IF" ] && break
|
||
sleep 0.5
|
||
done
|
||
|
||
# Ensure IPv6 is enabled on the LAN interface (may inherit host default)
|
||
sysctl -w "net.ipv6.conf.${LAN_IF}.disable_ipv6=0" >/dev/null 2>&1 || true
|
||
sysctl -w net.ipv6.conf.all.forwarding=1 >/dev/null 2>&1 || true
|
||
sysctl -w net.ipv6.conf.all.proxy_ndp=1 >/dev/null 2>&1 || true
|
||
|
||
# Start fips in background (gateway needs fips0)
|
||
fips --config "$CONFIG" &
|
||
# Wait for fips0 TUN device
|
||
for i in $(seq 1 30); do
|
||
[ -e /sys/class/net/fips0 ] && break
|
||
sleep 1
|
||
done
|
||
if [ ! -e /sys/class/net/fips0 ]; then
|
||
echo "FATAL: fips0 did not appear within 30s"
|
||
exit 1
|
||
fi
|
||
|
||
# Wait for the daemon's DNS responder to bind [::1]:5354 before
|
||
# exec'ing fips-gateway. The gateway binary's startup probe is
|
||
# bounded (5 attempts × 1s with retry); this harness wait is the
|
||
# belt to that suspenders so we get deterministic CI behaviour
|
||
# on slow runners. Bounded to ~30 seconds; if the daemon
|
||
# really never binds DNS, the gateway's own probe will report
|
||
# the definitive error after this wait expires.
|
||
for i in $(seq 1 30); do
|
||
if dig @::1 -p 5354 +tries=1 +time=1 test.fips >/dev/null 2>&1; then
|
||
echo "Daemon DNS ready (waited ~${i}s)"
|
||
break
|
||
fi
|
||
if [ "$i" -eq 30 ]; then
|
||
echo "WARNING: daemon DNS did not respond within ~30s; proceeding with gateway startup"
|
||
fi
|
||
sleep 1
|
||
done
|
||
|
||
echo "fips0 ready, starting gateway"
|
||
exec fips-gateway --config "$CONFIG" --log-level debug
|
||
;;
|
||
*)
|
||
echo "Unknown FIPS_TEST_MODE: $MODE"
|
||
echo "Valid modes: default, chaos, sidecar, tor-socks5, tor-directory, gateway"
|
||
exit 1
|
||
;;
|
||
esac
|