mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-22 07:48:26 +00:00
Moves both AEAD layers (ChaCha20-Poly1305, one round per layer per packet) plus the sendmsg syscall off the rx_loop task onto a per-shard worker pool, adds per-peer connect(2)-ed UDP with SO_REUSEPORT, and uses Linux UDP GSO (sendmsg+UDP_SEGMENT — kernel splits one super-skb into N on-the-wire datagrams in a single TX-stack walk) when packets in a batch are uniform-size. Same kernel primitive WireGuard's in-kernel module and BoringTun use to hit 2.5–3.2 Gbps single-stream. Single TCP stream on a 5-node docker-bridge mesh, 5 x 15 s x P=1: A→D: 1379 → 2708 Mbps (1.96x, RTT +0.12 ms) A→E: 1394 → 2663 Mbps (1.91x, RTT +0.11 ms) E→A: 1406 → 2624 Mbps (1.87x, RTT +0.19 ms) Static-peer pairs only — every CoV under 3%, 0 outliers, 0% ICMP loss. The ~+100 µs RTT is the worker queue handoff cost; AEAD + sendmmsg now run on a separate core in exchange. What lands: - src/node/encrypt_worker.rs: std::thread + crossbeam_channel workers; hash-by-destination dispatch pins a TCP flow to one worker so wire ordering is preserved; per-worker sendmmsg(2) batching up to 32; Linux uses sendmsg(2)+UDP_SEGMENT when packets in a group are uniform-size. - src/node/decrypt_worker.rs: receive-side mirror. Each shard owns its session's recv cipher + replay window in a thread-local HashMap (no shared RwLock/Mutex). Sessions are handed off at promote_connection and re-registered on K-bit flip / rekey cutover. - src/node/handlers/session.rs try_send_session_data_pipelined: FSP+FMP both seal in-place in the worker on one wire-buffer alloc; no intermediate inner_plaintext / fsp_payload Vecs. - src/transport/udp/connected_peer.rs + peer_drain.rs: per-peer connect(2)-ed UDP socket with SO_REUSEPORT (set on the listen socket too — without that, EADDRINUSE on activation and every packet falls back to the wildcard path); the worker sends with msg_name=NULL and the kernel uses its cached 5-tuple. Tick- driven activation in handlers/connected_udp.rs, idempotent. - src/transport/udp/mod.rs: mem::replace the recvmmsg backing buffer instead of buf.to_vec() per packet — single pointer swap, no MTU-sized memcpy. - src/protocol/link.rs SessionDatagramRef: zero-copy borrowed view used by handle_session_datagram for the bulk local-delivery path; handle_session_payload takes the borrowed payload directly (no payload[35..].to_vec()). - src/transport/mod.rs TransportAddr::from_socket_addr: collapses the two-alloc from_string(addr.to_string()) pattern to one. - src/node/handlers/rx_loop.rs: decrypt-fallback drain promoted ahead of packet_rx in the select! (TCP ACK starvation fix); interleaved fallback drain every 32 packets inside the rx burst loop. - noise::Session: send_cipher_clone / recv_cipher_clone / recv_replay_snapshot_owned / take_send_counter / accept_replay so off-task workers can hold a cloned cipher + reserved counter while the dispatcher keeps replay/counter sequencing serial. CipherState::cipher_clone returns a refcount-bumped LessSafeKey. AsyncUdpSocket: AsRawFd so workers issue raw sendmmsg / sendmsg without going through the tokio reactor. - Worker pool sizing: both default to num_cpus, overridable via FIPS_ENCRYPT_WORKERS=N / FIPS_DECRYPT_WORKERS=N. Per-peer connected UDP can be disabled via FIPS_CONNECTED_UDP=0. - src/perf_profile.rs: optional per-stage timing reporter under FIPS_PERF=1 (or FIPS_PIPELINE_TRACE=1). Off by default; zero overhead when disabled. - All cfg(unix)-gated. Windows continues on the existing tokio- based send/recv. Decrypt worker session lifecycle: - Node::unregister_decrypt_worker_session mirrors the existing register helper. Wired at the two natural sites that already iterate peers_by_index: the rekey drain-completion block in handlers/rekey.rs (drops the worker entry for the old our_index once the drain window has expired and the cache_key is unreachable to any in-flight OLD-K packet), and remove_active_peer in handlers/dispatch.rs (drops the worker entry for each of the four index slots: current, rekey, pending, previous). Only our_index is normally registered; unregister_session is fire- and-forget for missing entries, so calling unconditionally on all four slots is correct and bounds the cleanup without per- slot accounting. Without these callers the per-worker sessions HashMap and the Node's decrypt_registered_sessions set would grow monotonically per rekey on long-lived peers. Testing: - testing/static/scripts/bench-multirun.sh: multi-run iperf3 + ping bench. N reruns (default 5), median / min / max / CoV % / per-run outlier flag, avg ping RTT, ICMP loss %, TCP retransmit total. Plain client→dest labels + topology header. Pre-bench peer-convergence check (FIPS_BENCH_CONVERGE_SECS, default 15); per-path route verification via stats.bytes_sent deltas — fails fast if traffic exits via a non-static-peer link. - testing/static/docker-compose.yml: passes FIPS_ENCRYPT_WORKERS / FIPS_DECRYPT_WORKERS / FIPS_PERF through to containers for A/B benchmarking without rebuilds. - testing/static/scripts/iperf-test.sh: same plain client→dest labels + topology header (was multihop/direct/N hop, which conflated topology distance with on-wire path). - .config/nextest.toml: synthetic UDP node tests serialized through a max-threads=1 test group. Localhost handshakes drop on shared CI runners under parallel load; one-at-a-time keeps assertions reliable. - src/node/tests/spanning_tree.rs: repair_missing_edge_handshakes — retries up to 5 times for synthetic edges whose msg1 was dropped, with a drain after each edge retry instead of after each attempt's full burst. - src/node/decrypt_worker.rs::tests: two unit tests asserting WorkerMsg::UnregisterSession removes the worker-thread session HashMap entry (handle_msg_unregister_session_removes_entry) and is a no-op for never-seen cache_keys (handle_msg_unregister_session_idempotent_on_unknown_key), which is the safety invariant the unconditional unregister calls at the four index slots in remove_active_peer rely on. - src/node/encrypt_worker.rs::unix_tests pipelined_send_wire_layout_roundtrips_canonical_decoders: mirrors the encoder geometry of try_send_session_data_pipelined (no coords, the common established-session path), runs the worker's real seal + send via flush_direct_batch_sync, and decodes the resulting wire packet using only canonical receive-side decoders (EncryptedHeader::parse, SessionDatagramRef::decode, FSP header parse, noise::open). Any divergence between the hand-rolled encoder offsets (fsp_aad_offset, fsp_plaintext_offset) and the decoders fails at one of the parse / open / decode steps before the inner-plaintext assertion fires. Complements the existing fsp_preseal_runs_before_outer_fmp_seal test which covers the seal-ordering invariant with synthetic headers but does not exercise the wire-layout invariant. CHANGELOG.md [Unreleased] # Changed entry added describing the worker-pool threading model, hash-by-destination dispatch, sendmmsg/UDP_GSO, per-peer connected UDP, the operator-facing env vars, and the bench numbers above. Cherry-picks from mmalmi/master (paths translated from crates/fips-core/src/ to src/): 9b7c723, 0deb5cb, 13f7339, e036c0e, 3740a68, 3792f83, 8510193, 4910b07, e53f545, e4e2896, 5fe4af5, 1d01ada, 8c37008, e12469e, 6eb2860. Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
134 lines
4.6 KiB
Bash
Executable File
134 lines
4.6 KiB
Bash
Executable File
#!/bin/bash
|
|
# End-to-end iperf3 bandwidth test between FIPS nodes via DNS resolution.
|
|
# Usage: ./iperf-test.sh [mesh|chain] [--live]
|
|
#
|
|
# Requires containers to be running:
|
|
# docker compose --profile mesh up -d
|
|
# ./scripts/iperf-test.sh mesh
|
|
# ./scripts/iperf-test.sh mesh --live # Show live iperf3 output
|
|
set -e
|
|
|
|
# Exit entire script on Ctrl+C
|
|
trap 'echo ""; echo "Test interrupted"; exit 130' INT
|
|
|
|
PROFILE="${1:-mesh}"
|
|
LIVE_OUTPUT=false
|
|
if [ "$2" = "--live" ] || [ "$1" = "--live" ]; then
|
|
LIVE_OUTPUT=true
|
|
[ "$1" = "--live" ] && PROFILE="mesh"
|
|
fi
|
|
|
|
DURATION="${DURATION:-10}"
|
|
PARALLEL="${PARALLEL:-8}"
|
|
SETTLE_SECONDS="${SETTLE_SECONDS:-3}"
|
|
IPERF_TIMEOUT="${IPERF_TIMEOUT:-$((DURATION + 30))}"
|
|
PASSED=0
|
|
FAILED=0
|
|
|
|
# Node identities (from generated env file)
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
ENV_FILE="$SCRIPT_DIR/../generated-configs/npubs.env"
|
|
if [ ! -f "$ENV_FILE" ]; then
|
|
echo "Error: $ENV_FILE not found. Run generate-configs.sh first." >&2
|
|
exit 1
|
|
fi
|
|
# shellcheck source=../generated-configs/npubs.env
|
|
source "$ENV_FILE"
|
|
|
|
iperf_test() {
|
|
local server_node="$1"
|
|
local client_node="$2"
|
|
local dest_npub="$3"
|
|
local label="$4"
|
|
|
|
echo ""
|
|
echo "=== $label ==="
|
|
|
|
# iperf3 server is already running in daemon mode in each container
|
|
|
|
if [ "$LIVE_OUTPUT" = true ]; then
|
|
# Show live output
|
|
echo "Running iperf3 test (live output):"
|
|
if docker exec "fips-$client_node" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL"; then
|
|
PASSED=$((PASSED + 1))
|
|
else
|
|
echo "FAIL"
|
|
FAILED=$((FAILED + 1))
|
|
fi
|
|
else
|
|
# Capture and summarize output
|
|
echo -n "Running iperf3 test... "
|
|
local output
|
|
if output=$(docker exec "fips-$client_node" timeout "$IPERF_TIMEOUT" iperf3 -c "${dest_npub}.fips" -t "$DURATION" -P "$PARALLEL" 2>&1); then
|
|
# Check if we got valid results
|
|
if echo "$output" | grep -q "sender"; then
|
|
# Extract and display results (get SUM line for aggregate bandwidth)
|
|
local bandwidth=$(echo "$output" | grep "\[SUM\].*sender" | tail -1 | awk '{for(i=1;i<=NF;i++) if($i ~ /bits\/sec/) {print $(i-1), $i; exit}}')
|
|
echo "OK"
|
|
echo "Bandwidth: $bandwidth"
|
|
PASSED=$((PASSED + 1))
|
|
else
|
|
echo "FAIL (no bandwidth data)"
|
|
echo "Output: $output"
|
|
FAILED=$((FAILED + 1))
|
|
fi
|
|
else
|
|
echo "FAIL"
|
|
echo "Error output:"
|
|
echo "$output" | head -10
|
|
FAILED=$((FAILED + 1))
|
|
fi
|
|
fi
|
|
}
|
|
|
|
echo "=== FIPS iperf3 Bandwidth Test ($PROFILE topology) ==="
|
|
echo ""
|
|
|
|
# Print topology so the operator can see the configured routing
|
|
# context. The bench measures iperf3 throughput between the named
|
|
# nodes — labels DO NOT encode hop count, since peer discovery
|
|
# converges every same-subnet pair onto a direct UDP path within a
|
|
# few ticks regardless of the static `peers:` list.
|
|
if [ "$PROFILE" = "mesh" ] || [ "$PROFILE" = "mesh-public" ]; then
|
|
echo "Topology (static \`peers:\` from mesh.yaml):"
|
|
echo " A peers with: D, E"
|
|
echo " B peers with: C"
|
|
echo " C peers with: B, D, E"
|
|
echo " D peers with: A, C, E"
|
|
echo " E peers with: A, C, D"
|
|
echo " (Same docker-bridge subnet — discovery converges every"
|
|
echo " pair onto a direct UDP path; static \`peers:\` only seeds"
|
|
echo " the initial mesh, not the steady-state routing.)"
|
|
elif [ "$PROFILE" = "chain" ]; then
|
|
echo "Topology (static chain): A — B — C — D — E"
|
|
echo " (Same docker-bridge subnet — see mesh note above.)"
|
|
fi
|
|
echo ""
|
|
|
|
# Wait for nodes to converge
|
|
echo "Waiting ${SETTLE_SECONDS}s for mesh convergence..."
|
|
sleep "$SETTLE_SECONDS"
|
|
|
|
if [ "$PROFILE" = "mesh" ] || [ "$PROFILE" = "mesh-public" ]; then
|
|
echo ""
|
|
echo "Testing mesh topology paths:"
|
|
iperf_test node-d node-a "$NPUB_D" "A→D"
|
|
iperf_test node-e node-a "$NPUB_E" "A→E"
|
|
iperf_test node-b node-a "$NPUB_B" "A→B"
|
|
iperf_test node-c node-a "$NPUB_C" "A→C"
|
|
iperf_test node-a node-e "$NPUB_A" "E→A"
|
|
|
|
elif [ "$PROFILE" = "chain" ]; then
|
|
echo ""
|
|
echo "Testing chain topology paths:"
|
|
iperf_test node-b node-a "$NPUB_B" "A→B"
|
|
iperf_test node-c node-a "$NPUB_C" "A→C"
|
|
iperf_test node-d node-a "$NPUB_D" "A→D"
|
|
iperf_test node-e node-a "$NPUB_E" "A→E"
|
|
iperf_test node-a node-e "$NPUB_A" "E→A"
|
|
fi
|
|
|
|
echo ""
|
|
echo "=== Results: $PASSED passed, $FAILED failed ==="
|
|
[ "$FAILED" -eq 0 ] && exit 0 || exit 1
|