From cd9289e0f30b90e67b96f934fde607a2933d9846 Mon Sep 17 00:00:00 2001 From: Johnathan Corgan Date: Fri, 27 Feb 2026 00:52:17 +0000 Subject: [PATCH] Update test harness docs and add transport_mix support Chaos harness enhancements: - transport_mix config: weighted random transport assignment for random topologies (erdos_renyi, random_geometric) with UDP/Ethernet/TCP - Replace LoRa with Bluetooth L2CAP in cost-based/mixed-tech scenarios using realistic netem values (15-40ms delay, 5-15ms jitter, 2-8% loss) - New churn-20-mixed scenario: 20-node Erdos-Renyi with 60% UDP, 20% Ethernet, 20% TCP, full netem/link-flap/churn/bandwidth config - Expanded chaos README with full scenario catalog in four categories Static harness: - Updated README with topology table and scenario count --- testing/README.md | 28 +++-- testing/chaos/README.md | 108 +++++++++++++++--- .../chaos/scenarios/bottleneck-parent.yaml | 29 ++--- testing/chaos/scenarios/churn-20-mixed.yaml | 70 ++++++++++++ testing/chaos/scenarios/cost-avoidance.yaml | 16 +-- testing/chaos/scenarios/cost-mixed-7node.yaml | 14 +-- testing/chaos/scenarios/cost-reeval.yaml | 8 +- testing/chaos/scenarios/depth-vs-cost.yaml | 28 +++-- testing/chaos/scenarios/mixed-technology.yaml | 28 ++--- testing/chaos/sim/scenario.py | 24 ++++ testing/chaos/sim/topology.py | 25 +++- testing/static/README.md | 37 ++++-- 12 files changed, 316 insertions(+), 99 deletions(-) create mode 100644 testing/chaos/scenarios/churn-20-mixed.yaml diff --git a/testing/README.md b/testing/README.md index 3ffe2fd..4a563b1 100644 --- a/testing/README.md +++ b/testing/README.md @@ -7,16 +7,26 @@ containers running the full protocol stack. ### [static/](static/) -- Static Docker Network -Fixed topologies (mesh, chain) with 5 nodes. Manual scripts for -building, config generation, connectivity tests (ping, iperf), and -network impairment (netem). Useful for deterministic debugging and -validating specific topology configurations. +Fixed topologies with manual scripts for building, config generation, +connectivity tests (ping, iperf), and network impairment (netem). +Useful for deterministic debugging and validating specific topology +configurations. + +| Topology | Nodes | Transport | Description | +| ----------- | ----- | --------- | -------------------------------- | +| mesh | 5 | UDP | Sparse mesh, 6 links, multi-hop | +| chain | 5 | UDP | Linear chain, max 4-hop paths | +| mesh-public | 5+1 | UDP | Mesh with external public node | +| tcp-chain | 3 | TCP | Linear chain over TCP (port 443) | ### [chaos/](chaos/) -- Stochastic Simulation -Automated randomized testing with configurable node counts, topology -algorithms (random geometric, Erdos-Renyi, chain), and fault +Automated network testing with configurable node counts, topology +algorithms (random geometric, Erdos-Renyi, chain, explicit), and fault injection (netem mutation, link flaps, traffic generation, node -churn). Scenarios are defined in YAML and executed via a Python -harness that manages the full lifecycle: topology generation, Docker -orchestration, fault scheduling, log collection, and analysis. +churn). 16 scenarios covering general stress testing, cost-based parent +selection, mixed link technologies (fiber/Bluetooth/WiFi), and +transport-specific validation (UDP, TCP, Ethernet). Scenarios are +defined in YAML and executed via a Python harness that manages the full +lifecycle: topology generation, Docker orchestration, fault scheduling, +log collection, and analysis. diff --git a/testing/chaos/README.md b/testing/chaos/README.md index 2a25bb6..3a65669 100644 --- a/testing/chaos/README.md +++ b/testing/chaos/README.md @@ -1,10 +1,12 @@ # Stochastic Network Simulation -Automated stochastic network testing for FIPS. Generates random +Automated network testing for FIPS. Generates random or explicit topologies, spins up Docker containers, and applies configurable stressors (network impairment, link flaps, traffic generation, node -churn) over a timed simulation run. Logs are collected and analyzed -automatically. +churn) over a timed simulation run. Scenarios cover general stress +testing, cost-based parent selection, mixed link technologies +(fiber/Bluetooth/WiFi), and transport-specific validation (UDP, TCP, +Ethernet). Logs are collected and analyzed automatically. ## Prerequisites @@ -21,6 +23,10 @@ automatically. ## Available Scenarios +### General stress tests + +Random topologies with increasing stressor intensity. + | Scenario | Nodes | Topology | Duration | Netem | Link Flaps | Traffic | Node Churn | Bandwidth | | -------- | ----- | ---------------- | -------- | ----- | ---------- | ------- | ---------- | --------- | | smoke-10 | 10 | random_geometric | 60s | -- | -- | -- | -- | -- | @@ -28,6 +34,77 @@ automatically. | churn-10 | 10 | random_geometric | 600s | yes | yes | yes | yes | -- | | churn-20 | 20 | erdos_renyi | 600s | yes | yes | yes | yes | yes | +- **smoke-10**: Baseline sanity check. No stressors, just verify tree convergence. +- **chaos-10**: Network degradation (5-50ms delay, 0-2% loss), link flaps (max 2 + down, 10-30s), and iperf traffic (max 3 concurrent). Netem mutates 30% of + links every 15-30s between normal and degraded policies. +- **churn-10**: Extended run with node churn (1 node down at a time, 30-90s). + Tests tree re-convergence after node departure/rejoin. +- **churn-20**: Aggressive scale test. Erdos-Renyi topology, up to 5 nodes down + simultaneously, bandwidth tiers (1/10/100/1000 Mbps), `protect_connectivity` + disabled (partitions allowed). + +### Cost-based parent selection + +Explicit topologies with heterogeneous link types (fiber, Bluetooth, WiFi) to +test that the spanning tree selects optimal parents based on link cost. + +| Scenario | Nodes | Shape | Link types | Duration | What it tests | +| ----------------- | ----- | --------------- | ------------------------ | -------- | ------------------------------------------------------------------- | +| cost-avoidance | 4 | Diamond | Fiber + Bluetooth | 120s | n04 picks fiber parent (n03) over Bluetooth parent (n02) | +| depth-vs-cost | 4 | Linear tree | Fiber + Bluetooth | 120s | Cost tradeoff: depth vs. Bluetooth link quality | +| bottleneck-parent | 10 | Tree with BT | Fiber + Bluetooth | 120s | n06 avoids Bluetooth bottleneck via n02, picks fiber via n03 | +| cost-mixed-7node | 7 | Multi-type tree | Fiber + Bluetooth + WiFi | 180s | n06 prefers fiber (n03) over WiFi (n04) | +| cost-reeval | 4 | Diamond | Fiber (mutated) | 180s | Periodic re-evaluation triggers parent switch (reeval_interval=15s) | +| cost-stability | 4 | Diamond | WiFi (all) | 180s | Hysteresis prevents flapping when costs vary within 20% band | + +- **cost-avoidance**, **depth-vs-cost**: Minimal scenarios validating the core + cost formula. Bluetooth (L2CAP) links use 15-40ms delay and 2-8% loss; + fiber uses 1-5ms delay and 0-1% loss. +- **bottleneck-parent**: Larger topology where some nodes have both fiber and + Bluetooth paths to choose from, and one node (n09) is stuck with Bluetooth + (no alternative). +- **cost-mixed-7node**: Three link technologies in one mesh. Traffic enabled. +- **cost-reeval**: Netem mutation (50% fraction, every 12-18s) degrades random + links. FIPS override sets `reeval_interval_secs=15` so periodic re-evaluation + catches cost asymmetry. Look for `trigger=periodic` in logs. +- **cost-stability**: All links are WiFi. Mutation swings costs between + `slightly_better` and `slightly_worse` — within the hysteresis band. Expect + ≤ 5 parent switches over 180s. + +### Mixed-technology + +Larger explicit topologies combining multiple link technologies. + +| Scenario | Nodes | Link types | Duration | Netem mutation | What it tests | +| ---------------- | ----- | ------------------------ | -------- | -------------- | ------------------------------------------------ | +| mixed-technology | 10 | Fiber + Bluetooth + WiFi | 180s | 20%/30-60s | Tree convergence across heterogeneous link types | + +### Transport-specific + +Explicit topologies exercising non-UDP transports. + +| Scenario | Nodes | Transport | Shape | Duration | Netem | Link Flaps | What it tests | +| ------------- | ----- | -------------- | ----- | -------- | ----- | ---------- | ------------------------------------------ | +| ethernet-only | 4 | Ethernet | Ring | 90s | yes | -- | AF_PACKET transport with beacon discovery | +| ethernet-mesh | 6 | UDP + Ethernet | Mesh | 120s | yes | yes | Mixed UDP/Ethernet, netem mutation + flaps | +| tcp-only | 4 | TCP | Ring | 90s | yes | -- | TCP transport with static peer config | +| tcp-chain | 4 | TCP | Chain | 90s | yes | -- | TCP multi-hop routing through chain | +| tcp-mesh | 6 | UDP + TCP | Mesh | 120s | yes | yes | Mixed UDP/TCP, netem mutation + flaps | + +- **ethernet-only**: 4-node ring on raw Ethernet (AF_PACKET). Peers discovered + via beacons, not static config. Minimal netem (1-5ms delay). +- **ethernet-mesh**: Mirrors `tcp-mesh` topology but with Ethernet instead of + TCP. UDP edges use static config; Ethernet edges use beacon discovery. +- **tcp-only**: 4-node ring using TCP on port 443. Tests connect-on-send, + FMP framing over TCP, and reconnection. Netem enabled (1-10ms delay, 0-1% + loss). +- **tcp-chain**: 4-node linear chain, all TCP. Tests multi-hop routing over + TCP-only mesh. +- **tcp-mesh**: 6-node mesh with 4 UDP and 3 TCP edges. Both transports use + static peer config. Netem mutation (30% fraction, every 20-40s) and link + flaps (1 link max, 10-20s down). + ## CLI Options | Option | Description | @@ -109,23 +186,26 @@ logging: ## Topology Algorithms -| Algorithm | Parameters | Description | -| ---------------- | ------------------- | --------------------------------------------------------- | -| random_geometric | radius (default 0.5)| Place nodes in unit square, connect pairs within radius | -| erdos_renyi | p (default 0.3) | Include each edge independently with probability p | -| chain | -- | Linear chain: n01--n02--...--nN | +| Algorithm | Parameters | Description | +| ---------------- | -------------------- | ------------------------------------------------------- | +| random_geometric | radius (default 0.5) | Place nodes in unit square, connect pairs within radius | +| erdos_renyi | p (default 0.3) | Include each edge independently with probability p | +| chain | -- | Linear chain: n01--n02--...--nN | +| explicit | adjacency list | Hardcoded edges with optional per-edge transport type | When `ensure_connected` is true (default), the generator retries up to 50 times to produce a connected graph. ### Directed Outbound Configs -The config generator assigns each edge to exactly one node for outbound -connection using a BFS spanning tree rooted at the lowest node ID. Tree -edges are assigned parent-to-child; non-tree edges are assigned from the -lower node ID to the higher. This eliminates the dual-connect race -condition where both sides initiate simultaneously, and creates a clear -"owning side" for each link — relevant for auto-reconnect testing. +The config generator assigns each static-config edge (UDP or TCP) to +exactly one node for outbound connection using a BFS spanning tree rooted +at the lowest node ID. Tree edges are assigned parent-to-child; non-tree +edges are assigned from the lower node ID to the higher. This eliminates +the dual-connect race condition where both sides initiate simultaneously, +and creates a clear "owning side" for each link — relevant for +auto-reconnect testing. Ethernet edges are excluded from static config +since they use beacon discovery. ## Output diff --git a/testing/chaos/scenarios/bottleneck-parent.yaml b/testing/chaos/scenarios/bottleneck-parent.yaml index c5f4c66..cb847ae 100644 --- a/testing/chaos/scenarios/bottleneck-parent.yaml +++ b/testing/chaos/scenarios/bottleneck-parent.yaml @@ -1,8 +1,9 @@ -# Bottleneck Parent: 10-node focused LoRa bottleneck test +# Bottleneck Parent: 10-node focused Bluetooth bottleneck test # -# Explicit topology with two LoRa links that create bottleneck parent -# candidates. Tests that nodes avoid choosing LoRa parents when fiber -# alternatives exist at the same or slightly greater depth. +# Explicit topology with two Bluetooth (L2CAP) links that create +# bottleneck parent candidates. Tests that nodes avoid choosing +# Bluetooth parents when fiber alternatives exist at the same or +# slightly greater depth. # # Topology: # @@ -12,7 +13,7 @@ # / | \ \ # n02 n03 n04 n05 # | / | \ | -# LoRa f f LoRa +# BT f f BT # | / | \ | # n06 n07 n08 n09 # | @@ -21,15 +22,15 @@ # n10 # # Edges and link types: -# Fiber: n01-n02, n01-n03, n01-n04, n01-n05, -# n03-n06, n03-n07, n04-n08, n08-n10 -# LoRa: n02-n06, n05-n09 +# Fiber: n01-n02, n01-n03, n01-n04, n01-n05, +# n03-n06, n03-n07, n04-n08, n08-n10 +# Bluetooth: n02-n06, n05-n09 # # Cross-links: n03-n08 (fiber) — gives n08 a fiber alternative to n04 # # Test subjects: -# - n06 has LoRa (n02, depth 1) and fiber (n03, depth 1) — should pick n03 -# - n09 has only LoRa (n05) — no alternative, stuck with LoRa parent +# - n06 has Bluetooth (n02) and fiber (n03) at depth 1 — should pick n03 +# - n09 has only Bluetooth (n05) — no alternative, stuck with BT parent scenario: name: "bottleneck-parent" @@ -62,12 +63,12 @@ netem: jitter_ms: [0, 1] loss_pct: [0, 0.5] link_policies: - # LoRa links (high delay, moderate loss) + # Bluetooth (L2CAP) links - edges: ["n02-n06", "n05-n09"] policy: - delay_ms: [400, 600] - jitter_ms: [50, 100] - loss_pct: [5, 15] + delay_ms: [15, 40] + jitter_ms: [5, 15] + loss_pct: [2, 8] mutation: interval_secs: {min: 30, max: 60} fraction: 0.2 diff --git a/testing/chaos/scenarios/churn-20-mixed.yaml b/testing/chaos/scenarios/churn-20-mixed.yaml new file mode 100644 index 0000000..013beba --- /dev/null +++ b/testing/chaos/scenarios/churn-20-mixed.yaml @@ -0,0 +1,70 @@ +# 20-node churn test with mixed transports (UDP, Ethernet, TCP) +# +# Based on churn-20 but with transport_mix assigning each edge a +# random transport type by weight. Tests spanning tree convergence, +# link flaps, node churn, and traffic across heterogeneous transports. + +scenario: + name: "churn-20-mixed" + seed: 42 + duration_secs: 600 + +topology: + num_nodes: 20 + algorithm: erdos_renyi + params: + p: 0.3 + ensure_connected: true + subnet: "172.20.0.0/23" + ip_start: 10 + transport_mix: + udp: 0.6 + ethernet: 0.2 + tcp: 0.2 + +netem: + enabled: true + default_policy: + delay_ms: { min: 5, max: 50 } + jitter_ms: { min: 1, max: 10 } + loss_pct: { min: 0, max: 2 } + mutation: + interval_secs: { min: 20, max: 45 } + fraction: 0.3 + policies: + normal: + delay_ms: [5, 20] + loss_pct: [0, 1] + degraded: + delay_ms: [50, 100] + jitter_ms: [10, 30] + loss_pct: [3, 8] + +link_flaps: + enabled: true + interval_secs: { min: 30, max: 60 } + max_down_links: 3 + down_duration_secs: { min: 10, max: 30 } + protect_connectivity: true + +traffic: + enabled: true + max_concurrent: 10 + interval_secs: { min: 0, max: 30 } + duration_secs: { min: 5, max: 90 } + parallel_streams: 4 + +node_churn: + enabled: true + interval_secs: { min: 60, max: 90 } + max_down_nodes: 5 + down_duration_secs: { min: 30, max: 90 } + protect_connectivity: false + +bandwidth: + enabled: true + tiers_mbps: [1, 10, 100, 1000] + +logging: + rust_log: "debug" + output_dir: "./sim-results" diff --git a/testing/chaos/scenarios/cost-avoidance.yaml b/testing/chaos/scenarios/cost-avoidance.yaml index 296a50d..ffe63c2 100644 --- a/testing/chaos/scenarios/cost-avoidance.yaml +++ b/testing/chaos/scenarios/cost-avoidance.yaml @@ -8,13 +8,13 @@ # / \ # n02 n03 # \ / -# LoRa fiber +# BT fiber # \ / # n04 (test subject) # -# n04 has two candidate parents at depth 1: n02 (via LoRa) and n03 (via -# fiber). Cost-based selection should pick n03 because: -# effective_depth(n02) = 1 + ~6.0 (LoRa) = ~7.0 +# n04 has two candidate parents at depth 1: n02 (via Bluetooth L2CAP) +# and n03 (via fiber). Cost-based selection should pick n03 because: +# effective_depth(n02) = 1 + ~1.4 (Bluetooth) = ~2.4 # effective_depth(n03) = 1 + ~1.01 (fiber) = ~2.01 # # Validation: tree snapshot shows n04's parent is n03. @@ -44,12 +44,12 @@ netem: jitter_ms: [0, 1] loss_pct: [0, 0.5] link_policies: - # LoRa link from n02 to n04 + # Bluetooth (L2CAP) link from n02 to n04 - edges: ["n02-n04"] policy: - delay_ms: [400, 600] - jitter_ms: [50, 100] - loss_pct: [5, 15] + delay_ms: [15, 40] + jitter_ms: [5, 15] + loss_pct: [2, 8] link_flaps: enabled: false diff --git a/testing/chaos/scenarios/cost-mixed-7node.yaml b/testing/chaos/scenarios/cost-mixed-7node.yaml index f0d377b..cf13c96 100644 --- a/testing/chaos/scenarios/cost-mixed-7node.yaml +++ b/testing/chaos/scenarios/cost-mixed-7node.yaml @@ -4,7 +4,7 @@ # # n01 (root) # / | \ -# fiber | fiber LoRa +# fiber | fiber BT # / | \ # n02 n03 n04 # | | \ | @@ -16,8 +16,8 @@ # # Test subjects: # - n06 has edges to n03 (fiber) and n04 (wifi) — should prefer n03 -# - n04's link to root is LoRa, so n04 has high-cost parent link -# - n07 connects only to n04 (no choice, stuck with LoRa upstream) +# - n04's link to root is Bluetooth (L2CAP), so n04 has higher-cost parent link +# - n07 connects only to n04 (no choice, stuck with Bluetooth upstream) # # Validation: tree snapshot shows n06's parent is n03 (not n04). @@ -50,12 +50,12 @@ netem: jitter_ms: [0, 1] loss_pct: [0, 0.5] link_policies: - # LoRa link from n01 to n04 + # Bluetooth (L2CAP) link from n01 to n04 - edges: ["n01-n04"] policy: - delay_ms: [400, 600] - jitter_ms: [50, 100] - loss_pct: [5, 15] + delay_ms: [15, 40] + jitter_ms: [5, 15] + loss_pct: [2, 8] # WiFi link from n04 to n06 - edges: ["n04-n06"] policy: diff --git a/testing/chaos/scenarios/cost-reeval.yaml b/testing/chaos/scenarios/cost-reeval.yaml index afa48af..559966d 100644 --- a/testing/chaos/scenarios/cost-reeval.yaml +++ b/testing/chaos/scenarios/cost-reeval.yaml @@ -17,7 +17,7 @@ # n02 (smaller NodeAddr, tiebreak rule). # # Mutation: Stochastic netem mutation with a single "degraded" policy -# (LoRa-like: 400-600ms delay, 5-15% loss). With fraction=0.5, on +# (Bluetooth-like: 15-40ms delay, 2-8% loss). With fraction=0.5, on # average 2 of 4 edges degrade each round. Over 12 mutation rounds # (180s / 15s interval), n02-n04 will be degraded in some rounds. # @@ -62,9 +62,9 @@ netem: fraction: 0.5 policies: degraded: - delay_ms: [400, 600] - jitter_ms: [50, 100] - loss_pct: [5, 15] + delay_ms: [15, 40] + jitter_ms: [5, 15] + loss_pct: [2, 8] link_flaps: enabled: false diff --git a/testing/chaos/scenarios/depth-vs-cost.yaml b/testing/chaos/scenarios/depth-vs-cost.yaml index 41c997e..5fb845e 100644 --- a/testing/chaos/scenarios/depth-vs-cost.yaml +++ b/testing/chaos/scenarios/depth-vs-cost.yaml @@ -1,10 +1,10 @@ -# Cost-Based Parent Selection: Deeper Fiber Beats Shallow LoRa +# Cost-Based Parent Selection: Deeper Fiber Beats Shallow Bluetooth # # Topology (explicit 4-node): # # n01 (root) # / \ -# fiber LoRa +# fiber BT # / \ # n02 n04 (test subject) # | / @@ -13,18 +13,16 @@ # n03 # # n04 has two candidate parents: -# - n01 (root, depth 0) via LoRa: effective_depth = 0 + ~6.0 = ~6.0 -# - n03 (depth 2) via fiber: effective_depth = 2 + ~1.01 = ~3.01 +# - n01 (root, depth 0) via Bluetooth L2CAP: effective_depth = 0 + ~1.4 = ~1.4 +# - n03 (depth 2) via fiber: effective_depth = 2 + ~1.01 = ~3.01 # # Without cost-based selection, n04 would pick n01 (depth 0 < depth 2). -# With cost-based selection, n04 should pick n03 (lower effective_depth -# despite being 2 levels deeper). +# With cost-based selection and these Bluetooth impairments, n04 may +# still pick n01 since the Bluetooth cost (~1.4) is modest. This tests +# that the cost formula correctly weighs depth against link quality. # -# This tests the core depth-vs-cost tradeoff: a much deeper fiber path -# should beat a direct LoRa link to root when the cost difference is -# large enough. -# -# Validation: tree snapshot shows n04's parent is n03, n04's depth is 3. +# Validation: tree snapshot shows n04's parent selection reflects the +# actual cost tradeoff between depth and link quality. scenario: name: "depth-vs-cost" @@ -51,12 +49,12 @@ netem: jitter_ms: [0, 1] loss_pct: [0, 0.5] link_policies: - # LoRa link from n01 to n04 + # Bluetooth (L2CAP) link from n01 to n04 - edges: ["n01-n04"] policy: - delay_ms: [400, 600] - jitter_ms: [50, 100] - loss_pct: [5, 15] + delay_ms: [15, 40] + jitter_ms: [5, 15] + loss_pct: [2, 8] link_flaps: enabled: false diff --git a/testing/chaos/scenarios/mixed-technology.yaml b/testing/chaos/scenarios/mixed-technology.yaml index 8d18942..2da4c29 100644 --- a/testing/chaos/scenarios/mixed-technology.yaml +++ b/testing/chaos/scenarios/mixed-technology.yaml @@ -1,8 +1,8 @@ # Mixed Technology: 10-node heterogeneous network # -# Explicit topology with LoRa, WiFi, and fiber links. Tests that -# cost-based parent selection produces a tree favoring low-cost paths -# when multiple link technologies coexist. +# Explicit topology with Bluetooth (L2CAP), WiFi, and fiber links. +# Tests that cost-based parent selection produces a tree favoring +# low-cost paths when multiple link technologies coexist. # # Topology: # @@ -12,7 +12,7 @@ # / | \ # n02 n03 n04 # | \ | \ | \ -# f LoRa f f LoRa f +# f BT f f BT f # | \ | | \ | # n05 n06 n07 n08 n09 # \ / @@ -20,14 +20,14 @@ # n10 # # Edges and link types: -# Fiber: n01-n02, n01-n03, n01-n04, n02-n05, n03-n07, n04-n09 -# LoRa: n02-n06, n04-n08 -# WiFi: n03-n06, n08-n10 -# Fiber: n03-n08, n06-n10 +# Fiber: n01-n02, n01-n03, n01-n04, n02-n05, n03-n07, n04-n09 +# Bluetooth: n02-n06, n04-n08 +# WiFi: n03-n06, n08-n10 +# Fiber: n03-n08, n06-n10 # # Test subjects: -# - n06 has fiber (n03) and LoRa (n02) parents — should pick n03 -# - n08 has fiber (n03) and LoRa (n04) parents — should pick n03 +# - n06 has fiber (n03) and Bluetooth (n02) parents — should pick n03 +# - n08 has fiber (n03) and Bluetooth (n04) parents — should pick n03 # # Netem mutation shifts fiber-only links between normal and degraded. @@ -64,12 +64,12 @@ netem: jitter_ms: [0, 1] loss_pct: [0, 0.5] link_policies: - # LoRa links (high latency, moderate loss) + # Bluetooth (L2CAP) links - edges: ["n02-n06", "n04-n08"] policy: - delay_ms: [400, 600] - jitter_ms: [50, 100] - loss_pct: [5, 15] + delay_ms: [15, 40] + jitter_ms: [5, 15] + loss_pct: [2, 8] # WiFi links (moderate latency, low loss) - edges: ["n03-n06", "n08-n10"] policy: diff --git a/testing/chaos/sim/scenario.py b/testing/chaos/sim/scenario.py index 8d24bf8..6416ffc 100644 --- a/testing/chaos/sim/scenario.py +++ b/testing/chaos/sim/scenario.py @@ -34,6 +34,10 @@ class TopologyConfig: subnet: str = "172.20.0.0/24" ip_start: int = 10 default_transport: str = "udp" + # Optional transport mix for random topologies: {transport: weight}. + # When set, each edge is randomly assigned a transport based on weights. + # Only valid for non-explicit algorithms (explicit uses per-edge syntax). + transport_mix: dict[str, float] | None = None @dataclass @@ -191,6 +195,11 @@ def load_scenario(path: str) -> Scenario: s.topology.subnet = tc.get("subnet", "172.20.0.0/24") s.topology.ip_start = int(tc.get("ip_start", 10)) s.topology.default_transport = tc.get("default_transport", "udp") + if "transport_mix" in tc: + mix = tc["transport_mix"] + if not isinstance(mix, dict) or not mix: + raise ValueError("topology.transport_mix must be a non-empty dict") + s.topology.transport_mix = {str(k): float(v) for k, v in mix.items()} # Netem section nc = raw.get("netem", {}) @@ -289,6 +298,21 @@ def _validate(s: Scenario): f"topology.default_transport: '{s.topology.default_transport}' " f"not in {VALID_TRANSPORTS}" ) + if s.topology.transport_mix is not None: + if s.topology.algorithm == "explicit": + raise ValueError( + "topology.transport_mix cannot be used with explicit algorithm " + "(use per-edge transport syntax instead)" + ) + for transport, weight in s.topology.transport_mix.items(): + if transport not in VALID_TRANSPORTS: + raise ValueError( + f"topology.transport_mix: '{transport}' not in {VALID_TRANSPORTS}" + ) + if weight <= 0: + raise ValueError( + f"topology.transport_mix: weight for '{transport}' must be > 0" + ) if s.topology.algorithm == "explicit": adj = s.topology.params.get("adjacency") if not adj or not isinstance(adj, list): diff --git a/testing/chaos/sim/topology.py b/testing/chaos/sim/topology.py index b9a26d0..e2e1ab6 100644 --- a/testing/chaos/sim/topology.py +++ b/testing/chaos/sim/topology.py @@ -210,9 +210,9 @@ def generate_topology( else: raise ValueError(f"Unknown algorithm: {config.algorithm}") - # For non-explicit topologies, all edges use the default transport + # Assign transport types to edges if config.algorithm != "explicit": - edge_transport = {e: config.default_transport for e in edges} + edge_transport = _assign_edge_transports(edges, config, rng) # Build peer lists from edges for a, b in edges: @@ -243,6 +243,7 @@ def generate_topology( nodes[b].peers.append(a) topo.edges = edges + topo.edge_transport = _assign_edge_transports(edges, config, rng) if not topo.is_connected(): raise RuntimeError( @@ -319,6 +320,26 @@ def _generate_explicit( return edges, edge_transport +def _assign_edge_transports( + edges: set[tuple[str, str]], + config: TopologyConfig, + rng: random.Random, +) -> dict[tuple[str, str], str]: + """Assign transport types to edges. + + If ``config.transport_mix`` is set, each edge is randomly assigned + a transport based on the mix weights. Otherwise all edges use + ``config.default_transport``. + """ + if config.transport_mix is None: + return {e: config.default_transport for e in edges} + + transports = list(config.transport_mix.keys()) + weights = [config.transport_mix[t] for t in transports] + assignments = rng.choices(transports, weights=weights, k=len(edges)) + return dict(zip(sorted(edges), assignments)) + + def veth_interface_name(local: str, peer: str) -> str: """Generate the veth interface name inside a container. diff --git a/testing/static/README.md b/testing/static/README.md index 762898a..7ba9d20 100644 --- a/testing/static/README.md +++ b/testing/static/README.md @@ -2,10 +2,10 @@ Multi-node integration test for FIPS using Docker containers with fixed topologies. Multiple topologies are provided: a sparse mesh (5 nodes, 6 -links), a linear chain (5 nodes, 4 links), and a mesh with a public external -node. All exercise the full FIPS stack including TUN devices, DNS resolution, -peer link encryption, spanning tree construction, and discovery-driven -multi-hop routing. +links), a linear chain (5 nodes, 4 links), a mesh with a public external +node, and a TCP chain (3 nodes). All exercise the full FIPS stack including +TUN devices, DNS resolution, peer link encryption, spanning tree +construction, and discovery-driven multi-hop routing. ## Prerequisites @@ -56,14 +56,14 @@ are highlighted in blue in the diagram above. The ping test exercises all 20 directed pairs (5 nodes x 4 targets each), covering both direct-peer and multi-hop paths. -| Link | Type | -|------|------| +| Link | Type | +| ------ | --------------------------- | | A -- D | tree edge (D's parent is A) | | A -- E | tree edge (E's parent is A) | | C -- D | tree edge (C's parent is D) | | B -- C | tree edge (B's parent is C) | -| D -- E | non-tree link | -| C -- E | non-tree link | +| D -- E | non-tree link | +| C -- E | non-tree link | ### Chain @@ -88,6 +88,16 @@ topology is for testing mixed local/remote mesh operation. External nodes are not managed by Docker -- only their identity and address appear in the topology file so that Docker nodes can peer with them. +### TCP Chain + +Three nodes in a linear chain using TCP transport (port 443) instead of +UDP: A -- B -- C. Each node peers only with its immediate neighbors. +Tests basic TCP transport connectivity and multi-hop routing over TCP. + +The topology file sets `default_transport: tcp`, which causes config +generation to use TCP peer addresses (port 443), inject the TCP transport +section, and remove the UDP transport section. + ## Configuration Management ### File Structure @@ -103,15 +113,18 @@ testing/static/ │ └── topologies/ │ ├── mesh.yaml # Mesh topology definition │ ├── chain.yaml # Chain topology definition -│ └── mesh-public.yaml # Mesh + external public node +│ ├── mesh-public.yaml # Mesh + external public node +│ └── tcp-chain.yaml # TCP chain (3 nodes, port 443) ├── generated-configs/ # Auto-generated (gitignored) │ ├── npubs.env # NPUB_A=..., NPUB_B=..., etc. │ ├── mesh/ │ │ ├── node-a.yaml ... node-e.yaml │ ├── mesh-public/ │ │ ├── node-a.yaml ... node-e.yaml -│ └── chain/ -│ ├── node-a.yaml ... node-e.yaml +│ ├── chain/ +│ │ ├── node-a.yaml ... node-e.yaml +│ └── tcp-chain/ +│ ├── node-a.yaml ... node-c.yaml ├── scripts/ │ ├── build.sh # Build binary + generate configs │ ├── generate-configs.sh # Generate node configs from topology @@ -306,7 +319,7 @@ removing rules first. - **Capabilities**: `CAP_NET_ADMIN` (for TUN device creation) - **Devices**: `/dev/net/tun` mapped into each container - **DNS**: FIPS built-in resolver on `127.0.0.1:53` -- **Transport**: UDP on port 2121, MTU 1472 +- **Transport**: UDP on port 2121 (MTU 1472) or TCP on port 443 - **TUN**: `fips0` interface, MTU 1280 Each node resolves `.fips` DNS names to FIPS IPv6 addresses via its