laantungir/fips_setup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Hammered fips bugs

Browse Source
This commit is contained in:
Laan Tungir
2026-04-11 20:40:01 -04:00
parent a351883b67
commit a09c87ef10
10 changed files with 1214 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

56
README.md
View File

@@ -134,3 +134,59 @@ Now `sys-fips` and `sys-fips2` are independent FIPS nodes.
- The previous two-node local loopback approach was useful for local experiments but is not required for normal Qubes deployment.
- FIPS mesh encryption is end-to-end. A VPN upstream is optional and can be chained.
- `06-configure-dns.sh` removes dnsmasq `--local-service` from Debian's helper to avoid dropping AppVM DNS on Qubes `/32` links.
---
## Lessons learned from real `sys-fips` troubleshooting
These are the key fixes required to make AppVM → `sys-fips` → FIPS mesh work reliably in Qubes.
### 1) Intercept AppVM DNS in `sys-fips` with nftables
Qubes AppVM DNS normally targets the Qubes gateway addresses (for example `10.139.1.1` / `10.139.1.2`).
To make `.fips` resolution work from routed AppVMs, `sys-fips` must DNAT DNS to local dnsmasq.
Required nft hooks in `sys-fips`:
- `table ip qubes`, chain `dnat-dns`
- `table ip qubes`, chain `custom-input`
`dnat-dns` redirects UDP/TCP 53 traffic to local DNS (`10.139.1.1:53`) and `custom-input` allows DNS from AppVM interfaces (for example `vif+`).
### 2) IPv6 forwarding + route to FIPS space
`sys-fips` must forward IPv6 and route FIPS mesh space (`fd00::/8`) to `fips0`:
- enable IPv6 forwarding (`sysctl net.ipv6.conf.all.forwarding=1`)
- ensure route exists: `ip -6 route add fd00::/8 dev fips0`
Without this, AppVM traffic may resolve `.fips` names but fail to reach the resolved FIPS IPv6 destination.
### 3) AppVMs need a valid IPv6 source + route toward `sys-fips`
In testing, routed AppVM access required:
- a synthetic IPv6 on `eth0` (example: `fdaa::15/128`)
- route of FIPS space via Qubes link-local gateway:
- `ip -6 route add fd00::/8 via fe80::fcff:ffff:feff:ffff dev eth0`
This ensures traffic to resolved `.fips` AAAA records leaves the AppVM through `sys-fips`.
### 4) Persist network fixes in `/rw/config/rc.local`
For reboot safety:
- persist nftables DNS interception + IPv6 forwarding/route in `sys-fips` `/rw/config/rc.local`
- persist AppVM synthetic IPv6 + route in each AppVM `/rw/config/rc.local` that needs `.fips`
Do not rely on one-off manual commands for production.
### 5) Chromium-family browser caveat
Observed behavior:
- Firefox followed system resolver path and worked once networking was fixed.
- Chromium-family browsers (`agent-browser`, Brave) could still show `ERR_NAME_NOT_RESOLVED` even when `dig`, `getent`, `ping`, and `curl` worked.
Current operational workaround is to pin the `.fips` hostname in `/etc/hosts` for Chromium-family use-cases.
See `plans/chromium-fips-workaround.md` for tested commands and policy notes.

177
plans/FIPS_ADDRESSING.md Normal file
View File

@@ -0,0 +1,177 @@
# FIPS Addressing: How to Share Your Node Address
## Overview
There are **three ways** someone can reach your FIPS node, from most user-friendly to most raw:
| Method | Example | Config needed on visitor? |
|---|---|---|
| Friendly hostname | `http://mysite.fips` | Yes — host map entry or peer alias |
| Direct npub | `http://npub1qmc3...98.fips` | No — works on any FIPS node |
| Raw IPv6 | `http://[fdab:1234:...]:80` | Requires prior DNS lookup to prime cache |
---
## 1. Friendly Hostname: `mysite.fips`
This is the best option for sharing a web page address. You create a human-readable alias by adding an entry to `/etc/fips/hosts` on the visitor's node:
```
mysite npub1abc...xyz
```
Or, if the visitor has you as a configured peer with an `alias` in their `fips.yaml`, it works automatically:
```yaml
peers:
- npub: "npub1abc...xyz"
alias: "mysite"
addresses:
- transport: udp
addr: "1.2.3.4:2121"
```
Then they just browse to:
```
http://mysite.fips
```
### How it works
The FIPS DNS resolver on port 5354 sees `mysite.fips`, looks up `mysite` in the host map → gets the npub → derives the `fd00::/8` IPv6 address → primes the identity cache → returns the AAAA record. The browser connects transparently.
### The catch
This is a *local* mapping. There's no global DNS — each node maintains its own host map. So you'd share both the hostname convention and the npub, and visitors add the entry themselves. Think of it like a `/etc/hosts` entry but for the FIPS mesh.
Resolution priority:
1. `/etc/fips/hosts` entries (take precedence)
2. Peer `alias` fields from `fips.yaml`
3. Fall through to direct npub resolution
Changes to `/etc/fips/hosts` are picked up automatically on the next DNS query — no restart needed.
### Hosts file format
```
# /etc/fips/hosts
# hostname npub
mysite npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98
vps-tx npub10yffd020a4ag8zcy75f9pruq3rnghvvhd5hphl9s62zgp35s560qrksp9u
```
Rules:
- Lines starting with `#` are comments
- Empty lines are ignored
- Hostnames: a-z, 0-9, hyphens only; max 63 characters
- One hostname and one npub per line, separated by whitespace
---
## 2. Direct npub Address: `npub1abc...xyz.fips`
This is the **zero-configuration** option — no host map needed. You give someone your npub and they use it directly as a domain name:
```
http://npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips
```
### How it works
The FIPS DNS resolver extracts the npub from the label before `.fips`, derives the IPv6 address purely computationally (npub → public key → SHA-256 → node_addr → `fd00::/8` IPv6), and returns it as an AAAA record. No lookup table needed — the npub *is* the address.
This works on any FIPS node without any prior configuration. The downside is obvious: npubs are 63 characters of bech32, so the URL is long and ugly.
### Examples
```bash
# Web browsing
http://npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips
# SSH
ssh -6 npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips
# Ping
ping6 npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips
# Curl
curl -6 http://npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips
```
---
## 3. Raw IPv6 Address: `fd00:...`
Under the hood, every npub deterministically maps to an IPv6 address in the `fd00::/8` ULA range:
```
Public Key (32 bytes)
→ SHA-256 → node_addr (16 bytes)
→ fd + node_addr[0..15] → IPv6 address (16 bytes)
```
So you could share the raw IPv6:
```
http://[fdab:1234:5678:9abc:def0:1234:5678:9abc]:80
```
### Why this is the worst option
The derivation is **one-way** — the receiver's FIPS node can't recover the npub from the IPv6 address alone. The identity cache must be primed first (via a DNS lookup or inbound traffic from that node). If someone types the raw IPv6 without a prior DNS resolution, they get ICMPv6 "No route to destination."
Use raw IPv6 only for debugging or when you've already resolved the address via DNS.
---
## The Address Derivation Chain
```
Keypair (secp256k1)
│
├── npub1... (bech32-encoded public key)
│ ↓
│ npub1xxx.fips ← DNS name, resolvable on any FIPS node
│
├── node_addr (16 bytes = SHA-256 of pubkey, truncated)
│ ↓
│ Used internally for routing — intermediate nodes see only this
│
└── fd00::/8 IPv6 (fd + node_addr[0..15])
↓
Used by TUN interface for unmodified IPv6 apps
```
All three representations are **deterministically derived** from the same keypair. No registration, no central authority.
---
## What You'd Actually Tell People
In practice, you'd share something like:
> **My FIPS address:** `npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98`
>
> Browse to: `http://npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips`
>
> Or add to your `/etc/fips/hosts`:
> ```
> coolsite npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98
> ```
> Then just visit `http://coolsite.fips`
The **npub is the canonical address** — it's self-authenticating (derived from your keypair), works without any central registry, and any FIPS node can resolve it to an IPv6 address purely through computation. The friendly hostname is a local convenience layer on top.
---
## Key Properties
| Property | Detail |
|---|---|
| **Self-sovereign** | You generate your own address from a keypair — no one assigns it to you |
| **Self-authenticating** | The address IS the public key — connecting to an npub cryptographically proves you reached the right node |
| **Deterministic** | npub → IPv6 is pure computation, same result everywhere, no lookup needed |
| **No global registry** | No DNS authority, no CA, no registrar |
| **Friendly names are local** | `/etc/fips/hosts` is per-node; there's no global `.fips` name registry |
| **Privacy-preserving** | Intermediate routers see only the node_addr hash, not the npub identity |

53
plans/chromium-fips-workaround.md Normal file
View File

@@ -0,0 +1,53 @@
# Chromium/Brave `.fips` Access in `ai`
## Current proven behavior
- `.fips` works in Firefox (system resolver path).
- `.fips` works in `agent-browser` only when hostname is pinned in `/etc/hosts`.
- Raw FIPS network path is healthy in `ai`:
- `dig` resolves npub `.fips`
- `ping -6` to server FIPS IPv6 works
- `curl -6` to npub `.fips` returns HTTP 200
## Why Chromium-family is failing by default
Chromium/Brave path still returns name-resolution errors for `.fips` in this environment, even when system resolver succeeds. In practice here, forcing host mapping is the reliable path.
## Reliable workaround (now)
### 1) Resolve npub hostname to IPv6 and pin in hosts
```bash
HOST='npub1crpldvy49ef8z34wlacwujnfudy4nd7k96aqdx5wgn6ckztz7z8q9t59ud.fips'
IP=$(dig +short AAAA "$HOST" | head -1)
echo "$IP $HOST" | sudo tee -a /etc/hosts
```
### 2) Validate resolution and HTTP
```bash
getent ahosts "$HOST"
curl -6 -I "http://$HOST/"
```
### 3) Launch in Chromium tooling
```bash
AGENT_BROWSER_HEADED=1 agent-browser open "http://$HOST/"
```
## Maintenance note
If the target FIPS IPv6 changes, update `/etc/hosts` entry by re-running the resolve-and-pin step.
## Optional cleanup
To remove the pinned line later:
```bash
sudo sed -i '/npub1crpldvy49ef8z34wlacwujnfudy4nd7k96aqdx5wgn6ckztz7z8q9t59ud\.fips/d' /etc/hosts
```
## Deferred work (later)
For sys-fips ProxyVM mode, IPv6 forwarding/NAT and nftables flow still need a clean persistent design before re-enabling that path as default.

209
plans/fips-overview.md Normal file
View File

@@ -0,0 +1,209 @@
# FIPS: Free Internetworking Peering System — Overview
## 1. What It Does
FIPS is a **self-organizing, decentralized mesh network** written in Rust. It allows any set of machines to form an encrypted peer-to-peer network that routes traffic without any central authority, DNS, ISP, or infrastructure dependency.
### The Elevator Pitch
> Connect to one peer and you can reach the entire mesh. No accounts, no servers, no registration. Your cryptographic keypair **is** your address.
### Concrete Capabilities
| Capability | Description |
|---|---|
| **Mesh networking** | Nodes discover each other, build routing automatically, and relay traffic for each other across multiple hops |
| **Multi-transport** | Works over UDP, TCP, Ethernet, Tor, and Bluetooth (BLE) — designed for serial and radio too |
| **End-to-end encryption** | Double-layer: every hop is encrypted *and* every session is independently encrypted end-to-end |
| **Nostr-native identity** | Your node address is a Nostr keypair — `npub1...` — no registration, no CA, no central authority |
| **IPv6 compatibility** | A TUN interface maps npubs to `fd00::/8` IPv6 addresses so unmodified apps like SSH, curl, and ping6 just work |
| **DNS for the mesh** | Built-in `.fips` DNS resolver: `ping6 npub1abc...fips` resolves and routes through the mesh |
| **Zero configuration** | A node can start with no config file; add one peer address and you join the network |
| **Operator tooling** | `fipsctl` CLI and `fipstop` TUI dashboard for live monitoring of peers, links, sessions, and routing |
### What You Gain
1. **Sovereignty** — You own your identity. No one can revoke your address or deny you access.
2. **Resilience** — The mesh self-heals around failures. No single point of failure.
3. **Privacy** — Intermediate nodes route traffic without learning endpoint identities. Hop-by-hop + end-to-end encryption means even your direct peers cannot read your session traffic.
4. **Transport freedom** — If you can get packets between two machines by *any* means — WiFi, Bluetooth, a serial cable, Tor, the public internet — FIPS can build a mesh link over it.
5. **Application transparency** — Existing IPv6 apps work without modification through the TUN adapter.
---
## 2. How It Works (High Level)
### Architecture: Three Protocol Layers
```
┌─────────────────────────────────────────────────┐
│ Applications (SSH, curl, custom) │
│ ↕ IPv6 TUN adapter (fd00::/8) + .fips DNS │
├─────────────────────────────────────────────────┤
│ FSP — FIPS Session Protocol │
│ End-to-end Noise XK encryption between npubs │
│ Port-based service multiplexing │
├─────────────────────────────────────────────────┤
│ FMP — FIPS Mesh Protocol │
│ Peer auth (Noise IK), spanning tree, │
│ bloom filters, hop-by-hop forwarding │
├─────────────────────────────────────────────────┤
│ Transport Layer │
│ UDP | TCP | Ethernet | Tor | BLE │
└─────────────────────────────────────────────────┘
```
### Identity: Your Keypair Is Your Address
1. A node generates a **secp256k1 keypair** (same as Nostr)
2. The public key (`npub`) is the node's identity
3. A **node_addr** (16-byte SHA-256 hash of the pubkey) is used internally for routing — intermediate routers see only this hash, not the npub
4. An **IPv6 address** (`fd00::/8`) is deterministically derived from the node_addr for the TUN interface
All three representations — npub, node_addr, IPv6 — are derived from the same keypair. No registration needed.
### Self-Organization: Spanning Tree + Bloom Filters
The mesh organizes itself through two complementary mechanisms:
#### Spanning Tree (The Coordinate System)
- Nodes gossip `TreeAnnounce` messages with their direct peers
- Each node independently selects a **parent** based on cost-weighted depth
- The network converges on a single **root** (the node with the smallest node_addr) — no voting, no election
- Every node gets a **coordinate** — its path from itself to the root
- These coordinates enable **distance calculations** between any two nodes without global topology knowledge
- Tree state per node is O(peers × depth), **not** O(network size)
#### Bloom Filters (Reachability Knowledge)
- Each node maintains per-peer bloom filters summarizing which destinations are reachable through each peer
- Filters propagate transitively through tree edges using **split-horizon** (borrowed from distance-vector routing) to prevent loops
- At each hop, the router checks: "which of my peers can possibly reach destination D?" — definitive "no" eliminates a peer; "maybe" means worth checking
- Fixed 1 KB filter size regardless of network size; effective for networks up to ~2,000 nodes
#### Routing Decision (Per-Hop)
At each hop, the forwarding decision is:
1. **Local delivery?** — destination is this node
2. **Direct peer?** — destination is an authenticated neighbor
3. **Bloom-guided selection** — bloom filters identify candidate peers; tree coordinates rank them by distance
4. **Greedy tree routing** — forward to the peer closest to the destination in tree distance
### Two-Layer Encryption
Every packet is encrypted **twice**, independently:
| Layer | Scope | Noise Pattern | Purpose |
|---|---|---|---|
| **FMP** | Hop-by-hop | Noise IK | Encrypts ALL traffic on each peer link |
| **FSP** | End-to-end | Noise XK | Encrypts application payload between endpoints |
A packet from node A to node D through intermediaries B and C:
```
A encrypts payload with A↔D session key (FSP)
A wraps + encrypts with A↔B link key (FMP) → sends to B
B decrypts link layer, reads destination, re-encrypts with B↔C link key → forwards to C
C decrypts link layer, re-encrypts with C↔D link key → forwards to D
D decrypts link layer, then decrypts session layer → gets payload
```
Intermediate nodes B and C route by destination node_addr but **cannot read the session payload**.
### Transport Agnosticism
FIPS treats transports as dumb pipes. Each transport implements: send bytes, receive bytes, report MTU. The mesh layer handles everything above that. A single packet path might traverse:
```
WiFi → Ethernet → UDP/Internet → Tor → BLE
```
The application never knows or cares which transports are involved.
---
## 3. How It Is Innovative
### 3.1 Nostr Identity as Network Address
Most mesh networks invent their own identity system. FIPS reuses **Nostr keypairs** (secp256k1/Schnorr) as native node addresses. This is significant because:
- **No identity infrastructure** — no CA, no registrar, no DNS. Generate a keypair and you have an address.
- **Cross-ecosystem compatibility** — the same npub that identifies you on Nostr relays identifies your FIPS node. Your social identity and your network identity can be the same key.
- **Ephemeral or persistent** — generate a throwaway keypair for a session, or keep a persistent identity across restarts. The network treats both the same.
No other mesh network uses an existing social-protocol identity system as its native addressing scheme.
### 3.2 Spanning Tree Coordinates + Bloom Filter Routing
FIPS combines two well-understood techniques in a novel way:
- **Spanning tree coordinates** (adapted from Yggdrasil/Ironwood) give every node a position without global topology knowledge
- **Bloom filter reachability** (adapted from distance-vector split-horizon) narrows forwarding candidates probabilistically
The key insight: bloom filters are an **optimization**, not a correctness requirement. If filters saturate in a large network, routing still works via greedy tree distance — it just evaluates more candidates per hop. This graceful degradation is unusual; most systems that depend on probabilistic structures fail hard when those structures saturate.
Each node's state is **O(peers × depth)**, not O(N). A node in a 10,000-node network with 5 peers and depth 10 stores ~50 ancestry entries — not 10,000 routing table entries.
### 3.3 True Transport Agnosticism
Most "multi-transport" systems support maybe UDP and TCP. FIPS has working implementations for:
- **UDP** and **TCP** (overlay over existing internet)
- **Raw Ethernet** (no IP stack needed — MAC-level, with auto-discovery)
- **Tor** (SOCKS5 + onion service for anonymous peering)
- **Bluetooth Low Energy** (L2CAP channels with MTU negotiation)
The architecture is designed for **serial links and radio** too. This isn't theoretical — the transport interface is genuinely minimal (send/receive datagrams + report MTU), making new transports straightforward to add.
The implication: FIPS can build a mesh that spans the public internet, a local WiFi network, a Bluetooth PAN, and a Tor hidden service — all simultaneously, all transparently to applications.
### 3.4 Double-Layer Encryption with Metadata Minimization
The two independent encryption layers (hop-by-hop Noise IK + end-to-end Noise XK) are not just defense-in-depth — they serve distinct purposes:
- **Hop-by-hop (IK)**: Authenticates peers and encrypts ALL link traffic, including routing gossip. An observer on the transport sees only encrypted blobs.
- **End-to-end (XK)**: Protects application payload across untrusted intermediaries. XK is chosen over IK specifically because it **delays initiator identity disclosure** until msg3, providing stronger identity hiding for traffic traversing untrusted nodes.
Intermediate routers see only **node_addr** (a one-way hash) — they cannot learn the Nostr identities of the communicating endpoints. This is a deliberate metadata minimization design, not an accident.
### 3.5 IPv6 Adaptation Without Compromise
Many overlay networks require custom APIs or modified applications. FIPS provides a **TUN interface** that maps npubs to `fd00::/8` IPv6 addresses, complete with:
- `.fips` DNS resolution (`ping6 npub1abc...fips`)
- ICMPv6 Packet Too Big for path MTU discovery
- TCP MSS clamping
- Static hostname mapping via `/etc/fips/hosts`
Unmodified SSH, curl, web browsers, and any IPv6-capable application work transparently. This makes FIPS immediately useful without waiting for application ecosystem adoption.
### 3.6 Qubes OS Integration (This Repository)
The `fips_setup` repository demonstrates FIPS running as a **Qubes OS ProxyVM** (`sys-fips`), slotting into the Qubes network chain:
```
AppVMs → sys-fips (ProxyVM) → sys-firewall → sys-net → Internet
└─ fips0 (TUN) → FIPS mesh (encrypted)
```
Any AppVM pointed at `sys-fips` transparently gains access to the FIPS mesh. This is a practical deployment model that leverages Qubes' compartmentalization — the FIPS node runs in its own security domain, isolated from application VMs.
---
## Summary
| Dimension | FIPS Approach |
|---|---|
| **Identity** | Nostr keypairs — self-sovereign, no infrastructure |
| **Routing** | Spanning tree coordinates + bloom filter candidate selection |
| **Encryption** | Dual-layer: Noise IK hop-by-hop + Noise XK end-to-end |
| **Transports** | UDP, TCP, Ethernet, Tor, BLE — designed for radio/serial |
| **Application compat** | IPv6 TUN + .fips DNS — unmodified apps work |
| **State per node** | O(peers × depth), not O(network size) |
| **Configuration** | Zero-config possible; one peer address to join |
| **Implementation** | Rust, ~100+ source files, MIT licensed |
FIPS is not just another VPN or overlay network. It is a **complete replacement for the network layer** that can operate alongside or independently of the internet, using whatever physical media are available, with cryptographic identity as the foundation rather than an afterthought.

440
plans/laantungir-tor-fips-plan.md Normal file
View File

@@ -0,0 +1,440 @@
# laantungir.net: Tor + FIPS + Clearnet Access Plan
## Current Server State (discovered via SSH)
| Component | Status |
|---|---|
| **OS** | Ubuntu 24.04.3 LTS (VM410) |
| **nginx** | 1.24.0, running, with RTMP module |
| **Tor** | 0.4.8.10, running, SOCKS on 127.0.0.1:9050 |
| **Onion service** | `kn2jam4kyz6s5wacyozo3do3d2zsvh4to45uh7xliulwdust6zjuvnad.onion` — forwards ports 80/443 to 127.0.0.1:80/443 |
| **FIPS** | Not installed |
| **Nostr relay** | c-relay-pg (C Nostr Relay, PostgreSQL backend) on port 8888 |
| **Blossom** | didactyl.service on 127.0.0.1:8484 |
| **Website** | Static site served by nginx |
| **Gitea** | Port 3000 (web) + 2222 (SSH) |
| **PostgreSQL** | 18-main on 127.0.0.1:5432 |
| **RTMP** | nginx-rtmp on port 1935 with auth + HLS |
| **Streaming** | OBS → RTMP :1935/publish (auth via rtmp-auth.service on :8091) → ffmpeg forward → :1935/live → HLS at /var/www/html/hls/ |
| **Stream stats** | stream-stats.service on 127.0.0.1:8092, nginx internal at 127.0.0.1:8080/stat |
| **simon.service** | 127.0.0.1:8485 (unknown purpose) |
### Video Streaming Architecture
```
OBS (remote)
│
│ RTMP push to :1935/publish/{show}/{variant}/{secret_key}
│
▼
nginx-rtmp :1935
├── on_publish → 127.0.0.1:8091/auth (rtmp-auth.service, validates secret key)
├── exec_push → /usr/local/bin/rtmp-forward.sh
│ └── ffmpeg -c copy → rtmp://127.0.0.1:1935/live/{show}/{variant}
│
▼
nginx-rtmp :1935/live
├── HLS output → /var/www/html/hls/{show}/{variant}/index.m3u8
├── generate-master-playlist.py → /var/www/html/hls/{show}/master.m3u8
└── RTMP playback (allow play all)
```
**Viewer access (clearnet)**: `https://laantungir.net/hls/{show}/{variant}/index.m3u8` (HLS) or `rtmp://laantungir.net/live/{show}/{variant}` (RTMP direct)
**Current shows in HLS**: beat-topia, dub-techno, music, the-world-is-coming-to-an-end
### Current nginx virtual hosts (from /etc/nginx/conf.d/default.conf)
| Domain | Port 80 | Port 443 | Backend |
|---|---|---|---|
| `laantungir.net` (+ .com, .org, www) | Redirect to HTTPS | TLS, static site | Local files |
| `relay.laantungir.net` (+ .com, .org) | Redirect to HTTPS | TLS, WebSocket proxy | 127.0.0.1:8888 (c-relay-pg) |
| `blossom.laantungir.net` | Redirect to HTTPS | TLS, proxy | 127.0.0.1:8484 (didactyl) |
| `git.laantungir.net` (+ .com, .org) | Redirect to HTTPS | TLS, proxy | 127.0.0.1:3000 (gitea) |
| `auth.laantungir.net` (+ .com, .org) | Redirect to HTTPS | TLS, proxy | 127.0.0.1:3000 (gitea) |
### Current Tor onion service
```
HiddenServiceDir /var/lib/tor/nostr_client/
HiddenServicePort 80 127.0.0.1:80
HiddenServicePort 443 127.0.0.1:443
```
Onion address: `kn2jam4kyz6s5wacyozo3do3d2zsvh4to45uh7xliulwdust6zjuvnad.onion`
**Problem**: The onion service forwards to nginx ports 80/443, but nginx virtual hosts use `server_name` matching. Tor clients send the `.onion` hostname, which doesn't match any `server_name` directive — so nginx falls through to the default server block. This needs to be fixed.
---
## Target Architecture
```
┌─────────────────────────────────────────────────────────┐
│ laantungir.net (VM410) │
│ │
Clearnet user ───►│─── eth0 :80/:443 ──► nginx ──┬── / ──► static site │
│ ├── relay ──► :8888 │
│ └── blossom ──► :8484 │
│ │
Tor user ────────►│─── tor :80/:443 ──► nginx ──┬── / ──► static site │
│ (via .onion) ├── relay ──► :8888 │
│ └── blossom ──► :8484 │
│ │
FIPS user ───────►│─── fips0 :80 ──► nginx ──┬── / ──► static site │
│ (via fd00::/8) ├── relay ──► :8888 │
│ └── blossom ──► :8484 │
│ │
FIPS+Tor user ───►│─── tor transport ──► fips0 ──► nginx (same as FIPS) │
│ (FIPS peering over Tor) │
│ │
│ FIPS daemon: UDP :2121, TCP :8443, Tor .onion:8443 │
└─────────────────────────────────────────────────────────┘
```
### How each access path works
| Path | User needs | Connects via | Encryption |
|---|---|---|---|
| **Clearnet** | Browser | `https://relay.laantungir.net` | TLS (Let's Encrypt) |
| **Tor** | Tor Browser | `http://kn2jam...nad.onion/relay/` or subdomain routing | Tor onion encryption |
| **FIPS** | FIPS node | `ws://npub1<server-npub>.fips` | Noise IK + Noise XK (double) |
| **FIPS over Tor** | FIPS node + Tor | FIPS peers via Tor transport | Tor + Noise IK + Noise XK (triple) |
---
## Implementation Plan
### Phase 1: Fix Tor Onion Service Routing
The existing onion service already forwards ports 80/443 to nginx, but nginx doesn't know how to route `.onion` hostnames. Two approaches:
**Option A: Path-based routing (single .onion, recommended)**
Add the `.onion` hostname to each `server_name` directive:
```nginx
# In the relay server block:
server_name relay.laantungir.com relay.laantungir.net relay.laantungir.org
kn2jam4kyz6s5wacyozo3do3d2zsvh4to45uh7xliulwdust6zjuvnad.onion;
```
Tor users would access:
- Website: `http://kn2jam...nad.onion/`
- Relay: `ws://kn2jam...nad.onion/` (if relay is the default server)
- Blossom: `http://kn2jam...nad.onion/blossom/` (needs path-based routing)
**Problem**: relay and blossom are separate subdomains, not paths. With a single .onion, we need a way to distinguish them.
**Option B: Separate onion services per service (cleaner)**
Create three onion services in torrc:
```
# Website
HiddenServiceDir /var/lib/tor/hs_website/
HiddenServicePort 80 127.0.0.1:80
HiddenServicePort 443 127.0.0.1:443
# Relay
HiddenServiceDir /var/lib/tor/hs_relay/
HiddenServicePort 80 127.0.0.1:80
HiddenServicePort 443 127.0.0.1:443
# Blossom
HiddenServiceDir /var/lib/tor/hs_blossom/
HiddenServicePort 80 127.0.0.1:80
HiddenServicePort 443 127.0.0.1:443
```
Then add each `.onion` to the corresponding nginx `server_name`. Each service gets its own `.onion` address. Tor users connect to the right service by using the right `.onion`.
**Option C: Single .onion with default-server routing (simplest)**
Make the relay the default server (it's the most important service), and add the `.onion` to the website and blossom blocks too. Use a port-based split:
```
HiddenServiceDir /var/lib/tor/nostr_client/
HiddenServicePort 80 127.0.0.1:80 # website
HiddenServicePort 443 127.0.0.1:443 # website (TLS)
HiddenServicePort 7777 127.0.0.1:8888 # relay direct (no nginx)
HiddenServicePort 8484 127.0.0.1:8484 # blossom direct (no nginx)
```
Tor users connect to different ports on the same `.onion`:
- `http://kn2jam...nad.onion/` → website
- `ws://kn2jam...nad.onion:7777/` → relay (direct to c-relay-pg)
- `http://kn2jam...nad.onion:8484/` → blossom (direct to didactyl)
### Decision: Option B — Separate .onion per service
This gives the best compatibility with existing programs. Nostr clients expect to connect to a relay at a WebSocket URL like `ws://host/`. Blossom clients expect `http://host/`. Separate `.onion` addresses mean each service gets its own hostname — no path rewriting, no port gymnastics. Existing clients work unmodified.
### Steps for Phase 1
1. Add three HiddenService blocks to `/etc/tor/torrc` (website, relay, blossom)
2. Add each `.onion` hostname to the corresponding nginx `server_name` directive
3. Add plain-HTTP server blocks for `.onion` access (Tor already encrypts; no TLS needed)
4. Restart Tor (generates new onion keys), then restart nginx
5. Test from Tor Browser
---
### Phase 2: Install and Configure FIPS
#### 2.1 Build and install FIPS
```bash
# Install Rust toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env
# Install build deps for BLE (optional, can skip with --no-default-features)
sudo apt install -y bluez libdbus-1-dev
# Clone and build
git clone https://github.com/jmcorgan/fips.git
cd fips
cargo build --release
# Install via .deb
cargo install cargo-deb
cargo deb
sudo dpkg -i target/debian/fips_*.deb
```
#### 2.2 Configure FIPS
Edit `/etc/fips/fips.yaml`:
```yaml
node:
identity:
persistent: true # Stable npub for the server
tun:
enabled: true
name: fips0
mtu: 1280
dns:
enabled: true
bind_addr: "127.0.0.1"
port: 5354
transports:
udp:
bind_addr: "0.0.0.0:2121"
tcp:
bind_addr: "0.0.0.0:8443"
tor:
mode: "directory"
socks_addr: "127.0.0.1:9050"
directory_service:
hostname_file: "/var/lib/tor/fips_mesh/hostname"
bind_addr: "127.0.0.1:9443"
peers:
- npub: "npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98"
alias: "fips-test-node"
addresses:
- transport: udp
addr: "217.77.8.91:2121"
connect_policy: auto_connect
- npub: "npub1zv58cn7v83mxvttl70w5fwjwuclfmntv9cnmv5wmz2nzz88u5urqvdx96n"
alias: "fips.v0l.io"
addresses:
- transport: udp
addr: "fips.v0l.io:2121"
connect_policy: auto_connect
```
#### 2.3 Add FIPS Tor onion service to torrc
Add a separate onion service for FIPS mesh peering:
```
# FIPS mesh peering (separate from web services)
HiddenServiceDir /var/lib/tor/fips_mesh/
HiddenServicePort 9443 127.0.0.1:9443
```
#### 2.4 Open firewall for FIPS
```bash
sudo ufw allow 2121/udp comment "FIPS mesh UDP"
sudo ufw allow 8443/tcp comment "FIPS mesh TCP"
```
#### 2.5 Start FIPS
```bash
sudo systemctl enable --now fips
sudo systemctl enable --now fips-dns
```
#### 2.6 Configure DNS for .fips resolution on the server
```bash
sudo resolvectl dns fips0 127.0.0.1:5354
sudo resolvectl domain fips0 ~fips
```
---
### Phase 3: Make Services Reachable Over FIPS
#### 3.1 Make nginx listen on IPv6 (for FIPS fd00::/8 traffic)
Add `listen [::]:80;` and `listen [::]:443 ssl;` to each server block in `/etc/nginx/conf.d/default.conf`. This allows FIPS traffic (which arrives as IPv6 on the fips0 TUN interface) to reach nginx.
#### 3.2 Decision: Separate server blocks per service (matching Tor approach)
For maximum compatibility with existing programs, we use the same approach as Tor: each service gets its own server block matched by hostname. FIPS users use different `.fips` hostnames (via `/etc/fips/hosts`) to reach each service — just like Tor users use different `.onion` addresses.
Since blossom (didactyl) is a FastCGI extension of nginx, all three services must go through nginx.
```nginx
# FIPS: Website
server {
listen [::]:80;
server_name ~^npub1.+\.fips$ laantungir.fips;
# Same config as the clearnet website server block
# (static site root, locations, etc.)
}
# FIPS: Relay
server {
listen [::]:80;
server_name relay.fips;
location / {
proxy_pass http://127.0.0.1:8888;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}
# FIPS: Blossom
server {
listen [::]:80;
server_name blossom.fips;
# Same FastCGI config as the clearnet blossom server block
}
```
FIPS users access:
- `http://laantungir.fips/` → website
- `ws://relay.fips/` → Nostr relay (standard WebSocket URL, works with all Nostr clients)
- `http://blossom.fips/` → Blossom (standard HTTP, works with all Blossom clients)
The direct npub also works: `http://npub1<server-npub>.fips/` → website (default)
#### 3.3 Add /etc/fips/hosts entries
```
laantungir npub1<server-npub>
relay npub1<server-npub>
blossom npub1<server-npub>
```
This lets FIPS users access:
- `http://laantungir.fips/` → website
- `ws://relay.fips/` → relay (if using path-based routing)
- `http://blossom.fips/` → blossom
---
### Phase 4: Enable FIPS-over-Tor (Combined)
This is already handled by Phase 2.3. The FIPS config includes a Tor transport with a dedicated onion service for mesh peering. Remote FIPS nodes can peer with this server through Tor by adding:
```yaml
# On the remote FIPS node
peers:
- npub: "npub1<server-npub>"
alias: "laantungir"
addresses:
- transport: tor
addr: "<fips-mesh-onion>.onion:9443"
connect_policy: auto_connect
```
The remote node's FIPS traffic goes through Tor to reach this server's FIPS daemon, then the FIPS session provides end-to-end encryption on top. Triple encryption: Tor + Noise IK (hop) + Noise XK (session).
---
## Summary of Access Paths After Implementation
### Website (laantungir.net)
| Path | URL | Encryption |
|---|---|---|
| Clearnet | `https://laantungir.net` | TLS |
| Tor | `http://kn2jam...nad.onion/` | Tor |
| FIPS | `http://npub1<npub>.fips/` or `http://laantungir.fips/` | Noise IK+XK |
| FIPS+Tor | Same as FIPS (transport is transparent) | Tor + Noise IK+XK |
### Nostr Relay (relay.laantungir.net)
| Path | URL | Encryption |
|---|---|---|
| Clearnet | `wss://relay.laantungir.net` | TLS |
| Tor | `ws://<relay-onion>.onion/` | Tor |
| FIPS | `ws://relay.fips/` | Noise IK+XK |
| FIPS+Tor | Same as FIPS (transport is transparent) | Tor + Noise IK+XK |
### Blossom (blossom.laantungir.net)
| Path | URL | Encryption |
|---|---|---|
| Clearnet | `https://blossom.laantungir.net` | TLS |
| Tor | `http://<blossom-onion>.onion/` | Tor |
| FIPS | `http://blossom.fips/` | Noise IK+XK |
| FIPS+Tor | Same as FIPS (transport is transparent) | Tor + Noise IK+XK |
### Video Streaming (laantungir.net/hls/)
| Path | URL | Encryption | Notes |
|---|---|---|---|
| Clearnet | `https://laantungir.net/hls/{show}/master.m3u8` | TLS | HLS playback via static files |
| Clearnet RTMP | `rtmp://laantungir.net/live/{show}/{variant}` | None | Direct RTMP playback |
| Tor | `http://<website-onion>.onion/hls/{show}/master.m3u8` | Tor | HLS over Tor (same website .onion) |
| FIPS | `http://laantungir.fips/hls/{show}/master.m3u8` | Noise IK+XK | HLS over FIPS |
| FIPS+Tor | Same as FIPS | Tor + Noise IK+XK | HLS over FIPS+Tor |
**Streaming considerations:**
- **HLS playback** (HTTP-based) works naturally over Tor and FIPS — it's just static `.m3u8` and `.ts` files served by nginx. No special config needed beyond the website server block.
- **RTMP ingest** (OBS publishing on port 1935) stays clearnet-only. RTMP is a TCP protocol on a non-HTTP port; exposing it over Tor or FIPS would require additional transport config and is not needed — the publisher (you) controls the ingest path.
- **RTMP direct playback** (port 1935/live) is clearnet-only for the same reason. Viewers over Tor/FIPS use HLS instead.
- **Stream stats** (127.0.0.1:8092) and **RTMP auth** (127.0.0.1:8091) are internal-only and not exposed on any path.
---
## Decisions (Resolved)
1. **Tor routing**: Option B — separate `.onion` per service. Best compatibility with existing clients.
2. **FIPS service routing**: Separate `.fips` hostnames per service via `/etc/fips/hosts`. Best compatibility with existing clients.
3. **Blossom**: Stays behind nginx (it's a FastCGI extension, must go through nginx).
4. **Bootstrap peers**: fips-test-node (217.77.8.91:2121) + fips.v0l.io (fips.v0l.io:2121).
---
## Execution Order
1. Fix Tor onion routing (nginx server_name + torrc)
2. Test Tor access from Tor Browser
3. Build and install FIPS
4. Configure FIPS with persistent identity + Tor transport
5. Add FIPS onion service to torrc, restart Tor
6. Open firewall ports for FIPS
7. Start FIPS, verify fips0 interface and peer connectivity
8. Update nginx to listen on IPv6 and handle FIPS traffic
9. Test FIPS access from another FIPS node
10. Test FIPS-over-Tor access
11. Publish server npub and .onion addresses

113
plans/sys-fips-browser-fix-plan.md Normal file
View File

@@ -0,0 +1,113 @@
# sys-fips `.fips` Browser Access Remediation Plan
## Goal
Make `.fips` hostnames (especially the npub hostname for `laantungir.net`) resolve and load from the current AppVM browser path.
## Confirmed Root Cause
Current AppVM DNS path is returning public-DNS NXDOMAIN for `.fips`, which means `.fips` forwarding via `sys-fips` is not active in the effective network chain.
## Success Criteria
1. From AppVM, the npub `.fips` hostname returns an AAAA answer (not NXDOMAIN).
2. Browser can open the npub `.fips` URL successfully.
3. `sys-fips` is persistently configured so the behavior survives reboot.
---
## Phase 1 — Validate and fix the network chain in dom0
1. Ensure target AppVM uses `sys-fips` as netvm (not `sys-firewall`/`sys-whonix`/other).
2. Ensure `sys-fips` itself chains to intended upstream netvm (`sys-firewall` or `sys-vpn`).
3. Restart affected VMs after netvm changes so DHCP/DNS lease state is refreshed.
Reference workflow: [`README.md`](README.md:102)
---
## Phase 2 — Bring `sys-fips` services to healthy state
Inside `sys-fips`:
1. Start/enable FIPS services (`fips`, `fips-dns`).
2. Ensure FIPS config has DNS enabled on local resolver port 5354.
3. Ensure `fips0` exists and has IPv6 address.
Config reference: [`configs/fips.yaml`](configs/fips.yaml:6)
---
## Phase 3 — Re-apply DNS forwarding in `sys-fips`
Inside `sys-fips`:
1. Run DNS setup script to make dnsmasq forward `.fips` to local FIPS DNS (`127.0.0.1:5354`).
2. Verify dnsmasq is running and listening on port 53.
3. Verify local forwarding behavior using a `.fips` query.
Implementation reference: [`scripts/06-configure-dns.sh`](scripts/06-configure-dns.sh:1)
---
## Phase 4 — Re-apply IPv6 forwarding/routing for AppVM traffic
Inside `sys-fips`:
1. Enable IPv6 forwarding.
2. Ensure route for `fd00::/8` points to `fips0`.
3. Ensure ip6tables forwarding rules exist and are persisted.
Implementation reference: [`scripts/07-route-appvms.sh`](scripts/07-route-appvms.sh:1)
---
## Phase 5 — Verify mesh reachability from `sys-fips`
Inside `sys-fips`:
1. Confirm at least one reachable peer in FIPS config.
2. Confirm FIPS node has joined mesh and can resolve npub `.fips` locally.
3. If unresolved, add/bootstrap peers and restart FIPS.
Planning reference: [`plans/sys-fips-plan.md`](plans/sys-fips-plan.md:143)
---
## Phase 6 — Verify from AppVM (the failing client)
From the AppVM:
1. Confirm resolver points to `sys-fips`-provided DNS.
2. Query npub `.fips` against current resolver and confirm AAAA response.
3. Confirm no NXDOMAIN for `.fips`.
If still NXDOMAIN, restart AppVM networking context (or reboot AppVM) after Phase 1–4 changes.
---
## Phase 7 — Browser validation (headed)
1. Test npub `.fips` URL in headed browser automation.
2. Test alias hostname variant (if configured in hosts mapping).
3. Capture browser errors; expected fix is removal of name-resolution failure.
Operational context from previous troubleshooting: [`plans/laantungir-tor-fips-plan.md`](plans/laantungir-tor-fips-plan.md:95)
---
## Execution Order (Minimal Downtime)
1. dom0 netvm chain correction
2. `sys-fips` service health (`fips`, `fips-dns`)
3. DNS forwarding script
4. Routing script
5. Peer/mesh verification
6. AppVM DNS verification
7. Browser verification
---
## Expected Outcome
After these steps, the AppVM should resolve the npub `.fips` hostname through `sys-fips` DNS forwarding and reach the remote FIPS-hosted site in browser without `ERR_NAME_NOT_RESOLVED`.

2
scripts/02-install-fips.sh
View File

@@ -25,7 +25,7 @@ echo "Installing system dependencies..."
apt-get update -qq
apt-get install -y --no-install-recommends \
iproute2 iputils-ping dnsutils \
dnsmasq iptables \
dnsmasq iptables nftables \
curl python3 jq \
openssh-server openssh-client \
iperf3 2>/dev/null || {

74
scripts/06-configure-dns.sh
View File

@@ -5,6 +5,9 @@
# Configure dnsmasq in sys-fips so .fips queries are forwarded to local
# FIPS DNS resolver on 127.0.0.1:5354, and all other DNS goes upstream.
#
# Also applies Qubes-specific nftables DNS interception so AppVM DNS queries
# are forced to local dnsmasq in sys-fips.
#
# Usage:
# sudo bash 06-configure-dns.sh
# ==============================================================================
@@ -18,6 +21,23 @@ if [ "$(id -u)" -ne 0 ]; then
exit 1
fi
ensure_nft_rule() {
local table="$1"
local chain="$2"
local rule="$3"
if ! nft list chain "$table" "$chain" >/dev/null 2>&1; then
echo " WARN: nft chain $table $chain not found; skipping: $rule"
return 0
fi
if nft list chain "$table" "$chain" | grep -F -- "$rule" >/dev/null 2>&1; then
return 0
fi
nft add rule "$table" "$chain" $rule
}
systemctl stop dnsmasq 2>/dev/null || true
killall dnsmasq 2>/dev/null || true
@@ -61,7 +81,9 @@ fi
# which can reject DNS queries from downstream AppVMs in this topology.
DNSMASQ_HELPER="/usr/share/dnsmasq/systemd-helper"
if [ -f "$DNSMASQ_HELPER" ] && grep -q -- '--local-service' "$DNSMASQ_HELPER"; then
cp "$DNSMASQ_HELPER" "${DNSMASQ_HELPER}.bak"
if [ ! -f "${DNSMASQ_HELPER}.bak" ]; then
cp "$DNSMASQ_HELPER" "${DNSMASQ_HELPER}.bak"
fi
sed -i 's/--local-service//g' "$DNSMASQ_HELPER"
echo " Patched dnsmasq helper: removed --local-service for Qubes /32 compatibility"
fi
@@ -74,6 +96,56 @@ if ! pgrep -x dnsmasq >/dev/null 2>&1; then
dnsmasq
fi
echo "Applying Qubes nftables DNS interception rules..."
ensure_nft_rule ip qubes dnat-dns 'iifname "vif*" udp dport 53 dnat to 10.139.1.1:53'
ensure_nft_rule ip qubes dnat-dns 'iifname "vif*" tcp dport 53 dnat to 10.139.1.1:53'
ensure_nft_rule ip qubes custom-input 'iifname "vif*" ip daddr 10.139.1.1 udp dport 53 accept'
ensure_nft_rule ip qubes custom-input 'iifname "vif*" ip daddr 10.139.1.1 tcp dport 53 accept'
echo " nftables DNS interception ensured"
RCLOCAL="/rw/config/rc.local"
MARK_START="# === SYS-FIPS DNS START ==="
MARK_END="# === SYS-FIPS DNS END ==="
if [ ! -f "$RCLOCAL" ]; then
mkdir -p /rw/config
cat > "$RCLOCAL" << 'EOF'
#!/bin/sh
EOF
fi
sed -i "/$MARK_START/,/$MARK_END/d" "$RCLOCAL"
cat >> "$RCLOCAL" << 'EOF'
# === SYS-FIPS DNS START ===
# Keep dnsmasq alive with .fips forwarding
if [ -f /usr/share/dnsmasq/systemd-helper ]; then
sed -i 's/--local-service//g' /usr/share/dnsmasq/systemd-helper
fi
systemctl restart dnsmasq >/dev/null 2>&1 || true
pgrep -x dnsmasq >/dev/null 2>&1 || dnsmasq >/dev/null 2>&1 || true
# Qubes DNS interception for downstream AppVMs
nft list chain ip qubes dnat-dns >/dev/null 2>&1 && \
nft list chain ip qubes dnat-dns | grep -F 'iifname "vif*" udp dport 53 dnat to 10.139.1.1:53' >/dev/null 2>&1 || \
nft add rule ip qubes dnat-dns iifname "vif*" udp dport 53 dnat to 10.139.1.1:53 >/dev/null 2>&1 || true
nft list chain ip qubes dnat-dns >/dev/null 2>&1 && \
nft list chain ip qubes dnat-dns | grep -F 'iifname "vif*" tcp dport 53 dnat to 10.139.1.1:53' >/dev/null 2>&1 || \
nft add rule ip qubes dnat-dns iifname "vif*" tcp dport 53 dnat to 10.139.1.1:53 >/dev/null 2>&1 || true
nft list chain ip qubes custom-input >/dev/null 2>&1 && \
nft list chain ip qubes custom-input | grep -F 'iifname "vif*" ip daddr 10.139.1.1 udp dport 53 accept' >/dev/null 2>&1 || \
nft add rule ip qubes custom-input iifname "vif*" ip daddr 10.139.1.1 udp dport 53 accept >/dev/null 2>&1 || true
nft list chain ip qubes custom-input >/dev/null 2>&1 && \
nft list chain ip qubes custom-input | grep -F 'iifname "vif*" ip daddr 10.139.1.1 tcp dport 53 accept' >/dev/null 2>&1 || \
nft add rule ip qubes custom-input iifname "vif*" ip daddr 10.139.1.1 tcp dport 53 accept >/dev/null 2>&1 || true
# === SYS-FIPS DNS END ===
EOF
chmod +x "$RCLOCAL"
echo ""
echo "Verifying DNS..."
if dig @127.0.0.1 google.com A +short +time=2 >/dev/null 2>&1; then

71
scripts/07-route-appvms.sh
View File

@@ -2,7 +2,7 @@
# ==============================================================================
# 07-route-appvms.sh
#
# Configure IPv6 forwarding and ip6tables rules in sys-fips so AppVMs routed
# Configure IPv6 forwarding and firewall rules in sys-fips so AppVMs routed
# through sys-fips can reach FIPS fd00::/8 addresses via fips0.
#
# Usage:
@@ -18,6 +18,23 @@ if [ "$(id -u)" -ne 0 ]; then
exit 1
fi
ensure_nft_rule() {
local table="$1"
local chain="$2"
local rule="$3"
if ! nft list chain "$table" "$chain" >/dev/null 2>&1; then
echo " WARN: nft chain $table $chain not found; skipping: $rule"
return 0
fi
if nft list chain "$table" "$chain" | grep -F -- "$rule" >/dev/null 2>&1; then
return 0
fi
nft add rule "$table" "$chain" $rule
}
sysctl -w net.ipv6.conf.all.forwarding=1 >/dev/null
sysctl -w net.ipv6.conf.all.proxy_ndp=1 >/dev/null 2>&1 || true
sysctl -w net.ipv6.conf.all.disable_ipv6=0 >/dev/null 2>&1 || true
@@ -30,32 +47,56 @@ fi
ip -6 route replace fd00::/8 dev fips0 2>/dev/null || true
echo " route: fd00::/8 -> fips0"
# idempotent rule refresh
ip6tables -D FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT 2>/dev/null || true
ip6tables -D FORWARD -d fd00::/8 -o fips0 -j ACCEPT 2>/dev/null || true
ip6tables -D FORWARD -s fd00::/8 -i fips0 -j ACCEPT 2>/dev/null || true
# Prefer nftables (Qubes-native firewall stack)
echo " ensuring nftables forward rules"
ensure_nft_rule ip6 qubes-firewall forward 'iifname "vif*" ip6 daddr fd00::/8 accept'
ensure_nft_rule ip6 qubes-firewall forward 'iifname "fips0" ip6 saddr fd00::/8 accept'
ip6tables -I FORWARD 1 -m state --state RELATED,ESTABLISHED -j ACCEPT
ip6tables -I FORWARD 2 -d fd00::/8 -o fips0 -j ACCEPT
ip6tables -I FORWARD 3 -s fd00::/8 -i fips0 -j ACCEPT
echo " ip6tables forwarding rules installed"
# Keep ip6tables fallback for templates where FORWARD policy still depends on it.
if command -v ip6tables >/dev/null 2>&1; then
ip6tables -C FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT 2>/dev/null || \
ip6tables -I FORWARD 1 -m state --state RELATED,ESTABLISHED -j ACCEPT
ip6tables -C FORWARD -d fd00::/8 -o fips0 -j ACCEPT 2>/dev/null || \
ip6tables -I FORWARD 2 -d fd00::/8 -o fips0 -j ACCEPT
ip6tables -C FORWARD -s fd00::/8 -i fips0 -j ACCEPT 2>/dev/null || \
ip6tables -I FORWARD 3 -s fd00::/8 -i fips0 -j ACCEPT
echo " ip6tables forwarding fallback ensured"
fi
RCLOCAL="/rw/config/rc.local"
MARK_START="# === SYS-FIPS ROUTING START ==="
MARK_END="# === SYS-FIPS ROUTING END ==="
if [ -f "$RCLOCAL" ]; then
sed -i "/$MARK_START/,/$MARK_END/d" "$RCLOCAL"
if [ ! -f "$RCLOCAL" ]; then
mkdir -p /rw/config
cat > "$RCLOCAL" << 'EOF'
#!/bin/sh
EOF
fi
sed -i "/$MARK_START/,/$MARK_END/d" "$RCLOCAL"
cat >> "$RCLOCAL" << 'EOF'
# === SYS-FIPS ROUTING START ===
sysctl -w net.ipv6.conf.all.forwarding=1 >/dev/null 2>&1
sysctl -w net.ipv6.conf.all.disable_ipv6=0 >/dev/null 2>&1
ip6tables -I FORWARD 1 -m state --state RELATED,ESTABLISHED -j ACCEPT 2>/dev/null
ip6tables -I FORWARD 2 -d fd00::/8 -o fips0 -j ACCEPT 2>/dev/null
ip6tables -I FORWARD 3 -s fd00::/8 -i fips0 -j ACCEPT 2>/dev/null
ip -6 route replace fd00::/8 dev fips0 >/dev/null 2>&1 || true
# nftables forward allow rules for AppVM <-> fips0 traffic
nft list chain ip6 qubes-firewall forward >/dev/null 2>&1 && \
nft list chain ip6 qubes-firewall forward | grep -F 'iifname "vif*" ip6 daddr fd00::/8 accept' >/dev/null 2>&1 || \
nft add rule ip6 qubes-firewall forward iifname "vif*" ip6 daddr fd00::/8 accept >/dev/null 2>&1 || true
nft list chain ip6 qubes-firewall forward >/dev/null 2>&1 && \
nft list chain ip6 qubes-firewall forward | grep -F 'iifname "fips0" ip6 saddr fd00::/8 accept' >/dev/null 2>&1 || \
nft add rule ip6 qubes-firewall forward iifname "fips0" ip6 saddr fd00::/8 accept >/dev/null 2>&1 || true
# ip6tables fallback
if command -v ip6tables >/dev/null 2>&1; then
ip6tables -C FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT 2>/dev/null || ip6tables -I FORWARD 1 -m state --state RELATED,ESTABLISHED -j ACCEPT
ip6tables -C FORWARD -d fd00::/8 -o fips0 -j ACCEPT 2>/dev/null || ip6tables -I FORWARD 2 -d fd00::/8 -o fips0 -j ACCEPT
ip6tables -C FORWARD -s fd00::/8 -i fips0 -j ACCEPT 2>/dev/null || ip6tables -I FORWARD 3 -s fd00::/8 -i fips0 -j ACCEPT
fi
# === SYS-FIPS ROUTING END ===
EOF

36
scripts/test-connectivity.sh
View File

@@ -70,6 +70,42 @@ run_local_tests() {
fail "dnsmasq not running"
fi
if command -v nft >/dev/null 2>&1; then
if nft list chain ip qubes dnat-dns >/dev/null 2>&1; then
pass "nft chain exists: ip qubes dnat-dns"
if nft list chain ip qubes dnat-dns | grep -F 'iifname "vif*" udp dport 53 dnat to 10.139.1.1:53' >/dev/null 2>&1; then
pass "nft DNS DNAT rule present (UDP 53)"
else
fail "nft DNS DNAT rule missing (UDP 53)"
fi
if nft list chain ip qubes dnat-dns | grep -F 'iifname "vif*" tcp dport 53 dnat to 10.139.1.1:53' >/dev/null 2>&1; then
pass "nft DNS DNAT rule present (TCP 53)"
else
fail "nft DNS DNAT rule missing (TCP 53)"
fi
else
skip "nft chain missing: ip qubes dnat-dns"
fi
if nft list chain ip6 qubes-firewall forward >/dev/null 2>&1; then
pass "nft chain exists: ip6 qubes-firewall forward"
if nft list chain ip6 qubes-firewall forward | grep -F 'iifname "vif*" ip6 daddr fd00::/8 accept' >/dev/null 2>&1; then
pass "nft forward rule present (vif* -> fd00::/8)"
else
fail "nft forward rule missing (vif* -> fd00::/8)"
fi
if nft list chain ip6 qubes-firewall forward | grep -F 'iifname "fips0" ip6 saddr fd00::/8 accept' >/dev/null 2>&1; then
pass "nft forward rule present (fips0 -> fd00::/8)"
else
fail "nft forward rule missing (fips0 -> fd00::/8)"
fi
else
skip "nft chain missing: ip6 qubes-firewall forward"
fi
else
skip "nft not installed"
fi
if dig @127.0.0.1 test.fips AAAA +time=2 >/dev/null 2>&1; then
pass ".fips DNS query path reachable"
else