fips/docs/design/fips-gateway.md

# FIPS Gateway

The FIPS gateway lets unmodified IPv6 hosts on a LAN exchange traffic
with the mesh without running any FIPS software themselves. It is a
niche feature — most operators will never enable it. The gateway
runs most conveniently on a system that is already providing network
services (DHCP, DNS, RA) to a LAN segment, since hosts on that
segment already get IP assignment and a default route from that box.
The canonical example is an OpenWrt-based WiFi access point: every
client that associates with the AP already has the AP as default
router and DNS server, which is exactly the placement the gateway
needs. The OpenWrt ipk ships with the `gateway:` block of
`/etc/fips/fips.yaml` pre-populated and the integration glue
(dnsmasq forwarding, RA route for the virtual pool, global-scope
IPv6 prefix on `br-lan`) automated by the init script —
[`packaging/openwrt-ipk/files/etc/init.d/fips-gateway`](https://github.com/jmcorgan/fips/blob/master/packaging/openwrt-ipk/files/etc/init.d/fips-gateway).
The operator only needs to enable and start the service. Running the
gateway on a non-OpenWrt LAN-edge host (a Linux router/server, for
example) is technically possible but requires manual integration:
distributing a route to the virtual-IP pool, wiring DNS forwarding so
LAN clients send `.fips` queries to the gateway, configuring sysctls
and capabilities. That path is supported but tedious; it is the
secondary path.

The feature has two halves that share common machinery and have
their own unique parts.

The **outbound half** carries traffic from LAN to mesh. A non-FIPS
LAN workstation resolves `<npub>.fips` (or a `.fips` host alias) via
the gateway's DNS proxy, which returns a virtual IPv6 address from a
managed pool. The kernel routes the LAN packet to that virtual IP via
a route to the pool CIDR (RA-advertised, statically distributed, or
on-link via the default route). The gateway runs nftables NAT so the
packet appears on the mesh as if it had originated from the gateway's
own FIPS identity: prerouting DNAT rewrites the destination from the
virtual IP to the real `fd00::/8` mesh address, and postrouting
masquerade rewrites the source from the LAN host's address to the
gateway's `fips0` address. Return traffic follows the conntrack
reverse path back to the originating LAN host: conntrack reverses the
prerouting DNAT on each reply, restoring the virtual IP as source so
the client sees a response from the address it connected to.

The **inbound half** carries traffic from mesh to LAN. A
configuration entry in `gateway.port_forwards[]` exposes a LAN
service (`host:port`) on a port of the gateway's mesh-side `fips0`
address. Mesh peers reach it as `<gateway-npub>.fips:<listen_port>`.
A prerouting DNAT rule keyed on `(iif=fips0, l4proto, dport)`
rewrites the destination to the LAN target; a LAN-side masquerade in
postrouting rewrites the mesh peer's source so the LAN target sees a
reachable LAN address and conntrack steers replies back through the
gateway. This is the inverse of port-forwarding on a conventional NAT
router.

The two halves are independent and can be configured separately.
Inbound port-forwards work without any outbound configuration (just
a port-forward list and the table); outbound works without any
inbound forwards. They share the same nftables table, the same
binary, the same control socket, and the same atomic-rebuild
strategy. That shared machinery is what makes them halves of one
feature rather than two separate features.

## Architecture

### The `fips-gateway` Service

The gateway is a separate binary, [`fips-gateway`](https://github.com/jmcorgan/fips/blob/master/src/bin/fips-gateway.rs),
not part of the FIPS daemon. It reads the same `/etc/fips/fips.yaml`
the daemon reads (via `--config`, or the standard search path), but
acts on the `gateway.*` block. It needs `CAP_NET_ADMIN` to install
nftables rules, manage proxy NDP entries, and add the pool route.
The CLI is documented in
[../reference/cli-fips-gateway.md](../reference/cli-fips-gateway.md).

The gateway connects to the daemon indirectly. The outbound half
forwards `.fips` DNS queries to the daemon's built-in resolver
(default `[::1]:5354`); the daemon resolves the name to a mesh
address and primes its identity cache as a side effect. The inbound
half does not require any daemon plumbing at all — packets that
arrive on `fips0` after the daemon's TUN injection path are matched
by the nftables rules on `fips0` ingress. There is no shared memory,
no IPC channel, and no startup ordering coupling beyond "the daemon's
DNS responder must be reachable before the gateway starts serving
LAN queries", which the gateway enforces with a bounded reachability
probe at startup.

### nftables Table Layout

All gateway rules live in a single nftables table, `inet
fips_gateway`, with six chains and one set:

- `prerouting` — `type nat hook prerouting priority dstnat (-100)`,
  for both LAN→mesh DNAT (per virtual-IP mapping, matching
  `iifname <lan_interface>`) and mesh→LAN DNAT (per port-forward).
- `output` — `type nat hook output priority dstnat (-100)`, for the
  DNAT of the gateway host's own traffic to a virtual IP (per
  mapping, matching no interface).
- `postrouting` — `type nat hook postrouting priority srcnat (100)`,
  for the `oifname fips0` masquerade of LAN traffic, the `oifname
  fips0` masquerade of the gateway host's own traffic to a mapped
  mesh address, the per-mapping SNAT of new flows sourced from a
  mapped mesh address, and (when any port-forward is configured) the
  LAN-side masquerade for inbound traffic.
- `forward` — `type filter hook forward priority filter (0)`, which
  drops traffic into `fips0` from any interface other than the LAN
  interface unless it belongs to an established or related flow.
- `raw_prerouting` — `type filter hook prerouting priority raw
  (-300)`, which runs before connection tracking and drops traffic
  addressed to the pool that arrives on neither the LAN interface
  nor loopback, and traffic from a mapped mesh address that arrives
  on neither `fips0` nor loopback.
- `input` — `type filter hook input priority filter (0)`, which
  rejects traffic still addressed to the pool when it reaches the
  input hook, that is, traffic no mapping translated, except a
  neighbour solicitation.

The set, `fips_mesh_sources`, holds every mapped mesh address.

The table is rebuilt atomically on every change, in one netlink
batch that the kernel applies as a single transaction: add the
table, delete it, add it again, then the chains and the full rule
set. Because the delete and the recreate share one transaction, the
table never leaves the packet path; the leading add gives the
delete a target when no table exists yet, and a batch the kernel
refuses leaves the previous table in place. Rebuilding the whole
table avoids reliance on kernel rule-handle tracking, which the
rustables crate does not expose. The table holds the two masquerades
into `fips0`, the forward drop, the pool drop, the forged-source drop
and its set, the pool reject and its neighbour-solicitation exception,
three rules per live outbound mapping
(at most 1000 mappings), one rule per inbound forward, and one extra
masquerade when any forward is present.

### Control Socket

`fips-gateway` exposes a Unix-domain control socket at
`/run/fips/gateway.sock` (`root:fips`, mode `0770`) with two
commands: `show_gateway` and `show_mappings`. The protocol is the
same line-delimited JSON used by the daemon's control socket. The
shapes are documented in the
[Gateway command catalog](../reference/control-socket.md#gateway-command-catalog).
There is no `fipsctl gateway` subcommand; clients (including
`fipstop`'s gateway view) talk to the socket directly.

### Diagram

```text
                           LAN clients
                              │
        DNS query (.fips)     │     IPv6 packet
        for outbound          │     to virtual IP
                              │     or mesh peer
                              ▼
            ┌───────────────────────────────────┐
            │           fips-gateway            │
            │                                   │
            │  ┌──────────────┐  ┌───────────┐  │
            │  │   DNS proxy  │  │  Virtual  │  │
            │  │ ([::1]:5365) │─▶│  IP pool  │  │
            │  │   .fips only │  │ (state    │  │
            │  └──────┬───────┘  │  machine) │  │
            │         │          └─────┬─────┘  │
            │         │                │        │
            │  forward to              │ pool   │
            │  daemon resolver         │ events │
            │  ([::1]:5354)            ▼        │
            │         │          ┌───────────┐  │
            │         │          │   NAT     │  │
            │         │          │  manager  │  │
            │         │          │ (rebuild  │  │
            │         │          │  inet     │  │
            │         │          │  fips_    │  │
            │         │          │  gateway) │  │
            │         │          └─────┬─────┘  │
            │         │                │        │
            │         │          ┌─────▼─────┐  │
            │         │          │   net     │  │
            │         │          │  setup    │  │
            │         │          │ (proxy    │  │
            │         │          │  NDP, lo  │  │
            │         │          │  route)   │  │
            │         │          └───────────┘  │
            │         │                         │
            │         │   control socket        │
            │         │   /run/fips/            │
            │         │   gateway.sock          │
            └─────────┼─────────────────────────┘
                      │
                      ▼
              FIPS daemon resolver
              ([::1]:5354)
                      │
                      ▼
                fips0 TUN interface
                      │
                      ▼
                  the mesh
```

The DNS proxy and the virtual IP pool are exclusive to the outbound
half. The NAT manager and the kernel-side machinery (nftables table,
`fips0` and LAN interfaces, conntrack) are shared. The inbound half
contributes per-port-forward rules to the same table without
involving the DNS proxy or the pool.

## The Outbound Half (LAN → Mesh)

### DNS Resolution Flow

1. A LAN client sends a DNS query to the gateway's listener (default
   `[::1]:5365`, configurable via `gateway.dns.listen`). The default
   is not 5353, the mDNS port, which the daemon's LAN rendezvous and
   other mDNS responders hold. It is loopback-only on an unprivileged
   port: the canonical deployment
   has another resolver on the host (dnsmasq, systemd-resolved, BIND)
   holding port 53 and forwarding `.fips` queries to the gateway over
   loopback. Operators on a host without a pre-existing resolver on
   53 can override the listen value to `"[::]:53"` to let LAN clients
   query the gateway directly.
2. The gateway screens the datagram before it reads the question.
   It drops, with no answer, a response (QR set); a datagram not
   shaped like a query (more than one question, any answer or
   authority record, or more than one additional record); one
   shorter than a DNS header, or one that fills the 4096-byte
   receive buffer and so may have been cut short; and anything from
   source port 0, 7, 13, 17, 19 or 37, the UDP services that answer
   arbitrary input, so that a reply cannot start an endless exchange
   with one. A query with an opcode other than `QUERY` gets
   `NOTIMP`, one with no question or a malformed question name gets
   `FORMERR`, and one with a class other than IN or ANY gets
   `REFUSED`. If the question is not for a `.fips` domain, the
   gateway replies `REFUSED`. Every reply carries the client's ID and
   its RD and CD bits. The `NOTIMP` and `FORMERR` replies carry
   nothing else; every other reply carries the client's question
   byte for byte and only the records the gateway produced, nothing
   else from the query. The proxy is intentionally narrow — it does
   not resolve public DNS, and the LAN's primary resolver should hold
   port 53 on the gateway host (the OpenWrt init script wires dnsmasq
   to forward `.fips` queries to the loopback listener
   automatically).
3. The gateway forwards the query to the daemon resolver
   (`gateway.dns.upstream`, default `[::1]:5354`). The daemon must
   match: an IPv6 socket bound to `[::1]` does not accept v4-mapped
   traffic, so a `127.0.0.1:5354` upstream cannot reach a daemon
   bound on `[::1]:5354`.
4. If the daemon is unreachable or times out (5 s), the gateway
   replies `SERVFAIL`. If the daemon answers with an error such as
   `NXDOMAIN`, the gateway relays that response code. A relayed
   `NXDOMAIN` carries an SOA record, as NODATA does, so the client
   may cache it for up to `gateway.dns.ttl`; any other relayed error
   carries the question only. If the daemon answers without an AAAA
   record, the gateway replies `SERVFAIL`.
5. The gateway extracts the AAAA (`fd00::/8`) record from the
   daemon's response. This resolution primes the daemon's identity
   cache as a side effect — a prerequisite for `fips0` routing,
   because the daemon needs the cache entry to map the mesh address
   back to a `NodeAddr` for forwarding.
6. If the client asked for AAAA or ANY, the gateway allocates a
   virtual IP from the pool for that mesh address (idempotent: an
   existing mapping is reused, and its TTL refreshed only if it has
   carried traffic). Any other query type refreshes an existing
   mapping that has carried traffic, creates nothing, and is answered
   with NODATA. At the ceiling of 1000 live mappings, a new name
   replaces the oldest mapping that has never carried traffic.
7. If a new mapping was created, the pool emits `MappingCreated`
   (preceded by `MappingRemoved` for a mapping it replaced), which the
   main loop turns into a command for the NAT worker thread, which
   applies queued changes in one rebuild, and into `add_proxy_ndp` on
   the network setup. A NAT rule can land a few milliseconds after the
   answer.
8. The gateway returns an `AAAA` response containing the virtual IP,
   with the configured TTL (default 60 s) for a mapping that has
   carried traffic, and otherwise with at most the time left until
   TTL after the mapping's creation.

### Virtual IP Pool

The pool allocates IPv6 addresses from a required CIDR (commonly
`fd01::/112`). Each address maps to one mesh destination, keyed by
`NodeAddr` rather than by hostname — different `.fips` aliases for
the same node share a virtual IP. Address 0 (the network-equivalent)
is reserved; the rest are allocatable. The pool is capped at 2^16
addresses regardless of prefix length, to bound memory. A CIDR
written with host bits set, such as `fd01::1/112`, is used as its
prefix, with a warning; a `/128`, which has no address to issue, is
refused at start.

The pool tracks state per address:

```text
Allocated ──→ Active ──→ Draining ──→ Free
    │                       ▲
    └───────────────────────┘
      (TTL expired, no sessions)

Draining ──→ Active      traffic resumes before the grace period ends
Draining ──→ Allocated   a DNS query for the name
```

| State | Meaning |
| ----- | ------- |
| Allocated | DNS query created or renewed the mapping; no NAT sessions yet. |
| Active | Conntrack reports at least one session for this virtual IP. |
| Draining | TTL has expired with no sessions; the grace period is running. |
| Free | Reclaimed and available for new allocations. |

Transitions:

- **Allocated → Active**: conntrack sessions count goes above zero.
- **Allocated → Draining**: TTL expires before any session is
  observed.
- **Active → Draining**: TTL expires after the last session ends.
  Sessions refresh the mapping at every tick, so a mapping in use
  does not drain.
- **Draining → Active**: conntrack reports a session again before
  the grace period ends. The next drain starts a fresh grace period.
- **Draining → Allocated**: a DNS query for a name that has carried
  traffic, with or without an address in the answer. The client may
  now hold a fresh TTL, so reclamation is cancelled: the mapping gets
  the full TTL and, if it stays idle, a fresh grace period.
- **Draining → Free**: the grace period has elapsed since draining
  began with no session seen.

Timing:

- **TTL** (`gateway.dns.ttl`, default 60 s) is both the DNS TTL
  returned to the client and the mapping's idle lifetime. A DNS
  query for a mapped name that has carried traffic refreshes the
  mapping's idle clock, and so do conntrack sessions at each tick.
- **Grace period** (`gateway.pool_grace_period`, default 60 s) is
  the dwell time after the last session ends before the address is
  recycled. It prevents immediate reuse from confusing hosts with
  cached DNS responses.
- **Tick interval**: the pool re-evaluates state every 10 s.

Addresses are handed out in ring order from a moving cursor that
starts at a random address, so an address freed in one run is reused
only after the cursor has gone round the pool.

**Across a restart.** A client may cache an answer for up to the TTL,
so a restarted gateway must not hand that address to a different node.
The gateway keeps, in `/var/run/fips-gateway/pool.json`, how far it
may issue and which addresses a client may still hold an answer for;
the file holds no names, identities or mappings. It is written only to
tmpfs, so a reboot, which ends every client's cache with it, starts
from nothing. The first write with a stretch comes after the start
steps that can fail and before any name is answered. A start that
read a file writes what it holds back at once, with no stretch, so a
start that fails or is killed before its first write leaves the same
holds for the next one. The pool writes the
file before issuing past the stretch the last write covered (about 512
addresses at a time), off the DNS thread, and refuses new names for
the moment the cursor reaches that point before the next write is
confirmed. On a clean stop it writes the exact set of live addresses
with no stretch ahead. The next start reads the file, removes it, and
holds every address it names, and the stretch, for the TTL plus the
grace period, then issues from where the stretch ends. A write that
fails removes the file, and the gateway carries on issuing; the next
start then begins at a random address. Where `/var/run` is not on
tmpfs, as in most containers, the gateway warns once and keeps no
state.

After a crash or a kill, the last written stretch is held, which on a
pool with fewer than 512 free addresses (a `/119` or narrower, or a
`/118` with more than about 500 addresses in use) is every free address:
new names are refused for the TTL plus the grace period, 2 minutes at
the defaults. A clean stop has no such
cost. A pool resized with the same base address starts at a random
address with nothing held.

Active session counts come from `/proc/net/nf_conntrack`, or, on a
kernel without that file, from a dump of the IPv6 conntrack table over
`NETLINK_NETFILTER`, the request `conntrack -L` makes. The choice is
made on every tick, and the source is logged once at startup. Either
way an entry counts once toward each distinct IPv6 destination among
its original and reply tuples, so an entry counts as a session of a
virtual IP whose address is its original destination.

**Evidence of use.** A mapping counts as having carried traffic once a
conntrack entry for a flow to its virtual IP has seen a reply whose
source is the mapped mesh address. A reply from anywhere else does not
count: not from the virtual IP itself (an unmapped pool address is
local to the gateway, so a reply from it comes from the gateway, not
the mapped node; the input chain's reject of traffic to it is an
ICMPv6 error sent from the gateway's own address, not from the virtual
IP), not from an earlier holder of the address, and not from a host
forging the mesh address, since the `raw_prerouting` chain drops
packets from any mapped mesh address (the `fips_mesh_sources` set)
unless they arrive on `fips0` or loopback. An ICMPv6 error is not a
reply. The gateway routes its whole pool prefix to itself, so a name
whose mesh address lies inside that prefix is refused with `SERVFAIL`:
its traffic could never reach the mesh, and the gateway's own replies
from the address would read as the mapping's traffic. Only a mapping
that has carried traffic is refreshed by re-queries and answered with
the full TTL; a name nobody has used expires TTL plus grace after it
was created however often it is queried, and its answers never outlive
TTL after creation (TTL 0 if unreplied traffic keeps it mapped past
that). The flag is never cleared.

When the gateway cannot read conntrack at start (it warns that session
pinning is off), or reads fail for three ticks in a row (it warns
then), it stops trusting conntrack until three reads in a row succeed.
Mappings created in between can never count as having carried traffic:
re-queries do not extend them and they can be replaced at the ceiling.
Mappings that already counted keep their protection. At start the
gateway takes three reads before it answers any name, so that the
conntrack entries the previous run left are known before an address is
reissued; a mapping created before three reads have succeeded never
counts as having carried traffic.

**Limits.** A new mapping is refused, and the query answered
`SERVFAIL`, when the pool is exhausted, when the new-mapping rate limit
is spent (a bucket of 50 that refills at 10 per second), or when it
already holds 1000 live mappings that have all carried traffic. At
the ceiling a new name otherwise replaces the oldest mapping that has
never carried traffic; a client still holding an answer for the
replaced address then has its connections to it rejected, as for any
unmapped pool address, until that answer expires. A name that already
has a mapping keeps resolving while new names are refused. The first
refusal at a bound logs one warning; later ones are counted and
reported when the pool has refused nothing for 60 s. A pool smaller
than 1000 + 10 x (TTL + grace) addresses (2,200 at the defaults) can
be exhausted by one LAN host naming new names; the gateway warns at
start when it is. A host that can draw real replies from many mesh
destinations, such as nodes of its own, keeps that many mappings for
as long as it re-queries them, and with 1000 of them refuses new names
for everyone.

**Release.** A removed or replaced address returns to the free set
only once no answer naming it can still be cached (its creation plus
TTL plus grace, or its last answer plus grace, whichever is later),
the NAT table has reported its rules gone, and three conntrack reads
have followed. It is never handed back to the same node while a
conntrack entry from its old mapping survives, since nothing deletes
such entries and a forged reply on one would read as the new
mapping's traffic.

### NAT Pipeline (Outbound)

Three rule classes in `inet fips_gateway` together implement the
LAN→mesh path, two more carry the gateway host's own traffic to a
virtual IP, and two filter rules keep every other interface off it:

**Prerouting DNAT (per mapping)** rewrites the destination of LAN
traffic from the virtual IP to the corresponding mesh address:

```text
match:  iifname == <lan_interface> && nfproto ipv6 && ip6 daddr == <virtual_ip>
action: dnat to <mesh_addr>
```

After DNAT, the kernel routes the packet through `fips0` via the
standard routing table.

**Postrouting masquerade (`oifname fips0`)** rewrites the source of
LAN traffic exiting via `fips0` to the gateway's own `fips0` address:

```text
match:  iifname == <lan_interface> && oifname == "fips0"
action: masquerade
```

This rule is critical. Without it, LAN client source addresses (for
example `fd02::20` from the LAN's RA-advertised prefix) would appear
as the source on the mesh. Those addresses are meaningless to mesh
nodes, so return traffic would be black-holed. Masquerade ensures
LAN traffic appears on the mesh under the gateway's own FIPS
identity.

**Output DNAT (per mapping)** does for the gateway host's own traffic
what the prerouting DNAT does for the LAN's. Locally generated
packets pass the output hook, not prerouting, and every pool address
is local to the gateway through the `local <pool> dev lo` route, so
without this rule a connection the gateway host makes to a virtual IP
reaches the gateway itself:

```text
chain:  output
match:  nfproto ipv6 && ip6 daddr == <virtual_ip>
action: dnat to <mesh_addr>
```

The kernel reroutes the translated packet through `fips0`. The
`nfproto` match keeps the host's IPv4 traffic, which also passes this
`inet` chain, out of it.

**Host masquerade** gives that traffic the gateway's `fips0` address
as its source. The host may have chosen another source before the
reroute, such as its LAN address, which mesh nodes cannot answer:

```text
chain:  postrouting
match:  nfproto ipv6 && oifname == "fips0" && ip6 daddr in fips_mesh_sources
action: masquerade
```

The only flows that are new in `postrouting` with `oifname fips0` and
a mapped mesh destination are LAN flows, which the LAN masquerade
takes first; flows forwarded from any other interface, which the
forward drop has already dropped; and the host's own. A host flow
sent straight to a mapped node's mesh address already has the
`fips0` address as its source, so masquerading it changes nothing.

**Forward drop** keeps hosts on the gateway's other interfaces from
sending into the mesh:

```text
chain:  forward
match:  nfproto ipv6 && oifname == "fips0" && iifname != <lan_interface>
        && ct state not in { established, related }
action: drop
```

Packets of established or related flows still pass, such as replies
to flows a mesh peer opened and flows established before the rule was
installed; new, invalid and untracked packets do not.

**Pool drop** keeps them off the virtual IPs:

```text
chain:  raw_prerouting (before connection tracking)
match:  nfproto ipv6 && iifname != <lan_interface> && iifname != "lo"
        && ip6 daddr in <pool>
action: drop
```

It runs before conntrack because conntrack matches a packet to an
existing flow whatever interface it arrives on: a host that knows a
LAN flow's addresses and ports could otherwise send in that flow's
original direction and have it translated and forwarded. The
gateway host's own connections to a mapped virtual IP are translated
in `output` and leave on `fips0`, so they never reach this rule.
Loopback is exempt so that the gateway host's own connections to a
pool address with no mapping reach the pool reject below, as a LAN
client's do.

**Pool reject** keeps traffic to a pool address that has no mapping
(an expired one, or an address written into a configuration by hand)
from reaching whatever listens on the gateway host, such as its SSH
server:

```text
chain:  input
match:  nfproto ipv6 && ip6 daddr in <pool>
        && icmpv6 type == nd-neighbor-solicit
action: accept
match:  nfproto ipv6 && ip6 daddr in <pool>
action: reject (ICMPv6 port unreachable)
```

Every pool address is local to the gateway, and a mapping's DNAT
rewrites the destination before the input hook (LAN traffic in
`prerouting`, the host's own in `output`, and a flow to the gateway's
own node to its mesh address), so whatever reaches `input` still
addressed to the pool has no mapping. It covers the LAN and the
gateway host alike. The one exception is a neighbour solicitation: a
LAN client that reached a virtual IP by proxy NDP checks the neighbour
with a unicast solicitation addressed to the virtual IP, connection
tracking leaves neighbour discovery untracked so no DNAT rewrites it,
and proxy NDP must still answer it.

**Postrouting SNAT (per mapping)** rewrites the source of a new flow
sourced from a mapped mesh address to that mapping's virtual IP:

```text
match:  nfproto ipv6 && ip6 saddr == <mesh_addr>
action: snat to <virtual_ip>
```

NAT rules are consulted only for the first packet of a flow, so this
rule does not touch the replies of a connection a LAN client opened.
Those are translated by conntrack reversing the prerouting DNAT, which
is what makes the LAN client see replies from the virtual IP it
connected to rather than from the raw `fd00::/8` mesh address.

### Network Requirements (Outbound)

The gateway host needs IPv6 forwarding enabled
(`net.ipv6.conf.all.forwarding=1`), proxy NDP enabled on the LAN
interface, `CAP_NET_ADMIN` for `fips-gateway`, and a `local
<pool-cidr> dev lo` route so the kernel accepts packets to the pool
as locally owned and runs them through the NAT chains. LAN clients
need a route to the pool via the gateway and DNS resolution that
forwards `.fips` queries there. On OpenWrt the init script handles
all of this; on other Linux hosts the operator handles it manually.
Full setup is documented in
[../how-to/deploy-gateway.md](../how-to/deploy-gateway.md).

## The Inbound Half (Mesh → LAN)

### Configuration Shape

Inbound port-forwards live in `gateway.port_forwards[]`. Each entry
is a triple:

| Field | Type | Notes |
| ----- | ---- | ----- |
| `listen_port` | `u16` | Port on the gateway's `fips0` address. Must be non-zero. |
| `proto` | `tcp` \| `udp` | Match protocol. |
| `target` | `[ipv6]:port` | LAN destination. IPv4 targets are rejected at parse time by `SocketAddrV6`. |

Validation runs at startup (the gateway has no in-place reload; it
handles only SIGTERM and SIGINT, and a reload of the service restarts
it):
`(listen_port, proto)` must be unique across the list, and zero
listen ports are rejected. Forwards are independent of outbound
configuration: a gateway with no `pool` consumers can still expose
inbound services (the pool route and DNS proxy still run, since they
are part of the same binary, but they sit idle).

### NAT Pipeline (Inbound)

For each port-forward, a single prerouting DNAT rule matches
mesh-originated traffic landing on the gateway's `fips0` address
and rewrites it to the LAN target:

```text
match:  iifname == "fips0" && nfproto ipv6
        && l4proto == <tcp|udp> && th dport == <listen_port>
action: dnat to <target_ip>:<target_port>
```

The match clause is deliberately narrow:

- **`iifname == "fips0"`** restricts the rule to traffic that
  arrived from the mesh. LAN-side ingress is never subject to
  inbound forwarding.
- **`nfproto ipv6`** is enforced both here and at config-load time
  (`SocketAddrV6` rejects IPv4 targets); FIPS is IPv6-only end to
  end.
- **`l4proto + dport`** narrows the match to one
  `(listen_port, proto)` pair per rule. Unique-tuple validation
  ensures no two rules contend for the same packet.

When *any* port-forward is configured, a single LAN-side masquerade
is added to postrouting:

```text
match:  iifname == "fips0" && oifname == <lan_interface>
        && nfproto ipv6
action: masquerade
```

Without this rule, the LAN target would attempt to reply directly to
the mesh peer's `fd00::/8` source address, which is not reachable on
the LAN. Masquerade rewrites the source to the gateway's LAN-side
address so the target sees a reachable peer and conntrack routes
the reply back through the gateway.

This LAN-side masquerade is independent of the `oifname fips0`
masquerade in the outbound pipeline; the two have disjoint match
clauses (different `iifname`/`oifname` combinations) and coexist
without interaction when both directions are active.

The per-mapping SNAT rules match on source address alone, so an
inbound forwarded connection from a mesh peer that also holds a live
mapping matches both its SNAT and the LAN-side masquerade. NAT
statements are terminal, and the rebuild places the LAN-side
masquerade ahead of every per-mapping SNAT, so that peer's connection
reaches the LAN target from the gateway's LAN address like any other.

### Independence From Outbound

The inbound half does not require:

- A virtual-IP pool. Mesh peers connect directly to the gateway's
  own `fips0` address, which the FIPS daemon already owns.
- DNS resolution. Mesh peers reach the gateway as
  `<gateway-npub>.fips:<port>` using their own resolver (or a
  numeric mesh address); the gateway's DNS proxy is not in the path.
- A daemon-side identity cache for the LAN target. The target is a
  LAN-side IPv6 address, not a mesh address; no `fd00::/8` lookup
  happens for it.

A gateway configured with port-forwards but with no LAN clients ever
issuing `.fips` DNS queries will have an empty pool and zero
outbound mappings, but its inbound forwards work normally. The
inverse is also true: a gateway that serves only outbound LAN→mesh
traffic has zero entries in the port-forwards list and no LAN-side
masquerade.

## Atomic Table Rebuild (Common)

Both halves contribute rules to the same `inet fips_gateway` table,
and that table is rebuilt as one unit on every state change —
mapping added, mapping removed, port-forwards updated. The rebuild
sequence is:

1. Add the table (which succeeds whether or not it exists), delete
   it, and add it again, so the delete always has a target.
2. Add the six chains and the mesh-source set with its elements; the
   pool drop, the forged-source drop and the forward drop; the
   neighbour-solicitation accept and the pool reject; the two
   `oifname fips0` masquerades, LAN first; the LAN-side masquerade if
   any port-forwards exist; per-mapping prerouting DNAT, output DNAT
   and SNAT rules for every live pool entry; per-port-forward DNAT
   rules.
3. Send all of it as one batch, which the kernel applies as a single
   transaction. Only the last message before the batch end requests
   an acknowledgement, the socket's send buffer is sized to the
   batch, and a batch too large for any send buffer is refused
   before it is sent. A batch the kernel refuses leaves the previous
   table in place; the failure is logged, and the gateway keeps its
   record of the change. Pending changes are retried every ten seconds,
   even when no new mapping event arrives. Each retry rebuilds the latest
   desired state, so a newer change supersedes an earlier failed one.
   Successful rebuilds clear the pending flag; clean tables are not retried.

Rebuilds run on a NAT worker thread that owns the table, never on the
thread that answers `.fips` queries: a rebuild is a netlink round trip
that can wait up to five seconds for the kernel. The worker drains every
queued change into one rebuild. If it stops for any reason the gateway
exits non-zero so the service manager restarts it, rather than answer
with addresses that have no rules. Repeated failures to apply NAT rules
or proxy NDP entries log one warning or error per change of outcome,
with the repeats at debug level. Each mapping change forks
`ip -6 neigh` once, and a replaced mapping twice, so proxy NDP forks
at most 20 times a second at the admission rate. The main loop makes
these changes one at a time, and the DNS task hands them over through
a queue of 64, so a burst of new names at the ceiling (up to 50, each
replacing a mapping) can hold `.fips` answers back while the forks
drain: 100 forks took 0.26 s on an x86 host, and router hardware is
slower.

The rustables crate does not expose rule-handle tracking, so
incremental update of individual rules is not available. Atomic
rebuild was chosen for simplicity and correctness: it eliminates an
entire class of partial-update inconsistency bugs at the cost of
repeating the (cheap) rule construction on every change. The total
rule count is bounded by the live-mapping ceiling (1000 mappings,
two rules each) and the port-forward count.

## Configuration Reference

The full `gateway.*` block — pool CIDR, LAN interface, DNS
listen/upstream/TTL, pool grace period, conntrack timeouts, and
inbound port-forwards — is documented in the
[Gateway section](../reference/configuration.md#gateway-gateway)
of the configuration reference. The same block governs both halves;
fields specific to one half (`pool`, `dns.*` for outbound;
`port_forwards[]` for inbound) are simply unused when the other
half is not in play.

## Operations and Troubleshooting

- [../tutorials/deploy-fips-gateway.md](../tutorials/deploy-fips-gateway.md)
  — end-to-end walkthrough on OpenWrt.
- [../how-to/deploy-gateway.md](../how-to/deploy-gateway.md) —
  recipe for non-OpenWrt Linux hosts and inbound-port-forwarding
  configuration.
- [../how-to/troubleshoot-gateway.md](../how-to/troubleshoot-gateway.md)
  — diagnostic recipes (DNS failures, ping working but TCP not,
  conntrack inspection, pool exhaustion, port-53 conflicts,
  port-forward verification).
- [../reference/cli-fips-gateway.md](../reference/cli-fips-gateway.md)
  — command-line interface.
- [../reference/control-socket.md](../reference/control-socket.md#gateway-command-catalog)
  — `show_gateway` and `show_mappings` commands.

## Security Considerations

### Outbound

- **LAN trust boundary.** The DNS listener and the virtual-IP pool are
  reachable by every host on the LAN. Any LAN host that can resolve
  `.fips` and route to the pool CIDR can reach mesh destinations.
  There is no per-client authentication; access restriction is a
  network-level concern, enforced with firewall rules on the LAN
  interface or on the gateway host itself. The DNS listener does not
  answer queries that arrive on `fips0`, so a wildcard listen does not
  open the pool or the host map to the mesh; a local resolver that
  forwards `.fips` to the gateway's `fips0` address is refused the same
  way. Only traffic arriving on
  `lan_interface`, and the gateway host's own, is translated onto the
  mesh. Traffic into `fips0` from any other interface (a WAN port, a
  VPN interface, a container or VM bridge on the gateway host) is
  dropped unless it is a reply, and traffic to the pool from any other
  interface is dropped before connection tracking. The rules assume
  the TUN interface is named `fips0`. Three forms remain open, as they
  were before these rules. A host on another interface that knows the
  addresses and ports of a LAN flow sent straight to a mesh address
  (with no virtual IP) can inject packets in that flow's original
  direction, and can forge its replies, which the gateway then
  forwards to the LAN client. It can also forge the replies of an
  inbound port-forward flow, sent from the forward's target to the
  gateway's LAN address and the flow's masqueraded port (usually the
  mesh peer's own source port): connection tracking matches them as
  established, and the gateway forwards them to the mesh peer as if
  from the forwarded service. Dropping those would also break port
  forwards whose target is not on the LAN interface. Flows from other
  interfaces that were established before an upgrade to these rules
  continue until connection tracking expires them (`conntrack -F` or a
  reboot cuts them at once). A `lan_interface` that is a VLAN device
  on a bridge (for example `br-lan.1` under bridge VLAN filtering)
  does not serve clients whose frames enter the bridge untagged when
  `br_netfilter` is loaded and runs the IPv6 hooks for bridged
  traffic (`net.bridge.bridge-nf-call-ip6tables` 1, its default, or
  the bridge's `nf_call_ip6tables` option on). It runs the IPv6
  prerouting hooks at the bridge, with the parent bridge as the input
  interface and before the bridge assigns the port's VLAN, so LAN
  clients' DNAT never matches and the pool drop discards their
  traffic to the pool. `net.bridge.bridge-nf-pass-vlan-input-dev`
  does not help them, because their frames carry no tag at that
  point. Frames that arrive tagged skip these hooks unless
  `net.bridge.bridge-nf-filter-vlan-tagged` is 1, and are then served
  only with `bridge-nf-pass-vlan-input-dev` 1. With
  `bridge-nf-call-ip6tables` 0 and the bridge option off, or without
  `br_netfilter`, the gateway serves the VLAN device. The OpenWrt
  package sets `bridge-nf-call-ip6tables` to 0.
- **Identity masking.** All outbound LAN traffic appears on the
  mesh under the gateway's own FIPS identity. Mesh nodes cannot
  determine which LAN host originated a connection. This provides
  privacy for LAN hosts but means the gateway's reputation covers
  all of its clients — and that abusive behavior from one LAN host
  is attributed to the gateway, not to the host.
- **Plaintext between client and gateway.** Traffic between the LAN
  client and the gateway is unencrypted at the IP layer. FIPS
  encryption (FSP) protects the segment between the gateway and the
  destination mesh node; application-layer encryption (TLS, SSH,
  Noise) is the only thing that provides true end-to-end protection
  through the gateway.
- **Pool addresses are ephemeral.** Virtual IPs are allocated
  dynamically and recycled. They are not authenticated and not
  bound to client identity — a LAN host connecting to a virtual IP
  is trusting the gateway's recent DNS response.
- **DNS upstream trust.** The outbound half's correctness depends
  on the FIPS daemon's resolver returning honest `fd00::/8`
  answers; a compromised daemon could redirect LAN clients to
  arbitrary mesh nodes.

### Inbound

- **Port exposure.** Each entry in `port_forwards[]` exposes the
  matched `(listen_port, proto)` on the gateway's mesh-side
  address to every reachable mesh peer. Inbound port-forwards are
  not gated by any peer ACL beyond what FMP normally enforces;
  treat them with the same care as a public-internet port forward.
  The mesh firewall baseline's `forward` chain accepts them unless
  a drop-in under `/etc/fips/fips.d/forward/` restricts a forward
  to chosen mesh addresses (see
  [deploy-gateway.md](../how-to/deploy-gateway.md#compose-with-the-mesh-firewall)).
- **Mesh peer trust.** The LAN target sees connections that have
  been masqueraded to the gateway's LAN address. The target cannot
  distinguish one mesh peer from another, and there is no
  authenticated peer identity available to the LAN target — any
  application-layer authentication or rate-limiting must run on
  the target itself.
- **Return-path masquerade exposes the gateway's LAN address.**
  The LAN-side masquerade rewrites the mesh peer's source to the
  gateway's LAN address. A malicious or buggy LAN target can use
  this to send unsolicited traffic back at the gateway, or to
  probe other LAN hosts via the gateway's network position; LAN
  segmentation (VLANs, host firewalls) is the right control.

### Common

- **No client identity verification.** The gateway authenticates
  neither LAN clients nor mesh peers beyond what the underlying
  layers already do — `fips0` ingress carries an FSP-authenticated
  payload, the LAN side is whoever the LAN admits.

## References

- [fips-ipv6-adapter.md](fips-ipv6-adapter.md) — IPv6 adapter and
  TUN interface design.
- [fips-architecture.md](fips-architecture.md) — protocol layer
  architecture.
- [fips-concepts.md](fips-concepts.md) — protocol overview.
- [../reference/configuration.md](../reference/configuration.md) —
  configuration reference.
- [../reference/cli-fips-gateway.md](../reference/cli-fips-gateway.md)
  — `fips-gateway` CLI.
- [../reference/control-socket.md](../reference/control-socket.md) —
  control-socket protocol and command catalog.
- [../how-to/deploy-gateway.md](../how-to/deploy-gateway.md) —
  gateway host and LAN client setup.
- [../how-to/troubleshoot-gateway.md](../how-to/troubleshoot-gateway.md)
  — diagnostic recipes.
- [../tutorials/deploy-fips-gateway.md](../tutorials/deploy-fips-gateway.md)
  — OpenWrt walkthrough.