fips/examples/k8s-sidecar

FIPS Kubernetes Sidecar

Run FIPS as a sidecar container in a Kubernetes Pod, injecting the FIPS mesh network interface into every other container in the Pod. All containers in the Pod share the same network namespace, so once the sidecar creates fips0 it is immediately visible to the app container(s) without any further configuration.

Quick Start

# 1. Build the sidecar image
cd examples/k8s-sidecar
./scripts/build.sh --tag fips-k8s-sidecar:latest

# 2. Push to your registry (replace with your own)
docker tag fips-k8s-sidecar:latest registry.example.com/fips-k8s-sidecar:latest
docker push registry.example.com/fips-k8s-sidecar:latest

# 3. Create the identity secret
kubectl create secret generic fips-identity \
  --from-literal=nsec=$(openssl rand -hex 32)

# 4. Deploy
kubectl apply -f pod.yaml

How It Works

In Kubernetes, all containers in a Pod share the same network namespace. The fips sidecar:

1. Generates /etc/fips/fips.yaml from environment variables.
2. Rewrites /etc/resolv.conf to route .fips DNS through dnsmasq (which forwards .fips names to the FIPS daemon's built-in DNS resolver and everything else to the original cluster DNS).
3. Applies iptables isolation rules so that the pod's physical interface (eth0) only carries FIPS transport traffic.
4. Starts dnsmasq and then execs the FIPS daemon.

The app container starts concurrently and immediately sees lo, eth0, and fips0. DNS for <npub>.fips names resolves to fd00::/8 addresses via the dnsmasq → FIPS daemon pipeline.

┌────────────────────────────────────────────────────┐
│ Kubernetes Pod (shared network namespace)           │
│                                                     │
│ ┌─────────────────┐   ┌───────────────────────────┐│
│ │  fips (sidecar) │   │  app container(s)         ││
│ │                 │   │                           ││
│ │  fips daemon    │   │  your workload            ││
│ │  fipsctl        │   │  sees: lo, eth0, fips0    ││
│ │  dnsmasq        │   │                           ││
│ └─────────────────┘   └───────────────────────────┘│
│                                                     │
│ Interfaces:                                         │
│   lo    — loopback (unrestricted)                   │
│   eth0  — pod CNI interface (iptables: FIPS only)   │
│   fips0 — FIPS TUN (fd00::/8, unrestricted)           │
└────────────────────────────────────────────────────┘

Network Isolation

FIPS_ISOLATE controls whether the sidecar locks down eth0:

Value	Behaviour
false (default)	fips0 is added alongside normal cluster networking. eth0 continues to work — services, DNS, other pods, and the internet are all reachable as normal. Application traffic to FIPS mesh peers uses fips0.
true	All traffic on eth0 is dropped except FIPS transport (UDP 2121). The app container can only communicate via fips0. Use this for deployments where the workload must never bypass the mesh.

Common gotcha: if you deploy the sidecar and find that services or other pods are suddenly unreachable, check that FIPS_ISOLATE is not set to true. The mesh-only mode is intentionally strict and will break normal cluster connectivity.

Requirements

Requirement	Notes
NET_ADMIN capability	Required for TUN creation and iptables
/dev/net/tun	HostPath volume mount (see pod.yaml)
Linux kernel ≥ 4.9	Standard on all current distributions
No gVisor / kata-containers	Requires real kernel TUN support

Environment Variables

Variable	Default	Description
FIPS_NSEC	(required)	Node secret key (hex or nsec1 bech32)
FIPS_PEER_NPUB	(empty)	Single peer npub
FIPS_PEER_ADDR	(empty)	Single peer transport address (host:port)
FIPS_PEER_ALIAS	peer	Single peer alias
FIPS_PEER_TRANSPORT	udp	Single peer transport type
FIPS_PEERS_JSON	(empty)	JSON array of peer objects (overrides single-peer vars). Note: the included parser handles simple host:port addresses only; IPv6 addresses with brackets may not parse correctly.
FIPS_UDP_BIND	0.0.0.0:2121	UDP transport bind address
FIPS_UDP_PORT	(from FIPS_UDP_BIND)	UDP port for iptables rules
FIPS_TCP_BIND	(disabled)	TCP transport bind address
FIPS_TUN_NAME	fips0	TUN interface name
FIPS_TUN_MTU	1280	TUN MTU
FIPS_ISOLATE	false	Apply iptables isolation (blocks all eth0 traffic except FIPS transport)
FIPS_POD_IFACE	eth0	Pod physical interface name
FIPS_REWRITE_DNS	true	Rewrite /etc/resolv.conf
RUST_LOG	info	FIPS log level

Multiple Peers

Use FIPS_PEERS_JSON to configure more than one peer:

- name: FIPS_PEERS_JSON
  value: |
    [
      {"npub":"npub1abc...","alias":"gw1","addr":"203.0.113.10:2121","transport":"udp"},
      {"npub":"npub1def...","alias":"gw2","addr":"198.51.100.5:2121","transport":"udp"}
    ]

Each object supports the keys: npub (required), addr (required), alias, transport, priority.

Alternatively, mount a hand-crafted fips.yaml and set FIPS_NSEC to anything (the mount takes precedence because the entrypoint writes to /etc/fips/fips.yaml which is then overridden by the volume):

volumes:
  - name: fips-config
    configMap:
      name: my-fips-config
containers:
  - name: fips
    volumeMounts:
      - name: fips-config
        mountPath: /etc/fips/fips.yaml
        subPath: fips.yaml

JSON Parsing Limitations

FIPS_PEERS_JSON supports parsing with a simple awk-based parser. It handles common host:port addresses but does not support IPv6 addresses with brackets (e.g., [::1]:2121) or values containing embedded commas/colons. For complex configs, install jq in the container and modify the entrypoint, or use a mounted fips.yaml.

Files

File	Purpose
Dockerfile	Builds the sidecar image
entrypoint.sh	Generates config, rewrites DNS, applies iptables, starts fips
pod.yaml	Example Pod manifest (single app container + fips sidecar)
scripts/build.sh	Compiles FIPS and builds the Docker image

Troubleshooting

FIPS_NSEC is required — The secret was not injected. Verify the secretKeyRef name and key match the secret you created with kubectl create secret.

fips0 not visible in app container — Check that the fips sidecar started successfully: kubectl logs <pod> -c fips. The sidecar must start and run the daemon before fips0 appears. Add a startup probe or postStart lifecycle hook to your app container if you need to wait.

iptables errors — The NET_ADMIN capability and /dev/net/tun volume mount are both required. Verify they are present in the Pod spec.

.fips DNS not resolving — Check that FIPS_REWRITE_DNS=true and that dnsmasq started: kubectl exec <pod> -c fips -- pgrep dnsmasq. Verify the FIPS DNS listener: kubectl exec <pod> -c fips -- fipsctl show status.

CNI uses a different interface name — Set FIPS_POD_IFACE to match your CNI's interface name (e.g. ens3, net1 for Multus secondary interfaces).

gVisor / kata-containers — These sandboxed runtimes intercept syscalls and do not support AF_PACKET or /dev/net/tun in the same way as a standard kernel. Use a standard RuntimeClass for FIPS sidecar pods.