3 Commits

Include renames

Author	SHA1	Message	Date
Johnathan Corgan	462b9daf0a	docs(native): correct the end-of-file guidance given to client authors ... The how-to told an author writing a client in another language that a queued empty datagram leaves the socket with no events pending, while a closed peer sets POLLHUP. That is false, and a client written to it reproduces the defect the receive path has just had fixed, in their code rather than ours. It also told them to poll with an events mask of zero. The library stopped doing that when the macOS port landed, because a poll that requests nothing registers no filter on Darwin and returns no events for a peer that has in fact closed. A client following the old instruction would never detect a close on macOS at all. The reference page carried both errors and asserted that a zero-byte read is an empty datagram and only that. Both pages now describe what the code does. Request POLLIN. Treat a hang-up as end of file only when FIONREAD reports nothing queued behind the read. Design around the one case neither check can resolve, by never giving a zero-length payload a meaning of its own.	2026-08-30 08:27:16 +00:00
Johnathan Corgan	1ec62e5119	Build the native datagram API on macOS, over SOCK_DGRAM ... macOS does not implement SOCK_SEQPACKET for AF_UNIX, so the listener was gated to Linux and FreeBSD and a Mac got no API at all. It now uses SOCK_DGRAM there, which macOS does implement and which keeps the message boundaries the API's contract with its clients rests on. Both kernels were measured rather than reasoned about, and the Linux answer alone refuted the replacement the source had proposed. On Linux 6.8 a connected SOCK_DGRAM pair reports a closed peer not at all: revents stays empty and recv returns EAGAIN, which is exactly what an idle socket with a live peer does. SOCK_SEQPACKET on the same kernel sets POLLHUP and returns a zero-byte read, which is what the receive path keyed on. Darwin does report the close, with ECONNRESET, errno 54, and does not set POLLHUP. So the receive path treats ECONNRESET as end of file alongside the existing POLLHUP rule. One rule accepting either signal is correct on both kernels, where a rule split by platform would be silently wrong on whichever one it guessed at. EAGAIN is deliberately not in that company: it means the socket is empty and the peer alive, so it stays an error and the caller waits again. The measurements are asserted rather than only written down, so a kernel that gains or loses the signal reds a test and reopens the question instead of leaving a stale comment behind. Each carries its result in the assertion message, since a passing test prints nothing and a negative result would otherwise be as uninformative as no result: the close probe reports the poll return, the whole revents bitmask broken out by flag, the recv result and the errno, which is enough to write the real rule without another round trip. A portable test walks datagram sizes upward, because Darwin bounds a unix-domain datagram with the net.local.dgram.maxdgram sysctl, whose default is small and which is a system tunable rather than something this process controls, while the API advertises 1362 bytes to its clients. Every test recv in the seqpacket suite is bounded in time. This is not tidying. Simulating the Darwin configuration on a kernel that does not report the close, the end-of-file test hung for over ten minutes rather than failing, and a hang is not a red: it would have wedged the macOS runner with no diagnostic instead of naming the assertion. The same simulation now fails by name in five seconds. Three things in the native tree compiled on one platform only, all of them in code that had never been built for Darwin before. suseconds_t is i64 on Linux and i32 there, so the timeval microseconds field is a cast, matching the tv_sec line above it; it cannot truncate, because subsec_micros is below 1_000_000 by construction, and the cast is the only form that compiles on both, since From does not exist for the narrower width and try_from is a clippy error on the wider one. MSG_CMSG_CLOEXEC does not exist on Apple, so the recvmsg flags are chosen per platform and each received descriptor is marked close-on-exec with fcntl where there is no flag to pass; a failure to set it is reported rather than ignored, since the descriptor is live either way and the caller must not be told the receive was clean. socketpair takes SOCK_CLOEXEC in its type argument on Linux and FreeBSD and rejects it on macOS, so Darwin sets FD_CLOEXEC with a second fcntl. Both windows between a call and its fcntl are stated in the code rather than closed, since the daemon spawns no child on this path, and a test asserts both halves of a pair are close-on-exec on every platform, because the failure is a silent descriptor leak into a child and nothing else would report it. Every libc item the native tree uses was then checked against the crate's own Apple definitions rather than from memory, and those two constants are the only ones absent. The close difference turned out to be unhandled in six further places, and the whole native API agrees on it now. Darwin reports a closed AF_UNIX SOCK_DGRAM peer as ECONNRESET, and a later send on the disconnected survivor as EDESTADDRREQ, where Linux SOCK_SEQPACKET gives EPIPE on a write and a zero-byte read plus POLLHUP on a read. Each site below promised one of those spellings and saw another. - The client's recv and send passed ECONNRESET through, so a closed daemon half surfaced as errno 54 against the EPIPE the documentation promises. The translation is in one function in seqpacket rather than at each call site, since only this one condition has two spellings. - accept propagated the same errno instead of its documented EPIPE. The listener's read reports a closed peer as the empty chunk both of its callers already read as the far end going away, which leaves accept's contract true on both platforms without either caller knowing which it is on. - why() classified a failed hand-off by BrokenPipe alone, so every ordinary macOS listener close was counted under the counter an operator reads to find a client that stopped reading. It recognises all three errnos now, with a test over each. - A full client buffer ended a flow's only writer. On Linux that never arrives, because the send reports EAGAIN and waits for the client to drain; Darwin has no sender-side queue to wait on and reports ENOBUFS on the send itself. Returning left the registration, the port and the reader alive while every later inbound datagram was counted as a full queue for the rest of the flow's life, and a client that resumed reading never recovered. The datagram is dropped instead, which is what a datagram API does when the far end cannot take it. - The flow pair was never sized, and the two kernels charge a queued message to different ends: Linux to the sender's SO_SNDBUF, BSD to the receiver's so_rcv. Sizing only the sender, as the listener pair does, left the flow pair bounded on Darwin by a system default small enough that a batch held for an arriving client could not fit, and the whole flow was destroyed before its client ever saw it. Both halves are sized now. - peer_hung_up polled with an empty events field, on the rule that POLLHUP is reported whether or not it is requested. That holds on Linux, where it was measured, and not on Darwin, where a poll requesting nothing registers no filter. Nothing observable depended on it, because ECONNRESET arrives first and both callers act on it earlier. The cost was elsewhere: three assertions written as tripwires for a change in Darwin's behaviour could not fail there, which is a guard that executes and proves nothing. Requesting POLLIN fixes the function and the guards together. One difference is not an errno at all, and reading the kernel source rather than a manual page is what found it. Darwin's unp_disconnect sets SS_CANTRCVMORE and runs soisdisconnected on both ends for SOCK_STREAM. For SOCK_DGRAM it removes the reflink, clears SS_ISCONNECTED and stops: no sorwakeup, no socantrcvmore, no soisdisconnected. The closing peer deposits ECONNRESET in the survivor's so_error and wakes no knote. The registration is edge-triggered and was made while the socket was healthy, so nothing re-evaluates it, and recv awaited readiness before its syscall, which left the ECONNRESET arm sitting behind an await that never returns. A client closing its descriptor left the daemon's reader parked for ever, and the flow's port and registry entry held for the node's lifetime. recv reads before it waits now, because the latched error is visible to a syscall and only to a syscall, so the attempt that precedes the wait is what sees a close that has already happened. A close can also land while the task is parked, which no first attempt can catch, so on Darwin the wait is bounded and the syscall retried; the error is latched until a read consumes it, so the bound sets how long a dead flow holds its port rather than deciding whether the close is seen at all. On Linux this is one extra recv returning EAGAIN before the wait and changes nothing else, and everywhere else the readiness is authoritative and the wait stays unbounded. The three tests this predicted are the three that had failed: end of file on a closed client half, a listener's port unbound on close, and one flow's port freed while its connection stays open. One test asserted a delivery detail rather than the rule it exists to guard. a_descriptor_lands_on_the_last_complete_line_of_the_read_that_carried_it asserted that a plain write and the sendmsg following it arrive in one recvmsg. Linux coalesces them, so the read returns both lines and the descriptor together; Darwin stops a stream read at the ancillary boundary, so the plain line arrives by itself and the descriptor-bearing line comes on the next read. The rule the module rests on is unaffected, and Darwin satisfies it more easily than Linux, because the read it arrives on holds nothing later. The test fills until both lines are queued and asserts the rule instead of the number of reads it took. The client compiled in /run/fips/api.sock on every platform, and macOS has no /run for that path to be in. The daemon never had this problem: it resolves its socket at startup by looking for a directory, and its macOS branch lands on /var/run/fips. The constant is conditional the same way now, so a client that is told nothing looks where a packaged daemon on its own platform actually is. The reference documentation described that branch as FreeBSD-only and describes both. Windows stays excluded and cannot be included: it has no SCM_RIGHTS, so there is no way to pass a descriptor to another process at all, which is the whole mechanism rather than a detail of it. The platform statements in the source and in the shipped documentation all named Linux and FreeBSD and name macOS now, including the configuration reference, the security reference, the how-to and the walkthrough. The how-to also states how far the testing goes, because the person who would meet the gap first is the one enabling the API on a Mac. The end-to-end suite drives a client container against a node container over a shared volume, which is a Linux arrangement, so the socket lifecycle, the descriptor hand-off across a process boundary and the reclaiming of a port when a client exits are covered on macOS by unit tests rather than by anything that runs a daemon and a client as two real processes. That is a gap in testing and not a known defect, and it is a coverage gap rather than a discharged risk. The same place names the socket-type difference, since a reader who knows the descriptor is SOCK_DGRAM there can make sense of a close arriving as a different errno than the Linux documentation elsewhere describes. The changelog entry for the API is revised rather than followed by a second one: it now names the socket type each platform uses and the two end-of-file signals the receive path accepts. The entry describes what the release ships rather than the order the commits landed in.	2026-08-21 05:48:39 +00:00
Johnathan Corgan	3a789370b9	Add an experimental native datagram API addressed by public key ... A client process opens a flow to a peer's public key on a chosen port and sends and receives datagrams on a file descriptor the daemon hands it. No IPv6 emulation, no TUN device, no DNS: a datagram travels from key to key. The feature is off by default and is not a stable interface. The wire needs no change and gets none. Every FSP data packet has carried a port pair inside its AEAD envelope since v0.2.0, and port 256 is simply the IPv6 shim. What was missing was a way for a program to ask for a port of its own and be handed the traffic. Addressing is the part worth reading twice, because the obvious design is wrong. The x-only public key is the address. An npub is that key written in bech32, so converting between them is a local encoding rather than a lookup or a name service. The 16-byte node address that travels on the wire is the first half of a SHA-256 of the key: it is a truncated hash, it does not invert, and it appears nowhere a client can see. An earlier iteration of this work reported a peer by that hash and could supply a key only sometimes, which is what treating a wire identifier as an identity produces. An accepted flow therefore always knows its peer. The key is captured where the peer is authenticated rather than looked up when a report is rendered: every inbound datagram passes one call site inside a handler that refuses anything whose session is not established, and the responder has already rejected the session unless the claimed address derives from the key it proved. Reaching for the identity cache instead gives a best-effort answer from a structure that evicts. A listener is a descriptor. The daemon writes one message per arrival to it, carrying the new flow's descriptor and the peer's address, so poll, select and epoll work on a listener and accepting is a recvmsg. That is what lets the API be used from a program that already has an event loop, which a command-and-reply listener could not support: an arrival could not be waited on beside anything else. There is no accept command and no reject command. Refusing a flow is closing the descriptor you were handed. The Rust surface mirrors std::net. FipsStream::connect, FipsListener::bind, incoming, accept, io::Result and an errno mapping rather than a bespoke error type. An address is given as an npub, as a key, or as a pair, through one parameter, the way ToSocketAddrs takes several spellings of one thing. Each type holds its descriptor and copies of what setup told it and nothing else, so a stream that outlives its setup connection is not representable. set_nonblocking, AsFd and the four deadline methods carry the names and signatures std::net uses for the same jobs. They were asked for by a user integrating the API with tokio: AsyncFd requires a non-blocking descriptor, and anything receiving from a peer needs a bounded wait. AsFd is the better of the two descriptor accessors, because the borrow cannot outlive the value that owns the descriptor, so a reactor cannot hold a registration for a descriptor that has since been closed and its number reused by the next open. The non-blocking flag is read, modified and written back rather than assigned, since the flag word carries more than that one bit and a caller may have set O_ASYNC. A zero timeout is refused with EINVAL, because the kernel reads a zero timeval as "wait for ever", which inverts what a caller passing zero means; std::net refuses it for the same reason. The two directions are separate options and stay that way. FipsListener gets no timeout methods, matching TcpListener: bounding an accept is set_nonblocking plus the caller's own poll, which the reactor how-to builds. A flow taken from accept is blocking whatever the listener was set to, because the two are separate sockets and the daemon hands over a fresh one. One rule has no counterpart in Berkeley sockets and a client author must know it: the v1 wire carries no half-close, so nothing peer-driven ever closes a flow. A server written to read until the flow ends waits for a signal that cannot arrive, holding a thread and a flow per peer until its process exits. A program decides its own termination, and the example serves one datagram per flow. The tests reach a live daemon rather than a stand-in. Every public item had a unit test against a hand-written stand-in with canned replies, and the five entry points a program actually calls first, connect, connect_from, connect_at, bind and the SOCKET constant, had no coverage of any kind, because the tests that appear to cover them build a Wire over a socket pair and hand it to the private open and hold, so nothing ever resolved a socket path or mapped its errors. examples/native-surface.rs walks all thirty-eight items against a running daemon and reports the number of assertions it made. The count is read from the recorder rather than written as a literal, and the harness asserts the exit status, the completion marker and the count together, so deleting an assertion fails the check rather than quietly shrinking it. Watchdogs turn a hang into a named failure, which several of the walked behaviours would otherwise produce. The shared Docker image is built once for every integration leg, so the new binary is staged at all ten places the existing one is, the interop builder included, which gets a stub because those images exercise the wire between daemon versions and older refs do not carry the example. The platform gating was tested rather than reasoned about: flipping all eleven gates so the native API is excluded leaves the crate compiling clean across the workspace, every target and the profiling feature. The shipped docs tree gains what only the LaTeX manual under design/ had, which is not published with the daemon. A reference entry covers the whole surface: addressing and the port tiers, the Berkeley mapping, every method on FipsAddr, FipsStream, FipsListener and Incoming, the errno table, the ceilings, the four places data disappears with nothing reported, the line protocol and the command reference. The errno table gives names rather than numbers, since the client maps each name onto the libc constant for the platform it was built for and the supported platforms disagree on the numbers. A tutorial side trip stands up two throwaway nodes on one machine, peered over loopback UDP with no TUN and no DNS, then writes a listening program and a connecting program against them; it needs neither the public mesh nor root, because the native path is the one that does not go through the IPv6 adapter. The obligations a client in another language carries are a how-to of their own, since they are a task rather than a description: reading the setup connection with recvmsg, associating a descriptor with the last complete line, telling an empty datagram from a close, and six others. Serving many peers from one poll loop is another, with the whole program, because the straightforward listener spawns a thread per flow and that is wrong at the node's ceiling of 256. The drop causes are a table mapping each of the seven texts DropReason::as_str produces to the counter it increments, with drop_oversize called out as the ninth counter that is not in the table. What a daemon restart costs is a section of its own: every flow and listener ends, descriptors do not survive, there is no resumption, and datagrams sent but not yet forwarded are lost through a window nothing bounds. A stack comparison diagram places the interface against the stack a reader already knows: the same application over HTTP, TLS, TCP, IP and Ethernet on one side, and over its own format, FSP, FMP and a FIPS transport on the other, aligned so each row is one concern. The two columns are not alternatives and are not drawn as such. An unmodified IPv6 program's packets reach fips0, and the adapter hands each one to FSP as a payload, so the left stack runs inside the right one; the left column ends at a fork, eth0 for the ordinary internet and fips0 for the mesh, and an arrow leaves fips0 and runs back up into FSP's input. The row where TCP would be is empty on purpose and names Reliable Object Delivery, which is where that capability is expected to land. ROD is a v2 capability, the box is dashed because none of it exists yet, and the design entry says the part a reader needs most: nothing on the surface anticipates it, so a program written today should assume it does not exist. Both endpoints carry a scheme and a worked port, https://<npub>.fips:443 and fips://<npub>:443, with a footnote saying the two ports are not the same kind of thing, a TCP port inside the tunnel on the left and an FSP port on the right. The fips:// form is a coinage: nothing in the tree parses it, nothing registers the scheme, and the API takes a key and a port as separate arguments rather than a URL. The diagram also says where the right column stops, since FIPS over UDP still rides IP and Ethernet beneath. It appears in fips-concepts.md and fips-ipv6-adapter.md, which were making its argument in prose without a picture, and deliberately not in fips-architecture.md, which already carries the OSI mapping and makes the same point about the transport row. The gateway's control socket moves onto the same bind policy this API uses, which is the one change here that touches deployed behaviour: fips-gateway now tightens /run/fips to 0750. That is unreachable under the packaged deployment, where fips.service has already created the directory at that mode, and reachable for a source build or a container that starts the gateway alone. One changelog entry under Added, describing the released state: what a client opens and reads, the addressing and why the node address is not it, the listener being a descriptor, the std::net shape of the Rust surface, and the one rule Berkeley sockets have no counterpart for. It says in as many words that the wire is unchanged.	2026-08-21 05:48:23 +00:00