RCVD: Field Notes From a Live Router - Part 1

The value of a privacy tool isn’t in the design document. It’s in whether it holds up when the network tooling misbehaves and nobody is watching. These are notes from running RCVD in the field.

RCVD has been running on a small ARM router, on Alpine Linux — forwarding DNS for a live network over an encrypted upstream chain (DoQ to AdGuard, falling back to DoT at Cloudflare, then DoH at Quad9). It has been doing this, on real traffic, for roughly two months. A few things worth writing down.

Catching my own bugs

Building the thing is one job; operating it surfaces problems a test suite won’t.

• Graceful shutdown that wasn’t. I noticed the daemon was hitting its full stop timeout on every restart instead of stopping cleanly. The cause was an ordering bug in my own shutdown path: it waited on in-flight work before closing the listeners, but the read loops were parked blocking on the socket and couldn’t observe the cancellation until the socket was closed. Reorder to close listeners first, and the stop is instantaneous. I found it myself, fixed it, wrote a regression test that fails on the old ordering and passes on the new, and confirmed the fix live on the router.

• A statistics skew. Error paths were recording near-zero latencies into the min/avg/max, quietly poisoning the numbers. Gating the measurement on a successful query fixed the display. Small, but it’s the kind of thing you only see once you’re staring at real statistics from real traffic.

Dependency discipline

RCVD’s encrypted transports lean on the QUIC stack. At one point I was tracking a development pseudo-version of that dependency — fine for iterating, not fine for something running unattended on a router. I pinned it to a real tagged release, re-ran the full test suite and a set of container-based integration gates (including a real DoQ path that asserts zero SERVFAIL), and only then shipped the new binary to my router. The whole stack now sits on tagged dependencies, with a note to myself to pin forward to the next tag rather than chase the moving branch.

The day the guarantee mattered

The single most reassuring moment came from a real outage I didn’t cause. The router’s encrypted egress went dark for a few minutes, An upstream tunnel fault, not RCVD. This is exactly the scenario where a lesser stack leaks: encrypted path down, so it “helpfully” falls back to plaintext DNS to keep things working.

RCVD did the opposite, by design. It failed over across its encrypted upstreams, and when they were all unreachable it returned SERVFAIL and held the line — zero cleartext queries left the host during the outage. When the tunnel came back, the upstreams recovered automatically within about a second. The zero-cleartext promise isn’t a ‘slogan on a page’; I watched it hold under a fault I didn’t plan for.

Why this matters for the work I do

I spend many of my days as a infrastructure engineer. RCVD is where that experience turns into something built rather than configured — and running it in production is what lets me say the guarantees are real, not aspirational.

• Project site: rcvd.net
• Whitepaper: rcvd.net/whitepaper