← Blog
Postmortem

Turning on a DNSBL uncovered a bigger bug: our IRC network's bans were never actually saved

September 2026

We run CosmicIRC, a small independent IRC network on InspIRCd v4.11.0. Adding a DNSBL — a shared blocklist of known drones, open proxies, and spambots — should have been a five-minute config change: add a <dnsbl> block, reload, done. Instead it led us to a much bigger problem that had been sitting there, invisible, for who knows how long: every ban we'd ever issued was living in memory only, and a restart quietly threw all of them away.

Why a DNSBL at all

A DNSBL works by encoding lookup results in a DNS response's last octet. You query <reversed-ip>.rbl.ircbl.org, and if the IP is listed, you get back an address like 127.0.0.11 — the 11 is a bitmask telling you why it's listed (drone, open proxy, flooder, spambot, and so on). InspIRCd's dnsbl module does that lookup on every connecting client and can reject or mark the ones that come back positive.

We didn't want to block on every category IRCBL tracks — some of their lists are noisy for a small general-chat network, and we didn't want a plausible-looking exchange student flagged as an "insecure proxy" to get bounced. We scoped it to records 11, 13, and 16 — drones, flooders, and spambots specifically — and added the block to inspircd.conf, pointed at rbl.ircbl.org.

Making sure it actually covers real users

CosmicIRC's traffic isn't all native IRC clients. A meaningful share comes in through KiwiIRC, our web client, which connects to InspIRCd through a WebIRC gateway rather than a raw TCP connection. If a DNSBL check only fires on the class native clients land in, every web user sails through unchecked — which would make the whole feature mostly decorative.

We traced this through InspIRCd's connect-class logs rather than assuming it from the config. Watching real connections land, both the WebIRC-gateway path (KiwiIRC → gateway → the main connect class) and native client connections resolved into classes the DNSBL block actually covers. One old, seemingly-relevant detail turned out to be a non-issue: a proxied-flagged config option and port 6698 that looked like they might route around the check were confirmed dead and unused — nothing currently connects through them, so there was nothing to fix there.

The side quest that mattered more

While tracing how connections flow through InspIRCd for the DNSBL work, we ended up looking at the box's log directory permissions — ownership had drifted to the cosmic user rather than irc, the user InspIRCd actually runs as. Fixing that meant also fixing logrotate's create directive, which specifies the ownership newly-rotated log files get, so the fix wouldn't quietly undo itself on the next rotation.

That ownership check is what led us to look at /var/lib/inspircd and /etc/inspircd more broadly, and that's where we found the real problem: those directories had the same ownership mismatch, and InspIRCd's m_xline_db module — the one responsible for writing active Z-lines, G-lines, and K-lines to disk so they survive a restart — couldn't write to them. It wasn't erroring loudly. It was failing the write silently and carrying on, so from the operator's chair everything looked completely normal: you set a ban, it worked immediately, /stats showed it was active. The only way you'd ever notice was restarting the server and watching every ban you'd ever set simply not be there — and if you weren't specifically checking for that, you'd have no reason to suspect it.

The same ownership problem affected m_permchannels, which persists channel registration/mode state across restarts the same way.

Fixed both by correcting ownership on /var/lib/inspircd and /etc/inspircd to match the user InspIRCd actually runs as, then verified the fix the only way that actually proves it — set a test line, restart the service, confirm the line was still there afterward — rather than trusting that a clean-looking config meant a clean-working one.

The lesson

The DNSBL feature is maybe two lines of config. The bug we found while adding it had been silently discarding every persistent ban and permchannel setting on every restart, for an unknown length of time, with zero errors anywhere a normal operator would look. Nothing about the DNSBL work would have surfaced it on its own — it surfaced because tracing "does this check actually apply to real connections" led into directory permissions, and directory permissions led into "does this directory actually get written to."

If you run a service that's supposed to persist state to disk, the only real test is restarting it and checking the state is still there — not checking that the write appeared to succeed, and not checking that the directory exists. A permission problem that only affects a background write path can hide indefinitely behind an application that never bothers to check its own return code.


We run CosmicIRC, a free, independent IRC network. If you're running InspIRCd and want to compare DNSBL scoping or connect-class setups, we're at irc.cosmicirc.com, #cosmic.

← Previous: Pin every package in your Dockerfile