Match only the Spamhaus listing return codes

[AndrewSav]

Note

TLDR: Spamhaus check currently rejects emails on a query failure, not just when listed entry is found, this both breaks test intermittently, and rejects legitimate email in a production setup. This PR fixes it.

Postfix queries the two Spamhaus blocklists without saying which answers count as a listing. Spamhaus uses 127.255.255.0/24 to report a problem with the query — 127.255.255.254 "query via public/open resolver", 127.255.255.255 "excessive number of queries" — and because those are A records in 127.0.0.0/8, Postfix reads them as listings. While that lasts, every sender is rejected with a permanent 554, so mail is bounced and lost rather than delayed, and the reason given is untrue: the sender is not listed at all.

This is not hypothetical. It happened on master in CI: run 34746135638, attempt 1 failed the reverse suite with

554 5.7.1 Service unavailable; Sender address [user@gmail.com] blocked using
dbl.spamhaus.org; Error: open resolver; https://check.spamhaus.org/returnc/pub/13.66.150.136/

(the run was re-run afterwards and now shows green overall, so the failure is only visible under attempt 1).

The same change also settles those test failures, because nothing in the suite depends on the blocklist rejecting anything: no test references the DBL, and every fixture sender is gmail.com or domain.tld, neither of which is listed. The tests that break when a resolver is refused are all delivery assertions, and they stop breaking once senders are no longer rejected for a query error. No test loses coverage as a result.

• What is the current behavior (you can also link to an open issue here) ?

  reject_rhsbl_sender dbl.spamhaus.org and reject_rbl_client zen.spamhaus.org carry no =d.d.d.d filter, so any answer in 127.0.0.0/8 — including Spamhaus's query-error codes — is treated as a listing and the sender or client is rejected permanently.

• What is the new behavior (if this is a feature change) ?

  Only the documented listing codes reject. Query errors are logged and otherwise ignored, so a blocklist that cannot answer stops filtering instead of rejecting everything.

Match only the listing return codes

Postfix's postconf(5) is explicit about the unfiltered form:

If no "=d.d.d.d" is specified, reject the request when the MAIL FROM domain is listed with any A record under rbl_domain.

and Spamhaus is equally explicit that the error range is not a reputation:

[error responses] must not be interpreted as advisories of Spamhaus reputation data regarding the object which was queried

Any software which does not distinguish response codes from Spamhaus DNSBLs is, unfortunately, already out of date.

So both checks now carry the ranges Spamhaus publishes for use with Postfix, and a second line logs the error range without acting on it:

smtpd_sender_restrictions=
    ...
    reject_rhsbl_sender dbl.spamhaus.org=127.0.1.[2..99],
    warn_if_reject reject_rhsbl_sender dbl.spamhaus.org=127.255.255.[1..255],
    ...

smtpd_recipient_restrictions=
    ...
    reject_rbl_client zen.spamhaus.org=127.0.0.[2..11],
    warn_if_reject reject_rbl_client zen.spamhaus.org=127.255.255.[1..255]

warn_if_reject turns a refusal from a silent outage into a reject_warning line in the log; it queries the name the check above it has already queried, so the answer comes from the resolver's cache and no extra blocklist traffic is generated.

zen.spamhaus.org has the same defect and is fixed the same way. It never showed up in the tests because smtpd_recipient_restrictions begins with permit_mynetworks, permit_sasl_authenticated, which short-circuits for the loopback clients the fixtures use, whereas smtpd_sender_restrictions has no such permit and is evaluated for every client. In production a remote client reaches the zen check, so it was untested rather than unaffected.

What a refused resolver did to the test suite

reverse and ldap2 run with DISABLE_DNS_RESOLVER=true and inherit the host's resolver, so they are the suites exposed to this; everything else queries through the container's own unbound. When the inherited resolver is one Spamhaus refuses, no fixture mail is delivered and eight assertions fail, none of which mentions DNS:

test	reads	why it fails
34, 36	mail counts, alias delivery	nothing was delivered
35	"rejects mail to unknown user"	the sender was rejected before the recipient was evaluated
43	spam filtered	rspamd never saw a message
61, 62, 71	zeyple delivered / encrypted	zeyple never ran
69	john doe gpg key in keyring	first gpg of the run, see below

Test 69 is the most misleading of them: it fails on stray output rather than on its count.

gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
1

Normally zeyple has already run gpg while encrypting delivered mail, so the trustdb is rebuilt quietly much earlier; with nothing delivered, this test's gpg --list-keys is the first invocation and bats folds its stderr into $output. Test 70 — the same check — passes because 69 rebuilt it.

The whole cascade is one event: in that CI run all six rejections fall inside 700 ms and cover two sender domains, so at most two queries reached Spamhaus and the rest were the resolver's cached copy of the first refusal.

Record what the blocklist answered

A failing run could be diagnosed from its rejection messages, but a passing run recorded nothing at all about the blocklist — no line mentioning Spamhaus, the DBL, or which resolver was used — so a green run could not be told apart from one where the blocklist answered nothing and the check silently never matched.

fixtures_reverse, fixtures_ldap2 and fixtures_default now log, before any mail is sent, which resolver is in use and what it answers for dbltest.com (Spamhaus's permanent test listing), gmail.com (the fixtures' sender domain, the name Postfix actually looks up) and example.com (unlisted):

[dnsbl] resolver: nameserver=168.63.129.16 egress-seen-by-akamai=20.150.226.50 egress-seen-by-google=20.150.226.73
[dnsbl] dbltest.com.dbl.spamhaus.org   status=NOERROR   A=127.0.1.2   ttl=60   "Listed by DBL, see ..."
[dnsbl] gmail.com.dbl.spamhaus.org     status=NXDOMAIN  A=-           ttl=10   -
[dnsbl] example.com.dbl.spamhaus.org   status=NXDOMAIN  A=-           ttl=10   -

127.0.1.2 for the test listing means the blocklist is working; 127.255.255.x means this resolver is refused and the TXT names the address Spamhaus refused; NXDOMAIN for the test listing means the blocklist answered nothing and is inert. The resolver line is there because only a refusal names an address — a healthy answer does not. fixtures_default is the control: it queries through the container's unbound and should always show the listing. The probe is diagnostic only and cannot fail a run.

Type of change

• Bug fix (non-breaking change which fixes an issue)

Status

• Ready

Todo List

• Implementation
• Tests
• Documentation

How has this been tested ?

The full bats suite, one suite at a time: git submodule update --init --recursive, make build, then make default, make reverse, make ldap, make ldap2, make sieve, make ecdsa, make traefik_acmev1, make traefik_acmev2. Not as root: the Makefile passes id -u/id -g in as the vmail user and Dovecot refuses uid 0.

• default — 122 tests
• reverse — 79 tests
• ldap — 66 tests
• ldap2 — 29 tests
• sieve — 2 tests
• ecdsa — 4 tests
• traefik_acmev1 — 10 tests
• traefik_acmev2 — 11 tests

323 tests, 0 failures — unchanged from master, since this adds logging rather than assertions. The probe appears in the reverse, ldap2 and default logs immediately before the first fixture message.

Beyond the suite, the two restriction lists were exercised directly against a resolver Spamhaus refuses and against one it answers:

	refused resolver, ordinary sender	working resolver, listed sender (user@dbltest.com)
before	554, mail bounced, reason untrue	554
after	accepted	554 ... blocked using dbl.spamhaus.org; Listed by DBL

so the blocklist still rejects what it is supposed to reject, and stops rejecting what it never meant to.