Kolab Now: Grey Listing Applied

Aside from other anti-spam measures, we have applied a concept known as grey listing.

Here’s a summary of how grey listing works: When an email delivery attempt is made, we know the sending server’s IP address, the sender address, and the recipient address. If this is a previously unseen combination of facts, the delivery attempt will fail with an error message that indicates a temporary error — the sending server is responsible for trying again later.

This subject allows me to zoom in on what’s what as well. Please note that none of the terminology intends to refer to any societal norms and these things are solely intended to represent the technological parlance du jour;

White Listing

Accept whatever is on this list.

Grey Listing

What isn’t white listed is put on this list. What recurs under certain conditions can be white listed.

Black Listing

Do not accept whatever is on this list.

Grey listing makes it so that proper mail servers that send messages to our customers are required to use proper queuing — and that’s something that spam bots just do not do (most of the time, because they can’t).

That being said, this is not a blog post about how to install, configure and operate an environment in which you choose to apply grey listing — that one’s going to be a separate one.

But how do we run a cluster of inbound mail exchangers yet share what is known and what is new between nodes? Enter the sqlgrey package available in EPEL. It maintains the facts in a centralized fashion — in our case, a load-balanced MariaDB Galera cluster.

But how do we treat third party services that themselves cluster, or even geographically distribute — and thus originate from many sources, often not even the same subnet? Well, to be completely honest, most of them will have to retry delivery multiple times;

  • The first delivery attempt may originate from IP address a.b.c.$x,
  • The second delivery attempt may originate from IP address d.e.f.$y,
  • The third delivery attempt may originate from IP address g.h.i.$z,
  • and so forth.

We learn about these through analysis of our log files, and could whitelist such senders.

What does this all mean in real life?

A proper, legitimate, regular email delivery that is a new sender/recipient, from a new “Class C” (^1) network, is delayed by either of the following parameters;

  • The “new” delivery attempt is made within 5 minutes,
  • The sending party elects to not re-attempt delivery for $x minutes.

RFCs have suggested to re-attempt delivery every 30 minutes, entertain Delivery Status Notifications to be sent back to the original sender every 4 hours, and continue to attempt to deliver messages for up to 5 days.

However, many sending parties will re-attempt delivery well within 30 minutes. Some of them will fall within the (now mandatory) boundary of 5 minutes (i.e. “if not this MX, let’s try the next MX”). But email isn’t instant messaging.

Long story short, this means deliveries could be delayed. Sometimes this may take up to 30-some minutes.

If a combination of “Class C” (^1), sender and recipient doesn’t recur within 24 hours, the original entry (allowing it to be eligible with a re-attempt) is expired.

Domain name spaces that continue to recur are largely white listed. This avoids regularly recurring patterns from needing to be re-evaluated for all traffic.

Analysing Data

We need about a week’s worth of data to analyse in order to be able to recognize patterns. Luckily, we have that amount of information per tomorrow. It’ll take us some time to learn what we can from this data, and separately another amount of time to then apply that level of knowledge to our systems.

Opting Out

The mechanism I have elected to apply is not “opt-in”, nor “discriminatory” — but instead, “opt-out”.

We can entertain requests to have a certain group of recipients result in “no treatment” — basically implying the 3-legged tuple is considered white listed for as far as sqlgrey is concerned. Opting out is a discretionary effort though, and so you may wish to let our support know if you wish to not entertain this anti-spam measure being applied to you.

Footnotes

^1: “Class C” is not the correct terminology to apply to whatever is recognized by the first 3 octets of an IPv4 address, but it is inherently a “/24”, or “255.255.255.0” netmask. This can be a “Class C network”, or a “Class A or B sub-network (subnet)”.

Published by kanarip