SMC Milter - Email Filtering Technology

Filter working algorithms

In our opinion, mail server should receive e-mail only from the users who use their mail servers accounts legally, but SMTP protocol realisation being in use today does not forbid sending e-mail from the servers unrelated to the indicated mail domain. In SMC filter unverified e-mail may be rejected or sent to processing by the filter blocking algorithm. As mentioned above, by default junk e-mail blocking is accomplished by using Greylisting algorithm. The algorithm is based on checking server's execution of deferred delivery function when receiving server "temporary error". If the recipient mail server refuses to accept the letter ind informs about "temporary error", the sender mail server has to retry later. Bulk mailing software in that case usually does not try to do it. Successful check result is saved for the period determined by Lifetime parameter of configuration file. All subsequen letters from the same sender to the same recipient sent through the same server will be accepted without delay during this period. Algorithm functioning is adjusted by Maxdelay and Maxcount configuration file parameters. Maxdelay parameter determines maximum period of filter's waiting for repeat server call. If repeat call is registered within this period, the counter of successful attempts will augment 1, otherwise it will be nulled. Maxcount parameter determines total amount of such attempts after which a letter is accepted. Theoretically, this algorithm blocks only junk e-mail, but actually in practice this is not the case. Registration bots messages, news sites mailings etc. may be screened by mistake, if they use non-standard mail sending methods. Message delivery delays may be unacceptable in case of urgent mail. The filter contains a suit of automatic "white lists" enabling to avoid such delay or mistake. AutoSPF algorithm is a basic one among the filter's "white lists".

AutoSPF

The policy of mail legality defining (message sender verification) in AutoSPF algorithm is based on analysing the possibility of tie-up existing between a sender e-mail domain and transferring e-mail server. AutoSPF automatic sender verification technology is similar in many ways to SPF technology (Sender Policy Framework). In both cases DNS records are used for verification of messages issuing from the involved domain. In case of SPF technology application the administrator (owner) of the domain uses DNS to publish data describing potential e-mail sources which have sender addresses relating to the domain. In case of AutoSPF application it is considered that DNS records initially content enough information for sender verification. In te elementary case successful sender verification requires meeting 3 basic conditions:

There should be at least one MX record in an e-mail domain zone description (at the end of the sender's e-mail address);
Inverse resolving of the sender host name should be accomplished;
The sender host should have a record on one of the DNS servers supplying the e-mail domain zone.

By analysing DNS records, sender domain is checked on correspondence with address of the connecting server. I.e. the sender whos address is somebody@host.name can send an e-mail only through host.name domain mail server which would have DNS resolving and appropriate MX record corresponding to IP address of the connecting server.

In practice automatic sender verification algorithm looks a little bit more complex. At first stage a list of MX records in a mail domain zone description is analysed. Accomplishing direct name resolving for each of the indicated MX servers makes up a list of the domain mail servers. By comparing the sender's IP addresses with the resulting list's addresses we can determine whether the server belongs to the sender mail domain. The search could be finished here, but the reality shows that e-mail is often quite legally sent throuh relational nets, for example using provider e-mail service. Determining relational environment requires additional information, which in most cases is also contained in DNS records. In the case of using provider's mail (MX) server relational nets are usually supplied by the same name servers. Reasoning from the supposition, for a start we have a list of name servers (NS records) indicated in a mail domain zone description. To determine the sender's IP address connected name servers list we should create an inverse (PTR) record corresponding to the address. At the next stage we receive a list of name servers (NS records) indicated in the sender IP address inverse zone description. If at least one of the indicated name servers is present in the both zones, the sender is considered verified. At the last stage using inverse resolving of the sender IP address we determine domain name corresponding to the address. Then we get a list of name servers (NS records) indicated in the domain zone description and compare each of the found records with the corresponding records in the mail domain zone. If at least one of the indicated name servers is present in the both zones, the sender is considered verified.

All successful DNS queries are cached. A number of DNS queries per one connection usually does not exceed 20.

AutoSWL

Anoter white list is implemented by the filter's AutoSWL (Auto Sender White List) algorithm. The algrithm is very simple. In the mail contact - "sender-recipient" pair - the sender is considered authenticated (included in the recipient's "white list") if both outgoing message (request) and incoming message (reply) of the contact. Information about successful contact is saved for the period determined by Lifetime parameter of configuration file. The messages of the authenticated senders will not be blocked by the previous algorithm even if there was no successful sender verification. Thus, the filter learns during performance not to react upon active contacts, i.e. if you exchange messages with somebody, the incoming messages will be gated by the filter without additional check.

The filter by default gates e-mail from local reserved addresses (127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 etc.). Other addresses e-mail from which should be received without additional checks may be indicated in smc-milter.hosts configuration file. This file is a static "white list" of the filter. Below is the file content example:

#
# SMC-milter hosts file
#
# Pass through the mail from the indicated IP addresses
# or networks without any checks.
#
# lines preceded by a '#' are comments
#----------------------------------------------------------------------
12.107.209.244/32  # kernel.org
12.107.209.250/32  # sourceware.org
64.125.132.254/32  # collab.net
204.107.120.10/32  # Ameritrade
205.206.231.0/24   # securityfocus.com
209.132.176.174/32 # sourceware.org
213.136.52.31/32   # mysql.com

Extra information

E-mail headers

SMC filter marks incoming messages with the following range of commonly used headers:

Header	Description
X-Spam-Flag:	[YES/NO/WARN/PASS]
X-Spam-Report:	System message describing the reason of one or another Spam-Flag header value binding.
X-Spam-Checker-Version:	SMC-milter [software version]
X-Virus-Scanned:	Anti-virus software name (if used)

For example:

X-Spam-Flag: NO
X-Spam-Report: Host 195.206.40.177 is related to lust.icc.ru.
X-Spam-Checker-Version: SMC-milter 2.0
X-Virus-Scanned: ClamAV using SMC-milter