Most organisations today categorise email as mission-critical to their business, and so the ability to reliably send and receive emails receives a lot of attention from us. However, as we are all aware, the amount of junk mail – ‘SPAM’ – received is a continuing annoyance.
A quick check of our servers shows that over the past three months, a massive 53% (of nearly 1/2 million emails) of all emails received were suspect. Now obviously it would be great if we could simply block such messages, but detection of SPAM is more a black art than a defined science, and so SPAM filters must be continually adjusted and refined to ensure all real/valid emails get through (zero false-positives) while all SPAM is blocked. We could simply turn off SPAM filters, but generally this would result in 50% extra emails reaching each users’ inbox – a situation no user wants to contemplate.
Breakdown of the 53% Suspect Emails
Breaking down the above 53% shows that:
- 2% of all emails contain malware – Viruses, Trojans etc – we definitely do not want these passed through.
- 31% of all emails are from known spammers – we want these blocked automatically, but as some users want to see some of these, we need to provide a mechanism whereby they can be retrieved.
- 20% of all emails are where we have to be somewhat creative and apply our black-art talents to separating the wheat from the chaff, and it’s this 20% that seems to cause the most questions and mis-deliveries.
How to determine the validity of incoming email?
So what ‘black arts’ do we employ to determine the validity of this 20% (90,000) of messages? Essentially, each and every message is tested and given a Spam Confidence Level (SCL) score, and when it gets above a defined level, it is tagged as SPAM. Virtually all anti-SPAM solutions – whether in-house, or in-cloud, software-based or appliance-based – employ some or more of the following checks:
- Reputation of Sender:
- Is the sending server on a blacklist? This occurs when a recipient reports receiving SPAM from a server, and is placed on one or more of the 100+ blacklisting organisations.
- Is the sending server properly set up with a fixed address – determined by whether it has a Reverse-DNS (RDNS) entry. (Spammers will use arbitrary/impermanent servers).
- Does the sending organisation designate approved sending servers – via the Sender Policy Framework (SPF) stipulated in DNS
- Message Content:
- Does the message subject or content contain known SPAM-like words – such as ‘Drugs’, ‘Viagra’ and so on.
- Does the message contain multiple languages, such as combined English & Chinese?
- Message Distribution:
- How many recipients is the message sent to? More than a handful generally means the message is more likely to be Unsolicited Commercial Email (UCE).
- User Overrides:
- Has a given user either black-listed or white-listed a particular address?
- has a given user chosen to be more or less aggressive in their application of the above checks?
How to ensure that emails we send get read?
I trust from the above readers can get an idea of the issues we face in categorizing SPAM for delivery, and I trust that readers will also see that it’s very much an imperfect ‘guesstimation’ at best, and there is little as recipients we can do to more accurately qualify messages. But what should we do to ensure that messages we send get the highest possible chance to be delivered?
- We MUST have adequate malware protection for inbound emails, and ideally also scan outbound emails for malware to ensure we do not compromise our message recipients.
- We must properly designate and authenticate our outbound servers – via RDNS, SPF – and ensure all mails from our domain are sent only from these servers. This may not be just your corporate email server: – we must also check any other servers that send emails on behalf of our domain: -for example if our website sends emails, then it must also be properly designated and authenticated.
- We must regularly check that our sending servers have not been blacklisted.
- If we regularly need to send messages to more than a handful of recipients, we should consider using a third-party service rather than send such ‘mail-blasts’ through our own servers – which reduces the likelihood of getting our servers blacklisted.
- If we send Unsolicited Commercial Email (UCE) as a Singapore organisation, we must ensure the subject line contains <ADV> and we must provide (and act on) an unscubscribe link – to ensure we comply with legislation, or otherwise face the risk of legal action against us.
If you would like to discuss this issue and what it means for your business, please get in touch. Since inception in 1996, PASR Technologies has been providing SME business owners with a level of service and support to the SME business owner that is typically only directly available in very large organisations.
Servicing businesses from 10 to up to 200 employees, our clients range from local SMEs through to regional offices of larger MNCs, and include airlines.
At PASR Technologies, we solve your IT problems before you even realize you have one!
