Archive for the 'email' Category

Problems with spam filters…

Wednesday, January 10th, 2007

This last few months I have noticed an increasing problem with legitimate emails getting blocked by poorly conceived and configured spam filters.

It appears that many ISP’s (Internet Service Providers) are responding to complaints by their customers (about the torrent of spam they receive) by modifying and reconfiguring their spam filtering technology.

Unfortunately, these responses have not been well thought out and implemented.

I will provide a couple of examples but first I need to define some terms:

  • User PC – the personal computer used by an end-user to send or receive an email
  • Email Client – the software on the User PC used to compose and read email
  • Email Server – the server used by multiple users to handle both incoming and outgoing emails
  • ISP – a provider of internet connectivity to individuals and small businesses (includes big phone companies like Verizon, big cable TV operators like Comcast, and smaller local providers).
  • Spammer – the bad guys, typically using someone else’s hijacked PC’s or servers to broadcast thousands of undesirable junk emails to huge lists of email addresses

Example #1

I recently sent an email to a customer (an firm of architects) and had it bounced back to me as spam by the firm’s ISP. In this case the ISP was small, local outfit but a lot of the big guys are doing the same thing.

I created the email on my Windows XP computer using my email client (Mozilla Thunderbird). My email client is configured to use myname@salemdesign.com as the “From” address. It is also configured to send email via an email server provided by the hosting service I use to host salemdesign.com. This email server has a name along the lines of smtp.hostingserverdomain.net.

Why was my legitimate email bounced back to me? The local ISP analyzed the header information of my email and found that the email server I was using was not part of the same domain as my email address and decided that this meant my email was spam.

My “From” address was myname@salemdesign.com which had a different domain than the email server smtp.myhostingservice.net. The email header also includes the numeric IP address of the originating SMTP email server which would also be associated with my hosting service rather than my salemdesign.com domain.

Now, it is true that essentially all spam is sent from SMTP servers which are not associated with their purported “From” or “Reply-to” email addresses. But it is also true that the vast majority of small to medium size businesses and organizations do not have their own SMTP email servers. So this policy of blocking emails where the “From” domain is not associated with the originating email server means that a lot of legitimate emails are being blocked.

In this particular instance, I called up my customer on the phone and explained the situation and asked for his fax number… And then I faxed him what I would have otherwise emailed.

Note that this means that this firm of architects is having some portion of its legitimate incoming email blocked. If the sender is not that motivated, he or she may just shrug and “walk away” and some potential business is lost.

The big email providers such as Yahoo, MSN/Hotmail, and Google/GMail also follow this practice. They don’t block unassociated emails but they do automatically put them in the end-user’s spam or bulk folder.

SPF Record

There is a partial work-around but it’s a little complicated and very few small or medium size businesses know enough to apply it.

The partial work around is called the SPF Record. SPF stands for Sender Policy Framework and you can find out more about at www.openspf.org. But in essence, the SPF record is part of a domain’s registration record and it provides a list of domains and servers that can legitimately send email associated with that domain.

So I set up an SPF Record that included the email server information for my hosting provider’s email server (and a third party email server I use). I waited a few hours and then sent test emails to accounts I have on Yahoo, MSN, and GMail… And it worked. None treated my emails as spam.

I then sent a test email to my architect customer and it also got through fine.

To summarize… Once the SPF record is implemented, an ISP (such as Yahoo) receiving an email purported to be from me@salemdesign.com can check the domain records for salemdesign.com and confirm that the smtp server I used was authorized to send salemdesign.com emails.

Example #2

My hosting service is very good about ensuring that their servers are not used for spamming. This is critical because each of their servers hosts dozens of domains and they all share a single email server. If any one of those domains is sending spam (either deliberately or because they were hacked) then all mail from that email server gets blocked or labeled as spam by the big ISP’s.

But recently a new problem has arisen with one of the larger ISP’s (Comcast).

The scenario is that someone hosting their domain with the hosting service decides to forward their email (sent to, for example, userA@theirdomain.com) to an email address they have on Comcast (i.e. userA@comcast.net). Presumably Comcast is the ISP for either their small business or their home computer.

This should not be a problem but it is… Because Comcast views any spam that is being forwarded as being generated by the forwarding server and Comcast then blocks said forwarding server. This is causing such a problem that the hosting service has banned anyone from forwarding email to Comcast addresses. This is, of course, hurting Comcast’s own customers… Some are finding that they are not allowed to forward emails to their own comcast email accounts… And even more are having legitimate emails sent to them blocked.

I have a number of customers with comcast addresses, and I now send emails to them via my GMail account. That seems to work fine even though I am sure a lot of spam gets forwarded via GMail. Perhaps Comcast is scared of blocking emails from an entity as large as Google.

Dealing with Spam

Tuesday, October 31st, 2006

I am writing this with a view to advising clients who have their email hosted on my server environment but much of the discussion is relevent to anyone with an internet email address.

First some definitions:

Email Server: This is the computer out there somewhere on the internet that handles all the mail being sent to individual accounts under a given domain. For example, if you have an email account with Verizon or Comcast (i.e. yourname@verizon.net or yourname@comcast.net) then any mail sent to you is initially sent to the verizon.net or comcast.net email servers.

Email Client: Is the software mechanism that you use to access and read your email. Common email clients are Microsoft Outlook Express, Microsoft Outlook, Mozilla Thunderbird, Apple Mail, Microsoft Entourage, and Eudora. These clients just listed all run on your PC, Macintosh, or Linux machine. There are also web-based email clients such as Google GMail, Yahoo Mail, Microsoft Hotmail, Horde, SquirrelMail, and others.

So, if I were to send you an email from my office computer the steps involved would be as follows:

  1. I would compose the email on my local computer using an email client (in my case, Mozilla Thunderbird).
  2. Once I am ready to send the email (having addressed it to yourname@yourdomain.com), I would click the send button in my Thunderbird client.
  3. Thunderbird would then contact an outgoing email server (usually either an SMTP or Microsoft Exchange server) and request that the email be sent. The outgoing email server will usually require me to provide it a login and password combination. In my case, the outgoing email server could be owned by my broadband provider (Verizon), or by my hosting environment (SalemDesign.com).
  4. Assuming Thunderbird provided a valid login/password combination, the outgoing email server will upload my email. It then looks at the address yourname@yourdomain.com and sends the email off across the internet to your incoming email server. (It is a tad more complicated than that but we don’t want to get bogged down in those details.)
  5. The incoming email server associated with yourdomain.com receives the email and it will check to see if it “knows” about an email account belonging to “yourname”. If you do have a valid account on the incoming email server then the email gets stored in that account.
  6. The next time you run your email client, it will query the incoming email server and “ask” if you have any emails waiting to be read. If you do, those emails get downloaded to your email client and (usually) deleted off the incoming email server. You can then open the individual emails and read them.

One would like, of course, all these emails that get downloaded to our email clients to be ones we want to read (i.e. from friends, business associates, etc.). Unfortunately, as we all know too well, most of the email we receive is junk or worse from people trying to sell us something we don’t want or worse.

How do our email addresses get onto spammer lists?

Spambots: These are software mechanisms that “crawl” over the websites (in the same way as search engine spiders used by Google and Yahoo do) and identify and collect email addresses (basically anything that looks like blahblah@blahblah.com or .net, or .edu, etc.). Given the existence of these evil mechanisms, any time you have your email address listed on a website whether it is your own or someone else’s, then you will be getting spam.

SelfInflicted: Anytime you provide your email to someone else, they may turn around and use it to spam you or sell your address to someone else who does. So be careful who you give your email address to… Even if they are apparently legit, ask them if you have to provide your email address and ask them how they use their lists and whether they sell or provide them to third parties.

Domain Registrations: any email used as part of a domain registration is publi