Setting up an SPF record for a shared hosting service with lots of email gateways

Question

My website is on a Hostgator shared host, and I need to set up SPF for my email so my outgoing emails won't get bounced. So, I have to add a TXT record to the DNS for my domain, listing all possible senders of my domain's email. Shouldn't be a big deal, right?

Unfortunately, I found that Hostgator routes outgoing email from shared hosts through multiple email gateway servers, all with domains in the format gatewayXX.websitewelcome.com for some two-digit XX. So, I started sending test emails to myself, adding new servers as I found them. After finding six of them, I had the following SPF record...

v=spf1 +a +mx +ip4:50.87.144.137 +a:gateway33.websitewelcome.com +a:gateway26.websitewelcome.com +a:gateway20.websitewelcome.com +a:gateway24.websitewelcome.com +a:gateway22.websitewelcome.com +a:gateway36.websitewelcome.com ~all

... and kept finding more.

I finally wrote a script that went through the one hundred possible domains in the above format, and found that 28 of them existed, mapping to a total of 705 IP addresses. Nope: can't have that many domains in my SPF record, as it would be too many DNS lookups as well as just too many characters.

How do I accomplish this? Must I scan the 705 IP addresses and figure out what CIDR-notated networks cover them all? Am I then giving permission for anyone hosted at Hostgator to spoof my domain name in their outgoing mail? Is there a completely different approach I should take to this? Or should I be looking for a different hosting setup with a small number of publicly visible outgoing email servers?

Solution 1

You could have saved yourself a lot of time by just searching the hostgator support site. :)

From the above linked page:

v=spf1 a mx include:websitewelcome.com ~all

Solution 2

Hostgator, probably like most shared hosting services, has a master SPF record that is designed to cover all of its email servers. This allows the company to reorganize their servers without all of their customers having to edit their SPF records. To include Hostgator's record in my own, I needed to set my SPF record to the following:

v=spf1 +a +mx +ip4:50.87.144.137 +include:websitewelcome.com ~all 

where 50.87.144.137 is the primary IP address of my server, and ~all means to "soft-fail" if an outgoing email doesn't match the websitewelcome SPF record. (Eventually I'll change that to -all, which will bounce any non-matching email.)

However, there is a problem. Here's Hostgator's record at websitewelcome.com (line-wrapped):

v=spf1 a mx ip4:64.5.0.0/16 ip4:67.18.0.0/16 ip4:69.41.224.0/19 
ip4:69.56.0.0/16 ip4:69.93.0.0/16 ip4:70.85.0.0/16 ip4:74.52.0.0/16 
ip4:174.132.0.0/16 ip4:174.120.0.0/16 ip4:173.192.100.229 
include:spf2.websitewelcome.com include:spf.websitewelcome.com 

Here's the included record at spf2.websitewelcome.com:

v=spf1 a ip4:192.185.0.0/16 ip4:50.116.124.0/22 ip4:173.192.111.0/24

And here's the included spf.websitewelcome.com record (again, line-wrapped):

v=spf1 a mx ip4:173.192.0.0/16 ip4:174.121.0.0/16 ip4:174.122.0.0/16 
ip4:174.37.0.0/16 ip4:184.172.0.0/16 ip4:184.173.72.107/32 
ip4:50.22.0.0/16 ip4:70.84.0.0/16 ip4:70.86.0.0/16 ip4:70.87.0.0/16 
ip4:74.53.0.0/16 ip4:74.54.0.0/16 ip4:96.125.0.0/16

Hostgator had to split their record into three pieces because their list of IP ranges is almost twice the maximum length of an SPF record (256 chars). But, when I tested my SPF record with http://www.kitterman.com/spf/validate.html, I get an PermError SPF Permanent Error: Void lookup limit of 2 exceeded error. I get the same error even if I plug in the websitewelcome.com address.

My understanding is that this limitation is controversial; I'll have to hope that the destination email servers won't enforce it.

A secondary problem is the breadth of the resulting set of networks allowed in the SPF record: there may be millions of Hostgator clients who could spoof my domain name in their outgoing mail. That can't be changed, though, without switching providers. Maybe someday.

Solution 3

Did a little digging and here is what I've come up with:

v=spf1 +a +mx +ip4:50.87.144.50 +include:spf.websitewelcome.com +include:spf2.websitewelcome.com ~all

where 50.87.144.50 is my server’s main interface IP Address...

Avoiding the straight websitewelcome.com gets around the "too many lookups" issue and still covers all associated addresses since a lookup of websitewelcome.com now returns the following:

Found v=spf1 record for websitewelcome.com: 
v=spf1 include:spf.websitewelcome.com

It no longer has IP addresses of it's own. So, you can safely drop it from your record. And, since neither that record, nor spf.websitewelcome.com reference spf2, the only way to be sure you have them all is to base your record on the one that I posted.

And, try this testing tool out: https://www.mail-tester.com/ It's simple, easy to use, and can be really helpful when trying to track down issues.

Comments

[Jacob Evans]

You should find a better host or use Sendgrid/Mailchimp plugins for your website. dmarcian.com/spf-survey/websitewelcome.com

[Daniel Griscom]

Interesting. I just got an award for this question being viewed 1000 times, but it still has a net score of -2. Then again, I got 24 votes (and almost 4k views) for this not-all-that-substantial question. Ah, well...

[Daniel Griscom]

Just passed 2500 views, but the score is still -2... Stack Exchange voting is weird...

[Daniel Griscom]

Well, rats. I searched all over Hostgator, and in fact skimmed that page, but the page was presented as a generic description of SPF records rather than a "What to do as a Hostgator customer", and the critical "include..." statement was slipped in without comment, so I missed it.

[Daniel Griscom]

Most excellent information. If you copy the useful parts of my answer into yours in order to make the answer complete (e.g. that using the main Hostgator SPF record is problematic and why) then I'll switch the accepted answer to yours. Thanks.

[Akseli Palén]

"spoof my domain name in their outgoing mail" That sounds quite scary: could another user on the same shared host just send made-up invoices from a verified email address of my company?