Is SRS rewriting absolutely necessary for a forwarding mailserver?

Question

I am operating a Postfix email server for my domain, say mydomain.com. It mostly acts as a forwarding email server: users receive an email address @mydomain.com, but usually elect to have their address forward to an external inbox (Gmail, Yahoo, etc). There are a few thousand addresses being forwarded, so the server handles a fairly significant volume of mail traffic.

In the past, the server did not use SRS rewriting. This of course meant that forwarded mail would fail SPF checks, as my ip address is not technically authorized to send email on behalf of the original sender's domain. However, from what I can see it doesn't seem to be causing any significant issues. Generally no complaints from users as Gmail, Yahoo, etc. seem to be smart enough to ignore the SPF failures and deliver the messages anyway.

With this in mind, is it really necessary to enable SRS rewriting? I am considering enabling it but my main concern is that my domain will get blacklisted for sending spam when spam inevitably gets fowarded. Wouldn't the rewrite make it appear as if I'm the originator of the spam? (At least, this is my understanding from reading Gmail's Best Practices for Forwarding Mailservers).

Granted, I am already taking some of the recommended precautions like using SpamAssassin to add "SPAM" to the subject line of suspected spam before forwarding, not forwarding high confidence (score 15+) spam, and using the spamhaus blocklist, but these measures aren't perfect and spam can still slip through unmarked.

Is enabling SRS rewriting worth it, if it increases the risk of getting wrongly marked as a spammer? Or would it be safer to just leave it as is and ignore SPF failures?

Solution 1

It seems to me that what your question boils down to is "how many mail servers out there check SPF records on incoming email?". If it's most of them, SRS is an absolute requirement for a forwarding server; if it's none of them, you don't need SRS.

Unfortunately, I can't immediately put my hands on any academic work on this. But since I check SPF on incoming email, I can say with certainty that some mail servers do check it. Any of your clients who have your server forward to accounts on my server will lose email sent from senders who advertise an SPF that ends (as they all should) -all, unless you use SRS. So I can say with certainty that without SRS, some of your customers' email will not be delivered.

I apologise to Marc that I can't read German, so I can't say whether the PDF he quotes advances compelling arguments, but I can reiterate that without SRS, some fraction of your customers' email won't be delivered. I cannot say what that fraction is, but it isn't zero - and that given, I don't think you have any alternative but to run SRS.

I agree that your server will not be helping itself by forwarding SPAM, but in my experience most of the reputational damage is done to its IP address, not the envelope-From domain; this will be done regardless of SRS usage.

The deeper answer to your question is that, between SPF and its (ill-considered and internet-breaking) followup DMARC, it seems to me that mail forwarding services have had their day. I've already required all but one of my users to have final delivery on my server, and that one user will have to change or leave in 2016. Nowadays, many webmail systems will allow integration over multiple mailboxes by collecting off-server mail using IMAP or POP, and many mail clients allow multiple IMAP or POP accounts to present as a single integrated INBOX, so forwarding isn't the boon to centralised reading that it used to be.

In short, I'd say you need SRS in the short term, and a new business model in the longer term.

Solution 2

SRS seems to be a nice idea on the paper, but doesn't work very well in practice according to the folks of Heinlein Support (they are running a medium sized mail service with over 100000 accounts.)

Details are in their talk, though in German, why: https://www.heinlein-support.de/sites/default/files/SPF-DKIM-Greylisting_FrOSCon_2012.pdf

The main reason is that SRS is a small patch for serious problems of implementing SPF in reality, because SPF does not cover some common use cases of email very well. For SRS to make sense though it needs to be deployed on a large base of servers, which is unlikely to ever happen. So until it is deployed on that large base of servers, it doesn't make quite much sense at all.

The problem with the large mail providers though is that they nowadays do have a real big user base and they are implementing more and more techniques (the successor for DMARC is already in the pipeline), which does it make more and more difficult for a normal mail server setup to send mails to them in a reliable kind of way.

If you want to get your mail better delivered to the big mail providers like Gmail, Hotmail and so on you should implement at least DKIM and DMARC, but also set it to soft fail at best and maybe implementing some rate limiting mechanisms on the mail delivery would work wonders for you.

This problem with the big providers is the reason why there are nowadays services like Mailchimp, Mandrill or Returnpath. Those providers do pay money to Google&Co. for better delivery quality.

Solution 3

I agree with each word of @MadHatter, BUT important fact about Google!

If you provide a forwarding service to gmail, there is a good chance you provide SMTP access too so your gmail users can send mails from gmail on behalf the address stored by you.

In that case - gmail knows you are a forwarder for this email and doesn't flag your forwards as spam even though it's fail SPF check.

You can send your clients mails from [email protected]. The message will get to their inbox without any warning! (Microsoft does have -all in the spf record)

Checked and verified. Example attached.

this message went to the inbox.

Comments

[roaima]

I know from experience that some ISPs in the UK will reject inbound email that claims to be from their own customers, but that has not been sent from their own mailers. The same may also become true for Gmail, Yahoo, and AOL. Such situations can only be resolved by rewriting the sender address.

[Marc Stürmer]

The thing is that SRS is a solution to remedy forwarding issues of SPF. SRS rewrites e.g. user@A to A=user@B and the SPF records of B are in charge then. Problem: B can still forge adresses! So therefore some are adding crypt-hashes and time stamps to the rewritten address. For this to work on great scale it needs global adoption which isn't there. It also only works if something is being forwarded once, but not more. Also answers are a problem. Also keep in mind, that SPF is a technic to protect your own domain of abuse, nothing more.

[MadHatter]

@MarcStürmer "SRS is a solution to remedy forwarding issues of SPF": yes, that is exactly what it is. SPF is known to break simplistic forwarding; if you don't think SRS is a price worth paying, don't advertise an SPF record. "Problem: B can still forge adresses": not to A's domain, or any other domain protected by a decent SPF record, or the mail will be rejected under SPF; but other than that, yes, it can, and I don't see it as a problem. "SPF is a technic to protect your own domain of abuse, nothing more": I agree.

[Adrian Zaugg]

@MarcStürmer: "It also only works if something is being forwarded once, but not more." is wrong. SRS works completely well over several forwarding servers. It suffers only if there is a non tagging server in the line. But this is the same problem than with any non tagging server in general, be it a first or a later forward hop. In an SPF world, you do not need SRS, you just need to take over the responsibility of the forwarded mail and make sure you can deliver a possible bounce back. SRS is just one technique that does this, google e.g. uses something different.

[Adrian Zaugg]

The problem here is SPF not SRS. As long as some ISP do use SPF, you need to implement SRS (or something similar) to make forwarding working with all of them. The problem with greylisting is different, you should "unpack" SRS tagged sender addresses for greylisting (aswell as BATV tagged mails).

[mbirth]

The problem is, using SRS breaks the DMARC alignment check (i.e. envelope sender != From:-header) and will make Gmail reject messages if the domain in the From: header has p=reject in their DMARC policy. If you rewrite the From:, too, the mail will be checked according to your own domain's rules. But a DKIM check will fail and the sender shown in the mail client is mangled.

[MadHatter]

@mbirth afaik, you are right. But I personally regard DMARC as a complete disaster, not least as it unilaterally broke mailing lists, and (in my capacity as admin of a big community list) simply advise people against using any ISP that publishes a p=reject policy. If SRS breaks DMARC, well, that's just tough on DMARC.

[mbirth]

@MadHatter As much as I'd love to ignore/avoid the service with that p=reject rule, but it's the German DHL. Our main parcel service here. I was missing mails from them and noticed error reports in my forwarding server's mailbox. I now rewrite envelope- and header-senders to make the mails go through to Gmail. Luckily, that's the only service where that's needed for now.