Legitimate reasons SMTP “MAIL FROM:” will not match “From:” Header in DATA

Question

Is there ever a legitimate reason for the SMTP “MAIL FROM:” field to not match the “From:” field in the DATA section of a message, besides mailing lists?

From https://stackoverflow.com/questions/1750194/smtp-why-does-email-needs-envelope-and-what-does-the-envelope-mean:

“But, to continue your snail mail metaphor, most professional letters will contain the sender's and recipient's addresses printed on the letter itself. Those addresses are not necessary for the postman, but are instead a courtesy to the recipient. So it's sensible that email would work the same way.”

The problem with this line of logic lies here: “courtesy to the recipient”. Including the “From:” address in an email via SMTP is not a courtesy; it is required if the recipient is to be able to send a reply.

From: How to limit the From header to match MAIL FROM in postfix?:

“But if you really want to ensure From: and MAIL FROM then you have to apply header_checks so that Return-Path: matches From:”

What are the implications of doing this? Mailing lists would obviously be a problem. Are there any other legitimate uses of differing “MAIL FROM:” and “From:” header information?

Solution 1

There are many reasons why the Header and Envelope From addresses may not match. Most concern automated processes sending mail, where delivery issues need to be reported to an address that is not representative of who sent the mail, or who it was sent on behalf of, or who should be replied to. Mailing lists as you've pointed out are a good example.

The main reason why a message sent from a user's mail client might have differing from addresses is forwarded mail. The mail content should then be reasonably faithful to the original, but in case of delivery errors, those should be reported to the user who forwarded the email, not the original sender.

Besides the SMTP header, There are a variety of MIME headers that various programs use to try to distinguish between the original sender, and intermediate sender, and/or the preferred address to report errors to.Eg Reply-To, Sender, Originally-From, Errors-To, etc, etc, Each with different semantics. Some of these have standards support, while many more do not, but may be in use anyway. The way various mail programs behave in practice varies considerably.

Whether a manner of addressing mail is advisable is a different matter from whether it is "legitimate" as you ask. If you're considering legitimacy here in terms of something like policy for handling potential spam, then no, I don't think you'll be able to make a simple distinction in this way.

Have a think though about DKIM signing of email, and SPF authentication of mail servers for email domains. If you're sending much mail, it may be important to be able to authenticate your mail in these ways, and that may have implications for the addressing of mail From headers, as you can only authenticate mail related to domains for which you have authority.

--

Extended on request:

A MIME 'Reply-To' header directs a MUA (Mail User Agent, usually a person's mail client) to send replies to a different address, instead of the MIME 'From' address. This is not used by an MTA (Mail Transport Agent) for things like errors.

Usually an MTA would use the SMTP Envelope 'MAIL From' address to send errors to. IT can be overriden with a MIME 'Errors-To' header, which is an MTA instruction. Not all MTAs will honor it, so it's an inferior mechanism to setting the SMTP Envelope address, but there are many circumstances where it may be possible to set MIME Headers in a message, but not the SMTP Envelope From address. Eg software running in a shared hosting environment may find itself in this situation.

'Sender' is much more ambiguous as an instruction to software agents, but indicates who or what sent the email where that is distinct from the From address which is more like who the mail was sent on behalf of. Eg when you fill out an on-line mail-your-politician form, it would be very appropriate for the resulting email to use your mail in the From header, but have a Sender address related to the organisation who set up the form.

'Originally-From' is used by some MUA software when forwarding mail, with the forwarder's address being used for the 'From' header. Other MUAs will Leave the From address alone, and use a 'Resent-From' header. Whether MUAs recieving these various headers emails interpret the headers usefully, or even display them is quite variable. When replying to a mail that has been forwarded to you, who should the reply go to by default? Maybe best to set that 'Reply-To' header?

The behavior of MUAs is variable, aand poorly defined, although it does seem to be improving over time. By contrast, the semantics of the Envelope are much more defined. There's typically been a strong position that MTAs should never concern themselves with the MIME headers, but as MTAs are increasingly held responsible for mail content (eg see SPF and the emerging DMARC standards), there's pressure for the clarity of that position to be degraded. Long-standing mechanisms like Errors-To have also conflicted with the notion of MTAs not looking at header content, which is part of why those mechanisms have always been inconsistently applied. Philosophies of software authors vary.

You might find it useful to look over https://www.rfc-editor.org/rfc/rfc4021#section-2 , but remember that the actual practices of the multitude of mail software out there varies in ways which are not necessarily standards blessed.

It's fine to try to come up with a clear philosophy of how you think mail should be used, but don't expect that everyone else will do things how you think they should.

Solution 2

Automated processing is a big reason. You want to be able to send any bounces/autoreplies/errors to be processed separately, otherwise those messages disappear, or get ignored, or some poor sap has to dig through them. Yes, adding an X- header for processing is possible, but a lot of the time bounces/etc. won't include the original email or only a mangled portion of it and you won't be able to get the source.

For example, say someone signs up on your website, and you send them an email saying thanks for signing up. Your MAILFROM and From might look like this:

MAIL FROM: <[email protected]>
From: Website X <[email protected]>

This way, the user sees the "friendly from" in the mail client. But if the user doesn't exist, their MTA will send the bounce message to:

[email protected]

and an automated process can easily pull the userid (the 123123123 part) and the part of the system that created the bounce(the -signup- part) from the header and easily delete/mark that user as disabled.

Solution 3

The mail from: in the smtp conversation is designed to be the place where bounces will go The From: header in the message body is used to display to the recipient and as the reply address if the Reply-to: header isn't set.

Emails which shouldn't generate a bounce should set the empty sender in the envelope, for instance a return receipt will usually have: mail from:<> with the user's name in the from: header.

Another situation is where a mail server is using BATV to reject faked bounces. The mail from: will be in the form [email protected].

Solution 4

Unless I'm not reading my headers or the question correctly, emails from my BlackBerry are sent from the BlackBerry server and basically none of the fields match. Small percentage of users, I realize. I have been looking at this recently in evaluating my mail server. Below is an anonymized email sent from my BlackBerry to my Gmail account:

Delivered-To: [email protected]
Received: by 10.50.11.138 with SMTP id q10csp217364igb;
        Wed, 14 Aug 2013 00:18:53 -0700 (PDT)
X-Received: by 10.42.83.84 with SMTP id g20mr4290552icl.10.1376464731205;
        Wed, 14 Aug 2013 00:18:51 -0700 (PDT)
Return-Path: <SRS0=BQ46+U=R3=example.com=[e-mail]>
Received: from smtp08.bis6.us.blackberry.com (smtp08.bis6.us.blackberry.com. [74.82.85.8])
        by mx.google.com with ESMTP id lq6si7427361icb.102.2013.08.14.00.18.51
        for <[email protected]>;
        Wed, 14 Aug 2013 00:18:51 -0700 (PDT)
Received-SPF: pass (google.com: domain of SRS0=BQ46+U=R3=example.com=[e-mail] designates 74.82.85.8 as permitted sender) client-ip=74.82.85.8;
Authentication-Results: mx.google.com;
       spf=pass (google.com: domain of SRS0=BQ46+U=R3=example.com=[e-mail] designates 74.82.85.8 as permitted sender) smtp.mail=SRS0=BQ46+U=R3=example.com=[e-mail]
Received: from b15.c8.bise6.blackberry ([192.168.0.115])
    by srs.bis6.us.blackberry.com (8.13.7 TEAMON/8.13.7) with ESMTP id r7E7InfH019786
    for <[email protected]>; Wed, 14 Aug 2013 07:18:49 GMT
Received: from 172.29.201.172 (cmp2.c8.bise6.blackberry [172.29.201.172])
    by b15.c8.bise6.blackberry (8.13.7 TEAMON/8.13.7) with ESMTP id r7E7IlCt013236
    for <[email protected]>; Wed, 14 Aug 2013 07:18:47 GMT
X-rim-org-msg-ref-id: 587275596
Message-ID: <58727559[e-mail]>
Reply-To: [email protected]
X-Priority: Normal
Sensitivity: Normal
Importance: Normal
Subject: Test
To: "Recipient Name" <[email protected]>
From: [email protected]
Date: Wed, 14 Aug 2013 07:18:45 +0000
Content-Type: text/plain
MIME-Version: 1.0

Test
Sent via BlackBerry by AT&T

View more solutions

Comments

[Juanjo Daza]

I think I remember seeing a X-header for returns and NDRs. Do you know when and how this is used?

[Richard Salts]

X-headers were originally added as a means of MUAs and/or MTAs to put custom metadata and are not specified in any standards, although some do become defacto standards. You might be thinking of multipart/report mime type defined in rfc 6522 which was defined to help software classify messages which are bounced automatically. These will still be sent with an empty envelope in mail from:

[Steve Townsend]

There is an excellent reason for this - SPF rewriting. If BlackBerry wasn't doing this than you would get a lot more SPF failures sending from your device.

[Paul]

True, but that's because BlackBerry doesn't use my server for sending.

[Juanjo Daza]

I still have time to award this bounty and am interested in additional scenarios such that would require the use of the headers: ` Reply-To, Sender, Originally-From, Errors-To`

[Juanjo Daza]

Thank you for the additions. I'm hoping to get a clear understanding of what the MTA expected behaviors are, vs what they are implemented as. Also, you may be interested in this q: I just posted on Sec.StackExchange regarding email whitelisting (akin to BATV) security.stackexchange.com/q/41008/396