Why does DMARC operate on the From-address, and not the envelope sender (Return-Path)?

Question

Several emails sent from my webserver to a Gmail address, where the From: address is [email protected], have been marked as spam by Gmail. The From: field is populated from form data, and corresponds to the visitor's actual email address, which often is a Gmail address. The Return-Path: is always pointing to an address [email protected], which means that SPF and DKIM checks will work.

When I inspect the raw emails in the Gmail account, I see the following:

Delivered-To: [email protected]
...
Return-Path: <[email protected]>
Received: from mywebserver.com (mywebserver.com. [my:ipv6:address])
        by mx.google.com with ESMTPS id xxx
        for <[email protected]>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Tue, 02 Feb 2016 00:40:02 -0800 (PST)
Received-SPF: pass (google.com: domain of [email protected] designates my:ipv6:address as permitted sender) client-ip=xxx;
Authentication-Results: mx.google.com;
       spf=pass (google.com: domain of [email protected] designates my:ipv6:address as permitted sender) [email protected];
       dkim=pass [email protected];
       dmarc=fail (p=NONE dis=NONE) header.from=gmail.com
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mywebserver.com; s=mydkim;
    h=Date:Message-Id:Sender:From:Subject:To; bh=w2snQznwxlVRVACmfQELC7VGmD1dcYdiCXbCIRYFKRs=;
    b=a0Vy3Ky43J5FdiWSuQ4qvTTH47G+Js0W/qtRU5gMlxfesNqrlyaIyExaIZlWvHNL4o0LNOF1GI94w4C41mmH+2JIkMEQZazw0MainP7UyUgsm/RZbAWoRuecPv+k108FlsWMP/l1UttXAdlvBVJmV2UGsYYlSSjKErQEF8tv3K0=;
Received: from apache by mywebserver.com with local (Exim 4.80)
    (envelope-from <[email protected]>)
    id 1aQWVF-00009b-2X
    for [email protected]; Tue, 02 Feb 2016 09:40:01 +0100
To: [email protected]
From: Website User <[email protected]>
Sender: [email protected]
...

Note that both the SPF and DKIM checks are passing, but the DMARC check is not. After some searching, I tracked this down to DMARC using the From: address to get its reference domain from, according to this answer on stack overflow.

Three questions:

1. Is it likely that indeed the dmarc=fail is the cause of the email being assigned to spam by Gmail?
2. Why does DMARC operate on the From: address, rather than the Return-Path (envelope sender) as SPF and DKIM do?
3. If now also the From: header has to correspond to an address @mydomain.com then how should we specify the actual (logical, flesh and blood) sender of the message?

Solution 1

Think of SPF and DKIM as ways to validate the mail path, and think of DMARC as an extension that also validates the message sender. Think of this as delivering a FedEx letter. It's easy to validate where the envelope was shipped from, and that the courier was legitimate, but it doesn't provide a way to prove that the letter inside the envelope is really from the person whose name is printed on it.

Your webserver is a valid SMTP server for mywebserver.com and that your Sender address is legitimate, but that's not enough for other servers to trust that you have permission to send as [email protected] . How does GMail know that your server hasn't been hacked or otherwise used for malicious intent? Gmail's servers aren't going to blindly trust you to send mail as one of their users -- unless maybe you are hosted by them, and then you'd probably have trouble sending to Yahoo.

To address your first part of the question, yes, it's very likely that this is why GMail is categorizing it as spam. The oldest forms of spam center around spoofing the "From" address. This is what most users see when they get a message, and is the primary field they want to trust. When a message from a legitimate mail server is sent using a From address that doesn't belong to that mail server, it's still a red flag.

As you mentioned, DMARC operates on the From address as part of the specification. Granted, it makes it harder to write web apps that send on someone's behalf, but that's sort of the point. As to why they do it - well, that's up to the designers of the specification, but it's a trade-off. They are taking the high road and making a system that works very well if you stay within that limitation. Perhaps future mechanisms will find a way around this.

The unfortunate solution is to only use addresses that you have control of. To address your third question, sign your messages with your domain name, and mention in the body that it was sent on behalf of [email protected]. Otherwise you will have to request that your recipients add the address to their whitelist. It's not much fun for a legitimate web app developer, but it will protect the sanctity of the recipient's inbox. You might have luck using the Reply-To header with the web user's email address.

There is a discussion of this limitation on this DMARC thread.

In the mean time, you can try to make sure that your server isn't blacklisted on any RBLs. It could be that you can fail DMARC but still get through some spam filters if you have good enough reputation... but I wouldn't rely on it.

Solution 2

There are two "why" questions:

1. Why does a receiving mail server perform the check in this manner

• because that's what section 3.1 of RFC 7489 says to do

2. Why was DMARC designed that way?

• because the people who designed it apparently didn't read section 3.6.2 of RFC 5322, or misinterpreted it, or ignored it.

That section clearly establishes that a Sender: header, when present, takes priority over a From: header, for the purposes of identifying the party responsible for sending a message:

The "Sender:" field specifies the mailbox of the agent responsible for the actual transmission of the message. For example, if a secretary were to send a message for another person, the mailbox of the secretary would appear in the "Sender:" field and the mailbox of the actual author would appear in the "From:" field. If the originator of the message can be indicated by a single mailbox and the author and transmitter are identical, the "Sender:" field SHOULD NOT be used. Otherwise, both fields SHOULD appear.

Contrast this with the rationale given in RFC 7489:

DMARC authenticates use of the RFC5322.From domain by requiring that it match (be aligned with) an Authenticated Identifier. The RFC5322.From domain was selected as the central identity of the DMARC mechanism because it is a required message header field and therefore guaranteed to be present in compliant messages, and most Mail User Agents (MUAs) represent the RFC5322.From field as the originator of the message and render some or all of this header field's content to end users.

I contend that this logic is flawed, as RFC 5322 goes on to call out this error explicitly:

Note: The transmitter information is always present. The absence of the "Sender:" field is sometimes mistakenly taken to mean that the agent responsible for transmission of the message has not been specified. This absence merely means that the transmitter is identical to the author and is therefore not redundantly placed into the "Sender:" field.

I believe that DMARC is broken by design, because

• it conflates authority to send and proof of authorship;
• it misinterprets prior RFCs, and
• in doing so it breaks any previously compliant list-serv that identified itself by adding its own Sender: header.

If a Sender: field is present, DMARC should say to authenticate that field and ignore the From: field. But that's not what it says, and therefore I consider it to be broken.

RFC 7489 continues:

Thus, this field is the one used by end users to identify the source of the message and therefore is a prime target for abuse.

This is simply wrong (in the context of justifying ignoring the Sender: header). At the time that DMARC was designed, common email clients would routinely display a combination of the information from Sender: and From: fields, something like From name-for-mailing-list@server on behalf of [email protected]. So it was always clear to the user who was responsible for sending the message they were looking at.

---

Suggestions that Reply-To: is an adequate replacement are also flawed because that header is widely misinterpreted as "additional recipient" rather than "replacement recipient", and replacing the original sender's Reply-To: would impair the functionality for those users.

Solution 3

1) yes, likely the dmarc failure will cause gmail to junk your mails

2) also would be interested in an answer for this

3) I would (and we do) use the reply-to field for the customer address, our mails look like this:

from: [email protected]

to: [email protected]

subject: contact form

reply to: [email protected]

Hope this helps

Comments

[Ale]

Except for the desktop Outlook, common email clients display nothing about the Sender: header field. See other replies here.

[Martin Kealey]

Thanks for the mailing list link. It seems I'm not alone in my opinions, but there are reasoned arguments both ways (but I still contend that the opposing camp conflates "proof of authorship" with "authority to send"). Although DMARC's RFC was posted in 2015, it was in development for many years before that, perhaps as far back as 2010, when desktop programs were still a large proportion of email clients. Let's not forget that the prime proponents of DMARC were entities who built large webmail farms, to support their existing practice - like not showing the Sender.

[Martin Kealey]

Whilst it may now be true that the majority of clients only show the From: header, it seems to me that mostly they only show the name, which only proves my point further: that certifying something that the user never looks doesn't prevent abuse, and does impede reasonable uses. See rfc-editor.org/rfc/rfc7960.html for a more complete enumeration of the problems with DMARC as it stands.

[Ale]

Here is a list of workarounds.