Let's Encrypt DNS challenge with multiple public DNS providers

Question

We use two DNS providers for redundancy. Since the DynDNS DDOS attack of 2016, I expect we are not alone in this practice.

I am attempting to use the Let's Encrypt certbot with DNS challenge. Having two DNS providers seems to pose a problem. Do both DNS providers need to be updated with identical TXT records as part of the challenge process?

The real question is, how does the Let's Encrypt ACME Certificate Authority (CA) validate DNS TXT entries? Does it simply query the public DNS like any client would, or does it query against the specific DNS provider that maps to the certbot plugin I'm using? I suspect the former.

I attempted to create a certificate using only one of the plugins. The TXT entries were created, but validation failed. I expect I will need to update both DNS providers (or figure out how to link the two providers so they synchronize changes).

The documentation doesn't seem to indicate that using multiple plugins of the same type is possible, although I haven't tried it yet. One of my providers requires that I upgrade to gain access to their API.

If it is not possible to chain multiple authentication plugins, then I believe my only option for automated renewal is using the pre- and post-hooks with a custom script to modify both DNS providers.

I'd appreciate confirmation of this before I undertake the work and expense required (or choose to manage the renewal process manually).

EDIT: I'm aware that StackExchange wrote and published its own open-source DNS updater, dnscontrol, which could be used in pre- and post-hook scripts. However, one of our providers is not supported.

Solution 1

There are plenty of 3rd party ACME (LetsEncrypt) clients out there that provide more features than the default certbot. They maintain a list on their site here: ACME Client Implementations

A lot of them have some sort of DNS plugin architecture and some allow you to specify different plugins for different names in a cert. But none that I know of allow you to combine plugins per-zone (such that more than one plugin would service a single zone's TXT record).

However, if you're already considering writing pre/post hooks to get this done. I'd suggest using something like acme.sh. It has a ton of pre-written DNS plugins. You could probably fairly easily combine two or more of them into a single merged plugin that would effectively accomplish your goal.

Solution 2

Letsencrypt validates your TXT entries by just querying a random resolver, with no control whatsoever about which one of your name servers is going to be used.

• letsencrypt/boulder/bdns/dns.go

  chosenServer := dnsClient.servers[rand.Intn(len(dnsClient.servers))]

I recommend you delegate the _acme-challenge.example.com. zone to just one dns provider you have automated access to. You will never have a DoS situation at any of your DNS providers for longer than it takes to change this - or for longer than letsencrypt certificates are valid for. Therefore, its okay to skip the redundancy for ACME only, so you can use all the automation just like before you added more name servers.

edit based on comments: This strategy will not allow to continue certbot, as the respective certbot plugins is not correctly handling aliased or delegated validation domains. It should look to modify the zone of the validation record, but attempts to modify the zone of the validated domain, disregarding that it is very valid and for them to differ.

Other acme clients may require to explicitely state the location of the validation record, as is the case with acme.sh

Comments

[USD Matt]

I appreciate the possibility that a single DNS provider could have all their servers attacked, where-as using multiple providers gives the possibility of having one provider down while the other is working. I'm not sure how common it is for people to manage two completely independent DNS providers though. Usually you'd just want to have one master and let any other DNS servers pull data from that. If the master goes down, the slaves just don't update for a while

[USD Matt]

Let's Encrypt will just do a DNS lookup like any other client and could get either provider

[designarti]

@USDMatt Can you cite your evidence please?

[USD Matt]

specific DNS provider that maps to the certbot plugin I'm using not sure what you mean by that. The ACME clients all implement the same ACME protocol. They are given a token to insert in DNS, send a simple response to say it's ready to be checked, then the server tries to lookup that record via the normal DNS system. Trying to do anything else depending on the client/plugin would likely introduce methods to trick them into looking up records you control rather than those publicly available on the Internet.

[designarti]

The DNS provider in question, DNSMadeEasy, is not yet supported. There is a PR against dnscontrol, but it's stalled out. github.com/StackExchange/dnscontrol/pull/15

[Patrick Mevzek]

Your question is strange and not related to LE or anything else: if you use multiple provider/nameservers, whatever change you want to appear in your zone should be set on ALL your nameservers. Otherwise, expect major breakages. So any DNS change prompted by LE verification should appear in all of your nameservers (the problem is exactly the same if you use only one provider with X nameservers, it has to update all of them...)

[designarti]

I like this solution. I'd use Route53 for the sub. I've never delegated a subdomain before. This looks like the relevant doc, do you agree? docs.aws.amazon.com/Route53/latest/DeveloperGuide/…

[Zoredache]

delegate the _acme-challenge ... zone - this would work for a single record, but that doesn't seem like it would scale very well if you had many zones, or many records and didn't want a wildcard.

[Ryan Bolger]

Think of it less as taking another dependency and more of trading one dependency for another. Some of those 3rd party clients are better maintained (IMHO) than certbot. For example, acme.sh had support for the ACME v2 specification long before certbot did.

[designarti]

I attempted the subdomain solution and I think I hit a fatal blocker. The certbot-dns-route53 plugin expects to modify the primary domain, which lives in one Route53 hosted zone. I modified the IAM policy to allow it to modify the new subdomain, which is in a different hosted zone. The plugin died, complaining that it could not modify the hosted zone that it wanted to modify. It's opinionated about which hosted zone it wants to modify, and it doesn't appear to be smart enough to handle the subdomain solution.

[anx]

oh, i see the problem "_find_zone_id_for_domain" is supposed to figure out which one to modify, but they use that to figure out the zone responsible for the validation domain, not the actual record.