DNS over TLS with systemd-resolved

networking dns systemd-resolved

17,411

Solution 1

Recently I've implemented DNS over TLS for home network (using AsusWRT-Merlin powered router). While exploring approaches to implement DoT for Linux workstations and servers (in the cloud - I mean someone else' computers lol) outside of home network, I found systemd-resolved as recommended by the DNS Privacy Project.

While @opinion-no9 provided a Ubuntu 18.04 specific solution (limited by the systemd version shipped with the LTS), I'd like to share a more generic and close-to-upstream one:

A bit of background - timeline for DNS over TLS support in `systemd`

236 DNSSEC support for RFC 8080 (ed25519 keys and signatures)
239 systemd-resolved now supports opportunistic DNS-over-TLS, Off by default
243 systemd-resolved gained support for a new strict DNS-over-TLS mode

Oh NO! Ubuntu 18.04 LTS ships systemd 237...

UPDATE - 23 April 2020 Ubuntu 20.04 LTS which ships systemd 245 (without systemd-homed ;-).

Use DNS Over TLS (DoT) on generic Linux systems

So for generic Linux distro with relative close-to-upstream kernel, systemd, glibc, toolchain, GNU utils, etc

edit /etc/systemd/resolved.conf

[Resolve]
DNS=1.1.1.1 1.0.0.1 8.8.8.8
#FallbackDNS=1.1.1.1 9.9.9.10 8.8.8.8 2606:4700:4700::1111 2620:fe::10 2001:4860:4860::8888
#Domains=
#LLMNR=yes
#MulticastDNS=yes
#DNSSEC=allow-downgrade
#DNSOverTLS=opportunistic
DNSSEC=yes
DNSOverTLS=yes
#Cache=yes
#DNSStubListener=yes
#ReadEtcHosts=yes

Ubuntu 20.04 specific resolved.conf

[Resolve]
DNS=1.1.1.1 1.0.0.1 8.8.8.8
FallbackDNS=1.1.1.1 8.8.8.10 8.8.8.8
#Domains=
#LLMNR=no
#MulticastDNS=no
DNSSEC=yes
DNSOverTLS=yes
#Cache=yes
#DNSStubListener=yes
#ReadEtcHosts=yes

Assume systemd-resolved is enabled, restart the service

systemctl restart systemd-resolved.service

Use local stub resolver

systemd-resolved provides a local DNS stub listener on IP address 127.0.0.53 on the local loopback interface, so to use the DNS over TLS capable stub resolver, we'll need to somehow manage /etc/resolv.conf and make sure 127.0.0.53 is used as nameserver.

NOTE: systemd maintains /run/systemd/resolve/stub-resolv.conf for compatibility with traditional Linux programs. We can simply symlink to this file ;-)

ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf

NOTE: For Arch Linux, I have to replace openresolvconf with systemd-resolvconf.

Done.

Validation

Generate DNS query, then check TCP connection to specified upstream DNS server via port 853, in the following example we explicitly used Cloudflare's 1.1.1.1.

root@netbook:/etc# uname -a
Linux netbook 5.4.5-arch1-1 #1 SMP PREEMPT Wed, 18 Dec 2019 19:48:51 +0000 x86_64 GNU/Linux

root@netbook:~#  kdig -d github.com
;; DEBUG: Querying for owner(github.com.), class(1), type(1), server(127.0.0.53), port(53), protocol(UDP)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 55366
;; Flags: qr rd ra; QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 0

;; QUESTION SECTION:
;; github.com.                  IN      A

;; ANSWER SECTION:
github.com.             58      IN      A       13.236.229.21

;; Received 44 B
;; Time 2019-12-21 22:55:13 AEDT
;; From 127.0.0.53@53(UDP) in 58.0 ms

root@netbook:~# ss -tuna | grep :853
tcp   ESTAB  0       0        192.168.1.150:50504         1.1.1.1:853
tcp   ESTAB  0       0        192.168.1.150:50506         1.1.1.1:853

Or, if you want to be simple and rough, use tcpdump ;-)

tcpdump -tttt -nn -XX -vv -i <interface> dst 1.1.1.1 and port 853

Conclusion

Last but not least: This is a generic guide to enable DNS over TLS on a Linux host, it is not specific to distro or Desktop Environment (as we haven't even touched NetworkManager or alternatives ;-). Adjustments may be needed for different distros and different DEs/WMs.

Reference

Solution 2

DNS-over-TLS in Ubuntu 18.04 .... 20.04

Based on my experience, it will not work the way mentioned above on Ubuntu 18.04+ (i.e. U19, U20, U21).

Since Ubuntu 18+ uses Netplan in parallel with NetworkManager things have changed drastically:
No more manual config in the traditional ways ;-( Documentation is .... sparse.

snap may bring further changes. It establishes additional resolved.conf files. However, the following actually worked (/w good performance).

For Info about Netplan look here.

What DNS-over-TLS was working successful (works4me):

In /etc/systemd/resolved.conf ONLY change the DNSOverTLS setting to

DNSOverTLS=opportunistic

There is NO other option (see explanation here: DNS over TLS
Use a capable DNS Server. I prefer Digitalcourage servers

46.182.19.48 resp. 2a02:2970:1002::18

or https://dnsforge.de/

Why? Privacy!
This should be done in Network-Manager via GUI!

Enter the DNS server address in the GUI field for your connection under IPv4 Settings / DNS Servers and v6 respectively.

The entries will NOT show up in /etc/resolv.conf !! Which is correct. Instead, you will see nameserver 127.0.0.53

This is new Ubuntu .... no longer suitable for hobby-admins.
Not recommended but possible alternative

Setting appropriate DNS Servers can be done directly in /etc/resolv.conf in the usual format, remove the 127.0.0.53 or something else.

Problem: Gets overwritten by Network-Manager in Ubuntu!

Remedy: As true root (!) chattr the file /etc/resolv.conf

chattr +i /etc/resolv.conf

This is brute force and may disable automatic DNS caching via resolved.

Credit to the Arch documentation

However, works fine ;-) but needs manual maintenance as true root!

Tip:

You are well advised to make resolv.conf a link. This is required by resolved to work properly. As sudo-root move away the old file then

sudo ln -s /run/resolvconf/resolv.conf /etc/resolv.conf

I do not like this way but is inherently working proper.

Then reboot. Or restart network.

How to check

Some basic checks:

Check the DNS actually used by systemd-resolved:

resolvectl status

Check if DNS resolves with resolvectl:

resolvectl query archlinux.org

(Try some examples)

Check what DNS is actually used, check for leaks in VPN:

https://dnsleaktest.com/

. 2. Start Wireshark and filter for "port 53" and make web traffic.

This should not show connects on port 53 any longer. Then filter for port 853. Here should a lot go on.

Important: If all traffic uses port 853 and no traffic uses 53 you did it successfully!

Wireshark examples here.

Remark: I tried stubby. stubby does not integrate well in Ubuntu but you can get it working even with NetworkManager. There is one manual to do it successfully: How to use DNS-over-TLS on Ubuntu Linux Problem: performance was a bit of a pain. Something is weird and I did not find out the cause.

Enabling DNSSEC=yes in /etc/systemd/resolved.conf should be possible now.

Important:

This solution improves privacy a lot.

BUT is NOT sufficient if your personal integrity depends on data privacy and security!! See caveats in resolvd description. It is NOT sufficient to have opportunistic mode. Then better watch out for Tails Linux. A sad greeting to all political prisoners around the world.

updated version of this is available -- in german only -- via TOR url: http://mutant21.onastroidst6krpn.onion/schweinekraftland/DNS_sicherer_machen/sicherer%20surfen%20und%20mailen%20mit%20gesichertem%20DNS.htm

Solution 3

Just to be sure:

1.1.1.1 uses port 853 for DoT (DnsOverTLS), see page at https://1.1.1.1/
systemd-resolved uses the port automatically
remember to open the port on your firewall (imPORTant)

I still use shell commands with "ifconfig" or "ip" to up/down interfaces and set routes, "netstat" and "ss" for monitoring and so on, no systemd-networkd, network-manager or netplan. You just have to disable systemd-* packages or remove packages, to run your system the way you want to as there are no impediments for that.

BUT if you feel inclined to try new horizons and see the world from a different perspective, just in case you are configuring a desktop machine (not a workstation, nor a server) and you have a really well configured firewall all done by hand using shell scripting capabilities to configure it (cos you can) AND you also know your tcp flags well, when and HOW to allow PSH and URG, and you ALSO know sysctl options by heart and are not afraid to use it against the enemy, only then, you can use firefox implementation of a DoH client within the browser, and you can find it in configuration > General > Network Configuration, and use it in conjunction with nextdns, and you can also disable UDP and systemd-resolved entirelly and have just tcp working on your desktop machine ... Oh boy, pure bliss... or almost.

17,411

Fernando Gont

Updated on September 18, 2022

Comments

Fernando Gont over 1 year
I was trying to enable DNS over TLS via systemd-resolved. I changed /etc/systemd/resolved.conf as follows:
```
[Resolve]
DNS=1.1.1.1
#FallbackDNS=
Domains=~.
#LLMNR=no
#MulticastDNS=no
#DNSSEC=no
DNSOverTLS=opportunistic
#Cache=yes
#DNSStubListener=yes
```
While monitoring the network (with tcpdump) to see if the resulting behavior was the intended one, it seems that a TLS session is established with the target server; but, then the server closes the connection. I get the same results with 1.1.1.1, 8.8.8.8 and others.

Any ideas on how to fix this?

P.S.: systemd-resolved ends up doing parallel resolution with traditional DNS (despite the setting of "Domains" above). But my main question for this post is what may be going wrong with the TLS one.
- Daniel over 5 years
  
  What resolvectl say? Are you actually using systemd-resolved? Flip your DNSOverTLS setting to yes. You should also flip DNSSEC validation to yes. Few DNS over TLS implementation support reusing connections and will open one connection per request.
- opinion_no9 over 5 years
  
  @Aeyoun Yes, right thing, wrong order: First Fernando needs to get it working, then improve. HINT: DNSSEC works if you enable DNS-over-TLS the way described below. But it slows down significantly. Step by step getting closer to DNS-over-TLS in Ubuntu 18.04 ....
Freedo about 4 years

The systemd service doesn't seem to be working for me :( and I can't find your resolvectl command on my machine or a package with that name
opinion_no9 about 4 years

what Ubuntu / Linux you use? Can you install resolvectl from the CLI? I updated the whole description on my webpage. Assume it is not allowed to publish the link here and moreover its in German. Actually missing resolvectl will not stop your success ....
opinion_no9 about 4 years

in case it does not get deleted here the link: schweinekraftland.de/DNS_sicherer_machen/…
opinion_no9 about 4 years

Thanks for the great work! One remark: This is NOT a generic guide for all systems using NetworkManager since it destroys some mechanismns. You are well advised to make resolv.conf a link. This is required by resolved to work properly. Editing resolv.conf is not an option any more.
opinion_no9 about 4 years

ad 3) It is possible to do all network settings changes on the CLI for networkmanager. Example: nmcli connection modify CONNECTIONNAME ipv4.addresses 192.168.22.xxx/24 or a similar one: nmcli connection modify CONNECTIONNAME ipv6.dns 2a02:2970:1002::18,2001:67c:28a4::,2001:1608:10:25::9249:d69‌b ; see nmcli and nmtui manuals. Example: developer.gnome.org/NetworkManager/stable/…
opinion_no9 over 3 years

@Lorenz-Keel You do not need to like the way the posting is written. Please do not discourage people, just do a better attempt! I look forward to it. I can still learn and I need to.
opinion_no9 over 3 years

from Ubuntu 18 on this is no longer valid, sorry. See my hints about it. I do not really like the way the whole thing becomes more and more complicated and non-standard. However, editing resolv.conf is NOT AN OPTION any longer in Ubuntu.
opinion_no9 about 3 years

sorry, no, only the legacy-IPv4 address is replaced by a new one with a new provider. The Internet address does not exist with the new provider.