Wrong nameserver set by resolvconf and NetworkManager

Question

My DNS server is 192.168.1.152.

This DNS is provided to clients by DHCP. The windows clients on my LAN resolve names properly using that DNS, but my Ubuntu VM doesn't.

The VM is set up with bridge networking and is being properly provided the DNS server, but my local hostnames aren't being resolved by nslookup or browsers.

Here is an nslookup of one of my local domains:

# nslookup unraid.local
Server:     127.0.0.53
Address:    127.0.0.53#53

** server can't find unraid.local: SERVFAIL

Here is what it should resolve by using my DNS server:

# nslookup unraid.local 192.168.1.152
Server:     192.168.1.152
Address:    192.168.1.152#53

Name:   unraid.local
Address: 192.168.1.152

/etc/resolv.conf has a wrong nameserver:

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.

nameserver 127.0.0.53

I ran that command. Under DNS Servers, confusingly, it specifies the correct server (and my default gateway).

root@ubuntu:~# systemd-resolve --status
Global
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 2 (ens33)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 192.168.1.152
                      192.168.1.1

I don't want to "hard code" the DNS server's IP in a config file because I won't be able to resolve when I change networks.

How can I get resolvconf and NetworkManager to automatically set the DHCP server's IP in /etc/resolv.conf?

Solution 1

Known systemd bug.

Temporary workaround with no need to reconfigure if the DNS IP's changes:

sudo rm -f /etc/resolv.conf
sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf
reboot

Solution 2

Try editing /etc/systemd/resolved.conf, adding your desired DNS server:

change this:

[Resolve]
#DNS=

to this (but use the one you want - this is an example):

[Resolve]
DNS=192.168.1.152

after that, restart the service:

service systemd-resolved restart

And when you check the status you should see

$ systemd-resolve --status
Global
         DNS Servers: 192.168.1.152

      DNSSEC NTA: 10.in-addr.arpa
                  16.172.in-addr.arpa
                  168.192.in-addr.arpa
                  17.172.in-addr.arpa
                  18.172.in-addr.arpa
                  19.172.in-addr.arpa

Solution 3

I finally got a solution for this problem for ubuntu 17.10. By default this version of Ubuntu uses systemd-resolved, which I hope is going to be stable for the next versions.

In order to use custom dns instead of the local systemd-resolved cache, do the following:

1. add new nameservers. Edit the file in /etc/systemd/resolved.conf as sudoer. Here I've commented out the DNS entry and placed my dns [Resolve] DNS=10.96.0.10 8.8.8.8 8.8.4.4

2. cancel the actual symlink to /etc/resolv.conf

3. create a new symlink sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf
4. restart the service sudo service systemd-resolved restart
5. restart the network manager sudo systemctl restart networking

And now if you dig to a name provided by your add dns, you should see the record resolved dig nexus.default.svc.cluster.mydomain

Last step is to update the order of resolution in /etc/nsswitch.conf, by placing the dns before the mdns4_minimal

hosts           files dns mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] myhostname

Solution 4

Your /etc/resolv.conf is not the problem. systemd-resolved just defaults to unconfigured, so it fails all lookups. Feel free to rant about Unconfigured vs A Reasonable Default.

Manually add nameservers to systemd-resolved. (editing per Olorin's comment below to add mkdir, correct path /etc not /lib, in order to survive system updates)

sudo mkdir -p /etc/systemd/resolved.conf.d
sudo nano /etc/systemd/resolved.conf.d/00-my-dns-server-is.conf

Add:

[Resolve]
Cache=yes
DNS=192.168.1.152

Then...

sudo systemctl daemon-reload

systemd-resolved is smart, but, unconfigured as it is, by package maintainers, it just LOOKS stupid because package maintainers do not believe in A Reasonable Default. We can put 13 internet root servers in there aka "djb way", or 10 opennic servers: https://pastebin.com/JBfYVVtG or the three fastest opennic servers, as measured by namebench. Plus ISP nameservers, sure. Plus Google, sure. systemd-resolved is not the problem. I am the problem.

Solution 5

On my system I found a bad symlink: /etc/resolv.conf was a symlink which points to /run/systemd/resolve/stub-resolv.conf

This file contains only one line:

nameserver 127.0.0.53#53

As a result, the local network's DNS lookup was often missing.

So, instead I changed /etc/reolv.conf to point to /run/systemd/resolve/resolv.conf

and now works correctly.

View more solutions

Comments

[FireSpore]

Here is the systemd-resolve --status after changing /etc/systemd/resolved.conf pastebin.com/AeUFQkyB Browsers still fail to resolve names also.

[teknopaul]

systemd is so broken. DHCP tells the host what DNS should be, (and a bunch of other network settings) you should not have to change any files on the host for this to work.

[FireSpore]

This solution worked somewhat, host and nslookup commands resolve names properly, but wget and browsers are not

[Victor]

Does not look optimal, you will have to do this every time you change DNS server/network

[Victor]

You would need to reconfigure every time the DNS IP's changes (for instance on different networks).

[Victor]

You would need to reconfigure every time the DNS IP's changes (for instance on different networks).

[Victor]

You would need to reconfigure every time the DNS IP's changes (for instance on different networks).

[Fabio Fumarola]

Victor do you have a best solution? this is the same if you use static net config

[Victor]

This answer works for both static and dynamic configurations: askubuntu.com/a/974482/343617

[Fabio Fumarola]

thank you, the solution I have pointed out was static. I agree

[Rodrigo Ferrari]

it's not elegant, but it work's gcloud sometimes sabotage my bedtime.

[Ollie Harridge]

Editing /etc/resolv.conf doesn't work as the file will be overwritten during the systemd-resolved restart. Just edit /etc/systemd/resolved.conf. See my answer here: askubuntu.com/questions/977243/ubuntu-17-10-disable-netplan/‌​…

[Olorin]

Though you probably don't want to edit a file /usr/lib - those will likely be overwritten on package upgrade. I think a corresponding file somewhere in /etc/systemd is the way to go (it already should have an /etc/systemd/resolved.conf ready to be edited by admins).

[BobDodds]

man resolved.conf.d, with the d, yes /etc/systemd/resolved.d is the place. I'm noticing that we often have to mkdir /etc/[path].d. bob@laptop l /etc/systemd/resolved.conf.d ls: cannot access '/etc/systemd/resolved.conf.d': No such file or directory

[Mr Heelis]

this is correct ...the current bug is that the link to /etc/resolv.conf comes from /run/systemd/resolve/stub-resolv.conf which is bollox it should be /run/systemd/resolve/resolv.conf note that /etc/resolv.conf doesn't really exist in either case

[JamesCW]

This just worked for me on a new 18.04 install

[Hosam.Yousof]

@FireSpore: does ping work then? host and nslookup resolve host names differently from ping or wget. You may need to look into /etc/nsswitch.conf for a solutioin.

[Ismail Cherri]

This worked for me on Linux mint 19.1 fresh install as well

[Ben Mares]

Instead of reboot it worked for me to do service systemd-resolved restart && sudo systemctl restart networking

[Sruli]

This looked very promising but my system is still using /etc/resolv.conf (/run/systemd/resolve/stub-resolv.conf). I tried /etc/systemd/resolved.conf.d and /etc/systemd/resolved.d my /etc/systemd/resolved.conf is also configured with the correct DNS server but i nothing is updating /run/systemd/resolve/stub-resolv.conf and it seems this is the only file being used. (i also set in netplan yaml file dhcp4-overrides: use-dns: no for now I will need to symlink /etc/resolv.conf to /run/systemd/resolve/resolv.conf. Not sure if i missed anything but would really like a proper solution.

[Afshin]

It worked for me even without reboot

[Salem F]

Good news its 2021 and systemd-resolved still not stable

[levitopher]

Not a fix for me - now I just fail to name resolve quicker!

[levitopher]

This is what worked for me - just switching the link did not. But this should be flexible right, no matter what local network I'm on, won't 8.8.8.8 always be found?

[Blair Scott]

Thank you so much!

[FredG]

2022. Thank you.