Cannot contact any KDC for requested realm in log.winbindd-dc-connect every 10 seconds

Question

I have a Ubuntu box, used for cifs file shares, that is using Samba. It is joined to an Active Directory Domain. We are using trusted domains. We have a pretty large AD infrastructure with many child domains. This box is joined to one of the child domains.

My issue is that I am constantly getting this error(see below) about every 10 seconds in the log.winbindd-dc-connect. The interesting thing is that the cifs actually work but it seems like the performance is being impacted. I do see that the CPU is constantly high on this machine. I think the log errors and the performance hit are related. Just wondering if anybody has ever seen this.

I am more of the Windows individual than Linux so my understanding of Samba is pretty limited. Trying to learn as I go. Thank you in advance.

[2017/03/20 17:26:22.225186, 0, pid=19851] ../source3/libsmb/cliconnect.c:1921(cli_session_setup_spnego_send) Kinit for xxxxxx to access cifs/[email protected] failed: Cannot contact any KDC for requested realm

Solution

The error is telling you that your system is trying to authenticate against a domain controller, but is unable to do so because none are available. Since it's every 10 seconds, many of these could be retries as the system continues to try to hit the SAMBA share.

I noticed that your auth request has the last LOCAL in uppercase - try that in lowercase (since the hostname portion of the address is also in lowercase). Would think Samba would be case-insensitive, but I've been tripped up by case before.

Continue with verifying connection / bandwidth between this system and the KDC. Take into consideration ping times, dropped packet rate, and so forth.

Then verify that TCP ports 139 and 445 and UDP ports 137 and 138 are OPEN, in both egress and ingress directions, on any firewalls between your system and the domain controller. Check the firewall logs to see if packets in these ranges are getting dropped due to proc/network load. If the firewall is getting slammed, consider setting up a read-only DC in the same subnet as the Ubuntu system, so that the auth traffic doesn't have to bridge a firewall. The RDC will only need to sync a few times per day, rather than constantly pass auth requests. So it should reduce the firewall load.

Comments

[Admin]

So I think I found the issue. Like you said it has to do with the case of the file. I looked in the krb5.conf file and my in the realm is in lowercase. I have changed to uppercase and this fixed the error in the logs

[Travis-Zadara]

The ubuntu instance is in AWS so I will check the the firewall rules there. The strange thing is that I can connect to the DC on all the ports you mentioned from ubuntu and the cifs share actually work. So it is reaching the DC to authenticate users. That is why I am so baffled.

[Travis-Zadara]

All the firewall ports are open. A funny thing happens. If I shutdown the domain controller in question the error jumps to the next DC in the list and starts giving the same error. It is quite strange. Again no impact to actual connectivity just overall sluggishness. Thank for all the help.

[George Erhard]

So it's not completely blocked, but you've a lot of latency on the auth requests. Can you get a feel for the timing using Wireshark, take a look at the packets?

[Travis-Zadara]

So I think I found the issue. Like you said it has to do with the case of the file. I looked in the krb5.conf file and my in the realm is in lowercase. I have changed to uppercase and this fixed the error in the logs. Not sure it is actually helping performance but at least its not doing in the logs. I appreciate the help. Case Sensitivity is killing me coming from a windows world. Thanks again!!

[George Erhard]

Cool beans. I didn't think it would matter either, but as I said, that has bitten me in the behind more than once, so worth checking. Glad you were able to at least alleviate the log spam.