TFTP timeout problem during PXE boot

Question

While setting up a PXE boot server on a CentOS 7 machine, I encountered a strange problem with TFTP. I cannot retrieve any files from the TFTP server without encountering a timeout problem. The boot process comes so far that I correctly get an IP address and a file name from the DHCP server. However, when the boot files are to be retrieved from the TFTP server, a "TFTP open timeout" message comes. If I manually make a TFTP connection to the PXE server from a local computer, I immediately get access to the server. But if I try with a "get pxelinux.0" command, I get another timeout message. My firewall is set up correctly and it also makes no difference if I completely turn off the firewall. SeLinux is also disabled. If I make a tcpdump on port 69 I get the following message:

12:34:33.477401 IP 172.16.1.202.ah-esp-encap > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0
12:34:35.481131 IP 172.16.1.202.acp-port > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0
12:34:39.490793 IP 172.16.1.202.msync > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0
12:34:45.477712 IP 172.16.1.202.gxs-data-port > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0
12:34:53.441801 IP 172.16.1.202.vrtl-vmf-sa > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0
12:35:03.384065 IP 172.16.1.202.newlixengine > tools.dmz.tuxme.dk.tftp:  32 RRQ "pxelinux.0" octet blksize 1456
12:35:39.414843 IP 172.16.1.202.newlixconfig > tools.dmz.tuxme.dk.tftp:  32 RRQ "pxelinux.0" octet blksize 1456
12:36:51.422568 IP 172.16.1.202.tsrmagt > tools.dmz.tuxme.dk.tftp:  32 RRQ "pxelinux.0" octet blksize 1456
12:38:39.406732 IP 172.16.1.202.tpcsrvr > tools.dmz.tuxme.dk.tftp:  32 RRQ "pxelinux.0" octet blksize 1456

Unfortunately, the system log shows nothing useful:

Jan 15 13:13:19 tools xinetd[6993]: EXIT: tftp status=67 pid=7954 duration=0(sec)
Jan 15 13:13:21 tools xinetd[6993]: START: tftp pid=7955 from=172.16.1.202
Jan 15 13:13:21 tools in.tftpd[7955]: no user tftp: Success
Jan 15 13:13:21 tools xinetd[6993]: EXIT: tftp status=67 pid=7955 duration=0(sec)
Jan 15 13:13:25 tools xinetd[6993]: START: tftp pid=7956 from=172.16.1.202
Jan 15 13:13:25 tools in.tftpd[7956]: no user tftp: Success
Jan 15 13:13:25 tools xinetd[6993]: EXIT: tftp status=67 pid=7956 duration=0(sec)
Jan 15 13:13:31 tools xinetd[6993]: START: tftp pid=7957 from=172.16.1.202
Jan 15 13:13:31 tools in.tftpd[7957]: no user tftp: Success

The rights to the /var/lib/tftpboot directory are 0755.

Here are my setup file:

/etc/xinetd.d/tftp:

service tftp
socket_type             = dgram
protocol                = udp
wait                    = yes
user                    = root
server                  = /usr/sbin/in.tftpd
server_args             = -c -v -u tftp -p -U 117 -s /var/lib/tftpboot
disable                 = no
per_source              = 11
cps                     = 100 2
flags                   = IPv4

Solution

It looks like the tcpdump output contains only requests and not any responses at all. If this is what is actually happening, then a timeout error is to be expected.

In the server_args line of your TFTP configuration for xinetd, you have -u tftp. That tells in.tftpd to run as user tftp. In light of that, this message logged by in.tftpd might be important:

Jan 15 13:13:21 tools in.tftpd[7955]: no user tftp: Success

It says "no user tftp". Does the tftp user account actually exist on your system?

The Success at the end of the log message requires a bit of C programming knowledge to understand. It is likely to come from a minimalist error-handling function that probably just calls perror() and then does any necessary clean-up before exiting.

The perror() function takes a message from its caller and then appends to it a standard error message corresponding to the current value of the errno variable. It is designed to be used in situations where a previous system call has failed; the custom message should describe what the program was doing when the error was encountered, and the standard message should then clarify the type of problem encountered.

But if the programmer has used their error-handling function to report an error that has been caught some other way, the standard error message part will read Success.

My guess is that the in.tftpd process gets started by xinetd, prepares to switch to user tftp, and finds out that such user does not exist. So the in.tftpd process outputs that log message and dies without sending anything to the client.

The terse message with a misleading "Success" at the end is an example of the old concept of "if your only tool is a hammer, you tend to treat everything as nails." In this case, the programmer has probably used their only error handling function in a situation to which its output format does not quite fit.

Also, these requests look a little odd:

12:34:33.477401 IP 172.16.1.202.ah-esp-encap > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0
12:34:35.481131 IP 172.16.1.202.acp-port > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0
12:34:39.490793 IP 172.16.1.202.msync > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0
12:34:45.477712 IP 172.16.1.202.gxs-data-port > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0
12:34:53.441801 IP 172.16.1.202.vrtl-vmf-sa > tools.dmz.tuxme.dk.tftp:  27 RRQ "pxelinux.0" octet tsize 0

The tsize 0 indicates the client is expecting a TFTP transfer with a file size of 0 bytes total.

Are you aware that the UEFI PXE specification, as it existed in UEFI version 2.3 or so, requires the DHCP server to tell the PXE client the size of the file it's supposed to load? If you're using ISC DHCP server, the required option could be specified as

option boot-size <size value>; 

The <size value> should be the size of the boot file in bytes divided by 512, then rounded up.

Comments

[Stephen Harris]

in.tftpd[7957]: no user tftp and the server args -u tftp makes me think you need to have a user tftp that the daemon tries to run under, and this is failing

[Greenonline]

English only please...