Deploy Ubuntu 20.04 on bare metal or virtualbox VM by pxelinux, cloud-init doesn't pick up user-data file

Question

I've been deploying Ubuntu (since 12.04) along with other Linux with pxelinux many years. With the Debian-installer, it works just fine. These days I've been trying to deploy 20.04, which also works basically. Only one question, the "user-data" file never been picked up. No matter what I do, I always get all the questions asked from the installer. Here is my environment:

1. I created a user-data file under /var/www/html/ubuntu/cloud-init of the file server, and its content is as this:

## cloud-config

autoinstall:
  version: 1
  apt:
    preserve_sources_list: false
    primary:
    - arches: [default]
      uri: [...]/images/ubuntu

  identity: {realname: wrsadmin, username: wrsadmin}
  keyboard: {layout: us, toggle: null, variant: ''}
  locale: en_US
  network:
    ethernets:
      enp0s3:
        critical: true
        dhcp-identifier: mac
        dhcp4: true
        nameservers:
          addresses: [128.224.160.11, 128.224.160.12]
          search: [wrs.com., corp.ad.wrs.com.]
    version: 2
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: true

  late-commands:
    - rm -f /target/etc/resolv.conf
    - wget -O /target/etc/resolv.conf [...]/ubuntu/resolv.conf
    - chattr +i /target/etc/resolv.conf
    

2. "default" file of pxelinux. As you can see below, when I use the Debian-Installer netboot files, PXE works, and preseed works too. When I use cloud-init format netboot files, PXE works too, the 900MB server live image is loaded successfully. But, the "autoinstall ds=nocloud-net;s=[...]/ubuntu/cloud-init/" is like it doesn't exist.

Debian-installer + preseed:

LABEL Ubuntu 20.04 x64 legacy
  MENU LABEL Ubuntu 20.04 x64 legacy
  TEXT HELP
  Ubuntu 20.04 x64 legacy
  ENDTEXT
  KERNEL Linux/Ubuntu2004/linux
  APPEND vga=normal initrd=Linux/Ubuntu2004/initrd.gz locale=en_US.UTF-8 keyboard-configuration/layoutcode=us ipv6.disable=1 url=[...]/ubuntu/preseed/preseed2004.cfg

Cloud-init + user-data

LABEL Ubuntu 20.04 x64
  MENU LABEL Ubuntu 20.04 x64
  TEXT HELP
  Ubuntu 20.04 x64
  ENDTEXT
  KERNEL Linux/Ubuntu2004/vmlinuz
  APPEND initrd=Linux/Ubuntu2004/initrd ip=dhcp url=[...]/images/ubuntuExtra/ubuntu2004/ubuntu-20.04-live-server-amd64.iso autoinstall ds=nocloud-net;s=[...]/ubuntu/cloud-init/

Would you please help to diagnose which part I did wrong?

Solution 1

I was able to use these steps to do an autoinstall on a BIOS based VM. They are slightly modified from my UEFI steps. Hopefully they will provide an example to help you figure out your problem. You can tailor them to your environment

Build a tftp server

All the following steps are run as root. These were tested on an Ubuntu 18.04 server.

Install the tftp server, web server, and syslinux files

apt-get -y install tftpd-hpa apache2 pxelinux

Configure apache to serve files from the tftp directory

cat > /etc/apache2/conf-available/tftp.conf <<EOF
<Directory /var/lib/tftpboot>
        Options +FollowSymLinks +Indexes
        Require all granted
</Directory>
Alias /tftp /var/lib/tftpboot
EOF
a2enconf tftp
systemctl restart apache2

Copy the syslinux files to the tftp directory

cp /usr/lib/PXELINUX/gpxelinux.0 /var/lib/tftpboot/pxelinux.0.bios
cp /usr/lib/syslinux/modules/bios/*.c32 /var/lib/tftpboot

Download the live server iso

wget http://old-releases.ubuntu.com/releases/20.04/ubuntu-20.04-live-server-amd64.iso -O /var/lib/tftpboot/ubuntu-20.04-live-server-amd64.iso

Extract the kernel and initramfs from the live server iso

mount /var/lib/tftpboot/ubuntu-20.04-live-server-amd64.iso /mnt/
cp /mnt/casper/vmlinuz /var/lib/tftpboot/
cp /mnt/casper/initrd /var/lib/tftpboot/
umount  /mnt

Configure syslinux

MYIP=$(hostname --ip-address)
mkdir -p /var/lib/tftpboot/pxelinux.cfg
cat > /var/lib/tftpboot/pxelinux.cfg/default <<EOF
DEFAULT vesamenu.c32
TIMEOUT 600
ONTIMEOUT focal-live-install-autoinstall
PROMPT 0

NOESCAPE 1

LABEL focal-live-install
        MENU DEFAULT
        MENU label Install focal
        KERNEL vmlinuz
        INITRD initrd
        APPEND root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=http://${MYIP}/tftp/ubuntu-20.04-live-server-amd64.iso

LABEL focal-live-install-autoinstall
        MENU DEFAULT
        MENU label Install focal - autoinstall
        KERNEL vmlinuz
        INITRD initrd
        APPEND root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=http://${MYIP}/tftp/ubuntu-20.04-live-server-amd64.iso autoinstall ds=nocloud-net;s=http://${MYIP}/tftp/cloud-init-bios/ cloud-config-url=/dev/null

EOF

Configure cloud-init with the autoinstall configuration. I first ran the install manually to get the generated /var/log/installer/autoinstall-user-data file to use as the basis. I then made modifications based on my needs and errors encountered.

mkdir -p /var/lib/tftpboot/cloud-init-bios/
cat > /var/lib/tftpboot/cloud-init-bios/meta-data <<EOF
instance-id: focal-autoinstall
EOF

cat > /var/lib/tftpboot/cloud-init-bios/user-data <<'EOF'
#cloud-config
autoinstall:
  version: 1
  # use interactive-sections to avoid an automatic reboot
  #interactive-sections:
  #  - locale
  apt:
    # even set to no/false, geoip lookup still happens
    #geoip: no
    preserve_sources_list: false
    primary:
    - arches: [amd64, i386]
      uri: http://us.archive.ubuntu.com/ubuntu
    - arches: [default]
      uri: http://ports.ubuntu.com/ubuntu-ports
  # r00tme
  identity: {hostname: focal-autoinstall, password: $6$.c38i4RIqZeF4RtR$hRu2RFep/.6DziHLnRqGOEImb15JT2i.K/F9ojBkK/79zqY30Ll2/xx6QClQfdelLe.ZjpeVYfE8xBBcyLspa/,
    username: ubuntu}
  keyboard: {layout: us, variant: ''}
  locale: en_US.UTF-8
  # interface name will probably be different
  network:
    network:
      version: 2
      ethernets:
        ens192:
          critical: true
          dhcp-identifier: mac
          dhcp4: true
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: true
  # this creates an bios_grub partition, /boot partition, and root(/) lvm volume
  storage:
    config:
    - {ptable: gpt, path: /dev/sda, wipe: superblock, preserve: false, name: '', grub_device: true,
      type: disk, id: disk-sda}
    - {device: disk-sda, size: 1048576, flag: bios_grub, number: 1, preserve: false,
      type: partition, id: partition-0}
    - {device: disk-sda, size: 1073741824, wipe: superblock, flag: '', number: 2,
      preserve: false, type: partition, id: partition-1}
    - {fstype: ext4, volume: partition-1, preserve: false, type: format, id: format-0}
    - {device: disk-sda, size: -1, wipe: superblock, flag: '', number: 3,
      preserve: false, type: partition, id: partition-2}
    - name: ubuntu-vg
      devices: [partition-2]
      preserve: false
      type: lvm_volgroup
      id: lvm_volgroup-0
    - {name: ubuntu-lv, volgroup: lvm_volgroup-0, size: 100%, preserve: false,
      type: lvm_partition, id: lvm_partition-0}
    - {fstype: ext4, volume: lvm_partition-0, preserve: false, type: format, id: format-1}
    - {device: format-1, path: /, type: mount, id: mount-1}
    - {device: format-0, path: /boot, type: mount, id: mount-0}
write_files:
  # override the kernel package
  - path: /run/kernel-meta-package
    content: |
      linux-virtual
    owner: root:root
    permissions: "0644"
  # attempt to also use an answers file by providing a file at the default path.  It did not seem to have any effect
  #- path: /subiquity_config/answers.yaml
  #  content: |
  #    InstallProgress:
  #      reboot: no
  #  owner: root:root
  #  permissions: "0644"
EOF

Configure DHCP

Set the DHCP Options 66,67 according to the documentation for your DHCP server.

Boot your server

At this point, you should be able to boot your UEFI based server and perform a completely automatic install.

Solution 2

When I did this with a UEFI based server (which uses grub instead of syslinux) I had to escape the semicolon in the command line.

Try changing

ds=nocloud-net;s=http://blah/ubuntu/cloud-init/

to

ds=nocloud-net\;s=http://blah/ubuntu/cloud-init/

I found the easiest way to check in the installer environment is to use alt-f2 to get a console and use the command

dmesg | grep 'Command line'

That will show if the full ds argument is being passed or if it only passes up to the ;

Comments

[Admin]

Make sure you use an url that can be looked up properly and not use a short-name (i.e. non FQN). Try use the IP-address as source if that changes anything. Also if you use https with selfsigned certs will be problematic. Not sure how to solve that with autoinstall.

[Wei Du]

Thank you sir for you reply. "escape the semicolon" I've tried, which doesn't work. I tried both today and use "Alt+F2" to see the full arguments that had been passed, no matter escape or not, all arguments were passed.

[Andrew Lowther]

You're right, escaping the semicolon is not needed in the pxelinux.cfg. You can check your web server logs to see if the cloud-init files are being downloaded. The installer logs in /var/log might also contain any errors if they are being downloaded but the syntax is bad

[Wei Du]

Hello Sir, I check out the httpd logs on my http server. Here is what I got when the installer is running: comment is too short. Please see the answer from me below

[Andrew Lowther]

I posted another answer with complete steps that I used successfully. I hope it helps you figure out what you're missing

[Wei Du]

Hello Sir, I understand each part of your configuration. Thank you for your kindly help. But I failed to figure out why the live image doesn't load the autoinstall file I specified. I'll keep digging.