CentOS 7 ISO Kickstart Installation - "Pane is dead"

Question

EDIT: I retried this generically, and see the same problem. To reproduce, follow my steps below. I am starting to think that it might be a problem with the hardware, but I am not certain.

When installing a CentOS 7 ISO using kickstart, I get an error saying "Pane is dead" with no additional logs. Basically, I take the CentOS 7 ISO and added a kickstart file. When installing with this on VirtualBox, everything works fine. When I attempt to install it on real hardware with multiple interfaces, I get the "Pane is dead" error after all the packages are installed.

I am at a loss as to what could be the problem. Some things I've tried:

• There are no logs generated that indicate the problem. I've even added the logging --level=debug line to the kickstart, but there was nothing of value to see.
• There is no post-installation step (I removed that from the ks since that is last log before the problem is seen), so that cannot be the issue.
• I did see somewhere on the web that someone said there is a difference between CentOS 6 and 7 ISOs, but there was no additional information to indicate quantitatively what those differences might be.

Here are the complete instructions on how to reproduce. If you can see what I am doing wrong, I would greatly appreciate it.

Thanks!

Complete Instructions for How to Reproduce

My production machine is an up-to-date CentOS 7 server. If you are missing any applications, I am sure you can install them easily with yum.

The basic steps are:

• Create a working directory
• Create a kickstart file
• Update isolinux.cfg
• Download packages
• Create repository
• Create the ISO

Create Working Directory

I copied the CentOS 7 ISO to my temp directory and mounted it to get all the files:

>cd /tmp
>wget http://centos.mirror.iweb.ca/7/isos/x86_64/CentOS-7-x86_64-DVD-1503-01.iso
... hours later ...
>mkdir CentOS7-CD
>sudo mount -l /tmp/CentOS-7-x86_64-DVD-1503-01.iso /tmp/CentOS7-CD
mount: block device /tmp/CentOS-7-x86_64-DVD-1503-01.iso is write-protected, mounting read-only

I copied all the files out into a working directory.

>cd /tmp/
>cp -r CentOS7-CD/ CentOS7

Kickstart File

Add my kickstart file:

>cat << @@EOF@@ > /tmp/CentOS7/ks.cfg
#version=RHEL7
# System authorization information
auth --enableshadow --enablemd5
# Install OS instead of upgrade
install
# Reboot after installation
reboot --eject
# License agreement
eula --agreed
# Use text mode install
text
# Firewall configuration
firewall --disabled
firstboot --disable
ignoredisk --only-use=sda
# Keyboard layouts
# old format: keyboard us
# new format:
keyboard --vckeymap=us --xlayouts='us'
# System language
lang en_CA.UTF-8
# Installation logging level
logging --level=debug
# Network information
%include /tmp/network.ks

# Use CDROM installation media
cdrom
repo --name="CentOS" --baseurl=cdrom:sr0
# Use Network installation
#url --url=http://centos.mirror.iweb.ca/7/os/x86_64/

# Root password
rootpw --plaintext password
# SELinux configuration
selinux --disabled
# Do not configure the X Window System
skipx
# System timezone
timezone Etc/UTC --isUtc
# System bootloader configuration
bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=sda
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel 
# Disk partitioning information
part / --fstype="ext4" --size=10000
part /data --fstype="ext4" --size=18461
part /info --fstype="ext4" --size=2000
part /boot --fstype="ext3" --size=256
part swap --fstype="swap" --size=2048

%packages --nobase --excludedocs
@core
-aic94xx-firmware
-alsa-*
-iwl*-firmware
-ivtv-firmware
-libertas-*-firmware
-btrfs-progs
-cronie-anacron
-cronie
-crontabs
-ModemManager-glib
-NetworkManager*
-postfix
-wpa_supplicant
%end

%addon com_redhat_kdump --enable --reserve-mb='auto'

%end

%pre
interface=$(ip addr | grep -i broadcast | awk '{ print $2 }' | head -1 | sed 's/://g')
echo "network --bootproto=dhcp --device=${interface} --onboot=on" > /tmp/network.ks
%end
@@EOF@@

Note: If using these instructions and cutting and pasting, you may get "> " prefixing every line. Use the following to remove them.

>sed -i 's|^> ||' /tmp/CentOS7/ks.cfg

I've tried to limit the number of packages installed, hence all the exclusions in the %packages section.

ISOLINUX.CFG

Next, edit my isolinux.cfg file and add inst.ks=cdrom:/dev/cdrom:/ks.cfg to each append line and make the non-test version the default.

>cd /tmp/CentOS7/isolinux/
>sed -i 's|\(LABEL=\S*\)|\1 inst.ks=cdrom:/dev/cdrom:/ks.cfg|' isolinux.cfg
# Delete the old default (check)
>sed -i '/menu default/d' isolinux.cfg
# Add the new default (linux)
>sed -i '0,/  kernel vmlinuz/s//  menu default\n&/' isolinux.cfg

Add Packages

I empirically determined that there are 240 packages to add for my minimal installation. I did this by performing a network installation on VirtualBox with the following changes to the kickstart file:

# Use CDROM installation media
#cdrom                                    # <-- comment this out for ntwk install
#repo --name="CentOS" --baseurl=cdrom:sr0 # <-- comment this out for ntwk install
# Use Network installation
url --url=http://centos.mirror.iweb.ca/7/os/x86_64/

So I commented out the CDROM installation, and instead used the network repository. Installation took 10 minutes. Once complete, I logged in to get the list of installed packages:

VB> rpm -qa > pkg.list

I copied this file back to my production machine and obtained the locations for downloading these files:

> cd /tmp/CentOS7
> rm -rf Packages
> cd Packages
# Copy the pkg.list from the VB server to this location
> cat pkg.list | xargs yumdownloader --urls | grep "^http" > pkg.urls
> wget -nv -nc -i pkg.urls

This takes a few minutes to complete as it downloads all the packages.

Create Repository

Creating the repository is a simple task.

>cd /tmp/CentOS7
>mv repodata/*comps.xml.gz comps.xml.gz
>gzip -d comps.xml
>rm -rf repodata
>mkdir repodata
>cp comps.xml repodata/
>createrepo -g repodata/comps.xml .

Create the ISO

>cd /tmp
>mkisofs -o custom.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -V 'CentOS 7 x86_64' -boot-load-size 4 -boot-info-table -R -J -v -T CentOS7/

Solution 1

I've finally solved my own issue! I'll put this as an answer so that if anyone else sees it they will know the cause. However, I assume only Canadians will be affected...

It turns out that the problem was with the lang en_CA.UTF-8 line in the kickstart file. I changed this to lang en_US.UTF-8 and the installation completes normally. I may ask another question on how to use en_CA but at least I solved this one.

Thanks!

For anyone who is interested, how I solved it was by doing this:

• I created a DVD with the CentOS-7-x86_64-DVD-1503-01.iso on it
• Put this on my server and booted
• When asked, changed the grub configuration line and added text to it
• During anaconda installation, I configured minimum installation over the network.
• Installation completed and an anaconda-ks.cfg file was created in /root/
• I copied this file off and used it exactly as my own template for the my ks.cfg file.
• I also re-created the Packages and repodata directories based on the set of installed packages in this installation.
• Created a new iso with the new kickstart file and installation worked!
• Line by line, I changed the differences in the new ks.cfg file to match the old (problematic one), recreated the ISO and attempted installation.
• Painfully found that the only line that caused the error was the lang en_CA.UTF-8 line.

Solution 2

The issue seems to be quite clear as I might say. Notice that the Pane is dead comes when the post-installation script runs.

I'd rather cut that section from ks file or even better log it to understand what exactly went wrong.

In order to log you can use --log /path/to/logfile directive in your kickstart. Once you got pane message just switch to one of TTYs and check the logs.

So the offending section is %post.

I hope this helps.

Comments

[thrig]

logging --level=debug may also help. I've seen the intuitive "pane is dead" message, though that was due to an empty %pre block. However, all of your blocks look like they have statements in them so uh dunno.

[Trenin]

@CameronKerr I agree the sed line looks like it is replacing a ':' with a blank space. Not sure why. I got that from somewhere else. I will use sed 's/://g' next time. No logs of any interest in the places you mention. Interfaces line in ks looks fine. network --bootproto=dhcp --device=enp5s0f0 --onboot=on.

[Trenin]

@thrig I added the logging level, but not seeing much. I will compare with my virtual box deployment (which is successful).

[Trenin]

@thrig No useful information in the logs. See update to post on how I created the ISO.

[ostendali]

if the issue doesn't occur with virtualbox as you stated it migh be hardware issue. Try using VGA mode and if that helps. If it doesn't that you need to find out what exactly is offending part and blacklist the drive prior to boot.

[Trenin]

@ostendali It is an old server, Servaris Proserv 2650.

[ostendali]

sorry but I am not familiar with such hardware, I can only say that surely the problem is with hardware, therefore, the only way out is to enabled logging and check what is it, once you know what is it you will then blacklist it that specific driver during the boot and that will hopefully solve your issue.

[ostendali]

enable logging following CentOS wiki and see if that will give you some clue: wiki.centos.org/TipsAndTricks/KickStart

[ostendali]

where did you get that VGA mode is win stuff? you can set the VGA mode during boot stage (grub cmd) and add: vga=vesa (or give lower resolution, you need to try a few parameters)....

[Trenin]

@ostendali As I said in the post, I've already tried with logging enabled. There are no errors in the various log files to indicate anything went wrong.

[Trenin]

@ostendali Sorry, did a quick search for VGA mode and it seemed to be tied to Windows. Will try with the vga=vesa line on the grub installation line.

[ostendali]

you dodn't necessarily look for errors in the logs but anything, see where it hangs/freeze, that will give a clue at least:-)

[ostendali]

re to vga mode and this fixation with windoz, I know nothing about windoz so how would I get that info. Just type grub vga mode and let see how it is related to OSs other than linux: wiki.debian.org/GrubTransition

[Trenin]

@thrig Finally found the problem was with the en_CA line. See my answer.

[Trenin]

Thanks for the tip. However, I tried logging the post section (%post --log=/tmp/post.log in the kickstart) and got an empty file in /tmp/post.log. I suspect that the problem is not the post section because I see in the anaconda.log file that it successfully ran all the post installation scripts.

[Trenin]

Also, that wouldn't explain why this works with VirtualBox, but not with real hardware. The post installation script is pretty generic - simply disables some selinux configuration. I'll retry without the post installation script.

[Trenin]

I've confirmed that this is not the problem. I've removed %post section entirely and the problem persists.

[Trenin]

Correct. See the new screen shot. Although it looks the same, it is from the new installation attempt.

[Trenin]

Finally found the problem! Check out my own answer.