Building and running a ARM system image with QEMU?

Question

I am trying to set-up a full system image for ARM (armhf, armel or even aarch64) based on Debian that can be run with QEMU. Unfortunately, all the examples that I found on the Web start the image by passing the kernel (and possibly the initrd if it requires some modules).

But, I really would like to start the system as it is done with qemu-system-amd64, just by giving the disk image and the CD-ROM image (to start the installation).

I can understand that there is no default for -machine and -cpu, but I cannot find any clue on the web on how to do something like:

#> wget \
   http://cdimage.debian.org/.../armhf/iso-cd/debian-testing-armhf-netinst.iso
...
#> qemu-img create -f qcow2 debian-armhf.qcow 20G
...
#> qemu-system-arm -machine vexpress-a9 -cpu cortex-a9     \
                   -hda debian-armhf.qcow                  \
                   -cdrom debian-testing-amd64-netinst.iso \
                   -boot d
...
#> qemu-system-arm -machine vexpress-a9 -cpu cortex-a9     \
                   -hda debian-armhf.qcow                  \
                   -net nic -net user,hostfwd=tcp::2222-:22
...

So, I really wonder if such thing is just possible... And if yes, how? And if no, why?

Solution

This is a great question and one I chose to dig into myself a while ago, only for me it was using MIPS/MIP64.

There are a few issues which make this challenging.

When booting a Linux host the Kernel needs to understand the hardware which it is working with. For some architectures (like amd64) the discovery of firmware is handled by the BIOS via "Advanced Configuration and Power Interface" (ACPI). In other cases this is handled by an extensible firmware interface (EFI) or even ATAGS as a very old solution. In most modern embedded systems the use of ATAGS has been replaced with a mechanism called a "device tree blob" (DTB). A series of helpful users did a much better job explaining the difference than I can.

The purpose of all of these is to help hint the kernel as to the configuration of the hardware and generally works in tandem with the bootloader. In your case choosing between ARM little endian (armel), ARM hard floating point (armhf), and ARM 64 bit (aarch64) is a critical first decision, as it then has ramifications for the choice of compiled packages, Kernel, etc as everything will need to conform to the emulated "instruction set architecture" (ISA).

I understand that you want the simplicity of a blank disk and a CD image, but that's ultimately why there aren't "installers" of that form for systems like the Raspberry Pi*. When you download a Raspbian Image it includes a structured set of partitions which include a bootloader (das uBoot), a device tree blob for the corresponding hardware, and a base set of packages available with the package repositories configured.

This touches on another bit of interesting trivia.... Have you ever looked into what that ISO image is doing when you "install" Linux?

Let me use Anaconda based installers (Fedora, RHEL, CentOS, etc) as an example.

When you kick off the installation, it boots a minimal live system which automatically begins running the python utility Anaconda. Anaconda then prompts the user to answer a series of questions, populates a configuration file in its domain specific language, and then creates a chroot and executes a set of package installations inside of that chroot.

This... in a small way, it's just like using that Raspbian image.

So, what does this mean?

Well, first off there are answers I would recommend, depending on how you want to proceed?

If emulating an RPi is sufficient, let me suggest: Using QEMU to emulate a Raspberry Pi

If you want to emulate something more like a "cloud" based system, George Hawkins maintains a Gist with some great instructions.

If you really want to go from the ground up, the Debian User Aurel32 has maintained for years a set of base images ready to boot as well as a great writeup on their blog of a similar process.

Comments

[dirkt]

I would assume you'd need some sort of BIOS to be able to -boot (that's why the examples you found pass the kernel directly). After all, some program has to decode the ISO image. AFAIK, qemu only has a built-in PC BIOS. So where should the ARM BIOS come from? Which one would you use? Would it be OK to pass that BIOS directly, as you do with the kernel, and wouldn't that meet your requirements?

[dirkt]

Here is an example where they boot via uboot, so I assume you could compile uboot for your configuration, and pass it via -boot ..., if this meets your requirements.

[Philippos]

Are you looking for ELBE?