What's inside a Docker image/container?

Question

Considering the fact that docker images/containers come in various flavours - Ubuntu, CentOS, CoreOS etc.... I'm curious what actually makes up an image/container, and what is shared with the host OS? Where is the dividing line?

For example, I can download the base Ubuntu image and launch it on a CentOS host. Then, when I poke around inside the Ubuntu container I can see that it looks and feels like an Ubuntu server (filesystem layout etc). But if I run a uname command I see the kernel and the likes of the CentOS host....

Obviously I understand that the underlying kernel is shared by all containers on the same host. But what else is shared with the host OS, and what is part of the image/container?

E.g. the kernel is part of the host, the filesystem layout is part of the the image/container.... Is there a spec that defines this?

Solution 1

Docker is a wrapper over LXC Linux Containers and the documentation for that will let you know in detail what is shared and what is not.

In general the host machine sees/contains everything inside the containers from file system to processes etc. You can issue a ps command on the host vm and see processes inside the container.

Remember docker containers are not VMs - hence everything is actually running natively on the host and is using the host kernel directly. Each container has its own user namespace (similar to root jails of old). There are tools/features which make sure containers only see its own processes, have its own file-system layered onto host file system, and a networking stack which pipes to the host networking stack.

Solution 2

It can be helpful to distinguish between images and containers (docs). An image is static and lives only on disk. A container is a running instance of an image and it includes its own process tree as well as RAM and other runtime resources.

An image is a logical grouping of layers plus metadata about what to do when creating a container and how to assemble the layers. Part of that metadata is that each layer knows its parent's ID.

So, what goes into a layer? The files (and directories) you've added to the parent. There are also special files ("whiteout") that indicate that something was deleted from the parent.

When you docker run an image, docker creates a container: it unpacks all the layers in the correct order, creating a new "root" file system separate from the host. docker also reads the image metadata and starts either the "entrypoint" or "command" specified when the image was created -- that starts a new process sub-tree. From inside the container, that first process seems like the root of the tree, but from the host you can see it is a subtree of processes.

The root file system is what makes one Linux distro different from another (there can be some kernel module differences as well, and bootloader/boot file system differences, but these are usually invisible to the running processes). The kernel is shared with the host and is, in fact, still managing its usual responsibilities inside the container. But the root file system is different, and so when you're inside the container, it looks and feels like whatever distro was in the Docker image.

The container not only has its own file system and process tree, but also has its own logical network interface and, optionally, its own allocation of RAM and CPU time. You're in control over the container though, as the operator, so you can decide to share the host's network interface with the container, give it unlimited access to RAM and CPU, and even mount devices, files and directories from the host into the container. The default is to keep things separate, but you have the power to break the isolation model as much as you need to.

Comments

[Nigel Poulton]

So the container specific stuff is stuff thats implemented via kernel namespaces? So like each container having its own pid namespace, its own mnt namespace, its own net namespace.....? Everything else is shared with the host?

[Usman Ismail]

Exactly, there are some ways to tweek the container where you can use host stack directly but other than that its all namespaces.

[icecrime]

Docker does not rely on LXC by default since March.

[Nigel Poulton]

So that was gonna be my next question. I know that the default execution driver for docker is now libcontainer (not LXC).... and I'm sure Usman knows that. But will the LXC documentation still provide a decent place to find out under-the-hood detail?

[Howard Lee]

In addition, libcontainer wraps around existing Linux container APIs, particularly cgroups and namespaces. cgroups provide resource-level isolation: cpu, memory, block i/o, network. namespaces provide kernel-level isolation: process, network, user id, file system. More here

[Perishable Dave]

Thanks, this is super helpful. I was really confused with what is an image and how does it relate to the host OS. The docs on the website don't seem to explain this very well, but your answer here does.

[Nakilon]

Guys, how much these comments are relevant today? Maybe something big has changed and it's time for another Answer?