What is a container manifest?

Question

The only doc on this topic seems to assume I already know what a manifest is, the problem it solves, and how it fits into the docker ecosystem. After reading the doc I'm still not sure how manifests actually work.

My private GCR contains manifest files- don't really understand their purpose. Does docker hub also use manifest files? I can see they contain the layers and hashes of each layer, but I'm still unclear on how docker generates/uses them.

What is the purpose of a container manifest?

Solution 1

The manifest types are effectively the JSON-represented description of a named/tagged image. This description (manifest) is meant for consumption by a container runtime, like the Docker engine.

Any registry or runtime that claims to have Docker distribution v2 API/v2.2 image specification support will be interacting with the various manifest types to find out:

1. what actual filesystem content (layers) will be needed to build the root filesystem for the container, and..
2. any specific image configuration that is necessary to know how to run a container using this image. For example, information like what command to run when starting the container (as probably represented in the Dockerfile that was used to build the image).

As a prior answer mentioned, a client (such as the docker pull implementation) talking to a registry will interact over the Docker v2 API to first fetch the manifest for a specific image/tag and then determine what to download in addition to be able to run a container based on this image. The v2 manifest format does not have signatures encoded into it, but external verification using tools like a notary server can be used to validate external signatures on the same "blob"/hash of content for full cryptographic trust. Docker calls this "Docker Content Trust" but does not require it when talking to a registry, nor is it part of the API flow when talking to an image registry.

One additional detail about manifests in the v2.2 spec: there is not only a standard manifest type, but also a manifest list type which allows registries to represent support for multiple platforms (CPU or operating system variations) under a single "image:tag" reference. The manifest list simply has a list of platform entries with a redirector to an existing manifest so that an engine can go retrieve the correct components for that specific platform/architecture combination. In DockerHub today, all official images are now actually manifest lists, allowing for many platforms to be supported using the same image name:tag combination. I have a tool which can query entries in a registry and show whether they are manifest lists and also dump the contents of a manifest--both manifest lists and "regular" manifests. You can read more at the manifest-tool GitHub repository.

Slide 12 from this talk on containerd design also has a nice graphical representation of how manifest lists link to manifests, which represent the image config and layers for a specific platform.

Solution 2

An image is a combination of a JSON manifest and individual layer files. The process of pulling an image centers around retrieving these two components. So when you pull an Image file:

1. Get Manifest:

   GET /v2/<name>/manifests/<reference>
   

2. When the manifest is in hand, the client must verify the signature to ensure the names and layers are valid.

3. Then the client will then use the digests to download the individual layers. Layers are stored in as blobs in the V2 registry API, keyed by their digest.

Comments

[RichVel]

The containerd design slides linked near end are now behind the Slideshare/Scribd paywall - there's a similar slide deck that is still free to read. The same diagram seems to be at slide 13.