This article documents my experience in building Docker images for multi-system architectures, as well as some of my personal understandings.

Pre-requisite knowledge

CPU architectures

The mainstream CPU architectures are of two types: x86 and ARM, but they are not always named as such during development. For example, amd64 and x86_64 refer to the 64-bit architecture of x86, while arm64v8, aarch64, and arm64 refer to the 64-bit architecture of ARM.

docker hub

In the docker hub, the supported architectures are listed for all major images, and you can filter the images by Architectures.

docker buildx

Prior to docker buildx, we could only build images via docker build. As the name suggests, docker buildx is an extension to docker’s build capabilities, and one of its biggest highlights is its support for multi-system architecture builds.

docker buildx for Docker v19.03+

A docker buildx build example.

1

docker buildx build -t cop/cop-demo --platform linux/amd64 .

We will describe this command in detail below.

docker manifest

The docker manifest manifest, which is still experimental, is a key command for building multi-system architectures. It gives us information about the hierarchy of an image, its size, signature and, most critically, the architecture supported by the image.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

~ docker manifest inspect openjdk
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 954,
         "digest": "sha256:afbe5f6d76c1eedbbd2f689c18c1984fd67121b369fc0fbd51c510caf4f9544f",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 954,
         "digest": "sha256:0722e5cd28b8834d2c2e6a3659ba4631c6f6aea6aa88361feff58032bb3514e3",
         "platform": {
            "architecture": "arm64",
            "os": "linux",
            "variant": "v8"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2983,
         "digest": "sha256:5ecbb996abc91a17257ae0192f2b69a0a3096279a5b9167aef656d6b88972b65",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.20348.643"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2983,
         "digest": "sha256:702402cac2a4e078ec1df8aa23e0f13c7155621dffc520e5ac21e44d94d9ca76",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.17763.2803"
         }
      }
   ]
}

The platform column. This is the information about the architecture support that we are most interested in.

platform column

Comparing the digest information, you can see that it is the same as the docker hub information.

This article describes the environment

All operations in this article are based on Mac M1, Docker Desktop. All operations in this article are based on Mac M1, Docker Desktop, and may involve experimentation and buildkit features, which need to be enabled. My configuration reference:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

{
  "features": {
    "buildkit": true
  },
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "20GB"
    }
  },
  "experimental": true
}

Pulling multi-architecture mirrors

Before the Mac M1 / ARM architecture was used, pulling images didn’t seem to be that much of a hassle.

1

docker pull openjdk

As you can see from the previous article, openjdk has different digest for different architectures, and docker will determine the architecture of the current machine and pull the version of the corresponding architecture. For example, on Mac M1, I pull the arm64 version.

1
2

~ docker image inspect openjdk | grep Arch
        "Architecture": "arm64",

We can also specify the image corresponding to the OS & architecture to be pulled by using the --platform parameter.

1

docker pull --platform linux/amd64 openjdk

The same image tag, only one copy will be saved locally, check the architecture information of the local image again, it is already amd64.

1
2

~ docker image inspect openjdk | grep Arch
        "Architecture": "amd64",

The hub side supports storing multiple mirrors according to Arch, actually using mechanisms such as manifest, but not all mirrors support manifest.

This also means that the --platform argument does not apply to all mirrors, and you can check the Arch support of mirrors with docker manifest inspect.

Build Multi-Architecture Mirror

I had a lot of confusion and pitfalls while investigating the option of building multi-architecture mirrors, but I ended up with the option of building multi-architecture mirrors with docker buildx and merging manifest lists with docker manifest.

Finding a parent image that supports multiple architectures

Take openjdk for example, it provides both arm64 and amd64 versions, so we’ll use it for the demo.

Java demo.

1
2
3
4
5
6
7

public class Main {

    public static void main(String[] args) {
        System.out.println("hello world");
    }

}

Dockerfile:

1
2
3
4
5
6


FROM openjdk:17
COPY . /usr/src/myapp
WORKDIR /usr/src/myapp
RUN javac Main.java
CMD ["java", "Main"]

Build multi-architecture images locally

1
2
3
4
5
6
7
8
9

~ docker buildx inspect --bootstrap
Name:   default
Driver: docker

Nodes:
Name:      default
Endpoint:  default
Status:    running
Platforms: linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

The default builder of docker buildx supports building images for linux/arm64, linux/amd64, etc. Docker achieves this capability by cross-building, so it is not limited to the CPU architecture of the build machine.

The docker buildx supports the -platform argument, which specifies the OS & CPU architecture of the build image.

1
2

docker buildx build -t kiritomoe/java-multi-arch-demo:1.0-aarch64 --platform linux/arm64 -o type=docker .
docker buildx build -t kiritomoe/java-multi-arch-demo:1.0-x86_64 --platform linux/amd64 -o type=docker .

Creating a Push Manifest List

In the previous step, we have actually built a multi-architecture image, but at this point, different architectures have different tags, which is a bit different from the familiar openjdk solution. openjdk and other images use docker manifest to bind multiple architectures to the same tag.

1
2
3

docker manifest create kiritomoe/java-multi-arch-demo:1.0 kiritomoe/java-multi-arch-demo:1.0-x86_64 kiritomoe/java-multi-arch-demo:1.0-aarch64
docker manifest push kiritomoe/java-multi-arch-demo:1.0
docker manifest rm kiritomoe/java-multi-arch-demo:1.0

Note that the manifest kiritomoe/java-multi-arch-demo:1.0 was finally pushed, and no other images were pushed. And after the manifest is pushed, the local copy needs to be deleted, which makes it possible to perform operations such as docker manifest inspect kiritomoe/java-multi-arch-demo:1.0 locally in the future to ensure that it is loaded from the remote repository, and the manifest only makes sense if it exists in the remote repository to make sense.

View multi-arch images from remote repositories

remote repositories

Successfully bind multiple architectures to the same tag.

Use the command line to view it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

~ docker manifest inspect kiritomoe/java-multi-arch-demo:1.0
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1574,
         "digest": "sha256:6cceb21f1a225c9f309f51413fdb7cf8d8ea3980a832c84c07ce3e30fed41628",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1574,
         "digest": "sha256:0dddf9a86e60de3fd56d074a8f535a90e391b35a6e503fedd09f87c8c32ca75a",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      }
   ]
}

Some of the “best” practices

If you research the multi-architecture support, you will find that the above mentioned solutions are not the only ones supported, and with my limited energy, I didn’t look into the history of multi-architecture support for docker in detail. If not for the project, God knows I would have spent two days researching these things. But the above solution is the simplest one I have come up with so far.

While docker implements the ability to automatically pull native images based on the compiling machine, this capability is not available in all cases. For example

  1. if the build machine is not controllable, then the act of compiling will also become uncontrollable.
  2. the build machine is not necessarily the machine that will eventually run the image
  3. local builds for test development scenarios

To keep it all under control, my personal advice is to follow two principles.

  1. Business images offer multi-arch support. For example, I chose centos for my base image (centos supports multi-arch), my local environment is Mac M1, and our company’s build machine is x86, not everyone is a docker expert, I want to make the From centos strategy of pulling images manageable, and I am willing to write two Dockerfiles for it: Dockerfile_amd64 and Dockerfile_arm64. I would like to write two Dockerfiles for this purpose: Dockerfile_amd64 and Dockerfile_arm64, and eventually merge the manifest of my two products to implement multi-arch.
  2. Other generic mirrors support multi-arch while providing different Arch tags, for example, in business scenarios, several types of base mirrors are generally required
    • Base images for java applications: java-base:1.0, java-base:1.0-aarch64, java-base:1.0-x86_64
    • Base images for front-end applications: nginx-base:1.0, nginx-base:1.0-aarch64, nginx-base:1.0-x86_64
    • Base images for general-purpose applications: centos-base:1.0, centos-base:1.0-aarch64, centos-base:1.0-x86_64

While I am aware that it is possible to accurately pull the image of a given Arch via sha256, it would add a lot of cost to understanding.