The core goal of this article:
- Use the full docker cli command on a Mac, including support for basic
- Support for x86 emulation, with the ability to build or run images for x86.
- CPU architecture switching where possible, preferably for both arm64 and x86.
2. the choice of tools
First of all, we are most familiar with Docker Desktop, the installation package is huge and the UI is very laggy. Not to mention the startup speed and the occasional jam, so Docker Desktop is not considered at all; so the remaining options are as follows.
- VM solution
- Colima solution
- Lima solution
Let’s start with the conclusion: Lima YES! VM solution is expensive and unpleasant, Colima is temporary and unstable. See Section 5 directly for the Lima solution.
3. Virtual Machine Solution
Currently, the only available or usable virtual machine on M1 is Parallels Desktop, while others like VBox and VMware are still immature; if pure qemu is a bit too hardcore (if you want to package your own scripts, forget it); for Parallels Desktop, we need to buy the development version of the license , because we need to use
prlctl to achieve some automation, several hundred a year… After testing this solution is also somewhat feasible:
- first create a virtual machine like Ubuntu through PD
- install Docker in the virtual machine
- start the virtual machine through cli program, and mount
~rw to the virtual machine
Based on this solution I personally tried, I wrote a small tool PD to help with the mounting action. tool to assist in the mounting process. However, this tool has some obvious drawbacks:
- x86 emulation is not currently supported, which can be mitigated by binfmt, but is not perfect
- Virtual machines cost money and require virtual machine cli support
4. Colima solution
Colima is claimed to be a solution for Mac platform containerized toolchain, but actual testing found that Colima is not stable, sometimes there may be some small problems; of course, the biggest problem of Colima is: customizable degree is not high, the underlying layer is based on Lima. Colima specific use and so on here is not described in detail, is not stable Not really recommended.
5. Lima Solution
Lima is currently an automated VM solution based on QEMU, and thanks to its excellent design, Cloud Init can help us to complete the hook at many stages; so it is easy to install Docker or k8s, or get something else; and many solutions such as docker have official samples, so we can copy them directly and use them.
5.1. Lima installation
Lima is relatively easy to install on Mac, the following command will install the master branch version.
Under normal circumstances, the installation of Lima comes with the installation of QEMU, if QEMU is already installed locally, you may need to execute the following command Upgrade QEMU to 7.0 :
In order to use docker, you also need to install the docker cli via brew:
5.2. Using Lima
By default, Lima generates a
lima shortcut command after installation, but it is not recommended to use it, because it looks convenient but there is no control over too many parameters, so it is still recommended to use the standard
limactl command to operate. limactl is used in the following way:
5.3. Lima configuration file
Lima determines how to create a virtual machine by reading a yaml configuration description file, which has the following basic structure:
5.4. Starting a VM
The limactl command provides a
start subcommand to start a VM, which takes a single argument, the form of which varies depending on the behavior:
- If the argument is a file path, the file is assumed to be the yaml configuration of a lima virtual machine, which is read and started.
- If the argument is a simple string, it first tries to find a virtual machine with the same name among the existing ones, and starts it immediately if it finds one.
- If the argument is a simple string and no existing VM with the same name is found, then try to create a new VM using the built-in templates.
Using the docker configuration file I defined above as an example, we can create a docker virtual machine by starting this configuration directly:
After startup, you will be prompted whether to edit and then start again. This is to use the same configuration to start multiple vm’s, so just start them without editing.
After a few moments the virtual machine will start successfully:
After booting, execute the two commands printed at the bottom to use
docker on the host machine. This essentially uses the docker context feature, and then mounts the sock file from the virtual machine to the host, and configures the docker context for seamless use of docker commands.
5.5. Virtual machine tuning
In some cases, we need to customize the configuration of the VM, mainly by adjusting the
provision section of the configuration file; in this section, if
mode is defined as
system, the command will be executed as root user, otherwise it will be executed as normal user. It is important to note that the scripts we define need to be idempotent, because they will be executed once each time, so generally we need to write the logic for the commands that may cause data erasure actions, to avoid repeated execution.
For file mounting, we recommend using
9p type, future lima will switch to this mount mode completely; also after testing currently only
9p mount mode, local directory rw will not have permission problem when mapping to virtual machine, sshfs mount mode will cause permission error if encountering
chown or other commands, which may lead to container boot failure (e.g. mysql).
During test VM configuration, you can use
limactl delete -f xxxx to force delete the target VM, and then restart it; The VM name is the same as the yaml file name by default, you can use
limactl ls command to check it.
5.6. Multi-platform compatibility
In my docker configuration example above, binfmt is automatically installed every time the virtual machine finishes booting:
This ensures that docker images from other platforms can be run regardless of the original architecture of the Lima VM; typically, some openjdk8 images are only available in amd64, but can still be used if the lima VM is aarch64.
In addition to this “faster” cross-architecture approach, lima also supports defining the architecture directly in the VM, so that the target architecture is emulated directly from the VM system level when qemu is booted; this approach has the advantage of being very compatible with the target architecture, but it will run more slowly. Adjusting the VM architecture is a simple matter of modifying the
arch configuration (note that the image of the target architecture must be configured):
Overall, Docker Desktop is basically hard to use on mac, Colima is not too mature yet, suitable for light docker users; but heavy docker users with customization needs still recommend Lima virtual machine; Lima also supports many operating systems, there are a lot of official sample templates (including k8s, k3s, podman, etc.), which is very suitable for heavy container users.