Hacking around with Docker images on a laptop is a productive way to do
experimental development but if your building Ubuntu images locally and have a
lot of packages to
apt install your going to be stuck waiting for downloads
if you need to change something within a single layer.
If your living the dream and hanging out on cafe WIFI this can be a big deal.
A really simple fix for this would be to mount
/var/cache/apt as a volume
so that files can be cached on the host. Unfortunately support for
docker build -v ... doesn't exist yet:
- Feature request: Mount volume when run docker build
- build time only -v option
- Proposal: Nested builds
- Add support for named build stages
podman is a drop-in replacement for the
which brings many new features and Kubernetes integration. One of these
podman build support for volumes :)
There's a great
from Redhat that walks docker users through how
podman and its companion
buildah fit together.
Follow the podman installation guide when your ready to try it out.
Speeding up Ubuntu image building
podman installed, lets take a look at how we can plug-in a cache
directory for Docker builds.
/var/cache/apt as its cache directory, so if we wanted to cache
podman build command becomes something
mkdir -p ~/podman_apt_cache podman build -v ~/podman_apt_cache:/var/cache/apt .
Real world performance
With around 40 packages to install over installed
$ time docker build . > /dev/null real 5m52.357s user 0m0.338s sys 0m0.189s
$ time podman build -v ~/podman_apt_cache:/var/cache/apt . > /dev/null real 2m8.366s user 2m5.982s sys 0m27.275s
Build time reduced by around 63%!
No speed increase
ubuntu image cleans up
/var/cache/apt automatically. To fix this
we just need to tell
apt not to clean the cache by adding:
RUN rm /etc/apt/apt.conf.d/docker-clean
Could not get lock /var/lib/apt/lists/lock
If a build was interrupted the
lock file will still be present. Since we're
running apt inside a container it safe to just remove the file:
The package cache file is corrupted
Another error that can be caused by interrupted builds is:
E: Problem renaming the file /var/cache/apt/srcpkgcache.bin.iKB2BL to /var/cache/apt/srcpkgcache.bin - rename (2: No such file or directory) W: You may want to run apt-get update to correct these problems E: The package cache file is corrupted
The fix for this is to just cleanup the files:
rm ~/podman_apt_cache/pkgcache.bin rm ~/podman_apt_cache/srcpkgcache.bin
Sharing the apt cache between concurrent builds won't work properly because
apt expects exclusive access. Since this article is geared around solo
developers working on occasional builds on a laptop, this can be a problem for
Putting it all together
Dockerfile should look something like this:
FROM ubuntu:eoan-20200114 # stop package cleanup on exit RUN rm /etc/apt/apt.conf.d/docker-clean ENV DEBIAN_FRONTEND 'noninteractive' RUN apt-get update && apt-get install -y --no-install-recommends \ your package list here... && rm -rf /var/lib/apt/lists/*
Having to type out all the commands and remember the workarounds for corrupt
files isn't fun, so lets create a
Makefile to simply things:
build: mkdir -p ~/podman_apt_cache podman build -v ~/podman_apt_cache:/var/cache/apt . clean: rm -f ~/podman_apt_cache/pkgcache.bin rm -f ~/podman_apt_cache/srcpkgcache.bin rm -f ~/podman_apt_cache/archives/lock
Now we can build our image by typing:
And if we encounter corruption errors we can fix them by typing:
Caching external downloads between builds saves both time and data so
is a great way to speed up builds and prepare your systems for better Kubernetes
Need help optimising something else? Contact us!