Skip to content

BuildKit Dockerfile Frontend

I am a big fan of pandoc to generate documents instead of using a binary format like MS Word. Feels more natural to have a source and compile it into a delivarable.

Dockerfile Improvement

Installing LaTex and all the dependencies for pandoc is a hassle and the natural way of dealing with that is (of course) to create a container. I found the container cristiangreco/docker-pdflatex on the interweb.

But the image was rather big (4.4GB) and I thought:

Let's see if we can shave of a GB or so and while we are at it make the Dockerfile more readable

The Old Way

Let's tackle readablity first. Not sure if folks are broadly aware, but the syntax used within a Dockerfile can be swapped out, it's called Dockerfile frontend.

I got used to format my RUN lines in Dockerfiles like this:

FROM debian:bullseye-20230109-slim

ENV DEBIAN_FRONTEND noninteractive # (4)
RUN apt-get update \
 && apt-get install -y --no-install-recommends vim \ # (1)
 && apt-get clean -y \ # (2)
 && apt-get autoremove -y # (3)
  1. no-install-recommends should be a default I suppose - not that I use it always, but I should. 😄
  2. clean clears out the local repository of retrieved package files. It removes everything but the lock file from /var/cache/apt/archives/ and /var/cache/apt/archives/partial/.
  3. is used to remove packages that installed to satisfy dependencies for some package and that are no longer needed. Since I used no-install-recommends that should be obsolete.
  4. DEBIAN_FRONTEND will supress question (e.g. select your timezone)

I often see Dockerfile with the && \ at the end of the line and three spaces to intend the next commmand. When the engine prints out the complete multi-line section that introduces a lot of noise. But that's a personal preference, I suppose.

Here Documents

By using the latest frontend we can use here-documents (inline code) we can make this much more readable and clean. To do that, we need to put the frontend we want to use in the very first line of the Dockerfile

# syntax = docker/dockerfile:1.4 # (1)
FROM debian:bullseye-20230109-slim
  1. the latest version is 1.5, but I could not find new features (yet)

Now we are able to use inline code:

# syntax = docker/dockerfile:1.4 # (1)
FROM debian:bullseye-20230109-slim

ENV DEBIAN_FRONTEND noninteractive
RUN <<eot bash
  apt-get update
  apt-get install -y --no-install-recommends vim
  apt-get clean -y
  apt-get autoremove -y
eot

No more backslashes and ampersands - beautiful!

To build this Dockerfile you need to set DOCKER_BUILDKIT=1 to instruct docker to use BuildKit.

export DOCKER_BUILDKIT=1
docker build -t test .

Bash FTW

The nice thing about this is that you can use bash syntax all the way, using tests and such...

# syntax = docker/dockerfile:1.4 # (1)
ARG SRC_IMG=debian:bullseye-20230109-slim
FROM ${SRC_IMG}

ENV DEBIAN_FRONTEND=noninteractive
RUN <<eot bash
  apt-get update
  if [[ "$(cat /etc/os-release |awk -F= '/^ID=/{print $2}')" == debian ]];then
    echo ">>>>>>>>>> [INFO] I know that debian uses prefer 'nano'."
    apt-get install -y --no-install-recommends nano
  elif [[ "$(cat /etc/os-release |awk -F= '/^ID=/{print $2}')" == ubuntu ]];then
    echo ">>>>>>>>>> [INFO] I know that ubuntu folks like 'vim'."
    apt-get install -y --no-install-recommends vim
  else 
    echo "[ERROR] Not sure what there poison for $(cat /etc/os-release |awk -F= '/^ID=/{print $2}') is likely to be."
  fi
eot

Building this will check the operating system and install different editors.

$ docker build test .
*snip*
#3 resolve image config for docker.io/docker/dockerfile:1.4 # (1)
*snip*
#10 5.815 ">>>>>>>>>> [INFO] I know that debian uses prefer 'nano'. # (2)
  1. Using a different frontend
  2. Ok, we detected debian to be the OS.

The same command with a different SRC_IMG will install vim.

$ docker build --build-arg=SRC_IMG=ubuntu:22.04  -t test .
*snip*
#9 9.411 ">>>>>>>>>> [INFO] I know that ubuntu folks like 'vim'.

COPY and HereDocs

I'll leave you with another example of how to use here-docs. Using the COPY command from the github.com/moby/buildkit repository.

# syntax = docker/dockerfile:1.4
FROM alpine
COPY <<-"eot" /app/script.sh
    echo hello ${FOO}
eot
RUN FOO=abc ash /app/script.sh

This Dockerfile will output echo hello abc. On top of that we can even change the permission on-the-fly as well...

# syntax = docker/dockerfile:1.4
FROM alpine
COPY --chmod=0755 <<-"eot" /app/script.sh
    echo hello ${FOO}
eot
RUN FOO=abc /app/script.sh

Conclusion

Dockerfile frontends are a great way of removing workarounds in Dockerfiles and make them more readable. This post became to long already, so I'll stop here (😄), but there is more... I'll write another blog post about caching - that's especially important for us in the HPC world.

Further Reading

Comments