Decreasing Docker Build Times by 50% in Github Actions with Docker Buildx Caching

Written by Michael Shi • Published on Jan 14, 2023

One of the slower parts of our Cypress E2E test suite in CI used to be our Docker build step, taking up over 7 minutes to build our application before we can even start running our tests. We’ve spent some time setting up Docker’s new Buildx cache backend to cache build layers (allowing our CI to skip build steps when nothing has changed). We’ve gone from builds routinely taking 8 minutes, to 4 minutes on average, and down to 1 minute when the build is fully cached.

The best part is, it only takes a few lines of code in your workflow to enable caching!

Raking in Cache

We initially started with a simple docker compose build inside of Github Actions, without any caching built-in. We only had to make 2 modifications to our workflow to leverage Docker’s buildx Github Actions cache backend.

1. Setting up Github Actions cache API for Docker
2. Switching from docker compose build to docker buildx bake

Expose Github Action’s cache API key info to Docker

The first thing we’ll need to do is to have our cache API key be available as an environment variable for Docker to consume. This step is super simple as we can use crazy-max/ghaction-github-runtime@v2 to expose the information to Docker.

- name: Expose GitHub Runtime
  uses: crazy-max/ghaction-github-runtime@v2

Creating a new Buildx builder instance

This is the first step of two steps we do to set up buildx, creating a new builder using the docker-container driver is necessary to support using the Github Actions cache.

- name: Build images
  run: |
    docker buildx create --use --driver=docker-container

Replacing docker compose build with docker buildx bake

We’ll also need to modify our old docker compose build call to leverage buildx bake instead, which can leverage buildx.

- name: Build images
  run: |
    docker buildx create --use --driver=docker-container
    docker buildx bake -f ./docker-compose.ci.yml --set *.cache-to="type=gha,mode=max" --set *.cache-from="type=gha" --load

The last few flags configure the cache to Github Actions (gha) so that Docker cache layers are read and sent to Github Actions cache. We set mode=max so that intermediate layers get cached as well. This causes us to use more of our Github Actions cache, but increases our cache hit rate dramatically. The default mode is min which only cache final layer outputs, which did not benefit us as much.

The --load flag at the end ensures the built image is available in your local docker images so that it can be ran later on by Docker.

Complete Example

Here’s a complete example of what a Github Action workflow with caching enabled looks like:

name: E2E
 
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
 
jobs:
  e2e:
    timeout-minutes: 16
    runs-on: ubuntu-20.04
    steps:
      - name: Checkout
        uses: actions/checkout@v2
 
      - name: Expose GitHub Runtime
        uses: crazy-max/ghaction-github-runtime@v2
 
      - name: Build images
        run: |
          docker buildx create --use --driver=docker-container
          docker buildx bake -f ./docker-compose.ci.yml -f ./docker-compose.e2e.yml --set *.cache-to="type=gha,mode=max" --set *.cache-from="type=gha" --load
 
      - name: Spin up services
        run: |
          docker-compose -f ./docker-compose.ci.yml -f ./docker-compose.e2e.yml up -d
 
      - name: Install E2E deps
        uses: bahmutov/npm-install@v1
        with:
          working-directory: e2e
 
      - name: Run Tests
        working-directory: ./e2e
        run: yarn run test --browser chrome
        env:
          CYPRESS_DEPLOYSENTINEL_KEY: ${{ secrets.CYPRESS_DEPLOYSENTINEL_KEY }}

Now your Docker builds (and your overall Github Workflow) should be faster than ever before!