--mount=type=cache in buildkit

Question

I'm trying to get maven offline builds working from within a docker container. I've enabled buildkit. I've run mvn dependency:go-offline -s ~/checkouts/settings.xml to cache the dependencies in /root/.m2 of host machine. I wish to use this inside the container that builds maven project.

Here is my Dockerfile:

#syntax=docker/dockerfile:experimental
FROM maven:3.6.1-jdk-11 AS build
WORKDIR /
COPY . /
RUN --mount=type=cache,target=/root/.m2 mvn -o install 

FROM scratch
COPY --from=build /admin/admin- 
rest/target/admin-rest.war /webapps/ROOT.war

When I try to docker build this Dockerfile, I get the following error:

Plugin org.codehaus.mojo:build-helper-maven-plugin:3.0.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.mojo:build-helper-maven-plugin:jar:3.0.0 has not been downloaded from it before. -> [Help 1]

My Docker version:

Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77
 Built:             Sat May  4 02:35:57 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:19:41 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.5
  GitCommit:        bb71b10fd8f58240ca47fbb579b9d1028eea7c84
 runc:
  Version:          1.0.0-rc6+dev
  GitCommit:        2b18fe1d885ee5083ef9f0838fee39b62d653e30
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Am I right in my expectation that /root/.m2 from my host machine needs to get mounted on to /root/.m2 inside the docker container environment? And when I run mvn -o install it needs to refer to the mounted /root/.m2?

I added tail -f /dev/null, removed second build stage and changed the mvn install to mvn version with the cache-mount instruction intact in order to debug this. I see that nothing gets mounted on to /root/.m2 inside the container. It is empty. (Unsure if this is the right way to debug this as it may get unmounted after the image is built)

Solution

It's best to think of --mount=type=cache as being like a named volume in docker, managed by BuildKit, and potentially deleted if the BuildKit cache gets full or a prune is requested. The next time you run a build, that same named volume may be available, significiantly reducing the build time spent downloading dependencies. While very useful, that doesn't appear to be what you're looking for here. To use a cache like this, you'd need to include the go-offline as an earlier step in the Dockerfile:

#syntax=docker/dockerfile:experimental
FROM maven:3.6.1-jdk-11 AS build
WORKDIR /
# copy just the pom.xml for cache efficiency
COPY ./pom.xml /
# go-offline using the pom.xml
RUN --mount=type=cache,target=/root/.m2 mvn dependency:go-offline
# now copy the rest of the code and run an offline build
COPY . /
RUN --mount=type=cache,target=/root/.m2 mvn -o install 

FROM scratch
COPY --from=build /admin/admin-rest/target/admin-rest.war /webapps/ROOT.war

---

To mount a directory into the container from the host, what you appear to be looking for is a bind mount. And with BuildKit's experimental settings, that is available, but only to the build context, not to any arbitrary directory on the build host. For that, you can place your .m2 directory in the build context directory and then use the following line in your Dockerfile:

RUN --mount=type=bind,source=./.m2,target=/root/.m2,rw mvn -o install

Note if any of the dependencies change, then Maven may try to connect over the network again.

Comments

[A.R.K.S]

Initially I was thinking why would I do go-offline inside a container which is ephemeral.Thanks @BMitch that clears up my mis-understanding.I'll try this.

[BMitch]

@AshwiniR When you run the same command in a later build, docker will reuse the build cache results from before. To avoid breaking the cache, the COPY . / is not run until after the go-offline step.

[Evgeny]

The note about mounting the directory from the host is incorrect, --mount=type=bind,source=./.m2,target=/root/.m2,rw mounts the directory from either the context, or another container, depending on the from= parameter. Unfortunately it doesn't support mounting from the host afaik.

[BMitch]

@Evgeny the answer specifies "but only to the build context, not to any arbitrary directory on the build host". I did my best to indicate the constraints given the OP's request to mount an arbitrary host directory and explain why their command was not working.

[membersound]

The important part here is that every mvn statement must include the ` --mount=type=cache,target=/root/.m2` mount. Otherwise dependencies will be downloaded again.