Docker environment variables in multi-stage builds

docker environment-variables docker-compose dockerfile

10,258

Solution 1

So this is not a multi-stage issue.

It appears ENV variables are only used when running containers (docker-compose up). Not at build time (docker-compose build). So you have to use arguments:

.env:

TEST=11111

docker-compose.yaml:

version: '3'
services:
  test:
    build:
      context: .
      args:
        TEST: ${TEST}

Dockerfile:

FROM nginx:alpine
ARG TEST
ENV TEST ${TEST}
CMD ["sh", "-c", "echo $TEST"]

test command:

docker rmi test_test:latest ; docker-compose build && docker run -it --rm test_test:latest

Seriously the documentation is somewhat lacking.

Reference: https://github.com/docker/compose/issues/1837

Solution 2

The problem is not about multi-stage specifically.

It's about differences between Dockerfile ARG & docker-compose YAML build args ("build arguments"); and Dockerfile ENV & docker-compose YAML environment/.env.

The docs were updated (more recently than the original post), and it is fairly clear now:

args

Add build arguments, which are environment variables accessible only during the build process.

Example from the docker-compose docs

Starting simple, just showing the interaction between Dockerfile and the YAML:

ARG buildno
ARG gitcommithash

RUN echo "Build number: $buildno"
RUN echo "Based on commit: $gitcommithash"

build:
  context: .
  args:
    buildno: 1
    gitcommithash: cdc3b19

build:
  context: .
  args:
    - buildno=1
    - gitcommithash=cdc3b19

Example to tie it back to the question:

See the other answer in this thread.

Docs & deepening your understanding

Learn one layer of abstraction at a time

I recommend to go from the Dockerfile level of abstraction, upward. Making sure you understand each layer before you add the next layer of abstraction.

Dockerfile (and then play with running containers from your Dockerfile ... using default ENV, then playing with --env, then playing with ARG and --build-arg)
Then add docker-compose details in, and play with those.
Then loop back to Dockerfiles and understanding multi-stage builds.

Dockerfile

A helpful blog-post -- focuses on the Dockerfile but in all cases, it's best to understand Dockerfiles alone before adding the extra layers of abstraction on top of that, such as docker-compose YAML.

https://vsupalov.com/docker-arg-env-variable-guide/

docker-compose

Then docker-compose official docs:

multi-stage Dockerfiles

10,258

Author by

Mike Gleason jr Couturier

Updated on July 23, 2022

Comments

Mike Gleason jr Couturier almost 2 years
given this .env file:
```
TEST=33333
```
given this docker-compose.yml file:
```
  service_name:
    image: test
    env_file: .env
    environment:
      TEST: 22222
```
given this Dockerfile file:
```
FROM an_image AS builder

FROM another_image
ENV TEST 11111

CMD ["/bin/echo $TEST"]
```
Whenever I build and run this image in a container, it prints 11111.

If I remove the ENV 11111 line from the Dockerfile, my TEST environment variable is empty...

Is the parent image receiving the environment variables but not the child one?

Thanks!

EDIT:
1. trying ENV TEST ${TEST} didn't work ($TEST is empty)
2. removing ENV TEST didn't work ($TEST is empty)
BMitch over 6 years

I'm not able to reproduce your issue, are you really evaluating the variable in a CMD? The CMD is run time, not build time.
Rebar about 3 years

Is it possible to pass my .env file to the args?
Sean McCarthy over 2 years

ENV variables in a Dockerfile ARE actually available in both the build stage (i.e. RUN commands), and as environment variables in the running containers. See docs here: docs.docker.com/engine/reference/builder/#env