Externalising Spring Boot properties when deploying to Docker

Question

In my Spring Boot app I want to externalise the properties to run in a Docker container. When first deployed, the properties that are currently in my-server/src/main/resources/application.yml are loaded and used by the application as expected. All works fine.

However, my problem is that I need these properties to be updatable as needed, so I need access to the application.yml file once on the Docker container. But at this point, it's not included in the build/docker/ directory before running the buildDocker task, so won't be copied over or accessible after first deployment.

So, what I have tried is to copy the Yaml file into the docker/ build directory, copy it to an accessible directory (/opt/meanwhileinhell/myapp/conf), and use the spring.config.location property to pass a location of the config to the Jar in my Dockerfile:

ENTRYPOINT  ["java",\
...
"-jar", "/app.jar",\
"--spring.config.location=classpath:${configDirectory}"]

Looking at the Command running on the Docker container I can see that this is as expected:

/app.jar --spring.config.location=classpath:/opt/meanwhileinhell/myapp/conf]

However, when I update a property in this file and restart the Docker container, it isn't picking up the changes. File permissions are:

-rw-r--r-- 1 root root  618 Sep  5 13:59 application.yml

The documentation states:

When custom config locations are configured, they are used in addition to the default locations. Custom locations are searched before the default locations.

I can't seem to figure out what I'm doing wrong or misinterpreting, but probably more importantly, is this the correct way to externalise the config for this type of Docker scenario?

Solution 1

DOCKER IMAGE CONFIGURATION

If you look to the way Spring recommends to launch a Spring Boot powered docker container, that's what you find:

FROM openjdk:8-jdk-alpine
VOLUME /tmp
ARG JAR_FILE
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

That means your image extends openjdk and your container has its own environment. If you're doing like that, it would be enough to declare what you want to override as environment properties and Spring Boot will fetch them, since environment variables take precedence over the yml files.

Environment variables can be passed in your docker command too, to launch the container with your desired configuration. If you want to set some limit for the JVM memory, see the link below.

---

DOCKER COMPOSE SAMPLE

Here you have an example of how I launch a simple app environment with docker compose. As you see, I declare the spring.datasource.url property here as an environment variable, so it overrides whatever you've got in your application.yml file.

version: '2'
services:
    myapp:
        image: mycompany/myapp:1.0.0
        container_name: myapp
        depends_on:
        - mysql
        environment:
            - SPRING_DATASOURCE_URL=jdbc:mysql://mysql:3306/myapp?useUnicode=true&characterEncoding=utf8&useSSL=false
        ports:
            - 8080:8080

    mysql:
        image: mysql:5.7.19
        container_name: mysql
        volumes:
            - /home/docker/volumes/myapp/mysql/:/var/lib/mysql/
        environment:
            - MYSQL_USER=root
            - MYSQL_ALLOW_EMPTY_PASSWORD=yes
            - MYSQL_DATABASE=myapp
        command: mysqld --lower_case_table_names=1 --skip-ssl --character_set_server=utf8

See also:

• How do I pass environment variables to Docker containers?
• Limit JVM memory consumption in a Docker container

Solution 2

I personally would consider two options:

1. Using an environment variable per config

   app:
     image: my-app:latest
     ports:
       - "8080:8080"
     environment:
        SPRING_DATASOURCE_URL=jdbc:mysql://db:3306/table
   

2. Using SPRING_APPLICATION_JSON

   app:
     image: my-app:latest
     ports:
       - "8080:8080"
     environment:
       SPRING_APPLICATION_JSON: '{
         "spring.datasource.url": "jdbc:mysql://db:3306/table",
       }'
   

Solution 3

Personally I'd use Spring Cloud Config Server instead of trying to set up properties files all over the place.

tl;dr it allows you to hold properties in git (which allows version control, branching etc) at a per environment/profile level in a centralised location, which are then served up by REST. Spring Boot has full support for it; in effect it's just another property source that ends up in your Environment.

https://spring.io/guides/gs/centralized-configuration/

Solution 4

So I managed to get it working. Rather than passing the classpath to the directory in my DockerFile:

"--spring.config.location=classpath:${configDirectory}"]

I instead tried passing the full location of the file:

 "--spring.config.location=file:${configDirectory}/application.yml"]

This now updates upon restart of the Docker container.

Solution 5

A variation on Xtreme Biker's answer, this time for deployment of a Spring boot war into a dockerized TomCat…

I recommend including a nominal application.yml in your app, but use Docker environment variables to override any individual keys which need environment-specific variation.

The reason I recommend this approach (using Docker environment variables) is:

• your docker image can use exactly the same artefact as you might use for local development
• using volume-mounts is painful; you need to find somewhere for them to live on your docker host — which turns that host into a snowflake
• using docker secrets is painful; image or application layer need to be changed to explicitly lookup secrets from the filesystem

Spring Boot's Externalized Configuration docs explain two ways to supply environment via command-line:

• UN*X env vars (i.e. SPRING_DATASOURCE_USERNAME=helloworld)
• Java options (i.e. -Dspring.datasource.username=helloworld)

I prefer Java options, because they express an explicit intent: "this is intended for the following Java process, and only for that Java process".

Finally: I would use TomCat's CATALINA_OPTS as the mechanism for passing those Java options. Documentation from catalina.sh:

(Optional) Java runtime options used when the "start", "run" or "debug" command is executed. Include here and not in JAVA_OPTS all options, that should only be used by Tomcat itself, not by the stop process, the version command etc. Examples are heap size, GC logging, JMX ports etc.

Because CATALINA_OPTS is an easier route than making your Docker image responsible for creating a setenv.sh and passing the appropriate Docker env declarations into it.

---

Build your .war artefact like so:

./gradlew war

We expect a .war artefact to be output by Gradle to build/libs/api-0.0.1-SNAPSHOT.war.

Use such a Dockerfile:

FROM tomcat:8.5.16-jre8-alpine

EXPOSE 8080

COPY build/libs/api-0.0.1-SNAPSHOT.war /usr/local/tomcat/webapps/v1.war

CMD ["catalina.sh", "run"]

Build your Docker image like so:

docker build . --tag=my-api

Pass CATALINA_OPTS to your container like so:

docker run -it \
-p 8080:8080 \
-e CATALINA_OPTS="\
-Dspring.datasource.url='jdbc:mysql://mydatabase.stackoverflow.com:3306' \
-Dspring.datasource.username=myuser \
" \
my-api

And a docker-compose variant looks like this:

version: '3.2'
services:
  web:
    image: my-api
    ports:
      - "8080:8080"
    environment:
      - >
        CATALINA_OPTS=
        -Dspring.datasource.url='jdbc:mysql://mydatabase.stackoverflow.com:3306'
        -Dspring.datasource.username=myuser

View more solutions

Comments

[MeanwhileInHell]

Thanks for your answer. That is how my container is setup, using openjdk. I managed to get it working using a variation of the approach I was using, instead of passing the classpath, passing the full location of the file.

[MeanwhileInHell]

I totally agree, however requirements forbade it for some reason.

[Aritz]

I would strongly encourage you to use a docker based solution instead of having a boot application.yml external file. If you mess up with docker commands, your best is to go with docker-compose. This way you can have the whole application configuration in a compose file and change it whenever you want. As per the other answers, I would say the config server might be the best way to go for multi-instance applications, but it has some extra work.

[MeanwhileInHell]

Hmmm, I think I am going to have to rethink my solution. Currently I do use docker compose. So in the environment section, how do I overwrite a property from my application.yml that would be like com:mih:my-server:db:contactPoint?

[Aritz]

You must replace the colons (':') by dots ('.') or underscores ('_'). See this: If you use environment variables rather than system properties, most operating systems disallow period-separated key names, but you can use underscores instead (e.g. SPRING_CONFIG_NAME instead of spring.config.name).

[MeanwhileInHell]

Hotdog, got that working. That seems to be a much simpler way of controlling properties. Thanks a lot for your help!

[Nathan Niesen]

This answer is a bit dated. The Boot with Docker guide now uses Java as the entry point, not sh: ENTRYPOINT ["java",".... Also, the JDK_JAVA_OPTIONS (JDK-8170832) environment variable can now be used for VM options.

[surfealokesea]

plus one, by the docker run -e CATALINA_OPTS