Using env variable in Spring Boot's application.properties

Question

We are working on a Spring Boot web application, and the database we are using is MySQL;

• the setup we have is we first test it locally (means we need to install MySQL on our PC);

• then we push to Bitbucket;

• Jenkins automatically detects the new push to Bitbucket and does a build on it (for Jenkins mvn build to pass we also need to install MySQL on the virtual machines that is running Jenkins).

• if Jenkins build passes we push the code to our application on OpenShift (using the Openshift deployment plugin on Jenkins).

The problem we have, as you may have already figured it out, is that:

• in application.properties we can not hard code the MySQL info. Since our project will be running in 3 different places (local, Jenkins, and OpenShift), we need to make the datasource field dynamic in application.properties (we know there are different ways of doing it but we are working on this solution for now).

    spring.datasource.url = 
    spring.datasource.username = 
    spring.datasource.password = 
  

The solution we came up with is we create system environment variables locally and in the Jenkins VM (naming them the same way OpenShift names them), and assigning them the right values respectively:

export OPENSHIFT_MYSQL_DB_HOST="jdbc:mysql://localhost"
export OPENSHIFT_MYSQL_DB_PORT="3306"
export OPENSHIFT_MYSQL_DB_USERNAME="root"
export OPENSHIFT_MYSQL_DB_PASSWORD="123asd"

We have done this and it works. We have also checked with Map<String, String> env = System.getenv(); that the environment variables can be made into java variables as such:

String password = env.get("OPENSHIFT_MYSQL_DB_PASSWORD");   
String userName = env.get("OPENSHIFT_MYSQL_DB_USERNAME");   
String sqlURL = env.get("OPENSHIFT_MYSQL_DB_HOST"); 
String sqlPort = env.get("OPENSHIFT_MYSQL_DB_PORT");

Now the only thing left is we need to use these java variables in our application.properties, and that is what we are having trouble with.

In which folder, and how, do we need to assign the password, userName, sqlURL, and sqlPort variables for application.properties to be able to see them and how do we include them in application.properties?

We have tried many things one of them being:

spring.datasource.url = ${sqlURL}:${sqlPort}/"nameofDB"
spring.datasource.username = ${userName}
spring.datasource.password = ${password}

No luck so far. We are probably not putting these environment variables in the right class/folder or are using them incorrectly in application.properties.

Solution 1

You don't need to use java variables. To include system env variables add the following to your application.properties file:

spring.datasource.url = ${OPENSHIFT_MYSQL_DB_HOST}:${OPENSHIFT_MYSQL_DB_PORT}/"nameofDB"
spring.datasource.username = ${OPENSHIFT_MYSQL_DB_USERNAME}
spring.datasource.password = ${OPENSHIFT_MYSQL_DB_PASSWORD}

But the way suggested by @Stefan Isele is more preferable, because in this case you have to declare just one env variable: spring.profiles.active. Spring will read the appropriate property file automatically by application-{profile-name}.properties template.

Solution 2

The easiest way to have different configurations for different environments is to use spring profiles. See externalised configuration.

This gives you a lot of flexibility. I am using it in my projects and it is extremely helpful. In your case you would have 3 profiles: 'local', 'jenkins', and 'openshift'

You then have 3 profile specific property files: application-local.properties, application-jenkins.properties, and application-openshift.properties

There you can set the properties for the regarding environment. When you run the app you have to specify the profile to activate like this: -Dspring.profiles.active=jenkins

Edit

According to the spring doc you can set the system environment variable SPRING_PROFILES_ACTIVE to activate profiles and don't need to pass it as a parameter.

is there any way to pass active profile option for web app at run time ?

No. Spring determines the active profiles as one of the first steps, when building the application context. The active profiles are then used to decide which property files are read and which beans are instantiated. Once the application is started this cannot be changed.

Solution 3

Flyway doesn't recognize the direct environment variables into the application.properties (Spring-Boot V2.1). e.g

spring.datasource.url=jdbc:mysql://${DB_HOSTNAME}:${DB_PORT}/${DB_DATABASE}
spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASS}

To solve this issue I did this environment variables, usually I create the file .env:

SPRING_DATASOURCE_URL=jdbc:mysql://127.0.0.1:3306/place
SPRING_DATASOURCE_USERNAME=root
SPRING_DATASOURCE_PASSWORD=root

And export the variables to my environment:

export $(cat .env | xargs)

And finally just run the command

mvn spring-boot:run

Or run your jar file

java -jar target/your-file.jar

There another approach here: https://docs.spring.io/spring-boot/docs/2.1.0.RELEASE/maven-plugin/examples/run-env-variables.html

Solution 4

This is in response to a number of comments as my reputation isn't high enough to comment directly.

You can specify the profile at runtime as long as the application context has not yet been loaded.

// Previous answers incorrectly used "spring.active.profiles" instead of
// "spring.profiles.active" (as noted in the comments).
// Use AbstractEnvironment.ACTIVE_PROFILES_PROPERTY_NAME to avoid this mistake.

System.setProperty(AbstractEnvironment.ACTIVE_PROFILES_PROPERTY_NAME, environment);
ApplicationContext applicationContext = new ClassPathXmlApplicationContext("/META-INF/spring/applicationContext.xml");

Solution 5

Here is a snippet code through a chain of environments properties files are being loaded for different environments.

Properties file under your application resources ( src/main/resources ):-

 1. application.properties
 2. application-dev.properties
 3. application-uat.properties
 4. application-prod.properties

Ideally, application.properties contains all common properties which are accessible for all environments and environment related properties only works on specifies environment. therefore the order of loading these properties files will be in such way -

 application.properties -> application.{spring.profiles.active}.properties.

Code snippet here :-

    import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
    import org.springframework.core.io.ClassPathResource;
    import org.springframework.core.io.Resource;

    public class PropertiesUtils {

        public static final String SPRING_PROFILES_ACTIVE = "spring.profiles.active";

        public static void initProperties() {
            String activeProfile = System.getProperty(SPRING_PROFILES_ACTIVE);
            if (activeProfile == null) {
                activeProfile = "dev";
            }
            PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer
                    = new PropertySourcesPlaceholderConfigurer();
            Resource[] resources = new ClassPathResource[]
                    {new ClassPathResource("application.properties"),
                            new ClassPathResource("application-" + activeProfile + ".properties")};
            propertySourcesPlaceholderConfigurer.setLocations(resources);

        }
    }

View more solutions

Comments

[Edward J Beckett]

Read @ConfigurationProperties to learn more. However, this is a perfect use case for Profile specific configuration properties

[Alex Dvoretsky]

can you please give a link to the reference fragment about using env variables in the application.properties files?

[Alex Dvoretsky]

Found my self docs.spring.io/spring-boot/docs/current/reference/htmlsingle‌​/…

[Tom Hartwell]

I like this answer, but what if you want the profile name to come from the environment? I've tried -Dspring.active.profiles=$SPRING_ACTIVE_PROFILES, and setting the OS env var in /etc/profile.d/myenvvars.sh, but Spring Boot doesn't pick that up

[Fabian]

SPRING_PROFILES_ACTIVE works because of the relaxed binding feature of spring boot docs.spring.io/spring-boot/docs/1.3.0.BUILD-SNAPSHOT/referen‌​ce/…

[Rudi]

thanks for this answer Stefan, it worked for me, but with one change - the property is actually spring.profiles.active and not spring.active.profiles

[vishal]

is there any way to pass active profile option for web app at run time ? Default to production env but should be able to activate dev, staging etc at runtime (e.g. through query parameter)

[Fırat Küçük]

This method is more convenient for docker linking. For instance: docker run --name my-tomcat -p 127.0.0.1:8080:8080 -e APP_DB_DB=mydb -e APP_DB_USER=dbuser -e APP_DB_PASS=dbpass --link mongo-myapp:mongo -v /path-to/tomcat/webapps:/usr/local/tomcat/webapps -d tomcat:8-jre8-alpine

[kipper_t]

Whilst Spring profiles can be very useful, in relation to the OP they aren't suitable. This is due to how source code is stored and sensitivity of the properties information stored with that. The OP context is around Database access. For that situation you don't want prod details in plain text in source. This means if the source is compromised then the database is also compromised. It is better to use env variables or secret tools for this instead such as Vault. I prefer env. I'd also make all environments operate the same way in this regards for consistency. It avoids accidents in the future.

[kipper_t]

This is absolutely the best way to go. Using environment variables means you don't need to list secrets in plain text along side your application. This is significantly more secure and reduces dependency on your source code access security measures to protect your entire estate. An accidental SO post with properties included doesn't result in information leaking out.

[abe]

I wanted to add to this and mention that if you are using spring boot(didn't check if it works without boot), then any property can be overriden via an environment variable automatically without modifying your application.properties. ie, if you have a property called spring.activemq.broker-url then the corresponding environment variable would be: SPRING_ACTIVEMQ_BROKER_URL. periods and dashes are automatically converted to underscores. This is extremely convenient when working with containers/spring boot.

[Kevin Hawk]

it is spring.profiles.active, not spring.active.profiles

[Mikhail Golubtsov]

If you design for cloud it's not a preferable way to use Spring profiles. Using environment variables is recommended by the 12 factor app standard: 12factor.net/config

[Colin D Bennett]

You can use a Spring Boot profile properties file external to the application JAR. This environment-specific file, for instance, application-production.properties, would be deployed to the production machine in a secure way, and would not typically be in the application source code repository.

[PatS]

What is env-vars? How are they used. Your answer refers to things without a complete description and you do not include any links. I almost downvoted this, but I see your rep is 21 so you are new and one person found your answer useful, so I let it go, but try to provide more information in future answers, and welcome to SO (Stack Overflow). I hope you enjoy it as much as I do.

[Felipe Girotti]

Thanks @PatS, I added more details, hope it will useful.

[PatS]

Excellent changes. Thanks updating your answer.

[Ken Bekov]

@Carmageddon as far as I know, Spring cloud bootstrap phase happens before ApplicationContext is loaded. So Spring simply can't add vars from bootstrap.properties to the Environment.

[underscore_05]

I know this topic is a bit old. But you can combine both environment variable setup and spring profile setup. You dev profile should have static information while your production profile can make use of the environment variables. In this way dev's no longer need to define environment variables on their machine if they just want to deploy the development profile.

[danny]

@abe thanks for in the information. I checked the spring doc and it depends on how do you inject environment variables. according to docs.spring.io/spring-boot/docs/current/reference/html/…, @value doesn't support the flexible binding but ConfigurationProperties can do as what you described.

[Agustín Clemente]

This doesn't works for me, for some reason my spring app is trying to reach this route: ${mongo_port}:27017 instead of localhost:27017

[Master Chief]

Does the environment variables resolve in windows based environment? The syntax for environment variable seems like linux.

[Ken Bekov]

@Master Chief yes it does

[ChickenFeet]

Doesn't Spring Boot handle this scenario out of the box? See External Config documentation here

[GabrielBB]

@underscore_05 Yup, that's how I'm doing it

[GabrielBB]

@abe I think that only works for Spring own properties and not your custom properties

[antak]

There's an unfortunate gotcha with this: If Spring Boot can't find the environment variable specified by the placeholder, it silently leaves the placeholder unexpanded.

[JackTheKnife]

What about when datasource is MongoDB ReplicaSet and I need to pass X-number of members?

[Daniil]

Spring profiles do not solve the problem of having different set of database credentials - you won't store them in plain text anyway. Also spring profiles approach (though being useful before) is less and less relevant in containerized world, you don't want to create different images for different environments

[Raj]

Link returns 404

[apetrelli]

Putting configuration of your external servers inside the codebase is not a good idea. The codebase should be clean of external server configuration for security. Environment variables are safer because, especially for username and password, in a cloud environment like OKD or OpenShift, they can be stored encrypted using a "secret", so only the administrator can set the appropriate values, not the developer.

[ACV]

Nice thanks. How can I use the ${DB_PASSWORD} but make it encrypted

[MarkHu]

This seems really complex. Are you sure you aren't doing something the hard way, that Spring intended to make easy?