Spring Boot and multiple external configuration files

Question

I have multiple property files that I want to load from classpath. There is one default set under /src/main/resources which is part of myapp.jar. My springcontext expects files to be on the classpath. i.e.

<util:properties id="Job1Props"
    location="classpath:job1.properties"></util:properties>

<util:properties id="Job2Props"
    location="classpath:job2.properties"></util:properties>

I also need the option to override these properties with an external set. I have an external config folder in cwd. As per spring boot doc config folder should be on classpath. But its not clear from doc if it will only override the application.properties from there or all the properties in config.

When I tested it, only application.properties gets picked up and rest of properties are still picked up from /src/main/resources. I have tried supplying them as comma separated list to spring.config.location but the default set is still not being overriden.

How do I make multiple external config files override default ones?

As workaround I currently used app.config.location (app specific property) which I supply through the command line. i.e

java -jar myapp.jar app.config.location=file:./config

and I changed my applicationcontext to

<util:properties id="Job1Props" location="{app.config.location}/job1.properties"></util:properties>

<util:properties id="Job2Props"
    location="{app.config.location}/job2.properties"></util:properties>

And this is how I make separation between file and classpath while loading Application.
EDITS:

//pseudo code

if (StringUtils.isBlank(app.config.location)) {
            System.setProperty(APP_CONFIG_LOCATION, "classpath:");
}

I would really like not to use the above workaround and have Spring override all external config files on the classpath like it does for the application.properties file.

Solution 1

UPDATE: As the behaviour of spring.config.location now overrides the default instead of adding to it. You need to use spring.config.additional-location to keep the defaults. This is a change in behaviour from 1.x to 2.x

---

When using Spring Boot the properties are loaded in the following order (see Externalized Configuration in the Spring Boot reference guide).

1. Command line arguments.
2. Java System properties (System.getProperties()).
3. OS environment variables.
4. JNDI attributes from java:comp/env
5. A RandomValuePropertySource that only has properties in random.*.
6. Application properties outside of your packaged jar (application.properties including YAML and profile variants).
7. Application properties packaged inside your jar (application.properties including YAML and profile variants).
8. @PropertySource annotations on your @Configuration classes.
9. Default properties (specified using SpringApplication.setDefaultProperties).

When resolving properties (i.e. @Value("${myprop}") resolving is done in the reverse order (so starting with 9).

To add different files you can use the spring.config.location properties which takes a comma separated list of property files or file location (directories).

-Dspring.config.location=your/config/dir/

The one above will add a directory which will be consulted for application.properties files.

-Dspring.config.location=classpath:job1.properties,classpath:job2.properties

This will add the 2 properties file to the files that are loaded.

The default configuration files and locations are loaded before the additonally specified spring.config.location ones meaning that the latter will always override properties set in the earlier ones. (See also this section of the Spring Boot Reference Guide).

If spring.config.location contains directories (as opposed to files) they should end in / (and will be appended with the names generated from spring.config.name before being loaded). The default search path classpath:,classpath:/config,file:,file:config/ is always used, irrespective of the value of spring.config.location. In that way you can set up default values for your application in application.properties (or whatever other basename you choose with spring.config.name) and override it at runtime with a different file, keeping the defaults.

Solution 2

With Spring boot , the spring.config.location does work,just provide comma separated properties files.

see the below code

@PropertySource(ignoreResourceNotFound=true,value="classpath:jdbc-${spring.profiles.active}.properties")
public class DBConfig{

     @Value("${jdbc.host}")
        private String jdbcHostName;
     }
}

one can put the default version of jdbc.properties inside application. The external versions can be set lie this.

java -jar target/myapp.jar --spring.config.location=classpath:file:///C:/Apps/springtest/jdbc.properties,classpath:file:///C:/Apps/springtest/jdbc-dev.properties

Based on profile value set using spring.profiles.active property, the value of jdbc.host will be picked up. So when (on windows)

set spring.profiles.active=dev

jdbc.host will take value from jdbc-dev.properties.

for

set spring.profiles.active=default

jdbc.host will take value from jdbc.properties.

Solution 3

Spring boot 1.X and Spring Boot 2.X don't provide the same options and behavior about the Externalized Configuration.

The very good answer of M. Deinum refers to Spring Boot 1 specifities.
I will update for Spring Boot 2 here.

Environment properties sources and order

Spring Boot 2 uses a very particular PropertySource order that is designed to allow sensible overriding of values. Properties are considered in the following order:

• Devtools global settings properties on your home directory (~/.spring-boot-devtools.properties when devtools is active).

• @TestPropertySource annotations on your tests.

• @SpringBootTest#properties annotation attribute on your tests. Command line arguments.

• Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property).

• ServletConfig init parameters.

• ServletContext init parameters.

• JNDI attributes from java:comp/env.

• Java System properties (System.getProperties()).

• OS environment variables.

• A RandomValuePropertySource that has properties only in random.*.

• Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).

• Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).

• Application properties outside of your packaged jar (application.properties and YAML variants).

• Application properties packaged inside your jar (application.properties and YAML variants).

• @PropertySource annotations on your @Configuration classes. Default properties (specified by setting SpringApplication.setDefaultProperties).

To specify external properties files these options should interest you :

• Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).

• Application properties outside of your packaged jar (application.properties and YAML variants).

• @PropertySource annotations on your @Configuration classes. Default properties (specified by setting SpringApplication.setDefaultProperties).

You can use only one of these 3 options or combine them according to your requirements.
For example for very simple cases using only profile-specific properties is enough but in other cases you may want to use both profile-specific properties, default properties and @PropertySource.

Default locations for application.properties files

About application.properties files (and variant), by default Spring loads them and add their properties in the environment from these in the following order :

• A /config subdirectory of the current directory

• The current directory

• A classpath /config package

• The classpath root

The higher priorities are so literally :
classpath:/,classpath:/config/,file:./,file:./config/.

How to use properties files with specific names ?

The default locations are not always enough : the default locations like the default filename (application.properties) may not suit. Besides, as in the OP question you may need to specify multiple configuration files other than application.properties (and variant).
So spring.config.name will not be enough.

In this case you should provide an explicit location by using the spring.config.location environment property (which is a comma-separated list of directory locations or file paths).
To be free about the filenames pattern favor the list of file paths over the list of directories.
For example do like that :

java -jar myproject.jar --spring.config.location=classpath:/default.properties,classpath:/override.properties

That way is the most verbose that just specifying the folder but it is also the way to specify very finely our configuration files and to document clearly the properties effectively used.

spring.config.location now replaces default locations instead of adding to them

With Spring Boot 1, the spring.config.location argument adds specified locations in the Spring environment.
But from Spring Boot 2, spring.config.location replaces the default locations used by Spring by the specified locations in the Spring environment as stated in the documentation.

When custom config locations are configured by using spring.config.location, they replace the default locations. For example, if spring.config.location is configured with the value classpath:/custom-config/,file:./custom-config/, the search order becomes the following:

1. file:./custom-config/

2. classpath:custom-config/

spring.config.location is now a way to make sure that any application.properties file has to be explicitly specified.
For uber JARs that are not supposed to package application.properties files, that is rather nice.

To keep the old behavior of spring.config.location while using Spring Boot 2 you can use the new spring.config.additional-location property instead of spring.config.location that still adds the locations as stated by the documentation :

Alternatively, when custom config locations are configured by using spring.config.additional-location, they are used in addition to the default locations.

---

In practice

So supposing that as in the OP question, you have 2 external properties file to specify and 1 properties file included in the uber jar.

To use only configuration files you specified :

-Dspring.config.location=classpath:/job1.properties,classpath:/job2.properties,classpath:/applications.properties   

To add configuration files to these in the default locations :

-Dspring.config.additional-location=classpath:/job1.properties,classpath:/job2.properties

classpath:/applications.properties is in the last example not required as the default locations have that and that default locations are here not overwritten but extended.

Solution 4

Take a look at the PropertyPlaceholderConfigurer, I find it clearer to use than annotation.

e.g.

@Configuration
public class PropertiesConfiguration {


    @Bean
    public PropertyPlaceholderConfigurer properties() {
        final PropertyPlaceholderConfigurer ppc = new PropertyPlaceholderConfigurer();
//        ppc.setIgnoreUnresolvablePlaceholders(true);
        ppc.setIgnoreResourceNotFound(true);

        final List<Resource> resourceLst = new ArrayList<Resource>();

        resourceLst.add(new ClassPathResource("myapp_base.properties"));
        resourceLst.add(new FileSystemResource("/etc/myapp/overriding.propertie"));
        resourceLst.add(new ClassPathResource("myapp_test.properties"));
        resourceLst.add(new ClassPathResource("myapp_developer_overrides.properties")); // for Developer debugging.

        ppc.setLocations(resourceLst.toArray(new Resource[]{}));

        return ppc;
    }

Solution 5

this is one simple approach using spring boot

TestClass.java

@Configuration
@Profile("one")
@PropertySource("file:/{selected location}/app.properties")
public class TestClass {

    @Autowired
    Environment env;

    @Bean
    public boolean test() {
        System.out.println(env.getProperty("test.one"));
        return true;
    }
}

the app.properties context, in your selected location

test.one = 1234

your spring boot application

@SpringBootApplication

public class TestApplication {

    public static void main(String[] args) {
        SpringApplication.run(testApplication.class, args);
    }
}

and the predefined application.properties context

spring.profiles.active = one

you can write as many configuration class as you like and enable/disable them just by setting spring.profiles.active = the profile name/names {separated by commas}

as you can see spring boot is great it just needs sometime to get familiar with, it's worth mentioning that you may use @Value on your fields as well

@Value("${test.one}")
String str;

View more solutions

Comments

[M. Deinum]

The application.properties will always be loaded, with spring.config.location you can add additional configuration locations that are checked for files (that is when it ends with a /) however if you put a comma seperated list in there which points to files those will be loaded. This is also explained in the Spring Boot Reference Guide here

[nir]

Thanks but I have read this ref doc already and following is confusing to me "-Dspring.config.location=your/config/dir/ The one above will add a directory which will be consulted for application.properties files." What does it mean by application.properties files. That is only one file. In any case if it is able to pick up whole directory with "/" at end then I dont need to specify each as comma separated list. I think I have tried both approach as I mentioned in my post but I'll give it one more try

[M. Deinum]

As stated in the doc it will pick be consulted like the other default locations for the application.properties and application-[env].properties. It doesn't take into account other properties files. This is also stated in the reference guide (in the section the link leads to and the quote from the reference guide).

[nir]

Yes but that is what doesn't make sense to me.. why consider only one kind of file from a directory on classpath instead of whole directory. Its forces you to use one property file only which is not good imo. Like in tomcat I can configure common.loader to put particular directory (and everything insided it) on classpath why can't boot classloader can support it.

[M. Deinum]

Because that is how the Spring Boot guys made it work. If you want something else use a comma seperated list of files. Or specify another name (you can also override the default name application if you like). Next to that you can always use a @PropertySource annotation to load your properties files yourself.

[nir]

nice workaround. Like that java8 constructs! anyhow i cant use that as I need multiple Properties beans not just one. If you see my EDITS my workaround is pretty similar and neat for my use case.

[mxsb]

I posted a version for multiple files, just for completeness ;)

[eav]

This solved my problem after 4 hours of looking for why my tomcat sometimes imported the application.yml and sometimes not: If spring.config.location contains directories (as opposed to files) they should end in / (and will be appended with the names generated from spring.config.name before being loaded).

[Narfanator]

Quoting documentation isn't helpful. If the documentation was clear (enough? in the particularly needed way?) then the question wouldn't be necessary. For example, in this case, it's really not clear how config.location and config.names interact, although it probably appears to clear to people who already know how they interact. Can you update your answer to add something to the documentation?

[Xairoo]

This should be updated, as the behaviour of spring.config.location now overrides the default instead of adding to it. You need to use spring.config.additional-location to keep the defaults. This is a change in behaviour from 1.x to 2.x.

[Sowka]

I don't believe the first of the code blocks would work. I know as I stubbed myself on this one, and followed this answer. See jira.springsource.org/browse/SPR-8539 referenced in answer for decent explenation.

[Chetan]

Thank you very much for this answer. Can you please let me know how can I achieve the same in a project which has like XML configurations for different things with no base XML file? Your answer above did help me in other project which was annotation based. Thanks again for that.

[acaruci]

It will tell spring to use the "override" profile as your active profile; it would indeed overpass the value specified in the application.yml or application.properties file

[anakin59490]

I think I have similar problem . I have installed tomcat in opt folder. Where did you place your application file ? Should I change folder attributes too ?

[Tristan]

Your answer is really complete except on one thing : where will Spring find external configuration job1.properties on disk if you just specify : "classpath:/job1.properties" ? How did you add your directory containing external properties to the classpath here ?

[Mister_Jesus]

@Tristan, basically, spring can read one application.properties with all parameters and multiple ${file_name}.properties with partial defined sets of properties. So, if you use @PropertySource or other strong links to files, you can create other external file and override that properties (For ex: from classpath:file.properties).

[Ahmed Salem]

it will look inside the folder for any config file .ymal or .properties in my case I put only application-profile.yml then it takes correctly, Thanks @acaruci it was a nice trip

[Aakash Patel]

How can i get value by "${myprop}" for any controller level config?

[Poli]

By far simplest way to achieve properties separation. Thank you

[Rookie007]

I am able to inject values in @value only when I used @PropertyResource Can we inject value to @Value without specifying @PropertyResource as I have to use these field at lot of places, I dont want to use @PropertyResource all the time, is there way where I can switch .properties file in @Value annotation itself ?

[Pierre C]

Thanks for the update. This is exactly what I was looking for and it solves my problem.

[withoutOne]

Whats the vm parameter?