Loading multiple YAML files (using @ConfigurationProperties?)

Question

Using Spring Boot 1.3.0.RELEASE

I have a couple of yaml files that describe several instances of a program. I now want to parse all those files into a List<Program> (Map, whatever), so I can later on search for the most appropriate instance for a given criteria in all the programs.

I like the approach with @ConfigurationProperties a lot, and it works good enough for a single yaml-file, but I haven't found a way yet to read all files in a directory using that method.

Current approach working for a single file:

programs/program1.yml

name: Program 1
minDays: 4
maxDays: 6

can be read by

@Configuration
@ConfigurationProperties(locations = "classpath:programs/program1.yml", ignoreUnknownFields = false)
public class ProgramProperties {

private Program test; //Program is a POJO with all the fields in the yml.
//getters+setters

I tried changing the locations to an Array listing all of my files locations = {"classpath:programs/program1.yml", "classpath:programs/program2.yml"} as well as using locations = "classpath:programs/*.yml", but that still only loads the first file (array-approach) or nothing at all (wildcard-approach).

So, my question is, what is the best way in Spring Boot to load a bunch of yaml files in a classpath-directory and parse them into a (List of) POJO, so they can be autowired in a Controller? Do I need to use Snakeyaml directly, or is there an integrated mechanism that I just haven't found yet?

EDIT: A working approach is doing it manually:

    private static final Yaml yaml = new Yaml(new Constructor(Program.class));
private static final ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();

try {
        for (Resource resource : resolver.getResources("/programs/*.yml")) {

            Object data = yaml.load(resource.getInputStream());

            programList.add((Program) data);
        }
    }
    catch (IOException ioe) {
        logger.error("failed to load resource", ioe);
    }

Solution 1

In Spring, it is possible to load multiple configuration properties files using PropertySource annotation, but not YAML files. See section 26.6.4 in link below:

https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html#boot-features-external-config-typesafe-configuration-properties

However, from your problem, it seems that you can configure all your programs in single YAML and then get all list of programs in a single list.

Sample YAML (all.yaml)

programs:
  - name: A
    min: 1
    max: 2
  - name: B
    min: 3
    max: 4

Config.java

@Configuration
@ConfigurationProperties(locations={"classpath:all.yaml"})
public class Config{

    private List<Program> programs;

    public void setPrograms(List<Program> programs) {
        this.programs = programs;
    }

    public List<Program> getPrograms() {
        return programs;
    }
}

Solution 2

What I am currently doing, as far as I understood your question, is nearly the same. I am having an application.yml and also profile-specific yml files, e.g. application-{profile}.yml in my src/main/resources. In the application.yml I have defined the default profile key-values, which are partially overridden by the profile-specific yml files.

If you want to have a type-safe and well defined access of your YML key/values, then you can use the following approach:

 @ConfigurationProperties
 public class AppSettings {
     String name; // has to be the same as the key in your yml file

     // setters/getters

 }

In your Spring-Boot config, you have to add the following annotations onto your config class:

@ComponentScan
@EnableAutoConfiguration
@EnableConfigurationProperties( value = { AppSettings.class, SomeOtherSettings.class } )
public class SpringContextConfig {

     @Autowired
     private AppSettings appSettings;

     public void test() {
          System.out.println(appSettings.getName());
     }
}

The @Autowiring is also accessible from other Beans. The other way around (without an extra separated and type-safe class, is to access the YML-values via @Value("${name}").

To bring it together in a short manner: Yes, it is possible to use several YAML files for your application via Spring-profiles. You define your current active spring profile via command args, programmatically or via your system env (SPRING_PROFILES_ACTIVE=name1,name2). Therefore you can have several application.yml files for each profile (see above).

Comments

[Alex]

thanks mohit, this definitely works and was my first approach as well, but I wanted to go the separate file route as a single program can get quite big and the resulting "all.yaml" would be a horror to maintain...

[Mohit]

Well, in such case, you can use @ConfigurationProperties annotation on a method and then load the configuration by writing your own code. This approach is more of what you included in question with additional advantage of auto-wiring the List<Programs>.

[Peter Kirby]

Just and FYI: the locations argument in @ConfigurationProperties was depreciated in 1.4 "in favor of configuring the environment directly with additional locations".

[ampofila]

I do not think this was a question regarding how to name different profile application.yml files.