how to reference a YAML "setting" from elsewhere in the same YAML file?

Question

I have the following YAML:

paths:
  patha: /path/to/root/a
  pathb: /path/to/root/b
  pathc: /path/to/root/c

How can I "normalise" this, by removing /path/to/root/ from the three paths, and have it as its own setting, something like:

paths:
  root: /path/to/root/
  patha: *root* + a
  pathb: *root* + b
  pathc: *root* + c

Obviously that's invalid, I just made it up. What's the real syntax? Can it be done?

Solution 1

I don't think it is possible. You can reuse "node" but not part of it.

bill-to: &id001
    given  : Chris
    family : Dumars
ship-to: *id001

This is perfectly valid YAML and fields given and family are reused in ship-to block. You can reuse a scalar node the same way but there's no way you can change what's inside and add that last part of a path to it from inside YAML.

If repetition bother you that much I suggest to make your application aware of root property and add it to every path that looks relative not absolute.

Solution 2

Yes, using custom tags. Example in Python, making the !join tag join strings in an array:

import yaml

## define custom tag handler
def join(loader, node):
    seq = loader.construct_sequence(node)
    return ''.join([str(i) for i in seq])

## register the tag handler
yaml.add_constructor('!join', join)

## using your sample data
yaml.load("""
paths:
    root: &BASE /path/to/root/
    patha: !join [*BASE, a]
    pathb: !join [*BASE, b]
    pathc: !join [*BASE, c]
""")

Which results in:

{
    'paths': {
        'patha': '/path/to/root/a',
        'pathb': '/path/to/root/b',
        'pathc': '/path/to/root/c',
        'root': '/path/to/root/'
     }
}

The array of arguments to !join can have any number of elements of any data type, as long as they can be converted to string, so !join [*a, "/", *b, "/", *c] does what you would expect.

Solution 3

Another way to look at this is to simply use another field.

paths:
  root_path: &root
     val: /path/to/root/
  patha: &a
    root_path: *root
    rel_path: a
  pathb: &b
    root_path: *root
    rel_path: b
  pathc: &c
    root_path: *root
    rel_path: c

Solution 4

I've create a library, available on Packagist, that performs this function: https://packagist.org/packages/grasmash/yaml-expander

Example YAML file:

type: book
book:
  title: Dune
  author: Frank Herbert
  copyright: ${book.author} 1965
  protaganist: ${characters.0.name}
  media:
    - hardcover
characters:
  - name: Paul Atreides
    occupation: Kwisatz Haderach
    aliases:
      - Usul
      - Muad'Dib
      - The Preacher
  - name: Duncan Idaho
    occupation: Swordmaster
summary: ${book.title} by ${book.author}
product-name: ${${type}.title}

Example logic:

// Parse a yaml string directly, expanding internal property references.
$yaml_string = file_get_contents("dune.yml");
$expanded = \Grasmash\YamlExpander\Expander::parse($yaml_string);
print_r($expanded);

Resultant array:

array (
  'type' => 'book',
  'book' => 
  array (
    'title' => 'Dune',
    'author' => 'Frank Herbert',
    'copyright' => 'Frank Herbert 1965',
    'protaganist' => 'Paul Atreides',
    'media' => 
    array (
      0 => 'hardcover',
    ),
  ),
  'characters' => 
  array (
    0 => 
    array (
      'name' => 'Paul Atreides',
      'occupation' => 'Kwisatz Haderach',
      'aliases' => 
      array (
        0 => 'Usul',
        1 => 'Muad\'Dib',
        2 => 'The Preacher',
      ),
    ),
    1 => 
    array (
      'name' => 'Duncan Idaho',
      'occupation' => 'Swordmaster',
    ),
  ),
  'summary' => 'Dune by Frank Herbert',
);

Solution 5

YML definition:

dir:
  default: /home/data/in/
  proj1: ${dir.default}p1
  proj2: ${dir.default}p2
  proj3: ${dir.default}p3 

Somewhere in thymeleaf

<p th:utext='${@environment.getProperty("dir.default")}' />
<p th:utext='${@environment.getProperty("dir.proj1")}' /> 

Output: /home/data/in/ /home/data/in/p1

View more solutions

Comments

[dreftymac]

See also: stackoverflow.com/a/41620747/42223

[Eugene Gr. Philippov]

See also: The YAML specs: yaml.org/spec

[Andrew Bullock]

Ok thanks, yeah ill have to prepend the root in code. no biggie.

[Chris Johnson]

The accepted answer is not accurate. See my answer for a solution.

[Prateek Jain]

how to do this, if bill-to is in another file, which we have imported where ship-to is defined ?

[Anthon]

I like your solution, simpler in coding then mine at the cost of slightly less readable YAML.

[Anthon]

The highlight over the down-arrow says "this answer is not useful" and that is essentially the case for anyone not interested in the subject of YAML. Even worse nobody (apart from me, because I was the only commenter so far) will get a notification of your comment/question. Chances that someone who downvoted comes back here and reads your comment and answers are so low that I would consider those close to non-existent. Having said that, it wasn't me, and if I were you I wouldn't ask and delete the comment.

[Adrián González]

Thanks! This works even to create a list with previously defined elements

[dreftymac]

This answer deserves more up-votes. It is technically the most accurate answer pursuant to the YAML specification. There is one caveat, however, pursuant to actual YAML implementations, there are few that actually implement the full YAML spec. Python's pyyaml is above and beyond many others in terms of its uniformity with the specification.

[dreftymac]

@PrateekJain: if you are dealing with multiple files, you will probably do best to evaluate a standalone YAML-enhancement library, such as one listed here. github.com/dreftymac/dynamic.yaml/blob/master/…

[akostadinov]

See example 2.9 in yaml.org/spec/1.2/spec.html ; one can also reference scalars which is awesome

[user2020056]

The question seems to be about referencing a value IN a yaml file. Adding another layer of code around it would not be my preferred solution.

[Chris Johnson]

There is no other way.

[Honza Zidek]

@AndrewBullock I think this should be the accepted answer, as it exactly solves your issue.

[user5359531]

@ChrisJohnson Thanks for this answer, I was wondering if you had a reference document that listed this syntax. I've seen YAML spec explained in multiple places on the web so I just want to make sure I'm looking at the same reference you are. Thanks!

[Arthur Lacoste]

No, it is not a native usage of variable in YAML and it's not specified in any specification versions. After some test, this doesn't work.

[Andrew Wynham]

This probably worked for Pavol using something that pre-processed the yaml (i.e. maven-resources-plugin filtering)

[benjaminmgross]

This solution didn't work for me (python3?) however with a simple modification to the above it works as expected. Specifically: yaml.SafeLoader.add_constructor(tag='!join', constructor=join) yaml.load(open(fpth, mode='r'), Loader=yaml.SafeLoader)

[Dan Niero]

Not standard Yaml

[Dhiraj]

It's good to be aware of a library that adds this functionality on top of YAML

[dbaumann]

Thanks for making this. YAML is all the rage these days and it's nice to see a project to help contain the madness.