Golang nested Yaml values

Question

I am trying to access Yaml files and grab individual values, but am struggling with the Struct syntax to achieve this. The code below processes the Yaml and I can print the full Struct, but how can I access the individual ecs.services.name attribute?

Any recommendations on how to handle this is welcome as I have come across several Yaml libraries but haven't been able to get any of them to fully work.

test.yaml:

ecs:
  services:
    - name: my-service
      taskDefinition: my-task-def
      desiredCount: 1

Yaml.go

package main

import (
    "fmt"
    "io/ioutil"
    "path/filepath"

    "gopkg.in/yaml.v2"
)

type Config struct {
    //Ecs []map[string]string this works for ecs with name
    Ecs struct {
        Services []struct {
            Name           string
            TaskDefinition string
            DesiredCount   int
        }
    }
    //Services []map[string][]string
}

func main() {
    filename, _ := filepath.Abs("test.yaml")

    yamlFile, err := ioutil.ReadFile(filename)
    check(err)

    var config Config

    err = yaml.Unmarshal(yamlFile, &config)
    check(err)

    fmt.Printf("Description: %#v\n", config.Ecs.Services)
}

func check(e error) {
    if e != nil {
        panic(e)
    }
}

Output

$ go run yaml.go
Description: []struct { Name string; TaskDefinition string; DesiredCount int }{struct { Name string; TaskDefinition string; DesiredCount int }{Name:"my-service", TaskDefinition:"", DesiredCount:0}}

Solution 1

I had a similar requirement where on a yaml file i was needed to perform nested retrievals. As i found no out of the box solution i had to write it myself.

I have a yaml file having content like below

"a": "Easy!"
"b":
  "c": "2"
  "d": ["3", "4"]
"e":
  "f": {"g":"hi","h":"6"}

I wanted to access and print nested values from this structure and the output should be like below

--- yaml->a: Easy!
--- yaml->b->c: 2
--- yaml->b->x: None  //not existing in the yaml
--- yaml->y->w: None  //not existing in the yaml
--- yaml->b->d[0]: 3   //accessing value from a list
--- yaml->e->f->g: hi 

I also did not want to define a structure to hold the parsed yaml. The most generic structure in golang is interface{}. The most suitable structure to unmarshall the yaml is map[interface{}]interface{}. For folks coming from java this is akin to Map<Object,Object>. Once the data is unmarshalled i had to write a function which can traverse the structure using nested keys and return the value.

Below is the code to do it. Turn on the comments and execute to know how the code traverses nested structure and finally gets the value. Though this example assumes all the values in the yaml are string this can be extended for numerical keys and values as well.

package main

import (
    "fmt"
    "gopkg.in/yaml.v2"
    "io/ioutil"
    "reflect"
)

func main() {

    testFile := "test.yaml"
    testYaml, rerr := ioutil.ReadFile(testFile)
    if rerr != nil {
        fmt.Errorf("error reading yaml file: %v", rerr)
    }

    m := make(map[interface{}]interface{})
    if uerr := yaml.Unmarshal([]byte(testYaml), &m); uerr != nil {
        fmt.Errorf("error parsing yaml file: %v", uerr)
    }

    fmt.Printf("--- yaml->a: %v\n\n", getValue(m, []string{"a"}, -1))         //single value in a map
    fmt.Printf("--- yaml->b->c: %v\n\n", getValue(m, []string{"b", "c"}, -1)) //single value in a nested map
    fmt.Printf("--- yaml->b->x: %v\n\n", getValue(m, []string{"b", "x"}, -1)) //value for a non existent nest key
    fmt.Printf("--- yaml->y->w: %v\n\n", getValue(m, []string{"y", "w"}, -1)) //value for a non existent nest key
    fmt.Printf("--- yaml->b->d[0]: %v\n\n", getValue(m, []string{"b", "d"}, 0))
    fmt.Printf("--- yaml->e->f->g: %v\n\n", getValue(m, []string{"e", "f", "g"}, -1))
}

func getValue(obj map[interface{}]interface{}, keys []string, indexOfElementInArray int) string {

    //fmt.Printf("--- Root object:\n%v\n\n", obj)
    value := "None"
    queryObj := obj
    for i := range keys {
        if queryObj == nil {
            break
        }
        if i == len(keys)-1 {
            break
        }
        key := keys[i]
        //fmt.Printf("--- querying for sub object keyed by %v\n", key)
        if queryObj[key] != nil {
            queryObj = queryObj[key].(map[interface{}]interface{})
            //fmt.Printf("--- Sub object keyed by %v :\n%v\n\n", key, queryObj)
        } else {
            //fmt.Printf("--- No sub object keyed by %v :\n%v\n\n", key)
            break
        }
    }
    if queryObj != nil {
        lastKey := keys[len(keys)-1]
        //fmt.Printf("--- querying for value keyed by %v\n", lastKey)

        if queryObj[lastKey] != nil {
            objType := reflect.TypeOf(queryObj[lastKey])
            //fmt.Printf("Type of value %v\n", objType)
            if objType.String() == "[]interface {}" {
                //fmt.Printf("Object is a array %v\n", objType)
                tempArr := queryObj[lastKey].([]interface{})
                //fmt.Printf("Length of array is %v\n", len(tempArr))
                if indexOfElementInArray >= 0 && indexOfElementInArray < len(tempArr) {
                    value = queryObj[lastKey].([]interface{})[indexOfElementInArray].(string)
                }
            } else {
                value = queryObj[lastKey].(string)
            }
        }
    }

    return value
}

Solution 2

I think you have an extra level of nesting. The Config struct may not be required. Could you try the following definitions:

type Ecs struct {
    Services []Service
}

type Service struct {
    Name           string
    TaskDefinition string
    DesiredCount   int
}

And then try to unmarshal the yaml data. You can then perhaps access the data as ecs.Services.

Solution 3

You should use struct tags since you name the fields with lower case letters. This applies to different named fields as well.

See the example for how to fix this: https://play.golang.org/p/WMmlQsqYeB the other answer is incorrect.

Comments

[meh]

My use case was go templates. Unmarshalling yaml into map[interface{}]interface{} is working so far (I've been using it for a full 30 seconds).