Go parse yaml file

Question

I'm trying to parse a yaml file with Go. Unfortunately I can't figure out how. The yaml file I have is this:

---
firewall_network_rules:
  rule1:
    src:       blablabla-host
    dst:       blabla-hostname
...

I have this Go code, but it does not work:

package main

import (
    "fmt"
    "io/ioutil"
    "path/filepath"

    "gopkg.in/yaml.v2"
)

type Config struct {
    Firewall_network_rules map[string][]string
}

func main() {
    filename, _ := filepath.Abs("./fruits.yml")
    yamlFile, err := ioutil.ReadFile(filename)

    if err != nil {
        panic(err)
    }

    var config Config

    err = yaml.Unmarshal(yamlFile, &config)
    if err != nil {
        panic(err)
    }

    fmt.Printf("Value: %#v\n", config.Firewall_network_rules)
}

When I run this, I get an error. I think it's because I haven't created a struct for the src and dst key/values. FYI: when I change that to a list, it works.

So above code parses this:

---
firewall_network_rules:
  rule1:
    - value1
    - value2
...

Solution 1

If you're working with google cloud or kubernetes more specifically and want to parse a service.yaml like this:

apiVersion: v1
kind: Service
metadata:
  name: myName
  namespace: default
  labels:
    router.deis.io/routable: "true"
  annotations:
    router.deis.io/domains: ""
spec:
  type: NodePort
  selector:
    app: myName
  ports:
    - name: http
      port: 80
      targetPort: 80
    - name: https
      port: 443
      targetPort: 443

Supplying a real world example so you get the hang of how nesting can be written.

type Service struct {
    APIVersion string `yaml:"apiVersion"`
    Kind       string `yaml:"kind"`
    Metadata   struct {
        Name      string `yaml:"name"`
        Namespace string `yaml:"namespace"`
        Labels    struct {
            RouterDeisIoRoutable string `yaml:"router.deis.io/routable"`
        } `yaml:"labels"`
        Annotations struct {
            RouterDeisIoDomains string `yaml:"router.deis.io/domains"`
        } `yaml:"annotations"`
    } `yaml:"metadata"`
    Spec struct {
        Type     string `yaml:"type"`
        Selector struct {
            App string `yaml:"app"`
        } `yaml:"selector"`
        Ports []struct {
            Name       string `yaml:"name"`
            Port       int    `yaml:"port"`
            TargetPort int    `yaml:"targetPort"`
            NodePort   int    `yaml:"nodePort,omitempty"`
        } `yaml:"ports"`
    } `yaml:"spec"`
}

There's a convenient service called yaml-to-go https://zhwt.github.io/yaml-to-go/ which converts YAML to go structs, just input your YAML into that service and you get an autogenerated struct.

A JSON equivalent exists aswell: https://mholt.github.io/json-to-go/

And last unmarshal as a previous poster wrote:

var service Service

err = yaml.Unmarshal(yourFile, &service)
if err != nil {
    panic(err)
}

fmt.Print(service.Metadata.Name)

Solution 2

Well, I think I've figured it out by myself. The following piece of code works fine. Any suggestions/improvements?

package main

import (
    "fmt"
    "io/ioutil"
    "path/filepath"

    "gopkg.in/yaml.v2"
)

type Config struct {
    Firewall_network_rules map[string]Options
}

type Options struct {
    Src string
    Dst string
}

func main() {
    filename, _ := filepath.Abs("./fruits.yml")
    yamlFile, err := ioutil.ReadFile(filename)

    if err != nil {
        panic(err)
    }

    var config Config

    err = yaml.Unmarshal(yamlFile, &config)
    if err != nil {
        panic(err)
    }

    fmt.Printf("Value: %#v\n", config.Firewall_network_rules)
}

Solution 3

Why not organize your yaml file like below if you don't care about the rule name?

---
firewall_network_rules:
  - 
    name:      rule1
    src:       blablabla-host
    dst:       blabla-hostname
  - 
    name:      rule2
    src:       bla-host
    dst:       bla-hostname

So the code will be like this, it is clean and extensible:

type Rule struct {
    Name  string  `yaml:"name"`
    Src   string  `yaml:"src"`
    Dst   string  `yaml:"dst"`
}

type Config struct {
   FirewallNetworkRules []Rule  `yaml:"firewall_network_rules"`
}

Solution 4

If your YAML file is simple (single nesting) like following

mongo:
    DB: database
    COL: collection
log:
    error: log/error/error.log
api:
    key: jhgwewbcjwefwjfg

Here, you can use interface instead of declaring struct.

main(){
  config := Config()
  mongoConfig := config["mongo"]

  mongo.MongoDial(
    String(
        Get(mongoConfig, "DB")
    ), 
    String(
        Get(mongoConfig, "COL")
    )
  )
}

func Config() map[string]interface{} {
    filename, _ := filepath.Abs("configs/config.yaml")
    yamlFile, err := ioutil.ReadFile(filename)

    if err != nil {
        panic(err)
    }

    var config map[string]interface{}

    err = yaml.Unmarshal(yamlFile, &config)
    if err != nil {
        panic(err)
    }

    return config
}
func Get(this interface{}, key string) interface{}  {
    return this.(map[interface{}]interface{})[key]
}
func String(payload interface{}) string  {
    var load string
    if pay, oh := payload.(string); oh {
        load = pay
    }else{
        load = ""
    }
    return load
}

This works fine for level 1 nesting, if you have complex nesting then it is recommended to use struct.

View more solutions

Comments

[elithrar]

Try the more idiomatic FirewallNetworkRules and add a struct tag to capture the YAML formatting - e.g. ` yaml:"firewall_network_rules" ` See here for the docs on struct tag usage in the YAML lib: godoc.org/gopkg.in/yaml.v2#Marshal

[Rogier Lommers]

Thanks for your suggestion, it indeed clarifies my code.