How to parse and convert ini file into bash array variables?

Question

I'm trying to convert an ini file into bash array variables. The sample ini is as below:

[foobar]
session=foo
path=/some/path

[barfoo]
session=bar
path=/some/path

so these become:

session[foobar]=foo
path[foobar]=/some/path
session[barfoo]=bar

and so on.

Right now, I could come up with only this command

awk -F'=' '{ if ($1 ~ /^\[/) section=$1; else if ($1 !~ /^$/) print $1 section "=" $2 }'

Also, another problem is, it doesn't take spaces near = into consideration. I think sed is probably better suited for this job but I don't know how to hold and store a temporary variable for the section name in sed.

So any idea how to do this?

Solution 1

Gawk accepts regular expressions as field delimiters. The following eliminates spaces around the equal sign, but preserves them in the rest of the line. Quotes are added around the value so those spaces, if any, are preserved when the Bash assignment is performed. I'm assuming that the section names will be numeric variables, but if you're using Bash 4, it would be easy to adapt this to use associative arrays with the section names themselves as the indices.

awk -F ' *= *' '{ if ($1 ~ /^\[/) section=$1; else if ($1 !~ /^$/) print $1 section "=" "\"" $2 "\"" }'

Note that you may want to also do the space removal that Khaled shows (on only $1 and section) since Bash variable names can't contain spaces.

Also, this method won't work if the values contain equal signs.

Another technique would be to use a Bash while read loop and perform the assignments as the file is read, using declare which is safe from most malicious content.

foobar=1
barfoo=2  # or you could increment an index variable each time a section is found
while IFS='= ' read var val
do
    if [[ $var == \[*] ]]
    then
        section=$var
    elif [[ $val ]]
    then
        declare "$var$section=$val"
    fi
done < filename

Again, associative arrays could fairly easily be supported.

Solution 2

I would use simple python script for this job since it has built in INI parser:

#!/usr/bin/env python

import sys, ConfigParser

config = ConfigParser.ConfigParser()
config.readfp(sys.stdin)

for sec in config.sections():
    print "declare -A %s" % (sec)
    for key, val in config.items(sec):
        print '%s[%s]="%s"' % (sec, key, val)

and then in bash:

#!/bin/bash

# load the in.ini INI file to current BASH - quoted to preserve line breaks
eval "$(cat in.ini  | ./ini2arr.py)"

# test it:
echo ${barfoo[session]}

Sure, there are shorter implementations in awk, but I think this is more readable and easier to maintain.

Solution 3

If you want to eliminate the extra spaces, you can use the built-in function gsub. For example, you can add:

gsub(/ /, "", $1);

This will remove all spaces. If you want to remove spaces at the beginning or end of token, you can use

gsub(/^ /, "", $1);
gsub(/ $/, "", $1);

Solution 4

Here's a pure bash solution.

This is a new and improved version of what chilladx posted:

https://github.com/albfan/bash-ini-parser

For a really easy to follow initial example: After you download this, just copy the files bash-ini-parser, and scripts/file.ini to the same directory, then create a client test script using the example I've provided below to that same directory as well.

source ./bash-ini-parser
cfg_parser "./file.ini"
cfg_section_sec2
echo "var2=$var2"
echo "var5[*]=${var5[*]}"
echo "var5[1]=${var5[1]}"

Here are some further improvements I made to the bash-ini-parser script...

If you want to be able to read ini files with Windows line endings as well as Unix, add this line to the cfg_parser function immediately following the one which reads the file:

ini=$(echo "$ini"|tr -d '\r') # remove carriage returns

If you want to read files which have restrictive access permissions, add this optional function:

# Enable the cfg_parser to read "locked" files
function sudo_cfg_parser {

    # Get the file argument
    file=$1

    # If not "root", enable the "sudo" prefix
    sudoPrefix=
    if [[ $EUID -ne 0 ]]; then sudoPrefix=sudo; fi

    # Save the file permissions, then "unlock" the file
    saved_permissions=$($sudoPrefix stat -c %a $file)
    $sudoPrefix chmod 777 $file

    # Call the standard cfg_parser function
    cfg_parser $file

    # Restore the original permissions
    $sudoPrefix chmod $saved_permissions $file  
}

View more solutions

Comments

[Flint]

If there's other efficient way to do this, feel free to post your solution too :)

[Admin]

Or you can use this ini parser in bash.

[kenorb]

For simple solution, check: How do I grab an INI value within a shell script? at stackoverflow SE.

[Flint]

Very nice info and I particularly like the second technique since it uses bash built in function, instead of relying on external command.

[Felix Eve]

In bash versions prior to 4.2 it is necessary to declare an associate array before filling it, e.g. print "declare -A %s" % (sec)

[Richlv]

Had to downvote because of chmod 777. While a shady practice at best, there's surely no need to make the ini file executable. A better approach would be to use sudo to read the file, not to mess with the permissions.

[BuvinJ]

@Richlv Ok. I do appreciate the down vote explanation. But, that's a tiny little part of this, which is of minimal significance as far as answering the question as a whole. The "answer" is the link: github.com/albfan/bash-ini-parser. Rather than down vote the entire thing, for what is already label an optional wrapper function, you could have suggested an edit.

[Dennis Williamson]

Instead of eval: source <(cat in.ini | ./ini2arr.py)

[Dennis Williamson]

@TonyBarganski: That can be modified into one AWK call instead of piping one into another.

[shadowbq]

Make sure you are using bash 4.x as bash 3.x does not support associative arrays.

[John Greene]

WOW! A full programming in awk. Impressive. How did you even debug that?

[CodingInTheUK]

The awk isn't my own, sorry I should have clarified. I don't recall where i got it, on here somewhere no doubt. The rest is my own.