Use ConfigParser to read an array from ini file

Question

I have read this post, and defined an array in subscriber.ini

[smtp]
subscriber[] = [email protected]
subscriber[] = [email protected]
subscriber[] = [email protected]

Then I try to use ConfigParser to read the array

#!/usr/bin/python
import ConfigParser

CONFIG_FILE = 'subscriber.ini'

config = ConfigParser.ConfigParser()
config.read( CONFIG_FILE )

subscriber = config.get('smtp' , 'subscriber[]' )
print subscriber 

It will output the last element, [email protected]. But I expect a full subscriber list.

How do I get the array from ini file ?

Solution 1

This syntax, where subscriber[] automatically makes subscriber into a list of multiple values, is not a feature of .ini files in general, nor of ConfigParser; it's a feature of Zend_Config_Ini.

In Python, a ConfigParser ini file creates a dict mapping each key to its value. If you have more than one value, it will just override previous values. The magic [] suffix means nothing.

However, the ConfigParser constructor lets you specify a custom dictionary type or factory, in place of the default OrderedDict.

One simple solution would be to use a defaultdict(list) (or an OrderedDefaultDict, which there are recipes for in the docs) for the underlying storage, have __setitem__(self, key, value) do self.dd[key].append(value), and delegate everything else normally. (Or, if you prefer, inherit from defaultdict, override the constructor to pass list to the super, and then just don't override anything but __setitem__.) That will make all of your values into lists.

You could even do something hacky where a value that's only seen once is a single value, but if you see the same name again it becomes a list. I think that would be a terrible idea (do you really want to check the type of config.get('smtp', 'subscriber[]') to decide whether or not you want to iterate over it?), but if you want to, How to ConfigParse a file keeping multiple values for identical keys? shows how.

However, it's not at all hard to reproduce the exact magic you're looking for, where all keys ending in [] are lists (whether they appear once or multiple times), and everything else works like normal (keeps only the last value if it appears multiple times). Something like this:

class MultiDict(collections.OrderedDict):
    def __setitem__(self, key, value):
        if key.endswith('[]'):
            super(MultiDict, self).setdefault(key, []).append(value)
        else:
            super(MultiDict, self).__setitem__(key, value)

---

This obviously won't provide all of the extended features that Zend_Config_Ini adds on top of normal .ini files. For example, [group : subgroup : subsub] won't have any special meaning as a group name, nor will key.subkey.subsub as a key name. PHP values TRUE, FALSE, yes, no, and NULL won't get converted to Python values True, False, True, False, and None. Numbers won't magically become numbers. (Actually, this isn't a feature of Zend_Config_Ini, but a misfeature of PHP's leaky typing.) You have to use # comments, rather than freely mixing #, ;, and //. And so on. Any of those features that you want to add, you'll have to add manually, just as you did this one.

As I suggested in a comment, if you really want to have more than two levels of hierarchy, you may be better off with a naturally infinitely-hierarchical format, where any value can be a list or dict of other values.

JSON is ubiquitous nowadays. It may not be quite as human-editable as INI, but I think more people are familiar with it than INI in 2014. And it has the huge advantage that it's a standardized format, and that both Python (2.6+) and PHP (5.2+) come with parsers and pretty-printers for in their standard libraries.

YAML is a more flexible and human-editable format. But you will need third-party modules in both languages (see the list at the YAML site). And it can also bring in some security concerns if you're not careful. (See safe_load and friends in the PyYAML docs; most other libraries have similar features.)

Solution 2

Python ConfigParser doesn't provide this feature. Use following instead:

[smtp]
subscriber = [email protected] [email protected] [email protected]

Then in your script:

subscriber = config.get('smtp' , 'subscriber').split()

Solution 3

I wonder if you have considered using Michael Foord's configobj module? It seems to be capable of doing what you want, which might be better that trying to pervert ConfigParser to do what you apparently need.

Comments

[abarnert]

You read a question for PHP Zend_Config_Ini and expected it to automatically work for a completely different language and library?

[Brightshine]

May I seek a better format to represent array in .ini file? I hope it can work with both php and python.

[abarnert]

On further reading, I don't think this is a dup. The format that user wanted (where a key is a list iff it appears two or more times) is not what this user seems to want, and not what anyone should want. And there's a much simpler may to make this work just like (I'm pretty sure) Zend_Config_Ini works. See my answer.

[abarnert]

@Brightshine: If you specifically want to borrow this feature from Zend, you can, but you have to do it explicitly, as in my answer. It might be simpler to add your own simple syntax on top of the .ini syntax, as shown in Dmitry's answer, which of course you'd have to parse explicitly in both languages, but it should be dead-simple to parse.

[abarnert]

@Brightshine: Alternatively, have you considered using a recursively-structured format like JSON (which Python 2.6+ and PHP 5.2+ have stdlib parsers for) or YAML instead of a fixed-two-level format like INI?

[abarnert]

It's worth pointing out that Zend won't automatically parse this any more than Python, so if he wants to share config files (which he apparently does) he'll have to also do the equivalent in PHP. But it's just as trivial in PHP as in Python, so he should have no trouble doing that.

[abarnert]

I think this is more of a comment than an answer. Also, given that the OP wants to share config files between PHP and Python… is there a PHP parser for the configobj format?

[Brightshine]

Thanks, I think using JSON is a good idea to me !

[user1633272]

This seems to be good way to handle array in ini, some other languages do handle it this way.