What is the best method for setting up a config file in Python

Question

I realise this question as been asked before (What's the best practice using a settings file in Python?) but seeing as this was asked 7 years ago, I feel it is valid to discuss again seeing as how technologies have evolved.

I have a python project that requires different configurations to be used based on the value of an environment variable. Since making use of the environment variable to choose a config file is simple enough, my question is as follows:

What format is seen as the best practice in the software industry for setting up a configuration file in python, when multiple configurations are needed based on the environment?

I realise that python comes with a ConfigParser module but I was wondering if it might be better to use a format such as YAML or JSON because of there raise in popularity due to their ease of use across languages. Which format is seen as easier to maintain when you have multiple configurations?

Solution 1

If you really want to use an environment-based YAML configuration, you could do so like this:

config.py

import yaml
import os

config = None

filename = getenv('env', 'default').lower()
script_dir = os.path.dirname(__file__)
abs_file_path = os.path.join(script_dir, filename)
with open(abs_file_path, 'r') as stream:
    try:
        config = yaml.load(stream)
    except yaml.YAMLError as exc:
        print(exc)

Solution 2

I think looking at the standard configuration for a Python Django settings module is a good example of this, since the Python Django web framework is extremely popular for commercial projects and therefore is representative of the software industry.

It doesn't get too fancy with JSON or YAML config files - It simply uses a python module called settings.py that can be imported into any other module that needs to access the settings. Environment variable based settings are also defined there. Here is a link to an example settings.py file for Django on Github:

https://github.com/deis/example-python-django/blob/master/helloworld/settings.py

Solution 3

That really depends on your requirements, rather than the format's popularity. For instance, if you just need simple key-value pairs, an INI file would be more than enough. As soon as you need complex structures (e.g., arrays or dictionaries), I'd go for JSON or YAML. JSON simply stores data (it's more intended for automated data flow between systems), while YAML is better for human-generated (or maintained, or read) files, as it has comments, you can reference values elsewhere in the file... And on top of that, if you want robustness, flexibility, and means to check the correct structure of the file (but don't care much about the manual edition of the data), I'd go for XML.

Solution 4

I recommend giving trapdoor a try for turn-key configuration (disclaimer: I'm the author of trapdoor).

View more solutions