File based configuration handling in C (Unix)

Question

This is probably one of the most common tasks / problems when programming; You need to store the configuration of your application somewhere.

While I'm trying to create a webserver or other applications, I'd like to keep the code as clean as possible since my main interest in programming is architecture. This results in me wanting to store configurations in a file which can be changed without having to re-compile the software.

I'm not here to re-invent the wheel or anything like that, so what I'd like to do is creating a Configuration reader in C on *nix. The configuration might look a lot like any other software's configuration; Apache, vsftpd, MySQL, etc.

The basic question is: How do you read from a textfile and process each line efficiently (in pure C)? Do I need to use fgetc() and process each char?

Solution 1

Okay, so let's hit the other part. You need to think about what you'd like to have as your "language". In the UNIX world, the sort of canonical version is probably whitespace-delimited text (think /etc/hosts) or ":" delimited text (like /etc/passwd).

You have a couple of options, the simplest in some sense being to use scanf(3). Again, read the man page for details, but if a line entry is something like

port    100

then you'll be looking for something like

char inbuf[MAXLINE];
int  val;

scanf("%s %d\n", &inbuf[0], &val);

You can get a bit more flexibility if you write a simple FSA parse: read characters one at a time from the line, and use a finite automaton to define what to do.

Solution 2

Hmmm there is LibConfig.

I have found it in Wiki

Solution 3

Various people have given reasonably good advice - the Pure-FTP example is interesting.

You should also read TAOUP (The Art of Unix Programming) by E S Raymond. It has examples a-plenty of configuration files. It also outlines canonical idioms for the configuration files. For example, you should probably allow '#' to start a comment to the end of the line, and ignore blank lines. You should also decide what you will do if the configuration file contains a line you don't understand - whether to ignore and continue silently, or whether to complain. (I prefer things that complain; then I know why what I've just added isn't having any effect - whereas silent ignoring means I don't know whether the entry I just added has any effect.)

Another problem is locating the configuration file. Do you do that by compiled-in location, by a default install location with environment variable to override, or by some other piece of magic? Make sure there's a command line option to allow the configuration file to be specified absolutely - even consider making that the only way to do it.

Otherwise, within broad limits, keep it simple and everyone will be happier.

Solution 4

Okay, here's a real example of C code:

/* demo-fgets -- read a "demo.text", copy to stdout with line
   numbers. */

#include <stdio.h>
#include <stdlib.h>

#define MAXLINE 100

FILE * fp;
char bufr[MAXLINE];

extern int errno ;

int main(int argc, char ** argv){
    int count = 0 ;
    if((fp = fopen("demo.text","r")) != NULL){
        /* then file opened successfully. */
        while(fgets(bufr,MAXLINE,fp)!=NULL){
            /* then no read error */
            count +=1;
            printf("%d: %s",     /* no "\n", why? */
                   count, bufr);
        }
        /* fgets returned null */
        if(errno != 0){
            perror(argv[0]);    /* argv[0]?  Why that? */
            exit(1);
        }
        exit(0);                /* EOF found, normal exit */
    } else {                    /* there was an error on open */
        perror(argv[0]);
        exit(1);
    }
}

I run it with this input file:

520 $ cat demo.text 
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum
aliquet augue id quam. Sed a metus. Quisque sit amet quam. Sed id
ante. In egestas est non mi. Sed vel velit non elit vehicula
viverra. Curabitur eget tortor in ipsum vulputate
faucibus. Suspendisse imperdiet mauris at nibh. Sed interdum. Maecenas
vulputate, massa vel placerat mattis, ante est tincidunt sem, in
sollicitudin velit lacus non tortor. Etiam sagittis consequat nisl. 

Vestibulum id leo quis mauris gravida placerat. Donec aliquet justo a
tortor. Etiam nisi nibh, auctor non, luctus et, aliquam vitae,
metus. Cum sociis natoque penatibus et magnis dis parturient montes,
nascetur ridiculus mus. Nunc lacinia quam a ligula. Nulla quis nisi eu
nunc imperdiet cursus. Nunc vitae nisi vitae tellus posuere
sollicitudin. Nunc suscipit, dui ac interdum euismod, pede nisl varius
dui, sed mattis libero mauris eu felis. Nam mattis dui eget
nunc. Suspendisse malesuada, pede eget posuere pellentesque, neque
eros pretium nibh, ut blandit dui leo dapibus orci. Etiam lacinia
lectus at orci. Donec ligula lacus, sagittis nec, sodales et,
fringilla lobortis, eros. Etiam sit amet nulla. Aliquam mollis pede id
enim. Etiam ligula felis, pulvinar nec, vestibulum molestie, interdum
ut, urna. Ut porta ullamcorper diam. Nullam interdum arcu. 

Pellentesque habitant morbi tristique senectus et netus et malesuada
fames ac turpis egestas. Etiam eu enim quis sem accumsan
tristique. Proin non sem. Etiam quis ante. Aenean ornare pellentesque
dolor. Praesent sodales. Cras dui velit, scelerisque a, accumsan a,
vestibulum in, dui. Pellentesque sed sapien. Etiam augue est,
convallis eget, egestas vel, molestie id, turpis. Cum sociis natoque
penatibus et magnis dis parturient montes, nascetur ridiculus
mus. Cras posuere lorem eu diam. Ut ultricies velit. Nunc imperdiet
suscipit mauris. Vestibulum molestie elit id risus. Phasellus et
purus. Vestibulum id mauris. Fusce gravida elit quis turpis. Aliquam
ut est. 

Sed in mauris eu nulla rhoncus suscipit. Nam id dolor sit amet turpis
placerat sodales. Nunc ipsum. Quisque diam tellus, dapibus non,
interdum at, aliquam sit amet, tellus. Donec non pede eget massa
aliquam semper. Quisque dictum lacinia ipsum. Fusce magna purus,
mattis id, commodo et, lobortis eu, arcu. Vestibulum viverra neque a
nulla. Cum sociis natoque penatibus et magnis dis parturient montes,
nascetur ridiculus mus. Pellentesque vel felis in ligula blandit
auctor. Quisque quam. Curabitur turpis. Morbi molestie augue a
nisi. Nulla sollicitudin sagittis elit. Suspendisse in odio sed magna
dictum vestibulum. Duis facilisis lorem eget neque. Proin sit amet
urna eget velit scelerisque aliquam. Pellentesque imperdiet. Nullam
sapien. Nullam placerat ipsum eget metus. 

Mauris ornare risus eu velit. Morbi bibendum diam in sem. Morbi
aliquet nisl sit amet quam. Donec ornare sagittis nibh. Fusce ac
lectus. Sed sit amet risus. Integer facilisis commodo
sem. Pellentesque facilisis. Donec libero. Lorem ipsum dolor sit amet,
consectetur adipiscing elit.

Solution 5

There are a number of ways. You don't need to use fgetc. You should probably read the stdio man page, but the canonical thing would be to open the file with fopen(3), then read using fgets(3) to read a line at a time. That would look something like:

#include <stdio.h>

FILE * fp ;
char bufr[MAXLINE];

if((fp = fopen(filename, "r") != NULL){
    while(! feof(fp)){
         fgets(bufr, MAXLINE, fp);
         /* Do stuff */
    }
} else {
    /* error processing, couldn't open file */
}

You could also look at libini at Sourceforge.

View more solutions

Comments

[Charlie Martin]

XML. Ick. That's a LOT of overhead for a simple ini file.

[Filip Ekberg]

Indeed my thought. And it needs to be pure C and not use any external libraries. So i want to learn how to do this :)

[Filip Ekberg]

Ah i see, so it's like using fgets in ASM ( which i've worked with before ). Was a little curious if it was possible to use it with a file-stream. Thanks, very much appreciated!

[Filip Ekberg]

However, I'd like to know how to read to the "next line", do i need to create that myself? There's no "readline" which reads to a \n ? or to any other token

[Filip Ekberg]

Im however not completely sattisfied with the answer, since it does not really provide how i do the other managing efficient :)

[Charlie Martin]

fgets is a read line. When you do fgets, the "file pointer" advances past the next newline. The loop I show above will read a file line by line. Just a sec and I'll put up a real example.

[Filip Ekberg]

Oh i see, sorry for not understanding that! Very good answer!

[Filip Ekberg]

Looks very good. How would you suggest you process each line? Imagine the fist line might be PORT=1000 would you somehow create a function that would return that line and the value as a struct? This might be more fundamentals about "architecture" than pure "how to".

[Filip Ekberg]

Well to be honest we have an assignment in a course at my univ. where we need to create an application but i don't want to store configuration in the applicaton, so i want to go a step further. That's why. And to learn, i want to do it from scratch,how much knowledge would i gain from using pre-made

[Filip Ekberg]

Oh and by the way, "Efficient" is kind of abstract, when i use "efficiently" i mean Simplicity + Readabillity and other aspects. I also not want it to take forever to process of course :)

[Filip Ekberg]

It's not been considered and it feels "unsafe" since it makes the application less portable, right?

[ruslanab]

Why not? You could use them on unix/win/mac. Just google "environment variable mac" or unix, win.

[Jonathan Leffler]

Why is bufr not a local variable? It perfectly well could be - and therefore (in my book) should be. I'd also use sizeof(bufr) in the fgets() call. I'd also test for failure and exit to avoid the distant else clause. I'd also close the file, in case the code gets ripped out into a function. Etc.

[Filip Ekberg]

Yeah, but you'd have to set all the envoirnment variables, which seems hard? Or that could maybe be done by programming it..

[ruslanab]

It could be set with shell/bat script or any other program. Google: rc.conf - that is example of such config file in FreeBSD.

[Filip Ekberg]

Sorry but i don't like it. And the question was not really for alternative configuration types

[Filip Ekberg]

Well this is a very nice Library, which i would probably use if i didnt want to learn about the underlying structure.

[Ilya]

can't even imaging why it was down voted, even for homework it's valid and most efficient advice here ...

[Filip Ekberg]

It doesnt provide anything that i asked for in my question so i can't see why it would ever be upvoted.

[Filip Ekberg]

@Jonathan, having the buffer as a local variable is argumentative. He most likely used it to simplify the example.

[bortzmeyer]

Paul Betts' answer is the best advice: change the specifications because they are wrong. Often, this is what we must tell to people, even if it makes them angry.

[Flash]

This is vulnerable to buffer overflow if the field is longer than MAXLINE, right?

[Charlie Martin]

Yeah. And? This is, after all, a discussion of making a config file.

[Bora M. Alper]

@CharlieMartin "Yeah. And? This is, after all, a discussion of making a config file." aaand here is your exploit. Nearly all configuration files are, by definition, exposed to the outside world and they can be modified in any way by the users and other 3rd party programs. So it's not merely a discussion of making a config file "after all". See stackoverflow.com/questions/1621394/… for how to use scanf properly.

[Charlie Martin]

@boramalper So, yes, you're right, I didn't talk about buffer overflow vulnerabilities while explaining to a new programmer how to read a basic text file. I didn't explain endianness either. Or segmentation faults. I've often noticed that one of my advantages teaching programming is that I still remember when I didn't know everything.

[Bora M. Alper]

@CharlieMartin I don't know what made you think that the author is a "new programmer" as I cannot see any indicators of that in the question, but nevertheless, why teach things the wrong way in the very first place? Endianness aside (presuming the we are not moving configuration files around), buffer overflows and segmentation faults are also a bug right in the software and not just a security issue which you should be concerned with only later. What if a path in the config is longer than MAXLINE, considering that it will also be indented with tabs (or worse, spaces) as well.

[Charlie Martin]

@boramalper the part where he asks if he needs to use fgetc or what.

[hypers]

This answer should have more ups. I've tested it and it is more functional considering handling comments and LFs.

[hypers]

But you probably want to fclose the file in the end.

[Andrew Henle]

Waaay late here, but there's a huge flaw in this answer. See Why is “while ( !feof (file) )” always wrong?