Flex: Use Text File as Input Stream

Question

So I've used flex to generate a c file from my lex code, and then used gcc to create the corresponding actual parser .exe. However, I can't figure out how to get it to read from anything other than my own typed input. I am trying to get it to parse a huge dictionary file. Can anyone help?

Solution 1

You have two ways of solving it. The first is to redirect input from standard input with the command prompt < operation:

> parser.exe < some_file.txt

The other solution is to let the program open the file, and tell the lexer what the file is. For more information about it see the Flex manual. The important functions are yy_create_buffer and yy_switch_to_buffer.

Solution 2

Try to add the following code to your *.l file.

int main(int argc, char *argv[])
{
    yyin = fopen(argv[1], "r");
    yylex();
    fclose(yyin);
}

Solution 3

Adding onto the above answer by @Eliko, while using flex with yacc/bison, you can define FILE *yyin; in the global part of your grammar.y file. The definition in the generated lex.yy.c is an extern FILE *yyin by default. Thus, in your grammar.y, do something like this:

/* Some other global definitions */
FILE *yyin;
%%
/* Grammar rules*/
/* Grammar rules*/
%%
void main(int argc, char **argv) {
  /* Process command line args*/
  yyin = fopen("input.c", "r");
  yyparse();
  fclose(yyin);
  return 0;
}

Comments

[Jonathan Leffler]

There are a number of other relevant sections, such as the one about EOF handling and yywrap. The YY_INPUT macro is part of the answer. Nevertheless, the manual pointed to treated as a whole covers the issue.

[Jay Lee]

I had to use extern File *yyin just above the main function.

[Dean P]

In my instance, Im integrating flex code into my executable and all I had to do was add a line yyin = fopen(myFile, "r") and flex read from the file with no need to use yy_create_buffer or yy_switch_to_buffer. Im assuming the previous two functions are only needed if flex is not integrated into a user executable? Im trying to ensure Im not being naive in assuming just because in my case "it works", that its necessarily "right".