Java Tree parser output for ANTLR

Question

I've found a sample template in the ANTLR website, its the Javatreeparser.g, which the site says could produce the AST that I need, but since I'm new to ANTLR, how do I make it show? What I've done so far is placing the grammar file together with my existing java grammar. But I have no idea on how to use and output the AST that I need from the file. How do I do it?

Solution 1

I've found a sample template in the ANTLR website, its the Javatreeparser.g, which the site says could produce the AST that I need,

No, the combined grammar Java.g from the ANTLR wiki produces a lexer and parser for Java source files. The parser then constructs an AST of this source and this AST can then be used by JavaTreeParser.g to traverse it. The tree grammar JavaTreeParser.g is not used to create an AST. This is done by the parser created from Java.g.

What I've done so far is placing the grammar file together with my existing java grammar.

That is incorrect. The tree grammar JavaTreeParser.g expects an AST as input that the parser generated from Java.g produced. You can't just plug in another parser (or other tree grammar, for that matter).

But I have no idea on how to use and output the AST that I need from the file. How do I do it?

See this previous Q&A: Visualizing an AST created with ANTLR (in a .Net environment)

EDIT

I didn't want to post this immediately, because I wanted you to give it a try yourself first (yes, I'm mean!) ;)

Here's a quick demo:

1. copy the Java.g in a directory and remove the @header{...} and @lexer:::header{...} declarations from it;
2. copy antlr-3.3.jar into the same directory;
3. create the files Main.java and Test.java in this directory (see below).

Test.java

public class Test {

    int i = 1 + 2;
    String s;

    Test(String s) {
        this.s = s;
    }
}

Main.java

import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;

public class Main {
    public static void main(String[] args) throws Exception {
        JavaLexer lexer = new JavaLexer(new ANTLRFileStream("Test.java"));
        JavaParser parser = new JavaParser(new CommonTokenStream(lexer));
        CommonTree tree = (CommonTree)parser.javaSource().getTree();
        DOTTreeGenerator gen = new DOTTreeGenerator();
        StringTemplate st = gen.toDOT(tree);
        System.out.println(st);
    }
}

Now generate a lexer and parser:

java -cp antlr-3.3.jar org.antlr.Tool Java.g 

Then compile all .java source files:

javac -cp antlr-3.3.jar *.java 

And finally run the Main class and pipe the output to a file called ast.dot.

java -cp .:antlr-3.3.jar Main > ast.dot

(on Windows, do: java -cp .;antlr-3.3.jar Main > ast.dot)

If you now open the file ast.dot, you see a DOT representation of the AST produced by the parser. You can visualize this AST by copy-pasting the DOT-source in here: http://graphviz-dev.appspot.com resulting in the following image:

Solution 2

I really recommend you to use antlr4.

First, set your CLASSPATH (including antlr-4.5.3-complete.jar) and JAVA_HOME.

Second, generate a lexer and parser from the grammar Java.g4:

java -cp antlr-4.5.2-complete.jar Java.g4

Third, compile all Java*.java genereated:

javac Java*.java

Finally, run TestRig:

java org.antlr.v4.runtime.misc.TestRig Java compilationUnit -gui Test.java

You will see AST visually as follows: