Error whilst using StringTokenizer on text file with multiple lines

Question

I'm trying to read a text file and split the words individually using string tokenizer utility in java.

The text file looks like this;

a 2000

4  
b 3000  
c 4000  
d 5000

Now, what I'm trying to do is get each individual character from the text file and store it into an array list. I then try and print every element in the arraylist in the end.

Here is my code;

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.StringTokenizer;

public static void main(String[] args) {

    String fileSpecified = args[0];

    fileSpecified = fileSpecified.concat(".txt");
    String line;
    System.out.println ("file Specified = " + fileSpecified);

    ArrayList <String> words = new ArrayList<String> ();


    try {
        FileReader fr = new FileReader (fileSpecified);
        BufferedReader br = new BufferedReader (fr);
        line = br.readLine();

        StringTokenizer token;
        while ((line  = br.readLine()) != null) {
            token = new StringTokenizer (line);
            words.add(token.nextToken());
        }
    } catch (IOException e) {
        System.out.println (e.getMessage());
    }

    for (int i = 0; i < words.size(); i++) {
        System.out.println ("words = " + words.get(i));
    }



}

The error message I get is this;

Exception in thread "main" java.util.NoSuchElementException   
                at java.util.StringTokenizer.nextToken<Unknown Source>  
                at getWords.main<getWords.java:32>  

Where 'getWords' is the name of my java file.

Thankyou.

Solution 1

a) You always have to check StringTokenizer.hasMoreTokens() first. Throwing NoSuchElementException is the documented behaviour if no more tokens are available:

token = new StringTokenizer (line);
while(token.hasMoreTokens())
    words.add(token.nextToken());

b) don't create a new Tokenizer for every line, unless your file is too large to fit into memory. Read the entire file to a String and let the tokenizer work on that

Solution 2

Your general approach seems sound, but you have a basic problem in your code.

Your parser is most likely failing on the second line of your input file. This line is a blank line, so when you call words.add(token.nextToken()); you get an error, because there are no tokens. This also means you'll only ever get the first token on each line.

You should iterate on the tokes like this:

while(token.hasMoreTokens())
{
    words.add(token.nextToken())
}

You can find a more general example in the javadocs here:

http://download.oracle.com/javase/1.4.2/docs/api/java/util/StringTokenizer.html