info.informatica.text
Class TokenParser

java.lang.Object
  info.informatica.text.TokenParser

All Implemented Interfaces:

Iterator<String>

public final class TokenParser
extends Object
implements Iterator<String>

A StringTokenizer replacement with an Iterator interface and more flexibility.

Separates a String in tokens separated by sep, but grouped by delim.

Example: a,b,"c,d",e,f gives 5 tokens with default constructor.

If the separator contains white space ' ', consecutive separators are ignored.

Version:

1.03

Author:

amengual at informatica dot info

Constructor Summary

TokenParser(String line)
Separates a String in tokens separated by commas, and grouped by double quotes.

TokenParser(String line, String separator)
Separates a String in tokens separated by separator, and grouped by double quotes.

TokenParser(String line, String separator, String delimiters)
Separates a String in tokens separated by separator, and grouped by delimiters.

TokenParser(String line, String separator, String delimiters, boolean keepDelimiters)
Separates a String in tokens separated by separator, and grouped by delimiters.

Method Summary

void	findNext()
char	getLastSeparator()
char	getNextSeparator()
boolean	hasMoreTokens()
boolean	hasNext()
String	next()
String	nextToken()
void	remove()
static String[]	tokenize(String l, String sep, char[] delim, int init_size) Tokenizes a string l using sep as a token separator, while delim[] delimits a single token.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenParser

public TokenParser(String line)

Separates a String in tokens separated by commas, and grouped by double quotes.

Parameters:

line - the String to separate in tokens

TokenParser

public TokenParser(String line,
                   String separator)

Separates a String in tokens separated by separator, and grouped by double quotes.

Parameters:

line - the String to separate in tokens

separator - the separator String

TokenParser

public TokenParser(String line,
                   String separator,
                   String delimiters)

Separates a String in tokens separated by separator, and grouped by delimiters.

Parameters:

line - the String to separate in tokens

separator - the separator String

delimiters - a String of token delimiters. Note that using '(' as a delimiter implies ')', and the same for the "[]" and "{}" couples. Delimiters are removed from the tokens before returning them.

TokenParser

public TokenParser(String line,
                   String separator,
                   String delimiters,
                   boolean keepDelimiters)

Separates a String in tokens separated by separator, and grouped by delimiters.

Parameters:

line - the String to separate in tokens

separator - the separator String

delimiters - a String of token delimiters. Note that using '(' as a delimiter implies ')', and the same for the "[]" and "{}" couples.

keepDelimiters - true if we want to keep the string delimiters in the returned tokens, false otherwise.

Method Detail

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<String>

hasMoreTokens

public boolean hasMoreTokens()

next

public String next()

Specified by:

next in interface Iterator<String>

nextToken

public String nextToken()

remove

public void remove()

Specified by:

remove in interface Iterator<String>

findNext

public void findNext()

getLastSeparator

public char getLastSeparator()

getNextSeparator

public char getNextSeparator()

tokenize

public static String[] tokenize(String l,
                                String sep,
                                char[] delim,
                                int init_size)

Tokenizes a string l using sep as a token separator, while delim[] delimits a single token.

Example: a,b,"c,d",e,f gives 5 tokens.

Warning: to preserve backwards-compatibility, this static method does not behave exactly as the (newer and recommended) Iterator version (see comment below).

Parameters:

l - the input line to tokenize.

sep - the separator. In order to be compatible with legacy applications, this static version of the class handles sep as a separator which can be multiple-character.

delim - the delimiter (generally {'"'}).

init_size - a guess of the number of tokens to be found, used to set the initial size of the array.