Package org.carrot2.text.analysis

Lexical analysis utilities.

See: Description

Interface Summary

Interface	Description
ITokenizer	Splits input characters into tokens representing e.g.

Class Summary

Class	Description
ExtendedWhitespaceTokenizer	A tokenizer separating input characters on whitespace, but capable of extracting more complex tokens, such as URLs, e-mail addresses and sentence delimiters.
ExtendedWhitespaceTokenizerImpl	This class is a scanner generated by JFlex 1.5.0-SNAPSHOT on 8/24/12 11:36 AM from the specification file C:/carrot2/carrotsearch.trunk/carrot2/core/carrot2-util-text/src/org/carrot2/text/analysis/ExtendedWhitespaceTokenizerImpl.jflex
TokenTypeUtils	Utility methods for working with ITokenizer attributes.

Package org.carrot2.text.analysis Description

Lexical analysis utilities.