org.cogroo.analyzer

Class Tokenizer

java.lang.Object

org.cogroo.analyzer.Tokenizer

All Implemented Interfaces:

Analyzer

public class Tokenizer
extends Object
implements Analyzer

The Tokenizer class separates every word in a given sentence and allocates them in a list of tokens.

Constructor Summary

Constructors

Constructor and Description
Tokenizer(opennlp.tools.tokenize.TokenizerME tokenizer)

Method Summary

Modifier and Type	Method and Description
void	analyze(Document document) Analyzes part of a text or a word.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tokenizer

public Tokenizer(opennlp.tools.tokenize.TokenizerME tokenizer)

Method Detail

analyze

public void analyze(Document document)

Description copied from interface: Analyzer

Analyzes part of a text or a word.

For example, it can search all the sentences in the text or tag every word in a sentence.

Specified by:

analyze in interface Analyzer

Parameters:

document - contains the whole text given by the user. After an analysis it can store the text's sentences, words or its tags.