org.carrot2.text.linguistic

Class DefaultLexicalDataFactory

java.lang.Object

org.carrot2.text.linguistic.DefaultLexicalDataFactory

All Implemented Interfaces:

ILexicalDataFactory

@Bindable(inherit=LexicalDataLoader.class)
public class DefaultLexicalDataFactory
extends Object
implements ILexicalDataFactory

The default management of lexical resources. Resources are read from disk, cached and shared between all threads using this class. Additional attributes control resource reloading and merging: resourceLookup, reloadResources, mergeResources.

Field Summary

Fields

Modifier and Type	Field and Description
boolean	mergeResources Merges stop words and stop labels from all known languages.
boolean	reloadResources
ResourceLookup	resourceLookup

Constructor Summary

Constructors

Constructor and Description
DefaultLexicalDataFactory()

Method Summary

Methods

Modifier and Type	Method and Description
ILexicalData	getLexicalData(LanguageCode languageCode) The main logic for acquiring a shared ILexicalData instance.
static HashSet<String>	load(IResource resource) Loads words from a given IResource (UTF-8, one word per line, #-starting lines are considered comments).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

reloadResources

@Processing
@Input
@Attribute(key="reload-resources",
           inherit=true)
public boolean reloadResources

mergeResources

@Processing
@Input
@Attribute(key="merge-resources")
@Label(value="Merge lexical resources")
@Level(value=MEDIUM)
@Group(value="Preprocessing")
public boolean mergeResources

Merges stop words and stop labels from all known languages. If set to false, only stop words and stop labels of the active language will be used. If set to true, stop words from all LanguageCodes will be used together and stop labels from all languages will be used together, no matter the active language. Lexical resource merging is useful when clustering data in a mix of different languages and should increase clustering quality in such settings.

resourceLookup

@Processing
@Input
@Internal
@Attribute(key="resource-lookup",
           inherit=true)
public ResourceLookup resourceLookup

Constructor Detail

DefaultLexicalDataFactory

public DefaultLexicalDataFactory()

Method Detail

getLexicalData

public ILexicalData getLexicalData(LanguageCode languageCode)

The main logic for acquiring a shared ILexicalData instance.

Specified by:

getLexicalData in interface ILexicalDataFactory

load

public static HashSet<String> load(IResource resource)
                            throws IOException

Loads words from a given IResource (UTF-8, one word per line, #-starting lines are considered comments).

Throws:

IOException