Module org.apache.lucene.analysis.kuromoji

Package org.apache.lucene.analysis.ja.dict

Class TokenInfoDictionary

java.lang.Object

org.apache.lucene.analysis.ja.dict.BinaryDictionary

org.apache.lucene.analysis.ja.dict.TokenInfoDictionary

All Implemented Interfaces:

Dictionary

public final class TokenInfoDictionary
extends BinaryDictionary

Binary dictionary implementation for a known-word dictionary model: Words are encoded into an FST mapping to a list of wordIDs.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static class	TokenInfoDictionary.SingletonHolder

Nested classes/interfaces inherited from class org.apache.lucene.analysis.ja.dict.BinaryDictionary

BinaryDictionary.ResourceScheme

Field Summary

Fields

Modifier and Type	Field	Description
private final TokenInfoFST	fst
static final String	FST_FILENAME_SUFFIX

Fields inherited from class org.apache.lucene.analysis.ja.dict.BinaryDictionary

DICT_FILENAME_SUFFIX, DICT_HEADER, HAS_BASEFORM, HAS_PRONUNCIATION, HAS_READING, POSDICT_FILENAME_SUFFIX, POSDICT_HEADER, TARGETMAP_FILENAME_SUFFIX, TARGETMAP_HEADER, VERSION

Fields inherited from interface org.apache.lucene.analysis.ja.dict.Dictionary

INTERNAL_SEPARATOR

Constructor Summary

Constructors

Modifier	Constructor	Description
private	TokenInfoDictionary()
	TokenInfoDictionary(URL targetMapUrl, URL posDictUrl, URL dictUrl, URL fstUrl)	Create a TokenInfoDictionary from an external resource URL (e.g.
	TokenInfoDictionary(Path targetMapFile, Path posDictFile, Path dictFile, Path fstFile)	Create a TokenInfoDictionary from an external resource path.
	TokenInfoDictionary(BinaryDictionary.ResourceScheme resourceScheme, String resourcePath)	Deprecated, for removal: This API element is subject to removal in a future version. replaced by TokenInfoDictionary(Path, Path, Path, Path) for files and TokenInfoDictionary(URL, URL, URL, URL) for classpath/module resources
private	TokenInfoDictionary(IOSupplier<InputStream> targetMapResource, IOSupplier<InputStream> posResource, IOSupplier<InputStream> dictResource, IOSupplier<InputStream> fstResource)

Method Summary

Modifier and Type	Method	Description
(package private) static InputStream	getClassResource(String suffix)
TokenInfoFST	getFST()
static TokenInfoDictionary	getInstance()

Methods inherited from class org.apache.lucene.analysis.ja.dict.BinaryDictionary

getBaseForm, getInflectionForm, getInflectionType, getLeftId, getPartOfSpeech, getPronunciation, getReading, getResource, getRightId, getWordCost, lookupWordIds

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

FST_FILENAME_SUFFIX

public static final String FST_FILENAME_SUFFIX

See Also:

• Constant Field Values

fst

private final TokenInfoFST fst

Constructor Details

TokenInfoDictionary

@Deprecated(forRemoval=true,
            since="9.1")
public TokenInfoDictionary(BinaryDictionary.ResourceScheme resourceScheme,
 String resourcePath)
                    throws IOException

Deprecated, for removal: This API element is subject to removal in a future version.

replaced by TokenInfoDictionary(Path, Path, Path, Path) for files and TokenInfoDictionary(URL, URL, URL, URL) for classpath/module resources

Parameters:

resourceScheme - - scheme for loading resources (FILE or CLASSPATH).

resourcePath - - where to load resources (dictionaries) from.

Throws:

IOException

TokenInfoDictionary

public TokenInfoDictionary(Path targetMapFile,
 Path posDictFile,
 Path dictFile,
 Path fstFile)
                    throws IOException

Create a TokenInfoDictionary from an external resource path.

Parameters:

targetMapFile - where to load target map resource

posDictFile - where to load POS dictionary resource

dictFile - where to load dictionary entries resource

fstFile - where to load encoded FST data resource

Throws:

IOException - if resource was not found or broken

TokenInfoDictionary

public TokenInfoDictionary(URL targetMapUrl,
 URL posDictUrl,
 URL dictUrl,
 URL fstUrl)
                    throws IOException

Create a TokenInfoDictionary from an external resource URL (e.g. from Classpath with ClassLoader.getResource(String)).

Parameters:

targetMapUrl - where to load target map resource

posDictUrl - where to load POS dictionary resource

dictUrl - where to load dictionary entries resource

fstUrl - where to load encoded FST data resource

Throws:

IOException - if resource was not found or broken

TokenInfoDictionary

private TokenInfoDictionary()
                     throws IOException

Throws:

IOException

TokenInfoDictionary

private TokenInfoDictionary(IOSupplier<InputStream> targetMapResource,
 IOSupplier<InputStream> posResource,
 IOSupplier<InputStream> dictResource,
 IOSupplier<InputStream> fstResource)
                     throws IOException

Throws:

IOException

Method Details

getClassResource

static InputStream getClassResource(String suffix)
                             throws IOException

Throws:

IOException

getFST

public TokenInfoFST getFST()

getInstance

public static TokenInfoDictionary getInstance()