Class TokenInfoDictionary
java.lang.Object
org.apache.lucene.analysis.morph.BinaryDictionary<TokenInfoMorphData>
org.apache.lucene.analysis.ja.dict.TokenInfoDictionary
- All Implemented Interfaces:
Dictionary<TokenInfoMorphData>
Binary dictionary implementation for a known-word dictionary model: Words are encoded into an FST
mapping to a list of wordIDs.
-
Nested Class Summary
Nested Classes -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate final TokenInfoFST
static final String
private final TokenInfoMorphData
Fields inherited from class org.apache.lucene.analysis.morph.BinaryDictionary
buffer, DICT_FILENAME_SUFFIX, POSDICT_FILENAME_SUFFIX, TARGETMAP_FILENAME_SUFFIX
-
Constructor Summary
ConstructorsModifierConstructorDescriptionprivate
TokenInfoDictionary
(URL targetMapUrl, URL posDictUrl, URL dictUrl, URL fstUrl) Create aTokenInfoDictionary
from an external resource URL (e.g.TokenInfoDictionary
(Path targetMapFile, Path posDictFile, Path dictFile, Path fstFile) Create aTokenInfoDictionary
from an external resource path.private
TokenInfoDictionary
(IOSupplier<InputStream> targetMapResource, IOSupplier<InputStream> posResource, IOSupplier<InputStream> dictResource, IOSupplier<InputStream> fstResource) -
Method Summary
Modifier and TypeMethodDescription(package private) static InputStream
getClassResource
(String suffix) getFST()
static TokenInfoDictionary
Methods inherited from class org.apache.lucene.analysis.morph.BinaryDictionary
lookupWordIds
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.lucene.analysis.morph.Dictionary
getLeftId, getRightId, getWordCost
-
Field Details
-
FST_FILENAME_SUFFIX
- See Also:
-
fst
-
morphAtts
-
-
Constructor Details
-
TokenInfoDictionary
public TokenInfoDictionary(Path targetMapFile, Path posDictFile, Path dictFile, Path fstFile) throws IOException Create aTokenInfoDictionary
from an external resource path.- Parameters:
targetMapFile
- where to load target map resourceposDictFile
- where to load POS dictionary resourcedictFile
- where to load dictionary entries resourcefstFile
- where to load encoded FST data resource- Throws:
IOException
- if resource was not found or broken
-
TokenInfoDictionary
public TokenInfoDictionary(URL targetMapUrl, URL posDictUrl, URL dictUrl, URL fstUrl) throws IOException Create aTokenInfoDictionary
from an external resource URL (e.g. from Classpath withClassLoader.getResource(String)
).- Parameters:
targetMapUrl
- where to load target map resourceposDictUrl
- where to load POS dictionary resourcedictUrl
- where to load dictionary entries resourcefstUrl
- where to load encoded FST data resource- Throws:
IOException
- if resource was not found or broken
-
TokenInfoDictionary
- Throws:
IOException
-
TokenInfoDictionary
private TokenInfoDictionary(IOSupplier<InputStream> targetMapResource, IOSupplier<InputStream> posResource, IOSupplier<InputStream> dictResource, IOSupplier<InputStream> fstResource) throws IOException - Throws:
IOException
-
-
Method Details
-
getClassResource
- Throws:
IOException
-
getMorphAttributes
-
getFST
-
getInstance
-