Module org.apache.lucene.core

Package org.apache.lucene.codecs.lucene99

Class Lucene99HnswVectorsReader

java.lang.Object

org.apache.lucene.codecs.KnnVectorsReader

org.apache.lucene.codecs.lucene99.Lucene99HnswVectorsReader

All Implemented Interfaces:

Closeable, AutoCloseable, HnswGraphProvider, Accountable, QuantizedVectorsReader

public final class Lucene99HnswVectorsReader
extends KnnVectorsReader
implements QuantizedVectorsReader, HnswGraphProvider

Reads vectors from the index segments along with index data structures supporting KNN search.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
(package private) static class	Lucene99HnswVectorsReader.FieldEntry
private static final class	Lucene99HnswVectorsReader.OffHeapHnswGraph	Read the nearest-neighbors graph from the index input

Field Summary

Fields

Modifier and Type	Field	Description
private final FieldInfos	fieldInfos
private final Map<String,Lucene99HnswVectorsReader.FieldEntry>	fields
private final FlatVectorsReader	flatVectorsReader
private static final long	SHALLOW_SIZE
static final List<VectorSimilarityFunction>	SIMILARITY_FUNCTIONS
private final IndexInput	vectorIndex

Fields inherited from interface org.apache.lucene.util.Accountable

NULL_ACCOUNTABLE

Constructor Summary

Constructors

Constructor	Description
Lucene99HnswVectorsReader(SegmentReadState state, FlatVectorsReader flatVectorsReader)

Method Summary

Modifier and Type	Method	Description
void	checkIntegrity()	Checks consistency of this reader.
void	close()
ByteVectorValues	getByteVectorValues(String field)	Returns the ByteVectorValues for the given field.
FloatVectorValues	getFloatVectorValues(String field)	Returns the FloatVectorValues for the given field.
HnswGraph	getGraph(String field)	Return the stored HnswGraph for the given field.
private HnswGraph	getGraph(Lucene99HnswVectorsReader.FieldEntry entry)
ScalarQuantizer	getQuantizationState(String field)
QuantizedByteVectorValues	getQuantizedVectorValues(String field)
private static IndexInput	openDataInput(SegmentReadState state, int versionMeta, String fileExtension, String codecName, IOContext context)
long	ramBytesUsed()	Return the memory usage of this object in bytes.
private Lucene99HnswVectorsReader.FieldEntry	readField(IndexInput input, FieldInfo info)
private void	readFields(ChecksumIndexInput meta, FieldInfos infos)
static VectorSimilarityFunction	readSimilarityFunction(DataInput input)
static VectorEncoding	readVectorEncoding(DataInput input)
void	search(String field, byte[] target, KnnCollector knnCollector, Bits acceptDocs)	Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function.
void	search(String field, float[] target, KnnCollector knnCollector, Bits acceptDocs)	Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function.
private void	validateFieldEntry(FieldInfo info, Lucene99HnswVectorsReader.FieldEntry fieldEntry)

Methods inherited from class org.apache.lucene.codecs.KnnVectorsReader

getMergeInstance

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.lucene.util.Accountable

getChildResources

Field Details

SHALLOW_SIZE

private static final long SHALLOW_SIZE

fieldInfos

private final FieldInfos fieldInfos

fields

private final Map<String,Lucene99HnswVectorsReader.FieldEntry> fields

vectorIndex

private final IndexInput vectorIndex

flatVectorsReader

private final FlatVectorsReader flatVectorsReader

SIMILARITY_FUNCTIONS

public static final List<VectorSimilarityFunction> SIMILARITY_FUNCTIONS

Constructor Details

Lucene99HnswVectorsReader

public Lucene99HnswVectorsReader(SegmentReadState state,
 FlatVectorsReader flatVectorsReader)
                          throws IOException

Throws:

IOException

Method Details

openDataInput

private static IndexInput openDataInput(SegmentReadState state,
 int versionMeta,
 String fileExtension,
 String codecName,
 IOContext context)
                                 throws IOException

Throws:

IOException

readFields

private void readFields(ChecksumIndexInput meta,
 FieldInfos infos)
                 throws IOException

Throws:

IOException

validateFieldEntry

private void validateFieldEntry(FieldInfo info,
 Lucene99HnswVectorsReader.FieldEntry fieldEntry)

readSimilarityFunction

public static VectorSimilarityFunction readSimilarityFunction(DataInput input)
                                                       throws IOException

Throws:

IOException

readVectorEncoding

public static VectorEncoding readVectorEncoding(DataInput input)
                                         throws IOException

Throws:

IOException

readField

private Lucene99HnswVectorsReader.FieldEntry readField(IndexInput input,
 FieldInfo info)
                                                throws IOException

Throws:

IOException

ramBytesUsed

public long ramBytesUsed()

Description copied from interface: Accountable

Return the memory usage of this object in bytes. Negative values are illegal.

Specified by:

ramBytesUsed in interface Accountable

checkIntegrity

public void checkIntegrity()
                    throws IOException

Description copied from class: KnnVectorsReader

Checks consistency of this reader.

Note that this may be costly in terms of I/O, e.g. may involve computing a checksum value against large data files.

Specified by:

checkIntegrity in class KnnVectorsReader

Throws:

IOException

getFloatVectorValues

public FloatVectorValues getFloatVectorValues(String field)
                                       throws IOException

Description copied from class: KnnVectorsReader

Returns the FloatVectorValues for the given field. The behavior is undefined if the given field doesn't have KNN vectors enabled on its FieldInfo. The return value is never null.

Specified by:

getFloatVectorValues in class KnnVectorsReader

Throws:

IOException

getByteVectorValues

public ByteVectorValues getByteVectorValues(String field)
                                     throws IOException

Description copied from class: KnnVectorsReader

Returns the ByteVectorValues for the given field. The behavior is undefined if the given field doesn't have KNN vectors enabled on its FieldInfo. The return value is never null.

Specified by:

getByteVectorValues in class KnnVectorsReader

Throws:

IOException

search

public void search(String field,
 float[] target,
 KnnCollector knnCollector,
 Bits acceptDocs)
            throws IOException

Description copied from class: KnnVectorsReader

Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function. The score of each document is derived from the vector similarity in a way that ensures scores are positive and that a larger score corresponds to a higher ranking.

The search is allowed to be approximate, meaning the results are not guaranteed to be the true k closest neighbors. For large values of k (for example when k is close to the total number of documents), the search may also retrieve fewer than k documents.

The returned TopDocs will contain a ScoreDoc for each nearest neighbor, in order of their similarity to the query vector (decreasing scores). The TotalHits contains the number of documents visited during the search. If the search stopped early because it hit visitedLimit, it is indicated through the relation TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO.

The behavior is undefined if the given field doesn't have KNN vectors enabled on its FieldInfo. The return value is never null.

Specified by:

search in class KnnVectorsReader

Parameters:

field - the vector field to search

target - the vector-valued query

knnCollector - a KnnResults collector and relevant settings for gathering vector results

acceptDocs - Bits that represents the allowed documents to match, or null if they are all allowed to match.

Throws:

IOException

search

public void search(String field,
 byte[] target,
 KnnCollector knnCollector,
 Bits acceptDocs)
            throws IOException

Description copied from class: KnnVectorsReader

Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function. The score of each document is derived from the vector similarity in a way that ensures scores are positive and that a larger score corresponds to a higher ranking.

The search is allowed to be approximate, meaning the results are not guaranteed to be the true k closest neighbors. For large values of k (for example when k is close to the total number of documents), the search may also retrieve fewer than k documents.

The returned TopDocs will contain a ScoreDoc for each nearest neighbor, in order of their similarity to the query vector (decreasing scores). The TotalHits contains the number of documents visited during the search. If the search stopped early because it hit visitedLimit, it is indicated through the relation TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO.

The behavior is undefined if the given field doesn't have KNN vectors enabled on its FieldInfo. The return value is never null.

Specified by:

search in class KnnVectorsReader

Parameters:

field - the vector field to search

target - the vector-valued query

knnCollector - a KnnResults collector and relevant settings for gathering vector results

acceptDocs - Bits that represents the allowed documents to match, or null if they are all allowed to match.

Throws:

IOException

getGraph

public HnswGraph getGraph(String field)
                   throws IOException

Description copied from interface: HnswGraphProvider

Return the stored HnswGraph for the given field.

Specified by:

getGraph in interface HnswGraphProvider

Parameters:

field - the field containing the graph

Returns:

the HnswGraph for the given field if found

Throws:

IOException - when reading potentially off-heap graph fails

getGraph

private HnswGraph getGraph(Lucene99HnswVectorsReader.FieldEntry entry)
                    throws IOException

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

getQuantizedVectorValues

public QuantizedByteVectorValues getQuantizedVectorValues(String field)
                                                   throws IOException

Specified by:

getQuantizedVectorValues in interface QuantizedVectorsReader

Throws:

IOException

getQuantizationState

public ScalarQuantizer getQuantizationState(String field)

Specified by:

getQuantizationState in interface QuantizedVectorsReader