Module org.apache.lucene.codecs

Package org.apache.lucene.codecs.uniformsplit

Class BlockReader

java.lang.Object

org.apache.lucene.index.TermsEnum

org.apache.lucene.index.BaseTermsEnum

org.apache.lucene.codecs.uniformsplit.BlockReader

All Implemented Interfaces:

Accountable, BytesRefIterator

Direct Known Subclasses:

IntersectBlockReader, STBlockReader

public class BlockReader
extends BaseTermsEnum
implements Accountable

Seeks the block corresponding to a given term, read the block bytes, and scans the block terms.

Reads fully the block in blockReadBuffer. Then scans the block terms in memory. The details region is lazily decoded with termStatesReadBuffer which shares the same byte array with blockReadBuffer. See BlockWriter and BlockLine for the block format.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.index.TermsEnum

TermsEnum.SeekStatus

Field Summary

Fields

Modifier and Type	Field	Description
private static final long	BASE_RAM_USAGE
protected final BlockDecoder	blockDecoder
protected int	blockFirstLineStart	Offset of the start of the first line of the current block (just after the header), relative to the block start.
protected BlockHeader	blockHeader	Current block header.
protected BlockHeader.Serializer	blockHeaderReader
protected IndexInput	blockInput	IndexInput on the block file.
protected BlockLine	blockLine	Current block line.
protected BlockLine.Serializer	blockLineReader
protected ByteArrayDataInput	blockReadBuffer	In-memory read buffer for the current block.
protected long	blockStartFP	Current block start file pointer, absolute in the block file.
protected IndexDictionary.Browser	dictionaryBrowser	Holds the IndexDictionary.Browser once loaded.
protected final IndexDictionary.BrowserSupplier	dictionaryBrowserSupplier	IndexDictionary.Browser supplier for lazy loading.
protected final FieldMetadata	fieldMetadata
protected BytesRefBuilder	forcedTerm	Set when seekExact(BytesRef, TermState) is called.
protected int	lineIndexInBlock	Current line index in the block.
protected final PostingsReaderBase	postingsReader
protected BytesRef	scratchBlockBytes
protected BlockLine	scratchBlockLine
protected final BlockTermState	scratchTermState
protected BlockTermState	termState	Current block line details.
protected boolean	termStateForced	Whether the current TermState has been forced with a call to seekExact(BytesRef, TermState).
protected DeltaBaseTermStateSerializer	termStateSerializer
protected ByteArrayDataInput	termStatesReadBuffer	In-memory read buffer for the details region of the current block.

Fields inherited from class org.apache.lucene.index.TermsEnum

EMPTY

Fields inherited from interface org.apache.lucene.util.Accountable

NULL_ACCOUNTABLE

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	BlockReader(IndexDictionary.BrowserSupplier dictionaryBrowserSupplier, IndexInput blockInput, PostingsReaderBase postingsReader, FieldMetadata fieldMetadata, BlockDecoder blockDecoder)

Method Summary

Modifier and Type	Method	Description
protected void	clearTermState()	Called by the primary TermsEnum methods to clear the previous TermState.
protected int	compareToMiddleAndJump(BytesRef searchedTerm)	Compares the searched term to the middle term of the block.
protected BlockHeader.Serializer	createBlockHeaderSerializer()
protected BlockLine.Serializer	createBlockLineSerializer()
protected DeltaBaseTermStateSerializer	createDeltaBaseTermStateSerializer()
protected BytesRef	decodeBlockBytesIfNeeded(int numBlockBytes)
int	docFreq()	Returns the number of documents containing the current term.
protected IndexDictionary.Browser	getOrCreateDictionaryBrowser()
ImpactsEnum	impacts(int flags)	Return a ImpactsEnum.
protected void	initializeBlockReadLazily()
protected void	initializeHeader(BytesRef searchedTerm, long targetBlockStartFP)	Reads and sets blockHeader.
protected boolean	isBeyondLastTerm(BytesRef searchedTerm, long blockStartFP)	Indicates whether the searched term is beyond the last term of the field.
protected boolean	isCurrentTerm(BytesRef searchedTerm)
protected CorruptIndexException	newCorruptIndexException(String msg, Long fp)
BytesRef	next()	Increments the iteration to the next BytesRef in the iterator.
protected BytesRef	nextTerm()	Moves to the next term line and reads it, it may be in the next block.
long	ord()	Returns ordinal position for current term.
PostingsEnum	postings(PostingsEnum reuse, int flags)	Get PostingsEnum for the current term, with control over whether freqs, positions, offsets or payloads are required.
long	ramBytesUsed()	Return the memory usage of this object in bytes.
protected BlockHeader	readHeader()	Reads the block header.
protected BlockLine	readLineInBlock()	Reads the current block line.
protected BlockTermState	readTermState()	Reads the BlockTermState on the current line.
protected BlockTermState	readTermStateIfNotRead()	Reads the BlockTermState if it is not already set.
TermsEnum.SeekStatus	seekCeil(BytesRef searchedTerm)	Seeks to the specified term, if it exists, or to the next (ceiling) term.
void	seekExact(long ord)	Not supported.
boolean	seekExact(BytesRef searchedTerm)	Attempts to seek to the exact term, returning true if the term is found.
void	seekExact(BytesRef term, TermState state)	Positions this BlockReader without re-seeking the term dictionary.
protected TermsEnum.SeekStatus	seekInBlock(BytesRef searchedTerm)	Seeks to the provided term in this block.
protected TermsEnum.SeekStatus	seekInBlock(BytesRef searchedTerm, long blockStartFP)	Seeks to the provided term in the block starting at the provided file pointer.
BytesRef	term()	Returns current term.
TermState	termState()	Expert: Returns the TermsEnums internal state to position the TermsEnum without re-seeking the term dictionary.
long	totalTermFreq()	Returns the total number of occurrences of this term across all documents (the sum of the freq() for each doc that has this term).

Methods inherited from class org.apache.lucene.index.BaseTermsEnum

attributes

Methods inherited from class org.apache.lucene.index.TermsEnum

postings

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.lucene.util.Accountable

getChildResources

Field Details

BASE_RAM_USAGE

private static final long BASE_RAM_USAGE

blockInput

protected IndexInput blockInput

IndexInput on the block file.

postingsReader

protected final PostingsReaderBase postingsReader

fieldMetadata

protected final FieldMetadata fieldMetadata

blockDecoder

protected final BlockDecoder blockDecoder

blockHeaderReader

protected BlockHeader.Serializer blockHeaderReader

blockLineReader

protected BlockLine.Serializer blockLineReader

blockReadBuffer

protected ByteArrayDataInput blockReadBuffer

In-memory read buffer for the current block.

termStatesReadBuffer

protected ByteArrayDataInput termStatesReadBuffer

In-memory read buffer for the details region of the current block. It shares the same byte array as blockReadBuffer, with a different position.

termStateSerializer

protected DeltaBaseTermStateSerializer termStateSerializer

dictionaryBrowserSupplier

protected final IndexDictionary.BrowserSupplier dictionaryBrowserSupplier

IndexDictionary.Browser supplier for lazy loading.

dictionaryBrowser

protected IndexDictionary.Browser dictionaryBrowser

Holds the IndexDictionary.Browser once loaded.

blockStartFP

protected long blockStartFP

Current block start file pointer, absolute in the block file.

blockHeader

protected BlockHeader blockHeader

Current block header.

blockLine

protected BlockLine blockLine

Current block line.

termState

protected BlockTermState termState

Current block line details.

blockFirstLineStart

protected int blockFirstLineStart

Offset of the start of the first line of the current block (just after the header), relative to the block start.

lineIndexInBlock

protected int lineIndexInBlock

Current line index in the block.

termStateForced

protected boolean termStateForced

Whether the current TermState has been forced with a call to seekExact(BytesRef, TermState).

See Also:

• forcedTerm

forcedTerm

protected BytesRefBuilder forcedTerm

Set when seekExact(BytesRef, TermState) is called.

This optimizes the use-case when the caller calls first seekExact(BytesRef, TermState) and then postings(PostingsEnum, int). In this case we don't access the terms block file (we don't seek) but directly the postings file because we already have the TermState with the file pointers to the postings file.

scratchBlockBytes

protected BytesRef scratchBlockBytes

scratchTermState

protected final BlockTermState scratchTermState

scratchBlockLine

protected BlockLine scratchBlockLine

Constructor Details

BlockReader

protected BlockReader(IndexDictionary.BrowserSupplier dictionaryBrowserSupplier,
 IndexInput blockInput,
 PostingsReaderBase postingsReader,
 FieldMetadata fieldMetadata,
 BlockDecoder blockDecoder)
               throws IOException

Parameters:

dictionaryBrowserSupplier - to load the IndexDictionary.Browser lazily in seekCeil(BytesRef).

blockDecoder - Optional block decoder, may be null if none. It can be used for decompression or decryption.

Throws:

IOException

Method Details

seekCeil

public TermsEnum.SeekStatus seekCeil(BytesRef searchedTerm)
                              throws IOException

Description copied from class: TermsEnum

Seeks to the specified term, if it exists, or to the next (ceiling) term. Returns SeekStatus to indicate whether exact term was found, a different term was found, or EOF was hit. The target term may be before or after the current term. If this returns SeekStatus.END, the enum is unpositioned.

Specified by:

seekCeil in class TermsEnum

Throws:

IOException

seekExact

public boolean seekExact(BytesRef searchedTerm)
                  throws IOException

Description copied from class: TermsEnum

Attempts to seek to the exact term, returning true if the term is found. If this returns false, the enum is unpositioned. For some codecs, seekExact may be substantially faster than TermsEnum.seekCeil(org.apache.lucene.util.BytesRef).

Overrides:

seekExact in class BaseTermsEnum

Returns:

true if the term is found; return false if the enum is unpositioned.

Throws:

IOException

isCurrentTerm

protected boolean isCurrentTerm(BytesRef searchedTerm)

isBeyondLastTerm

protected boolean isBeyondLastTerm(BytesRef searchedTerm,
 long blockStartFP)

Indicates whether the searched term is beyond the last term of the field.

Parameters:

blockStartFP - The current block start file pointer.

seekInBlock

protected TermsEnum.SeekStatus seekInBlock(BytesRef searchedTerm,
 long blockStartFP)
                                    throws IOException

Seeks to the provided term in the block starting at the provided file pointer. Does not exceed the block.

Throws:

IOException

seekInBlock

protected TermsEnum.SeekStatus seekInBlock(BytesRef searchedTerm)
                                    throws IOException

Seeks to the provided term in this block.

Does not exceed this block; TermsEnum.SeekStatus.END is returned if it follows the block.

Compares the line terms with the searchedTerm, taking advantage of the incremental encoding properties.

Scans linearly the terms. Updates the current block line with the current term.

Throws:

IOException

compareToMiddleAndJump

protected int compareToMiddleAndJump(BytesRef searchedTerm)
                              throws IOException

Compares the searched term to the middle term of the block. If the searched term is lexicographically equal or after the middle term then jumps to the second half of the block directly.

Returns:

The comparison between the searched term and the middle term.

Throws:

IOException

readLineInBlock

protected BlockLine readLineInBlock()
                             throws IOException

Reads the current block line. Sets blockLine and increments lineIndexInBlock.

Returns:

The BlockLine; or null if there no more line in the block.

Throws:

IOException

seekExact

public void seekExact(BytesRef term,
 TermState state)

Positions this BlockReader without re-seeking the term dictionary.

The block containing the term is not read by this method. It will be read lazily only if needed, for example if next() is called. Calling postings(org.apache.lucene.index.PostingsEnum, int) after this method does require the block to be read.

Overrides:

seekExact in class BaseTermsEnum

Parameters:

term - the term the TermState corresponds to

state - the TermState

seekExact

public void seekExact(long ord)

Not supported.

Specified by:

seekExact in class TermsEnum

next

public BytesRef next()
              throws IOException

Description copied from interface: BytesRefIterator

Increments the iteration to the next BytesRef in the iterator. Returns the resulting BytesRef or null if the end of the iterator is reached. The returned BytesRef may be re-used across calls to next. After this method returns null, do not call it again: the results are undefined.

Specified by:

next in interface BytesRefIterator

Returns:

the next BytesRef in the iterator or null if the end of the iterator is reached.

Throws:

IOException - If there is a low-level I/O error.

nextTerm

protected BytesRef nextTerm()
                     throws IOException

Moves to the next term line and reads it, it may be in the next block. The term details are not read yet. They will be read only when needed with readTermStateIfNotRead().

Returns:

The read term bytes; or null if there is no more term for the field.

Throws:

IOException

initializeHeader

protected void initializeHeader(BytesRef searchedTerm,
 long targetBlockStartFP)
                         throws IOException

Reads and sets blockHeader. Sets null if there is no block for the field anymore.

Parameters:

searchedTerm - The searched term; or null if none.

targetBlockStartFP - The file pointer of the block to read.

Throws:

IOException

initializeBlockReadLazily

protected void initializeBlockReadLazily()
                                  throws IOException

Throws:

IOException

createBlockHeaderSerializer

protected BlockHeader.Serializer createBlockHeaderSerializer()

createBlockLineSerializer

protected BlockLine.Serializer createBlockLineSerializer()

createDeltaBaseTermStateSerializer

protected DeltaBaseTermStateSerializer createDeltaBaseTermStateSerializer()

readHeader

protected BlockHeader readHeader()
                          throws IOException

Reads the block header. Sets blockHeader.

Returns:

The block header; or null if there is no block for the field anymore.

Throws:

IOException

decodeBlockBytesIfNeeded

protected BytesRef decodeBlockBytesIfNeeded(int numBlockBytes)
                                     throws IOException

Throws:

IOException

readTermStateIfNotRead

protected BlockTermState readTermStateIfNotRead()
                                         throws IOException

Reads the BlockTermState if it is not already set. Sets termState.

Throws:

IOException

readTermState

protected BlockTermState readTermState()
                                throws IOException

Reads the BlockTermState on the current line. Sets termState.

Overriding method may return null if there is no BlockTermState (in this case the extending class must support a null termState).

Returns:

The BlockTermState; or null if none.

Throws:

IOException

term

public BytesRef term()

Description copied from class: TermsEnum

Returns current term. Do not call this when the enum is unpositioned.

Specified by:

term in class TermsEnum

ord

public long ord()

Description copied from class: TermsEnum

Returns ordinal position for current term. This is an optional method (the codec may throw UnsupportedOperationException). Do not call this when the enum is unpositioned.

Specified by:

ord in class TermsEnum

docFreq

public int docFreq()
            throws IOException

Description copied from class: TermsEnum

Returns the number of documents containing the current term. Do not call this when the enum is unpositioned. TermsEnum.SeekStatus.END.

Specified by:

docFreq in class TermsEnum

Throws:

IOException

totalTermFreq

public long totalTermFreq()
                   throws IOException

Description copied from class: TermsEnum

Returns the total number of occurrences of this term across all documents (the sum of the freq() for each doc that has this term). Note that, like other term measures, this measure does not take deleted documents into account.

Specified by:

totalTermFreq in class TermsEnum

Throws:

IOException

termState

public TermState termState()
                    throws IOException

Description copied from class: TermsEnum

Expert: Returns the TermsEnums internal state to position the TermsEnum without re-seeking the term dictionary.

NOTE: A seek by TermState might not capture the AttributeSource's state. Callers must maintain the AttributeSource states separately

Overrides:

termState in class BaseTermsEnum

Throws:

IOException

See Also:

• TermState
• TermsEnum.seekExact(BytesRef, TermState)

postings

public PostingsEnum postings(PostingsEnum reuse,
 int flags)
                      throws IOException

Description copied from class: TermsEnum

Get PostingsEnum for the current term, with control over whether freqs, positions, offsets or payloads are required. Do not call this when the enum is unpositioned. This method will not return null.

NOTE: the returned iterator may return deleted documents, so deleted documents have to be checked on top of the PostingsEnum.

Specified by:

postings in class TermsEnum

Parameters:

reuse - pass a prior PostingsEnum for possible reuse

flags - specifies which optional per-document values you require; see PostingsEnum.FREQS

Throws:

IOException

impacts

public ImpactsEnum impacts(int flags)
                    throws IOException

Description copied from class: TermsEnum

Return a ImpactsEnum.

Specified by:

impacts in class TermsEnum

Throws:

IOException

See Also:

• TermsEnum.postings(PostingsEnum, int)

ramBytesUsed

public long ramBytesUsed()

Description copied from interface: Accountable

Return the memory usage of this object in bytes. Negative values are illegal.

Specified by:

ramBytesUsed in interface Accountable

getOrCreateDictionaryBrowser

protected IndexDictionary.Browser getOrCreateDictionaryBrowser()
                                                        throws IOException

Throws:

IOException

clearTermState

protected void clearTermState()

Called by the primary TermsEnum methods to clear the previous TermState.

newCorruptIndexException

protected CorruptIndexException newCorruptIndexException(String msg,
 Long fp)