Module org.apache.lucene.core
Package org.apache.lucene.util.compress

Class LZ4

java.lang.Object
org.apache.lucene.util.compress.LZ4
public final class LZ4 extends Object
LZ4 compression and decompression routines.

https://github.com/lz4/lz4/tree/dev/lib http://fastcompression.blogspot.fr/p/lz4.html

The high-compression option is a simpler version of the one of the original algorithm, and only retains a better hash table that remembers about more occurrences of a previous 4-bytes sequence, and removes all the logic about handling of the case when overlapping matches are found.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final class 
    LZ4.FastCompressionHashTable
    Simple lossy LZ4.HashTable that only stores the last ocurrence for each hash on 2^14 bytes of memory.
    (package private) static class 
    LZ4.HashTable
    A record of previous occurrences of sequences of 4 bytes.
    static final class 
    LZ4.HighCompressionHashTable
    A higher-precision LZ4.HashTable.
    private static class 
    LZ4.Table
     
    private static class 
    LZ4.Table16
    16 bits per offset.
    private static class 
    LZ4.Table32
    32 bits per value, only used when inputs exceed 64kB, e.g.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    (package private) static final int
    HASH_LOG_HC
     
    (package private) static final int
    HASH_TABLE_SIZE_HC
     
    (package private) static final int
    LAST_LITERALS
     
    static final int
    MAX_DISTANCE
    Window size: this is the maximum supported distance between two strings so that LZ4 can replace the second one by a reference to the first one.
    (package private) static final int
    MEMORY_USAGE
     
    (package private) static final int
    MIN_MATCH
     

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    private
    LZ4()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    private static int
    commonBytes(byte[] b, int o1, int o2, int limit)
     
    static void
    compress(byte[] bytes, int off, int len, DataOutput out, LZ4.HashTable ht)
    Compress bytes[off:off+len] into out using at most 16kB of memory.
    static void
    compressWithDictionary(byte[] bytes, int dictOff, int dictLen, int len, DataOutput out, LZ4.HashTable ht)
    Compress bytes[dictOff+dictLen:dictOff+dictLen+len] into out using at most 16kB of memory.
    static int
    decompress(DataInput compressed, int decompressedLen, byte[] dest, int dOff)
    Decompress at least decompressedLen bytes into dest[dOff:].
    private static void
    encodeLastLiterals(byte[] bytes, int anchor, int literalLen, DataOutput out)
     
    private static void
    encodeLen(int l, DataOutput out)
     
    private static void
    encodeLiterals(byte[] bytes, int token, int anchor, int literalLen, DataOutput out)
     
    private static void
    encodeSequence(byte[] bytes, int anchor, int matchRef, int matchOff, int matchLen, DataOutput out)
     
    private static int
    hash(int i, int hashBits)
     
    private static int
    hashHC(int i)
     
    private static int
    readInt(byte[] buf, int i)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

    • LZ4

      private LZ4()

  • Method Details

    • hash

      private static int hash(int i, int hashBits)

    • hashHC

      private static int hashHC(int i)

    • readInt

      private static int readInt(byte[] buf, int i)

    • commonBytes

      private static int commonBytes(byte[] b, int o1, int o2, int limit)

    • decompress

      public static int decompress(DataInput compressed, int decompressedLen, byte[] dest, int dOff) throws IOException
      Decompress at least decompressedLen bytes into dest[dOff:]. Please note that dest must be large enough to be able to hold all decompressed data (meaning that you need to know the total decompressed length). If the given bytes were compressed using a preset dictionary then the same dictionary must be provided in dest[dOff-dictLen:dOff].
      Throws:
      IOException

    • encodeLen

      private static void encodeLen(int l, DataOutput out) throws IOException
      Throws:
      IOException

    • encodeLiterals

      private static void encodeLiterals(byte[] bytes, int token, int anchor, int literalLen, DataOutput out) throws IOException
      Throws:
      IOException

    • encodeLastLiterals

      private static void encodeLastLiterals(byte[] bytes, int anchor, int literalLen, DataOutput out) throws IOException
      Throws:
      IOException

    • encodeSequence

      private static void encodeSequence(byte[] bytes, int anchor, int matchRef, int matchOff, int matchLen, DataOutput out) throws IOException
      Throws:
      IOException

    • compress

      public static void compress(byte[] bytes, int off, int len, DataOutput out, LZ4.HashTable ht) throws IOException
      Compress bytes[off:off+len] into out using at most 16kB of memory. ht shouldn't be shared across threads but can safely be reused.
      Throws:
      IOException

    • compressWithDictionary

      public static void compressWithDictionary(byte[] bytes, int dictOff, int dictLen, int len, DataOutput out, LZ4.HashTable ht) throws IOException
      Compress bytes[dictOff+dictLen:dictOff+dictLen+len] into out using at most 16kB of memory. bytes[dictOff:dictOff+dictLen] will be used as a dictionary. dictLen must not be greater than 64kB, the maximum window size.

      ht shouldn't be shared across threads but can safely be reused.

      Throws:
      IOException