Class IndexInfo

  • All Implemented Interfaces:
    IndexMonitor

    public class IndexInfo
    extends java.lang.Object
    implements IndexMonitor
    The information that makes up an index. IndexInfoVersion Repeated information of the form
    1. Index Type.
    2. sub-directory name.
    3. Status
      1. Indexes, sub indexes, and overlays must be committed. Status is ACTIVE, MERGING, COMPLETING_INDEX
      2. Delta: Transaction status
      3. Overlay: Transaction status
    Merges always take place to new indexes so we can detect merge failure or partial merges. Or we do not know what has merged. Incomplete delete merging does not matter - the overlay would still exist and be treated as such. So a document may be deleted in the index as well as in the applied overlay. It is still correctly deleted. NOTE: Public methods lock as required, the private methods assume that the appropriate locks have been obtained. TODO: Write element status into individual directories. This would be enough for recovery if both index files are lost or corrupted. TODO: Tidy up index status at start up or after some time. How long would you leave a merge to run?

    The index structure is duplicated to two files. If one is currupted the second is used.

    TODO:

    1. make the index sharing configurable
    2. use a thread pool for deletions, merging and index deletions
    3. something to control the maximum number of overlays to limit the number of things layered together for searching
    4. look at lucene locking again post 2.0, to see if it is improved
    5. clean up old data files (that are not old index entries) - should be a config option
    Author:
    Andy Hind
    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String MAIN_READER  
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void addApplicationListener​(org.springframework.context.ApplicationListener listener)
      Registers a listener for events on this index.
      void closeDelta​(java.lang.String id)
      Make sure the writer and reader for TX data are closed.
      void closeDeltaIndexReader​(java.lang.String id)
      Manage closing and unregistering an index reader.
      void closeDeltaIndexWriter​(java.lang.String id)
      Manage closing and unregistering an index writer .
      void delete​(java.lang.String deltaId)  
      static void destroy()  
      void destroyInstance()  
      <R> R doReadOnly​(IndexInfo.LockWork<R> lockWork)  
      long getActualSize()
      Gets the actual size of the index in bytes.
      java.util.Set<java.lang.String> getDeletions​(java.lang.String id)
      Get the deletions for a given index (there is no check if they should be applied that is up to the calling layer)
      org.apache.lucene.index.IndexReader getDeltaIndexReader​(java.lang.String id)
      This method should only be called from one thread as it is bound to a transaction.
      org.apache.lucene.index.IndexWriter getDeltaIndexWriter​(java.lang.String id, org.apache.lucene.analysis.Analyzer analyzer)
      Manage getting a lucene index writer for transactional data - looks after registration and checking there is no active reader.
      static IndexInfo getIndexInfo​(java.io.File file, LuceneConfig config)
      Get the IndexInfo object based in the given directory.
      org.apache.lucene.index.IndexReader getMainIndexReferenceCountingReadOnlyIndexReader()
      Get the main reader for committed index data
      org.apache.lucene.index.IndexReader getMainIndexReferenceCountingReadOnlyIndexReader​(java.lang.String id, java.util.Set<java.lang.String> deletions, java.util.Set<java.lang.String> containerDeletions, boolean deleteOnlyNodes)
      Get the main index reader augmented with the specified TX data As above but we add the TX data
      int getNumberOfDocuments()
      Gets the number of documents in the index.
      int getNumberOfFields()
      Gets the number of fields known to the index.
      int getNumberOfIndexedFields()
      Gets the number of indexed fields.
      java.lang.String getRelativePath()
      Gets the relative path of the index directory.
      java.util.Map<java.lang.String,​java.lang.Integer> getStatusSnapshot()
      Gets a snapshot of the statuses of the individual entries in this index.
      long getUsedSize()
      Gets the size used on disk by the index directory.
      static void main​(java.lang.String[] args)
      Helper to print out index information
      void setPreparedState​(java.lang.String id, java.util.Set<java.lang.String> toDelete, java.util.Set<java.lang.String> containersToDelete, long documents, boolean deleteNodesOnly)
      Set the aux data for the index entry for a transactional unit of work.
      void setStatus​(java.lang.String id, TransactionStatus state, java.util.Set<org.apache.lucene.index.Term> toDelete, java.util.Set<org.apache.lucene.index.Term> read)  
      java.lang.String toString()  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Method Detail

      • destroy

        public static void destroy()
      • destroyInstance

        public void destroyInstance()
      • delete

        public void delete​(java.lang.String deltaId)
      • getIndexInfo

        public static IndexInfo getIndexInfo​(java.io.File file,
                                             LuceneConfig config)
                                      throws IndexerException
        Get the IndexInfo object based in the given directory. There is only one object per directory per JVM.
        Parameters:
        file - File
        config - LuceneConfig
        Returns:
        IndexInfo
        Throws:
        IndexerException
      • getDeltaIndexReader

        public org.apache.lucene.index.IndexReader getDeltaIndexReader​(java.lang.String id)
                                                                throws java.io.IOException
        This method should only be called from one thread as it is bound to a transaction.
        Parameters:
        id - String
        Returns:
        IndexReader
        Throws:
        java.io.IOException
      • getDeltaIndexWriter

        public org.apache.lucene.index.IndexWriter getDeltaIndexWriter​(java.lang.String id,
                                                                       org.apache.lucene.analysis.Analyzer analyzer)
                                                                throws java.io.IOException
        Manage getting a lucene index writer for transactional data - looks after registration and checking there is no active reader.
        Parameters:
        id - String
        analyzer - Analyzer
        Returns:
        IndexWriter
        Throws:
        java.io.IOException
      • closeDeltaIndexReader

        public void closeDeltaIndexReader​(java.lang.String id)
                                   throws java.io.IOException
        Manage closing and unregistering an index reader.
        Parameters:
        id - String
        Throws:
        java.io.IOException
      • closeDeltaIndexWriter

        public void closeDeltaIndexWriter​(java.lang.String id)
                                   throws java.io.IOException
        Manage closing and unregistering an index writer .
        Parameters:
        id - String
        Throws:
        java.io.IOException
      • closeDelta

        public void closeDelta​(java.lang.String id)
                        throws java.io.IOException
        Make sure the writer and reader for TX data are closed.
        Parameters:
        id - String
        Throws:
        java.io.IOException
      • getDeletions

        public java.util.Set<java.lang.String> getDeletions​(java.lang.String id)
                                                     throws java.io.IOException
        Get the deletions for a given index (there is no check if they should be applied that is up to the calling layer)
        Parameters:
        id - String
        Throws:
        java.io.IOException
      • setPreparedState

        public void setPreparedState​(java.lang.String id,
                                     java.util.Set<java.lang.String> toDelete,
                                     java.util.Set<java.lang.String> containersToDelete,
                                     long documents,
                                     boolean deleteNodesOnly)
                              throws java.io.IOException
        Set the aux data for the index entry for a transactional unit of work.
        Parameters:
        id - - the tx id
        toDelete - - noderefs that should be deleted from previous indexes (not this one)
        documents - - the number of docs in the index
        deleteNodesOnly - - should deletions on apply to nodes (ie not to containers)
        Throws:
        java.io.IOException
      • getMainIndexReferenceCountingReadOnlyIndexReader

        public org.apache.lucene.index.IndexReader getMainIndexReferenceCountingReadOnlyIndexReader()
                                                                                             throws java.io.IOException
        Get the main reader for committed index data
        Returns:
        IndexReader
        Throws:
        java.io.IOException
      • getMainIndexReferenceCountingReadOnlyIndexReader

        public org.apache.lucene.index.IndexReader getMainIndexReferenceCountingReadOnlyIndexReader​(java.lang.String id,
                                                                                                    java.util.Set<java.lang.String> deletions,
                                                                                                    java.util.Set<java.lang.String> containerDeletions,
                                                                                                    boolean deleteOnlyNodes)
                                                                                             throws java.io.IOException
        Get the main index reader augmented with the specified TX data As above but we add the TX data
        Parameters:
        id - String
        deleteOnlyNodes - boolean
        Returns:
        IndexReader
        Throws:
        java.io.IOException
      • setStatus

        public void setStatus​(java.lang.String id,
                              TransactionStatus state,
                              java.util.Set<org.apache.lucene.index.Term> toDelete,
                              java.util.Set<org.apache.lucene.index.Term> read)
                       throws java.io.IOException
        Throws:
        java.io.IOException
      • main

        public static void main​(java.lang.String[] args)
                         throws java.lang.Throwable
        Helper to print out index information
        Parameters:
        args - String[]
        Throws:
        java.lang.Throwable
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object
      • getRelativePath

        public java.lang.String getRelativePath()
        Description copied from interface: IndexMonitor
        Gets the relative path of the index directory.
        Specified by:
        getRelativePath in interface IndexMonitor
        Returns:
        the relative path
      • getStatusSnapshot

        public java.util.Map<java.lang.String,​java.lang.Integer> getStatusSnapshot()
        Description copied from interface: IndexMonitor
        Gets a snapshot of the statuses of the individual entries in this index.
        Specified by:
        getStatusSnapshot in interface IndexMonitor
        Returns:
        a map of entry status names to entry counts
      • getActualSize

        public long getActualSize()
                           throws java.io.IOException
        Description copied from interface: IndexMonitor
        Gets the actual size of the index in bytes.
        Specified by:
        getActualSize in interface IndexMonitor
        Returns:
        the actual size in bytes
        Throws:
        java.io.IOException - Signals that an I/O exception has occurred.
      • getUsedSize

        public long getUsedSize()
                         throws java.io.IOException
        Description copied from interface: IndexMonitor
        Gets the size used on disk by the index directory. A large discrepancy from the value returned by IndexMonitor.getActualSize() may indicate that there are unused data files.
        Specified by:
        getUsedSize in interface IndexMonitor
        Returns:
        the size on disk in bytes
        Throws:
        java.io.IOException - Signals that an I/O exception has occurred.
      • getNumberOfDocuments

        public int getNumberOfDocuments()
                                 throws java.io.IOException
        Description copied from interface: IndexMonitor
        Gets the number of documents in the index.
        Specified by:
        getNumberOfDocuments in interface IndexMonitor
        Returns:
        the number of documents
        Throws:
        java.io.IOException - Signals that an I/O exception has occurred.
      • getNumberOfFields

        public int getNumberOfFields()
                              throws java.io.IOException
        Description copied from interface: IndexMonitor
        Gets the number of fields known to the index.
        Specified by:
        getNumberOfFields in interface IndexMonitor
        Returns:
        the number of fields
        Throws:
        java.io.IOException - Signals that an I/O exception has occurred.
      • getNumberOfIndexedFields

        public int getNumberOfIndexedFields()
                                     throws java.io.IOException
        Description copied from interface: IndexMonitor
        Gets the number of indexed fields.
        Specified by:
        getNumberOfIndexedFields in interface IndexMonitor
        Returns:
        the number of indexed fields
        Throws:
        java.io.IOException - Signals that an I/O exception has occurred.
      • addApplicationListener

        public void addApplicationListener​(org.springframework.context.ApplicationListener listener)
        Description copied from interface: IndexMonitor
        Registers a listener for events on this index.
        Specified by:
        addApplicationListener in interface IndexMonitor
        Parameters:
        listener - the listener