Package org.alfresco.repo.search.impl.lucene.analysis

Class MLTokenDuplicator

  • public class MLTokenDuplicator
extends org.apache.lucene.analysis.Tokenizer
    Create duplicate tokens for multilingual varients The forms are Tokens: Token - all languages {fr}Token - if a language is specified {fr_CA}Token - if a language and country is specified {fr_CA_Varient}Token - for all three {fr__Varient}Token - for a language varient with no country
    Author:
    andyh

    • Field Summary

      • Fields inherited from class org.apache.lucene.analysis.Tokenizer

        input

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.util.Iterator<org.apache.lucene.analysis.Token> buildIterator​(org.apache.lucene.analysis.Token token)  
      org.apache.lucene.analysis.Token next()  

      • Methods inherited from class org.apache.lucene.analysis.Tokenizer

        close, reset

      • Methods inherited from class org.apache.lucene.analysis.TokenStream

        next, reset

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • MLTokenDuplicator

        public MLTokenDuplicator​(org.apache.lucene.analysis.TokenStream source,
                         java.util.Locale locale,
                         java.io.Reader reader,
                         MLAnalysisMode mlAnalaysisMode)

      • MLTokenDuplicator

        public MLTokenDuplicator​(java.util.Locale locale,
                         MLAnalysisMode mlAnalaysisMode)

    • Method Detail

      • next

        public org.apache.lucene.analysis.Token next()
                                      throws java.io.IOException
        Overrides:
        next in class org.apache.lucene.analysis.TokenStream
        Throws:
        java.io.IOException

      • buildIterator

        public java.util.Iterator<org.apache.lucene.analysis.Token> buildIterator​(org.apache.lucene.analysis.Token token)