AnalyzerUtil error on Lucene - lucene

Question

I'm learning to work with lucene. I wrote a simple program to test lucene analyzers like:
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.analysis.WhitespaceAnalyzer;
import org.apache.lucene.analysis.SimpleAnalyzer;
import org.apache.lucene.analysis.StopAnalyzer;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.util.Version;
import org.apache.lucene.wordnet.AnalyzerUtils;
import java.io.IOException;
public class AnalyzerDemo {
private static final String[] examples = {
"The quick brown fox jumped over the lazy dog",
"XY&Z Corporation - xyz#example.com"
};
private static final Analyzer[] analyzers = new Analyzer[] {
new WhitespaceAnalyzer(),
new SimpleAnalyzer(),
new StopAnalyzer(Version.LUCENE_30),
new StandardAnalyzer(Version.LUCENE_30)
};
public static void main(String[] args) throws IOException {
String[] strings = examples;
if (args.length > 0) {
strings = args;
}
for (String text : strings) {
analyze(text);
}
}
private static void analyze(String text) throws IOException {
System.out.println("Analyzing \"" + text + "\"");
for (Analyzer analyzer : analyzers) {
String name = analyzer.getClass().getSimpleName();
System.out.println(" " + name + ":");
System.out.print(" ");
AnalyzerUtils.displayTokens(analyzer, text);
System.out.println("\n");
}
}
}
but I got the following error:
AnalyzerDemo.java:7: package org.apache.lucene.wordnet does not exist
import org.apache.lucene.wordnet.AnalyzerUtils;
^
AnalyzerDemo.java:35: cannot find symbol
symbol : variable AnalyzerUtils
location: class AnalyzerDemo
AnalyzerUtils.displayTokens(analyzer, text);
^
Note: AnalyzerDemo.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2 errors
I think library wordnet or AnalyzerUtils is not available. How can I install this part of lucene? Do you have any ideas? Why is that missing? I've installed lucene 3.5.0.

Answer 1

lucene-wordnet contrib module was removed in Lucene 3.4.0. AnalyzerUtils also doesn't exist, so you either have to get Lucene 3.3.0 or write your own for 3.5.0 based on this one.

Answer 2

In case of word-net, the word-net contrib was removed from lucene 3.4.0 and the functionality was merged with analyzers contrib. Point number 4 at: http://apache.spinellicreations.com/lucene/java/3.4.0/changes-3.4.0/Contrib-Changes.html#3.4.0.new_features.
Java docs can be found for the same at: https://lucene.apache.org/core/old_versioned_docs/versions/3_5_0/api/all/org/apache/lucene/analysis/synonym/SynonymFilter.html

Answer 3

Instead of AnalyzerUtils.displayTokens(analyzer,text);
Use the function:
private static void displayTokens(Analyzer analyzer,String text) throws IOException
{
TokenStream stream=analyzer.tokenStream(null,new StringReader(text));
CharTermAttribute cattr = stream.addAttribute(CharTermAttribute.class);
stream.reset();
while (stream.incrementToken()){
System.out.print(cattr.toString()+" ");
}
stream.end();
stream.close();
}