When I run my program, I index the documents each time I run the program in eclipse. However, I want to just index once. Perhaps by deleting the index after each use, but I don't know how to go about doing that.
Set your IndexWriter to OpenMode.CREATE. It's probably set to OpenMode.CREATE_OR_APPEND now. Setting it to CREATE will cause the existing index at the specified directory to be overwritten when you open the indexwriter, to make way for the new one.
Like:
IndexWriterConfig config = new IndexWriterConfig(version, analyzer);
config.setOpenMode(IndexWriterConfig.OpenMode.CREATE);
//etc.....
IndexWriter writer = new IndexWriter(directory, config);
Related
I am using Lucene.Net.
On some clients' machines all the indexes are getting deleted when we are trying to add a new document.
Process:
Check if document already exists
If Exists delete
Else add a new document
Say I had 2000 indexes, now adding new one, it deletes all the indexed documents and just keeps the one which we have added recently. However, a few times it also deletes that.
It is only happening on a few customers' machines and we are also not able to reproduce on our environment.
var dir = new DirectoryInfo(_reportIndexDirectory);
_directory = FSDirectory.Open(dir);
_ireader = IndexReader.Open(_directory, true);
_iwriter = new IndexWriter(_directory, _analyzer, (!dir.Exists || dir.GetFiles().Length == 0) ? true : false, IndexWriter.MaxFieldLength.LIMITED);
_iwriter.SetMaxFieldLength(25000);
_iwriter.SetSimilarity(_similarityOne);
_iwriter.SetRAMBufferSizeMB(900);
Getting Lucene 4.10 read 3.2 version indexes
Upgraded to 4.10 still need to read 3.2 indexes. Deployed jre 7 as required. Made all changes within a existing code base which became erroneous. Still need to read 3.2 indexes before going to take on re-indexing. How to read existing 3.2 indexes by Lucene 4.10 ( what changes to make if any in a code )
You can use IndexUpgrader, something like:
IndexUpgrader upgrader = new IndexUpgrader(myIndexDirectory, Version.LUCENE_4_10_0);
upgrader.upgrade();
or run it from the command line:
java -cp lucene-core.jar org.apache.lucene.index.IndexUpgrader myIndexDirectory
You can set the codec used to decode the indexes in the IndexWriterConfig. Lucene3xCodec would be the codec to use here:
IndexWriterConfig config = new IndexWriterConfig(Version.LATEST, analyzer);
config.setCodec(new Lucene3xCodec());
IndexWriter writer = new IndexWriter(directory, config);
IndexSearcher searcher = new IndexSearcher(new DirectoryReader.open(writer));
Bear in mind, this codec is strictly read-only. Any attempt to add, delete, or update a document will result in an UnsupportedOperationException being thrown. If you wish to support writing to the index, you must upgrade your index (see my original answer).
how to reopen a closed indexWriter in lucene 3.2
and how to testify whether an indexWriter is closed?
When we create instance of IndexWriter, we should do like this way
IndexWriterConfig iwc = new IndexWriterConfig(analyzer);
iwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
If IndexWriterConfig.OpenMode.CREATE_OR_APPEND is used IndexWriter will create a new index if there is not already an index at the provided path and otherwise open the existing index.
The above is from : https://lucene.apache.org/core/4_6_0/core/org/apache/lucene/index/IndexWriter.html
I use Hibernate search for full text search in my web application. I have button for index creation in admin panel. I do it by this code:
fullTextSession.createIndexer()
.purgeAllOnStart(true)
.optimizeAfterPurge(true)
.optimizeOnFinish(true)
.batchSizeToLoadObjects( 25 )
.threadsToLoadObjects( 5 )
.threadsForSubsequentFetching( 20 )
.startAndWait();
If index was build correctly and then I push this button again old index files still on disk and program creates new index. And so on. Can you help me remove old index files before creating new?
I do a similar thing but only when I shut my app down do I remove the indexes and then I set them up again on startup.
Have you tried calling purgeAll() instead of purgeAllOnStart()? That is what I call on app shutdown and it works. To be safe, after I purge the indexes I actually delete my index directory and all files / folders it contains from disk as well.
I've learned from reading the available documentation that an IndexSearcher instance should be shared across searches, for optimal performance, and that a new instance must be created in order to load any changes made to the index. This implies that the index is writable (using IndexWriter) after having created an instance of IndexSearcher that points to the same directory. However, this is not the behaviour I see in my implementation of Lucene.Net. I'm using FSDirectory. RAMDirectory is not a viable option. The IndexSearcher locks one of the index files (in my implementation it's the _1.cfs file) making the index non-updatable during the lifetime of the IndexSearcher instance.
Is this a known behaviour? Can't I rebuild the index from scratch while using an IndexSearcher instance created prior to rebuilding? Is it only possible to to modifications to the index, but not to rebuild it?
Here is how I create the IndexSearcher instance:
// Create FSDirectory
var directory = FSDirectory.GetDirectory(storagePath, false);
// Create IndexReader
var reader = IndexReader.Open(directory);
// I get the same behaviour regardless of whether I close the directory or not.
directory.Close();
// Create IndexSearcher
var searcher = new IndexSearcher(reader);
// Closing the reader will cause "object reference not set..." when searching.
//reader.Close();
Here is how I create the IndexWriter:
var directory = FSDirectory.GetDirectory(storagePath, true);
var indexWriter = new IndexWriter(directory, new StandardAnalyzer(), true);
I'm using Lucene.Net version 2.0.
Edit:
Upgrading to Lucene.Net 2.1 (thx KenE) and slightly modifying the way I create my IndexWriter fixed the problem:
var directory = FSDirectory.GetDirectory(storagePath, false);
var indexWriter = new IndexWriter(directory, new StandardAnalyzer(), true);
The latest version of Lucene.Net (2.1) appears to support opening an IndexWriter with create=true even when there are open readers:
http://incubator.apache.org/lucene.net/docs/2.1/Lucene.Net.Index.IndexWriter.html
Earlier versions are not clear as to whether they support this or not. I would try using 2.1.