How to optimize Lucene.Net indexing - lucene

Question

I need to index around 10GB of data. Each of my "documents" is pretty small, think basic info about a product, about 20 fields of data, most only a few words. Only 1 column is indexed, the rest are stored. I'm grabbing the data from text files, so that part is pretty fast.
Current indexing speed is only about 40mb per hour. I've heard other people say they have achieved 100x faster than this. For smaller files (around 20mb) the indexing goes quite fast (5 minutes). However, when I have it loop through all of my data files (about 50 files totalling 10gb), as time goes on the growth of the index seems to slow down a lot. Any ideas on how I can speed up the indexing, or what the optimal indexing speed is?
On a side note, I've noticed the API in the .Net port does not seem to contain all of the same methods as the original in Java...
Update--here are snippets of the indexing C# code:
First I set thing up:
directory = FSDirectory.GetDirectory(#txtIndexFolder.Text, true);
iwriter = new IndexWriter(directory, analyzer, true);
iwriter.SetMaxFieldLength(25000);
iwriter.SetMergeFactor(1000);
iwriter.SetMaxBufferedDocs(Convert.ToInt16(txtBuffer.Text));
Then read from a tab-delim data file:
using (System.IO.TextReader tr = System.IO.File.OpenText(File))
{
string line;
while ((line = tr.ReadLine()) != null)
{
string[] items = line.Split('\t');
Then create the fields and add the document to the index:
fldName = new Field("Name", items[4], Field.Store.YES, Field.Index.NO);
doc.Add(fldName);
fldUPC = new Field("UPC", items[10], Field.Store.YES, Field.Index.NO);
doc.Add(fldUPC);
string Contents = items[4] + " " + items[5] + " " + items[9] + " " + items[10] + " " + items[11] + " " + items[23] + " " + items[24];
fldContents = new Field("Contents", Contents, Field.Store.NO, Field.Index.TOKENIZED);
doc.Add(fldContents);
...
iwriter.AddDocument(doc);
Once its completely done indexing:
iwriter.Optimize();
iwriter.Close();

Answer 1

Apparently, I had downloaded a 3 yr old version of Lucene that is prominently linked to for some reason from the home page of the project...downloaded the most recent Lucene source code, compiled, used the new DLL, fixed about everything. The documentation kinda sucks, but the price is right and its real fast.
From a helpful blog
First things first, you have to add
the Lucene libraries to your project.
On the Lucene.NET web site, you’ll see
the most recent release builds of
Lucene. These are two years old. Do
not grab them, they have some bugs.
There has not been an official release
of Lucene for some time, probably due
to resource constraints of the
maintainers. Use Subversion (or
TortoiseSVN) to browse around and grab
the most recently updated Lucene.NET
code from the Apache SVN Repository.
The solution and projects are Visual
Studio 2005 and .NET 2.0, but I
upgraded the projects to Visual Studio
2008 without any issues. I was able
to build the solution without any
errors. Go to the bin directory, grab
the Lucene.Net dll and add it to your
project.

Answer 2

Since I can't comment on the marked answer above related to a 3 year old version, I would highly recommend installing the Visual Studio extension for NuGet Package Manager when adding Lucene.NET to your projects. It should add the most recent DLL version for you unless you need a specific later version.