RavenDb : Search occurrences in text is slow - ravendb

Question

I would like to find the occurrences of a word in a text.
I have a class like this
public class Page
{
public string Id { get; set; }
public string BookId { get; set; }
public string Content { get; set; }
public int PageNumber { get; set; }
}
I have my index like this :
class Pages_SearchOccurrence : AbstractIndexCreationTask<Page, Pages_SearchOccurrence.ReduceResult>
{
public class ReduceResult
{
public string PageId { get; set; }
public int Count { get; set; }
public string Word { get; set; }
public string Content { get; set; }
}
public Pages_SearchOccurrence()
{
Map = pages => from page in pages
let words = page.Content
.ToLower()
.Split(new string[] { " ", "\n", ",", ";" }, StringSplitOptions.RemoveEmptyEntries)
from w in words
select new
{
page.Content,
PageId = page.Id,
Count = 1,
Word = w
};
Reduce = results => from result in results
group result by new { PageId = result.PageId, result.Word } into g
select new
{
Content = g.First().Content,
PageId = g.Key.PageId,
Word = g.Key.Word,
Count = g.ToList().Count()
};
Index(x => x.Content, Raven.Abstractions.Indexing.FieldIndexing.Analyzed);
}
}
Finally, my query is like this :
using (var session = documentStore.OpenSession())
{
RavenQueryStatistics stats;
var occurence = session.Query<Pages_SearchOccurrence.ReduceResult, Pages_SearchOccurrence>()
.Statistics(out stats)
.Where(x => x.Word == "works")
.ToList();
}
But I realize that RavenDb is very slow (or my query is not good  )
stats.IsStale = true and raven studio take too much time and give only few results.
I have 1000 document “Pages” with a content of 1000 words per Page .
Why is my query not okay and how can I find the occurrences in a page ?
Thank you for your help!

Answer 1

You are doing it wrong. You should set the Content field as Analyzed and use RavenDB's Search() operator. The slowness is most likely because of the amount of un-optimized work your index code is doing.

Answer 2

I had found a partial result.
Perhaps I'm not clear : my goal is to find the occurrences of a word in the page.
I search the hits count of a word in the page and I would like to order by this count.
I changed my index like this :
class Pages_SearchOccurrence : AbstractIndexCreationTask<Page, Pages_SearchOccurrence.ReduceResult>{
public class ReduceResult
{
public string Content { get; set; }
public string PageId { get; set; }
public string Count { get; set; }
public string Word { get; set; }
}
public Pages_SearchOccurrence()
{
Map = pages => from page in pages
let words = page.Content.ToLower().Split(new string[] { " ", "\n", ",", ";" }, StringSplitOptions.RemoveEmptyEntries)
from w in words
select new
{
page.Content,
PageId = page.Id,
Count = 1,
Word = w
};
Index(x => x.Content, Raven.Abstractions.Indexing.FieldIndexing.Analyzed);
Index(x => x.PageId, Raven.Abstractions.Indexing.FieldIndexing.NotAnalyzed);
}
Finally, my new query looks like this :
using (var session = documentStore.OpenSession())
{
var query = session.Query<Pages_SearchOccurrence.ReduceResult, Pages_SearchOccurrence>()
.Search((x) => x.Word, "works")
.AggregateBy(x => x.PageId)
.CountOn(x => x.Count)
.ToList()
.Results
.FirstOrDefault();
var listFacetValues = query.Value.Values;
var finalResult = listFacetValues.GroupBy(x => x.Hits).OrderByDescending(x => x.Key).Take(5).ToList();
}
The finalResult gives me a group of Facetvalue which have a property Hits
( the properties Hits and Count of my FacetValue are the same here )
The Hits property gives me the result that I want but for me this code is not correct and ravendb studio doesn't like this too.
Do you have a better solution ?