Ravendb Live projections produces null values (Ravendb Lucene, Multimap, Live projections) - lucene

Firstly, i am sorry for my english and i will be very happy if i can tell my problem very simply.
I spend so much time to solve multimap index and live projection problem. I read too much on stackoverflow, google, ayende blog etc... However couldn't solve my problem.
What i want:
I have an app and want a twitter like search which is the twitter search box searches from multiple sources, such as from twit content, user names and hashtags. While i get result, i want to apply transform on results and shape index result into FullSearchResult model. Also i want to find that where the result is found. In the post, in user or in tag?
The problem :
i have 3 type of docs (Post, User, Tag) and multimap index. When i create my multimap index with TransformResults i get all my results with null values. (I query my docs with multimap index by full text search).
My Docs
public class Post
{
public string Id { get; set; }
public long SqlDbId { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public string ContentAsHtml { get; set; }
public Status Status { get; set; }
public DenormalizedUser User { get; set; }
public DenormalizedTagCollection Tags { get; set; }
}
public class User
{
public string Id { get; set; }
public string Name { get; set; }
public string Surname { get; set; }
public string Email { get; set; }
public string MobileNumber { get; set; }
}
public class Tag
{
public string Id { get; set; }
public long SqlDbId { get; set; }
public string TagName { get; set; }
public DenormalizedUser TagInserterDenormalizedUser { get; set; }
public bool IsSystemTag { get; set; }
public Status Status { get; set; }
}
public class FullSearchIndex : AbstractMultiMapIndexCreationTask<FullSearchResult>
{
public FullSearchIndex()
{
AddMap<Post>(posts => from post in posts
let tags = post.Tags
where post.Status == Status.Active
select new
{
UserId = post.User != null ? post.User.Id.ToString() : (string)null,
PostId = post.Id,
TagIds = tags != null ? tags.Select(tag => tag.Id).ToArray() : new string[0],
SearchQuery = new object[]
{
post.Title,
post.Content,
post.Tags != null ? tags.Select(x => x.TagName).ToArray() : new string[0]
},
Source = SearchResultSource.ResultIsFromPost
});
AddMap<User>(users => from user in users
select new
{
UserId = user.Id,
PostId = (string)null,
TagIds = new string[0],
SearchQuery = new object[]
{
user.Name,
user.Surname
},
Source = SearchResultSource.ResultIsFromUser
});
AddMap<Tag>(tags => from tag in tags
where tag.Status == Status.Active
select new
{
UserId = (string)null,
PostId = (string)null,
TagIds = new string[] { tag.Id },
SearchQuery = new object[]
{
tag.TagName
},
Source = SearchResultSource.ResultIsFromTag
});
Index(searchResult => searchResult.SearchQuery, FieldIndexing.Analyzed);
TransformResults = (clientSideDatabase, results) =>
from result in results
let post = clientSideDatabase.Load<Post>(result.PostId)
let tags = clientSideDatabase.Load<Tag>(result.TagIds)
let user = clientSideDatabase.Load<User>(result.UserId)
select new
{
PostId = post != null ? post.Id : (string)null,
PostTitle = post != null ? post.Title : (string)null,
PostContent = post != null ? post.Content : (string)null,
PostTags = tags != null ? tags.Select(x => x.TagName).ToArray() : (string[])null,
UserId = user != null ? user.Id : (string)null,
UserName = user != null ? user.Name : (string)null,
UserSurname = user != null ? user.Surname : (string)null,
UserEmail = user != null ? user.Email : (string)null,
UserMobileNumber = user != null ? user.MobileNumber : (string)null
};
}
}
When i query using multimap index and lucene search i have 4 results. However, all values are null
query = "Tag50";
session.Query<FullSearchResult, FullSearchIndex>()
.Search(resultItem => resultItem.SearchQuery, query)
.As<FullSearchResultViewModel>()
.ToList();

Nkaya,
You assumed that the input of the TransformResults is the output of the Maps, but that isn't the case.
The input to TransformResults are the actual documents. The output of the maps is used to generate the index for searching ,not to shape how the input to the transform results looks like.

Related

Asp.Net Core EF Core many to many relation update command

I followed this link enter link description here to create many to many relationship. But, I do not know how to create and update Tag value to Post Object.
Any help would be appreciated.
Update, related code
class MyContext : DbContext
{
public DbSet<Post> Posts { get; set; }
public DbSet<Tag> Tags { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<PostTag>()
.HasKey(t => new { t.PostId, t.TagId });
modelBuilder.Entity<PostTag>()
.HasOne(pt => pt.Post)
.WithMany(p => p.PostTags)
.HasForeignKey(pt => pt.PostId);
modelBuilder.Entity<PostTag>()
.HasOne(pt => pt.Tag)
.WithMany(t => t.PostTags)
.HasForeignKey(pt => pt.TagId);
}
}
public class Post
{
public int PostId { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public List<PostTag> PostTags { get; set; }
}
public class Tag
{
public string TagId { get; set; }
public List<PostTag> PostTags { get; set; }
}
public class PostTag
{
public int PostId { get; set; }
public Post Post { get; set; }
public string TagId { get; set; }
public Tag Tag { get; set; }
}
Update2: Code for Update record
With below code, it will create records in both three tables.
var p = new Post { Content = "C1" };
var t = new Tag { TagId = "T1" };
var pt = new PostTag { Post = p, Tag = t };
_context.PostTag.Add(pt);
_context.SaveChanges();
But, with below code, it will insert new records in middle table PostTag instead of update the previous records.
var t1 = new Tag { TagId = "T3" };
var t2 = new Tag { TagId = "T4" };
var p =_context.Posts.Find(1);
p.PostTags = new List<PostTag>() {
new PostTag{ Post=p, Tag=t1},
new PostTag{ Post=p, Tag=t2}
};
_context.Posts.Update(p);
_context.SaveChanges();
Let's create some sample data
var p = new Post { ... };
var t = new Tag { ... };
var pt = new PostTag { Post = p, Tag = t };
This is still in-memory so we have to add it to the context, starting with:
db.Posts.Add(p);
db.Tags.Add(t);
Note that at this stage neither p nor t has any knowledge of pt. There are two ways to insert the third entity into your database.
1) The easiest way is to add the DbSet<PostTag> property and then the line
db.PostTags.Add(pt);
2) Without that extra DbSet you will need to ensure the linking from the other 2 sides. You also need to set up the List properties:
p.PostTags = new List<PostTag> { pt };
t.PostTags = new List<PostTag> { pt };
EF will now fill out any missing link fields.

Take function doesn't work and cannot be sent to RavenDB for query

Query Code:
var query = session.IndexQuery<App_OrgSearch.IndexResult, App_OrgSearch>();
var organizationUnitResults = query.Statistics(out stats)
.Skip(0)
.Take(5)
.AsProjection<Org>().ToList();
public static IRavenQueryable<TResult> IndexQuery<TResult, TIndex>(this IDocumentSession session)
where TIndex : AbstractIndexCreationTask, new()
{
return session.Query<TResult, TIndex>();
}
App_OrgSearch is the index I defined as below:
public class App_OrgSearch : AbstractIndexCreationTask<Org, App_OrgSearch.IndexResult>
{
public class IndexResult
{
public string Id { get; set; }
public string BusinessName { get; set; }
public string ShortName { get; set; }
public IList<string> Names { get; set; }
public List<string> PhoneNumbers { get; set; }
public List<OrganizationUnitPhone> OrganizationUnitPhones { get; set; }
}
public App_OrganizationUnitSearch()
{
Map = docs => from doc in docs
select new
{
Id = doc.Id,
Names = new List<string>
{
doc.BusinessName,
doc.ShortName,
},
BusinessName = doc.BusinessName,
ShortName = doc.ShortName,
PhoneNumbers = doc.OrganizationUnitPhones.Where(x => x != null && x.Phone != null).Select(x => x.Phone.Number),
};
Indexes.Add(x => x.Names, FieldIndexing.Analyzed);
}
}
I have 27 records in database. I want to take 5, but after query, all 27 records are returned. Why does Take function not work?
Your sample code seems wrong.
var query = session.IndexQuery<App_OrgSearch.IndexResult, App_OrgSearch>();
var organizationUnitResults = organizationUnitsQuery.Statistics(out stats)
What is organizationUnitsQuery ? You have the query as query, but there is no IndexQuery method on the session

inserting data in database table in one to many relationship in code first

I am using Code First Approach with Mvc4.For authentication and authorization, simple membership is being used.
My UserProfile class have these fields where a user can have multiple posts and comments.
[Table("UserProfile")]
public class UserProfile
{
public UserProfile()
{
this.PostComments = new HashSet<PostComment>();
this.Posts = new HashSet<Post>();
}
[Key]
[DatabaseGeneratedAttribute(DatabaseGeneratedOption.Identity)]
public int UserId { get; set; }
public string UserName { get; set; }
public string AvatarExt { get; set; }
public virtual ICollection<PostComment> PostComments { get; set; }
public virtual ICollection<Post> Posts { get; set; }
}
and my Post class is something like this.here i have configured one to many relationship between userprofile class and Post class.
public class Post
{
public Post()
{
this.PostComments = new HashSet<PostComment>();
}
[Key]
public int PostId { get; set; }
public string Message { get; set; }
public int PostedBy { get; set; }
public System.DateTime PostedDate { get; set; }
public virtual ICollection<PostComment> PostComments { get; set; }
public virtual UserProfile UserProfile { get; set; }
}
and this is my controller's action method for adding posts to the post table in the database.
public HttpResponseMessage PostPost(Post post)
{
post.PostedBy = WebSecurity.CurrentUserId;
post.PostedDate = DateTime.UtcNow;
// post.UserProfile.UserId = WebSecurity.CurrentUserId;
ModelState.Remove("post.PostedBy");
ModelState.Remove("post.PostedDate");
// ModelState.Remove("post.UserProfile.UserId");
if (ModelState.IsValid)
{
db.Posts.Add(post);
db.SaveChanges();
var usr = db.UserProfiles.FirstOrDefault(x => x.UserId == post.PostedBy);
var ret = new
{
Message = post.Message,
PostedBy = post.PostedBy,
PostedByName = usr.UserName,
PostedByAvatar = imgFolder + (String.IsNullOrEmpty(usr.AvatarExt) ? defaultAvatar : post.PostedBy + "." + post.UserProfile.AvatarExt),
PostedDate = post.PostedDate,
PostId = post.PostId
};
HttpResponseMessage response = Request.CreateResponse(HttpStatusCode.Created, ret);
response.Headers.Location = new Uri(Url.Link("DefaultApi", new { id = post.PostId }));
return response;
}
else
{
return Request.CreateErrorResponse(HttpStatusCode.BadRequest, ModelState);
}
}
when i tried to debug this action method, i found out that it is passing null at this line---
PostedByName = usr.UserName
what should i do to pass current loggedIn userName to the database.
One more information i want to share is--
In Sql Server, it is creating 5 columns----
PostId(which is primary key),
Message,PostedBy(int),PostedDate, UserProfile_UserId(Foreign key column).
Now, everything is working fine but in UserProfile_UserId column, it is storing null.
I know it should not be null but how.
What i am missing.what should i do.There was slight change in this code using dbFirst approach, and that code is working fine.
To get post on the view page, action method is----
public dynamic GetPosts()
{
var ret = (from post in db.Posts.ToList()
orderby post.PostedDate descending
select new
{
Message = post.Message,
PostedBy = post.PostedBy,
PostedByName = post.UserProfile.UserName,
PostedByAvatar = imgFolder + (String.IsNullOrEmpty(post.UserProfile.AvatarExt) ? defaultAvatar : post.PostedBy + "." + post.UserProfile.AvatarExt),
PostedDate = post.PostedDate,
PostId = post.PostId,
PostComments = from comment in post.PostComments.ToList()
orderby comment.CommentedDate
select new
{
CommentedBy = comment.CommentedBy,
CommentedByName = comment.UserProfile.UserName,
CommentedByAvatar = imgFolder + (String.IsNullOrEmpty(comment.UserProfile.AvatarExt) ? defaultAvatar : comment.CommentedBy + "." + comment.UserProfile.AvatarExt),
CommentedDate = comment.CommentedDate,
CommentId = comment.CommentId,
Message = comment.Message,
PostId = comment.PostId
}
}).AsEnumerable();
return ret;
}
this is the action method where i want to show post and comment with their UserName.here, PostedByName is returning null.

RavenDB - How to merge related docs of same type into an index projection

Assuming the following class:
public class Thing
{
public string Id { get; set; }
public string Title{ get; set; }
public string KeyId { get; set; }
public CultureInfo Culture { get; set; }
}
and the following result class:
public class ProjectedThing
{
public string Id { get; set; }
public string Title{ get; set; }
public string KeyId { get; set; }
public CultureInfo Culture { get; set; }
public IEnumerable<Thing> Things { get; set; }
}
How can I build an index that holds the result class?
The closest I've come is with the following index definition:
public class ProjectedThings : AbstractIndexCreationTask<Thing,ProjectedThing>
{
public ProjectedThings()
{
Map = docs => from doc in docs
select new
{
Title = doc.Title,
KeyId = doc.KeyId,
Culture = doc.Culture,
Things = new[] {
new Thing{
Id = doc.Id,
Title = doc.Title,
KeyId = doc.KeyId,
Culture = doc.Culture,
TitlePluralized = doc.TitlePluralized
}
}
};
Reduce = results => from r in results
group r by r.KeyId into g
select new
{
Title = g.FirstOrDefault(x => x.Id == x.KeyId).Title,
KeyId = g.Key,
Culture = g.FirstOrDefault(x => x.Id == x.KeyId).Culture,
Things = from thing in g.SelectMany(x => x.Things).Where(x => x.Id != x.KeyId)
select new
{
Id = thing.Id,
Title = thing.Title,
KeyId = thing.Key,
Culture = thing.Culture
}
};
}
}
That's almost doing the trick, but I can't collect the Title, KeyId, and Culture in the reduction. Only the Things property is being populated.
Look at your code:
g.FirstOrDefault(x => x.Id == x.KeyId)
I don't understand this, but this is likely to be always false.
You probably want:
g.FirstOrDefault().Title,

RavenDb : Search occurrences in text is slow

I would like to find the occurrences of a word in a text.
I have a class like this
public class Page
{
public string Id { get; set; }
public string BookId { get; set; }
public string Content { get; set; }
public int PageNumber { get; set; }
}
I have my index like this :
class Pages_SearchOccurrence : AbstractIndexCreationTask<Page, Pages_SearchOccurrence.ReduceResult>
{
public class ReduceResult
{
public string PageId { get; set; }
public int Count { get; set; }
public string Word { get; set; }
public string Content { get; set; }
}
public Pages_SearchOccurrence()
{
Map = pages => from page in pages
let words = page.Content
.ToLower()
.Split(new string[] { " ", "\n", ",", ";" }, StringSplitOptions.RemoveEmptyEntries)
from w in words
select new
{
page.Content,
PageId = page.Id,
Count = 1,
Word = w
};
Reduce = results => from result in results
group result by new { PageId = result.PageId, result.Word } into g
select new
{
Content = g.First().Content,
PageId = g.Key.PageId,
Word = g.Key.Word,
Count = g.ToList().Count()
};
Index(x => x.Content, Raven.Abstractions.Indexing.FieldIndexing.Analyzed);
}
}
Finally, my query is like this :
using (var session = documentStore.OpenSession())
{
RavenQueryStatistics stats;
var occurence = session.Query<Pages_SearchOccurrence.ReduceResult, Pages_SearchOccurrence>()
.Statistics(out stats)
.Where(x => x.Word == "works")
.ToList();
}
But I realize that RavenDb is very slow (or my query is not good  )
stats.IsStale = true and raven studio take too much time and give only few results.
I have 1000 document “Pages” with a content of 1000 words per Page .
Why is my query not okay and how can I find the occurrences in a page ?
Thank you for your help!
You are doing it wrong. You should set the Content field as Analyzed and use RavenDB's Search() operator. The slowness is most likely because of the amount of un-optimized work your index code is doing.
I had found a partial result.
Perhaps I'm not clear : my goal is to find the occurrences of a word in the page.
I search the hits count of a word in the page and I would like to order by this count.
I changed my index like this :
class Pages_SearchOccurrence : AbstractIndexCreationTask<Page, Pages_SearchOccurrence.ReduceResult>{
public class ReduceResult
{
public string Content { get; set; }
public string PageId { get; set; }
public string Count { get; set; }
public string Word { get; set; }
}
public Pages_SearchOccurrence()
{
Map = pages => from page in pages
let words = page.Content.ToLower().Split(new string[] { " ", "\n", ",", ";" }, StringSplitOptions.RemoveEmptyEntries)
from w in words
select new
{
page.Content,
PageId = page.Id,
Count = 1,
Word = w
};
Index(x => x.Content, Raven.Abstractions.Indexing.FieldIndexing.Analyzed);
Index(x => x.PageId, Raven.Abstractions.Indexing.FieldIndexing.NotAnalyzed);
}
Finally, my new query looks like this :
using (var session = documentStore.OpenSession())
{
var query = session.Query<Pages_SearchOccurrence.ReduceResult, Pages_SearchOccurrence>()
.Search((x) => x.Word, "works")
.AggregateBy(x => x.PageId)
.CountOn(x => x.Count)
.ToList()
.Results
.FirstOrDefault();
var listFacetValues = query.Value.Values;
var finalResult = listFacetValues.GroupBy(x => x.Hits).OrderByDescending(x => x.Key).Take(5).ToList();
}
The finalResult gives me a group of Facetvalue which have a property Hits
( the properties Hits and Count of my FacetValue are the same here )
The Hits property gives me the result that I want but for me this code is not correct and ravendb studio doesn't like this too.
Do you have a better solution ?