Iterating through a result set and updating is often failing to persist the update
if (actionObject.ActionType==ActionType.TradeComplete)
{
var results = _session.Query<Model.ActionObject>().Where(x => x.ActionType == ActionType.TradeRequest && x.ActionObjectId==actionObject.ActionObjectId);
foreach (var result in results)
{
result.State = State.Closed;
}
}
_session.Store(actionObject);
_session.SaveChanges();
Often, the object is not getting its state set to state.closed..
I see the discussion around patch commands, but there's little to no documentation on how to do that with a query that has multiple parameters..
Any idea why it isn't persisting?
Edit: my object does not have an ID, could that be the problem? It seems Raven is supposed to track the objects, and often is...
Related
I am new to RavenDB and could really use some help.
I have a collection of ~20M documents, and I need to add a key to each document. The challenge is that the value of the key needs to be derived from another document.
For instance, given the following document:
{
"Name" : "001A"
"Date" : "09-09-2013T00:00:00.0000000"
"Related" : [
"002B",
"003B"
]
}
The goal is to add a key that holds the dates for the related documents, i.e. 002B and 003B, by looking up the related documents in the collection and returning their date. E.g.:
{
"Name" : "001A"
"Date" : "09-09-2013T00:00:00.0000000"
"Related" : [
"002B",
"003B"
]
"RelatedDates" : [
"08-10-2013T00:00:00.0000000",
"08-15-2013T00:00:00.0000000"
]
}
I realize that I'm trying to treat the collection somewhat like a relational database, but this is the form that my data is in to begin with. I would prefer not to put everything into a relational dataset first in order to structure the data for RavenDB.
I first tried doing this on the client side, by paging through the collection and updating the records. However, I quickly reach the maximum number of request for the session.
I then tried patching on the server side with JavaScript, but I'm not sure if this is possible.
At this point I would greatly appreciate some strategic guidance on the right way to approach this problem, as well as, more tactical guidance on how to implement it.
The recommended way of doing this is via a Console application that loops thru all your records, similar to what you have already done but in a way that pages the data so you dont hit the maximum number of requests per session.
See this example from the ravendb source code example application:
you need to do something like this:
using (var store = new DocumentStore { ConnectionStringName = "RavenDB" }.Initialize())
{
int start = 0;
while (true)
{
using (var session = store.OpenSession())
{
var posts = session.Query<Post>()
.OrderBy(x => x.CreatedAt)
.Include(x => x.CommentsId)
.Skip(start)
.Take(128)
.ToList();
if (posts.Count == 0)
break;
foreach (var post in posts)
{
session.Load<PostComments>(post.CommentsId).Post = new PostComments.PostReference
{
Id = post.Id,
PublishAt = post.PublishAt
};
}
session.SaveChanges();
start += posts.Count;
Console.WriteLine("Migrated {0}", start);
}
}
}
I've done this sort of thing with about ~1.5M records and it wasnt exactly quick to do the migration. If your records are small then you can just Load<> and SaveChanges on each one as from experience programmatically patching the documents did not speed things up materially
As a side note, the ravendb google groups is very active if you want to ask specifically about doing this from the studio
I am using criteria to query the database based on the unique key. But I am coming through a weird scenario. After two or three queries, it starts giving me timeout expired error.
using (NHibernate.ISession session = m_SessionFactory.OpenSession())
{
using (ITransaction transacion = session.BeginTransaction())
{
if (cashActivity.ActivityState == ApplicationConstants.TaxLotState.Deleted || cashActivity.ActivityState == ApplicationConstants.TaxLotState.Updated)
{
IList<CashActivity> lsCActivity = RetrieveEquals<CashActivity>("UniqueKey",cashActivity.UniqueKey);
if (lsCActivity != null && lsCActivity.Count > 0)
cashActivity.CashActivityID = lsCActivity[0].CashActivityID;
}
if (cashActivity.ActivityState == ApplicationConstants.TaxLotState.Deleted)
{
session.Delete(cashActivity);
}
else
session.SaveOrUpdate(cashActivity);
}
}
}
public IList<T> RetrieveEquals<T>(string propertyName, object propertyValue)
{
using (Isession session = m_SessionFactory.OpenSession())
{
Icriteria criteria = session.CreateCriteria(typeof(T));
criteria.Add(Restrictions.Eq(propertyName, PropertyValue));
IList<T> matchingObjects = criteria.List<T>();
return matchingObjects;
}
}
I made changes in the code and start using StateLess Session but that change only reduces the frequency of timeout error.
After decugging , I found IList matchingObjects = criteria.List(); is cause of the exception. But this is only returning only one value, so it should not result timeout error since table also doesnt contain more than 100 rows as of now. Any Suggestions??
Unless you have wrapped NHibernate's ISessionFactory in something else, each call to OpenSession() will yield a new session. So the above code involves multiple sessions and it isn't clear if this is required.
Theoretically, a query on the session in RetrieveEquals() could block because of locks taken on the connection used in the calling method. But given the code as shown I can't see anything to prove this.
The calling method first updates a property of cashActivity, then in some cases goes on to delete the object. And there is no Commit(). This seems strange - is this really the used code or might there be a copy/paste error?
You also say "after two or three queries"... do you imply that there is a loop somewhere which isn't shown?
We seem to have verified that RavenDB is getting stale results even when we use various flavors of "WaitForNonStaleResults". Following is the fully-functional sample code (written as a standalone test so that you can copy/paste it and run it as is).
public class Cart
{
public virtual string Email { get; set; }
}
[Test]
public void StandaloneTestForPostingOnStackOverflow()
{
var testDocument = new Cart { Email = "test#abc.com" };
var documentStore = new EmbeddableDocumentStore { RunInMemory = true };
documentStore.Initialize();
using (var session = documentStore.OpenSession())
{
using (var transaction = new TransactionScope())
{
session.Store(testDocument);
session.SaveChanges();
transaction.Complete();
}
using (var transaction = new TransactionScope())
{
var documentToDelete = session
.Query<Cart>()
.Customize(x => x.WaitForNonStaleResultsAsOfLastWrite())
.First(c => c.Email == testDocument.Email);
session.Delete(documentToDelete);
session.SaveChanges();
transaction.Complete();
}
RavenQueryStatistics statistics;
var actualCount = session
.Query<Cart>()
.Statistics(out statistics)
.Customize(x => x.WaitForNonStaleResultsAsOfLastWrite())
.Count(c => c.Email == testDocument.Email);
Assert.IsFalse(statistics.IsStale);
Assert.AreEqual(0, actualCount);
}
}
We have tried every flavor of WaitForNonStaleResults and there is no change. Waiting for non-stale results seems to work fine for the update, but not for the delete.
Update
Some things which I have tried:
Using separate sessions for each action. Outcome: no difference. Same successes and fails.
Putting Thread.Current.Sleep(500) before the final query. Outcome: success. If I sleep the thread for a half second, the count comes back zero like it should.
Re: my comment above on stale results, AllowNonAuthoritiveInformation wasn't working. Needing to put WaitForNonStaleResults in each query, which is the usual "answer" to this issue, feels like a massive "code smell" (as much as I normally hate the term, it seems completely appropriate here).
The only real solution I've found so far is:
var store = new DocumentStore(); // do whatever
store.DatabaseCommands.DisableAllCaching();
Performance suffers accordingly, but I think slower performance is far less of a sin than unreliable if not outright inaccurate results.
This is an old question, but I recently ran across this problem as well. I was able to work around it by changing the convention on the DocumentStore used by the session to make it wait for non stale as of last write:
session.DocumentStore.DefaultQueryingConsistency = ConsistencyOptions.AlwaysWaitForNonStaleResultsAsOfLastWrite;
This made it so that I didn't have to customize every query run after. That said, I believe this only works for queries. It definitely doesn't work on patches as I have found out through testing.
I would also be careful about this and only use it around the code that's needed as it can cause performance issues. You can set the store back to its default with the following:
session.DocumentStore.DefaultQueryingConsistency = ConsistencyOptions.None;
The problem isn't related to deletes, it is related to using TransactionScope. The problem here is that DTC transaction complete in an asynchronous manner.
To fix this issue, what you need to do is call:
session.Advanced.AllowNonAuthoritiveInformation = false;
Which will force RavenDB to wait for the transaction to complete.
I am trying to find the perfect way to handle this exception and force client changes to overwrite any other changes that caused the conflict. The approach that I came up with is to wrap the call to Session.Transaction.Commit() in a loop, inside the loop I would do a try-catch block and handle each stale object individually by copying its properties, except row-version property then refreshing the object to get latest DB data then recopying original values to the refreshed object and then doing a merge. Once I loop I will commit and if any other StaleObjectStateException take place then the same applies. The loop keeps looping until all conflicts are resolved.
This method is part of a UnitOfWork class. To make it clearer I'll post my code:
// 'Client-wins' rules, any conflicts found will always cause client changes to
// overwrite anything else.
public void CommitAndRefresh() {
bool saveFailed;
do {
try {
_session.Transaction.Commit();
_session.BeginTransaction();
saveFailed = false;
} catch (StaleObjectStateException ex) {
saveFailed = true;
// Get the staled object with client changes
var staleObject = _session.Get(ex.EntityName, ex.Identifier);
// Extract the row-version property name
IClassMetadata meta = _sessionFactory.GetClassMetadata(ex.EntityName);
string rowVersionPropertyName = meta.PropertyNames[meta.VersionProperty] as string;
// Store all property values from client changes
var propertyValues = new Dictionary<string, object>();
var publicProperties = staleObject.GetType().GetProperties();
foreach (var p in publicProperties) {
if (p.Name != rowVersionPropertyName) {
propertyValues.Add(p.Name, p.GetValue(staleObject, null));
}
}
// Get latest data for staled object from the database
_session.Refresh(staleObject);
// Update the data with the original client changes except for row-version
foreach (var p in publicProperties) {
if (p.Name != rowVersionPropertyName) {
p.SetValue(staleObject, propertyValues[p.Name], null);
}
}
// Merge
_session.Merge(staleObject);
}
} while (saveFailed);
}
The above code works fine and handle concurrency with the client-wins rule. However, I was wondering if there is any built-in capabilities in NHibernate to do this for me or if there is a better way to handle this.
Thanks in advance,
What you're describing is a lack of concurrency checking. If you don't use a concurrency strategy (optimistic-lock, version or pessimistic), StaleStateObjectException will not be thrown and the update will be issued.
Okay, now I understand your use case. One important point is that the ISession should be discarded after an exception is thrown. You can use ISession.Merge to merge changes between a detached a persistent object rather than doing it yourself. Unfortunately, Merge does not cascade to child objects so you still need to walk the object graph yourself. So the implementation would look something like:
catch (StaleObjectStateException ex)
{
if (isPowerUser)
{
var newSession = GetSession();
// Merge will automatically get first
newSession.Merge(staleObject);
newSession.Flush();
}
}
Given 1000 documents with a complex data structure. for e.g. a Car class that has three properties, Make and Model and one Id property.
What is the most efficient way in C# to push these documents to raven db (preferably in a batch) without having to query the raven collection individually to find which to update and which to insert. At the moment I have to going like so. Which is totally inefficient.
note : _session is a wrapper on the IDocumentSession where Commit calls SaveChanges and Add calls Store.
private void PublishSalesToRaven(IEnumerable<Sale> sales)
{
var page = 0;
const int total = 30;
do
{
var paged = sales.Skip(page*total).Take(total);
if (!paged.Any()) return;
foreach (var sale in paged)
{
var current = sale;
var existing = _session.Query<Sale>().FirstOrDefault(s => s.Id == current.Id);
if (existing != null)
existing = current;
else
_session.Add(current);
}
_session.Commit();
page++;
} while (true);
}
Your session code doesn't seem to track with the RavenDB api (we don't have Add or Commit).
Here is how you do this in RavenDB
private void PublishSalesToRaven(IEnumerable<Sale> sales)
{
sales.ForEach(session.Store);
session.SaveChanges();
}
Your code sample doesn't work at all. The main problem is that you cannot just switch out the references and expect RavenDB to recognize that:
if (existing != null)
existing = current;
Instead you have to update each property one-by-one:
existing.Model = current.Model;
existing.Make = current.Model;
This is the way you can facilitate change-tracking in RavenDB and many other frameworks (e.g. NHibernate). If you want to avoid writing this uinteresting piece of code I recommend to use AutoMapper:
existing = Mapper.Map<Sale>(current, existing);
Another problem with your code is that you use Session.Query where you should use Session.Load. Remember: If you query for a document by its id, you will always want to use Load!
The main difference is that one uses the local cache and the other not (the same applies to the equivalent NHibernate methods).
Ok, so now I can answer your question:
If I understand you correctly you want to save a bunch of Sale-instances to your database while they should either be added if they didn't exist or updated if they existed. Right?
One way is to correct your sample code with the hints above and let it work. However that will issue one unnecessary request (Session.Load(existingId)) for each iteration. You can easily avoid that if you setup an index that selects all the Ids of all documents inside your Sales-collection. Before you then loop through your items you can load all the existing Ids.
However, I would like to know what you actually want to do. What is your domain/use-case?
This is what works for me right now. Note: The InjectFrom method comes from Omu.ValueInjecter (nuget package)
private void PublishSalesToRaven(IEnumerable<Sale> sales)
{
var ids = sales.Select(i => i.Id);
var existingSales = _ravenSession.Load<Sale>(ids);
existingSales.ForEach(s => s.InjectFrom(sales.Single(i => i.Id == s.Id)));
var existingIds = existingSales.Select(i => i.Id);
var nonExistingSales = sales.Where(i => !existingIds.Any(x => x == i.Id));
nonExistingSales.ForEach(i => _ravenSession.Store(i));
_ravenSession.SaveChanges();
}