Just tracked down a bug relating to what appears to be a facet query being cached in RavenDB.
I found the line of code that sets aggresive caching but only for the given session. In this session, it appears the code is only getting a 'FacetSetup' doc and not executing a query on facets. However, I have confirmed that when the given line of code is commented out, I receive the current facet count and when I reinstate the comment and make an update, I receive the stale facet count.
My question is: What does this line actually tell Raven to cache?
/// <summary>
/// Gets all the facet names from the main facet document.
/// That way, we know all the possible facets to query over.
/// We will execute a facet query for each facet in the FacetSetup
/// </summary>
private IEnumerable<string> GetFacetNamesFromFacetSetup(IDocumentStore store, string facetDocumentName)
{
IList<string> facetFromDocument = new List<string>();
using (IDocumentSession docSession = store.OpenSession())
{
//The line below here!!
docSession.Advanced.DocumentStore.AggressivelyCacheFor(TimeSpan.FromMinutes(30));
FacetSetup facetSetup = docSession.Load<FacetSetup>(facetDocumentName);
foreach (Facet facet in facetSetup.Facets)
{
facetFromDocument.Add(facet.Name);
}
}
return facetFromDocument;
}
AggressivelyCacheFor is global for the document store.
And it returns an IDisposable.
Until you dispose it, all operations will be eagerly cached.
Related
If you do not have experience with or aren't currently using EntitySpaces ("ES") ORM this question is not meant for you.
I have a 10 year old application that after 4 years now needs my attention. My application uses a now defunct ORM called EntitySpaces and I'm hoping if you're reading this you have experience or maybe still use it too! Switching to another ORM is not an option at this time so I need to find a way to make this work.
Between the time I last actively worked on my application and now (ES Version 2012-09-30), EntitySpaces ("ES") has gone through a significant change in the underlying ADO.net back-end. The scenario that I'm seeking help on is when an entity collection is loaded with only a subset of the columns:
_products = new ProductCollection();
_products.Query.SelectAllExcept(_products.Query.ImageData);
_products.LoadAll();
I then override the properties that weren't loaded in the initial select so that I may lazyload them in the accessor. Here is an example of one such lazy-loaded property that used to work perfectly.
public override byte[] ImageData
{
get
{
bool rowIsDirty = base.es.RowState != DataRowState.Unchanged;
// Check if we have loaded the blob data
if(base.Row.Table != null && base.Row.Table.Columns.Contains(ProductMetadata.ColumnNames.ImageData) == false)
{
// add the column before we can save data to the entity
this.Row.Table.Columns.Add(ProductMetadata.ColumnNames.ImageData, typeof(byte[]));
}
if(base.Row[ProductMetadata.ColumnNames.ImageData] is System.DBNull)
{
// Need to load the data
Product product = new Product();
product.Query.Select(product.Query.ImageData).Where(product.Query.ProductID == base.ProductID);
if(product.Query.Load())
{
if (product.Row[ProductMetadata.ColumnNames.ImageData] is System.DBNull == false)
{
base.ImageData = product.ImageData;
if (rowIsDirty == false)
{
base.AcceptChanges();
}
}
}
}
return base.ImageData;
}
set
{
base.ImageData = value;
}
}
The interesting part is where I add the column to the underlying DataTable DataColumn collection:
this.Row.Table.Columns.Add(ProductMetadata.ColumnNames.ImageData, typeof(byte[]));
I had to comment out all the ADO.net related stuff from that accessor when I updated to the current (and open source) edition of ES (version 2012-09-30). That means that the "ImageData" column isn't properly configured and when I change it's data and attempt to save the entity I receive the following error:
Column 'ImageData' does not belong to table .
I've spent a few days looking through the ES source and experimenting and it appears that they no longer use a DataTable to back the entities, but instead are using a 'esSmartDictionary'.
My question is: Is there a known, supported way to accomplish the same lazy loaded behavior that used to work in the new version of ES? Where I can update a property (i.e. column) that wasn't included in the initial select by telling the ORM to add it to the entity backing store?
After analyzing how ES constructs the DataTable that is uses for updates it became clear that columns not included in the initial select (i.e. load) operation needed to be added to the esEntityCollectionBase.SelectedColumns dictionary. I added the following method to handle this.
/// <summary>
/// Appends the specified column to the SelectedColumns dictionary. The selected columns collection is
/// important as it serves as the basis for DataTable creation when updating an entity collection. If you've
/// lazy loaded a column (i.e. it wasn't included in the initial select) it will not be automatically
/// included in the selected columns collection. If you want to update the collection including the lazy
/// loaded column you need to use this method to add the column to the Select Columns list.
/// </summary>
/// <param name="columnName">The lazy loaded column name. Note: Use the {yourentityname}Metadata.ColumnNames
/// class to access the column names.</param>
public void AddLazyLoadedColumn(string columnName)
{
if(this.selectedColumns == null)
{
throw new Exception(
"You can only append a lazy-loaded Column to a partially selected entity collection");
}
if (this.selectedColumns.ContainsKey(columnName))
{
return;
}
else
{
// Using the count because I can't determine what the value is supposed to be or how it's used. From
// I can tell it's just the number of the column as it was selected: if 8 colums were selected the
// value would be 1 through 8 - ??
int columnValue = selectedColumns.Count;
this.selectedColumns.Add(columnName, columnValue);
}
}
You would use this method like this:
public override System.Byte[] ImageData
{
get
{
var collection = this.GetCollection();
if(collection != null)
{
collection.AddLazyLoadedColumn(ProductMetadata.ColumnNames.ImageData);
}
...
It's a shame that nobody is interested in the open source EntitySpaces. I'd be happy to work on it if I thought it had a future, but it doesn't appear so. :(
I'm still interested in any other approaches or insight from other users.
There are a number of questions on StackOverflow that talk about getting the Minification failed. Returning unminified contents error from the MVC4 minification.
I'd like to know if there is a way to be notified about this error when it happens and to be able to log it.
It is nice that when there is an error the bundler returns the original contents so my site doesn't break, but I would like to know about these errors automatically rather than having to visit each css/js bundle url to see if there is an error.
So that logic is actually in the implementation of the default transforms that Script/StyleBundle are using. If you want to catch those errors yourself, you can change the transforms on your bundles to something that surfaces those errors:
So to actually detect the errors, you would have to manually enumerate all of your bundles (to trigger them to get generated), and also be able to listen to errors that happened (so the GenerateErrorResponse equivalent below would need to report any errors to someplace that you would see)
Here's what JsMinify does in its process for reference:
/// <summary>
/// Transforms the bundle contents by applying javascript minification
/// </summary>
/// <param name="context">The <see cref="BundleContext"/> object that contains state for both the framework configuration and the HTTP request.</param>
/// <param name="response">A <see cref="BundleResponse"/> object containing the bundle contents.</param>
public virtual void Process(BundleContext context, BundleResponse response) {
if (!context.EnableInstrumentation) {
Minifier min = new Minifier();
// NOTE: Eval immediate treatment is needed for WebUIValidation.js to work properly after minification
// NOTE: CssMinify does not support important comments, so we are going to strip them in JS minification as well
string minifiedJs = min.MinifyJavaScript(response.Content, new CodeSettings() { EvalTreatment = EvalTreatment.MakeImmediateSafe, PreserveImportantComments = false });
if (min.ErrorList.Count > 0) {
GenerateErrorResponse(response, min.ErrorList);
}
else {
response.Content = minifiedJs;
}
}
response.ContentType = JsContentType;
}
Given a changeset c and given that c contains merge operations, I would like to get a list of all changesets that have been merged and resulted in c.
For example:
Changeset 1 contains some edits for some files.
Changeset 2 contains some edits for some other files.
Changeset 3 is a merge of changesets 1+2 to a parent branch.
Now I would like to get changesets 1+2 from asking changeset 3 which changeset merges it contained.
I want to do this using the TFS API. I came across the versionControlServer.TrackMerges method, however I do not understand what the ItemIdentifiers that the method expects should be. Unfortunately, I cannot find an example of how to use this method. Also I'm not sure if that is really the correct one.
Okay, it took me really long, but I think I found out how to do this. This is the Code that will find all the "parent" changesets:
/// <summary>
/// Gets the changesets which have resulted in the given changeset due
/// to a merge operation.
/// </summary>
/// <param name="changeset">The changeset.</param>
/// <param name="versionControlServer">The version control server.</param>
/// <returns>
/// A list of all changesets that have resulted into the given changeset.
/// </returns>
public static List<Changeset> GetMergedChangesets(Changeset changeset, VersionControlServer versionControlServer)
{
// remember the already covered changeset id's
Dictionary<int, bool> alreadyCoveredChangesets = new Dictionary<int, bool>();
// initialize list of parent changesets
List<Changeset> parentChangesets = new List<Changeset>();
// go through each change inside the changeset
foreach(Change change in changeset.Changes)
{
// query for the items' history
var queryResults = versionControlServer.QueryMergesExtended(
new ItemSpec(change.Item.ServerItem, RecursionType.Full),
new ChangesetVersionSpec(changeset.ChangesetId),
null,
null);
// go through each changeset in the history
foreach (var result in queryResults)
{
// only if the target-change is the given changeset, we have a hit
if (result.TargetChangeset.ChangesetId == changeset.ChangesetId)
{
// if that hit has already been processed elsewhere, then just skip it
if (!alreadyCoveredChangesets.ContainsKey(result.SourceChangeset.ChangesetId))
{
// otherwise add it
alreadyCoveredChangesets.Add(result.SourceChangeset.ChangesetId, true);
parentChangesets.Add(versionControlServer.GetChangeset(result.SourceChangeset.ChangesetId));
}
}
}
}
return parentChangesets;
}
Edit:
I just realized that there is a small "bug" in the above code, I used to write "VersionSpec.Latest" in the query, however: "new ChangesetVersionSpec(changeset.ChangesetId)" would be better, because then the changesets would also be tracked once the source branch has been deleted.
I think this page by a Ben Clark-Robinson answers the original question how to use the TrackMerges() API:
Here's a verified example:
using tfvcc = Microsoft.TeamFoundation.VersionControl.Client;
var sourcePath = "$/projectName/branchObjectName1";
var targetPath = "$/projectName/branchObjectName2";
versionCtl.TrackMerges(
sourceChangesetIds: new[] { 1000 },
sourceItem: new tfvcc.ItemIdentifier(sourcePath),
targetItems: new[] { new tfvcc.ItemIdentifier(targetPath) },
pathFilter: null)
I'm running two instances of my application. In one instance, I save one of my entities. When I check the RavenDB (http://localhost:8080/raven), I can see the change. Then, in my other client, I do this (below), but I don't see the changes from the other application. What do I need to do in order to get the most recent data in the DB?
public IEnumerable<CustomVariableGroup> GetAll()
{
return Session
.Query<CustomVariableGroup>()
.Customize(x => x.WaitForNonStaleResults());
}
Edit: The code above works if I try to make a change and get a concurrency exception. After that, when I call refresh (which invokes the above code), it works.
Here is the code that does the save:
public void Save<T>(T objectToSave)
{
Guid eTag = (Guid)Session.Advanced.GetEtagFor(objectToSave);
Session.Store(objectToSave, eTag);
Session.SaveChanges();
}
And here is the class that contains the Database and Session:
public abstract class DataAccessLayerBase
{
/// <summary>
/// Gets the database.
/// </summary>
protected static DocumentStore Database { get; private set; }
/// <summary>
/// Gets the session.
/// </summary>
protected static IDocumentSession Session { get; private set; }
static DataAccessLayerBase()
{
if (Database != null) { return; }
Database = GetDatabase();
Session = GetSession();
}
private static DocumentStore GetDatabase()
{
string databaseUrl = ConfigurationManager.AppSettings["databaseUrl"];
DocumentStore documentStore = new DocumentStore();
try
{
//documentStore.ConnectionStringName = "RavenDb"; // See app.config for why this is commented.
documentStore.Url = databaseUrl;
documentStore.Initialize();
}
catch
{
documentStore.Dispose();
throw;
}
return documentStore;
}
private static IDocumentSession GetSession()
{
IDocumentSession session = Database.OpenSession();
session.Advanced.UseOptimisticConcurrency = true;
return session;
}
}
Lacking more detailed information and some code, I can only guess...
Please make sure that you call .SaveChanges() on your session. Without explicitly specifiying an ITransaction your IDocumentSession will be isolated and transactional between it's opening and the call to .SaveChanges. Either all operations succeed or none. But if you don't call it all your previous .Store calls will be lost.
If I was wrong, please post more details about your code.
EDIT: Second answer (after additional information):
Your problem has to do with the way RavenDB caches on the client-side. RavenDB by default caches every GET request throughout a DocumentSession. Plain queries are just GET queries (and no, it has nothing to do wheter your index in dynamic or manually defined upfront) and therefore they will be cached. The solution in your application is to dispose the session and open a new one.
I suggest you rethink your Session lifecycle. It seems that your sessions live too long, otherwise this concurrency wouldn't be an issue. If you're building a web-application I recommend to open and close the session with the beginning and the end of your request. Have a look at RaccoonBlog to see it implemented elegantly.
Bob,
It looks like you have but a single session in the application, which isn't right. The following article talks about NHibernate, but the session management parts applies to RavenDB as well:
http://archive.msdn.microsoft.com/mag200912NHibernate
This code is meaningless:
Guid eTag = (Guid)Session.Advanced.GetEtagFor(objectToSave);
Session.Store(objectToSave, eTag);
It basically a no op, but one that looks important. You seems to be trying to work with a model where you have to manually manage all the saves, don't do that. You only need to manage things yourself when you create a new item, that is all.
As for the reason you get this problem, here is a sample:
var session = documentStore.OpenSession();
var post1 = session.Load<Post>(1);
// change the post by another client
post2 = session.Load<Post>(1); // will NOT go to the server, will give the same instance as post1
Assert.ReferenceEquals(post1,post2);
Sessions are short lived, and typically used in the scope of a single form / request.
I have an entity CustomerActivityReport which I'm trying to submit to the server via WCF. On the server end I'm using the repository + UOW patterns to update/insert the entity into the db.
CustomerActivityReport has a many to many relationship to another entity LookupValue. When I try and submit an instance of CustomerActivityReport, the DataContractSerializer throws the error: "Object graph for type 'FixupCollection[CustomerActivityReport]' contains cycles and cannot be serialized if reference tracking is disabled". I am getting this error even when I don't set the relationship on the LookupValue entities.
To get around this I've tried applying [DataContract(IsReference = true)] to both the entities in question and also to FixupCollection. But then I get different problems.
Has anybody else run into similar problems when trying to submit related entities over WCF?
Thanks in advance for any replies.
Ryan
The times that we have had a similar problem we were missing an attribute on a sub object.
I couldn't get this working with FixupCollection, and so I've had to submit all my entity collections as standard Collection, and then add logic server end to change them back into FixupCollection.
Client:
convertedCustomerActivityReport.LookupValues = new Collection<LookupValue>()
Server:
public virtual ICollection<LookupValue> LookupValues
{
get
{
if (_lookupValues == null || _lookupValues is Array)
{
var newCollection = new FixupCollection<LookupValue>();
newCollection.CollectionChanged += FixupLookupValues;
newCollection.AddRange(_lookupValues);
_lookupValues = newCollection;
}
return _lookupValues;
}
I've also added an AddRange method to FixupCollection:
/// <summary>
/// Adds multiple items.
/// </summary>
/// <param name="items">The items to add.</param>
public void AddRange(IEnumerable<T> items)
{
if (items == null)
{
return;
}
foreach (var item in items)
{
this.Add(item);
}
}