I used Entity Framework to create a prototype for a project and now that it's working I want to make the program ready for production.
I face many challenges with EF, the biggest one being the concurrency management (it's a financial software).
Given that it seems to have no way to handle pessimistic concurrency with EF, I have to switch to stored procs in SQL.
To be honest I'm a bit afraid of the workload that may represent.
I would like to know if anybody have been in the same situation before and what is the best strategy to convert a .net code using EF to raw SQL.
Edit:
I'm investigating CLR but it's not clear if pessimistic concurency can be manage with it. is it an option more interesting than TSQl in this case ? It would allow me to reuse part of my C# code and structure of function calling another functions, if I understand well.
I was there and the good news is you don't have to give up Entity Framework if you don't want to. The bad news is you have to update the database yourself. Which isn't as hard as it seems. I'm currently using EF 5 but plan to go to EF 6. I don't see why this still wouldn't work for EF 6.
First thing is in the constructor of the DbContext cast it to IObjectContextAdapter and get access to the ObjectContext. I make a property for this
public virtual ObjectContext ObjContext
{
get
{
return ((IObjectContextAdapter)this).ObjectContext;
}
}
Once you have that subscribe to the SavingChanges event - this isn't our exact code some things are copied out of other methods and redone. This just gives you an idea of what you need to do.
ObjContext.SavingChanges += SaveData;
private void SaveData(object sender, EventArgs e)
{
var context = sender as ObjectContext;
if (context != null)
{
context.DetectChanges();
var tsql = new StringBuilder();
var dbParams = new List<KeyValuePair<string, object>>();
var deletedEntites = context.ObjectStateManager.GetObjectStateEntries(EntityState.Deleted);
foreach (var delete in deletedEntites)
{
// Set state to unchanged - so entity framework will ignore
delete.ChangeState(EntityState.Unchanged);
// Method to generate tsql for deleting entities
DeleteData(delete, tsql, dbParams);
}
var addedEntites = context.ObjectStateManager.GetObjectStateEntries(EntityState.Added);
foreach (var add in addedEntites)
{
// Set state to unchanged - so entity framework will ignore
add.ChangeState(EntityState.Unchanged);
// Method to generate tsql for added entities
AddData(add, tsql, dbParams);
}
var editedEntites = context.ObjectStateManager.GetObjectStateEntries(EntityState.Modified);
foreach (var edit in editedEntites)
{
// Method to generate tsql for updating entities
UpdateEditData(edit, tsql, dbParams);
// Set state to unchanged - so entity framework will ignore
edit.ChangeState(EntityState.Unchanged);
}
if (!tsql.ToString().IsEmpty())
{
var dbcommand = Database.Connection.CreateCommand();
dbcommand.CommandText = tsql.ToString();
foreach (var dbParameter in dbParams)
{
var dbparam = dbcommand.CreateParameter();
dbparam.ParameterName = dbParameter.Key;
dbparam.Value = dbParameter.Value;
dbcommand.Parameters.Add(dbparam);
}
var results = dbcommand.ExecuteNonQuery();
}
}
}
Why we set the entity to unmodified after the update because you can do
var changed properties = edit.GetModifiedProperties();
to get a list of all the changed properties. Since all the entities are now marked as unchanged EF will not send any updates to SQL.
You will also need to mess with the metadata to go from entity to table and property to fields. This isn't that hard to do but messing the metadata does take some time to learn. Something I still struggle with sometimes. I refactored all that out into an IMetaDataHelper interface where I pass it in the entity type and property name to get the table and field back - along with caching the result so I don't have to query metadata all the time.
At the end the tsql is a batch that has all the T-SQL how we want it with the locking hints and containing the transaction level. We also change numeric fields from just being set to nfield = 10 but to be nfield = nfield + 2 in the TSQL if the user updated them by 2 to avoid the concurrency issue as well.
What you wont get to is having SQL locked once someone starts to edit your entity but I don't see how you would get that with stored procedures as well.
All in all it took me about 2 solid days to get this all up and running for us.
Related
I am trying to update an object after retrieving it from a database.
This fires 2 queries , one for the select and the other for the update, is there any way of update an object using Fluent NHiberNate firing only one query ?
My code is as below:
var userProfile = userProfileRepository
.Find(x => x.ClientId == clientId)
.FirstOrDefault();
/* update UserProfile object here */
userProfileRepository.SaveOrUpdate(userProfile);
the SaveOrUpdate Method looks as such :
public bool SaveOrUpdate(T instance)
{
using (var session = SessionManager.OpenSession())
{
using (var transaction = session.BeginTransaction())
{
session.SaveOrUpdate(instance);
transaction.Commit();
}
return true;
}
}
In case that your issue is:
regardless of what I do, SaveOrUpdate() always sends SELECT then UPDATE
You should check the doc:
5.1.4.7. Assigned Identifiers
If you want the application to assign identifiers (as opposed to having NHibernate generate them), you may use the assigned generator. This special generator will use the identifier value already assigned to the object's identifier property. Be very careful when using this feature to assign keys with business meaning (almost always a terrible design decision).
Due to its inherent nature, entities that use this generator cannot be saved via the ISession's SaveOrUpdate() method. Instead you have to explicitly specify to NHibernate if the object should be saved or updated by calling either the Save() or Update() method of the ISession.
So, if your Fluent configuration sets the ID to be assigned - NHibernate has no other way then check if it
exists
or is new
because used method was SaveOrUpdate()
Solution(s)
1) Change the ID to be generated by DB or NHiberante 2) use explicit Update()
Are you trying to create an
UPDATE ... WHERE ...
statement?
AFAIK the NHibernate way to do this, is to select the appropriate objects (using the WHERE clause), update the fields, and persist them again.
var tempObjects = _session.Query<myObject>.Where(o => o.Id > 500);
// update proxy objects
foreach (var o in tempObjects)
{
o.MyValue = updatedValue;
}
// commit updated objects
_session.Update(tempObjects);
To be honest, we've used ISession.CreateSQLQuery ourselves. I hate using SQL in code because it breaks in refactoring, but if you must - here's how:
_session.CreateSQLQuery(
#"UPDATE [MyTable] SET [MyValue]=:updatedvalue WHERE Id > 500")
.SetParameter("updatedvalue", updatedValue)
.ExecuteUpdate();
I have a bunch of existing sagas in various states of a long running process.
Recently we decided to make one of the properties on our IContainSagaData implementation unique by using the Saga.UniqueAttribute (about which more here http://docs.particular.net/nservicebus/nservicebus-sagas-and-concurrency).
After deploying the change, we realized that all our old saga instances were not being found, and after further digging (thanks Charlie!) discovered that by adding the unique attribute, we were required to data fix all our existing sagas in Raven.
Now, this is pretty poor, kind of like adding a index to a database column and then finding that all the table data no longer select-able, but being what it is, we decided to create a tool for doing this.
So after creating and running this tool we've now patched up the old sagas so that they now resemble the new sagas (sagas created since we went live with the change).
However, despite all the data now looking right we're still not able to find old instances of the saga!
The tool we wrote does two things. For each existing saga, the tool:
Adds a new RavenJToken called "NServiceBus-UniqueValue" to the saga metadata, setting the value to the same value as our unique property for that saga, and
Creates a new document of type NServiceBus.Persistence.Raven.SagaPersister.SagaUniqueIdentity, setting the SagaId, SagaDocId, and UniqueValue fields accordingly.
My questions are:
Is it sufficient to simply make the data look correct or is there something else we need to do?
Another option we have is to revert the change which added the unique attribute. However in this scenario, would those new sagas which have been created since the change went in be OK with this?
Code for adding metadata token:
var policyKey = RavenJToken.FromObject(saga.PolicyKey); // This is the unique field
sagaDataMetadata.Add("NServiceBus-UniqueValue", policyKey);
Code for adding new doc:
var policyKeySagaUniqueId = new SagaUniqueIdentity
{
Id = "Matlock.Renewals.RenewalSaga.RenewalSagaData/PolicyKey/" + Guid.NewGuid().ToString(),
SagaId = saga.Id,
UniqueValue = saga.PolicyKey,
SagaDocId = "RenewalSaga/" + saga.Id.ToString()
};
session.Store(policyKeySagaUniqueId);
Any help much appreciated.
EDIT
Thanks to David's help on this we have fixed our problem - the key difference was we used the SagaUniqueIdentity.FormatId() to generate our document IDs rather than a new guid - this was trivial tio do since we were already referencing the NServiceBus and NServiceBus.Core assemblies.
The short answer is that it is not enough to make the data resemble the new identity documents. Where you are using Guid.NewGuid().ToString(), that data is important! That's why your solution isn't working right now. I spoke about the concept of identity documents (specifically about the NServiceBus use case) during the last quarter of my talk at RavenConf 2014 - here are the slides and video.
So here is the long answer:
In RavenDB, the only ACID guarantees are on the Load/Store by Id operations. So if two threads are acting on the same Saga concurrently, and one stores the Saga data, the second thread can only expect to get back the correct saga data if it is also loading a document by its Id.
To guarantee this, the Raven Saga Persister uses an identity document like the one you showed. It contains the SagaId, the UniqueValue (mostly for human comprehension and debugging, the database doesn't technically need it), and the SagaDocId (which is a little duplication as its only the {SagaTypeName}/{SagaId} where we already have the SagaId.
With the SagaDocId, we can use the Include feature of RavenDB to do a query like this (which is from memory, probably wrong, and should only serve to illustrate the concept as pseudocode)...
var identityDocId = // some value based on incoming message
var idDoc = RavenSession
// Look at the identity doc's SagaDocId and pull back that document too!
.Include<SagaIdentity>(identityDoc => identityDoc.SagaDocId)
.Load(identityDocId);
var sagaData = RavenSession
.Load(idDoc.SagaDocId); // Already in-memory, no 2nd round-trip to database!
So then the identityDocId is very important because it describes the uniqueness of the value coming from the message, not just any old Guid will do. So what we really need to know is how to calculate that.
For that, the NServiceBus saga persister code is instructive:
void StoreUniqueProperty(IContainSagaData saga)
{
var uniqueProperty = UniqueAttribute.GetUniqueProperty(saga);
if (!uniqueProperty.HasValue) return;
var id = SagaUniqueIdentity.FormatId(saga.GetType(), uniqueProperty.Value);
var sagaDocId = sessionFactory.Store.Conventions.FindFullDocumentKeyFromNonStringIdentifier(saga.Id, saga.GetType(), false);
Session.Store(new SagaUniqueIdentity
{
Id = id,
SagaId = saga.Id,
UniqueValue = uniqueProperty.Value.Value,
SagaDocId = sagaDocId
});
SetUniqueValueMetadata(saga, uniqueProperty.Value);
}
The important part is the SagaUniqueIdentity.FormatId method from the same file.
public static string FormatId(Type sagaType, KeyValuePair<string, object> uniqueProperty)
{
if (uniqueProperty.Value == null)
{
throw new ArgumentNullException("uniqueProperty", string.Format("Property {0} is marked with the [Unique] attribute on {1} but contains a null value. Please make sure that all unique properties are set on your SagaData and/or that you have marked the correct properties as unique.", uniqueProperty.Key, sagaType.Name));
}
var value = Utils.DeterministicGuid.Create(uniqueProperty.Value.ToString());
var id = string.Format("{0}/{1}/{2}", sagaType.FullName.Replace('+', '-'), uniqueProperty.Key, value);
// raven has a size limit of 255 bytes == 127 unicode chars
if (id.Length > 127)
{
// generate a guid from the hash:
var key = Utils.DeterministicGuid.Create(sagaType.FullName, uniqueProperty.Key);
id = string.Format("MoreThan127/{0}/{1}", key, value);
}
return id;
}
This relies on Utils.DeterministicGuid.Create(params object[] data) which creates a Guid out of an MD5 hash. (MD5 sucks for actual security but we are only looking for likely uniqueness.)
static class DeterministicGuid
{
public static Guid Create(params object[] data)
{
// use MD5 hash to get a 16-byte hash of the string
using (var provider = new MD5CryptoServiceProvider())
{
var inputBytes = Encoding.Default.GetBytes(String.Concat(data));
var hashBytes = provider.ComputeHash(inputBytes);
// generate a guid from the hash:
return new Guid(hashBytes);
}
}
}
That's what you need to replicate to get your utility to work properly.
What's really interesting is that this code made it all the way to production - I'm surprised you didn't run into trouble before this, with messages creating new saga instances when they really shouldn't because they couldn't find the existing Saga data.
I almost think it might be a good idea if NServiceBus would raise a warning any time you tried to find Saga Data by anything other than a [Unique] marked property, because it's an easy thing to forget to do. I filed this issue on GitHub and submitted this pull request to do just that.
Afaik Entity Framework 6 doesn't support for batch insert/update/delete.
Is there anyway to make an batch update over an IQueryable object. As an example I have
var query = _db.People.Where(x=>x.Name.Contains(parameter));
an IQueryable (query) object and I want to get the generated sql. Then I hope I can create an update command with this select query like this
Update filteredPerson
Set filteredPerson.Status = 'Updated'
from (here it comes IQueryable Generated SQL :) ) as filteredPerson
over DbContext raw sql execution commands. BTW I don't need EF properties like change tracking and auto detecting. It is just a batch operation.
I know it is pretty risky but I am going to use it for a small piece of code.
Some other logics are appricated. If you know something better, I would like to hear it.
REASON: Why I want to do it this way, because I don't want to spoil the seperation of layers. And there is some validation and filtering comes into the queryable object from other layers. So it is hard to convert it to stored procedure. At the other hand it must be faster than other standard queries.
Again I know there is no support in Entity Framework 6 for batch operations. But other questions are bit outdated. That's another reason why I want to ask this again.
While I was writing the question, I was guessing how I am going to solve it. But I was looking for some more proper way of it. In the end, I know what am I doing and tried to be simple for my colleagues who looking to the same code after me. I know it has some risky usages but I let the exceptions to CLR to handle it. After this excuse :) , I wrote the code like this:
Let's say I have an IQueryable object which is generated with this way:
string parameter = "John";
AdventureWorks2012Entities _db = new AdventureWorks2012Entities();
var query = _db.People.AsQueryable();
//Some parameters added from different layers
query = query.Where(x => x.FirstName.Contains(parameter));
Then I want a batch update over this IQueryable object.
var sqlFrom = query.ToString(); //This is the query which becomes "from table"
var dbq = query.AsDbQuery().GetObjectQuery(); //This does some magic with reflection
var linqParams = dbq.Parameters
.Select(x => new System.Data.SqlClient.SqlParameter(x.Name, x.Value)).ToList();
linqParams.Add(new System.Data.SqlClient.SqlParameter("#ModDate", DateTime.Now));
var sqlBatchUpdate = #"Update filteredPerson Set ModifiedDate = #ModDate From (#FilteredPerson) as filteredPerson"
.Replace("#FilteredPerson", sqlFrom);
var affectedRows = _db.Database.ExecuteSqlCommand(sqlBatchUpdate, linqParams.ToArray());
That's it! Now I don't have to repeat same business logic in stored procedure again. And it is more faster than a foreach and SaveChanges combo.
So I ended up with this for very basic usage. As a fast solution It brings more problems no doubt! But I know I can easily wrap around it for new purposes. So It is up to programmer who wants to use it with more preferences.
Also the code which does the reflection and casting is below and I added a gist for full code:
public static ObjectQuery<T> GetObjectQuery<T>(this DbQuery<T> query)
{
var internalQueryField = query.GetType()
.GetFields(BindingFlags.NonPublic | BindingFlags.Instance)
.Where(f => f.Name.Equals("_internalQuery"))
.FirstOrDefault();
var internalQuery = internalQueryField.GetValue(query);
var objectQueryField = internalQuery.GetType()
.GetFields(BindingFlags.NonPublic | BindingFlags.Instance)
.Where(f => f.Name.Equals("_objectQuery"))
.FirstOrDefault();
var objectQuery = objectQueryField.GetValue(internalQuery) as ObjectQuery<T>;
return objectQuery;
}
Here is the Gist file. Hope It helps somebody out there.
I've been banging my head off what should be a simple issue. I'm trying to do the following basic operation
1) Check if entity exists by a field other than ID
2) If not, create entity
Problem is this is in a console app that is multi-threaded, so I need to somehow get an entity by a field other than the ID and set the LockMode to Upgrade (or at least I think thats what needs to be done). From what I see there is no way to do that with ISession.
Any ideas?
in a single process use a global lockobject
lock(existsLocker)
{
var entity = session.Query<Entity>().Where(...).FirstOrDefault();
if (entity == null)
{
entity = new Entity();
session.Save(entity);
session.Flush();
}
}
All,
I have a requirement to hide my EF implementation behind a Repository. My simple question: Is there a way to execute a 'find' across both a DbSet AND the DbSet.Local without having to deal with them both.
For example - I have standard repository implementation with Add/Update/Remove/FindById. I break the generic pattern by adding a FindByName method (for demo purposes only :). This gives me the following code:
Client App:
ProductCategoryRepository categoryRepository = new ProductCategoryRepository();
categoryRepository.Add(new ProductCategory { Name = "N" });
var category1 = categoryRepository.FindByName("N");
Implementation
public ProductCategory FindByName(string s)
{
// Assume name is unique for demo
return _legoContext.Categories.Where(c => c.Name == s).SingleOrDefault();
}
In this example, category1 is null.
However, if I implement the FindByName method as:
public ProductCategory FindByName(string s)
{
var t = _legoContext.Categories.Local.Where(c => c.Name == s).SingleOrDefault();
if (t == null)
{
t = _legoContext.Categories.Where(c => c.Name == s).SingleOrDefault();
}
return t;
}
In this case, I get what I expect when querying against both a new entry and one that is only in the database. But this presents a few issues that I am confused over:
1) I would assume (as a user of the repository) that cat2 below is not found. But it is found, and the great part is that cat2.Name is "Goober".
ProductCategoryRepository categoryRepository = new ProductCategoryRepository();
var cat = categoryRepository.FindByName("Technic");
cat.Name = "Goober";
var cat2 = categoryRepository.FindByName("Technic");
2) I would like to return a generic IQueryable from my repository.
It just seems like a lot of work to wrap the calls to the DbSet in a repository. Typically, this means that I've screwed something up. I'd appreciate any insight.
With older versions of EF you had very complicated situations that could arise quite fast due to the required references. In this version I would recomend not exposing IQueryable but ICollections or ILists. This will contain EF in your repository and create a good seperation.
Edit: furthermore, by sending back ICollection IEnumerable or IList you are restraining and controlling the queries being sent to the database. This will also allow you to fine tune and maintain the system with greater ease. By exposing IQueriable, you are exposing yourself to side affects which occur when people add more to the query, .Take() or .Where ... .SelectMany, EF will see these additions and will generate sql to reflect these uncontrolled queries. Not confining the queries can result in queries getting executed from the UI and is more complicated tests and maintenance issues in the long run.
since the point of the repository pattern is to be able to swap them out at will. the details of DbSets should be completly hidden.
I think that you're on a good path. The only thing I probaly ask my self is :
Is the context long lived? if not then do not worry about querying Local. An object that has been Inserted / Deleted should only be accessible once it has been comitted.
if this is a long lived context and you need access to deleted and inserted objects then querying the Local is a good idea, but as you've pointed out, you may run into difficulties at some point.