The setup
Some of the "old old old" tables of our database use an exotic primary key generation scheme [1] and I'm trying to overlay this part of the database with NHibernate. This generation scheme is mostly hidden away in a stored procedure called, say, 'ShootMeInTheFace.GetNextSeededId'.
I have written an IIdentifierGenerator that calls this stored proc:
public class LegacyIdentityGenerator : IIdentifierGenerator, IConfigurable
{
// ... snip ...
public object Generate(ISessionImplementor session, object obj)
{
var connection = session.Connection;
using (var command = connection.CreateCommand())
{
SqlParameter param;
session.ConnectionManager.Transaction.Enlist(command);
command.CommandText = "ShootMeInTheFace.GetNextSeededId";
command.CommandType = CommandType.StoredProcedure;
param = command.CreateParameter() as SqlParameter;
param.Direction = ParameterDirection.Input;
param.ParameterName = "#sTableName";
param.SqlDbType = SqlDbType.VarChar;
param.Value = this.table;
command.Parameters.Add(param);
// ... snip ...
command.ExecuteNonQuery();
// ... snip ...
return ((IDataParameter)command
.Parameters["#sTrimmedNewId"]).Value as string);
}
}
The problem
I can map this in the XML mapping files and it works great, BUT....
It doesn't work when NHibernate tries to batch inserts, such as in a cascade, or when the session is not Flush()ed after every call to Save() on a transient entity that depends on this generator.
That's because NHibernate seems to be doing something like
for (each thing that I need to save)
{
[generate its id]
[add it to the batch]
}
[execute the sql in one big batch]
This doesn't work because, since the generator is asking the database every time, NHibernate just ends up getting the same ID generated multiple times, since it hasn't actually saved anything yet.
The other NHibernate generators like IncrementGenerator seem to get around this by asking the database for the seed value once and then incrementing the value in memory during subsequent calls in the same session. I would rather not do this in my implementation if I have to, since all of the code that I need is sitting in the database already, just waiting for me to call it correctly.
Is there a way to make NHibernate actually issue the INSERT after each call to generating an ID for entities of a certain type? Fiddling with the batch size settings don't seem to help.
Do you have any suggestions/other workarounds besides re-implementing the generation code in memory or bolting on some triggers to the legacy database? I guess I could always treat these as "assigned" generators and try to hide that fact somehow within the guts of the domain model....
Thanks for any advice.
The update: 2 months later
It was suggested in the answers below that I use an IPreInsertEventListener to implement this functionality. While this sounds reasonable, there were a few problems with this.
The first problem was that setting the id of an entity to the AssignedGenerator and then not actually assigning anything in code (since I was expecting my new IPreInsertEventListener implementation to do the work) resulted in an exception being thrown by the AssignedGenerator, since its Generate() method essentially does nothing but check to make sure that the id is not null, throwing an exception otherwise. This is worked around easily enough by creating my own IIdentifierGenerator that is like AssignedGenerator without the exception.
The second problem was that returning null from my new IIdentifierGenerator (the one I wrote to overcome the problems with the AssignedGenerator resulted in the innards of NHibernate throwing an exception, complaining that a null id was generated. Okay, fine, I changed my IIdentifierGenerator to return a sentinel string value, say, "NOT-REALLY-THE-REAL-ID", knowing that my IPreInsertEventListener would replace it with the correct value.
The third problem, and the ultimate deal-breaker, was that IPreInsertEventListener runs so late in the process that you need to update both the actual entity object as well as an array of state values that NHibernate uses. Typically this is not a problem and you can just follow Ayende's example. But there are three issues with the id field relating to the IPreInsertEventListeners:
The property is not in the #event.State array but instead in its own Id property.
The Id property does not have a public set accessor.
Updating only the entity but not the Id property results in the "NOT-REALLY-THE-REAL-ID" sentinel value being passed through to the database since the IPreInsertEventListener was unable to insert in the right places.
So my choice at this point was to use reflection to get at that NHibernate property, or to really sit down and say "look, the tool just wasn't meant to be used this way."
So I went back to my original IIdentifierGenreator and made it work for lazy flushes: it got the high value from the database on the first call, and then I re-implemented that ID generation function in C# for subsequent calls, modeling this after the Increment generator:
private string lastGenerated;
public object Generate(ISessionImplementor session, object obj)
{
string identity;
if (this.lastGenerated == null)
{
identity = GetTheValueFromTheDatabase();
}
else
{
identity = GenerateTheNextValueInCode();
}
this.lastGenerated = identity;
return identity;
}
This seems to work fine for a while, but like the increment generator, we might as well call it the TimeBombGenerator. If there are multiple worker processes executing this code in non-serializable transactions, or if there are multiple entities mapped to the same database table (it's an old database, it happened), then we will get multiple instances of this generator with the same lastGenerated seed value, resulting in duplicate identities.
##$##$#.
My solution at this point was to make the generator cache a dictionary of WeakReferences to ISessions and their lastGenerated values. This way, the lastGenerated is effectively local to the lifetime of a particular ISession, not the lifetime of the IIdentifierGenerator, and because I'm holding WeakReferences and culling them out at the beginning of each Generate() call, this won't explode in memory consumption. And since each ISession is going to hit the database table on its first call, we'll get the necessary row locks (assuming we're in a transaction) we need to prevent duplicate identities from happening (and if they do, such as from a phantom row, only the ISession needs to be thrown away, not the entire process).
It is ugly, but more feasible than changing the primary key scheme of a 10-year-old database. FWIW.
[1] If you care to know about the ID generation, you take a substring(len - 2) of all of the values currently in the PK column, cast them to integers and find the max, add one to that number, add all of that number's digits, and append the sum of those digits as a checksum. (If the database has one row containing "1000001", then we would get max 10000, +1 equals 10001, checksum is 02, resulting new PK is "1000102". Don't ask me why.
A potential workaround is to generate and assign the ID in an event listener rather than using an IIdentifierGenerator implementation. The listener should implement IPreInsertEventListener and assign the ID in OnPreInsert.
Why dont you just make private string lastGenerated; static?
Related
I have been assigned a task to verify the count of changes done using SaveChanges().
It is expected that the developer should know how many records will be changed before-hand when SaveChanges() will be called.
To implement it, I have created an extension method for DbContext called SaveChangesAndVerify(int expectedChangeCount) where I am using transaction and equating this parameter with the return value of SaveChanges().
If the values match, the transaction is committed and if it doesn't match, the transaction is rolled back.
Please check the code below and let me know if it would work and if there are any considerations that I need to make. Also, is there a better way to do this?
public static class DbContextExtensions
{
public static int SaveChangesAndVerify(this DbContext context, int expectedChangeCount)
{
context.Database.BeginTransaction();
var actualChangeCount = context.SaveChanges();
if (actualChangeCount == expectedChangeCount)
{
context.Database.CommitTransaction();
return actualChangeCount;
}
else
{
context.Database.RollbackTransaction();
throw new DbUpdateException($"Expected count {expectedChangeCount} did not match actual count {actualChangeCount} while saving the changes.");
}
}
public static async Task<int> SaveChangesAndVerifyAsync(this DbContext context, int expectedChangeCount, CancellationToken cancellationToken = default)
{
await context.Database.BeginTransactionAsync();
var actualChangeCount = await context.SaveChangesAsync();
if(actualChangeCount == expectedChangeCount)
{
context.Database.CommitTransaction();
return actualChangeCount;
}
else
{
context.Database.RollbackTransaction();
throw new DbUpdateException($"Expected count {expectedChangeCount} did not match actual count {actualChangeCount} while saving the changes.");
}
}
}
A sample usage would be like context.SaveChangesAndVerify(1) where a developer is expecting only 1 record to update.
Ok so some points.
Unless you've disabled it SaveChanges works as a transaction. Nothing will be changed if anything fails
Furthermore use context.ChangeTracker.Entries() and from there you can get the count of the number of the changed entities. So this will not require you handle transactions. Also SaveChanges() simply return the numbers of rows affected so it may not tell the full story.
Generally I dislike the idea of having this kind of check from a project architecture standpoint, increases complexity of code for dynamic changes and simply adds complexity without bringing any kind of security or safety. Data integrity and proper behavior should be validated using Unit test not those kinds of methods. For example you could add Unit Tests that validate that the rows that got changed are the same as those as you expected. But that should be test code. Not code that will be shipped to production
But if you need to do it dont use transaction and count the entities before changing anything as it is much cheaper. You can even use a "cheap" forloop so you can log what entities failed and so on. Furthermore since we are policing the developers you use extensions which means a developer can freely use the SaveChanges() as far as I can tell. You should create a new custom class for DbContext and expose only those 2 methods for saving changes.
I am attempting to use JMSSerializerBundle to consume JSON into Doctrine entities. I need to both create new entities where they do not already exist in the database, and update existing entities when they do already exist. I am using the DoctrineObjectConstructor included in the JMSSerializer package to help with this. When I consume JSON which contains a property designated as an identifier, such as:
{
"id": 1,
"some_other_attribute": "stuff"
}
by attempting to deserialize it, JMSSerializer causes warnings and eventually dies with an exception for attempting to utilize reflection to set properties on a null value. The warnings all look like this:
PHP Warning: ReflectionProperty::setValue() expects parameter 1 to be object, null given in /Users/cdonadeo/Repos/Ubertester/vendor/jms/serializer/src/JMS/Serializer/GenericDeserializationVisitor.php on line 176
If I manually insert an entity with ID 1 in my database and make another attempt then I receive no errors and everything appears to be working correctly, but I'm now short half my functionality. I looked at the code for the DoctrineObjectConstructor class, and at the top is a comment:
/**
* Doctrine object constructor for new (or existing) objects during deserialization.
*/
But I don't see how it could possibly create a new a new entity because after the construct() function has done all of its checks, at the end it calls:
$object = $objectManager->find($metadata->name, $identifierList);
And since the identifier does not exist in the database the result is null which is ultimately what gets returned from the function. This explains why inserting a row in the database with the appropriate ID makes things work: find() now returns a proper Entity object, which is what the rest of the library expects.
Am I using the library wrong or is it broken? I forked the Git repo and made an edit, and trying it out everything seems to work more or less the way I expected. That edit does have some drawbacks that make me wonder if I'm not just making this more difficult than it has to be. The biggest issue I see is that it will cause persisted and unpersisted entities to be mixed together with no way to tell which ones are which, but I don't know if that's even a big deal.
For Doctrine entities use configuration:
jms_serializer:
object_constructors:
doctrine:
fallback_strategy: "fallback" # possible values ("null" | "exception" | "fallback")
see configuration reference https://jmsyst.com/bundles/JMSSerializerBundle/master/configuration
I am taking over a project that was written by third party consultants who have already left.
I come from EF backgournd. One of the DAO class has the following which I find very hard to get my head around on details of what is exactly happening step by step. If anyone could kindly help me to understand this code section will be much appreciated.
return HibernateTemplate.Execute(
delegate(ISession hbSession) // <<--What is this code actually trying to do?
{
string queryText = "from {0} x where x.Code = :Code";
queryText = string.Format(queryText, typeof(Product));
IQuery query = hbSession.CreateQuery(queryText);
query.SetParameter("Code", productCode);
query.SetCacheable(true);
query.SetCacheRegion(CoreCacheConstants.ProductQueryCacheRegion); // <-- What is this code trying to do.
var fund = query.UniqueResult(); // <-- Is this similar to DISTINCT keyword in LINQ?
if (fund == null)
throw new ArgumentException(String.Format("No product found with productcode: {0}", productCode: ));
NHibernateUtil.Initialize(((Product)Product).Details); // <--What is this code trying to do. And where is the execute method for above queries.
return fund;
}
) as Product
Basically I am confused with delegate part and why delegate is being used instead of simple query to database. And what is the benefit of above approach.
Also I cant see any nHibernate ORM mapping xml. Does Spring.NET requires mapping files in order to pass data from/to data source?In order words how does ISession knows which database to connect to and which table to use etc
This is what in the spring documents is referred to as Classic Hibernate Usage. It is not the currently recommended approach to work with NHibernate, which is described in the chapter on object relational mappers.
The "convenient" usage of delegates here is basically done to provide the HibernateTemplate the means to manage a session and hand this managed session over to other custom methods (in this particular case an anonymous method). (I think it's an implementation of the visitor pattern, btw).
Using this approach, the classic HibernateTemplate can provide functionality to methods it doesn't "know of", such as opening and closing sessions correctly and participating in transactions.
The query is actually being executed by HibernateTemplate.Execute(myMethod); I imagine it creates and initializes a session for you, does transaction management, executes your method with the managed session and cleans everything up.
I've never used HibernateTemplate, but I'm sure it would require mapping files and a SessionFactory, so if this code is hit during execution and doesn't throw any exceptions, the configuration for those has to be there somewhere!
With respect to the questions (besides the delegate part) within your posted code: the use of NHibernateTemplate hasn't really got anything to do with it: you can just as well run this code in any piece of code where you've got hold of a valid ISession instance:
the session executes a HQL query; this query is added to the query cache. I've never used SetCacheRegion myself, but apparently it gives you "fine-grained control over query cache expiration policies".
query.UniqueResult
NHibernateUtil.Initialize
I have the following pseduo code in some Repository Pattern project that uses EF4.
public void Delete(int someId)
{
// 1. Load the entity for that Id. If there is none, then null.
// 2. If entity != null, then DeleteObject(..);
}
Pretty simple but I'm getting a run-time error:-
ConcurrencyException: Store, Update,
Insert or Delete statement affected an
unexpected number of rows (0).
Now, this is what is happening :-
Two instances of EF4 are running inthe app at the same time.
Instance A calls delete.
Instance B calls delete a nano second later.
Instance A loads the entity.
Instance B also loads the entity.
Instance A now deletes that entity - cool bananas.
Instance B tries to delete the entity, but it's already gone. As such, the no-count or what not is 0, when it expected 1 .. or something like that. Basically, it figured out that the item it is suppose to delete, didn't delete (because it happened a split sec ago).
I'm not sure if this is like a race-condition or something.
Anyways, is there any tricks I can do here so the 2nd call doesn't crash? I could make it into a stored procedure.. but I'm hoping to avoid that right now.
Any ideas? I'm wondering If it's possible to lock that row (and that row only) when the select is called ... forcing Instance B to wait until the row lock has been relased. By that time, the row is deleted, so when Instance B does it's select, the data is not there .. so it will never delete.
Normally you would catch the OptimisticConcurrencyException and then handle the problem in a manner that makes sense for your business model - then call SaveChanges again.
try
{
myContext.SaveChanges();
}
catch (OptimisticConcurrencyException e)
{
if (e.StateEntries.FirstOrDefault() is DeletingObject)
myContext.Detach(e.StateEntries.First();
myContext.SaveChanges();
}
Give that a whirl and see how you get on - not tested/compiled or anything - I've used DeletingObject as the type of the entity you're trying to delete. Substitute for your entity type.
If I call Save on a new object, then populate its properties, NHibernate generates and insert statement that contains only the default values for the properties. For example (session is an open ISession):
var homer = new Person();
session.Save(homer);
homer.Name = "Homer J. Simpson";
session.Flush();
I thought that calling Save would make homer persistent and that NH would track any changes and include them in the insert. Instead, it issues an insert with the name property parameter set to null. If I put the Save call after the assignment then it works. This object has a GUID id assigned by NH so it's not doing a premature insert to get an identity.
ETA I'm using session-per-request in an ASP.NET app and the pattern I want to follow is:
MyObject myObject;
if (id == null)
{
myObject = new MyObject();
repository.Add(myObject);
}
else
{
myObject = repository.GetMyObject(id);
}
// populate myObject's properties
// NH magic happens here when the HTTP request ends
I think your assumption in this case is simply incorrect.
Reading the code sample you provided, you could just as well expect NHibernate to insert the object, and then subsequently change the Name and then issue an Update. That, however, would assume that Flush implicitly saves the changed state.
I also wonder why this happens. NH should really wait to insert the object to the database.
Reasons why could do this:
the id, you already said that you are using guids, so this shouldn't be the reason.
there is a query. To ensure that it is performed on actual data, the session is flushed.
there are calculated columns, which need to be read back from the database
there might be other reasons I don't remember.
Is this really the code you are running to reproduce the test?
How does the mapping file look like?
You just mentioned it in the answer to my (perhaps rather naive) comment. You have set session FlushMode to Auto. Change that to Manual and you're more likely to see the behavior you are seeking.
It's still a rather wild guess, simply because so many other properties of your configuration can be at play.