How to delete data in DB efficiently using LinQ to NHibernate (one-shot-delete) - sql

Producing software for customers, mostly using MS SQL but some Oracle, a decision was made to plunge into Nhibernate (and C#).
The task is to delete efficiently e.g. 10 000 rows from 100 000 and still stay sticked to ORM.
I've tried named queries - link already,
IQuery sql = s.GetNamedQuery("native-delete-car").SetString(0, "Kirsten");
sql.ExecuteUpdate();
but the best I have ever found seems to be:
using (ITransaction tx = _session.BeginTransaction())
{
try
{
string cmd = "delete from Customer where Id < GetSomeId()";
var count = _session.CreateSQLQuery(cmd).ExecuteUpdate();
...
Since it may not get into dB to get all complete rows before deleting them.
My questions are:
If there is a better way for this kind of delete.
If there is a possibility to get the Where condition for Delete like this:
Having a select statement (using LinQ to NHibernate) => which will generate appropriate SQL for DB => we get that Where condition and use it for Delete.

If there is a better way for this kind of delete.
Yes, you could use HQL instead of SQL.
If there is a possibility to get the Where condition for Delete [using Expressions]:
No, AFAIK that's not implemented. Since NHibernate is an open source project, I encourage you to find out if anyone has proposed this, and/or discuss it on the mailing list.

Thanks for your quick reply. Now I've probably got the difference.
session.CreateSQLQuery(cmd).ExecuteUpdate();
must have cmd with Delete From DbTable. On the contrary the HQL way
session.CreateQuery(cmd).ExecuteUpdate();
needs cmd with Delete From MappedCollectionOfObjects.
In that case it possibly solves my other question as well.

There now is a better way with NHibernate 5.0:
var biggestId = GetSomeId();
session.Query<Customer>()
.Where(c => c.Id < biggestId)
.Delete();
Documentation:
//
// Summary:
// Delete all entities selected by the specified query. The delete operation is
// performed in the database without reading the entities out of it.
//
// Parameters:
// source:
// The query matching the entities to delete.
//
// Type parameters:
// TSource:
// The type of the elements of source.
//
// Returns:
// The number of deleted entities.
public static int Delete<TSource>(this IQueryable<TSource> source);

Related

Is any method exist for QueryOver like Load for Get?

I'm using Fluent NHibernate and try to do many updates. First came in mind is code like this:
using (ISession s = OpenSession())
using (s.BeginTransaction())
{
IList<SomeType> items s.QueryOver<SomeType>()
.Where(someCondition)
.List();
items.ForEach(i => {
i.Foo = "bar";
s.Update(i);
});
s.Transaction.Commit();
}
But problem is in that every updating item must be loaded before do update. So database queried twice. In SQL it can be done with one query and i believe exist some way to do this with one query in NHibernate. And found doc for Load method that not actually load item from db and only work with some proxy instead and hit database only when i do update/delete.
Exist some method in NHibernate that load not items itself but proxy like Load?
For now for update large data best way is use HQL or raw SQL how mentioned #DavidOsborne.
I unfortunately not found any other methods that works lazy as Load method.

Linq order by with a field to retrieve dynamically in vb.net

I have a object Ob with several fields f1,..,fn (of different types).
Now a list of object is shown in a GridView and I need to implement the sorting method.
The real problem is:
how can I run
(from ob in Ob_list orderby ob.f1 ascending)
when the sorting field is represented by a string (i.e. "f1")?
Unfortunately I am not able to get it with the reflection (I am not able to do something like ob.GetType().GetField("f1"), this is not mapped into sql code).
I have several fields to possibly sort the rows, which is the best&fastest approach to this?
Thank you very much!
LINQ execution is deferred until you actually enumerate over the results or access the "count", etc. Because of this, you can build up your LINQ statement in stages.
The below code is done in C#, but I'm sure the equivalent is possible in VB.NET.
First setup your basic query:
var query = (from ob in Ob_list);
At this point, nothing has actually gone to the database due to deferred execution.
Next, conditionally add your order by components:
if (sortField == "f1")
{
query = query.OrderBy(o => o.f1);
}
else if (sortField == "f2")
{
query = query.OrderBy(o => o.f2);
}
else
{
//...
}
And finally, collect your results
foreach (var item in query)
{
// Process the item
}
I've found this question: How do I specify the Linq OrderBy argument dynamically?
I'm using Entity Framework, so the first answer did not solved my problem. The second one however, worked great!
Hope it helps!

Symfony2, Doctrine, add\insert\update best solution for big count of queries

Let's imagine we have this code:
while (true)
{
foreach($array as $row)
{
$item = $em->getRepository('reponame')->findOneBy(array('filter'));
if (!$item)
{
$needPersist = true;
$item = new Item();
}
$item->setItemName()
// and so on ...
if ($needPersist)
{
$em->persist();
}
}
$em->flush();
}
So, the point is that code will be executed a lot of times (while server won't die :) ). And we want to optimize it. Every time we:
Select already entry from repository.
If entry not exists, create it.
Set new (update) vars to it.
Apply actions (flush).
So question is - how to avoid unnecessary queries and optimize "check if entry is exist"? Because when there are 100-500 queries it's not so scary... But when it comes up to 1000-10000 for one while loop - it's too much.
PS: Each entry in DB is unique by several columns (not only by ID).
Instead of fetching results one-by-one, load all results with one query.
Eg.
let's say your filter wants to load ids 1, 2, 10. So QB would be something like:
$allResults = ...
->where("o.id IN (:ids)")->setParameter("ids", $ids)
->getQuery()
->getResults() ;
"foreach" of these results, do your job of updating them and flushing
While doing that loop, save ids of those fetched objects in new array
Compare that array with original one using array_diff. Now you have ids that were not fetched the first time
Rinse and repeat :)
And don't forget $em->clear() to free memory
While this can still be slow when working with 10.000 records (dunno, never tested), it will be much faster to have 2 big queries than 10.000 small ones.
Regardless if you need them to persist or not after the update, retrieving 10k+ and up entries from the database and hydrating them to php objects is going to need too much memory. In such cases you should better fallback to the Doctrine DBAL Layer and fire pure SQL queries.

Nhibernate QueryOver don't get latest database changes

I am trying get a record updated from database with QueryOver.
My code initially creates an entity and saves in database, then the same record is updated on database externally( from other program, manually or the same program running in other machine), and when I call queryOver filtering by the field changed, the query gets the record but without latest changes.
This is my code:
//create the entity and save in database
MyEntity myEntity = CreateDummyEntity();
myEntity.Name = "new_name";
MyService.SaveEntity(myEntity);
// now the entity is updated externally changing the name property with the
// "modified_name" value (for example manually in TOAD, SQL Server,etc..)
//get the entity with QueryOver
var result = NhibernateHelper.Session
.QueryOver<MyEntity>()
.Where(param => param.Name == "modified_name")
.List<T>();
The previous statement gets a collection with only one record(good), BUT with the name property established with the old value instead of "modified_name".
How I can fix this behaviour? First Level cache is disturbing me? The same problem occurs with
CreateCriteria<T>();
The session in my NhibernateHelper is not being closed in any moment due application framework requirements, only are created transactions for each commit associated to a session.Save().
If I open a new session to execute the query evidently I get the latest changes from database, but this approach is not allowed by design requirement.
Also I have checked in the NHibernate SQL output that a select with a WHERE clause is being executed (therefore Nhibernate hits the database) but don´t updates the returned object!!!!
UPDATE
Here's the code in SaveEntity after to call session.Save: A call to Commit method is done
public virtual void Commit()
{
try
{
this.session.Flush();
this.transaction.Commit();
}
catch
{
this.transaction.Rollback();
throw;
}
finally
{
this.transaction = this.session.BeginTransaction();
}
}
The SQL generated by NHibernate for SaveEntity:
NHibernate: INSERT INTO MYCOMPANY.MYENTITY (NAME) VALUES (:p0);:p0 = 'new_name'.
The SQL generated by NHibernate for QueryOver:
NHibernate: SELECT this_.NAME as NAME26_0_
FROM MYCOMPANY.MYENTITY this_
WHERE this_.NAME = :p0;:p0 = 'modified_name' [Type: String (0)].
Queries has been modified due to company confidential policies.
Help very appreciated.
As far as I know, you have several options :
have your Session as a IStatelessSession, by calling sessionFactory.OpenStatelesSession() instead of sessionFactory.OpenSession()
perform Session.Evict(myEntity) after persisting an entity in DB
perform Session.Clear() before your QueryOver
set the CacheMode of your Session to Ignore, Put or Refresh before your QueryOver (never tested that)
I guess the choice will depend on the usage you have of your long running sessions ( which, IMHO, seem to bring more problems than solutions )
Calling session.Save(myEntity) does not cause the changes to be persisted to the DB immediately*. These changes are persisted when session.Flush() is called either by the framework itself or by yourself. More information about flushing and when it is invoked can be found on this question and the nhibernate documentation about flushing.
Also performing a query will not cause the first level cache to be hit. This is because the first level cache only works with Get and Load, i.e. session.Get<MyEntity>(1) would hit the first level cache if MyEntity with an id of 1 had already been previously loaded, whereas session.QueryOver<MyEntity>().Where(x => x.id == 1) would not.
Further information about NHibernate's caching functionality can be found in this post by Ayende Rahien.
In summary you have two options:
Use a transaction within the SaveEntity method, i.e.
using (var transaction = Helper.Session.BeginTransaction())
{
Helper.Session.Save(myEntity);
transaction.Commit();
}
Call session.Flush() within the SaveEntity method, i.e.
Helper.Session.Save(myEntity);
Helper.Session.Flush();
The first option is the best in pretty much all scenarios.
*The only exception I know to this rule is when using Identity as the id generator type.
try changing your last query to:
var result = NhibernateHelper.Session
.QueryOver<MyEntity>()
.CacheMode(CacheMode.Refresh)
.Where(param => param.Name == "modified_name")
if that still doesn't work, try add this after the query:
NhibernateHelper.Session.Refresh(result);
After search and search and think and think.... I´ve found the solution.
The fix: It consist in open a new session, call QueryOver<T>() in this session and the data is succesfully refreshed. If you get child collections not initialized you can call HibernateUtil.Initialize(entity) or sets lazy="false" in your mappings. Take special care about lazy="false" in large collections, because you can get a poor performance. To fix this problem(performance problem loading large collections), set lazy="true" in your collection mappings and call the mentioned method HibernateUtil.Initialize(entity) of the affected collection to get child records from database; for example, you can get all records from a table, and if you need access to all child records of a specific entity, call HibernateUtil.Initialize(collection) only for the interested objects.
Note: as #martin ernst says, the update problem can be a bug in hibernate and my solution is only a temporal fix, and must be solved in hibernate.
People here do not want to call Session.Clear() since it is too strong.
On the other hand, Session.Evict() may seem un-applicable when the objects are not known beforehand.
Actually it is still usable.
You need to first retrieve the cached objects using the query, then call Evict() on them. And then again retrieve fresh objects calling the same query again.
This approach is slightly inefficient in case the object was not cached to begin with - since then there would be actually two "fresh" queries - but there seems to be not much to do about that shortcoming...
By the way, Evict() accepts null argument too without exceptions - this is useful in case the queried object is actually not present in the DB.
var cachedObjects = NhibernateHelper.Session
.QueryOver<MyEntity>()
.Where(param => param.Name == "modified_name")
.List<T>();
foreach (var obj in cachedObjects)
NhibernateHelper.Session.Evict(obj);
var freshObjects = NhibernateHelper.Session
.QueryOver<MyEntity>()
.Where(param => param.Name == "modified_name")
.List<T>()
I'm getting something very similar, and have tried debugging NHibernate.
In my scenario, the session creates an object with a couple children in a related collection (cascade:all), and then calls ISession.Flush().
The records are written into the DB, and the session needs to continue without closing. Meanwhile, another two child records are written into the DB and committed.
Once the original session then attempts to re-load the graph using QueryOver with JoinAlias, the SQL statement generated looks perfectly fine, and the rows are being returned correctly, however the collection that should receive these new children is found to have already been initialized within the session (as it should be), and based on that NH decides for some reason to completely ignore the respective rows.
I think NH makes an invalid assumption here that if the collection is already marked "Initialized" it does not need to be re-loaded from the query.
It would be great if someone more familiar with NHibernate internals could chime in on this.

NHibernate CreateCriteria query problem

I hope someone can help with this please.
I am trying to query an OLAP Fact table with NHibernate, but am struggling to get it to work. Its seems a simple requirement but I just cant see what the problem could be.
I have a central Fact table with several Dimension tables, one of the Dimensions has a secondary Dimension.
So ERD is. Fact >---1 Factor_Dim >---1 Target_Dim
My NHibernate query is.
facts = session.CreateCriteria(typeof(Fact), "facts")
.CreateAlias("facts.FactorDimension", "factDim", JoinType.InnerJoin)
.CreateAlias("factDim.TargetDimension", "targetDim",JoinType.InnerJoin)
.Add(Restrictions.Eq("targetDim.TargetID", targetId))
.List();
The error is "The multi-part identifier "targetdim2_.TargetID" could not be bound.". The generated SQL does not have the Factor_DIM or Target_DIM tables in the From clause.
Are there any alternative techniques to get this query to work? Id like to stick to this style as opposed to CreateSQLQuery() if possible.
Please help. Thanks.
Linq or QueryOver will be your cleanest solutions. If you are determined to stay with ICriteria you probably would want to wrap each of your entities with a class with common crud methods, it also makes your code access common, so code corrections are done in one place, not over hundres of files or classes.
Theres plenty of projects at http://nhforge.org/wikis/general/open-source-project-ecosystem.aspx which can help you out. I know NhGen ( http://sourceforge.net/projects/nhgen/ ) creates a CRUD class for each entity based on the NHibernate.Burrows GenericDao class with a few CRUD methods. It takes care of all the aliases and joins so queries become as simple as
IMessageDao messageDao = new MessageDao();
// Get All
IList<IMessage> messageList1 dao.FindAll();
// Find using QueryByExample
IList<IMessage> messageList2 = dao.FindByExample(messageDetails, orderBy)).ToList();
// Find using a simple entity query
IList<IMessage> messageList3 = messageDao.Find( new [] { Restrictions.Le(MessageHelper.Columns.Date, dateLastChecked) } );
// Find using a join and a query on said joined entities
IList<IMessage> messageList4 = messageDao.Find
( new []
{
Restrictions.Le(MessageHelper.Columns.Date, dateLastChecked),
Restrictions.Eq(MessageHelper.Columns.IsActive, true))
}, new[]
{
Restrictions.Eq(CategoryHelper.KeyColumns.Rsn, categoryRsn),
Restrictions.Eq(CategoryHelper.Columns.IsActive, true))
}, new []
{
Restrictions.Eq(ChannelHelper.KeyColumns.Rsn, channelRsn),
Restrictions.Eq(ChannelHelper.Columns.IsActive, true))
}
);
Theres plenty of overrides so you can specify your join type or it naturally assumes inner join.