We're using NHibernate with Memcache as the second level cache. Occasionally there is a need for more advanced queries or bulk query operations. From the book Nhibernate in Action they recommend the following:
"It’s our view that ORM isn’t suitable for mass-update (or mass-delete) operations. If
you have a use case like this, a different strategy is almost always better: call a stored
procedure in the database, or use direct SQL UPDATE and DELETE statements for that
particular use case."
My concern is that queries against the underlying database do not reflect in the cache (at least until cache expiry) and I was wondering if anyone has come up with any effective strategies for mixing and matching NHibernate with custom SQL statements?
Is there any way of getting say a bulk Update statement (executed with custom sql) to reflect in the second level cache? I am aware of being able to manually evict, but this removes the items from cache and thefore increases hits on the database.
Does the community have any solutions that have been found to be effective in dealing with this problem?
As far as I know there is no method to keep the 2nd level cache up to date with massupdates. But you can partially evict the cache as described in: http://www.nhforge.org/doc/nh/en/index.html#performance-sessioncache.
Related
I have windows server 2008 r2 with microsoft sql server installed.
In my application, I am currently designing a tool for my users, that is querying database to see, if user has any notifications. Since my users can access the application multiple times in a short timespan, i was thinking about putting some kind of a cache on my query logic. But then I thought, that my ms sql server probably does that already for me. Am I right? Or do I need to configure something to make it happen? If it does, then for how long does it keep the cache up?
It's safe to assume that MSSQL will has the caching worked out pretty well =)
Don't bother trying to build anything yourself on top of it, simply make sure that the method you use to query for changes is efficient (eg. don't query on non-indexed columns).
PS: wouldn't caching locally defeat the whole purpose of checking for changes on the database?
Internally the database does all sorts of things, including 'caching', but at all times it works incredibly hard to make sure your users see up-to-date data. So it has to do some work each time your application makes a request.
If you want to reduce the workload by keeping static data in your application then you have to implement it yourself.
The later versions of the .net framework have caching features built in so you should take a look at those (building your own caching can get very complex).
SQL Server will handle caching for you, yes. When you create a query or a stored procedure SQL Server will cache that execution plan and reuse it accordingly. From MSDN:
SQL Server execution plans have the following main components: Query
Plan The bulk of the execution plan is a re-entrant, read-only data
structure used by any number of users. This is referred to as the
query plan. No user context is stored in the query plan. There are
never more than one or two copies of the query plan in memory: one
copy for all serial executions and another for all parallel
executions. The parallel copy covers all parallel executions,
regardless of their degree of parallelism.
Execution Context, each user that is currently executing the query has a data structure that holds
the data specific to their execution, such as parameter values. This
data structure is referred to as the execution context. The execution
context data structures are reused. If a user executes a query and one
of the structures is not being used, it is reinitialized with the
context for the new user.
If you wish to clear this cache you can execute sp_recompile or DBCC FREEPROCHCACHE
In the application which I am working on I use spring, hibernate and envers for auditing. envers works with calls like, hibernateTemplate.insert, hibernateTemplate.save, hibernateTemplate.saveOrUpdate. But it doesnt seem to work when i call hibernateTemplate.bulkUpdate.
I googled for solutions and found that envers doesnt support bulkUpdate.
A work around has been provided in the link below but i am not able to get it.
Envers Bulk insert/updates
It would be of help if someone can provide a workaround/sample for this.
Thanks
The documentation is correct. HQL and native SQL operations are not audited.
Since the performance of the bulk update will be affected by auditing, you may wish to change your design - for example, if you have a parent entity with related children, and you are performing a bulk update on the child records, you could update attributes of the parent record and then call saveOrUpdate after doing the bulk update.
Another option, is to manually perform the bulk update on the audit table(s) also, but its not going to be elegant. I managed to get around the issue in my case by changing the design as per the above.
I have a big SQL Server 2008 R2 database with many rows that are updated constantly. Updating is done by a back end service application that calls stored procedures. Within one of those stored procedures there is a SQL cursor that recalculates and updates data. This all runs fine.
But, our frontend web application needs to search through these rows and this search sometimes results in a
Lock request time out period exceeded. at
Telerik.OpenAccess.RT.Adonet2Generic.Impl.PreparedStatementImp.executeQuery()..
After doing some research I have found that the best way to make this query to run without problems is to make it run with "read uncommitted isolation level". I've found that this setting can be made in the Telerik OpenAccess settings, but that's a setting that affects the complete database ORM project. That's not what I want! I want this level for this query only.
Is there a way to make this specific LINQ query to run in this uncommitted isolation level?
Or can we make this one query to use a WITH NOLOCK hint?
Use
SET LOCK_TIMEOUT -1
in the beginning of your query.
See the reference manual
Runnung the queries in read uncommitted isolation level (and using NOLOCK hint) can cause many strange problems, you have to clearly understand why do you do this and how it can interfere with your dataflow
Does anyone know if NHibernate supports returning output parameters from stored procedures? I've had a search in the documentation but can't really find anything that confirms either way.
I was facing the same problem. NHibernate does not let you use stored procedures in this manner. But it does allow a way to make calls using the plain old ADO.NET API. Here's an example -
http://refactoringaspnet.blogspot.com/2009/06/how-to-use-legacy-stored-procedures-in.html
I can't officially confirm for you, but as far as I know, not directly. Stored procedure use in NHibernate is very specific to doing standard CRUD.
If you want to grab output paramaters (that are't the standard row count output parameter for INSERT, UPDATE, and DELETE), you could fall back to a different (or the standard) database access tools that give you direct access to SQL and the result set. (Assuming you can get by with bypassing NHibernate's cache. You'll want to make sure you flush NHibernate before you run the query too, etc.)
Given a small set of entities (say, 10 or fewer) to insert, delete, or update in an application, what is the best way to perform the necessary database operations? Should multiple queries be issued, one for each entity to be affected? Or should some sort of XML construct that can be parsed by the database engine be used, so that only one command needs to be issued?
I ask this because a common pattern at my current shop seems to be to format up an XML document containing all the changes, then send that string to the database to be processed by the database engine's XML functionality. However, using XML in this way seems rather cumbersome given the simple nature of the task to be performed.
It depends on how many you need to do, and how fast the operations need to run. If it's only a few, then doing them one at a time with whatever mechanism you have for doing single operations will work fine.
If you need to do thousands or more, and it needs to run quickly, you should re-use the connection and command, changing the arguments for the parameters to the query during each iteration. This will minimize resource usage. You don't want to re-create the connection and command for each operation.
You didn't mention what database you are using, but in SQL Server 2008, you can use table variables to pass complex data like this to a stored procedure. Parse it there and perform your operations. For more info, see Scott Allen's article on ode to code.
Most databases support BULK UPDATE or BULK DELETE operations.
From a "business entity" design standpoint, if you are doing different operations on each of a set of entities, you should have each entity handle its own persistence.
If there are common batch activities (like "delete all older than x date", for instance), I would write a static method on a collection class that executes the batch update or delete. I generally let entities handle their own inserts atomically.
The answer depends on the volume of data you're talking about. If you've got a fairly small set of records in memory that you need to synchronise back to disk then multiple queries is probably appropriate. If it's a larger set of data you need to look at other options.
I recently had to implement a mechanism where an external data feed gave me ~17,000 rows of dta that I needed to synchronise with a local table. The solution I chose there was to load the external data into a staging table and call a stored proc that did the synchronisation completely within the database.