I'm using Castle ActiveRecord in a warehouse project. I have several tables that get frequently updated: infeed, stack, stack_location. If a lot is happening (infeed is added, stacks are formed) sometimes the id that is set on an object after calling CreateAndFlush is the ID from another table. The database is MySQL, the ID columns are int(11) not null auto_increment primary key. I use PrimaryKeyType.Native for the ID property.
I also suffered from the following issue:
How to retrieve the last insert ID in Castle ActiveRecord?
but only with my infeed table (and I normally have something like 4 stacks / infeed). I added a catch to that situation with a 5000ms sleep and a SaveAndFlush afterwards, which makes sure I get an ID at that point.
I need to call flush directly because I need the ID to write it to a PLC which will hand it to me at a later point. My application is multi-threaded, but if I'm correct there's just one thread writing to the database when all of this goes wrong. I don't have anything marked as volatile, there are no locks in place preventing database access from multiple threads, but I assume Castle ActiveRecord does locking where appropiate.
As already mentioned in a comment on my question:
I hate to answer my own questions, but seems I've found that I'm doing it all wrong. Apparently, nHibernate is intentionally not thread-aware. Because of that I need to take care of the nHibernate sessions that are used inside different threads. With Castle ActiveRecord the solution seems to be to use 'using(new SessionScope())' blocks in threads to cater for this issue. If anyone has a better suggestion or a more clear explanation I'm obviously interested.
All issues I had have not returned anymore.
Related
There is a trigger in postgres that gets called whenever a particular table is updated.
It is used to send updates to another API.
Is there a way one can control the firing of this trigger?
Sometimes when I update the table I don't want the trigger to be fired. How do I do this?
Is there a silence trigger sql syntax?
If not
Can I fire triggers when a row is updated by PG user X and when PG user Y updates the table no trigger should be fired?
In recent Postgres versions, there is a when clause that you can use to conditionally fire the trigger. You could use it like:
... when (old.* is distinct from new.*) ...
I'm not 100% this one will work (can't test atm):
... when (current_user = 'foo') ...
(If not, try placing it in an if block in your plpgsql.)
http://www.postgresql.org/docs/current/static/sql-createtrigger.html
(There also is the [before|after] update of [col_name] syntax, but I tend to find it less useful because it'll fire even if the column's value remains the same.)
Adding this extra note, seeing that #CraigRinger's answer highlights what you're up to...
Trying to set up master-master replication between Salesforce and Postgres using conditional triggers is, I think, a pipe dream. Just forget it... There's going to be a lot more to it than that: you'll need to lock data as appropriate on both ends (which won't necessarily be feasible in a reasonable way), manage the resulting deadlocks (which might not automatically get detected), and deal with conflicting data.
Your odds of successfully pulling this off with a tiny team is about about zero -- especially if your Postgres skills are at the level where investing time in reading the manual would answer your own questions. You can safely bet that someone much more competent at Salesforce or some major SQL shop (e.g. like the one Craig works for) considered the same, and either miserably failed or ruled it out.
Moreover, I'd stress that implementing efficient, synchronous, multi-master replication is not a solved problem. You read that right: not solved. Just a few years ago, doing it at all wasn't well solved enough to make it in the Postgres core. So you've no prior art that works well to base your work on and iterate upon.
This seems to be the same problem as this post a few minutes ago, approaching it from a different direction.
If so, while you can indeed do as Denis suggests, don't attempt to reinvent this wheel. Use an established tool like Slony-I or Bucardo if you are attempting two-way (multi-master) replication. You also need to understand the major limitations involved in multi-master when dealing with conflicting updates.
In general, there are a few ways to control trigger firing:
Let the trigger fire, then put logic in the PL/PgSQL trigger body to cause it to take no action if a certain condition is met. This is often the only option when the rules are complex.
As Denis points out, use a trigger WHEN clause to conditionally fire the trigger
Use session_replication_role to control the firing of all triggers
Directly enable/disable triggers.
In particular, if your application shares a single SQL-level user ID for all database access and does its own user management above the SQL level, and you want to control trigger firing on a per-user basis, the only way to do it will be with in-trigger logic. You might find this prior answer about getting user IDs within triggers useful:
Passing user id to PostgreSQL triggers
We're building a large application with NHibernate as ORM layer. We've tried to apply as many best practices as possible, among which setting FlushMode to Never. However, this is giving us pain, for example the following scenario:
There is a table with an end date column. From this table, we delete the last (by end date) record:
The record is deleted;
After the delete, we do a (repository) query for the last record (by end date);
This last record is updated because it's the new active record.
This is a very simple scenario, of which many exist. The problem here is that when we do the query, we get the deleted record back, which of course is not correct. This roughly means that we cannot do queries in business logic that may touch the entity being inserted or deleted, because its resp. not there yet or still there.
How can I work with this scenario? Are there ways to work around this without reverting the FlushMode setting or should I just give up on the FlushMode setting all together?
How can I work with this scenario? Are there ways to work around this
without reverting the FlushMode setting
FlushMode.Never does not prevent you from manually calling Flush() when you want to deal with up-to-date data. I guess it is the way to work this scenario without changing the FlushMode
or should I just give up on the FlushMode setting all together?
Could you provide some reference on FlushMode.Never being a good practice in the general case ? Seems like FlushMode.Never is fit when dealing with large, mostly readonly, sets of objects.
http://jroller.com/tfenne/entry/hibernate_understand_flushmode_never
FlushMode.Never is a best practice only when you absolutely require fine-grained control. FlushMode.Auto will cover 99.99% of the cases without a problem. That said, decorating you CUD operations with a ISession.FLush() will not hurt as it only involves a database roundtrip if there are any CUD actions in the internal action queue
Flush mode Never means NHibernate will never flush the session, it's up to you to do that. So, session.Delete() will not actually delete the record from database, just mark the object for delete in session's cache. You can force a flush by calling session.Flush() after calling session.Delete().
I think Auto is a better option, with Auto, NHibernate will flush the session automatically before querying for data.
I want to avoid generating duplicate numbers in my system.
CreateNextNumber() will:
Find the last number created.
Increment the value by one.
Update the value in the database with the new number.
I want to avoid two clients calling this method at the same time. My fear is they will pull the same last number created, increment it by one, and return the duplicate number for both clients.
Questions:
Do I need to use single mode here? I'd rather use Per Call if possible.
The default concurrency mode is single. I don't understand how Per Call would create multiple instances, yet have a single thread. Does that mean that even though multiple instances are created, only one client at a time can call a method in their instance?
If you use InstanceContextMode.Single and ConcurrentcyMode.Single your service will handle one request at a time and so would give you this feature - however, this issue is better handled in the database
Couple of options:
make the field that requires the unique number an identity column and the database will ensure no duplicate values
Wrap the incrementing of the control value in a stored procedure that uses isolation level RepeatableRead and read, increment and write in a transaction
for your questions you might find my blog article on instancing and concurrency useful
Single instance will not stop the service from handling requests concurrently, I don't think. You need a server side synchronisation mechanism, such as a Mutex, so that all code that tries to get this number first locks. You might get away with a static locking object inside the service code actually, which will likely be simpler than a mutex.
Basically, this isn't a WCF configuration issue, this is a more general concurrency issue.
private static object ServiceLockingObject = new object();
lock (ServiceLockingObject)
{
// Try to increment your number.
}
Don't bother with WCF settings, generate unique numbers in the database instead. See my answer to this question for details. Anything you try to do in WCF will have the following problems:
If someone deploys multiple instances of your service in a web farm, each instance will generate clashing numbers.
If there is a database error during the reading or writing of the table, then problems will ensue.
The mere act of reading and writing to the table in separate steps will introduce massive concurrency problems. Do you really want to force a serializable table lock and have everything queue up on the unique number generator?
If you begin a transaction in your service code, all other requests will block on the unique number table because it will be part of a long-running transaction.
I'm new to NHibernate and am having difficulties setting it up for my current website. This website will run on multiple webservers with one database server, and this leaves me facing some concurrency issues.
The website will have an estimated 50.000 users or so registered, and each user will have a profile page. On this page, other users can 'like' another user, much like Facebook. This is were the concurrency problem kicks in.
I was thinking of using second-level cache, most likely using the MemChached provider since I'll have multiple webservers. What is the best way to implement such a 'Like' feature using NHibernate? I was thinking of three options:
Use a simple Count() query. There will be a table 'User_Likes' where each row would represent a like from one user to another. To display the number the number of likes, I would simply ask the number of Likes for a user, which would be translated to the database as a simple SELECT COUNT(*) FROM USER_LIKES WHERE ID = x or something. However, I gather this would be come with a great performance penalty as everytime a user would visit a profile page and like another user, the number of likes would have to be recalculated, second-level cache or not.
Use an additional NumberOfLikes column in the User table and increment / decrement this value when a user likes or dislikes another user. This however gives me concurrency issues. Using a simple for-loop, I tested it by liking a user 1000 times on two servers and the result in the db was around 1100 likes total. That's a difference of 900. Whether a realistic test or not, this is of course not an option. Now, I looked at optimistic and pessimistic locking as a solution (is it?) but my current Repository pattern is, at the moment, not suited to use this I'm afraid, so before I fix that, I'd like to know if this is the right way to go.
Like 2, but using custom HQL and write the update statement myself, something along the lines of UPDATE User SET NumberOfLikes = NumberOfLikes + 1 WHERE id = x. This won't give me any concurrency issues in the database right? However, I'm not sure if I'll have any datamismatch on my multiple servers due to the second level caching.
So... I really need some advice here. Is there another option? This feels like a common situation and surely NHibernate must support this in an elegant manner.
I'm new to NHIbernate so a clear, detailed reply is both necessary and appreciated :-) Thanks!
I suspect you will see this issue in more locations. You could solve this specific issue with 3., but that leaves other locations where you're going to encounter concurrency issues.
What I would advise is to implement pessimistic locking. The usual way to do this is to just apply a transaction to the entire HTTP request. With the BeginRequest in your Global.asax, you start a session and transaction. Then, in the EndRequest you commit it. With the Error event, you go the alternative path of doing a rollback and discarding the session.
This is quite an accepted manner of applying NHibernate. See for example http://dotnetslackers.com/articles/aspnet/Configuring-NHibernate-with-ASP-NET.aspx.
I'd go with 3. I believe this in this kind of application it's not so critical if some pages show a slightly outdated value for a while.
IIRC, HQL updates do not invalidate the entity cache entry, so you might have to do it manually.
In an app, Users and Cases have a many-to-many relationship. Users pull their list of Cases often, Users can update a single case at a time (a 1-10 second operation, requiring more than one UPDATE). Under READCOMMITTED, any in-use Case would block all associated Users from pulling their list of Cases. Also, the most recent data is a hotspot for both reads and writes to the Cases table.
I think I want to employ dirty reads to keep the experience snappy. READPAST on Cases won't work for this purpose. NOLOCK will work, but I'd like to be able to show which records are dirty when they are listed.
I don't know of any native way to show which records are dirty, so I'm thinking that for each update or insert to Cases, an INUSE flag will be set. This flag must be cleared by the end of the updating transaction such that under READCOMMITTED, this flag will never appear to be set. Note that this is NOT to replace concurrency management, only to show which records are potentially dirty to the User.
My question is whether this is reliable - if we UPDATE two or more fields (INUSE plus the other fields) in a single statement, is it possible that a concurrent NOLOCK query would read some of the new values but not others? If so, is it possible to guarantee that INUSE be set first?
And if I'm thinking about this all wrong, please enlighten me. My ideal situation would be to, in a manageable way, be able to show the values as they were PRIOR to any related transaction so the data is immediately available and always consistent (but partially out-dated). But I don't think this is available - especially in the more complex actual database.
Thanks!
Restating the problem just to be sure: User A on connection A updates two columns (col1, col2) in MyTable. While this is going on, user B on connection B issues a dirty read, selecting data from that row. You are wondering if user B could get, say, the updated value in col1 AND the old/not updated value in col2. Correct?
I have to say: no way could this happen. As I understand it, updates are indeed an atomic transaction, and if you're writing data to the page (in memory), then the entire row update would have to finish on that set of bytes before anything else (another thread) could get access to them.
But I don't know for sure, and I can't imagine how to set up a test to confirm or deny this. The only answer I'd rely on would have to come from someone who actually had a hand in writing the code, or perhaps a Microsoft technician who has similar access. If you don't get any good answers here, posting the question on the appropriate MSDN forum (link) might get a good answer.
Have you considered using SNAPSHOT isolation level? When used for a query, it requires no locks whatsoever, and it gives precisely the semantics that you're asking for:
show the values as they were PRIOR to any related transaction so the data is immediately available and always consistent (but partially out-dated)