I've been reading about locking the various persistent variables, application, session, etc. and I understand the race conditions that exists with reading application variables that are updated often.
I'm not so sure about session variables and application variables that don't change. I am on a single server and CF9. I'm locking the session variables when I set them, but is it really necessary to lock them when reading them. Doesn't each user have their own set of session variables. How would changing values for one user effect an different user viewing the same page at the same time.
Also, I set application variables in my application.cfc file and they do not change. Are locks needed every time I read the application.dsn variable?
You don't need to lock session variables unless you have part of your app that is going to be modifying the user's session directly (which is part of an undocumented feature in CF, so it's not likely you're doing that anyway).
As for Application variables, you shouldn't need to lock those either if you're just setting them once and never messing with them again.
Related
I have an Apache Geode setup, connected with external Postgres datasource. I have a scenario where I define an expiration time for a key. Let's say after T time the key is going to expire. Is there a way so that the keys which are going to expire can make a call to an external datasource and update the value incase the value has been changed? I want a kind of automatic syncing for my keys which are there in Apache Geode. Is there any interface which i can implement and get the desired behavior?
I am not sure I fully understand your question. Are you saying that the values in the cache may possibly be more recent than what is currently stored in the database?
Whether you are using Look-Aside Caching, Inline Caching, or even Near Caching, Apache Geode combined with Spring would take care of ensuring the cache and database are kept in sync, to some extent depending on the caching pattern.
With Look-Aside Caching, if used properly, the database (i.e. primary System of Record (SOR), e.g. Postgres in your case) should always be the most current. (Look-Aside) Caching is secondary.
With Synchronous Inline Caching (using a CacheLoader/CacheWriter combination for Read/Write-Through) and in particular, with emphasis on CacheWriter, during updates (e.g. Region.put(key, value) cache operations), the DB is written to first, before the entry is stored (or overwritten) in the cache. If the DB write fails, then the cache entry is not written or updated. This is true each time a value for a key is updated. If the key has not be updated recently, then the database should reflect the most recent value. Once again, the database should always be the most current.
With Asynchronous Inline Caching (using AEQ + Listener, for Write-Behind), the updates for a cache entry are queued and asynchronously written to the DB. If an entry is updated, then Geode can guarantee that the value is eventually written to the underlying DB regardless of whether the key expires at some time later or not. You can persist and replay the queue in case of system failures, conflate events, and so on. In this case, the cache and DB are eventually consistent and it is assumed that you are aware of this, and this is acceptable for your application use case.
Of course, all of these caching patterns and scenarios I described above assume nothing else is modifying the SOR/database. If another external application or process is also modifying the database, separate from your Geode-based application, then it would be possible for Geode to become out-of-sync with the database and you would need to take steps to identify this situation. This is rather an issue for reads, not writes. Of course, you further need to make sure that stale cache entries does not subsequently overwrite the database on an update. This is easy enough to handle with optimistic locking. You could even trigger a cache entry remove on an DB update failure to have the cache refreshed on read.
Anyway, all of this is to say, if you applied 1 of the caching patterns above correctly, the value in the cache should already be reflected in the DB (or will be in the Async, Write-Behind Caching UC), even if the entry eventually expires.
Make sense?
The situation
Suppose we have a web application connected to a (Postgre)SQL database whose task can be summarized as:
A SELECT operation to visualize the data.
An UPDATE operation that stores modifications based on the visualized data.
Simple, but... the data involved isn't user specific, so it might potentially be changed during the process by other users. The editing task may take long time (perhaps more than an hour), meaning that the probability of these collisions happening isn't low: it makes sense to implement a robust solution to the problem.
The approach
The idea would be that, once the user tries to submit the changes (i.e. firing the UPDATE operation), a number of database checks will be triggered to ensure that the involved data didn't change in the meantime.
Assuming we have timestamped every change on the data, it would be as easy as keeping the access time when the data was SELECTed and ensuring that no new changes were done after that time on the involved data.
The problem
We could easily just keep that access time in the frontend application while the user performs the editing, and later provide it as an argument to the trigger function when performing the UPDATE, but that's not desirable for security reasons. The database should store the user's access time.
An intuitive solution could be a TEMPORARY TABLE associated to the database session. But, again, the user might take a long time doing the task, so capturing a connection from the pool and keeping it idle for such a long time doesn't seem like a good option either. The SELECT and the UPDATE operations will be performed under different sessions.
The question
Is there any paradigm or canonical way to address and solve this problem efficiently?
This problem is known as the "lost update" problem.
There are several solutions that depend on whether a connection pool is used or not and on the transaction isolation level used:
pessimistic locking with SELECT ... FOR UPDATE without connection pool
optimistic locking with timestamp column if connection pool is used.
If all connections with identical sql connection strings are dropped regardless of the individual instance calling the clearpool method, this sounds like a difficulty to me. We have an issue where the close and dispose methods of a sql connection don't actually clear it from the list of connections in the sql activity monitor, and we get a backlog of instances of this same stored procedure being called or active in some way. Based on this idea of all instances of the same process being cleared from the pool based on a single call from a single instance, it sounds as if any instance performing a sql transaction at the time it's being called would be dropped and cause an outage in the transaction that's occurring in mid-process.
A particular wrinkle in this for us is that several people are using our software product at the same time, and the sql connection strings referenced in the vb code are set up using the same variable name for everyone-- but that doesn't mean that all the actual strings assigned to the variable at runtime are the same, does it?
Is the backup of calls to the same procedure something that would be fully cleared from the queue using the .clearpool method, or would only the single instance be cleared? If the single instance is cleared, that's great.
I'm planning to test the sqlconnection.state to see if it's performing an action before using .clearpool to be sure it doesn't drop the connection while the stored procedure is running.
Many misconceptions here.
regardless of the individual instance calling the clearpool method
You cannot call this method on any instance. It is static. C# allows you to write it like an instance call but really it is not.
We have an issue where the close and dispose methods of a sql connection don't actually clear it from the list of connections in the sql activity monitor
That is the whole purpose of pooling. The physical connection stays alive. All settings made on the logical connection are reset, through.
and we get a backlog of instances of this same stored procedure being called or active in some way
Highly unlikely. When a connection is recycled it is reset. All transactions are rolled back. When you close a connection all running statements are killed. Note, though, that the reset happens when the connection is taken. Not when it is put back. For that reason you should explicitly rollback transactions that you do not wish to commit. Do this simply by disposing the reader and transaction objects.
it sounds as if any instance performing a sql transaction at the time it's being called would be dropped and cause an outage in the transaction that's occurring in mid-process.
Clearing the pool only affects connections that are not in use. This is transparent to you.
the sql connection strings referenced in the vb code are set up using the same variable name for everyone-- but that doesn't mean that all the actual strings assigned to the variable at runtime are the same, does it?
Why wouldn't it? Not enough information here to see any reason why.
Is the backup of calls to the same procedure something that would be fully cleared from the queue using the .clearpool method, or would only the single instance be cleared?
This statement is based on false assumptions. Clearing the pool has no effect on connections that are in use. That would be a horrible design choice.
Never clear the pool. Simply dispose of your connections when you no longer need them.
I am working on a VB.NET application.
As per the nature of the application, One module has to monitor database (SQLite DB) each second. This Monitoring is done by simple select statement which run to check data against some condition.
Other Modules performs a select,Insert and Update statements on same SQLite DB.
on SQLite concurrent select statements are working fine, but I'm having hard time here to find out, why it is not allowing Inset and Update.
I understand it's a file based lock, but is there anyway to get it done?
each module, in fact statement opens and close the connection to DB.
I've restricted user to run single statement at a time by GUI design.
any Help will be appreciated.
If your database file is not on a network, you could allow a certain amount of read/write concurrency by enabling WAL mode.
But perhaps you should use only a single connection and do your own synchronization for all DB accesses.
You can use some locking mechanism to make sure the database works in a multithreading situation. Since your application is a read intensive one according to what you said, you can consider using a ReaderWriterLock or ReaderWriterLockSlim. (refer to here and here for more details)
If you have only one database, then creating just one instance of the lock is OK; if you have more than one database, each of them can be assigned a lock. Every time you do some read or write, enter the lock (by EnterReadLock() for ReaderWriterLockSlim, or by AcquireReaderLock() for ReaderWriterLock) before you do something, and after you're done exit the lock. Note that you can place the exit of the lock in a finally clause lest you forget to release it.
The strategy above is being used in our production applications. It's not so good as to use a single thread in your case because you have to take performance into account.
I have two applications running on a machine, where NHibernate is used as an ORM. One app is managing objects (CRUD operations), while the other is processing the objects (get, process, set status and save).
First I let the processing app process an object and set the status to processed. Then I change a text property manually in the database and reset the status (to make it process it again). The manual DB edit is to simulate the managing app. Then I start to see problems:
The read object still has the old text property, event though I've changed it in the DB. I guess NHibernate caching is the problem here.
When I set the object's status to processed, it uses all properties in the where clause when updating, which means it doesn't get updated in the database. This is because it has the wrong text in a property. I would guess this also has to do with caching.
The consequence of the status not being updated is that the same object (with wrong text) is processed over and over and over...
Anyone out there who can help me with how I should set up NHibernate to make this problem disappear?
Better call refresh method on the object you want, because flush can have unwanted side-effects.