I know maybe I'm asking something stupid in my application users can create a sort of agendas but only a specific number of agendas is allowed per day. So, users perform this pseudo-code:
select count(*) as created
from Agendas
where agendaDay = 'dd/mm/yyyy'
if created < allowedAgendas {
insert into Agendas ...
}
All this obviously MUST be executed in mutual exclusion. Only one user at time can read the number of created agendas and, possibly, insert a new one if allowed.
How can I do this?
I tried to open a transaction with default Read Committed isolation level but this doesn't help because during the transaction the other users can still get the number of the created agendas at the same time with a select query and so try
to insert a new one even if it wouldn't be allowed.
I don't think changing the isolation level could help.
How can I do this?
For testing I'm using SQL Server 2008 while in our production server SQL Server 2012 is run.
it sounds like you have an architecture problem there, but you may be able to achieve this requirement with:
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
If you're reading an inserting within the same transaction, I don't see where the problem will be, but if you're expecting interactive input on the basis of the count then you should probably ensure you do this within a single session of implement some kind of queuing functionality
Related
Is there (simple) way in a CDatabase (against an MDB file), after BeginTrans, to check how many records will be affected by an ExecuteSQL call that does a DELETE, INSERT, UPDATE or ALTER? In CDaoDatabase there is GetRecordsAffected - but I can't find anything similar for CDatabase.
I was (previously) using the number of affected records as a level of confirmation that my SQL was doing what I intended before either committing or rolling back the transaction. However I haven't been able to replicate that functionality since changing DB API tech.
I could probably auto-formulate a SELECT statement with the same WHERE parameters and count the results - but that is a lot of work, and I was hoping I missing something simple...
I have a very large Redshift database that contains billions of rows of HTTP request data.
I have a table called requests which has a few important fields:
ip_address
city
state
country
I have a Python process running once per day, which grabs all distinct rows which have not yet been geocoded (do not have any city / state / country information), and then attempts to geocode each IP address via Google's Geocoding API.
This process (pseudocode) looks like this:
for ip_address in ips_to_geocode:
country, state, city = geocode_ip_address(ip_address)
execute_transaction('''
UPDATE requests
SET ip_country = %s, ip_state = %s, ip_city = %s
WHERE ip_address = %s
''')
When running this code, I often receive errors like the following:
psycopg2.InternalError: 1023
DETAIL: Serializable isolation violation on table - 108263, transactions forming the cycle are: 647671, 647682 (pid:23880)
I'm assuming this is because I have other processes constantly logging HTTP requests into my table, so when I attempt to execute my UPDATE statement, it is unable to select all rows with the ip address I'd like to update.
My question is this: what can I do to update these records in a sane way that will stop failing regularly?
Your code is violating the serializable isolation level of Redshift. You need to make sure that your code is not trying to open multiple transactions on the same table before closing all open transactions.
You can achieve this by locking the table in each transaction so that no other transaction can access the table for updates until the open transaction gets closed. Not sure how your code is architected (synchronous or asynchronous), but this will increase the run time as each lock will force others to wait till the transaction gets over.
Refer: http://docs.aws.amazon.com/redshift/latest/dg/r_LOCK.html
Just got the same issue on my code, and this is how I fixed it:
First things first, it is good to know that this error code means you are trying to do concurrent operations in redshift. When you do a second query to a table before the first query you did moments ago was done, for example, is a case where you would get this kind of error (that was my case).
Good news is: there is a simple way to serialize redshift operations! You just need to use the LOCK command. Here is the Amazon documentation for the redshift LOCK command. It works basically making the next operation wait until the previous one is closed. Note that, using this command your script will naturally get a little bit slower.
In the end, the practical solution for me was: I inserted the LOCK command before the query messages (in the same string, separated by a ';'). Something like this:
LOCK table_name; SELECT * from ...
And you should be good to go! I hope it helps you.
Since you are doing a point update in your geo codes update process, while the other processes are writing to the table, you can intermittently get the Serializable isolation violation error depending on how and when the other process does its write to the same table.
Suggestions
One way is to use a table lock like Marcus Vinicius Melo has suggested in his answer.
Another approach is to catch the error and re run the transaction.
For any serializable transaction, it is said that the code initiating the transaction should be ready to retry the transaction in the face of this error. Since all transactions in Redshift are strictly serializable, all code initiating transactions in Redshift should be ready to retry them in the face of this error.
Explanations
The typical cause of this error is that two transactions started and proceeded in their operations in such a way that at least one of them cannot be completed as if they executed one after the other. So the db system chooses to abort one of them by throwing this error. This essentially gives control back to the transaction initiating code to take an appropriate course of action. Retry being one of them.
One way to prevent such a conflicting sequence of operations is to use a lock. But then it restricts many of the cases from executing concurrently which would not have resulted in a conflicting sequence of operations. The lock will ensure that the error will not occur but will also be concurrency restricting. The retry approach lets concurrency have its chance and handles the case when a conflict does occur.
Recommendation
That said, I would still recommend that you don't update Redshift in this manner, like point updates. The geo codes update process should write to a staging table, and once all records are processed, perform one single bulk update, followed by a vacuum if required.
Either you start a new session when you do second update on the same table or you have to 'commit' once you transaction is complete.
You can write set autocommit=on before you start updating.
in an existing application there are several dynamically generated SQL statements that are being executed. Some of those are very slow in performance and block the UI.
Without changing the code that generates the SQL I was wondering if there is a way to prematurely stop an Oracle/SQL statement that exceeds
a) an execution time threshold
b) number of result rows
While b) sounds easy, it is not easy in the application infrastructure I am dealing with, because I do not get a recordset back that I can iterate over as the SQL is being executed. I guess in some ways a huge number of result rows would eventually trigger the time threshold.
I read something about using the Oracle Resource Manager, but I wasn't sure if that can address a) and b) and if that is the easiest way to solve this. I was hoping there are session/connection options that would help me.
Thanks in advance!
Create a profile to limit the CPU time on a single SQL call. Assign that profile to the application user.
--Create profile that limits CPU per call to 1 second.
create profile temp_profile limit cpu_per_call 100;
--Create user, assign profile.
create user profile_test_user identified by "asDF1234!";
alter user profile_test_user profile temp_profile;
grant connect to profile_test_user;
That user will get errors like this:
PROFILE_TEST_USER#someDB> select count(*) from user_objects;
COUNT(*)
----------
0
PROFILE_TEST_USER#someDB> select count(*) from all_objects;
select count(*) from all_objects
*
ERROR at line 1:
ORA-02393: exceeded call limit on CPU usage
In general this approach should be a last resort. It's usually better to spend time tuning queries and databases.
Look into Resource Consumer groups. You can limit things like CPU time to sessions/queries for specfic groups of users.
This would be a database solution, not an application one - so it would work for that user regardless of how they are logged into the database.
I'm currently working with SQL Server 2008 R2, and I have only READ access to a few tables that house production data.
I'm finding that in many cases, it would be extremely nice if I could run something like the following, and get the total record count back that was affected :
USE DB
GO
BEGIN TRANSACTION
UPDATE Person
SET pType = 'retailer'
WHERE pTrackId = 20
AND pWebId LIKE 'rtlr%';
ROLLBACK TRANSACTION
However, seeing as I don't have the UPDATE permission, I cannot successfully run this script without getting :
Msg 229, Level 14, State 5, Line 5
The UPDATE permission was denied on the object 'Person', database 'DB', schema 'dbo'.
My questions :
Is there any way that my account in SQL Server can be configured so that if I want to run an UPDATE script, it would automatically be wrapped in a transaction with an rollback (so no data is actually affected)
I know I could make a copy of that data and run my script against a local SSMS instance, but I'm wondering if there is a permission-based way of accomplishing this.
I don't think there is a way to bypass SQL Server permissions. And I don't think it's a good idea to develop on production database anyway. It would be much better to have development version of the database you work with.
If the number of affected rows is all you need then you can run select instead of update.
For example:
select count(*)
from Person
where pTrackId = 20
AND pWebId LIKE 'rtlr%';
If you are only after the amount of rows that would be affected with this update, that would be same amount of rows that currently comply to the WHERE clause.
So you can just run a SELECT statement as such:
SELECT COUNT(pType)
FROM Person WHERE pTrackId = 20
AND pWebId LIKE 'rtlr%';
And you'd get the resulting potential rows affected.
1.First Login as admin in sqlserver
2.Goto login->your name->Check the roles.
3.IF u have write access,then you can accomplish the above task.
4.If not make sure you grant access to write.
If it's strictly necessary to try the update, you could write a stored procedure, accepting dynamic SQL as a string (Your UPDATE query) and wrapping the dynamic SQL in a transaction context which is then rolled back. Your account could then be granted access to that stored procedure.
Personally, I think that's a terrible idea, and incredibly unsafe - some queries break out of such transaction contexts (e.g. ALTER TABLE). You may be able to block those somehow, but it would still be a security/auditing problem.
I recommend writing a query to count the relevant rows:
SELECT COUNT(*)
FROM --tables
WHERE --your where clause
-- any other clauses here e.g. GROUP BY, HAVING ...
I do have a webapp written in Java on Tomcat, all connections should be autoCommit=false by default. Now, if I do run SELECT statement only in a transaction. Do I still need to call commit() or is it sufficient just to close the connection?
For what it's worth: I am on Oracle 11.2.
There is a similar question but does not actually give an answer for this case.
It is sufficient to close the connection, no need to call commit or rollback.
But according to connection.close(), it is recommended to call either commit or rollback.
Select statements do not disturb the underlying model or the data contained within the model. It is safe to close the connection without calling any commands related to transactions (like commit).
Actually strike that. I had not considered adjacent selects made to a model in my first answer. Say you execute select id from users where age > 20 and follow it up with select id from users where age = 20, any updates made between these queries would affect the ACID nature of the selects and return duplicate results within the 2 queries. To guarantee consistent results you would need to wrap both selects in the same transaction with a commit().
So yes, It makes sense to commit your selects.