The current situation is that I have an application that scales horizontally with one SQL database. Periodically, a background process is ran but I only want one invocation of this background process running at a time. I have tried to accomplish this by using a database row and locking but I am stuck. The requirement is that only one batch job should have successfully completed per day.
Currently I have a table called lock which has three columns: timestamp, lock_id, status. Status is an enum that has three values 0 = not running, 1 = running, 2 = completed.
The issue is that if a batch job fails and status is equal to 0, How can I make sure that only one background process will retry. How do I guarantee that only one background process is running in the retry scenario?
In an ideal world, I would like to do a SELECT statement that checks for the STATUS in the locking table, if status is = 0 meaning not running then start the background job and change status to 1 = running. However, if all horizontally scaled processes do this at the same time, is it guaranteed that only one is executed?
Thanks!
Related
enter image description here
I'm trying to find an efficient way to select all dependencies. The idea behind this table is:
Each taskID is a task that will be display to the user.
When taskid 22180 is selected by the user and completed it will
automatically enable the other tasks (activateTaskId = 22180) to
become available to the user hence inactive = 0. Up to that point
everything is fine.
What i need help at is when you have subsequent tasks.
So when taskid (22180) is selected and completed by the user the following tasks become available (22181,22182,22185 and 22186). As you can see if the user selects 22182 then the task dependent on it is 22183 and subsequently 22184 will .
how would i efficiently select all the task that branch off the initial one (22182) no matter how many may exist ?
Thank you
I am working on a server that accesses a database. It is possible for multiple people to access the same record. Will each request wait in line, or will they all try to modify that record at the same time.
Example:
I have an image, and the database will keep track of how many "likes" that image has.
UPDATE `images` SET `image_likes` = `image_likes` + 1 WHERE `image_id` = 0;
Assuming that specific image has 0 "likes" and 3 people at the same time "like" that image, would those 3 request properly be processed, resulting in that image having 3 likes, or is there a chance that the record can be corrupted, or at the very least be incorrect, maybe only showing 2 "likes"?
My Database uses the MyISAM engine and I am using it through GoDaddy.
Thank you
Php by itself is not thread safe but MySQL is , in this case MySQL will handle this issue and you will get 3 likes. Unless there is any other operation involved this should not be a problem
You can give it a try by calling that script via console multiple times to see what happens
In my code, i need to assign the Pallet number to the selected carton boxes.
once the user the selects the boxes(15-30 boxes) and press OK, i run the following code.
//UPDATE THE PALLET NO FOR ALL THE SELECTED CARTONS
foreach (DataGridViewRow item in dgvCartonDetails.Rows)
{
dbLayer.tblCartonUpdatePalletid(item.Cells["CM_ID"].Value.ToString(), Pno, _Settings.Line.ToString());
//STORED PROCEDURE:tblCartonUpdatePalletid
//update tblCarton set CM_palletid = #palletid, cm_cartoncompletetime = getdate() where cm_id = #cm_id
}
//PRINT ALL THE BOXES IN THE PALLET
dbLayer.tblPrintAllCartonsOfthePallet(PalletID);
//STORED PROCEDURE: tblPrintAllCartonsOfthePallet
//select * from tblCarton where cm_palletid = #PalletID
Some times i face the lock error (ref pic).
I have given the stored procedure data as well for referance. Carton table will increase at the rate of 5000 records/day.
I dont know what i am missing. where should i look into? Thanks in advance.
There must be another process running at the same time. You need to identify what it is. The best thing to do if possible is to capture a deadlock in profiler as it will show you exactly which processes deadlocked and one which resource.
I assume you run those 2 queries in 1 transaction.
If there are 2 processes running the code above, what may happen is they update tblCarton at the same time. The first one updates a record on page 1 and the second updates a record on page 2. Then they need to update the pages in reverse: first needs to update page 2 and second page 1. That will result in a deadlock.
Without getting a deadlock report from profiler, it's hard to say if this is happening for sure, though.
I am a developer on a web app that uses an Oracle database. However, often the UI will trigger database operations that take a while to process. As a result, the client would like a progress bar when these situations occur.
I recently discovered that I can query V$SESSION_LONGOPS from a second connection, and this is great, but it only works on operations that take longer than 6 seconds. This means that I can't update the progress bar in the UI until 6 seconds has passed.
I've done research on wait times in V$SESSION but as far as I've seen, that doesn't include the waiting for the query.
Is there a way to get the progress of the currently running query of a session? Or should I just hide the progress bar until 6 seconds has passed?
Are these operations Pl/SQL calls or just long-running SQL?
With PL/SQL operations we can write messages with SET_SESSION_LONGOPS() in the DBMS_APPLICATION_INFO package. We can monitor these messages in V$SESSION_LONGOPS. Find out more.
For this to work you need to be able to quantify the operation in units of work. These must be iterations of something concrete, and numeric not time. So if the operation is insert 10000 rows you could split that up into 10 batches. The totalwork parameter is the number of batches (i.e. 10) and you call SET_SESSION_LONGOPS() after every 1000 rows to increment the sofar parameter. This will allow you to render a thermometer of ten blocks.
These messages are session-based but there's no automatic way of distinguishing the current message from previous messages from the same session & SID. However if you assign a UID to the context parameter you can then use that value to filter the view.
This won't work for a single long running query, because there's no way for us to divide it into chunks.
i found this very usefull
dbms_session.set_module("MY Program" , "Kicking off ... ")
..
dbms_session.set_action("Extracting data ... ")
..
dbms_session.set_action("Transforming data ... ")
..
you can monitor the progress using
select module , action from v$session where sid = :yoursessionid
I've done quite a lot of web development with Oracle over the years and found that most users prefer an indeterminate progress bar, than a determinate bar that is inaccurate (a la pretty much any of Microsoft's progress bars which annoy me no end), and unfortunately there is no infallible way of accurately determining query progress.
Whilst your research into the long ops capability is admirable and would definitely help to make the progress of the database query more reliable, it can't take into account the myriad of other variables that may/will affect the web operation's transactional progress (network load, database load, application server load, client-side data parsing, the user clicking on a submit button 1,000 times, etc and so on).
I'd stick to the indeterminate progress method using Javascript callbacks. It's much easier to implement and it will manage your user's expectations as appropriate.
Using V$_SESSION_LONGOPS requires to set TIMED_STATISTICS=true or SQL_TRACE=true. Your database schema must be granted the ALTER SESSION system privilege to do so.
I once tried using V$_SESSION_LONGOPS with a complex and long running query. But it turned up that V$_SESSION_LONGOPS may show the progress of parts of the query like full table scans, join operations, and the like.
See also: http://www.dba-oracle.com/t_v_dollar_session_longops.htm
What you can do is just to show the user "the query is still running". I implemented a <DIV> nested into a <TD> that gets longer with every status request sent by the browser. Status requests are initiated by window.SetTimeout (every 3 seconds) and are AJAX calls to a server-side procedure. The status report returned by the server-side procedure simply says "we are still running". The progress bar's width (i.e. the <DIV>'s width) increments by 5% of the <TD>s width every time and is reset to 5% after showing 100%.
For long running queries you might track the time they took in a separate table, possibly with individual entries for varying where clauses. You could use this to display the average time plus the time that just elapsed in the client-side dialog.
If you have a long running PL/SQL procedure or the like on the server side doing several steps, try this:
create a table for status messages
use a unique key for any process the user starts. Suggestion: client side's javascript date in milliseconds + session ID.
in case the long running procedure is to be started by a link in a browser window, create a job using DBMS_JOB.SUBMIT to run the procedure instead of running the procedure directly
write a short procedure that updates the status table, using PRAGMA AUTONOMOUS_TRANSACTION. This pragma allows you to commit updates to the status table without committing your main procedure's updates. Each major step of your main procedure should have an entry of its own in this status table.
write a procedure to query the status table to be called by the browser
write a procedure that is called by an AJAX call if the use clicks "Cancel" or closes the window
write a procedure that is called by the main procedure after completion of each step: it queries the status table and raises an exception with an number in the 20,000s if the cancel flag was set or the browser did not query the status for, say, 60 seconds. In the main procedure's exception handler look for this error, do a rollback, and update the status table.
Node.js & Redis:
I have a LIST (users:waiting) storing a queue of users waiting to join games.
I have SORTED SET (games:waiting) of games waiting for users. This is updated by the servers every 30s with a new date. This way I can ensure if a server crashes, the game is no longer used. If the server is running and fills up, it'll remove itself from the sorted set.
Each game has a SET (game:id:users) containing the users that are in it. Each game can accept no more than 6 players.
Multiple servers are using BRPOP to pick up users from the LIST (users:waiting).
Once a server has a user id, it gets the waiting games ids, then proceeds to run SCARD on their game:id:users SET. If the result of this is less than 6, it adds them to the set.
The problem:
If multiple servers are doing this at once, we could end up with more than 6 users being added to a set at a time. For example if one server requests SCARD and immediately after another runs SADD, the number in the set will have increased but the first server won't know.
Is there anyway of preventing this?
You need transactions, which redis supports: http://redis.io/topics/transactions
in your case in particular, you want to pay attention to the watch command: http://redis.io/topics/transactions#cas