UPDATE 3
Hm,...i was using an old, old, version of delayed_job_active_record. All is well.
I have implemented a push notification service in an API i'm working on. I process the push notifications using delayed job. My problem is that sometimes it seems that the worker process don't obtain a lock on the job table. That is, 2 workers sometimes pick up the same job. I can't reproduce the problem consistently, but i'm wondering if anyone else has experienced this? Here is the code for enqueueing jobs:
Device.where("platform = ? AND enabled = ?", 'ios', true ).find_in_batches( batch_size: 2000 ) do |batch|
Delayed::Job.enqueue APNWorker.new( params[:push_notification], batch )
end
devices is a table containing mobile device tokens. Testing is done locally with Foreman.
UPDATE 1
Here is some output from Foreman
13:10:41 worker.1 | started with pid 2489
13:10:41 worker.2 | started with pid 2492
13:10:41 worker.3 | started with pid 2495
Then, when i enqueue a job using the above code, sometimes, rather randomly i get
13:15:55 worker.1 | work
13:15:55 worker.3 | work
Here, 'work' indicates that the job is being executed. And i receive a duplicate push notification. If i check the delayed_jobs table i see only one, locked, job. Still, 2 workers are picking it up.
UPDATE 2
Here are some logs from Rails
Delayed::Backend::ActiveRecord::Job Load (1.1ms) SELECT "delayed_jobs".* FROM "delayed_jobs" WHERE ((run_at <= '2014-02-02 17:42:37.813835' AND (locked_at IS NULL OR locked_at < '2014-02-02 13:42:37.813853') OR locked_by = 'host:positive-definite-fakta-vbox pid:4114') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1
Delayed::Backend::ActiveRecord::Job Load (4.8ms) SELECT "delayed_jobs".* FROM "delayed_jobs" WHERE ((run_at <= '2014-02-02 17:42:37.772102' AND (locked_at IS NULL OR locked_at < '2014-02-02 13:42:37.772130') OR locked_by = 'host:positive-definite-fakta-vbox pid:4118') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1
(0.1ms) BEGIN
Delayed::Backend::ActiveRecord::Job Load (0.7ms) SELECT "delayed_jobs".* FROM "delayed_jobs" WHERE "delayed_jobs"."id" = $1 LIMIT 1 FOR UPDATE [["id", 537]]
(0.4ms) UPDATE "delayed_jobs" SET "locked_at" = '2014-02-02 17:42:37.844545', "locked_by" = 'host:positive-definite-fakta-vbox pid:4118', "updated_at" = '2014-02-02 17:42:37.954756' WHERE "delayed_jobs"."id" = 537
(0.6ms) COMMIT
(3.0ms) BEGIN
Delayed::Backend::ActiveRecord::Job Load (7.0ms) SELECT "delayed_jobs".* FROM "delayed_jobs" WHERE "delayed_jobs"."id" = $1 LIMIT 1 FOR UPDATE [["id", 537]]
(0.4ms) UPDATE "delayed_jobs" SET "locked_at" = '2014-02-02 17:42:37.869191', "locked_by" = 'host:positive-definite-fakta-vbox pid:4114', "updated_at" = '2014-02-02 17:42:37.997562' WHERE "delayed_jobs"."id" = 537
(0.8ms) COMMIT
Device Load (0.6ms) SELECT "devices".* FROM "devices" WHERE "devices"."id" = $1 LIMIT 1 [["id", "18"]]
Device Load (0.6ms) SELECT "devices".* FROM "devices" WHERE "devices"."id" = $1 LIMIT 1 [["id", "18"]]
As can be seen, both workers get to do the job( 'Device Load...' is the actual work ).
In the delayed_jobs table there is a single entry, locked by:
host:positive-definite-fakta-vbox pid:4114
What i don't really get is that the above seems like a perfectly normal, highly likely scenario. The only thing happening is that two workers poll the job_queue at just about the same time. Nothing strange about that i guess...but of course the result is disastrous.
How come that the select for update statement:
Delayed::Backend::ActiveRecord::Job Load (0.7ms) SELECT "delayed_jobs".* FROM "delayed_jobs" WHERE "delayed_jobs"."id" = $1 LIMIT 1 FOR UPDATE [["id", 537]]
is not checking that the job is not already locked? The regular queue-polling seems to do just that.
Related
I have a system that requests information by sending 3 parameters to an external system: user, start_date and end_date.
I have a table
request (
id,
user,
start_date,
end_date,
status
)
that logs these requests and their status (Done for the requests that have returned, Waiting for the requests that havent yet returned).
Every few hours I will resubmit the requests that havent yet returned, even though the initial request could still return some time in the future.
After some time, my table will have multiple requests for the same user/start_date/end_date, some of them Waiting, some Done.
What I need is a query that returns a list of ids of all duplicate requests with the exception of 1 Done, where at least one request has status=Done.
In summary, I need a way to clear the exceeding requests for a given user/start_date/end_date, if at least one of them has status=Done (doesnt matter which one, I just need to keep 1 status = Done for a given user/start_date/end_date).
So far I've been able to pinpoint the duplicate requests that have at least 1 Done. To select all but one complete from this query, I would most likely wrap this entire query into 2 more selects and do the magic, but the query as is, is already really slow. Can someone help me refactor it and select the end result i need?
http://sqlfiddle.com/#!5/10c25a/1
I'm using SQLite
The expected result from the dataset provided in the sqlfiddle is this:
454, 457, 603, (604 or 605 not both), 607, 608
select r.id from request r inner join (
select user, start_date, end_date,
min(case when status = 'Done' then id end) as keep_id
from request
group by user, start_date, end_date
having count(case when status = 'Done' then 1 end) > 0 and count(*) > 1
) s on s.user = r.user and s.start_date = r.start_date and s.end_date = r.end_date
and s.keep_id <> r.id
What you're after are records that match this criteria...
There exists another record with Status "Done"
That other "Done" record matches user, start_date and end_date
That other record has a lower id value (because you need something to identify the record to keep) or the other record has a higher id but the record you're looking at has Status "Waiting"
With all that in mind, here's your query
SELECT id FROM request r1
WHERE EXISTS (
SELECT 1 FROM request r2
WHERE r2.Status = 'Done'
AND r1.user = r2.user
AND r1.start_date = r2.start_date
AND r1.end_date = r2.end_date
AND (r1.id > r2.id OR r1.Status = 'Waiting')
)
ORDER BY id
http://sqlfiddle.com/#!5/10c25a/26 ~ produces IDs 454, 457, 603, 605, 607 and 608
I'm trying to port Delayed Job to Haskell and am unable to understand the WHERE clause in the query that DJ fires to poll the next job:
UPDATE "delayed_jobs"
SET locked_at = '2017-07-18 03:33:51.729884',
locked_by = 'delayed_job.0 host:myhostname pid:21995'
WHERE id IN (
SELECT id FROM "delayed_jobs"
WHERE
(
(
run_at <= '2017-07-18 03:33:51.729457'
AND (locked_at IS NULL OR locked_at < '2017-07-17 23:33:51.729488')
OR locked_by = 'delayed_job.0 host:myhostname pid:21995'
)
AND failed_at IS NULL
) ORDER BY priority ASC, run_at ASC LIMIT 1 FOR UPDATE) RETURNING *
The structure of the WHERE clause is the following:
(run_at_condition AND locked_at_condition OR locked_by_condition)
AND failed_at_condition
Is there a set of inner parentheses missing in run_at_condition AND locked_at_condition OR locked_by_condition? In what precedence are the AND/OR clauses evaluated?
What is the purpose of the locked_by_condition where it seems to be picking up jobs that have already been locked by the current DJ process?!
The statement is probably fine. The context of the whole statement is to take the lock on the highest-priority job by setting its locked_at/locked_by fields.
The where condition is saying something like: "if run_at is sooner than now (it's due) AND, it's either not locked or it was locked more than four hours ago... alternatively that's all overridden if it was me that locked it, and of course, if it hasn't failed THEN lock it." So if I'm reading it correctly it looks kinda like it's running things that are ready to run but with a timeout so that things can't be locked-out forever.
To your second question, AND has a higher precedence than OR:
SELECT 'yes' WHERE false AND false OR true; -- 'yes', 1 row
SELECT 'yes' WHERE (false AND false) OR true; -- 'yes', 1 row
SELECT 'yes' WHERE false AND (false OR true); -- 0 rows
The first two statements mean the same thing, the third one is different.
The second point may just be a rough sort of ownership system? If the current process is the one that locked something, it should be able to override that lock.
hey guys so i have these 2 lists being pulled out of sqlite3
connection = sqlite3.connect('intel.db')
database = connection.cursor()
top = database.execute("SELECT logtext from logs order by length(logtext) desc limit 10").fetchall()
usr = database.execute("SELECT loguser from logs order by length(logtext) desc limit 10").fetchall()
print(usr,top)
('DIRECT_TEST',), ('ROBOT',), ('ZERO',), ('TEST',), ('Yan Yan',),
('Toasty',), ('!☆♡Fox Wizard♡☆',), ('Agibear',), ('ZEROBIT',), ('Big
Bad Bug',)] [('30000',), ('1605',), ('715',), ('333',), ('209',)
('260',), ('128',), ('376',), ('86',), ('0',)]
how can i make them print like this :
Tiny update.. I changed the logtext from TEXT to INT sorting got
solved.
Try this
for row in database.execute("SELECT logtext, loguser from logs"):
print(row)
Your output is not ordered, so not sure why you need that
But if you did...
SELECT logtext, cast(loguser as INTEGER) loguser from logs ORDER BY loguser DESC
I have some code like this:
stages.each do |stage|
start_activity = stage.activities.where('created_at < ?', #start_time).first
end_activity = stage.activities.where('created_at < ?', #end_time).first
# ...
end
Which fills my logs with a few screens worth of queries like these:
Activity Load (0.3ms) SELECT "activities".* FROM "activities"
WHERE "activities"."stage_id" = $1
AND (created_at < '2017-01-31 08:00:00.000000')
ORDER BY id DESC, "activities"."created_at" DESC
LIMIT 1
[["stage_id", 11548]]
Activity Load (0.3ms) SELECT "activities".* FROM "activities"
...
(Activity has a default scoped order of id DESC)
So I'm hoping there is a way to fetch all the activities I need per timestamp in a single query. This would reduce my number of from activity_count * 2 to simply 2.
Here's what I tried, but I don't think distinct is properly being used here.
Activity
.select('DISTINCT ON (stage_id) *')
.where(stage: stages)
.where('created_at < ?', #start_time)
.reorder('id DESC')
Which spews out:
ActiveRecord::StatementInvalid (PG::InvalidColumnReference: ERROR: SELECT DISTINCT ON expressions must match initial ORDER BY expressions
LINE 1: SELECT DISTINCT ON (stage_id) * FROM "activities" WHERE "act...
^
: SELECT DISTINCT ON (stage_id) * FROM "activities" WHERE "activities"."stage_id" IN (11548, 11549, <more ids snipped>, 12432, 12433) AND (created_at < '2017-01-31 08:00:00.000000') ORDER BY id DESC):
What I mean to do here is to say: give me all activities that have a stage_id in an array of stages, where the created_at date is nearest to (without going over) a timestamp. What I want to get back is a single activity record for every stage in stages.
Is it possible to do this in 1 query rather than stages.count queries?
By taking a look at your error...
ERROR: SELECT DISTINCT ON expressions must match initial ORDER BY expressions
...I got to this Stack Overflow post.
You need to include 'account_id' in your order function somewhere.
So I think you should be able to do...
Activity
.select('DISTINCT ON (stage_id)')
.where(stage: stages)
.where('created_at < ?', #start_time)
.order(:stage_id, :created_at)
c = Conversation.joins(:messages).random
>> Conversation Load (8.1ms) SELECT `conversations`.* FROM `conversations` INNER JOIN `messages` ON `messages`.`conversation_id` = `conversations`.`id` ORDER BY created_at DESC, RAND() LIMIT 1
Count:
c.messages.count
>> (5.7ms) SELECT COUNT(*) FROM `messages` WHERE `messages`.`conversation_id` = 74
Length:
c.messages.length
>> Message Load (1.2ms) SELECT `messages`.* FROM `messages` WHERE `messages`.`conversation_id` = 82
How to not perform additional queries above? I thought I have already joined messages for the random conversation using INNER JOIN, and yet a new query is performed to count those results?
The second issue is attempting to get all joined messages with a specific user_id:
u = User with id 27 # has messages in 'c' results above
c.messages.where('user_id = ?', u.id).all
>> Message Load (5.3ms) SELECT `messages`.* FROM `messages` WHERE `messages`.`conversation_id` = 82 AND (user_id = 27)
I used select to do this without additional queries:
c.messages.select { |msg| msg.user_id == u.id }
>> returns messages without query logged
I'd still appreciate how I might reduce or optimize these queries.
Try using includes instead of joins to avoid the n+1 problem.
Railscast #181 By Ryan Bates, http://railscasts.com/episodes/181-include-vs-joins?view=asciicast, explains the difference in an easy way.