I have a rails app with sidekiq-pro version 5.2.1 So recently we introduced super_fetch in our code. It does rescue the jobs when the Sidekiq shuts down, now when these rescued jobs are executed again, does it restart from the beginning of the worker or it starts from where it was interrupted?
Sidekiq calls perform each time. There’s no magic to restart at some arbitrary line of Ruby.
Related
I am using delayed jobs, because I have some long running processes running in the background.
The annoying part is that I need to run 'rake jobs:work' before I start my application everytime.
Is there any way to let my Rails application know that I want it to start every time I restart my server? or refresh my homepage?
Is this on your local machine? If so, you could just set an alias that you could run that would run 'rake jobs:work' when you use the alias. For example:
alias startserver='rake jobs:work & rails s'
I'm currently using Heroku's scheduler to run a script. However, the time that the script takes to run is going to increase from a few milliseconds to a few minutes. I'm looking at using the delayed_job gem to push this process off to a Worker Dyno. I want to continue to run this script once-a-day, just offload it to the worker. My current rake task is:
desc "This task updates some stuff for you."
task :update_some_stuff => :environment do
puts "Updating some stuff ..."
SomeClass.new.process
puts "... done."
end
Once the gem is installed, migration run, and worker dyno started, will the script just need to change to:
desc "This task updates some stuff for you."
task :update_some_stuff => :environment do
puts "Updating some stuff ..."
SomeClass.new.delay.process
puts "... done."
end
With this task still being a rake task scheduled by Heroku's Scheduler, is the only thing that needs to happen here the introduction of the delay method to put this in the Worker's queue?
Thanks in advance for any help.
Yes, DJ is a very simple gem, and you should be able to see this working correctly locally by adding a record to your delayed_jobs table in your database. With a worker running on Heroku everything should be fine.
Just make sure you have the following in your Procfile:
worker: bundle exec rake jobs:work
More information can be found here: https://devcenter.heroku.com/articles/delayed-job
I have a rails app using delayed job, it works fine in development.
However, when I pushed to production it doesn't do anything.
Delayed job starts up fine using either one of these commands:
RAILS_ENV=production rake jobs:work
or
RAILS_ENV=production script/delayed_job start
However, once the process has started it just sits there and does not process any of the jobs in the delayed_backend_mongoid table.
I checked the delayed_job.log and it shows that the process starts up fine but does not throw any errors.
And for the heck of it I checked the production.log and there are no errors there as well.
Also if I run
RAILS_ENV=production rake jobs:clear
It will delete the jobs that are in the que, so I don't think it has an issue communicating with mongodb.
I'm using
Rails 3.1.1
gem 'delayed_job'
gem 'delayed_job_mongoid'
Any insight will be greatly appreciated.
Ok so I finally got it to work, it wasn't related to delayed job at all. It had to do with the ruby gem Anemone. Anemone could not get proper authorization to the mongodb. Fixed, thanks anyways.
I'm using delayed_job and I am able to run jobs using rake jobs:work but using the daemonized version, it does nothing although I see it in the process list.
I'm using:
rails (3.0.9)
delayed_job (2.1.4)
daemons (1.0.10)
I'm running delayed_job using:
unix>RAILS_ENV=development script/delayed_job start
It could be a problem loading a custom job class file. To test that, try this:
Enter the rails console rails console --sandbox.
Make sure you have a job in the table job = Delayed::Job.first.
Try YAML.load(job.handler). If you get an error that looks like this: ArgumentError: undefined class/module MyCustomClass, it's probably a problem loading your custom job
Still in the rails console, run require 'My_Custom_Class. Then run the YAML.load(job.handler) command again. If this returns the appropriate object it's definitely a class loading problem.
To fix the problem create the file config/initializers/custom.rb and in it put require 'My_Custom_Class'.
You should then be able to run rake jobs::workoff and get something that looks like this:
[Worker(host:my.host pid:5085)] Starting job worker
[Worker(host:my.host pid:5085)] MyCustomJob completed after 0.0774
[Worker(host:my.host pid:5085)] 1 jobs processed at 9.1935 j/s, 0 failed ...
[Worker(host:my.host pid:5085)] No more jobs available. Exiting
To answer your question we may need more information.
Are jobs added to database? Are there any errors in jobs?
What's the result of RAILS_ENV=development script/delayed_job status as I already mentioned?
Second, did you went through the most common problems Wiki page?
https://github.com/collectiveidea/delayed_job/wiki/Common-problems
I am using delayed_job gem for sending emails in my rails app.
delayed_job was working well but from last 5 days, it is not working and throwing following error in delayed_job.log file.
2011-10-09T01:53:04+0530: [Worker(delayed_job host:backupserver pid:23953)] Syck::DomainType#private_group_join_request failed with NoMethodError: undefined method private_group_join_request' for # - 11 failed attempts
2011-10-09T01:53:04+0530: [Worker(delayed_job host:backupserver pid:23953)] 1 jobs processed at 1.4503 j/s, 1 failed ...
2011-10-09T01:54:40+0530: [Worker(delayed_job host:backupserver pid:23953)] Syck::DomainType#contact_us_email failed with NoMethodError: undefined method contact_us_email for # - 11 failed attempts
2011-10-09T01:54:40+0530: [Worker(delayed_job host:backupserver pid:23953)] 1 jobs processed at 4.3384 j/s, 1 failed ...
Following is one of the example how I am calling delayed job for sending email.
UserMailer.delay(:run_at => 10.seconds.from_now).contact_us_email(self)
I am starting delayed job with
RAILS_ENV=production script/delayed_job start
It is working correctly in development as well as production environment on my local machine.
Environment Which I am using in Rails App.
Rails 3.0.8
Ruby 1.9.2 in Linux(Ubuntu)
rake 0.9.2
delayed_job 2.1.4
This is same as
Undefined Method Error when creating delayed_job workers with script/delay_job
But solution is not working for me.
I figured it out. It was due to package "libyaml" package, which was not present on my local system but was installed on server.
Is it possible that you didn't stop and start your delayed_job worker when you deployed some new code? If a worker that was running before the deploy is trying to run new methods, it will fail.
Is it possible that YAML (or Syck) running in the worker process doesn't know about the method in question? Take a look at:
https://github.com/collectiveidea/delayed_job/wiki/Common-problems#wiki-jobs_are_silently_removed_from_the_database
... the relevant part is:
One common cause of deserialization errors is that the YAML references
a class not known to the worker. If this is the case, you can add
# file: config/initializers/custom.rb
require 'my_custom_class'
which will force my_custom_class to be loaded when the worker starts.
I had to restart my unicorns on the production server, by hand because for some reason cap deploy was not doing it for me.
So what I had to do was:
sudo /etc/init.d/unicorn_myapp stop
sudo /etc/init.d/unicorn_myapp start
But unicorn wasn't able to start, so I had to
sudo rm /tmp/unicorn.my_app.sock
And
sudo /etc/init.d/unicorn_myapp start