This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 9 years ago.
I am using a delayed_jobs to run a task in the background.
I am starting a task using ajax, the worker gets some uuid and writes to cache the status of the task and the result.
then I use another ajax to poll every second and see if I got a result.
It works well on my localhost, but when I upload to heroku it does not.
I checked the logs, and I can see that the worker can read the cache it has writen, but when the main thread tries to access it its empty.
I am using thin server, memcachier and dalli.
This is the code used to write to the cache:
def self.get_meta_info(link_url,job_uuid)
begin
#..........
result = {
title: "stuff here..."
}
#..........
Rails.cache.write({job_uuid: job_uuid,type: 'result'},result.to_json)
Rails.cache.write({job_uuid: job_uuid,type: 'status'},'success')
#the next to lines return the data in the logs
Rails.logger.info("get_meta_info written to hash at #{job_uuid}")
Rails.logger.info("get_meta_info result for #{job_uuid} was: #{Rails.cache.read({job_uuid: job_uuid,type: 'result'})}")
rescue Exception => ex
Rails.cache.write({job_uuid: job_uuid,type: 'result'},ex)
Rails.cache.write({job_uuid: job_uuid,type: 'status'},'error')
end
end
This is the server side code I use for polling: (it is called by ajax every second)
def get_meta_info_result
job_uuid = params[:job_uuid]
status = Rails.cache.read({job_uuid: job_uuid,type: 'status'})
#the next to lines return nothing in the logs
Rails.logger.info("nlp_provider_controller.get_meta_info_result for uuid #{job_uuid} read status #{status}")
Rails.logger.info("nlp_provider_controller.get_meta_info_result for uuid #{job_uuid} read result #{Rails.cache.read({job_uuid: job_uuid,type: 'result'})}")
respond_to do |format|
if status=='success'
format.json {render json: Rails.cache.read({job_uuid: job_uuid,type: 'result'})}
elsif status=='error'
format.json{render :nothing => true, status: :no_content }
else
format.json{render :nothing => true, status: :partial_content }
end
end
I have no idea how to solve that.
Tank You!
Two days to solve this. Stupid mistake.
There are two configs, development.rb and production.rb. Not that I did not know that, but usually I config in a separate initializer.
I had the Redis configured in the delvelopment and not in the production one.
Added:
redis_url = ENV["REDISTOGO_URL"] || "redis://127.0.0.1:6379/0/MyApp"
MyApp::Application.config.cache_store = :redis_store, redis_url
(based on: http://blog.jerodsanto.net/2011/06/connecting-node-js-to-redis-to-go-on-heroku/)
and it works.
Related
The test below runs great... I just want to know specifically how long each test step takes. The test was originally exported from the selenium firefox plugin to ruby.
My plan is to add one more additional step that verifies the 'average chute time' page has completely loaded. And I'd really like to know how long it takes from the moment that the test clicks on "average chute time" until the page is fully loaded.
require "json"
require "selenium-webdriver"
gem "test-unit"
require "test/unit"
class LoginToChute< Test::Unit::TestCase
def setup
#driver = Selenium::WebDriver.for :chrome
#base_url = "https://devdb5.esosuite.net/EsoSuiteHotfixDaily/"
#accept_next_alert = true
#driver.manage.timeouts.implicit_wait = 30
#verification_errors = []
end
def teardown
assert_equal [], #verification_errors
#driver.quit
end
def test_login_to_chute
#driver.get(#base_url)
assert_equal "ESO Solutions :: NextGen", #driver.title
#driver.find_element(:id, "UserName").clear
#driver.find_element(:id, "UserName").send_keys "jenna"
#driver.find_element(:id, "Password").clear
#driver.find_element(:id, "Password").send_keys ".alice77."
#driver.find_element(:id, "AgencyLoginId").clear
#driver.find_element(:id, "AgencyLoginId").send_keys "wonderland"
#driver.find_element(:id, "btnLogin").click
#driver.find_element(:css, "img[alt=\"analytics\"]").click
#driver.find_element(:xpath, "//div[text() = 'ePCR Reports']").click
#driver.find_element(:xpath, "//div[text() = 'Operational Reports']").click
#driver.find_element(:xpath, "//div[text() = 'Average Chute Time']").click
end
def element_present?(how, what)
#driver.find_element(how, what)
true
rescue Selenium::WebDriver::Error::NoSuchElementError
false
end
def alert_present?()
#driver.switch_to.alert
true
rescue Selenium::WebDriver::Error::NoAlertPresentError
false
end
def verify(&blk)
yield
rescue Test::Unit::AssertionFailedError => ex
#verification_errors << ex
end
def close_alert_and_get_its_text(how, what)
alert = #driver.switch_to().alert()
alert_text = alert.text
if (#accept_next_alert) then
alert.accept()
else
alert.dismiss()
end
alert_text
ensure
#accept_next_alert = true
end
end
You should analyze this with a client side analysis like google dev tools. Determine what the last thing on the page is to actually load. This of course would give you a good analysis of performance itself, but going with the automated approach...
You then take a system timestamp and save it as a variable before the #driver.find_element(:xpath, "//div[text() = 'Average Chute Time']").click line of your test. Execute all your code in your test and finish with a wait for the element you identified as the last to be loaded after your execution finishes. Right after the wait take a timestamp and get the difference between the two timestamps.
Naturally this method would work for any duration you are wanting to measure you just need to organize your test accordingly so that you are measuring the exact step executions you want to measure. The waits for specific elements are important to ensure the client is finished rendering everything from a user standpoint. In general performance tricks are rendering the page and controls and then modifying the DOM via javascript so that it appears the page is there before all components and control contents are actually actionable to the user. So be careful what you are actually measuring and don't just measure a raw pageload.
Repeat this for your specific loads and have separate measures for each. I have also implemented a wrapper that provides times for every step. Then have separate functions which I can start and stop a timer for groups of steps to measure. The choice is really up to what numbers you specifically want to capture.
I have a controller action that creates 2 background jobs to be run at a future date.
I am trying to test that the background jobs get run
# spec/controllers/job_controller_spec.rb
setup
post :create, {:job => valid_attributes}
Delayed::Job.count.should == 2
Delayed::Worker.logger = Rails.logger
#Delayed::Worker.new.work_off.should == [2,0]
Delayed::Worker.new.work_off
Delayed::Job.count.should == 0 # this is where it fails
This is the error:
1) JobsController POST create with valid params queues up delayed job and fires
Failure/Error: Delayed::Job.count.should == 0
expected: 0
got: 2 (using ==)
For some reason it seems like it is not firing.
You can try to use
Delayed::Worker.new(quiet: false).work_off
to debug the result of your background jobs, this could help you to find out if the fact that they're supposed to run in the future is messing with the assert itself.
Don't forget to take off the "quiet:false" when you're done, otherwise your tests will always output the results of the background jobs.
I have a Cucumber Scenario for testing UI features. Sometimes due to one of the several issues, web-page takes lot of time to respond and Capybara times out with following error.
ruby-1.9.3-p327/lib/ruby/1.9.1/net/protocol.rb:146:in `rescue in rbuf_fill'
ruby-1.9.3-p327/lib/ruby/1.9.1/net/protocol.rb:140:in `rbuf_fill'
ruby-1.9.3-p327/lib/ruby/1.9.1/net/protocol.rb:122:in `readuntil'
ruby-1.9.3-p327/lib/ruby/1.9.1/net/protocol.rb:132:in `readline'
ruby-1.9.3-p327/lib/ruby/1.9.1/net/http.rb:2562:in `read_status_line'
ruby-1.9.3-p327/lib/ruby/1.9.1/net/http.rb:2551:in `read_new'
My question is-
Can I somehow force Cucumber scenario or Capybara to retry (for constant number of times) whole scenario or step respectively, on timeout error?
Maybe, you can do it like this:
Around do |scenario, block|
for i in 1..5
begin
block.call
break
rescue Timeout::Error
next
end
end
end
But I can't figure out if this code works because of the bug (It's not possible to call block several times in Around hook)
From The Cucumber book:
Add a eventually method that keeps trying to run a block of code until it either stops raising an error or it reaches a time limit.
Here is the code for that method:
module AsyncSupport
def eventually
timeout = 2
polling_interval = 0.1
time_limit = Time.now + timeout
loop do
begin
yield
rescue Exception => error
end
return if error.nil?
raise error if Time.now >= time_limit sleep polling_interval
end
end
end
World(AsyncSupport)
The method called be called as follows from a step_definition:
Then /^the balance of my account should be (#{CAPTURE_CASH_AMOUNT})$/ do |amount|
eventually { my_account.balance.should eq(amount) }
end
I recollect getting log files that were nicely ordered, so that you could follow one request, then the next, and so on.
Now, the log files are, as my 4 year old says "all scroggled up", meaning that they are no longer separate, distinct chunks of text. Loggings from two requests get intertwined/mixed up.
For instance:
Started GET /foobar
...
Completed 200 OK in 2ms (Views: 0.4ms | ActiveRecord: 0.8ms)
Patient Load (wait, that's from another request that has nothing to do with foobar!)
[ blank space ]
Something else
This is maddening, because I can't tell what's happening within one single request.
This is running on Passenger.
I tried to search for the same answer but couldn't find any good info. I'm not sure if you should fix server or rails code.
If you want more info about the issue here is the commit that removed old way of logging https://github.com/rails/rails/commit/04ef93dae6d9cec616973c1110a33894ad4ba6ed
If you value production log readability over everything else you can use the
PassengerMaxInstancesPerApp 1
configuration. It might cause some scaling issues. Alternatively you could stuff something like this in application.rb:
process_log_filename = Rails.root + "log/#{Rails.env}-#{Process.pid}.log"
log_file = File.open(process_log_filename, 'a')
Rails.logger = ActiveSupport::BufferedLogger.new(log_file)
Yep!, they have made some changes in the ActiveSupport::BufferedLogger so it is not any more waiting until the request has ended to flush the logs:
http://news.ycombinator.com/item?id=4483390
https://github.com/rails/rails/commit/04ef93dae6d9cec616973c1110a33894ad4ba6ed
But they have added the ActiveSupport::TaggedLogging which is very funny and you can stamp every log with any kind of mark you want.
In your case could be good to stamp the logs with the request UUID like this:
# config/application.rb
config.log_tags = [:uuid]
Then even if the logs are messed up you still can follow which of them correspond to the request you are following up.
You can make more funny things with this feature to help you in your logs study:
How to log user_name in Rails?
http://zogovic.com/post/21138929607/running-time-in-rails-logs
Well, for me the TaggedLogging solution is a no go, I can live with some logs getting lost if the server crashes badly, but I want my logs to be perfectly ordered. So, following advice from the issue comments I'm applying this to my app:
# lib/sequential_logs.rb
module ActiveSupport
class BufferedLogger
def flush
#log_dest.flush
end
def respond_to?(method, include_private = false)
super
end
end
end
# config/initializers/sequential_logs.rb
require 'sequential_logs.rb'
Rails.logger.instance_variable_get(:#logger).instance_variable_get(:#log_dest).sync = false
As far as I can say this hasn't affected my app, it is still running and now my logs make sense again.
They should add some quasi-random reqid and write it in every line regarding one single request. This way you won't get confused.
I haven't used it, but I believe Lumberjack's unit_of_work method may be what you're looking for. You call:
Lumberjack.unit_of_work do
yield
end
And all logging done either in that block or in the yielded block are tagged with a unique ID.
Let's say I have delayed_job running in the background. Tasks can be scheduled or run immediately(some are long tasks some are not)
If a task is too long, a user should be able to cancel it. Is it possible in delayed job? I checked the docs and can't seem to find a terminate method or something. They only provide a catch to cancel delayed job itself(thus cancelling all tasks...I need to just cancel a certain running task)
UPDATE
My boss(who's a great programmer btw) suggested to use Ruby Threading for this feature of ours. Is this possible? Like creating new threads per task and killing that thread while it's running?
something like:
t1 = Thread.new(task.run)
self.delay.t1.join (?) -- still reading on threads so correct me if im wrong
then to stop it i'll just use t1.stop (?) again don't know yet
Is this possible? Thanks!
It seems that my boss hit the spot so here's what we did(please tell us if there's some possibility this is bad practice so I can bring it up):
First, we have a Job model that has def execute! (which runs what it's supposed to do).
Next, we have delayed_job worker in the background, listening for new jobs. Now when you create a job, you can schedule it to run immediately or run every certain day (we use rufus for this one)
When a job is created, it checks if its supposed to run immediately. If it is, it adds itself to the delayed job queue. The execute function creates a Thread, so each job has its own thread.
User in the ui can see if a job is running(if there's a started_at and no finished_at). If it IS running, there's a button to cancel it. Canceling it just sets the job's canceled_at to Time.now.
While the job is running it also checks itself if it has a canceled_at or if Time.now is > finished_at. If so, kill the thread.
Voila! We've tested it for one job and it seems to work. Now the only problem is scaling...
If you see any problems with this please do so in the comments or give more suggestions if ever :) I hope this helps some one too!
Delayed::Job is an < ActiveRecord::Base model, so you can query it just like you normally would like Delayed::Job.all(:conditions => {:last_error => nil}).
Delayed::Job objects have a payload field which contain a serialized version of the method or job that you're attempting to run. This object is accessed by their '#payload_object' method, which loads the object in question.
You can combine these two capabilities to make queriable job workers, for instance, if you have a User model, and the user has a paperclip'ed :avatar, then you can make a method to delete unprocessed jobs like so:
class User < ActiveRecord::Base
has_attached_file :avatar, PaperclipOptions.new(:avatar)
before_create :'process_avatar_later'
def process_avatar_later
filename = Rails.root.join('tmp/avatars_for_processing/',self.id)
open(filename, 'w') do |file| file <<self.avatar.to_file end
Delayed::Job.enqueue(WorkAvatar.new(self.id, filename))
self.avatar = nil
end
def cancel_future_avatar_processing
WorkAvatar.future_jobs_for_user(self.id).each(&:destroy)
#ummm... tell them to reupload their avatar, I guess?
end
class WorkAvatar < Struct.new(:user_id, :path)
def user
#user ||= User.find(self.user_id)
end
def self.all_jobs
Delayed::Job.scoped(:conditions => 'payload like "%WorkAvatar%"')
end
def self.future_jobs_for_user(user_id)
all_jobs.scoped(:conditions => {:locked_at => nil}).select do |job|
job.payload_object.user_id == user_id
end
end
def perform
#user.avatar = File.open(path, 'rb')
#user.save()
end
end
end
It's possible someone has made a plugin make queryable objects like this. Perhaps searching on GitHub would be fruitful.
Note also that you'd have to work with any process monitoring tools you might have to cancel any running job worker processes that are being executed if you want to cancel a job that has locked_at and locked_by set.
You can wrap the task into a Timeout statement.
require 'timeout'
class TaskWithTimeout < Struct.new(:parameter)
def perform
Timeout.timeout(10) do
# ...
end
rescue Timeout::Error => e
# the task took longer than 10 seconds
end
end
No, there's no way to do this. If you're concerned about a runaway job you should definitely wrap it in a timeout as Simone suggests. However, it sounds like you're in search of something more but I'm unclear on your end goal.
There will never be a way for a user to have a "cancel" button since this would involve finding a method to directly communicate with the worker running process running the job. It would be possible to add a signal handler to the worker so that you could do something like kill -USR1 pid to have it abort the job it's currently working and move on. Would this accomplish you goal?