I have the following line in my config/environments/development.rb:
config.log_tags = [lambda { |req| Time.now }]
which works perfectly for the rails log (it prepends every entry with a timestamp).
I'd like to have the same behaviour in my resque workers.
The output of
p Rails.application.config.log_tags
in a resque job's perform method shows me, that there is a proc in it:
[#<Proc:0x007ffb36c14920#/.../config/environments/development.rb:49 (lambda)>]
But still, the log entries don't have timestamps.
I'm running out of ideas on why this happens...
Log tags are evaluated once per request in Rack middleware (Rails::Rack::Logger#call). There is no Web request when Resque runs and so you see nothing.
To get what you want, you should add code similar to Rails::Rack::Logger#compute_tags to Resque as a job hook, e.g., in before_perform.
Related
The idea is:
Perform some time consuming action in background.
Have the results from that action be propagated back to the controller using a callback.
Store the result in an in memory session store.
Have the result in session be used and available from that point onward.
Controller receives the results in the callback:
# controller callback, saves result to session
# this method is executed as expected
# session id matches the one from other controller actions
def on_counter_value_calculated(context, new_value)
#counter = new_value
session[:counter] = #counter
end
However, stored session is lost in subsequent calls:
# although the same session is targeted (same id)
# the saved value is not the same
def index
#counter = session[:counter] || 0
end
I've created a small Rails project that demonstrates the issue:
https://github.com/elvanja/controller_callbak_store_in_session
Any input appreciated.
Checked Rails code, and if I understand correctly, session in fact comes from request:
# actionpack-3.2.11/lib/action_controller/metal.rb:132
delegate :session, :to => "#_request"
It seems session is valid only within request cycle context and although it can be accessed, the changes are not saved, as demonstrated by the project.
Hence, this will not work. As suggested # Ruby on Rails Google Group and Ruby Rogues, the best way to deal with this is to use Sidekiq, DelayedJob, Resque or similar frameworks.
EDIT: Access `session` within rails controller thread is actually the reason (background processing in the example is done in a separate thread).
I want to prepopulate my cache with an initializer, but I don't need this code to run every time I run rake or rails g, etc. Rake and Bundler are easy to deal with, but a similar solution does not work for the generators:
# config/initializers/prepop_cache.rb
if !defined?(::Bundler) and !defined?(::Rake) and !defined(Rails::Generators)
# do stuff
end
This must be because rails/generators (or something similar) is requireed at runtime. How can I check to see if the command being run is rails g xyz?
Update:
Two solutions can be found here: Rails 3 initializers that run only on `rails server` and not `rails generate`, etc
Still would like to know if it's possible in the manner I've tried above.
In Rails 3, what you're looking to do is conceivably possible, but in a hacky way. Here's how:
When you make a rails generate call, the callpath looks like this:
bin/rails is called, which eventually routes you to execute script/rails
script/rails is executed which requires rails/commands
rails/commands is loaded, which is the main point of focus.
Within rails/commands the code that runs for generate:
ARGV << '--help' if ARGV.empty?
aliases = {
"g" => "generate",
"c" => "console",
"s" => "server",
"db" => "dbconsole"
}
command = ARGV.shift # <= #1
command = aliases[command] || command
case command
when 'generate', 'destroy', 'plugin', 'benchmarker', 'profiler'
require APP_PATH
Rails.application.require_environment! # <= #2
require "rails/commands/#{command}" # <= #3
The points of interest are numbered above. Namely, that at point #1 the command that you're running is shifting off of ARGV. Which in your case means generate is going to be removed from the command line args.
At point #2 your environment gets loaded, at which point your initializers are going to be executed. And herein is the tough part - because nothing indicating a specific command has been loaded at this point (this occurs at #3) there is no information to determine a generator is being run!
Let's insert a script into config/initializer/debug.rb to see what is available if we ran rails generate model meep:
puts $0 #=> "script/rails"
puts ARGV #=> ["model", "meep"]
As you can see, there is no direct information that a generator is being run. That said, there is indirect information. Namely ARGV[0] #=> "model". Conceivably you could create a list of possible generators and check to see if that generator has been called on ARGV[0]. The responsible developer in me says this is a hack and may break in ways you'd not expect so I'd use this cautiously.
The only other option is to modify script/rails like you suggested -- which isn't too bad a solution, but would likely break when you upgrade to Rails 4.
In Rails 4, you've got more hope! By the time the application environment is being loaded, the generators namespace has already been loaded. This means that in an initializer you could do the following:
if defined? Rails::Generators #=> "constant"
# code to run if generators loaded
else
# code to run if generators not loaded
end
Is there a way to add the session id to each log in Rails.
Now, I've added this in my environment.rb:
class Logger
def format_message(severity, timestamp, progname, msg)
"#{timestamp.to_formatted_s(:db)} #{severity} #{msg}\n"
end
end
First of all, is this best-practice? And how do I add the session id?
Unfortunately, giving :session_id to log_tags don't work yet (as of May 23 2012), since there is no session_id in Rack middleware.
Since log_tags array accept a Proc object, and the proc object is invoked with a request object, I could configure the log_tags following way:
config.log_tags = [ :uuid, :remote_ip,
lambda {|req| "#{req.cookie_jar["_session_id"]}" } ]
It's working with Rails 3.2.3, with ActiveRecord's session store is in use.
when you are working with rails 3.2 you can use log tags:
Update Rails::Rack::Logger middleware to apply any tags set in
config.log_tags to the newly ActiveSupport::TaggedLogging
Rails.logger. This makes it easy to tag log lines with debug
information like subdomain and request id -- both very helpful in
debugging multi-user production applications DHH
with some luck, adding this to your environment might work:
config.log_tags = [:uuid, :remote_ip, :session_id]
UPDATE
for the unlucky, the solution is from the answer of #shigeya.
unfortunately rails does NOT provide a way to access your sessionid in a sane manner.
you always have to rely on what rack is doing in it's internals... accessing the cookie_jar with your cookie key is the "recommended" way of doing so.
this is what i use for my application:
config.log_tags = [
:host,
:remote_ip,
lambda { |request| "#{request.uuid}"[0..15] },
lambda { |request| "#{request.cookie_jar["_session_id"]}"[0..15] },
]
output then looks like this:
[hamburg.onruby.dev] [127.0.0.1] [27666d0897c88b32] [BAh7B0kiD3Nlc3Np] Completed 200 OK in 298ms (Views: 286.2ms | ActiveRecord: 9.2ms)
For Rails 3.2 with ActiveSupport::TaggedLogging, if you're using :cookie_store:
config.log_tags = [ :uuid, :remote_ip,
lambda { |r| "#{r.cookie_jar.signed["_session_id"]["session_id"]}" } ]
Note: Change the "_session_id" with your :key value at config/initializers/session_store.rb
Unfortunately this isn't easy with Rails log tags. Moreover, it clutters your logs to the point where they are unreadable.
I'd recommend something like timber, it captures session ids (and more) by simply augmenting your logs with metadata. It's automatic and you don't lose readability.
My thinking is to capture the session_id and store it in thread local storage, e.g.
Thread.current[:session_id] = session[:session_id]
But some rails logging occurs before my filter is called.
I thought I might capture the session_id by writing a middleware plug-in. But again, I don't seem to get it early enough for all logging.
What is the earliest that I can capture session_id?
Thanks!
Since Rails 3.2 support tagged logging using log_tags configuration array, log_tags array accept a Proc object, and the proc object is invoked with a request object, I could configure the log_tags following way:
config.log_tags = [ :uuid, :remote_ip,
lambda {|req| "#{req.cookie_jar["_session_id"]}" } ]
It's working with Rails 3.2.3, with ActiveRecord's session store is in use.
Okay, I finally figured this out.
I moved ActionDispatch::Cookies and ActionDispatch::Session::CookieStore way earlier in the rack stack. This appears to be safe, and is necessary because otherwise some logging happens before the session is initialized.
I added my own rack middleware component that sets the session_id in thread local storage.
I override the rails logger and prepend the session_id to each log message.
This is very helpful in being able to separate out and analyze all logs for particular user session.
I'd be interested to know how anyone else accomplishes this.
Based on #Felix's answer. i've done these in rails 4:
# config/application.rb
config.middleware.delete "ActionDispatch::Cookies"
config.middleware.delete "ActionDispatch::Session::CookieStore"
config.middleware.insert_before Rails::Rack::Logger, ActionDispatch::Cookies
config.middleware.insert_before Rails::Rack::Logger, ActionDispatch::Session::CookieStore
# config/environment/development.rb and production.rb
config.log_tags = [
lambda {|req| "#{req.subdomain}/#{req.session["user_id"]}" },
:uuid
]
config.log_formatter = Logger::Formatter.new
This produces logs like this:
I, [2015-11-05T15:45:42.617759 #22056] INFO -- : [verimor/2] [77e593dc-c852-4102-a999-5c90ea0c9d66] Started GET "/home/dashboard" for 192.168.1.37 at 2015-11-05 15:45:42 +0200
[verimor/2] is subdomain/user_id (this is a multitenant app).
[77e593dc-c852-4102-a999-5c90ea0c9d66] is a unique id for this request. Useful for keeping track of the lifecycle of requests.
HTH.
For Rails 3.2 with ActiveSupport::TaggedLogging, if you're using :cookie_store:
config.log_tags = [ :uuid, :remote_ip,
lambda { |r| "#{r.cookie_jar.signed["_session_id"]["session_id"]}" } ]
Note: Change the "_session_id" with your :key value at config/initializers/session_store.rb
Related: https://stackoverflow.com/a/22487360/117382
In my application, in the homepage action, I call a specific web service that returns JSON.
parsed = JSON.parse(open("http://myservice").read)
#history = parsed['DATA']
This data will not change more than once per 60 seconds and does not change on a per-visitor basis, so i would like to, ideally, cache the #history variable itself (since the parsing will not result in a new result) and auto invalidate it if it is more than a minute old.
I'm unsure of the best way to do this. The default Rails caching methods all seem to be more oriented towards content that needs to be manually expired. I'm sure there is a quick and easy method to do this, I just don't know what it is!
You can use the built in Rails cache for this:
#history = Rails.cache.fetch('parsed_myservice_data', :expires_in => 1.minute) do
JSON.parse connector.get_response("http://myservice")
end
One problem with this approach is when the rebuilding of the data to be cached takes
quite a long time. If you get many client requests during this time, each of them will
get a cache miss and call your block, resulting in lots of duplicated effort, not to mention slow response times.
EDIT: In Rails 3.x you can pass the option :race_condition_ttl to the fetch method to avoid this problem. Read more about it here.
A good solution to this in previous versions of Rails is to setup a background/cron job to be run at regular intervals that will fetch and parse the data and update the cache.
In your controller or model:
#history = Rails.cache.fetch('parsed_myservice_data') do
JSON.parse connector.get_response("http://myservice")
end
In your background/cron job:
Rails.cache.write('parsed_myservice_data',
JSON.parse connector.get_response("http://myservice"))
This way, your client requests will always get fresh cached data (except for the first
request if the background/cron job hasn't been run yet.)
I don't know of an easy railsy way of doing this. You might want to look into using redis. Redis lets you set expiration times on the data you store in it. Depending on which redis gem you use it'd look something like this:
#history = $redis.get('history')
if not #history
#history = JSON.parse(open("http://myservice").read)['DATA']
$redis.set('history', #history)
$redis.expire('history', 60)
end
Because there's only one redis service this will work for all your rails processes.
We had a similar requirement and we ended up using Squid as a forward proxy for all the webservice calls from the rails server. Squid was configured to have a cache-expiry time of 60 seconds.
http_connection_factory.rb:
class HttpConnectionFactory
def self.connection
AppConfig.use_forward_proxy ? Net::HTTP::Proxy(AppConfig.forward_proxy_host, AppConfig.forward_proxy_port) : Net::HTTP
end
end
In your application's home page action, you can use the proxy instead of making the call directly.
connector = HttpConnectionFactory.connection
parsed = JSON.parse(connector.get_response("http://myservice"))
#history = parsed['DATA']
We had second thoughts about using Redis or Memcache. But, we had several service calls and wanted to avoid all the hassles of generating keys and sweeping them at appropriate times.
So, in our case, the forward proxy took care of all those nitty gritties. Please refer to Squid Wiki for the configuration parameters necessary.