Resque with Rails 3 + Heroku + RedisToGo giving Postgresql error - ruby-on-rails-3

Question

I'm writing an app which needs to send many emails and creates many user notifications because of these emails. This task produces a timeout in Heroku. To solve this, I decided to use Resque and RedistToGo.
What I did was to send the email (it's actually just one email because we use Sendgrid to handle this) and create the notifications using a Resque worker. The email is already created, so I send its id to the worker, along with all the recipients.
This works fine locally. In production, unless we restart our app in Heroku, it only works once. I will post some of my code and the error message:
#lib/tasks/resque.rake
require 'resque/tasks'
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
end
desc "Alias for resque:work (To run workers on Heroku)"
task "jobs:work" => "resque:work"
#config/initalizers/resque.rb
ENV["REDISTOGO_URL"] ||= "redis://redistogo:some_hash#some_url:some_number/"
uri = URI.parse(ENV["REDISTOGO_URL"])
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
Dir["#{Rails.root}/app/workers/*.rb"].each { |file| require file }
#app/workers/massive_email_sender.rb
class MassiveEmailSender
#queue = :massive_email_queue
def self.perform(email_id, recipients)
email = Email.find(email_id.to_i)
email.recipients = recipients
email.send_email
end
end
I've got an Email model which has an after_create that enqueues the worker:
class Email < ActiveRecord::Base
...
after_create :enqueue_email
def enqueue_email
Resque.enqueue(MassiveEmailSender, self.id, self.recipients)
end
...
end
This Email model also has the send_email method which does what I said before
I'm getting the following error message. I'm gonna post all the information Resque gives to me:
Worker
9dddd06a-2158-464a-b3d9-b2d16380afcf:1 on massive_email_queue at just now
Retry or Remove
Class
MassiveEmailSender
Arguments
21
["some_email_1#gmail.com", "some_email_2#gmail.com"]
Exception
ActiveRecord::StatementInvalid
Error
PG::Error: SSL error: decryption failed or bad record mac : SELECT a.attname, format_type(a.atttypid, a.atttypmod), d.adsrc, a.attnotnull FROM pg_attribute a LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum WHERE a.attrelid = '"emails"'::regclass AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1139:in `async_exec'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1139:in `exec_no_cache'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:663:in `block in exec_query'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/abstract_adapter.rb:280:in `block in log'
/app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.2/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/abstract_adapter.rb:275:in `log'
/app/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.3.2/lib/new_relic/agent/instrumentation/active_record.rb:31:in `block in log_with_newrelic_instrumentation'
/app/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.3.2/lib/new_relic/agent/method_tracer.rb:242:in `trace_execution_scoped'
/app/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.3.2/lib/new_relic/agent/instrumentation/active_record.rb:28:in `log_with_newrelic_instrumentation'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:662:in `exec_query'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1264:in `column_definitions'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:858:in `columns'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/schema_cache.rb:12:in `block in initialize'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:228:in `yield'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:228:in `default'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:228:in `columns'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:237:in `columns_hash'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/delegation.rb:7:in `columns_hash'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/finder_methods.rb:330:in `find_one'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/finder_methods.rb:311:in `find_with_ids'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/finder_methods.rb:107:in `find'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/querying.rb:5:in `find'
/app/app/workers/massive_email_sender.rb:5:in `perform'
According to this, the first argument is the email id, and the second one is the list of all recipients... exactly as it should be.
Can anyone help me? Thanks!

Answer 1

I've run into the same problem. Assuming you're using Active Record you have to call ActiveRecord::Base.establish_connection for each forked Resque worker to make sure it doesn't have a stale database connection. Try putting this in your lib/tasks/resque.rake
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
end