ruby resque without loading rails environment - ruby-on-rails-3

Question

I have a resque worker which works great but is just too slow. The main reason for this is I'm using activerecord and having to load the entire environment which takes at least 10-20 seconds just to load up (I don't keep a running worker at all times as I'm using Heroku and pay for the time the worker runs). I'm using a resque worker to grab & parse data from an external website and then dumping the data into my database.
My question is whether I should rewrite the method to not use Rails and instead use DataMapper? Or something else which would load faster than activerecord.
Or If I should extract the code (using ActiveRecord) which figures out what to do with the external data and move it out of the worker and back into the app?
Hope that makes sense.

Answer 1

I have the same problem.
you could setup your environment on the rake resque:setup rake task
I tried this. assuming my rake resque task is on lib/tasks/resque.rake
require "resque/tasks"
task "resque:setup" do
root_path = "#{File.dirname(__FILE__)}/../.."
db_config = YAML::load(File.open(File.join(root_path,'config','database.yml')))["development"]
ActiveRecord::Base.establish_connection(db_config)
require "#{root_path}/app/workers/photo_downloader.rb" #workers
#Dir.glob("#{root_path}/app/models/*").each { |r| puts r; require r } #require all model
require "#{root_path}/app/models/photo.rb" # require model individually
end
I haven't completely success beacuse I use the Paperclip gem which require rails environment

Answer 2

Rails’ bootstrap is really slow; it is intended to be kept running, until certain time for restart (to eliminate some memory leaks there most likely is, any software is not bug-free), and is not intended to be used as a site that is launched for one request and then shut down.
That kind of usage more resembles a script. If you need to launch it with browser, you can easily use something like Erubis to write the page and use ActiveRecord in the script (I think it was useable outside of rails) or similar abstraction layer. Myself, for small tasks, I just use Mysql2.

Answer 3

Use bundler to get active_record and other gem to you without rails application .
require 'rubygems'
require 'logger'
require 'active_record'
require 'bundler'
require "active_support"
require "spreadsheet"
require 'net/ping'
require 'net/http'
Bundler.setup
Bundler.require(:default) if defined?(Bundler)
$config_logger = Logger.new("./log/dev.log")
class Dbconnect
def initialize
#settings = YAML.load_file('./config/database.yml')["development"]
#adapter = #settings["adapter"] if #settings["adapter"]
#database = #settings["database"] if #settings["database"]
#pool = #settings["pool"] if #settings["pool"]
#timeout = #settings["timeout"] if #settings["timeout"]
end
def connect_to_db
ActiveRecord::Base.establish_connection(
:adapter => #adapter,
:database => #database,
:reconnect => #reconnect,
:pool => #pool,
:timeout => #timeout)
$config_logger.info "\n db Connected: to => #{#database} "
end
end
end
}
Example Gemfile :
source "http://rubygems.org"
gem 'mail'
gem "escape_utils"
gem 'json',:require => "json"
gem 'json_pure'
gem 'resque'
gem 'resque-scheduler'
gem 'redis-namespace'
gem 'resque-status'
gem 'rake'
gem 'em-udns'
gem 'sqlite3'
gem 'spreadsheet'
gem 'activerecord', '3.2.1', :require => "active_record"
gem 'net-scp', :require => 'net/scp'
gem 'net-sftp', :require => 'net/sftp'
gem 'net-ssh', :require => 'net/ssh'
gem 'dir'
gem 'amatch'
gem 'haml'
gem 'net-ping'
gem install bundler
rest of the thing : bundle install .