.worker file with `gemfile "Gemfile", "group"` increases remote build time x10 - iron.io

To better manage the required gems across my application, I created a Gemfile group called :iron:
group :default, :iron do
  gem "activerecord", require: 'active_record
  gem 'mysql2'
  gem 'aws-sdk'
  gem 'yajl-ruby'
  gem 'hashie'
  gem 'require_all'
end
This would help consolidate the required gems for iron worker. My .worker file is now as follows:
gemfile '../Gemfile', 'iron'
instead of redundantly listing all the gems individually:
gem "activerecord"
gem 'mysql2'
...
Unfortunately, this increased my remote build time from 1 minute to 10 minutes. The output attempts to include far more gems, seemingly bubbling up nested dependencies:
BEFORE:
------> Creating code package
Found workerfile with path='iron_job.worker'
Detected exec with path='iron_job_bootstrapper.rb' and args='{}'
Merging dir with path='../app/models' and dest=''
Merging dir with path='../lib' and dest=''
Merging file with path='iron.json' and dest=''
Adding ruby gem dependency with name='activerecord' and version='>= 0'
Adding ruby gem dependency with name='mysql2' and version='>= 0'
Adding ruby gem dependency with name='aws-sdk' and version='>= 0'
Adding ruby gem dependency with name='yajl-ruby' and version='>= 0'
Adding ruby gem dependency with name='hashie' and version='>= 0'
Adding ruby gem dependency with name='require_all' and version='>= 0'
AFTER:
------> Creating code package
Found workerfile with path='iron_job.worker'
Detected exec with path='iron_job_bootstrapper.rb' and args='{}'
Merging dir with path='../app/models' and dest=''
Merging dir with path='../lib' and dest=''
Merging file with path='iron.json' and dest=''
Adding ruby gems dependencies from iron group of ../Gemfile
Adding ruby gem dependency with name='i18n' and version='0.6.5'
Adding ruby gem dependency with name='minitest' and version='4.7.5'
Adding ruby gem dependency with name='multi_json' and version='1.8.1'
Adding ruby gem dependency with name='atomic' and version='1.1.14'
Adding ruby gem dependency with name='thread_safe' and version='0.1.3'
Adding ruby gem dependency with name='tzinfo' and version='0.3.37'
Adding ruby gem dependency with name='activesupport' and version='4.0.0'
Adding ruby gem dependency with name='builder' and version='3.1.4'
Adding ruby gem dependency with name='activemodel' and version='4.0.0'
Adding ruby gem dependency with name='activerecord-deprecated_finders' and version='1.0.3'
Adding ruby gem dependency with name='arel' and version='4.0.0'
Adding ruby gem dependency with name='activerecord' and version='4.0.0'
Adding ruby gem dependency with name='json' and version='1.8.0'
Adding ruby gem dependency with name='mini_portile' and version='0.5.1'
Adding ruby gem dependency with name='nokogiri' and version='1.6.0'
Adding ruby gem dependency with name='uuidtools' and version='2.1.4'
Adding ruby gem dependency with name='aws-sdk' and version='1.11.1'
Adding ruby gem dependency with name='bundler' and version='1.3.5'
Adding ruby gem dependency with name='hashie' and version='2.0.5'
Adding ruby gem dependency with name='mysql2' and version='0.3.13'
Adding ruby gem dependency with name='require_all' and version='1.3.1'
Adding ruby gem dependency with name='yajl-ruby' and version='1.1.0'
This output was local, but the time stretched out even more on the IronWorker service because it reinstalls gems more than once (uninstall then reinstalls bundler). The log below has the details.
https://hud.iron.io/tq/projects/5254773dd05880000d000003/tasks/525ee1d9f8953468b927e83f/log
We have since gone back to a .worker file that simply lists out the gems we need rather than invoke gemfile "Gemfile", "iron". Any help on how to use gemfile without reinstalling bundler and have a 10m upload time would be much appreciated!
Thanks.
Edit
Adding full Gemfile. Please note that we have no desire to pull in all the gems on the :default group. That is overkill for our workers. We just want :iron.
Bundler version 1.3.5
source 'https://rubygems.org'
ruby '1.9.3'
gem 'typhoeus' # Only for ruby 1.9.3
gem 'rails', '4.0.0'
gem 'sass-rails', '~> 4.0.0'
gem 'uglifier', '>= 1.3.0'
gem 'coffee-rails', '~> 4.0.0'
gem 'jquery-rails'
gem 'turbolinks'
gem 'haml-rails'
gem 'thin-rails'
gem 'therubyracer'
gem 'less-rails'
gem 'twitter-bootstrap-rails'
gem 'settingslogic'
gem 'iron_worker_ng'
group :default, :iron do
gem "activerecord", require: 'active_record'
gem 'mysql2'
gem 'aws-sdk'
gem 'yajl-ruby'
gem 'require_all'
end
group :doc do
# bundle exec rake doc:rails generates the API under doc/api.
gem 'sdoc', require: false
end
group :production, :qa do
gem 'rails_12factor' #for heroku
end
group :development, :test do
gem 'factory_girl_rails'
gem 'ffaker'
gem 'shoulda-matchers'
gem 'pry'
gem 'debugger', '>= 1.6.1'
gem 'pry-debugger'
gem 'pivotal_git_scripts'
gem 'rspec-rails'
gem 'capybara'
gem 'fuubar'
end
group :test do
gem 'database_cleaner'
end
grouped Gemfile:
source 'https://rubygems.org'
ruby '1.9.3'
group :rails do
gem 'typhoeus' # Only for ruby 1.9.3
gem 'rails', '4.0.0'
gem 'sass-rails', '~> 4.0.0'
gem 'uglifier', '>= 1.3.0'
gem 'coffee-rails', '~> 4.0.0'
gem 'jquery-rails'
gem 'turbolinks'
gem 'haml-rails'
gem 'thin-rails'
gem 'therubyracer'
gem 'less-rails'
gem 'twitter-bootstrap-rails'
gem 'settingslogic'
gem 'rack-mini-profiler'
end
group :rails, :iron do
gem "activerecord", require: 'active_record'
gem 'mysql2'
gem 'aws-sdk'
gem 'yajl-ruby'
gem 'require_all'
gem 'hashie'
gem 'iron_worker_ng'
end
group :doc do
# bundle exec rake doc:rails generates the API under doc/api.
gem 'sdoc', require: false
end
group :production, :qa do
gem 'rails_12factor' #for heroku
end
group :development, :test do
gem 'factory_girl_rails'
gem 'ffaker'
gem 'shoulda-matchers'
gem 'pry'
gem 'debugger', '>= 1.6.1'
gem 'pry-debugger'
gem 'pivotal_git_scripts'
gem 'rspec-rails'
gem 'capybara'
gem 'fuubar'
end
group :test do
gem 'database_cleaner'
end

IronWorker already announced a "Docker Workflow" and you don't need the .worker file anymore. You just need to install all dependencies locally in docker image reproducing the same environment as running on IronWorker servers. Here you can find a ruby "HelloWorld" example.

Related

rails 5.1.4 console in production doesn't find models

I think I got a configuration issue with my rails setup in production.
I've got a puma working fine without any issues (connecting to db, loading models etc) and previously was able to use the console (I've encountered this [bug][https://github.com/rails/rails/issues/19256] but worked around it by filling in the database.yml)
When I try to open the console to perform little operations, none of my model classes are found.
✗ bundle exec rails console production
Running via Spring preloader in process 18313
Loading production environment (Rails 5.1.4)
irb(main):001:0> User.count
NameError: uninitialized constant User
from (irb):1
irb(main):002:0>
Similar questions on SO recommend to run ActiveRecord::Base.subclasses and eventually Rails.application.eager_load!
When I do try to run Rails.application.eager_load! I get uninitializaed constant error
irb(main):005:0> Rails.application.eager_load!
NameError: uninitialized constant ApplicationController
Did you mean? ApplicationCable
from app/controllers/accounts_controller.rb:1:in `<top (required)>'
from (irb):5
So I guess my console is now somehow lost and hasn't loaded the project source correctly.
How can I troubleshoot this situation ?
edit 2018-02-15 : I connected my workstation to the prod environment to use the console. This worked at first, the console was behaving correctly. After a couple times the same issues appeared. After doing spring stop I was able again to user my local workstation to open a console in prod.
Interesting fact : this never happens in dev environment, although spring had an app running in this environment.
It turns out spring is also running in prod, although the spring gem is only in dev group. (see below)
Gemfile
source 'https://rubygems.org'
git_source(:github) do |repo_name|
repo_name = "#{repo_name}/#{repo_name}" unless repo_name.include?("/")
"https://github.com/#{repo_name}.git"
end
gem 'rails', '~> 5.1.4'
gem 'puma', '~> 3.7'
gem 'sass-rails', '~> 5.0'
gem 'uglifier', '>= 1.3.0'
gem 'therubyracer', platforms: :ruby
gem 'coffee-rails', '~> 4.2'
gem 'jbuilder', '~> 2.5'
group :development, :test do
gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
gem 'capybara', '~> 2.13'
gem 'selenium-webdriver'
gem 'rails-erd', require: false, group: :development
end
group :development do
gem 'web-console', '>= 3.3.0'
gem 'listen', '>= 3.0.5', '< 3.2'
gem 'spring'
gem 'spring-watcher-listen', '~> 2.0.0'
gem 'pry'
gem 'pry-byebug'
end
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]
gem 'devise'
gem 'devise-i18n'
gem 'cancancan', '~> 2.0'
gem 'rolify'
gem 'redis', '~> 3.2'
group :production do
gem "sidekiq"
gem "sentry-raven"
end
gem 'pg'
gem "paperclip", "~> 5.0.0"
gem 'i18n-country-translations'
gem 'rails-i18n', '~> 5.0.0'
gem 'i18n_alchemy'
gem 'wicked_pdf'
gem 'wkhtmltopdf-binary'
If you deployed through Docker make sure using -ti prefix in running command like:
docker exec -ti 4d318d505e9f rails c production

undefined method `page' for # Kaminari::PaginatableArray

I have been using Kaminari since a while. I use to paginate generic arrays using the following line.
Kaminari.paginate_array([1,2,3]).page(1).per(1)
That use to work before but I noticed that since yesterday this show the error
undefined method `page' for #<Kaminari::PaginatableArray:0x007f8e5d10fde0>
However calling page on ActiveRecord relation is working fine. something like this is working
User.page(1).per(1)
My Gem file
source 'https://rubygems.org'
ruby '1.9.3'
gem 'rails', '3.2.8'
# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'
gem 'mysql2'
gem 'pg'
# Ok lets try out the heroku business
gem 'heroku'
# Gems used only for assets and not required
# in production environments by default.
group :assets do
gem 'sass-rails', '~> 3.2.3'
gem 'coffee-rails', '~> 3.2.1'
# See https://github.com/sstephenson/execjs#readme for more supported runtimes
# gem 'therubyracer', :platforms => :ruby
gem 'uglifier', '>= 1.0.3'
end
gem 'jquery-fileupload-rails'
gem 'jquery-rails'
gem 'jquery-ui-rails'
# Twitter Boostrap and Upgrade dependencies
gem 'therubyracer'
gem 'less-rails'
gem 'twitter-bootstrap-rails'
# To use ActiveModel has_secure_password
# gem 'bcrypt-ruby', '~> 3.0.0'
# To use Jbuilder templates for JSON
# gem 'jbuilder'
# Use unicorn as the app server
gem 'unicorn'
# Deploy with Capistrano
gem 'capistrano'
gem 'cap_bootstrap', github: 'benrs44/cap_bootstrap'
gem 'awesome_print'
group :test, :development do
gem 'rspec-rails', '>= 2.0.1'
gem "capybara-webkit"
gem 'capybara'
gem 'launchy'
gem 'selenium-webdriver'
gem 'shoulda-matchers'
gem 'ci_reporter'
gem "factory_girl_rails", "~> 4.0"
# DRb server RSpec
gem "spork-rails"
# To use debugger
gem 'debugger'
end
group :test do
gem "guard-rspec"
gem 'rb-fsevent', '~> 0.9.1'
gem 'database_cleaner'
gem 'fuubar'
end
#Image Uploader
gem 'carrierwave'
#Image Processor
gem "rmagick"
#Video Processor
gem 'video_info'
gem 'hpricot'
#Authenitcation
gem 'omniauth-facebook', '1.4.0'
#Messaging
gem 'mailboxer', github: 'rralston/mailboxer'
gem 'haml-rails'
gem 'simple_form'
#Gelocation
gem 'geocoder'
#Search
gem 'ransack'
# For easily making nouns possessive
gem 'possessive'
#Pagination
gem "kaminari"
gem 'will_paginate', '~> 3.0'
#Web App Monitoring
gem 'newrelic_rpm'
#FB Style Newsfeeds
gem 'public_activity'
# Heroku CDN Link to AWS
gem "asset_sync"
# Carrier Wave link to AWS S#
gem 'fog', '~> 1.12.1'
gem 'cancan'
# client side validtions
gem 'client_side_validations'
# tagging
gem 'acts-as-taggable-on'
# remote file uploads
gem "remotipart", "~> 1.2.1"
# social sharing
gem 'social-share-button'
gem 'devise', "~> 2.2.3"
gem 'omniauth'
gem 'devise-async'
running bundle show bundle show kaminari gives /Users/***/.rvm/gems/ruby-1.9.3-p448/gems/kaminari-0.14.1
Turned out to be that my colleague has added a configuration file for Kaminari and changed this page function with this line.
config.page_method_name = :kaminari_page
So instead of using Kaminari.paginate_array([1,2,3]).page(1).per(1) I should now use
Kaminari.paginate_array([1,2,3]).kaminari_page(1).per(1)

Bundled versions of rspec and cucumber gems in a new rails 3.2 app

I am experiencing a problem with the cucumber and rspec gems bundled in a rails app.
This is what my Gemfile looks like in a new Rails 3.2.11 application with RSpec support added:
source 'https://rubygems.org'
gem 'rails', '3.2.11'
gem 'sqlite3'
gem 'jquery-rails'
group :assets do
gem 'sass-rails', '~> 3.2.3'
gem 'coffee-rails', '~> 3.2.1'
gem 'uglifier', '>= 1.0.3'
end
group :test do
gem 'rspec-rails'
end
After running bundle, Gemfile.lock reports that rspec 2.12.2 is being used (the latest version of the gem, as of this writing).
But I also want to use cucumber, so I run bundle update after modifying Gemfile like so:
group :test do
gem 'rspec-rails'
gem 'cucumber-rails'
end
To my surprise, the bundled rspec is now version 2.0.1, definitely NOT cool.
In fact, this was causing all kinds of errors when running specs, and it took me a while to figure out that they were old rspec errors, fixed long ago.
I can force bundler to use the latest gem:
group :test do
gem 'rspec-rails', '>= 2.12.0'
gem 'cucumber-rails'
end
but I am unhappy, because I do not understand:
if I did something wrong to begin with
if the fix I applied is going to cause other problems down the road
if there was a way to predict this behavior
Any insights?
According to https://github.com/cucumber/cucumber-rails/blob/master/cucumber-rails.gemspec#L22, cucumber-rails depends on 'rspec-rails', '~> 2.10.1' (which means >= 2.10.1 and < 2.11). If any other upstream dependency depends on a version of rspec < 2.10.1 or >= 2.11, bundler should raise an error, so I can't understand how this happened.
As an aside, you should include rspec-rails in both the :development and :test groups (per https://github.com/rspec/rspec-rails#configure).

test-unit gem displays test stat for every rake run

I using test-unit gem with a Rails 3.1 project, and I have a small issue. If I start rake tasks, I always see a test result, with zeros.
My Gemfile is:
source :rubygems
gem 'rails', '3.1.2'
gem 'sqlite3'
gem 'json_pure'
# Gems used only for assets and not required
# in production environments by default.
group :assets do
# gem 'sass-rails', '~> 3.1.5.rc.2'
# gem 'coffee-rails', '~> 3.1.1'
gem 'therubyracer'
gem 'uglifier', '>= 1.0.3'
end
gem 'jquery-rails'
# Rails i18n
gem 'rails-i18n', :git => 'git://github.com/hron84/rails-i18n.git'
# Paginator
gem 'will_paginate'
# To use ActiveModel has_secure_password
gem 'bcrypt-ruby', '~> 3.0.0'
# Use unicorn as the web server
gem 'thin'
# Deploy with Capistrano
gem 'capistrano'
# To use debugger
# gem 'ruby-debug'
group :development do
gem 'rails3-generators'
gem 'nifty-generators'
end
group :test do
gem 'rspec'
gem 'rspec-rails'
gem 'ci_reporter'
gem 'factory_girl'
unless defined? Rubinius
gem 'rcov'
gem 'rcov_rails'
gem 'test-unit'
end
gem 'ruby-prof'
gem 'faker'
end
# Application-related stuffs
gem 'authlogic'
# Slug helper
gem 'slug'
# vim: ft=Gemfile
Is there a way to avoid it? What I do wrong?
Check your RAILS_ENV. Make sure it's set to "development", not "test". If that's not the culprit, try removing gems (particularly test/spec-related ones) one by one until the problem goes away. Then you'll know which gem is giving you the problem.

Rails 3 Cucumber Load Error

I am trying to get cucumber to work with Rails 3.
But whenever I try to run the cucumber features I get an load error.
cucumber-0.7.3/lib/cucumber/ast/feature_element.rb:2:in `require': no such file to load -- gherkin/parser/tag_expression (LoadError)
I have added the following gems to the Gemfile
gem "rspec", '>= 2.0.0.beta.19'
gem 'capybara'
gem 'database_cleaner'
gem 'cucumber-rails', '~> 0.1.1.rc6'
gem 'cucumber', '~> 0.7.0.beta.8'
gem 'rspec-rails', '~> 2.0.0.beta.19'
gem 'spork'
gem 'launchy'
I'm running on ruby-1.9.2-p0
Any suggestions?
I was getting the same error and was able to move past it by forcing gherkin to be loaded in Gemfile instead of as a cucumber dependency.
So, assigning specific version to Gherkin and >= to cucumbers:
...
gem 'gherkin', '2.1.5'
gem 'cucumber-rails', '>=0.3.2'
gem 'cucumber', '>=0.8.5'
...