I've read Richard Schneeman's article, and a bunch of other ones. ;-)
I'm still struggling with this.
Here's few gems I've added in my Gemfile to benchmark my app:
gem 'airbrake'
gem 'newrelic_rpm'
gem 'stackprof'
gem 'derailed', group: :development
gem 'rack-mini-profiler'
gem 'flamegraph'
gem 'memory_profiler'
gem "skylight"
After a lots of benchmarks in development and in staging env, I know where my app is not fast enough but there's not memory leak (some small mem bloats sometimes maybe).
newapp-staging app is the new version (aka: new frontend, upgraded gems, optimized queries, ...) of oldapp-production app.
Please have a look at the screenshots (oldapp-production use webrick, newapp-staging use puma)
So here comes 2 simple questions:
question #1
newapp-staging app is using ruby '2.2.0' & rails '3.2.22.2' and I can't ensure that it is threadsafe because of my code and the associated gems, so... I must use 1 thread at a time. Is puma an advantage here? Metrics are telling me not.
OR... my configuration is not good. (missing preload_app! maybe, or other things?) Here's my Procfile:
web: bundle exec puma -t 1:1 -p ${PORT:-3000} -e ${RACK_ENV:-development}
worker: bundle exec rake jobs:work
question #2
Unicorn could be used as a replacement?
Thank you for your time and your advices.
Cheers
Using unicorn is the best move here. Here's my configuration if that could help anyone.
Gemfile:
gem 'unicorn'
gem 'unicorn-rails'
group :production, :staging do
gem 'unicorn-worker-killer'
end
Procfile:
web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
worker: bundle exec rake jobs:work
config/unicorn.rb
worker_processes Integer(ENV["WEB_CONCURRENCY"] || 2)
timeout 15
preload_app true
before_fork do |server, worker|
Signal.trap 'TERM' do
puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
Process.kill 'QUIT', Process.pid
end
defined?(ActiveRecord::Base) and
ActiveRecord::Base.connection.disconnect!
end
after_fork do |server, worker|
Signal.trap 'TERM' do
puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to send QUIT'
end
defined?(ActiveRecord::Base) and
ActiveRecord::Base.establish_connection
end
config.ru
if ENV['RAILS_ENV'] == 'production' || ENV['RAILS_ENV'] == 'staging'
require 'unicorn/worker_killer'
use Unicorn::WorkerKiller::MaxRequests, 768, 1024, true
use Unicorn::WorkerKiller::Oom, (450*(1024**2)), (490*(1024**2)), 16, true
end
require ::File.expand_path('../config/environment', __FILE__)
use Rack::Deflater
run MyApp::Application
On Heroku:
2 x `Standard 2X dynos` for web
1 x `Standard 1X dyno` for worker
Heroku config vars:
SENSIBLE_DEFAULTS: enabled (just in case) & WEB_CONCURRENCY: 2
Cheers
Related
I'm trying to tests my puppet modules, but I get an error message that it can't find class.
I've written a couple of internally used Puppet modules; now that I have my head around puppet, I want to write tests. Since I ran into issues, I decided to essentially start over with a new clean module to figure out what I need to do.
$puppet module generate wet-splunk
<<snip interactive questions>>
$cd splunk
$rake spec
<path snip>ruby -I/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-support-3.4.1/lib:/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/lib /Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/exe/rspec --pattern spec/\{classes,defines,unit,functions,hosts,integration,types\}/\*\*/\*_spec.rb --color
F
Failures:
1) splunk with defaults for all parameters should contain Class[splunk]
Failure/Error: it { should contain_class('splunk') }
Puppet::PreformattedError:
Evaluation Error: Error while evaluating a Function Call, Could not find class ::splunk for host.example.com at line 1:1 on node host.example.com
# ./spec/classes/init_spec.rb:5:in `block (3 levels) in <top (required)>'
Finished in 0.21712 seconds (files took 3.14 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/classes/init_spec.rb:5 # splunk with defaults for all parameters should contain Class[splunk]
/Users/wet/.rbenv/versions/1.9.3-p0/bin/ruby -I/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-support-3.4.1/lib:/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/lib /Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/exe/rspec --pattern spec/\{classes,defines,unit,functions,hosts,integration,types\}/\*\*/\*_spec.rb --color failed
Checking the files though I think they contain what they should:
manifests/init.pp
# <snip documentation boilterplate>
class splunk {
}
spec/classes/init_spec.rb
require 'spec_helper'
describe 'splunk' do
context 'with defaults for all parameters' do
it { should contain_class('splunk') }
end
end
The Could not find class ::splunk is the part that has me confused.
The way I read the spec file is it looking inside init for class{'splunk': ... } That could be me misreading it as it seems odd they would start with a broken spec file, but just in case I tried testing for package instead.
manifests/init.pp
class splunk {
package{'splunk':
ensure => 'present',
}
}
spec/classes/init_spec.rb
require 'spec_helper'
describe 'splunk' do
context 'with defaults for all parameters' do
it { should contain_package('splunk') }
end
end
I still get a similar error message:
$ rake spec
/Users/wet/.rbenv/versions/1.9.3-p0/bin/ruby -I/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-support-3.4.1/lib:/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/lib /Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/exe/rspec --pattern spec/\{classes,defines,unit,functions,hosts,integration,types\}/\*\*/\*_spec.rb --color
F
Failures:
1) splunk with defaults for all parameters should contain Package[splunk]
Failure/Error: it { should contain_package('splunk') }
Puppet::PreformattedError:
Evaluation Error: Error while evaluating a Function Call, Could not find class ::splunk for host.example.com at line 1:1 on node host.example.com
# ./spec/classes/init_spec.rb:6:in `block (3 levels) in <top (required)>'
Finished in 0.21168 seconds (files took 3.17 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/classes/init_spec.rb:6 # splunk with defaults for all parameters should contain Package[splunk]
Sorry to ask such a basic question; I really appreciate your help. I'd really love links to good tutorials on testing Puppet code; I'd love to use Vagrant to have some confidence my code is actually going to do what I want it to.
Update:
I believe I have followed #Peter_Souter's suggestions; I still get the error that it can't find the class. I'm posting the contents of the files referenced to try and validate I did the right thing and in case there are other errors.
$ cat .fixtures.yml
fixtures:
symlinks:
"splunk": "#{source_dir}"
$ cat spec_helper.rb
require 'puppetlabs_spec_helper/module_spec_helper'
$ cat Rakefile
require 'puppetlabs_spec_helper/rake_tasks'
require 'puppet-lint/tasks/puppet-lint'
PuppetLint.configuration.send('disable_80chars')
PuppetLint.configuration.ignore_paths = ["spec/**/*.pp", "pkg/**/*.pp"]
desc "Validate manifests, templates, and ruby files"
task :validate do
Dir['manifests/**/*.pp'].each do |manifest|
sh "puppet parser validate --noop #{manifest}"
end
Dir['spec/**/*.rb','lib/**/*.rb'].each do |ruby_file|
sh "ruby -c #{ruby_file}" unless ruby_file =~ /spec\/fixtures/
end
Dir['templates/**/*.erb'].each do |template|
sh "erb -P -x -T '-' #{template} | ruby -c"
end
end
$ cat Gemfile
source 'https://rubygems.org'
puppetversion = ENV.key?('PUPPET_VERSION') ? "#{ENV['PUPPET_VERSION']}" : ['>= 3.3']
gem 'puppet', puppetversion
gem 'puppetlabs_spec_helper', '>= 0.8.2'
gem 'puppet-lint', '>= 1.0.0'
gem 'facter', '>= 1.7.0'
gem 'rspec-puppet', '~> 2.1', :require => false
gem 'rspec-core', '3.1.7', :require => false
The way that rspec-puppet simulates the catalog for testing, it makes a fixtures directory with the required modules in it. So right now it's running it on an empty directory so it can't find your module. We generally fix that with the puppetlabs-spec-helper which makes a simlink to your current repository in that fixtures directory.
I'd recommend adding the following into your files:
# .fixtures.yml
fixtures:
symlinks:
"splunk": "#{source_dir}"
# spec_helper.rb
require 'puppetlabs_spec_helper/module_spec_helper'
# Rakefile
require 'puppetlabs_spec_helper/rake_tasks'
# Gemfile
gem 'rspec-puppet', '~> 2.1', :require => false
gem 'rspec-core', '3.1.7', :require => false
Can you try that and tell me if it works for you?
If somebody stumble around this Thread as I do. The for me correct answer is given in:
Why do I get puppet-rspec 'class does not exist' when it does?
means you have to do:
puppet module generate foo-bar
and then to fix this issue
bundle exec rspec-puppet-init
It's not nice but better then handling with hidden dot files.
I am using the following configurations in my deploy.rb file for capistrano:
require 'bundler/capistrano'
require 'rvm/capistrano'
set :bundle_cmd, "/home/deployment/.rvm/gems/ruby-1.9.3-p194#global/bin/bundle"
set :default_environment, {
'PATH' => "/home/deployment/.rvm/gems/ruby-1.9.3-p194/bin:/home/deployment/.rvm/bin:$PATH",
'RUBY_VERSION' => 'ruby 1.9.3',
'GEM_HOME' => "/home/deployment/.rvm/gems/ruby-1.9.3-p194",
'GEM_PATH' => "/home/deployment/.rvm/gems/ruby-1.9.3-p194",
'BUNDLE_PATH' => "/home/deployment/.rvm/gems/ruby-1.9.3-p194"
}
But when I run cap deploy:update I get this:
* executing "cd /var/www/currienet/marketplace/releases/20120928140140 && /home/deployment/.rvm/gems/ruby-1.9.3-p194#global/bin/bundle install --gemfile /var/www/currienet/marketplace/releases/20120928140140/Gemfile --path /var/www/currienet/marketplace/shared/bundle --deployment --quiet --without development test"
That is, it's not setting the bundle path (the --path argument) to what I want it to be.
I've tried a number of tutorials, including the rvm capistrano tutorial but nothing seems to work. It continues to use the capistrano default.
Capistrano also creates an application with the following .bundler/config
BUNDLE_FROZEN: '1'
BUNDLE_PATH: /var/www/currienet/marketplace/shared/bundle
BUNDLE_DISABLE_SHARED_GEMS: '1'
BUNDLE_WITHOUT: development:test
Development Machine: Windows 7, bundler (1.0.22), capistrano (2.12.0), rvm-capistrano (1.2.7), rails (3.2.8), (no rvm)
Production: Debian, bundler (1.2.1) (no capistrano), (no rvm-capistrano), rails (3.2.8), rvm 1.16.5
Thanks to Joseph Holsten's blog I was able to ascertain my problem was I was not defining the bundler variables in my deploy.rb before I required 'bundler/capistrano'. I also needed to define the bundle_dir variable, to create code that looks like the following:
set :bundle_cmd, "/home/deployment/.rvm/gems/ruby-1.9.3-p194#global/bin/bundle"
set :bundle_dir, "/home/deployment/.rvm/gems/ruby-1.9.3-p194"
require 'bundler/capistrano'
I have a problem with Capistrano deploying to a server with RVM and gemsets
executing `deploy:assets:precompile'
* executing "cd /var/app/releases/20111229233555 && rake RAILS_ENV=production RAILS_GROUPS=assets assets:precompile"
*** [err ::] rake aborted!
*** [err ::] Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available runtimes.
*** [deploy:update_code] rolling back
failed: "rvm_path=$HOME/.rvm/ $HOME/.rvm/bin/rvm-shell '1.9.2#mygemset' -c 'cd /var/app/releases/20111229234715 && rake RAILS_ENV=production RAILS_GROUPS=assets assets:precompile'"
This is my Capfile
$:.unshift(File.expand_path('./lib', ENV['rvm_path']))
require 'rvm/capistrano'
set :rvm_bin_path, "$HOME/.rvm/bin"
set :rvm_ruby_string, '1.9.2#mygemset'
set :rvm_type, :user
load 'deploy/assets'
When I log in as the very same user which is used by Capistrano the failing command is executed without any problems.
It seems that Capistrano is using some other environment.
Here is my ~/.rvmrc file:
rvm_gemset_create_on_use_flag=1 rvm_trust_rvmrcs_flag=1
Is there a way how to check which ruby and gemset is used at the execution time?
node.js is and V8 are installed on the system, and all needed gems are in Gemfile:
here is my rvm info:
ruby:
interpreter: "ruby"
version: "1.9.2p290"
date: "2011-07-09"
platform: "x86_64-linux"
patchlevel: "2011-07-09 revision 32553"
full_version: "ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-linux]"
homes:
gem: "/home/capistrano/.rvm/gems/ruby-1.9.2-p290#mygemset"
ruby: "/home/capistrano/.rvm/rubies/ruby-1.9.2-p290"
binaries:
ruby: "/home/capistrano/.rvm/rubies/ruby-1.9.2-p290/bin/ruby"
irb: "/home/capistrano/.rvm/rubies/ruby-1.9.2-p290/bin/irb"
gem: "/home/capistrano/.rvm/rubies/ruby-1.9.2-p290/bin/gem"
rake: "/home/capistrano/.rvm/gems/ruby-1.9.2-p290#mygemset/bin/rake"
setting up PATH in the deploy.rb solved this issue:
default_environment['PATH'] = "/usr/local/bin:/usr/bin:/bin:/opt/bin:$PATH"
it's a Gentoo distribution and the PATH var seems to be empty when using cap deploy, so I copied the content of server PATH and now it works
This issue went away for me when I installed nodejs (using apt-get) on my server. As I stated in this Stackoverflow answer, it's the most practical choice because you don't need to include any dependencies in your Gemfile.
I have upgraded a Rails 2.3.5 app to Rails 3.0.6. In the process I have also upgraded the resque from 1.9.1 to 1.15.0 and resque_scheduler from 1.9.1 to 1.9.9. The following commands used to work fine with Rails 2.3.5
COUNT=1 QUEUE=scheduled_1_queue,another_queue,yet_another_queue,slow_queue,redis_cleanup_queue,immediate_queue RAILS_ENV=development JOBS_PER_FORK=500 rake resque:workers
rake resque:scheduler
But now after the upgrade when I run above commands I get following error:
rake aborted!
wrong number of arguments (0 for 1)
Tasks: TOP => resque:work => resque:setup => environment
The RakeFile is as following:
require File.expand_path('../config/application', __FILE__)
require 'rake'
require 'resque/tasks'
require 'resque_scheduler/tasks'
MyApp::Application.load_tasks
task "resque:setup" => :environment
If i comment the last line 'task "resque:setup" => :environment' in RakeFile the resque scheduler and worker commands run without any error, but the scheduler does not schedule any tasks at all and stays on this:
2011-06-16 09:33:45 Schedule empty! Set Resque.schedule
I have also tried with older version of resque(1.9.1) and resque_scheduler(1.9.1) with Rails 3, but to no avail. I have also tried to run above commands with --trace, but it does not provide any trace info about the "wrong number of arguments (0 for 1)" error. I suspect I might need to provide more info, but don't know exactly what, please let me know what else is needed to answer this question. Thanks a lot.
I have s sinatra app warbled and deployed on glassfish. I have used bundler to manage dependencies and warbler has included those gems in WEB-INF/gems directory.
But when i try to run a rake task from WEB-INF directory, that task is not running.
gfish#server2:~/glassfish/domains/domain1/applications/sinatra_app/WEB-INF$ rake resque:work
(in /home/gfish/glassfish/domains/domain1/applications/sinatra_app/WEB-INF)
rake aborted!
Could not find gem 'sinatra (>= 0, runtime)' in any of the gem sources listed in your Gemfile.
/home/gfish/glassfish/domains/domain1/applications/sinatra_app/WEB-INF/Rakefile:5:in `(root)'
(See full trace by running task with --trace)
gfish#server2:~/glassfish/domains/domain1/applications/sinatra_app/WEB-INF$ **ls -l** gems/gems/
total 104
...
drwxr-xr-x 4 root root 4096 2011-02-24 14:38 sinatra-1.1.3
I finally found it. I used the following code to do it.
# Add warbled gems to the $LOAD_PATH
if ENV['RACK_ENV'] == "production"
puts "Adding warbled gems to the load path..."
local_gems_path = Dir[File.expand_path(".") + "/gems/gems/*"]
local_gems_path.each do |g|
$LOAD_PATH.unshift "#{g}/lib/"
end
end