Puppet: rake spec could not find class ::splunk - testing

I'm trying to tests my puppet modules, but I get an error message that it can't find class.
I've written a couple of internally used Puppet modules; now that I have my head around puppet, I want to write tests. Since I ran into issues, I decided to essentially start over with a new clean module to figure out what I need to do.
$puppet module generate wet-splunk
<<snip interactive questions>>
$cd splunk
$rake spec
<path snip>ruby -I/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-support-3.4.1/lib:/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/lib /Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/exe/rspec --pattern spec/\{classes,defines,unit,functions,hosts,integration,types\}/\*\*/\*_spec.rb --color
F
Failures:
1) splunk with defaults for all parameters should contain Class[splunk]
Failure/Error: it { should contain_class('splunk') }
Puppet::PreformattedError:
Evaluation Error: Error while evaluating a Function Call, Could not find class ::splunk for host.example.com at line 1:1 on node host.example.com
# ./spec/classes/init_spec.rb:5:in `block (3 levels) in <top (required)>'
Finished in 0.21712 seconds (files took 3.14 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/classes/init_spec.rb:5 # splunk with defaults for all parameters should contain Class[splunk]
/Users/wet/.rbenv/versions/1.9.3-p0/bin/ruby -I/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-support-3.4.1/lib:/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/lib /Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/exe/rspec --pattern spec/\{classes,defines,unit,functions,hosts,integration,types\}/\*\*/\*_spec.rb --color failed
Checking the files though I think they contain what they should:
manifests/init.pp
# <snip documentation boilterplate>
class splunk {
}
spec/classes/init_spec.rb
require 'spec_helper'
describe 'splunk' do
context 'with defaults for all parameters' do
it { should contain_class('splunk') }
end
end
The Could not find class ::splunk is the part that has me confused.
The way I read the spec file is it looking inside init for class{'splunk': ... } That could be me misreading it as it seems odd they would start with a broken spec file, but just in case I tried testing for package instead.
manifests/init.pp
class splunk {
package{'splunk':
ensure => 'present',
}
}
spec/classes/init_spec.rb
require 'spec_helper'
describe 'splunk' do
context 'with defaults for all parameters' do
it { should contain_package('splunk') }
end
end
I still get a similar error message:
$ rake spec
/Users/wet/.rbenv/versions/1.9.3-p0/bin/ruby -I/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-support-3.4.1/lib:/Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/lib /Users/wet/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/rspec-core-3.4.1/exe/rspec --pattern spec/\{classes,defines,unit,functions,hosts,integration,types\}/\*\*/\*_spec.rb --color
F
Failures:
1) splunk with defaults for all parameters should contain Package[splunk]
Failure/Error: it { should contain_package('splunk') }
Puppet::PreformattedError:
Evaluation Error: Error while evaluating a Function Call, Could not find class ::splunk for host.example.com at line 1:1 on node host.example.com
# ./spec/classes/init_spec.rb:6:in `block (3 levels) in <top (required)>'
Finished in 0.21168 seconds (files took 3.17 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/classes/init_spec.rb:6 # splunk with defaults for all parameters should contain Package[splunk]
Sorry to ask such a basic question; I really appreciate your help. I'd really love links to good tutorials on testing Puppet code; I'd love to use Vagrant to have some confidence my code is actually going to do what I want it to.
Update:
I believe I have followed #Peter_Souter's suggestions; I still get the error that it can't find the class. I'm posting the contents of the files referenced to try and validate I did the right thing and in case there are other errors.
$ cat .fixtures.yml
fixtures:
symlinks:
"splunk": "#{source_dir}"
$ cat spec_helper.rb
require 'puppetlabs_spec_helper/module_spec_helper'
$ cat Rakefile
require 'puppetlabs_spec_helper/rake_tasks'
require 'puppet-lint/tasks/puppet-lint'
PuppetLint.configuration.send('disable_80chars')
PuppetLint.configuration.ignore_paths = ["spec/**/*.pp", "pkg/**/*.pp"]
desc "Validate manifests, templates, and ruby files"
task :validate do
Dir['manifests/**/*.pp'].each do |manifest|
sh "puppet parser validate --noop #{manifest}"
end
Dir['spec/**/*.rb','lib/**/*.rb'].each do |ruby_file|
sh "ruby -c #{ruby_file}" unless ruby_file =~ /spec\/fixtures/
end
Dir['templates/**/*.erb'].each do |template|
sh "erb -P -x -T '-' #{template} | ruby -c"
end
end
$ cat Gemfile
source 'https://rubygems.org'
puppetversion = ENV.key?('PUPPET_VERSION') ? "#{ENV['PUPPET_VERSION']}" : ['>= 3.3']
gem 'puppet', puppetversion
gem 'puppetlabs_spec_helper', '>= 0.8.2'
gem 'puppet-lint', '>= 1.0.0'
gem 'facter', '>= 1.7.0'
gem 'rspec-puppet', '~> 2.1', :require => false
gem 'rspec-core', '3.1.7', :require => false

The way that rspec-puppet simulates the catalog for testing, it makes a fixtures directory with the required modules in it. So right now it's running it on an empty directory so it can't find your module. We generally fix that with the puppetlabs-spec-helper which makes a simlink to your current repository in that fixtures directory.
I'd recommend adding the following into your files:
# .fixtures.yml
fixtures:
symlinks:
"splunk": "#{source_dir}"
# spec_helper.rb
require 'puppetlabs_spec_helper/module_spec_helper'
# Rakefile
require 'puppetlabs_spec_helper/rake_tasks'
# Gemfile
gem 'rspec-puppet', '~> 2.1', :require => false
gem 'rspec-core', '3.1.7', :require => false
Can you try that and tell me if it works for you?

If somebody stumble around this Thread as I do. The for me correct answer is given in:
Why do I get puppet-rspec 'class does not exist' when it does?
means you have to do:
puppet module generate foo-bar
and then to fix this issue
bundle exec rspec-puppet-init
It's not nice but better then handling with hidden dot files.

Related

What configuration for Rails 3.2.22.2 + Puma + Heroku?

I've read Richard Schneeman's article, and a bunch of other ones. ;-)
I'm still struggling with this.
Here's few gems I've added in my Gemfile to benchmark my app:
gem 'airbrake'
gem 'newrelic_rpm'
gem 'stackprof'
gem 'derailed', group: :development
gem 'rack-mini-profiler'
gem 'flamegraph'
gem 'memory_profiler'
gem "skylight"
After a lots of benchmarks in development and in staging env, I know where my app is not fast enough but there's not memory leak (some small mem bloats sometimes maybe).
newapp-staging app is the new version (aka: new frontend, upgraded gems, optimized queries, ...) of oldapp-production app.
Please have a look at the screenshots (oldapp-production use webrick, newapp-staging use puma)
So here comes 2 simple questions:
question #1
newapp-staging app is using ruby '2.2.0' & rails '3.2.22.2' and I can't ensure that it is threadsafe because of my code and the associated gems, so... I must use 1 thread at a time. Is puma an advantage here? Metrics are telling me not.
OR... my configuration is not good. (missing preload_app! maybe, or other things?) Here's my Procfile:
web: bundle exec puma -t 1:1 -p ${PORT:-3000} -e ${RACK_ENV:-development}
worker: bundle exec rake jobs:work
question #2
Unicorn could be used as a replacement?
Thank you for your time and your advices.
Cheers
Using unicorn is the best move here. Here's my configuration if that could help anyone.
Gemfile:
gem 'unicorn'
gem 'unicorn-rails'
group :production, :staging do
gem 'unicorn-worker-killer'
end
Procfile:
web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
worker: bundle exec rake jobs:work
config/unicorn.rb
worker_processes Integer(ENV["WEB_CONCURRENCY"] || 2)
timeout 15
preload_app true
before_fork do |server, worker|
Signal.trap 'TERM' do
puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
Process.kill 'QUIT', Process.pid
end
defined?(ActiveRecord::Base) and
ActiveRecord::Base.connection.disconnect!
end
after_fork do |server, worker|
Signal.trap 'TERM' do
puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to send QUIT'
end
defined?(ActiveRecord::Base) and
ActiveRecord::Base.establish_connection
end
config.ru
if ENV['RAILS_ENV'] == 'production' || ENV['RAILS_ENV'] == 'staging'
require 'unicorn/worker_killer'
use Unicorn::WorkerKiller::MaxRequests, 768, 1024, true
use Unicorn::WorkerKiller::Oom, (450*(1024**2)), (490*(1024**2)), 16, true
end
require ::File.expand_path('../config/environment', __FILE__)
use Rack::Deflater
run MyApp::Application
On Heroku:
2 x `Standard 2X dynos` for web
1 x `Standard 1X dyno` for worker
Heroku config vars:
SENSIBLE_DEFAULTS: enabled (just in case) & WEB_CONCURRENCY: 2
Cheers

Capistrano: cannot load such file -- rvm/capistrano (LoadError)

This is really bugging me as there is a bit (not a lot) on the internet about the following error when running cap deploy:cold
cannot load such file -- rvm/capistrano (LoadError)
The solution seems to be gem install rvm-capistrano. Everyone else says installing that gem fixed the error, but for me it does nothing.
The offending lines are:
# Add RVM's lib directory to the load path.
$:.unshift(File.expand_path('./lib', ENV['rvm_path']))
# Load RVM's capistrano plugin.
require "rvm/capistrano"
set :rvm_ruby_string, ENV['GEM_HOME'].gsub(/.*\//,"")
set :rvm_type, :user
###
You should remove the following line too.
$:.unshift(File.expand_path('./lib', ENV['rvm_path']))
That, and installing the rvm-capistrano gem solved similar issue for me.

Heroku paypal_adaptive gem setup

I have paypal_adaptive gem on my local development machine working. But when deployed heroku throws me error that:
ArgumentError: syntax error on line 20, col 2: ` ssl_cert_file:'
/usr/ruby1.9.2/lib/ruby/1.9.1/syck.rb:135:in `load'
/usr/ruby1.9.2/lib/ruby/1.9.1/syck.rb:135:in `load'
/app/.bundle/gems/ruby/1.9.1/bundler/gems/paypal_adaptive-a3853ca1635b/lib/paypal_adaptive/config.rb:32:in `load
I tried with and without line "ssl_cert_file:" in YML file and variables are saved to ENV variables to Heroku. I suspect that gem can't find YML file. How to force gem to find that YML file.
I sent a fix related to this that has been pulled in. http.rb can't handle non strings
See: https://github.com/tc/paypal_adaptive/commit/3e94637d56959994dc4c57186209f071fecc46bb
The gem just needs a version bump.

Upgrading to Rails 3, resque scheduler and worker raise error

I have upgraded a Rails 2.3.5 app to Rails 3.0.6. In the process I have also upgraded the resque from 1.9.1 to 1.15.0 and resque_scheduler from 1.9.1 to 1.9.9. The following commands used to work fine with Rails 2.3.5
COUNT=1 QUEUE=scheduled_1_queue,another_queue,yet_another_queue,slow_queue,redis_cleanup_queue,immediate_queue RAILS_ENV=development JOBS_PER_FORK=500 rake resque:workers
rake resque:scheduler
But now after the upgrade when I run above commands I get following error:
rake aborted!
wrong number of arguments (0 for 1)
Tasks: TOP => resque:work => resque:setup => environment
The RakeFile is as following:
require File.expand_path('../config/application', __FILE__)
require 'rake'
require 'resque/tasks'
require 'resque_scheduler/tasks'
MyApp::Application.load_tasks
task "resque:setup" => :environment
If i comment the last line 'task "resque:setup" => :environment' in RakeFile the resque scheduler and worker commands run without any error, but the scheduler does not schedule any tasks at all and stays on this:
2011-06-16 09:33:45 Schedule empty! Set Resque.schedule
I have also tried with older version of resque(1.9.1) and resque_scheduler(1.9.1) with Rails 3, but to no avail. I have also tried to run above commands with --trace, but it does not provide any trace info about the "wrong number of arguments (0 for 1)" error. I suspect I might need to provide more info, but don't know exactly what, please let me know what else is needed to answer this question. Thanks a lot.

warbled gems not seen by Rakefile

I have s sinatra app warbled and deployed on glassfish. I have used bundler to manage dependencies and warbler has included those gems in WEB-INF/gems directory.
But when i try to run a rake task from WEB-INF directory, that task is not running.
gfish#server2:~/glassfish/domains/domain1/applications/sinatra_app/WEB-INF$ rake resque:work
(in /home/gfish/glassfish/domains/domain1/applications/sinatra_app/WEB-INF)
rake aborted!
Could not find gem 'sinatra (>= 0, runtime)' in any of the gem sources listed in your Gemfile.
/home/gfish/glassfish/domains/domain1/applications/sinatra_app/WEB-INF/Rakefile:5:in `(root)'
(See full trace by running task with --trace)
gfish#server2:~/glassfish/domains/domain1/applications/sinatra_app/WEB-INF$ **ls -l** gems/gems/
total 104
...
drwxr-xr-x 4 root root 4096 2011-02-24 14:38 sinatra-1.1.3
I finally found it. I used the following code to do it.
# Add warbled gems to the $LOAD_PATH
if ENV['RACK_ENV'] == "production"
puts "Adding warbled gems to the load path..."
local_gems_path = Dir[File.expand_path(".") + "/gems/gems/*"]
local_gems_path.each do |g|
$LOAD_PATH.unshift "#{g}/lib/"
end
end