So, I am using the Bullet gem to try and catch N+1 queries in my Rails 3 app. However, I hit one query that I can't seem to get around.
Here's the bullet message:
N+1 Query detected
User => [:profile]
Add to your finder: :include => [:profile]
N+1 Query method call stack
.../app/models/user.rb:38:in `full_name'
.../config/initializers/paperclip.rb:14:in `block in <top (required)>'
.../app/views/photos/_photo_tiles.html.haml:4:in `_app_views_photos__photo_tiles_html_haml__787318603078146010_2192645460_1634077202131545355'
.../app/views/photos/index.html.haml:30:in `_app_views_photos_index_html_haml___2562222078013468078_2155167020__3275303796392914006'
It doesn't really make sense why it would do this, though, because every user always has profile data that needs to be pulled with them, so my User model has this in it:
# SCOPES
default_scope includes(:profile)
# DELEGATES
delegate :first_name, :last_name, :full_name,
:to => :profile
So, I'm always pulling the profile along with the user. The problem seems to stem from the inclusion of the user's full name in the file name for their uploads, found in my paperclip initializer:
Paperclip.interpolates :username do |attachment, style|
attachment.instance.user.full_name.dehumanize
end
So, that call to user.full_name is being passed through to user.profile -- but profile is included!
So, anyone have any ideas how to eliminate this N+1 query?
Thanks!
Are you sure this is a true instance of N+1 query rather than a mistake by Bullet? Maybe it gets confused by the whole default_scope + delegate thing.
Have you tried running the method in question in script/console development while in another shell doing tail -f log/development.log?
This way you can see which queries are getting executed and make sure that Bullet isn't just confused.
Related
I have this problem related to testing model errors with Mocha:
This is my controller (Api / Artist controller):
class Api::ArtistsController < ApplicationController
respond_to :json
def create
artist = Artist.new(params[:artist])
if artist.save <-- This is where the test fails
render :json=>artist
else
respond_with artist
end
end
end
This is my model (Artist Model):
class Artist < ActiveRecord::Base
include Core::BaseModel
attr_accessible :name
has_many :albums
validates :name, :presence=>true, :uniqueness=>{:case_sensitive=> false}
default_scope where :deleted=>false
end
This is the test where it fails, about Artist controller:
it "should not save a duplicated artist" do
Artist.any_instance.stubs(:is_valid?).returns(false)
Artist.any_instance.stubs(:errors).returns({:name=>[I18n.t('activerecord.errors.messages.taken')]})
post :create, :format => :json
expect(response).not_to be_success
expect(response.code).to eq("422")
results = JSON.parse(response.body)
expect(results).to include({
"errors"=>{
"name"=>[I18n.t('activerecord.errors.messages.taken')]
}
})
end
When I run the tests, this is the error I get on the above test:
Failure/Error: post :create, :format => :json
NoMethodError:
undefined method `add_on_blank' for {}:Hash
# ./app/controllers/api/artists_controller.rb:17:in `create'
# ./spec/controllers/api/artists_controller_spec.rb:56:in `block (3 levels) in <top (required)>'
I'm starting to use Mocha, so I don't know if there's a way to test the json result for the specific case when I want to test the validation for the duplicated name.
ActiveRecord::Base#errors (i.e. Artist#errors) isn't a simple hash. It's supposed to be an instance of ActiveModel::Errors. You're stubbing it with a hash, and ActiveRecord is trying to call add_on_blank on it, which is failing.
I don't think save invokes is_valid? at all, and I suspect it's running the validations and then trying to call add_on_blank to append an error, but since you've stubbed out errors, that's failing.
This isn't really a good way to test the controller. It's making too many assumptions about the internals of Artist. You're also testing things that aren't part of the controller at all; errors isn't referenced anywhere in the action. The only behavior worth testing in the controller is whether or not it creates an Artist; if that Artist fails to save, that it renders JSON with it; and if the save succeeds, that it redirects. That's all of the controller's responsibility.
If you want to test that errors are rendered a certain way, you should write a separate view spec. If you want to test that missing fields generate errors, you should write a model spec. If you don't want to write a view spec, it's still sufficient to rely on the model to populate errors (tested in a model spec), and in your controller, just test that render is called with json set to the Artist instance.
Generally speaking it's best to avoid stubbing as much as possible, but in this case, the only things I'd consider stubbing are Artist.new to return a mock, and save on that mock to return false. Then I'd check to make sure it rendered with the mock.
The easier option is to just create an actual Artist record, then call post with duplicate params to trigger a validation failure. The downside is that you hit the database, and avoiding that in a controller spec is laudable, but generally more convenient. You could instead do that in a Capybara feature spec if you want to avoid DB hits in your controller specs.
If you want to try testing the way you are, you can manually create an instance of ActiveModel::Errors and populate that, or stub methods on it, and stub out Artist.any_instance.stubs(:errors) to return your mock with ActiveModel::Errors-compatible behavior, but that's a lot of mocking.
One final tip: don't use post :create, :format => :json. Use xhr :post, :create to generate a real Ajax request rather than relying on a format param. It's a more robust test of your routing and response code.
I'm trying to track user login history for stat purposes but its not clear to me what the best way to go about it would be. I could have a separate table that records users and their login stats with a date, but that table could get REALLY big. I could track some historic fields in the User model/object itself in a parse-able field and just update it (them) with some delimited string format. e.g. split on :, get the last one, if an included date code isn't today, add an item (date+count) otherwise increment, then save it back. At least with this second approach it would be easy to remove old items (e.g. only keep 30 days of daily logins, or IPs), as a separate table would require a task to delete old records.
I'm a big fan of instant changes. Tasks are useful, but can complicate things for maintenance reasons.
Anyone have any suggestions? I don't have an external data caching solution up or anything yet. Any pointers are also welcome! (I've been hunting for similar questions and answers)
Thanks!
If you have the :trackable module, I found this the easiest way. In the User model (or whichever model you're authenticating)
def update_tracked_fields!(request)
old_signin = self.last_sign_in_at
super
if self.last_sign_in_at != old_signin
Audit.create :user => self, :action => "login", :ip => self.last_sign_in_ip
end
end
(Inspired by https://github.com/plataformatec/devise/wiki/How-To:-Turn-off-trackable-for-admin-users)
There is a nice way to do that through Devise.
Warden sets up a hook called after_set_user that runs after setting a user. So, supposed you have a model Login containing an ip field, a logged_in_at field and user_id field, you can only create the record using this fields.
Warden::Manager.after_set_user :except => :fetch do |record, warden, options|
Login.create!(:ip => warden.request.ip, :logged_in_at => Time.now, :user_id => record.id)
end
Building upon #user208769's answer, the core Devise::Models::Trackable#update_tracked_fields! method now calls a helper method named update_tracked_fields prior to saving. That means you can use ActiveRecord::Dirty helpers to make it a little simpler:
def update_tracked_fields(request)
super
if last_sign_in_at_changed?
Audit.create(user: self, action: 'login', ip: last_sign_in_ip)
end
end
This can be simplified even further (and be more reliable given validations) if audits is a relationship on your model:
def update_tracked_fields(request)
super
audits.build(action: 'login', ip: last_sign_in_ip) if last_sign_in_at_changed?
end
Devise supports tracking the last signed in date and the last signed in ip address with it's :trackable module. By adding this module to your user model, and then also adding the correct fields to your database, which are:
:sign_in_count, :type => Integer, :default => 0
:current_sign_in_at, :type => Time
:last_sign_in_at, :type => Time
:current_sign_in_ip, :type => String
:last_sign_in_ip, :type => String
You could then override the Devise::SessionsController and it's create action to then save the :last_sign_in_at and :last_sign_in_ip to a separate table in a before_create callback. You should then be able to keep them as long you would like.
Here's an example (scribd_analytics)
create_table 'page_views' do |t|
t.column 'user_id', :integer
t.column 'request_url', :string, :limit => 200
t.column 'session', :string, :limit => 32
t.column 'ip_address', :string, :limit => 16
t.column 'referer', :string, :limit => 200
t.column 'user_agent', :string, :limit => 200
t.column 'created_at', :timestamp
end
Add a whole bunch of indexes, depending on queries
Create a PageView on every request
We used a hand-built SQL query to take out the ActiveRecord overhead on
this
Might try MySQL's 'insert delayed´
Analytics queries are usually hand-coded SQL
Use 'explain select´ to make sure MySQL isusing the indexes you expect
Scales pretty well
BUT analytics queries expensive, can clog upmain DB server
Our solution:
use two DB servers in a master/slave setup
move all the analytics queries to the slave
http://www.scribd.com/doc/49575/Scaling-Rails-Presentation-From-Scribd-Launch
Another option to check is Gattica with Google Analytics
I hate answering my own questions, especially given that you both gave helpful answers. I think answering my question with the approach I took might help others, in combination with your answers.
I've been playing with the Impressionist Gem (the only useful page view Gem since the abandoned RailStat) with good results so far. After setting up the basic migration, I found that the expected usage follows Rail's MVC design very closely. If you add "impressionist" to a Controller, it will go looking for the Model when logging the page view to the database. You can modify this behaviour or just call impressionist yourself in your Controller (or anywhere really) if you're like me and happen to be testing it out on a Controller that doesn't have a Model.
Anyways, I got it working with Devise to track successful logins by overriding the Devise::SessionsController and just calling the impressionist method for the #current_member: (don't forget to check if it's nil! on failed login)
class TestSessionController < Devise::SessionsController
def create
if not #current_member.nil?
impressionist(#current_member)
end
super
end
end
Adding it to other site parts later for some limited analytics is easy to do. The only other thing I had to do was update my routes to use the new TestSessionController for the Devise login route:
post 'login' => 'test_session#create', :as => :member_session
Devise works like normal without having to modify Devise in anyway, and my impressionist DB table is indexed and logging logins. I'll just need a rake task later to trim it weekly or so.
Now I just need to work out how to chart daily logins without having to write a bunch of looping, dirty queries...
There is also 'paper_trail' gem, that allows to track model changes.
I'm fairly new to rails and TDD (as will no doubt be obvious from my post) and am having a hard time wrapping my brain around Rspec and FactoryGirl.
I'm using Rails 3, rspec and factory girl:
gem 'rails', '3.0.3'
# ...
gem 'rspec-rails', '~>2.4.0'
gem 'factory_girl_rails'
I have a user model that I've been successfully running tests on during development, but then needed to add an attribute to, called "source". It's for determining where the user record originally came from (local vs LDAP).
In my factories.rb file, I have several factories defined, that look something like the following:
# An alumnus account tied to LDAP
Factory.define :alumnus, :class => User do |f|
f.first_name "Mickey"
f.last_name "Mouse"
f.username "mickeymouse"
f.password "strongpassword"
f.source "directory"
end
I have a macro defined (that's been working up until now) that looks like this:
def login(user)
before(:each) do
sign_out :user
sign_in Factory.create(user)
end
end
I'm calling it in multiple specs like so (example from users_controller_spec.rb):
describe "for non-admins or managers" do
login(:alumnus)
it "should deny access" do
get :index
response.should redirect_to(destroy_user_session_path)
end
end
If I don't specify the "source" attribute, everything works OK, but as soon as I do, I get an error like so when running the test
12) UsersController for non-admins or managers should deny access
Failure/Error: Unable to find matching line from backtrace
NoMethodError:
undefined method `source=' for #<User:0x00000100e256c0>
I can access the attribute no problem from the rails console and the app itself, and it's listed in my attr_accessible in the user model. It's almost as though Rspec is seeing an old version of my model and not recognizing that I've added an attribute to it. But if I put the following line into my user model, the error disappears
attr_accessor :source
... which indicates to me that it is actually looking at the correct model.
Help!
How about running this?
rake db:test:load
[If you added a new attribute you'd need to migrate it to the test database.]
if you don't use schema.rb (e.g. you have set config.active_record.schema_format = :sql)
you should run
rake db:test:prepare
I'm using RSpec and CanCan in a project. I'm testing my permission logic in specs related to the Ability class. For the controllers I really just want to make sure I'm doing an authorization check. I set up a controller macro, but it doesn't seem to be working correctly.
So really I have two questions. One, is the strategy sufficient for testing the permission logic of my controllers (or should I be testing the controller authorization logic more)?
Two, does anyone see what I'm doing wrong to make this not work?
#plan_orders_controller.rb
def approve
plan_order = PlanOrder.find(params[:id])
authorize! :update, plan_order
current_user.approve_plan_order(plan_order)
redirect_to plan_order_workout_plan_url(plan_order)
end
#controller_macros.rb
def it_should_check_permissions(*actions)
actions.each do |action|
it "#{action} action should authorize user to do this action" do
#plan_order = Factory(:plan_order, :id=>1)
ability = Object.new
ability.extend(CanCan::Ability)
controller.stub!(:current_ability).and_return(ability)
get action, :id => 1
ability.should_receive(:can?)
end
end
end
The output I get from RSpec is the following
Failure/Error: ability.should_receive(:can?)
(#<Object:0x00000006d4fa20>).can?(any args)
expected: 1 time
received: 0 times
# ./spec/controllers/controller_macros/controller_macros.rb:27:in `block (2 levels) in it_should_check_permissions'
I'm not really sure which method I should be checking when I call !authorize in the controller (or it is automatically called via load_and_authorize_resource)
should_receive is an expectation of something that happens in the future. Reverse these two lines:
get action, :id => 1
ability.should_receive(:can?)
so you have this:
ability.should_receive(:can?)
get action, :id => 1
I would like to see the SQL statement that a given ActiveRecord Query will generate. I recognize I can get this information from the log after the query has been issued, but I'm wondering if there is a method that can be called on and ActiveRecord Query.
For example:
SampleModel.find(:all, :select => "DISTINCT(*)", :conditions => ["`date` > #{self.date}"], :limit => 1, :order => '`date`', :group => "`date`")
I would like to open the irb console and tack a method on the end that would show the SQL that this query will generate, but not necessarily execute the query.
Similar to penger's, but works anytime in the console even after classes have been loaded and the logger has been cached:
For Rails 2:
ActiveRecord::Base.connection.instance_variable_set :#logger, Logger.new(STDOUT)
For Rails 3.0.x:
ActiveRecord::Base.logger = Logger.new(STDOUT)
For Rails >= 3.1.0 this is already done by default in consoles. In case it's too noisy and you want to turn it off you can do:
ActiveRecord::Base.logger = nil
Stick a puts query_object.class somewhere to see what type of object your working with, then lookup the docs.
For example, in Rails 3.0, scopes use ActiveRecord::Relation which has a #to_sql method. For example:
class Contact < ActiveRecord::Base
scope :frequently_contacted, where('messages_count > 10000')
end
Then, somewhere you can do:
puts Contact.frequently_contacted.to_sql
just use to_sql method and it'll output the sql query that will be run. it works on an active record relation.
irb(main):033:0> User.limit(10).where(:username => 'banana').to_sql
=> "SELECT "users".* FROM "users" WHERE "users"."username" = 'banana'
LIMIT 10"
when doing find, it won't work, so you'll need to add that id manually to the query or run it using where.
irb(main):037:0* User.where(id: 1).to_sql
=> "SELECT "users".* FROM "users" WHERE "users"."id" = 1"
This may be an old question but I use:
SampleModel.find(:all,
:select => "DISTINCT(*)",
:conditions => ["`date` > #{self.date}"],
:limit=> 1,
:order => '`date`',
:group => "`date`"
).explain
The explain method will give quite a detailed SQL statement on what its going to do
This is what I usually do to get SQL generated in console
-> script/console
Loading development environment (Rails 2.1.2)
>> ActiveRecord::Base.logger = Logger.new STDOUT
>> Event.first
You have to do this when you first start the console, if you do this after you have typed some code, it doesn't seem to work
Can't really take credit for this, found it long time ago from someone's blog and can't remember whose it is.
When last I tried to do this there was no official way to do it. I resorted to using the function that find and its friends use to generate their queries directly. It is private API so there is a huge risk that Rails 3 will totally break it, but for debugging, it is an ok solution.
The method is construct_finder_sql(options) (lib/active_record/base.rb:1681) you will have to use send because it is private.
Edit: construct_finder_sql was removed in Rails 5.1.0.beta1.
Create a .irbrc file in your home directory and paste this in:
if ENV.include?('RAILS_ENV') && !Object.const_defined?('RAILS_DEFAULT_LOGGER')
require 'logger'
RAILS_DEFAULT_LOGGER = Logger.new(STDOUT)
end
That will output SQL statements into your irb session as you go.
EDIT: Sorry that will execute the query still, but it's closest I know of.
EDIT: Now with arel, you can build up scopes/methods as long as the object returns ActiveRecord::Relation and call .to_sql on it and it will out put the sql that is going to be executed.
My typical way to see what sql it uses is to introduce a "bug" in the sql, then you'll get an error messages spit out to the normal logger (and web screen) that has the sql in question. No need to find where stdout is going...
Try the show_sql plugin. The plugin enables you to print the SQL without running it
SampleModel.sql(:select => "DISTINCT(*)", :conditions => ["`date` > #{self.date}"], :limit => 1, :order => '`date`', :group => "`date`")
You could change the connection's log method to raise an exception, preventing the query from being run.
It's a total hack, but it seems to work for me (Rails 2.2.2, MySQL):
module ActiveRecord
module ConnectionAdapters
class AbstractAdapter
def log_with_raise(sql, name, &block)
puts sql
raise 'aborting select' if caller.any? { |l| l =~ /`select'/ }
log_without_raise(sql, name, &block)
end
alias_method_chain :log, :raise
end
end
end
You can simply use to_sql() function with the active record
Form.where(status:"Active").to_sql
In Rails 3 you can add this line to the config/environments/development.rb
config.active_record.logger = Logger.new(STDOUT)
It will however execute the query. But half got answered :