I was wondering if there was a way to use "find_by_sql" within a named_scope. I'd like to treat custom sql as named_scope so I can chain it to my existing named_scopes. It would also be good for optimizing a sql snippet I use frequently.
While you can put any SQL you like in the conditions of a named scope, if you then call find_by_sql then the 'scopes' get thrown away.
Given:
class Item
# Anything you can put in an sql WHERE you can put here
named_scope :mine, :conditions=>'user_id = 12345 and IS_A_NINJA() = 1'
end
This works (it just sticks the SQL string in there - if you have more than one they get joined with AND)
Item.mine.find :all
=> SELECT * FROM items WHERE ('user_id' = 887 and IS_A_NINJA() = 1)
However, this doesn't
Items.mine.find_by_sql 'select * from items limit 1'
=> select * from items limit 1
So the answer is "No". If you think about what has to happen behind the scenes then this makes a lot of sense. In order to build the SQL rails has to know how it fits together.
When you create normal queries, the select, joins, conditions, etc are all broken up into distinct pieces. Rails knows that it can add things to the conditions without affecting everything else (which is how with_scope and named_scope work).
With find_by_sql however, you just give rails a big string. It doesn't know what goes where, so it's not safe for it to go in and add the things it would need to add for the scopes to work.
This doesn't address exactly what you asked about, but you might investigate 'contruct_finder_sql'. It lets you can get the SQL of a named scope.
named_scope :mine, :conditions=>'user_id = 12345 and IS_A_NINJA() = 1'
named_scope :additional {
:condtions => mine.send(:construct_finder_sql,{}) + " additional = 'foo'"
}
sure why not
:named_scope :conditions => [ your sql ]
Related
In my controller method for the the index view I have the following line.
#students_instance = Student.includes(:memo_tests => {:memo_target => :memo_level})
So for each Student I eager-load all necessary info.
Later on in a .map block, I call the .where() method on one of the relations as shown below.
#all_students = #students_instance.map do |student|
...
last_pass = student.memo_tests.where(:result => true).last.created_at.utc
difference_in_weeks = ((last_pass.to_i - current_date.to_i) / 1.week).round
...
end
This leads to a single SQL query for each student. And since I have over 300+ students, leads to very slow load times and over 300+ SQL queries.
Am I right in thinking that this is caused by the .where() method. I think this because I have checked everything else and these are the two lines that cause all of the queries.
More importantly, is there a better way to do this that reduces these queries to a single query?
The moment you ask where, the statement is translated to a query. Normally, the result should be sql-cached...
Anyway, in order to be sure, you can instead add programming logic to your statement. That way, you are not requesting a NEW sql statement.
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.sort.last
EDIT
I see the OP's question does not require sorting... So, leaving the sorting out:
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.last
compact is required to remove nil results from the array.
I'm trying to work out a more efficient way to add a note count, with a couple of simple where conditions applied to the query. This can take forever, though, as there are as many as 20K records to iterate over. Would welcome any thinking on this.
def reblog_array(notes)
data = []
notes.select('note_type, count(*) as count').where(:note_type => 'reblog', :created_at => Date.today.years_ago(1)..Date.today).group('DATE(created_at)').each do |n|
data << n.count
end
return data
end
This is what's passed to reblog_array(notes) from my controller.
#tumblr = Tumblr.find(params[:id])
#notes = Note.where("tumblr_id = '#{#tumblr.id}'")
From what I can tell, you are trying to calculate how many reblogs/day this Tumblr account/blog had? If so,
notes.where(:note_type => 'reblog', :created_at => Date.today.years_ago(1)..Date.today).group('DATE(created_at)').count.values
should give you the right result, without having to iterate over the result list again. One thing to note, your call right now won't indicate when there are days with 0 reblogs. If you drop the call to #values, you'll get a hash of date => count.
As an aside and in case you didn't know, I'd also suggest making more use of the ActiveRecord relations:
Class Tumblr
has_many :notes
end
#tumblr = Tumblr.find(params[:id])
#notes = #tumblr.notes
this way you avoid writing code like Note.where("tumblr_id = '#{#tumblr.id}'"). It's best to avoid string-interpolated parameters, in favour of code like Note.where(:tumblr_id => #tumblr.id) or Note.where("tumblr_id = ?", #tumblr.id) to leave less chance that you'll write code vulnerable to SQL injection
I have rails controller coding as below:
#checked_contact_ids = #list.contacts.all(
:conditions => {
"contacts_lists.contact_id" => #list.contacts.map(&:id),
"contacts_lists.is_checked" => true
}
).map(&:id)
its equivalent to sql
SELECT *
FROM "contacts"
INNER JOIN "contacts_lists" ON "contacts".id = "contacts_lists".contact_id
WHERE ("contacts_lists".list_id = 67494 )
This above query takes more time to run, I want another way to run the same query with minimum time.
Is anyone knows please notice me Or is it possible? or is the above query enough for give output?
I am waiting information...................
I think the main problem with your original AR query is that it isn't doing any joins at all; you pull a bunch of objects out of the database via #list.contacts and then throw most of that work away to get just the IDs.
A first step would be to replace the "contacts_lists.contact_id" => #list.contacts.map(&:id) with a :joins => 'contact_lists' but you'd still be pulling a bunch of stuff out of the database, instantiating a bunch of objects, and then throwing it all away with the .map(&:id) to get just ID numbers.
You know SQL already so I'd probably go straight to SQL via a convenience method on your List model (or whatever #list is), something like this:
def checked_contact_ids
connection.execute(%Q{
SELECT contacts.id
FROM contacts
INNER JOIN contacts_lists ON contacts.id = contacts_lists.contact_id
WHERE contacts_lists.list_id = #{self.id}
AND contacts_lists.is_checked = 't'
}).map { |r| r['id'] }
end
And then, in your controller:
#checked_contact_ids = #list.checked_contact_ids
If that isn't fast enough then review your indexes on the contacts_lists table.
There's no good reason not go straight to SQL when you know exactly what data you need and you need it fast; just keep the SQL isolated inside your models and you shouldn't have any problems.
I'm writing a Rails 3 ActiveRecord query using the "where" syntax, that uses both the SQL IN and the SQL OR operator and can't figure out how to use both of them together.
This code works (in my User model):
Question.where(:user_id => self.friends.ids)
#note: self.friends.ids returns an array of integers
but this code
Question.where(:user_id => self.friends.ids OR :target => self.friends.usernames)
returns this error
syntax error, unexpected tCONSTANT, expecting ')'
...user_id => self.friends.ids OR :target => self.friends.usern...
Any idea how to write this in Rails, or just what the raw SQL query should be?
You don't need to use raw SQL, just provide the pattern as a string, and add named parameters:
Question.where('user_id in (:ids) or target in (:usernames)',
:ids => self.friends.ids, :usernames => self.friends.usernames)
Or positional parameters:
Question.where('user_id in (?) or target in (?)',
self.friends.ids, self.friends.usernames)
You can also use the excellent Squeel gem, as #erroric pointed out on his answer (the my { } block is only needed if you need access to self or instance variables):
Question.where { user_id.in(my { self.friends.ids }) |
target.in(my { self.friends.usernames }) }
Though Rails 3 AR doesn't give you an or operator you can still achieve the same result without going all the way down to SQL and use Arel directly. By that I mean that you can do it like this:
t = Question.arel_table
Question.where(t[:user_id].in(self.friends.ids).or(t[:username].in(self.friends.usernames)))
Some might say it ain't so pretty, some might say it's pretty simply because it includes no SQL. Anyhow it most certainly could be prettier and there's a gem for it too: MetaWhere
For more info see this railscast: http://railscasts.com/episodes/215-advanced-queries-in-rails-3
and MetaWhere site: http://metautonomo.us/projects/metawhere/
UPDATE: Later Ryan Bates has made another railscast about metawhere and metasearch: http://railscasts.com/episodes/251-metawhere-metasearch
Later though Metawhere (and search) have become more or less legacy gems. I.e. they don't even work with Rails 3.1. The author felt they (Metawhere and search) needed drastic rewrite. So much that he actually went for a new gem all together. The successor of Metawhere is Squeel. Read more about the authors announcement here:
http://erniemiller.org/2011/08/31/rails-3-1-and-the-future-of-metawhere-and-metasearch/
and check out the project home page:
http://erniemiller.org/projects/squeel/
"Metasearch 2.0" is called Ransack and you can read something about it from here:
http://erniemiller.org/2011/04/01/ransack-the-library-formerly-known-as-metasearch-2-0/
Alternatively, you could use Squeel. To my eyes, it is simpler. You can accomplish both the IN (>>) and OR (|) operations using the following syntax:
Question.where{(:user_id >> my{friends.id}) | (:target >> my{friends.usernames})}
I generally wrap my conditions in (...) to ensure the appropriate order of operation - both the INs happen before the OR.
The my{...} block executes methods from the self context as defined before the Squeel call - in this case Question. Inside of the Squeel block, self refers to a Squeel object and not the Question object (see the Squeel Readme for more). You get around this by using the my{...} wrapper to restore the original context.
raw SQL
SELECT *
FROM table
WHERE user_id in (LIST OF friend.ids) OR target in (LIST OF friends.usernames)
with each list comma separate. I don't know the Rails ActiveRecord stuff that well. For AND you would just put a comma between those two conditions, but idk about OR
In a Rails app, I have a model, Machine, that contains the following named scope:
named_scope :needs_updates, lambda {
{ :select => self.column_names.collect{|c| "\"machines\".\"#{c}\""}.join(','),
:group => self.column_names.collect{|c| "\"machines\".\"#{c}\""}.join(','),
:joins => 'LEFT JOIN "machine_updates" ON "machine_updates"."machine_id" = "machines"."id"',
:having => ['"machines"."manual_updates" = ? AND "machines"."in_use" = ? AND (MAX("machine_updates"."date") IS NULL OR MAX("machine_updates"."date") < ?)', true, true, UPDATE_THRESHOLD.days.ago]
}
}
This named scope works fine in development mode. In production mode, however, it returns the 2 models as expected, but the models are empty or uninitialized; that is, actual objects are returned (not nil), but all the fields are nil. For example, when inspecting the return value of the named scope in the console, the following is returned:
[#<Machine >, #<Machine >]
But, as you can see, all the fields of the objects returned are set to nil.
The production and development environments are essentially the same. Both are using a SQLite database. Here is the SQL statement that is generated for the query:
SELECT
"machines"."id",
"machines"."machine_name",
"machines"."hostname",
"machines"."mac_address",
"machines"."ip_address",
"machines"."hard_drive",
"machines"."ram",
"machines"."machine_type",
"machines"."use",
"machines"."comments",
"machines"."in_use",
"machines"."model",
"machines"."vendor_id",
"machines"."operating_system_id",
"machines"."location",
"machines"."acquisition_date",
"machines"."rpi_tag",
"machines"."processor",
"machines"."processor_speed",
"machines"."manual_updates",
"machines"."serial_number",
"machines"."owner"
FROM
"machines"
LEFT JOIN
"machine_updates" ON "machine_updates"."machine_id" = "machines"."id"
GROUP BY
"machines"."id",
"machines"."machine_name",
"machines"."hostname",
"machines"."mac_address",
"machines"."ip_address",
"machines"."hard_drive",
"machines"."ram",
"machines"."machine_type",
"machines"."use",
"machines"."comments",
"machines"."in_use",
"machines"."model",
"machines"."vendor_id",
"machines"."operating_system_id",
"machines"."location",
"machines"."acquisition_date",
"machines"."rpi_tag",
"machines"."processor",
"machines"."processor_speed",
"machines"."manual_updates",
"machines"."serial_number",
"machines"."owner"
HAVING
"machines"."manual_updates" = 't'
AND "machines"."in_use" = 't'
AND (MAX("machine_updates"."date") IS NULL
OR MAX("machine_updates"."date") < '2010-03-26 13:46:28')
Any ideas what's going wrong?
This might not be related to what is happening to you, but it sounds similar enough, so here it goes: are you using the rails cache for anything?
I got nearly the same results as you when I tried to cache the results of a query (as explained on railscast #115).
I tracked down the issue to a still open rails bug that makes cached ActiveRecords unusable - you have to choose between not using cached AR or applying a patch and getting memory leaks.
The cache works ok with non-AR objects, so I ended up "translating" the stuff I needed to integers and arrays, and cached that.
Hope this helps!
Seems like the grouping may be causing the problem. Is the data also identical in both dev & production?
Um, I'm not sure you're having the problem you think you're having.
[#<Machine >, #<Machine >]
implies that you have called "inspect" on the array... but not on each of the individual machine-objects inside it. This may be a silly question, but have you actually tried calling inspect on the individual Machine objects returned to really see if they have nil in the columns?
Machine.needs_updates.each do |m|
p m.inspect
end
?
If that does in fact result in nil-column data. My next suggestion is that you copy the generated SQL and go into the standard mysql interface and see what you get when you run that SQL... and then paste it into your question above so we can see.