Optimize the query PostgreSql-8.4 - ruby-on-rails-3

I have rails controller coding as below:
#checked_contact_ids = #list.contacts.all(
:conditions => {
"contacts_lists.contact_id" => #list.contacts.map(&:id),
"contacts_lists.is_checked" => true
}
).map(&:id)
its equivalent to sql
SELECT *
FROM "contacts"
INNER JOIN "contacts_lists" ON "contacts".id = "contacts_lists".contact_id
WHERE ("contacts_lists".list_id = 67494 )
This above query takes more time to run, I want another way to run the same query with minimum time.
Is anyone knows please notice me Or is it possible? or is the above query enough for give output?
I am waiting information...................

I think the main problem with your original AR query is that it isn't doing any joins at all; you pull a bunch of objects out of the database via #list.contacts and then throw most of that work away to get just the IDs.
A first step would be to replace the "contacts_lists.contact_id" => #list.contacts.map(&:id) with a :joins => 'contact_lists' but you'd still be pulling a bunch of stuff out of the database, instantiating a bunch of objects, and then throwing it all away with the .map(&:id) to get just ID numbers.
You know SQL already so I'd probably go straight to SQL via a convenience method on your List model (or whatever #list is), something like this:
def checked_contact_ids
connection.execute(%Q{
SELECT contacts.id
FROM contacts
INNER JOIN contacts_lists ON contacts.id = contacts_lists.contact_id
WHERE contacts_lists.list_id = #{self.id}
AND contacts_lists.is_checked = 't'
}).map { |r| r['id'] }
end
And then, in your controller:
#checked_contact_ids = #list.checked_contact_ids
If that isn't fast enough then review your indexes on the contacts_lists table.
There's no good reason not go straight to SQL when you know exactly what data you need and you need it fast; just keep the SQL isolated inside your models and you shouldn't have any problems.

Related

Why does the where() method run SQL queries after all nested relations are eager-loaded?

In my controller method for the the index view I have the following line.
#students_instance = Student.includes(:memo_tests => {:memo_target => :memo_level})
So for each Student I eager-load all necessary info.
Later on in a .map block, I call the .where() method on one of the relations as shown below.
#all_students = #students_instance.map do |student|
...
last_pass = student.memo_tests.where(:result => true).last.created_at.utc
difference_in_weeks = ((last_pass.to_i - current_date.to_i) / 1.week).round
...
end
This leads to a single SQL query for each student. And since I have over 300+ students, leads to very slow load times and over 300+ SQL queries.
Am I right in thinking that this is caused by the .where() method. I think this because I have checked everything else and these are the two lines that cause all of the queries.
More importantly, is there a better way to do this that reduces these queries to a single query?
The moment you ask where, the statement is translated to a query. Normally, the result should be sql-cached...
Anyway, in order to be sure, you can instead add programming logic to your statement. That way, you are not requesting a NEW sql statement.
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.sort.last
EDIT
I see the OP's question does not require sorting... So, leaving the sorting out:
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.last
compact is required to remove nil results from the array.

How to use LINQ to Entities to make a left join using a static value

I've got a few tables, Deployment, Deployment_Report and Workflow. In the event that the deployment is being reviewed they join together so you can see all details in the report. If a revision is going out, the new workflow doesn't exist yet new workflow is going into place so I'd like the values to return null as the revision doesn't exist yet.
Complications aside, this is a sample of the SQL that I'd like to have run:
DECLARE #WorkflowID int
SET #WorkflowID = 399 -- Set to -1 if new
SELECT *
FROM Deployment d
LEFT JOIN Deployment_Report r
ON d.FSJ_Deployment_ID = r.FSJ_Deployment_ID
AND r.Workflow_ID = #WorkflowID
WHERE d.FSJ_Deployment_ID = 339
The above in SQL works great and returns the full record if viewing an active workflow, or the left side of the record with empty fields for revision details which haven't been supplied in the event that a new report is being generated.
Using various samples around S.O. I've produced some Entity to SQL based on a few multiple on statements but I feel like I'm missing something fundamental to make this work:
int Workflow_ID = 399 // or -1 if new, just like the above example
from d in context.Deployments
join r in context.Deployment_Reports.DefaultIfEmpty()
on
new { d.Deployment_ID, Workflow_ID }
equals
new { r.Deployment_ID, r.Workflow_ID }
where d.FSJ_Deployment_ID == fsj_deployment_id
select new
{
...
}
Is the SQL query above possible to create using LINQ to Entities without employing Entity SQL? This is the first time I've needed to create such a join since it's very confusing to look at but in the report it's the only way to do it right since it should only return one record at all times.
The workflow ID is a value passed in to the call to retrieve the data source so in the outgoing query it would be considered a static value (for lack of better terminology on my part)
First of all don't kill yourself on learning the intricacies of EF as there are a LOT of things to learn about it. Unfortunately our deadlines don't like the learning curve!
Here's examples to learn over time:
http://msdn.microsoft.com/en-us/library/bb397895.aspx
In the mean time I've found this very nice workaround using EF for this kind of thing:
var query = "SELECT * Deployment d JOIN Deployment_Report r d.FSJ_Deployment_ID = r.Workflow_ID = #WorkflowID d.FSJ_Deployment_ID = 339"
var parm = new SqlParameter(parameterName="WorkFlowID" value = myvalue);
using (var db = new MyEntities()){
db.Database.SqlQuery<MyReturnType>(query, parm.ToArray());
}
All you have to do is create a model for what you want SQL to return and it will fill in all the values you want. The values you are after are all the fields that are returned by the "Select *"...
There's even a really cool way to get EF to help you. First find the table with the most fields, and get EF to generated the model for you. Then you can write another class that inherits from that class adding in the other fields you want. SQL is able to find all fields added regardless of class hierarchy. It makes your job simple.
Warning, make sure your filed names in the class are exactly the same (case sensitive) as those in the database. The goal is to make a super class model that contains all the fields of all the join activity. SQL just knows how to put them into that resultant class giving you strong typing ability and even more important use-ability with LINQ
You can even use dataannotations in the Super Class Model for displaying other names you prefer to the User, this is a super nice way to keep the table field names but show the user something more user friendly.

More efficient Active Record query for large number of columns

I'm trying to work out a more efficient way to add a note count, with a couple of simple where conditions applied to the query. This can take forever, though, as there are as many as 20K records to iterate over. Would welcome any thinking on this.
def reblog_array(notes)
data = []
notes.select('note_type, count(*) as count').where(:note_type => 'reblog', :created_at => Date.today.years_ago(1)..Date.today).group('DATE(created_at)').each do |n|
data << n.count
end
return data
end
This is what's passed to reblog_array(notes) from my controller.
#tumblr = Tumblr.find(params[:id])
#notes = Note.where("tumblr_id = '#{#tumblr.id}'")
From what I can tell, you are trying to calculate how many reblogs/day this Tumblr account/blog had? If so,
notes.where(:note_type => 'reblog', :created_at => Date.today.years_ago(1)..Date.today).group('DATE(created_at)').count.values
should give you the right result, without having to iterate over the result list again. One thing to note, your call right now won't indicate when there are days with 0 reblogs. If you drop the call to #values, you'll get a hash of date => count.
As an aside and in case you didn't know, I'd also suggest making more use of the ActiveRecord relations:
Class Tumblr
has_many :notes
end
#tumblr = Tumblr.find(params[:id])
#notes = #tumblr.notes
this way you avoid writing code like Note.where("tumblr_id = '#{#tumblr.id}'"). It's best to avoid string-interpolated parameters, in favour of code like Note.where(:tumblr_id => #tumblr.id) or Note.where("tumblr_id = ?", #tumblr.id) to leave less chance that you'll write code vulnerable to SQL injection

Rails SQL efficiency for where statement

Is there a more efficient method for doing a Rails SQL statement of the following code?
It will be called across the site to hide certain content or users based on if a user is blocked or not so it needs to be fairly efficient or it will slow everything else down as well.
users.rb file:
def is_blocked_by_or_has_blocked?(user)
status = relationships.where('followed_id = ? AND relationship_status = ?',
user.id, relationship_blocked).first ||
user.relationships.where('followed_id = ? AND relationship_status = ?',
self.id, relationship_blocked).first
return status
end
In that code, relationship_blocked is just an abstraction of an integer to make it easier to read later.
In a view, I am calling this method like this:
- unless current_user.is_blocked_by_or_has_blocked?(user)
- # show the content for unblocked users here
Edit
This is a sample query.. it stops after it finds the first instance (no need to check for a reverse relationship)
Relationship Load (0.2ms) SELECT "relationships".* FROM "relationships" WHERE ("relationships".follower_id = 101) AND (followed_id = 1 AND relationship_status = 2) LIMIT 1
You can change it to only run one query by making it use an IN (x,y,z) statement in the query (this is done by passing an array of ids to :followed_id). Also, by using .count, you bypass Rails instantiating an instance of the model for the resulting relationships, which will keep things faster (less data to pass around in memory):
def is_blocked_by_or_has_blocked?(user)
relationships.where(:followed_id => [user.id, self.id], :relationship_status => relationship_blocked).count > 0
end
Edit - To get it to look both ways;
Relationship.where(:user_id => [user.id, self.id], :followed_id => [user.id, self.id], :relationship_status => relationship_blocked).count > 0

Encapsulating SQL in a named_scope

I was wondering if there was a way to use "find_by_sql" within a named_scope. I'd like to treat custom sql as named_scope so I can chain it to my existing named_scopes. It would also be good for optimizing a sql snippet I use frequently.
While you can put any SQL you like in the conditions of a named scope, if you then call find_by_sql then the 'scopes' get thrown away.
Given:
class Item
# Anything you can put in an sql WHERE you can put here
named_scope :mine, :conditions=>'user_id = 12345 and IS_A_NINJA() = 1'
end
This works (it just sticks the SQL string in there - if you have more than one they get joined with AND)
Item.mine.find :all
=> SELECT * FROM items WHERE ('user_id' = 887 and IS_A_NINJA() = 1)
However, this doesn't
Items.mine.find_by_sql 'select * from items limit 1'
=> select * from items limit 1
So the answer is "No". If you think about what has to happen behind the scenes then this makes a lot of sense. In order to build the SQL rails has to know how it fits together.
When you create normal queries, the select, joins, conditions, etc are all broken up into distinct pieces. Rails knows that it can add things to the conditions without affecting everything else (which is how with_scope and named_scope work).
With find_by_sql however, you just give rails a big string. It doesn't know what goes where, so it's not safe for it to go in and add the things it would need to add for the scopes to work.
This doesn't address exactly what you asked about, but you might investigate 'contruct_finder_sql'. It lets you can get the SQL of a named scope.
named_scope :mine, :conditions=>'user_id = 12345 and IS_A_NINJA() = 1'
named_scope :additional {
:condtions => mine.send(:construct_finder_sql,{}) + " additional = 'foo'"
}
sure why not
:named_scope :conditions => [ your sql ]