In my controller method for the the index view I have the following line.
#students_instance = Student.includes(:memo_tests => {:memo_target => :memo_level})
So for each Student I eager-load all necessary info.
Later on in a .map block, I call the .where() method on one of the relations as shown below.
#all_students = #students_instance.map do |student|
...
last_pass = student.memo_tests.where(:result => true).last.created_at.utc
difference_in_weeks = ((last_pass.to_i - current_date.to_i) / 1.week).round
...
end
This leads to a single SQL query for each student. And since I have over 300+ students, leads to very slow load times and over 300+ SQL queries.
Am I right in thinking that this is caused by the .where() method. I think this because I have checked everything else and these are the two lines that cause all of the queries.
More importantly, is there a better way to do this that reduces these queries to a single query?
The moment you ask where, the statement is translated to a query. Normally, the result should be sql-cached...
Anyway, in order to be sure, you can instead add programming logic to your statement. That way, you are not requesting a NEW sql statement.
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.sort.last
EDIT
I see the OP's question does not require sorting... So, leaving the sorting out:
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.last
compact is required to remove nil results from the array.
Related
I've not played with Ruby in a while, and was just writing a simple db query tool.
The tool connects fine and returns the correct number of results for the query (56 rows in this case), but the value returned for each element is 'nil'. Executing the query in sqlplus works fine.
I've found similar problems on StackExchange, but most of the solutions don't apply, or require using ODBC directly. Ugh.
I'm including a stripped down version of what I've written. Any ideas what I'm doing wrong?
require 'dbi'
dbh = DBI.connect('DBI:OCI8:foodb', 'user', 'password')
rs = dbh.prepare('select field_name from foo_user.cdr_fields where layout like ?')
rs.execute('phi_outage')
while rsRow = rs.fetch do
p rsRow
end
rs.finish
dbh.disconnect
The fetch method is an iterator so no need for a while loop. Try
rs.fetch do|row|
p row unless p.nil?
end
The API states that the method gets called for all remaining rows and will return nil when done.
I'm missing something simple - I do not want to access the results of this query in a view.
Here is the query:
#adm = Admin.where({:id => {"$ne" => params[:id].to_s},:email => params[:email]})
And of course when you inspect you get:
#adm is #<MongoMapper::Plugins::Querying::DecoratedPluckyQuery:0x007fb4be99acd0>
I understand (from asking the MM guys) why this is the case - they wished to delay the results of the actual query as long as possible, and only get a representation of the query object until we render (in a view!).
But what I'm trying to ascertain in my code is IF one of my params matches or doesn't match the result of my query in the controller so I can either return an error message or proceed.
Normally in a view I'm going to do:
#adm.id
To get the BSON out of this. When you try this on the Decorated Query of course it fails:
NoMethodError (undefined method `id' for #<MongoMapper::Plugins::Querying::DecoratedPluckyQuery:0x007fb4b9e9f118>)
This is because it's not actually a Ruby Object yet, it's still the query proxy.
Now I'm fundamentally missing something because I never read a "getting started with Ruby" guide - I just smashed my way in here and learned through brute-force. So, what method do I call to get the results of the Plucky Query?
The field #adm is set to a query as you've seen. So, to access the results, you'll need to trigger execution of the query. There are a variety of activation methods you can call, including all, first, and last. There's a little documentation here.
In this case, you could do something like:
adm_query = Admin.where({:id => {"$ne" => params[:id].to_s},:email => params[:email]})
#adm_user = adm_query.first
That would return you the first user and after checking for nil
if #adm_user.nil?
# do something if no results were found
end
You could also limit the query results:
adm_query = Admin.where( ... your query ...).limit(1)
I have this site with the following parameters:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc
I use the values of each of the parameters as a value in a SQL query.
I am trying to test my application and ultimately hack my own application for learning purposes.
I'm trying to inject this statement:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc' or 1=1 --
But It fails, and MySQL says this:
Warning: mysql_fetch_assoc() expects parameter 1 to be resource,
boolean given in /home/dir/public_html/pagination.php on line 132
Is my application completely free from SQL injection, or is it still possible?
EDIT: Is it possible for me to find a valid sql injection statement to input into one of the parameters of the URL?
The application secured from sql injection never produces invalid queries.
So obviously you still have some issues.
Well-written application for any input produces valid and expected output.
That's completely vulnerable, and the fact that you can cause a syntax error proves it.
There is no function to escape column names or order by directions. Those functions do not exist because it is bad style to expose the DB logic directly in the URL, because it makes the URLs dependent on changes to your database logic.
I'd suggest something like an array mapping the "order" parameter values to column names:
$order_cols = array(
'time' => 'comment_time',
'popular' => 'comment_score',
... and so on ...
);
if (!isset($order_cols[$_GET['order'])) {
$_GET['order'] = 'time';
}
$order = $order_cols[$_GET['order']];
Restrict "sc" manually:
if ($_GET['sc'] == 'asc' || $_GET['sc'] == 'desc') {
$order .= ' ' . $_GET['sc'];
} else {
$order .= ' desc';
}
Then you're guaranteed safe to append that to the query, and the URL is not tied to the DB implementation.
I'm not 100% certain, but I'd say it still seems vulnerable to me -- the fact that it's accepting the single-quote (') as a delimiter and then generating an error off the subsequent injected code says to me that it's passing things it shouldn't on to MySQL.
Any data that could possibly be taken from somewhere other than your application itself should go through mysql_real_escape_string() first. This way the whole ' or 1=1 part gets passed as a value to MySQL... unless you're passing "sc" straight through for the sort order, such as
$sql = "SELECT * FROM foo WHERE page='{$_REQUEST['page']}' ORDER BY data {$_REQUEST['sc']}";
... which you also shouldn't be doing. Try something along these lines:
$page = mysql_real_escape_string($_REQUEST['page']);
if ($_REQUEST['sc'] == "desc")
$sortorder = "DESC";
else
$sortorder = "ASC";
$sql = "SELECT * FROM foo WHERE page='{$page}' ORDER BY data {$sortorder}";
I still couldn't say it's TOTALLY injection-proof, but it's definitely more robust.
I am assuming that your generated query does something like
select <some number of fields>
from <some table>
where sc=desc
order by comment_time
Now, if I were to attack the order by statement instead of the WHERE, I might be able to get some results... Imagine I added the following
comment_time; select top 5 * from sysobjects
the query being returned to your front end would be the top 5 rows from sysobjects, rather than the query you try to generated (depending a lot on the front end)...
It really depends on how PHP validates those arguments. If MySQL is giving you a warning, it means that a hacker already passes through your first line of defence, which is your PHP script.
Use if(!preg_match('/^regex_pattern$/', $your_input)) to filter all your inputs before passing them to MySQL.
I have rails controller coding as below:
#checked_contact_ids = #list.contacts.all(
:conditions => {
"contacts_lists.contact_id" => #list.contacts.map(&:id),
"contacts_lists.is_checked" => true
}
).map(&:id)
its equivalent to sql
SELECT *
FROM "contacts"
INNER JOIN "contacts_lists" ON "contacts".id = "contacts_lists".contact_id
WHERE ("contacts_lists".list_id = 67494 )
This above query takes more time to run, I want another way to run the same query with minimum time.
Is anyone knows please notice me Or is it possible? or is the above query enough for give output?
I am waiting information...................
I think the main problem with your original AR query is that it isn't doing any joins at all; you pull a bunch of objects out of the database via #list.contacts and then throw most of that work away to get just the IDs.
A first step would be to replace the "contacts_lists.contact_id" => #list.contacts.map(&:id) with a :joins => 'contact_lists' but you'd still be pulling a bunch of stuff out of the database, instantiating a bunch of objects, and then throwing it all away with the .map(&:id) to get just ID numbers.
You know SQL already so I'd probably go straight to SQL via a convenience method on your List model (or whatever #list is), something like this:
def checked_contact_ids
connection.execute(%Q{
SELECT contacts.id
FROM contacts
INNER JOIN contacts_lists ON contacts.id = contacts_lists.contact_id
WHERE contacts_lists.list_id = #{self.id}
AND contacts_lists.is_checked = 't'
}).map { |r| r['id'] }
end
And then, in your controller:
#checked_contact_ids = #list.checked_contact_ids
If that isn't fast enough then review your indexes on the contacts_lists table.
There's no good reason not go straight to SQL when you know exactly what data you need and you need it fast; just keep the SQL isolated inside your models and you shouldn't have any problems.
I was wondering if there was a way to use "find_by_sql" within a named_scope. I'd like to treat custom sql as named_scope so I can chain it to my existing named_scopes. It would also be good for optimizing a sql snippet I use frequently.
While you can put any SQL you like in the conditions of a named scope, if you then call find_by_sql then the 'scopes' get thrown away.
Given:
class Item
# Anything you can put in an sql WHERE you can put here
named_scope :mine, :conditions=>'user_id = 12345 and IS_A_NINJA() = 1'
end
This works (it just sticks the SQL string in there - if you have more than one they get joined with AND)
Item.mine.find :all
=> SELECT * FROM items WHERE ('user_id' = 887 and IS_A_NINJA() = 1)
However, this doesn't
Items.mine.find_by_sql 'select * from items limit 1'
=> select * from items limit 1
So the answer is "No". If you think about what has to happen behind the scenes then this makes a lot of sense. In order to build the SQL rails has to know how it fits together.
When you create normal queries, the select, joins, conditions, etc are all broken up into distinct pieces. Rails knows that it can add things to the conditions without affecting everything else (which is how with_scope and named_scope work).
With find_by_sql however, you just give rails a big string. It doesn't know what goes where, so it's not safe for it to go in and add the things it would need to add for the scopes to work.
This doesn't address exactly what you asked about, but you might investigate 'contruct_finder_sql'. It lets you can get the SQL of a named scope.
named_scope :mine, :conditions=>'user_id = 12345 and IS_A_NINJA() = 1'
named_scope :additional {
:condtions => mine.send(:construct_finder_sql,{}) + " additional = 'foo'"
}
sure why not
:named_scope :conditions => [ your sql ]