After writing some large and complex SQL statements I asked sometime ago how could I do it in a better way with Rails. However I ended up using Arel again, because I couldn't nest more conditions if I were calling "find_by_sql".
Questions.find_by_sql(HUGE_SQL).by_filter(:popular).order('created_at').limit(5)
My goals are:
Organize sqls in separate files
Convert those string sqls into Arel objects
Nest this arel object with common reusable conditions like "limit, order, .."
Thanks
The solution is to use the magical scuttle.io to convert complex SQL queries to Arel. Then, you can create separate methods for each part of your query, and still be able to combine them together. More detail is in this presentation.
I needed something like this so that I could take a filtered scope from a web dashboard and pass it to a background task to generate a CSV export using the same scope. Here's what I came up with:
def sql_to_scope(model_class, sql)
model_class.from("(#{sql}) as \"#{model_class.table_name}\"")
end
Given a Customer model:
sql_to_scope(Customer, Customer.where('id % 2 = 0').to_sql).find_each { |c| puts c.id }
# 2
# 4
# 6
sql_to_scope(Customer, "select * from customers where id % 2 = 0").pluck(:id)
# => [2, 6, 4]
It works be replacing the default FROM clause with a subquery containing the original SQL. I wouldn't be surprised if it fell apart in more complex cases but it works for my simple needs.
Related
I would like to use find_by_sql and includes at the same time.
I use find_by_sql because I write select inside from to utilize index.
Somehow, index is ignored if I use left join.
But, find_by_sql does not return ActiveRecord_Relation but returns Array objects, so I cannot write
like Model.find_by_sql("select * from (select * from table limit 10)table left join rel_table on ...").includes(:rel_table,...) .
I can run two queries and hand-includes after that.
Is there any way to solve it as one SQL?
You can't call includes on the resulting array, but you can call the Rails preloader manually to do the same thing.
See ARel mimic includes with find_by_sql.
array = Model.find_by_sql("select * from ...")
# Rails 3 and 4.0.x
ActiveRecord::Associations::Preloader.new(array, [:rel_table]).run
# Rails 4.1+
ActiveRecord::Associations::Preloader.new.preload(array, [:rel_table])
I would like to have a method called feed in my User model that returns all the entries from two tables (discussions, exchanges).
In User.rb
def feed
SELECT * FROM discussions, exchanges GROUP BY created_at
end
This doesn't work, i get a problem in rails console
syntax error, unexpected ';', expecting '='
Can anyone show me how to write SQL in here? Basically I want to return and sort entries from two different tables..
if you want actual ActiveRecord objects you can try the following
def feed
exchanges = Exchange.all
discussions = Discussion.all
(exchanges + discussions).sort! { |a, b| a.created_at <=> b.created_at }
end
this is quite ineffective, as the sorting could be done in sql, but ActiveRecord cannot instantiate records selected from different tables (you can somehow override this by using STI)
Firstly - you can't just write plain SQL in your ruby code and expect it to work.
It's ruby, not SQL. They are different languages.
If you can - use the ruby-way with associations instead (as per the other example).
However - if you desperately need to use raw SQL (eg you have legavy tables that don't match to models or have some complex combination-logic in teh SQL that doesn't easily map to assocations); then you need to pass SQL to the database... which means using a connection via Active Record.
Try:
def feed
ActiveRecord::Base.connection.execute("SELECT * FROM discussions, exchanges GROUP BY created_at")
end
It will not return ruby models for you - just a raw results-object.
I'd recommend trying this in script/console and then doing a "puts my_user.feed.inspect" to have a look at the kind of thing it returns so you know how to use it.
Note: the presence of this kind of thing is considered a strong code smell - only use it where you really need it
Is there a way to get the sql for JUST a scope? So I want to do something like:
class Presentation < ActiveRecord::Base
has_many :calls
has_many :recordings, :through => :calls
scope :with_recordings, joins(:calls).joins(:recordings)
end
And then be able to get the sql for that scope.
Presentations.with_recordings.sql returns the entire sql statement, including the SELECT statement. All I want is the sql added by the scope. Figure there ought to be a way to do this.
I agree with ctcherry about this not being very useful, but having said that, I needed to do this for a project I was working on. We needed to duplicate the sql in the scopes to allow us to reuse the sql across different types of searches. Rather that have to maintain the same sql in two different places, I choose to extract the sql from the scope.
The code below is what I came up with. It's ugly, but works under Rails 3.0
def extract_named_scope_clause(scope, args)
# where_clauses will return an array of clauses for an entire relationship.
# As this is only run a single scope, we only ever care about the first.....
clause, *bind_vars = self.send(scope, args).where_clauses.first
# prefix 'and ' to the string, add some spaces and append any bind variables
if clause
[" and #{clause} ", bind_vars]
else
nil
end
end
This wouldn't really make sense, as there is no standard way to represent SQL "fragments".
The different kinds of SQL "fragments" that can be added and manipulated by a scope don't really have a clean way to be represented by themselves without being part of a complete SQL statement. A fragment could be "JOIN users ON users.id = orders.user_id" or it could be "WHERE active = 1". How would you return these without them being part of a complete SQL statement? This is most likely why there is no mechanism to retrieve them other than the one you have already discovered that just returns the complete SQL statement.
I am migrating raw PHP code to CakePHP and have some problems. As I have big problems with query to ORM transformation I temporary use raw SQL. All is going nice, but I met the ugly code and don't really know how to make it beautiful. I made DealersController and added function advanced($condition = null) (it will be called from AJAX with parameters 1-15 and 69). function looks like:
switch ($condition) {
case '1':
$cond_query = ' AND ( (d.email = \'\' OR d.email IS NULL) )';
break;
case '2':
$cond_query = ' AND (d.id IN (SELECT dealer_id FROM dealer_logo)';
break;
// There are many cases, some long, some like these two
}
if($user_group == 'group_1') {
$query = 'LONG QUERY WITH 6+ TABLES JOINING' . $cond_query;
} elseif ($user_group == 'group_2'){
$query = 'A LITLE BIT DIFFERENT LONG QUERY WITH 6+ TABLES JOINING' . $cond_query;
} else {
$query = 'A LITLE MORE BIT DIFFERENT LONG QUERY WITH 10+ TABLES JOINING' . $cond_query;
}
// THERE IS $this->Dealer->query($query); and so on
So.. As you see code looks ugly. I have two variants:
1) get out query addition and make model methods for every condition, then these conditions seperate to functions. But this is not DRY, because main 3 big queries is almost the same and if I will need to change something in one - I will need to change 16+ queries.
2) Make small reusable model methods/queries whitch will get out of DB small pieces of data, then don't use raw SQL but play with methods. It would be good, but the performance will be low and I need it as high as possible.
Please give me advice. Thank you!
If you're concerned about how CakePHP makes a database query for every joined table, you might find that the Linkable behaviour can help you reduce the number of queries (where the joins are simple associations on the one table).
Otherwise, I find that creating simple database querying methods at the Model level to get your smaller pieces of information, and then combining them afterwards, is a good approach. It allows you to clearly outline what your code does (through inline documentation). If you can migrate to using CakePHP's find methods instead of raw queries, you will be using the conditions array syntax. So one way you could approach your problem is to have public functions on your Model classes which append their appropriate conditions to an inputted conditions array. For example:
class SomeModel extends AppModel {
...
public function addEmailCondition(&$conditions) {
$conditions['OR'] = array(
'alias.email_address' => null,
'alias.email_address =' => ''
);
}
}
You would call these functions to build up one large conditions array which you can then use to retrieve the data you want from your controller (or from the model if you want to contain it all at the model layer). Note that in the above example, the conditions array is being passed by reference, so it can be edited in place. Also note that any existing 'OR' conditions in the array will be overwritten by this function: your real solution would have to be smarter in terms of merging your new conditions with any existing ones.
Don't worry about 'hypothetical' performance issues - if you've tried to queries and they're too slow, then you can worry about how to increase performance. But for starters, try to write the code as cleanly as possible.
You also might want to consider splitting up that function advanced() call into multiple Controller Actions that are grouped by the similarity of their condition query.
Finally, in case you haven't already checked it out, here's the Book's entry on retrieving data from models. There might be some tricks you hadn't seen before: http://book.cakephp.org/view/1017/Retrieving-Your-Data
If the base part of the query is the same, you could have a function to generate that part of the query, and then use other small functions to append the different where conditions, etc.
In Ruby on rails 3 I want to query on a has_many field of a model as follows:
#project.items.where(:status => 1)
The problem is I'm trying to get the exact opposite result than this. What i want is all items of #project where the status is not 1. Been looking for the answer to this for a while, anyone?
There are many ways to accomplish what you are trying to do, however, some are better than others. If you will always be searching for a hardcoded number (i.e. 1 in this case), then the following solution will work:
#project.items.where('status != 1')
However, if this value is not hard-coded, you are openly vulnerable to SQL injection as Rails will not (cannot) escape this kind of query. As a result, it is preferred among Rails developers to user the following syntax for most custom conditions (those that can't be constructed via Hash):
#project.items.where(['status != ?', 1])
This syntax is slightly confusing, so let me go over it. Basically you are providing the where clause an Array of values. The first value in the array is a String representing the query you want executed. Anywhere you want a value in that string, you place a ?. This serves as a placeholder. Next, you add an element for every question mark in you query. For example, if I had the following:
where(['first_name = ? AND last_name = ?', params[:first_name], params[:last_name]]
Rails will automatically match these up forming the query for you. In that process, it also escapes potentially unsafe characters, preventing injection.
In general, it is preferred to use the Array syntax, even for a hardcoded value. I've been told that pure string conditions in Rails 3.5 will raise a warning (unverified), so it doesn't hurt to get in the process of using the Array syntax now.