data = Program.joins(:program_schedules, :channel).
where(
"program_schedules.start" => options[:time_range],
"programs.ptype" => "movie",
"channels.country" => options[:country]).
where("programs.official_rating >= ?", options[:min_rating]).
group("programs.id").
order("programs.popularity DESC")
This query retrieve only the "programs" table (I think because the "group by" clause).
How I could retrieve all data from all tables (programs, programs_schedules, channel) ?
class Program < ActiveRecord::Base
belongs_to :channel
has_many :program_schedules
Ruby on Rails 3.2
Postgresql 9.2
Are you looking for the eager loading of associations provided by the includes method ?
Eager loading is a way to find objects of a certain class and a number of named associations. This is one of the easiest ways of to prevent the dreaded 1+N problem in which fetching 100 posts that each need to display their author triggers 101 database queries. Through the use of eager loading, the 101 queries can be reduced to 2.
Update:
You can get the SQL as a string by appending .to_sql to your query. .explain can also help.
If you want to get the data as a hash and no more as model instances, you can serialize the result of the query and include the associations using .serializable_hash :
data.serializable_hash(include: [:program_schedules, :channel])
You can define which fields to include using :except or :only options in the same way as described here for .as_json (which acts similarly) :
http://api.rubyonrails.org/classes/ActiveModel/Serializers/JSON.html#method-i-as_json
Related
Lets say I have two models: :User and :Ringtone. A :User can have many :Ringtones and a :Ringtone belongs to a user. I would like to be able to use the ActiveRecord Query interface to calculate the average number of :Ringtones that belongs to a :User, but am struggling to figure out how.
Using raw sql I can get the data like:
ActiveRecord::Base.connection.execute("
(with tones_count as
(select
user_id, count(ringtones.id)
from
ringtones
group by 1)
select avg(count) from tones_count)").values
But this is not ideal, and I would much rather be able to use ActiveRecord for this query/calculation...
I would much rather be able to use ActiveRecord, you could do like so
table = Arel::Table.new(:ringtones)
Ringtone.from(
Ringtone.select(
table[:user_id],
table[:id].count.as("tones_count")
).group(1)
).average("tones_count")
I have two models Issue and Label. They have a many to many relationship.
I have a method that returns the ten labels that point to the most issues.
class Label < ApplicationRecord
has_many :tags
has_many :issues, through: :tags
def self.top
Label.joins(:issues)
.group(:name)
.order('count_id desc')
.count(:id)
.take(10)
end
end
It does exactly what I expect it to but I want to know if it's possible to compose the query without the SQL string.
order('count_id DESC') is confusing me. Where does count_id come from? There isn’t a column named count_id.
Label.joins(:issues).group(:name).column_names
#=> ["id", "name", "created_at", "updated_at"]
I’ve found some SQL examples here. I think it’s basically the same as ORDER BY COUNT(Id):
SELECT COUNT(Id), Country
FROM Customer
GROUP BY Country
ORDER BY COUNT(Id) DESC
Is it possible to perform the same query without passing in the SQL string? Can it be done with the ActiveRecord querying interface alone?
If you look at your query log, you'll see something like:
select count(labels.id) as count_id ...
The combination of your group call (with any argument) and the count(:id) call gets ActiveRecord to add the count_id column alias to the query. I don't think this is documented or specified anywhere (at least that I can find) but you can see it happen if you're brave enough to walk through the Active Record source.
In general, if you add a GROUP BY and then count(:x), Active Record will add a count_x alias. There's no column for this so you can't say order(:count_id), order(count_id: :desc), or any of the other common non-String alternatives. AFAIK, you have to use a string but you can wrap it in an Arel.sql to prevent future deprecation issues:
Label.joins(:issues)
.group(:name)
.order(Arel.sql('count_id desc'))
.count(:id)
.take(10)
There's no guarantee about this so if you use it, you should include something in your test suite to catch any problems if the behavior changes in the future.
We have landed ourself with a column 'data' on User table that has a huge json dump in it.
Now every time we load a bunch of users we get all this data into the memory resulting into out of memory errors.
We want to be able to write some computed columns that we can use in our select statements.
eg:
Instead of doing this
user.data['profile_data']['data']['image']
We would like to add a column :image and then write a query like:
Here :name and :email are actual columns on the table and :image is a computed column:
Users.where(SOME_CONDITION).select(:name,:email,:image)
The main use case is the index page where we display all users which basically loads data column for all users
This will avoid loading the huge data column in memory and help us load the fields we want from the data column
Whats the best way to do this in Rails4?
Updates:
We use postgres on Heroku.
I would move the data column to another table, but if that's not an option try the lazy_columns gem.
class User < ActiveRecord::Base
lazy_load :data
end
Now the data column will be excluded during the initial load, but if you try to access the .data it will be retrieved from the database.
Add the column :image using the migration. Then add the below code :
class User < ActiveRecord::Base
before_save :extract_image
private
def extract_image
self.image = self.data['profile_data']['data']['image']
self.save
end
end
before_save: Is called before Base.save (regardless of whether it’s a create or update save).
Postgresql support two json data type, as described in the documentation
There are two JSON data types: json and jsonb. They accept almost identical sets of values as input. The major practical difference is one of efficiency. The json data type stores an exact copy of the input text, which processing functions must reparse on each execution; while jsonb data is stored in a decomposed binary format that makes it slightly slower to input due to added conversion overhead, but significantly faster to process, since no reparsing is needed. jsonb also supports indexing, which can be a significant advantage.
So to solve your problem, you need to change the type of the data column to jsonb through a migration:
# This should use the up and down methods, because change_column
# is not reversible
class ChangeUsersDataColumnTypeToJsonb < ActiveRecord::Migration
def up
change_column :users, :data, :jsonb
end
def down
change_column :users, :data, :text # or whatever datatype it was
end
end
than to query the image field with you use the functions that postgres provides to query the json data type :
Users.where(SOME_CONDITION).select(:name,:email,"data::json->'image' as image")
than you access the image attribute like any other attribute.
You have also to define the :data attribute as a lazy loading column, like , so that column do not get loaded when the user object is instantiated.
class User < ActiveRecord::Base
lazy_load :data
end
#integration = Integration.first(:conditions=> {:integration_name => params[:integration_name]}, :joins => :broker, :select => ['`integrations`.*, `brokers`.*'])
$stderr.puts #integration.broker.id # This line causes Brokers to be queried again
Results in:
Integration Load (0.4ms) SELECT `integrations`.*, `brokers`.* FROM `integrations` INNER JOIN `brokers` ON `brokers`.id = `integrations`.broker_id WHERE (`integrations`.`integration_name` = 'chicke') LIMIT 1
Integration Columns (1.5ms) SHOW FIELDS FROM `integrations`
Broker Columns (1.6ms) SHOW FIELDS FROM `brokers`
Broker Load (0.3ms) SELECT * FROM `brokers` WHERE (`brokers`.`id` = 1)
Any ideas why Rails would hit the databse again for brokers even though I already joined/selected them?
Here are the models (Broker -> Integration is a 1-to-many relationship). Note that this is incomplete, and I have only included the lines that establish their relationship
class Broker < ActiveRecord::Base
# ActiveRecord Associations
has_many :integrations
class Integration < ActiveRecord::Base
belongs_to :broker
I'm using Rails/ActiveRecord 2.3.14, so keep that in mind.
When I do Integration.first(:conditions=> {:integration_name => params[:integration_name]}, :include => :broker) that line causes two SELECTs
Integration Load (0.6ms) SELECT * FROM `integrations` WHERE (`integrations`.`integration_name` = 'chicke') LIMIT 1
Integration Columns (2.4ms) SHOW FIELDS FROM `integrations`
Broker Columns (1.9ms) SHOW FIELDS FROM `brokers`
Broker Load (0.3ms) SELECT * FROM `brokers` WHERE (`brokers`.`id` = 1)
Use include instead of joins to avoid reloading Broker object.
Integration.first(:conditions=>{:integration_name => params[:integration_name]},
:include => :broker)
There is no need to give the select clause as you are not trying to normalize the brokers table columns.
Note 1:
While eager loading dependencies, AR executes one SQL per dependency. In your case AR will execute main sql + broker sql. Since you are trying to get one row there isn't much gain. When you are trying to access N rows you will avoid the N+1 problem if you eager-load the dependencies.
Note 2:
In some cases it might be beneficial to use custom eager loading strategies. Let us assume that you just want to get the associated broker name for the integration. You can optimize your sql as follows:
integration = Integration.first(
:select => "integrations.*, brokers.name broker_name",
:conditions=>{:integration_name => params[:integration_name]},
:joins => :broker)
integration.broker_name # prints the broker name
The object returned by the query will have all the aliased columns in the select clause.
Above solution will not work when you want to return the Integration object even when there is no corresponding Broker object. You have to use OUTER JOIN.
integration = Integration.first(
:select => "integrations.*, brokers.name broker_name",
:conditions=>{:integration_name => params[:integration_name]},
:joins => "LEFT OUTER JOIN brokers ON brokers.integration_id = integrations.id")
The :joins options just makes active record add a join clause to the query. It doesn't actually make active record do anything with the rows that have been returned. The association isn't loaded and so accessing it triggers a query
The :include option is all about loading associations ahead of time. Active record has two strategies for doing this. One is via a big join query and one is by firing one query per association. The default is the latter, which is why you see two queries.
On rails 3.x you can decide which of those strategies you want by doing Integration.preload(:broker) or Integration.eager_graph(:broker).
There is no such facility in rails 2.x, so the only thing you can do is trick the heuristics used to determine the strategy. Whenever rails thinks that the order clause, select clause or conditions refer to columns on the included association it switches to the joins strategy (because it is the only one that works in that case).
For example doing something like
Integration.first(:conditions => {...}, :include => :broker, :select => 'brokers.id as ignored')
should force the alternate strategy (and active record actually ignores the select option in this case).
In my Rails3 app, I am using ActiveRecord and Postgresql.
Say I have a model called Parts. The model has a small list of standard attributes such as price, quantity, etc.
However, Customer A might want to add LotNumber and CustomerB might want to add OriginalLocation.
How would I allow them to do that?
I thought about creating a PartsDetail model that allowed them to have a type.
class PartsDetail < ActiveRecord::Base
attr_accessible :type, :value, :part_id
belongs_to :parts
end
So that "type" could be "LotNumber", etc.
But I'm not quite sure how that would work in my associations and querying.
Any ideas?
Thanks.
Since you're using PostgreSQL, you could use hstore to store arbitrary hashes in database columns:
This module implements the hstore data type for storing sets of key/value pairs within a single PostgreSQL value. This can be useful in various scenarios, such as rows with many attributes that are rarely examined, or semi-structured data. Keys and values are simply text strings.
There's even a gem for adding hstore support to ActiveRecord:
https://github.com/softa/activerecord-postgres-hstore
Then you could create an hstore column called, say, client_specific and look inside it with things like:
M.where("client_specific -> 'likes' = 'pancakes'")
M.where("client_specific #> 'likes=>pancakes'")
to see which clients have noted that they like pancakes.
You might want to store a list of customer-specific fields somewhere with the customer record to make the UI side of things easier to deal with but that's pretty simple to do.
Here's a gist that someone wrote that allows you to use hstore attributes in your models more seamlessly: https://gist.github.com/2834785
To use add this in an initializer ( or create a new initializer called active_record_extensions.rb )
require "hstore_accessor"
Then in your model you can have:
Class User < ActiveRecord::Base
hstore_accessor :properties, :first_name, :last_name
end
That allows you to do:
u = User.new
u.first_name = 'frank'
You can still do add attributes to the hstore column and bypass the hstore_attributes though:
u.properties['middle_name'] = 'danger'