I'm trying to increase my app's efficiency by doing work in the database rather than in the app layer, and I'm wondering if I can move this calculation into the database.
Models:
class Offer < ActiveRecord::Base
has_many :lines
has_many :items, :through => :lines
end
class Line < ActiveRecord::Base
belongs_to :offer
belongs_to :item
# also has a 'quantity' attribute (integer)
end
class Item < ActiveRecord::Base
has_many :lines
has_many :offers, :through => :lines
# also has a 'price' attribute (decimal)
end
What I want to do is calculate the price of an offer. Currently I have a price method in the Offer class:
def price
self.lines.inject(0) do |total, line|
total + line.quantity * line.item.price
end
end
I suspect it may be possible to do a Offer.sum calculation instead that would get the answer directly from the DB rather than looping through the records, but the Calculations section of the ActiveRecord query guide doesn't have enough detail to help me out. Anybody?
Thanks!
You're correct that you can do this with sum. Something like this:
class Offer < ActiveRecord::Base
# ...
def price
self.lines.sum 'lines.quantity * items.price', :joins => :item
end
end
When you call e.g. Offer.find( some_id ).price the above will construct a query something like this:
SELECT SUM( lines.quantity * items.price ) AS total
FROM lines
INNER JOIN items ON items.id = lines.item_id
WHERE lines.offer_id = <some_id>
;
Sometimes you're better off with SQL.
SELECT SUM( lines.quantity * items.price ) AS total
FROM offers
INNER JOIN lines ON offers.id = lines.offer_id
INNER JOIN items ON items.id = lines.item_id
WHERE offers.id = 1
;
Related
I'm still learning ruby, rails and ActiveRecord everyday. Right now I'm learning SQL through a new small app I'm building but the problem is that the main view of my app currently does ~2000 queries per page refresh, oouuuppps.
So now that I know I have all the required information in my DB and that I can display them correctly, it is time for me to optimise them but I just don't know where to start to be honest.
These are my models associations
class League < ActiveRecord::Base
belongs_to :user
has_many :league_teams
has_many :teams, :through => :league_teams
end
class Team < ActiveRecord::Base
has_many :gameweeks
has_many :league_teams
has_many :leagues, :through => :league_teams
end
class Gameweek < ActiveRecord::Base
belongs_to :team
has_and_belongs_to_many :players
has_and_belongs_to_many :substitutes, class_name: "Player", join_table: "gameweeks_substitutes"
belongs_to :captain, class_name: "Player"
belongs_to :vice_captain, class_name: "Player"
end
class Player < ActiveRecord::Base
serialize :event_explain
serialize :fixtures
serialize :fixture_history
has_many :gameweeks, class_name: "captain"
has_many :gameweeks, class_name: "vice_captain"
has_and_belongs_to_many :gameweeks
has_many :player_fixtures
end
So this is my controller:
#league = League.includes(teams: [{gameweeks: [{players: :player_fixtures} , :captain]}]).find_by(fpl_id:params[:fpl_id])
#teams = #league.teams
#defense_widget_leaderboard = #league.position_based_leaderboard_stats(#teams, ['Defender', 'Goalkeeper'])
And this is one of the method in my League Model:
def position_based_leaderboard_stats(teams,positions_array)
leaderboard = []
teams.each do |team|
position_points = 0
gameweeks = team.gameweeks
gameweeks.each do |gameweek|
defense = gameweek.players.where(type_name:positions_array)
defense.each do |player|
player.player_fixtures.where(gw_number: gameweek.number).each do |p|
position_points += p.points
end
end
end
leaderboard << [team.team_name,position_points]
end
return leaderboard.sort_by {|team| team[1]}.reverse
end
I have 4 methods that look more or less the same thing as the one above. Each are doing between 300 and 600 queries.
As far as I read it only, it is a typical case of N+1 queries. I tried to reduce with the includes in the #league but it got me down from 2000 to 1800 queries.
I looked into group_by, joins and sum but I couldn't make it work.
The closest thing I got to working was this
players = PlayerFixture.group("player_id").sum(:points)
Where I could then query by doing players[player.id] but that doesn't give me the right results anyway because it doesn't take into account the Gameweeks > Players > Player_fixtures relationship.
How can I reduce the numbers of queries I'm doing? I went on #RubyOnRails on freenode and people told me it can be done in 1 query but wouldn't point me in any directions or help me...
Thanks
In your position_based_leaderboard_stats N+1 problem appears, too. So you can preload all your associations before each cycles:
def position_based_leaderboard_stats(teams,positions_array)
leaderboard = []
Team.preload(gameweeks: players).where('players.type_name=?', positions_array )
your code
Also, you could add player_fixtures to preload statement, but I can't understand dependencies of those associations, sorry.
Spend some time with SQL. Finally found the query that can help me with that. I also discovered SQL Views and how to use them through Activerecord which is pretty neat.
Final successful query
CREATE VIEW team_position_points AS
select teams.id as team_id, teams.team_name, players.type_name, sum(points) as points
from teams
inner join gameweeks on teams.id = gameweeks.team_id
inner join gameweeks_players on gameweeks.id = gameweeks_players.gameweek_id
inner join players on gameweeks_players.player_id = players.id
inner join player_fixtures on players.id = player_fixtures.player_id AND player_fixtures.gw_number = gameweeks.number
group by teams.id, players.type_name
I think I want to do a union in Rails, but according to this post rails union hack, how to pull two different queries together unions aren't natively supported in Rails. I'm wondering if there is a better way to approach this problem.
I have table of items, each item has many prices, but I only want to join one price to each item.
To determine the proper price for an item I have two additional foreign keys in the price model: category_id and discount_id. Each could independently declare a price for an item.
Ex.
Item + Category = Price1 and Item + Discount = Price 2
If discount_id matches a passed id I want to exclude the price results FOR THAT ITEM ONLY that match Item + Category. Also I'm trying not to loose lazy loading.
Hopefully the problem is clear! If not I'll try to clarify more, thanks in advance.
Your models would start off looking something like this:
class Price < ActiveRecord::Base
belongs_to :item
belongs_to :category
belongs_to :discount
scope :category, where("prices.category_id IS NOT NULL")
scope :discount, where("prices.discount_id IS NOT NULL")
end
class Item < ActiveRecord::Base
has_many :prices
end
class Category < ActiveRecord::Base
has_many :prices
end
class Discount < ActiveRecord::Base
has_many :prices
end
One way of doing this is to add a class method to Price that encapsulates this logic:
class Price < ActiveRecord::Base
def self.used
discount_items_sql = self.discount.select("prices.item_id").to_sql
where("prices.discount_id IS NOT NULL OR prices.item_id NOT IN (#{discount_items_sql})")
end
end
This is effectively the same as this query:
SELECT * FROM prices
WHERE prices.discount_id IS NOT NULL -- the discount_id is present on this record,
OR prices.item_id NOT IN ( -- or no discount_id is present for this item
SELECT item_id FROM prices WHERE discount_id IS NOT NULL)
You can add these helper methods on your Item model for simplicity:
class Item < ActiveRecord::Base
def category_price
prices.category.first
end
def discount_price
prices.discount.first
end
def used_price
prices.used.first
end
end
Now you can easily get each individual 'type' of price for a single item (will be nil for prices that aren't available):
item.category_price
item.discount_price
item.used_price
I'm trying to figure out how to query this relationship without using find_by_sql
class User < ActiveRecord::Base
has_many :lists
end
class List < ActiveRecord::Base
has_many :list_items
belongs_to :user
end
class ListItem < ActiveRecord::Base
belongs_to :list
belongs_to :item
end
class Item < ActiveRecord::Base
has_many :list_items
end
this should be what we are using but How would I do this not by find_by_sql
in user.rb
def self.find_users_who_like_by_item_id item_id
find_by_sql(["select u.* from users u, lists l, list_items li where l.list_type_id=10 and li.item_id=? and l.user_id=u.id and li.list_id=l.id", item_id])
end
I've tried several different includes / joins / merge scenarios but am not able to get at what I'm trying to do.
thx
It's a bit difficult to tell exactly what query you're trying to do here, but it looks like you want the user records where the user has a list with a particular list_type_id and containing a particular item. That would look approximately like this:
User.joins(:lists => [:list_items]).where('lists.list_type_id = ? and list_items.item_id = ?', list_type_id, item_id)
This causes ActiveRecord to execute a query like the following:
SELECT "users".* FROM "users" INNER JOIN "lists" ON "lists"."user_id" = "users"."id" INNER JOIN "list_items" ON "list_items"."list_id" = "lists"."id" WHERE (lists.list_type_id = 10 and list_items.item_id = 6)
and return the resulting collection of User objects.
I have an application where users can customize a calendar and fill it with a given pool of events. A user can also overwrite a title for his own calendar by an alias. So I have the following has_many :through relation:
class Calendar < ActiveRecord::Base
has_many :event_aliases
has_many :events, :through => :event_aliases
end
class Event < ActiveRecord::Base
attr_accessible :title
has_many :event_aliases
has_many :calendars, :through => :event_aliases
end
class EventAliases < ActiveRecord::Base
attr_accessible :course_id, :calendar_id, :custom_name
belongs_to :event
belongs_to :calendar
end
No I want to deliver the calendar with the aliases. If an event has an alias (custom_name), it should be displayed. Otherwise the default event name (title) should be displayed.
Is there a way to easily set up a query that returns all events for the current calendar whether with a custom_name (if exists) or with the default title?
My current solution is to hardcode an if condition into the query which I would like to avoid.
title_column = "case when custom_name IS NOT NULL then custom_name else title end as title"
# assume we are given a calendar_id
Calendar.find(calendar_id).event_aliases.joins(:event).select(title_column, :event_id).each do |event_alias|
# do further stuff here
end
I also could fetch all event_aliases and run through each of them to get the default title if necessary.
# assume we are given a calendar_id
Calendar.find(calendar_id).event_aliases.each do |event_alias|
title = event_alias.custom_name
if title.nil?
title = Event.find(event_alias.event_id).title
# do further stuff here
end
But this one results in too many queries to me.
So is there any smarter way of accomplishing what I want? Maybe using named scopes or another fancy rails technique?
UPDATE
I ended up with making a "custom" select via the has_many :through relationship. So the only thing changes is the Calendar model:
class Calendar < ActiveRecord::Base
has_many :event_aliases
has_many :events, :through => :event_aliases,
:select => "event_aliases.custom_name as custom_name, events.*"
end
So accessing the custom_name / the title now happens a little like #Doon suggested:
Calendar.find(1).courses.each do |course|
title = course.custom_name || course.title
end
This creates only 2 queries instead of 3:
Calendar Load (0.6ms) SELECT `calendars`.* FROM `calendars` WHERE `calendars`.`id` = 1 LIMIT 1
Event Load (0.7ms) SELECT event_aliases.custom_name as custom_name, events.* FROM `events` INNER JOIN `event_aliases` ON `events`.`id` = `event_aliases`.`event_id` WHERE `event_aliases`.`calendar_id` = 1
what about using includes to grab the events at the same time as you pull the aliases.
Calendar.find(1).event_aliases.includes(:event).each do |e|
puts e.custom_name.blank? ? e.event.title : e.custom_name
end
the SQL Rails generates will look something like this:
Calendar Load (0.2ms) SELECT "calendars".* FROM "calendars" WHERE "calendars"."id" = ? LIMIT 1
EventAlias Load (0.2ms) SELECT "event_aliases".* FROM "event_aliases" WHERE "event_aliases"."calendar_id" = 1
Event Load (0.2ms) SELECT "events".* FROM "events" WHERE "events"."id" IN (1, 2)
also if you want to clean it up a bit you can add a virtual field to the EventAlias
class EventAlias < ActiveRecord::Base
def name
custom_name || self.event.title
end
end
As long as you use the includes, the queries will be be the same.
Can someone help me understand how to convert this sql query to a named_scope or maybe a method?
Background: A trip can have many trip_runs. I'm trying to be able to say TripRun.upcoming and return only valid runs from valid trips based on the following query
SELECT r.*
FROM trip_run r
LEFT JOIN trips t
ON r.trip_id = t.id
WHERE r.starts_on > NOW()
AND t.is_booked = 1
AND t.is_cancelled IS NULL
thank you
this assumes you're using Rails 3 and AREL. Edit your model files for Trip and TripRun like so:
class Trip < ActiveRecord::Base
has_many :trip_runs
scope :booked, where("is_booked = 1 and isnull(is_cancelled)")
end
class TripRun < ActiveRecord::Base
belongs_to :trip
scope :upcoming, where("starts_on > NOW()")
end
Then access like this:
Trip.booked.trip_runs.upcoming
There are lots of ways to mix and match these patterns to get similar effects and create methods to access the data.
Based on Joshua's answer, but for rails 2.3:
class Trip < ActiveRecord::Base
has_many :trip_runs
named_scope :booked, :conditions => 'is_booked = 1 AND is_cancelled IS NULL'
end
class TripRun < ActiveRecord::Base
belongs_to :trip
named_scope :upcoming, :conditions => 'starts_on > NOW()'
end
Trip.booked.trip_runs.upcoming
or alternatively:
class Trip < ActiveRecord::Base
has_many :trip_runs
end
class TripRun < ActiveRecord::Base
belongs_to :trip
named_scope :upcoming,
:conditions => 'trip.is_booked = 1 AND trip.is_cancelled IS NULL
AND trip_runs.starts_on > NOW()',
:joins => :trip
end
TripRun.upcoming
That will use an INNER JOIN not a LEFT JOIN, but since you're looking for rows with trip.is_booked set to a non-null value, the results will be the same and the query will be no slower.