I'm still learning ruby, rails and ActiveRecord everyday. Right now I'm learning SQL through a new small app I'm building but the problem is that the main view of my app currently does ~2000 queries per page refresh, oouuuppps.
So now that I know I have all the required information in my DB and that I can display them correctly, it is time for me to optimise them but I just don't know where to start to be honest.
These are my models associations
class League < ActiveRecord::Base
belongs_to :user
has_many :league_teams
has_many :teams, :through => :league_teams
end
class Team < ActiveRecord::Base
has_many :gameweeks
has_many :league_teams
has_many :leagues, :through => :league_teams
end
class Gameweek < ActiveRecord::Base
belongs_to :team
has_and_belongs_to_many :players
has_and_belongs_to_many :substitutes, class_name: "Player", join_table: "gameweeks_substitutes"
belongs_to :captain, class_name: "Player"
belongs_to :vice_captain, class_name: "Player"
end
class Player < ActiveRecord::Base
serialize :event_explain
serialize :fixtures
serialize :fixture_history
has_many :gameweeks, class_name: "captain"
has_many :gameweeks, class_name: "vice_captain"
has_and_belongs_to_many :gameweeks
has_many :player_fixtures
end
So this is my controller:
#league = League.includes(teams: [{gameweeks: [{players: :player_fixtures} , :captain]}]).find_by(fpl_id:params[:fpl_id])
#teams = #league.teams
#defense_widget_leaderboard = #league.position_based_leaderboard_stats(#teams, ['Defender', 'Goalkeeper'])
And this is one of the method in my League Model:
def position_based_leaderboard_stats(teams,positions_array)
leaderboard = []
teams.each do |team|
position_points = 0
gameweeks = team.gameweeks
gameweeks.each do |gameweek|
defense = gameweek.players.where(type_name:positions_array)
defense.each do |player|
player.player_fixtures.where(gw_number: gameweek.number).each do |p|
position_points += p.points
end
end
end
leaderboard << [team.team_name,position_points]
end
return leaderboard.sort_by {|team| team[1]}.reverse
end
I have 4 methods that look more or less the same thing as the one above. Each are doing between 300 and 600 queries.
As far as I read it only, it is a typical case of N+1 queries. I tried to reduce with the includes in the #league but it got me down from 2000 to 1800 queries.
I looked into group_by, joins and sum but I couldn't make it work.
The closest thing I got to working was this
players = PlayerFixture.group("player_id").sum(:points)
Where I could then query by doing players[player.id] but that doesn't give me the right results anyway because it doesn't take into account the Gameweeks > Players > Player_fixtures relationship.
How can I reduce the numbers of queries I'm doing? I went on #RubyOnRails on freenode and people told me it can be done in 1 query but wouldn't point me in any directions or help me...
Thanks
In your position_based_leaderboard_stats N+1 problem appears, too. So you can preload all your associations before each cycles:
def position_based_leaderboard_stats(teams,positions_array)
leaderboard = []
Team.preload(gameweeks: players).where('players.type_name=?', positions_array )
your code
Also, you could add player_fixtures to preload statement, but I can't understand dependencies of those associations, sorry.
Spend some time with SQL. Finally found the query that can help me with that. I also discovered SQL Views and how to use them through Activerecord which is pretty neat.
Final successful query
CREATE VIEW team_position_points AS
select teams.id as team_id, teams.team_name, players.type_name, sum(points) as points
from teams
inner join gameweeks on teams.id = gameweeks.team_id
inner join gameweeks_players on gameweeks.id = gameweeks_players.gameweek_id
inner join players on gameweeks_players.player_id = players.id
inner join player_fixtures on players.id = player_fixtures.player_id AND player_fixtures.gw_number = gameweeks.number
group by teams.id, players.type_name
Related
I have two models, Apartments and Amenities, which are associated through ApartmentAmenities. I am trying to implement a filter where I only show apartments that have all of the amenities specified.
class Amenity < ActiveRecord::Base
has_many :apartment_amenities
has_many :apartments, through: :apartment_amenities
end
class ApartmentAmenity < ActiveRecord::Base
belongs_to :apartment
belongs_to :amenity
end
class Apartment < ActiveRecord::Base
has_many :apartment_amenities
has_many :amenities, through: :apartment_amenities
end
I've got a query working that will return all apartments that match at least one of the amenities of given set like so:
Apartment.joins(:apartment_amenities).where('apartment_amenities.amenity_id IN (?)', [1,2,3])
but this isn't quite what I'm going for.
Alright, after giving up for a few days then getting back to it, I finally found this question: How to find records, whose has_many through objects include all objects of some list?
Which led me to the answer that works properly:
def self.with_amenities(amenity_ids)
where("NOT EXISTS (SELECT * FROM amenities
WHERE NOT EXISTS (SELECT * FROM apartment_amenities
WHERE apartment_amenities.amenity_id = amenities.id
AND apartment_amenities.apartment_id = apartments.id)
AND amenities.id IN (?))", amenity_ids)
end
i have the models User, Company, Product, View
class Company < ActiveRecord::Base
has_many :users
end
class Product < ActiveRecord::Base
has_many :views_by_user, -> { where viewable_type: User },
as: :viewable, class_name: "View"
end
class User < ActiveRecord::Base
has_many :viewed, as: :viewer, class_name: "View"
belongs_to :company
end
class View < ActiveRecord::Base
belongs_to :viewable, polymorphic: true
belongs_to :viewer, polymorphic: true
end
What i did with the above is, when a user views product, i save the data in the views
Now i want the list of distinct companies that have looked at my product(via user) and total count for my serializer. what i have done is,
distinct_users = #product.views_by_user
.includes(viewer: [:company])
.joins("left outer join users on views.viewer_id = users.id")
.select("distinct users.company_id, views.*")
but with this, i would have to do something like
distinct_users.will_paginate(...).map(&:viewer).map(&:company)
is there a better way to do it? also if i use distinct_users.count it throws me an error
PG::UndefinedFunction: ERROR: function count(integer, views) does not exist
LINE 1: SELECT COUNT(distinct users.company_id,...
Start from Company if this is the type of record you actually want. You can use merge to combine the conditions on a relation with those from another. Try this:
Company.joins(:users => :viewed).merge(View.where(viewable: #product))
HTH
Consider the following:
class User < ActiveRecord::Base
has_many :events
end
class Event < ActiveRecord::Base
belongs_to :user #this user is the event owner
has_many :members
end
class Members < ActiveRecord::Base
belongs_to :user
belongs_to :event
end
Now, I need to list all the members for which current_user is the owner. so I have come up with this:
#members = Member.where event_id: current_user.events
which produces the following query:
SELECT "members".* FROM "members" WHERE "members"."event_id" IN (SELECT "events"."id" FROM "events" WHERE "events"."user_id" = 1)
This works as expected but uses subqueries instead of JOIN. Does anyone know a better way to write this same query?
Add a has_many :through association to your User model:
class User < ActiveRecord::Base
has_many :events
has_many :members, :through => :events
end
Now you can query for all a user's members through the members association:
user.members
The SQL generated will look something like:
SELECT "members".* FROM "members" INNER JOIN "events" ON "members"."id" = "events"."member_id" WHERE "events"."user_id" = 1
Transformed to JOIN syntax (with table aliases to make it shorter and easier to read):
SELECT m.*
FROM events e
JOIN members m ON m.event_id = e.id
WHERE e.user_id = $1
I guess this will work.
Member.joins(:event).where("events.user_id = ?" , current_user.id)
You could do something like :
Member.joins(:event).where(events: {user_id: current_user.id})
I have the following:
has_many :sports, :through => :user_sports
has_one :primary_sport, class_name: "UserSport", conditions: ["user_sports.primary = ?", true]
has_many :user_sports
When I run this in console:
athlete = Athlete.all.last
athlete.primary_sport
The record that is returned is the record from the join table instead of the record joining the sports table. Any way to return the actual sport from the join?
You might probably do something like this:
class UserSport < ActiveRecord::Base
has_many :athletes
has_many :sports
end
athlete = Athlete.all.last
athlete.primary_sport.sport
Didn't try it by myself, just check and see :)
I'm trying to figure out how to query this relationship without using find_by_sql
class User < ActiveRecord::Base
has_many :lists
end
class List < ActiveRecord::Base
has_many :list_items
belongs_to :user
end
class ListItem < ActiveRecord::Base
belongs_to :list
belongs_to :item
end
class Item < ActiveRecord::Base
has_many :list_items
end
this should be what we are using but How would I do this not by find_by_sql
in user.rb
def self.find_users_who_like_by_item_id item_id
find_by_sql(["select u.* from users u, lists l, list_items li where l.list_type_id=10 and li.item_id=? and l.user_id=u.id and li.list_id=l.id", item_id])
end
I've tried several different includes / joins / merge scenarios but am not able to get at what I'm trying to do.
thx
It's a bit difficult to tell exactly what query you're trying to do here, but it looks like you want the user records where the user has a list with a particular list_type_id and containing a particular item. That would look approximately like this:
User.joins(:lists => [:list_items]).where('lists.list_type_id = ? and list_items.item_id = ?', list_type_id, item_id)
This causes ActiveRecord to execute a query like the following:
SELECT "users".* FROM "users" INNER JOIN "lists" ON "lists"."user_id" = "users"."id" INNER JOIN "list_items" ON "list_items"."list_id" = "lists"."id" WHERE (lists.list_type_id = 10 and list_items.item_id = 6)
and return the resulting collection of User objects.