I have 2 tables with millions of records:
First table is Subscription and second table is Message.
In Subscription table, I have 3 indexes:
subscription_id
msisdn
reporting_id
However, in Message table, I only have 2 indexes:
message_id
reporting_id
I have msisdn in Message table but it is not indexed.
So, I have a dashboard UI that let me search for Message.
I want to search Message using msisdn but since msisdn is not indexed, the query is taking forever!
but I want to put the idea of "Indexing msisdn in Message table" as plan B.
So my Plan A is(which is where I need help for):
Since both table has indexed reporting_id, I want to search for Message using subscription.msisdn in the dashboard, and then from there, I want to use the subscription.reporting_id and list all the Message with that subscription.reporting_id.
Something like subscription.reporting_id = message.reporting_id.
I have the view, controller and model for these 2 tables,
I tried doing this in both models
has_many :message, foreign_key: :reporting_id in Subscription.rb
and
belongs_to :subscription in Message.rb
but it didn't work.
In my view, the search_field for MSISDN is currently search for record from subscription.msisdn
Does anyone knows how can I achieve that?
PS: I want to prevent from editing the table structure.
If you want to establish a has_many :messages association in the Subscription class, you'll need to specify the primary_key.
class Subscription < ApplicationRecord
has_many :messages, foreign_key: "reporting_id", primary_key: "reporting_id"
end
With this, the SQL generated will look like this:
SELECT "messages".* FROM "messages" WHERE "messages"."reporting_id" = $1 [["reporting_id", "#{reporting_id}"]]
The primary_key ensures the reporting_id value of the Subscription model is used, else it will default to the id column.
However, rather than establishing this association which doesn't seem exactly normalized(atleast 3NF), you can just have a query to find all messages using the msisdn and reporting_id. Something like this should still be effective:
Message.
joins("INNER JOIN subscriptions ON subscriptions.reporting_id = messages.reporting_id").
merge(Subscription.where(msisdn: "#{value}"))
have to add primary key as well
has_many :message, foreign_key: :reporting_id, primary_key: :reporting_id in Subscription.rb
Found the solution!
In my Message.rb, I did:
belongs_to :subscription, foreign_key: :reporting_id, primary_key: :reporting_id
and it generated this query
SELECT `subscription`.* FROM `subscription` WHERE `subscription`.`reporting_id` = '1eaa7a61fe635005c81df347b1a7c401' LIMIT 1
Related
I'm trying to create an attribute in my select statement that depends on whether or not an association exists. I'm not sure if it's possible with a single query, and the goal is to not have to iterate a list afterward.
Here is the structure.
class Project < ApplicationRecord
has_many :subscriptions
has_many :users, through: :subscriptions
end
class User < ApplicationRecord
has_many :subscriptions
has_many :projects, through: :subscriptions
end
class Subscription < ApplicationRecord
belongs_to :project
belongs_to :user
end
Knowing a project, the goal of the query is to return ALL users and include on them a new attribute call subscribed - denoting whether or not they are subscribed.
non-working code (pseudo code):
project = Project.find_by(name: 'has_subscribers')
query = 'users.*, (subscriptions.project_id = ?) AS subscribed'
users = User.includes(:subscriptions).select(query, project.id)
user.first.subscribed
# => true or false
I'm open to whether or not there is a better way of going about this. However, the information is:
You know the project record.
You query a list of ALL users
Each user record has a subscribed attribute, denoting whether its
subscribed to the given project
Solution:
I was able to figure out a straight forward solution using the bool_or aggregate method. Coalesce ensures that the value returned is false instead of nil, should no subscriptions exists.
query = "users.*, COALESCE(bool_or(subscriptions.project_id = '#{project_id}'::uuid), false) as subscribed"
User.left_outer_joins(:subscriptions)
.select(query)
.group('users.id')
Yep, you can do this:
User.joins(:projects).select(Arel.star, Subscription.arel_table[:project_id])
Which will result in a SQL query like this:
SELECT *, "subscriptions"."project_id" FROM "users" INNER JOIN "subscriptions" ON "subscriptions"."user_ud" = "users"."id";
If you want to specify a specific project (i.e. use an expression), you can do it with Arel like this:
User.joins(:projects).select(Arel.star, Subscription.arel_table[:project_id].eq(42))
Unfortunately, you won't have a column name alias, and you can't call as on an Arel::Nodes::Equality instance. I don't know enough about the internals of Arel to have a way out of that box. But you can do this if you want the composability of Arel (e.g. if this is going to be something that needs to work with multiple models or columns):
User.joins(:projects).select(Arel.star, Subscription.arel_table[:project_id].eq(42).to_sql + " as has_project")
This is a bit clunky, but it works and provides a user.has_project method that returns a boolean. You can pretty it up like so:
class User
scope :with_project_status, lambda do |project_id|
has_project =
Subscription.arel_table[:project_id].
eq(project_id).to_sql + " as has_project"
joins(:projects).select(Arel.star, has_project)
end
end
User.with_project_status(42).where(active: true)
Alrigth, so I'm not sure if this isn't too specific... But I really don't have a clue how to construct such a query, neither in AR, nor in SQL. So here's the situation:
I have a User model. User has_many Projects. A Project, in turn, have a following associations:
Project has_one BasicEvent
Project had_many AdditionalEvents
BasicEvent and AdditionalEvent classes are built on inheritance from a AR model class, Event
Now, the goal is this: on my view, I need:
access to all Projects count per User
access to all Events count, where happened_at attribute in nil
have all the Users sorted, by the count of Events, where happened_at attribute in nil
I made a couple of attempts at it so far, but didn't really get too far... I will appreciate any help with this complicated (at least from my perspective) query.
You can start from code like:
# -*- frozen-string-literal: true -*-
class UserStatService
SELECT_CLAUSE = <<~SQL
(SELECT count(1) FROM projects WHERE user_id=users.id) AS projects_count,
(SELECT count(1) FROM events WHERE (happened_at IS NULL)
AND (project_id IN (SELECT id FROM projects WHERE user_id=users.id)
) AS events_count,
users.*
SQL
def call
User
.select( SELECT_CLAUSE )
.order( 'events_count' )
end
end
Each User instance in resultset will have projects_count and events_count attributes.
Also any kind of cache (counter_cache or handmade) is recommended. Because the query is not expected to be fast.
Projects count per User: Project.group(:user_id).count
Events count, where happened_at attribute in nil: Event.where(happened_at: nil).count
Users sorted, by the count of Events, where happened_at attribute in nil: This is the trickiest but it's also not particularly difficult:
User.
select("users.*, (SELECT COUNT(*) FROM events WHERE user_id = users.id AND happened_at IS NULL) AS unhappened_events_count").
order("unhappened_events_count DESC")
This is not a particularly efficient query but it should do its job well for several thousand records (easily) – especially if you set an index on user_id and happened_at in the events table.
Assuming basic_events and additional_events have essentially the same columns, a great solution to this problem would be to change your database/app to use single table inheritance.
Assuming you're using Rails 5.x your models would look like this:
class User < ApplicationRecord
has_many :projects
has_many :basic_events, through: :projects
has_many :additional_events, through: :projects
end
class Project < ApplicationRecord
belongs_to :user
has_one :basic_event
has_many :additional_events
end
class Event < ApplicationRecord
belongs_to :project
end
class BasicEvent < Event
belongs_to :project
end
class AdditionalEvent < Event
belongs_to :project
end
This way, there are only three tables involved users, projects and events. Your events table will have a :type column where you can specify if it's basic or additional.
For your queries:
access to all Projects count per User
User
.select("users.*, count(projects.id) AS projects_count")
.joins(:projects)
.group('users.id')
.order('projects_count DESC')
Using select like this give you access to a :projects_count method on each User object returned in this active record relation, so if you assigned this query to a variable called users_with_projects_count you could do users_with_projects_count.first.projects_count and it would return the number of projects associated with that user.
access to all Events count, where happened_at attribute in nil
User
.select("users.*, count(events.id) AS events_count")
.joins(:events)
.where('events.happened_at IS NULL')
.group('users.id')
You can access :events_count the same way you did :projects_count in the last example.
have all the Users sorted, by the count of Events, where happened_at attribute in nil
User
.select("users.*, count(events.id) AS events_count")
.joins(:events)
.where('events.happened_at IS NULL')
.group('users.id')
.order('events_count DESC')
You use the same query as the last example just add an :order.
I have a users table and a tasks table with a model with the following association:
class Task < ActiveRecord::Base
belongs_to :owner, class_name: User
end
I'm trying to build a query that will search the task's description and its owner's name as such:
includes(:owner).where("LOWER(tasks.description) LIKE ? OR LOWER(owner.name) LIKE ?", "%#{q.downcase}%", "%#{q.downcase}%")
...but I don't know how to properly query the users table for the task owners. In place of owner.name, I've tried users.name, tasks.users.name, and probably a few other things, all to no avail. How can I do this?
Note: I do not want to add a gem for this. I'm looking for a solution that is native to rails.
EDIT:
The foreign key as it exists in my schema.rb
add_foreign_key "tasks", "users", :name => "tasks_owner_id_fk", :column => "owner_id", :dependent => :nullify
SECOND EDIT:
I also need a solution that will return an AREL. I can get this to work by returning an array of objects, but I need to add other query methods to the result, so it has to be AREL.
You need to include a foreign_key of owner_id if you want to call owner.name in your view or query.
Second, you should be able to still call user, but since it's a belongs_to, it would be user.name, not users.name
If you set up the foreign_key you can call owner.name like you did before and it will work.
This should help you set it up:
http://guides.rubyonrails.org/active_record_migrations.html#foreign-keys
After digging around all day, this post helped me out, and my solution is the following:
joins("LEFT OUTER JOIN users ON users.id = tasks.owner_id").where("LOWER(users.name) LIKE ? OR
LOWER(tasks.description) LIKE ?",
"%#{q.downcase}%", "%#{q.downcase}%")
I am working with a has_many through for the first time, and despite a lot of reading here and in the guide I am not understanding the correct way to access attributes on the through table. My tables are the same as this example from another post.
class Product < ActiveRecord::Base
has_many :collaborators
has_many :users, :through => :collaborators
end
class User < ActiveRecord::Base
has_many :collaborators
has_many :products, :through => :collaborators
end
class Collaborator < ActiveRecord::Base
belongs_to :product
belongs_to :user
end
Assuming that the collaborators table has additional attributes, say hours_spent, what is the correct way to find the hours_spent from the collaborator table for a particular user and product?
When I have found my users via the product, and am iterating over them as in
#product.users.each do |user|
This seems to work
user.collaborator[0].hours_spent
I get the correct value, but since there should only be one collaborator record for each User/Product pair, the index is throwing me off, making me think I’m doing something wrong.
Thank you for reading!
EDIT
Perhaps I am not getting the has_many through concept. Maybe a MySQL example would help.
What I was thinking is that if I did
SELECT * FROM collaborators where user_id = 1;
I would expect a set (zero or more) as the result. Similarly
SELECT * FROM collaborators where product_id = 1;
would also give me a set, but
SELECT * FROM collaborators where user_id = 1 and product_id = 1;
would give at most 1 row.
If I am understanding properly, all 3 queries return a set. So I guess I need some kind of uniqueness constraint, but that would have to be a compound key of sorts, on both of the belongs to keys. Is that even possible? Is there a structure that better models this?
Thanks so much for the quick and helpful responses!
There may be a single database row per pair, but when considering a single user, that user can be associated to many products, so a user can have many rows in the collaborators table. Similarly, when considering a single product, that product can be associated to many users, so a product can have many rows in the collaborators table.
Also, instead of using user.collaborators[0].hours_spent, use user.collaborators.first.try(:hours_spent) (which may return null), if you only want the first collaborator's hours spent.
If a single user can only have one single product and a single product can only have a single user, then switch the has_many's to has_one's for everything.
Update: The preceding is the answer to the original question which has since been clarified via comments. See comments for detail and see comments on other answer by Peter.
Perhaps you should use has_and_belongs_to_many. If your Collaborator is used only to make link between User and Product without having more fields.
class Product < ActiveRecord::Base
has_and_belongs_to_many :users
end
class User < ActiveRecord::Base
has_and_belongs_to_many :products
end
The beetween migration would be:
class CreateUsersProducts < ActiveRecord::Migration
def change
create_table "users_products", :id => false do |t|
t.integer :user_id
t.integer :product_id
end
end
end
After implementing this, what I found was that I think I had the correct relationships setup, I had to use the has_many :though as users could have many products, and it needed to be :through because there are additional attributes on the collaborator table. The sticking point was how to get there to only be a single Collaborator record for each user/product pair, and then how do I guarantee I got it. And to this point the answer I've found is it has to be done in code.
To make sure there is only a single record for each pair, I used
class Collaborator < ActiveRecord::Base
validates :product_id, :presence => true, :uniqueness => {:scope => [:user_id], :message => "This is a duplicate join"}
And then to make doubly sure I'm finding the right record, I have a scope
scope :collaboration_instance, lambda {|p_id, u_id| where("collaborations.product_id = ? && collaborations.user_id = ?", p_id, u_id)}
If someone has a more elegant solution, or just wants to improve this one, please post and I will change yours to the selected answer.
I have tables 'orders' and 'items' with has_many association in the model.
class Order < ActiveRecord::Base
has_many :items
class Item < ActiveRecord::Base
belongs_to :order
Item consists of 'quantity' field, and Order consists of 'quantity_sum' field to track sum of associated items quantity.
For eg:
Order 1 : name='Toms shopping cart', quantity_sum=12
Item 1 : name='T-shirt', quantity=10
Itme 2 : name='Shoes', quantity=2
I have been looking for a way so that whenever new item is added/edited/deleted, the field 'quantity_sum' of Order gets updated automatically. Presently I have been using after_save method in Item, to update 'quantity_sum' field of Order.
Is there any other neat way of doing this besides 'after_save' ???
Similar to "counter_cache" for tracking count of associations, does rails have support for automatically keeping track of sum of some fields in the association?
Thanks
Remove the quantity_sum field from your table and add a quantity_sum method to the order class that sums up the quantity_values
class Order < ActiveRecord::Base
has_many :items
def quantity_sum
self.items.sum(:quantity)
end
end
Should do the trick. All you then need to do is remove any code you may have that updates the quantity_sum field. You will find that because the name of the method is the same as the field name (That you must not forget to delete) you won't have to refactor any of your code that makes use of it.
Obviously you need to be careful not to use this field unneccesarily like in a list of all orders in the system as this will be quite heavy on the database. O.K for a few hundred records but you'll notice a performance issue over thousands of orders.
Don't forget to remove that quantity_sum field from the order table
I think that this gem is what your'e looking for.
Look under "Totaling instead of counting" in the docs.
It should allow you to to something like this:
class Item < ActiveRecord::Base
belongs_to :order
counter_culture :order, :column_name => 'quantity_sum', :delta_column => 'quantity'
end
class Order < ActiveRecord::Base
has_many :items
end