Is there any Eloquent way to get sum of relational table and sort by it? - sql

Example:
table Users
ID | Username | sex
1 | Tony | m
2 | Andy | m
3 | Lucy | f
table Scores
ID | user_id | score
1 | 2 | 4
2 | 1 | 3
3 | 1 | 4
4 | 2 | 3
5 | 1 | 1
6 | 3 | 3
7 | 3 | 2
8 | 2 | 3
Expected Result:
ID | Username | sex | score_sum (sum) (desc)
2 | Andy | m | 10
1 | Tony | m | 8
3 | Lucy | f | 5
The code I use so far:
User model:
class User extends Authenticatable
{
...
public function scores()
{
return $this->hasMany('App\Score');
}
...
}
Score model
class Job extends Model
{
//i put nothing here
}
Code in controller:
$users = User::all();
foreach ($users as $user){
$user->score_sum = $user->scores()->sum('score');
}
$users = collect($users)->sortByDesc('score_sum');
return view('homepage', [
'users' => $users->values()->all()
]);
Hope my example above make sense. My code does work, but I thought there must be an Eloquent and elegant way to do this without foreach?

There are 2 options for doing this in an Eloquent way.
Option 1
The first way is to do this to add the score_sum as an attribute that is always included when querying the users model. This is only a good idea if you will be using the score_sum the majority of the time when querying the users table. If you only need the score_sum on very specific view or for specific business logic then I would use the second option below.
To do this you will add the attribute to the users model, you can look here for documentation: https://laravel.com/docs/5.6/eloquent-mutators#defining-an-accessor
Here is an example for your use case:
/app/User.php
class User extends Model
{
.
.
.
public function getScoreSumAttribute($value)
{
return $this->scores()->sum('score');
}
}
Option 2
If you just want to do this for a single use case, then the easiest solution is just to use the sum() function in the eventual foreach loop you will be using (most likely in the view).
For example in a view:
#foreach($users as $user)
<div>Username: {{$user->username}}</div>
<div>Sex: {{$user->sex}}</div>
<div>Score Sum: {{$user->scores()->sum('price')}}</div>
#endforeach
Additionally, if you do not want to do this in a foreach loop you can use a raw query in the Eloquent call in your Controller gets the `score_sum'. Here is an example of how that can be done:
$users = User::select('score_sum',DB::raw(SUM(score) FROM 'scores'))->get();
I did not have a quick environment to test this, you might need a WHERE clause in the DB::raw query
Hope this helps!

This is as nice as it gets:
User::selectRaw('*, (SELECT SUM(score) FROM scores WHERE user_id = users.id) as score_sum')
->orderBy('score_sum', 'DESC')
->get();

Related

Weighted + ordered tag search using Postgres

After an AI file analysis across tens of thousands of audio files I end up with this kind of data structure in a Postgres DB:
id | name | tag_1 | tag_2 | tag_3 | tag_4 | tag_5
1 | first song | rock | pop | 80s | female singer | classic rock
2 | second song | pop | rock | jazz | electronic | new wave
3 | third song | rock | funk | rnb | 80s | rnb
Tag positions are really important: the more "to the left", the more prominent it is in the song. The number of tags is also finite (50 tags) and the AI always returns 5 of them for every song, no null values expected.
On the other hand, this is what I have to query:
{"rock" => 15, "pop" => 10, "soul" => 3}
The key is a Tag name and the value an arbitrary weight. Numbers of entries could be random from 1 to 50.
According to the example dataset, in this case it should return [1, 3, 2]
I'm also open for data restructuring if it could be easier to achieve using raw concatenated strings but... is it something doable using Postgres (tsvectors?) or do I really have to use something like Elasticsearch for this?
After a lot of trials and errors this is what I ended up with, using only Postgres:
Turn all data set to integers, so it moves on to something like this (I also added columns to match the real data set more closely) :
id | bpm | tag_1 | tag_2 | tag_3 | tag_4 | tag_5
1 | 114 | 1 | 2 | 3 | 4 | 5
2 | 102 | 2 | 1 | 6 | 7 | 8
3 | 110 | 1 | 9 | 10 | 3 | 12
Store requests in an array as strings (note that I sanitized those requests before with some kind of "request builder"):
requests = [
"bpm BETWEEN 110 AND 124
AND tag_1 = 1
AND tag_2 = 2
AND tag_3 = 3
AND tag_4 = 4
AND tag_5 = 5",
"bpm BETWEEN 110 AND 124
AND tag_1 = 1
AND tag_2 = 2
AND tag_3 = 3
AND tag_4 = 4
AND tag_5 IN (1, 3, 5)",
"bpm BETWEEN 110 AND 124
AND tag_1 = 1
AND tag_2 = 2
AND tag_3 = 3
AND tag_4 IN (1, 3, 5),
AND tag_5 IN (1, 3, 5)",
....
]
Simply loop in the requests array, from the most precise to the most approximate one:
# Ruby / ActiveRecord example
track_ids = []
requests.each do |request|
track_ids += Track.where([
"(#{request})
AND tracks.id NOT IN ?", track_ids
]).pluck(:id)
break if track_ids.length > 200
end
... and done! All my songs are ordered by similarity, the closest match at the top, and the more to the bottom, the more approximate they get. Since everything is about integers, it's pretty fast (fast enough on a 100K rows dataset), and the output looks like pure magic. Bonus point: it is still easily tweakable and maintainable by the whole team.
I do understand that's rough, so I'm open to any more efficient way to do the same thing, even if something else is needed in the stack (ES ?), but so far: it's a simple solution that just works.

How to expand on result of first query?

This is my first time using Postgres and I would like to write a query that expands on the elements in an array. The example is as follows.
I have some object in a table, say
+-----+------+----+
| 123 | john | AZ |
+-----+------+----+
| 456 | carl | CA |
+-----+------+----+
Another table has an object that contains an array of user ids.
+-----+-----------+
| 999 | {123,456} |
+-----+-----------+
Given the two case classes
case class User(userId: Int, name: String, country: String)
case class Group(groupId: Int, users: List[User])
I would love to write a function with this signature:
def getGroupById(groupId: Int): Future[Group] // or Future[Option[Group]]
so that
getGroupById(999) ---> Group(999, List(User(123, john, AZ), User(456, carl, CA))
For the time being I am doing it the 'brute force' way:
obtain group object with user ids
---> Future.sequence(query each user id)
---> map to desired final object
But, could I achieve this without application logic, in one single query?
I am using the slick-pg extensions for Slick to manipulate arrays in Postgres.

Do While statement not progressing to second result set in laravel using PDO

I have a stored procedure that returns multiple result sets. The results contain stats for a player, each result represents a year in which we have stats for him.
Stored procedure return from mysql CLI:
+----------+-----------+---------+------+
| compperc | passyards | passtds | ints |
+----------+-----------+---------+------+
| 61.4 | 319 | 2 | 1 |
| 85.7 | 76 | 0 | 0 |
| 20.0 | 9 | 0 | 1 |
| 57.1 | 30 | 0 | 0 |
| 100.0 | 59 | 1 | 0 |
| 66.7 | 21 | 0 | 0 |
| 50.0 | 86 | 1 | 0 |
| 60.0 | 38 | 0 | 0 |
+----------+-----------+---------+------+
8 rows in set (0.00 sec)
+----------+-----------+---------+------+
| compperc | passyards | passtds | ints |
+----------+-----------+---------+------+
| 80.0 | 40 | 0 | 0 |
| 0.0 | 0 | 0 | 0 |
| 100.0 | 40 | 0 | 0 |
+----------+-----------+---------+------+
3 rows in set (0.00 sec)
I'm using Laravel 4.1.2 and call the procedure in my Player Model with a raw PDO prepared statement:
$statDB = DB::connection('mysql')->getPdo();
$statDB->setAttribute(PDO::ATTR_EMULATE_PREPARES, true);
$results = $statDB->prepare("CALL fullstats(:id);");
$results->execute(array(':id'=> $id));
The previous block of code pulls in the proper result sets (manually iterating using
if($statDB->nextRowset()) { $statArr[] = $results->fetchAll(PDO::FETCH_ASSOC); } works) but when I try to iterate through it using a do-while statement it never gets to the second result set.
$statArr = array();
do
{
$statArr[] = $results->fetchAll(PDO::FETCH_ASSOC);
} while ($results->nextRowset());
I can add dd($statArr); immediately after I set the initial $statArr[] and it will return the set of stats for the first year. I can also add dd($results->nextRowset()); after I set the $statArr[] and it returns true so it theoretically should move through the additional result sets. If I let the statement execute I get a generic error from Laravel: PDOException SQLSTATE[HY000]: General error. It provides no additional details as to what's going wrong. I've tried the same do-while statement in a raw php file (on a different domain but the same server) using PDO and it works without a problem.
Is there some configuration option that I need to set to get this to work? I've been beating my head against the problem for an entire day and can't figure out why this isn't working. Any help is greatly appreciated.
Update:
The PDOException SQLSTATE[HY000]: General error is coming from the second $statArr[] = $results->fetchAll(); so it's entering the second result set, it just won't fetch the data. I also removed the PDO::FETCH_ASSOC from fetchAll() as I've read it doesn't work properly but the issue persists.
It turns out this is a bug in the PDO/mysql driver. The nextRowset() method doesn't return false when it's out of rowsets so the fetch statement tries to access data that isn't there. This causes the nondescript general error on the fetchall() method and in turn breaks the script.
The PHP bug report:
https://bugs.php.net/bug.php?id=62820
Since the nextRowset() method wasn't working properly I added a build query that gives me the count of the results I should expect and then have a counter in the loop with an if statement that breaks the loop if the loop count is equal to the count returned by the query.
Here's the working function:
public function allstats($id) {
$statDB = DB::connection('mysql')->getPdo();
$statDB->setAttribute(PDO::ATTR_EMULATE_PREPARES, true);
$years = $statDB->prepare('select count(distinct year) from pass_stats where player_id = :id');
$years->execute(array(':id'=> $id));
$years = (int) $years->fetchColumn();
$results = $statDB->prepare("CALL fullstats(:id);");
$results->execute(array(':id'=> $id));
$statArr = array();
$i = 1; // Set to one to mimic sql count
do
{
if($i === $years) { break; }
foreach($results->fetchall(PDO::FETCH_ASSOC) as $g) {
$statArr[$g['year']][] = $g;
}
$i++;
} while ($results->nextRowset()); // Check for next rowset
return $statArr;
}

Rails, SQL: private chat, how to find last message in each conversation

I'v got the folowing schema
+----+------+------+-----------+---------------------+--------+
| id | from | to | message | timestamp | readed |
+----+------+------+-----------+---------------------+--------+
| 46 | 2 | 6 | 123 | 2013-11-19 19:12:19 | 0 |
| 44 | 2 | 3 | 123 | 2013-11-19 19:12:12 | 0 |
| 43 | 2 | 1 | ????????? | 2013-11-19 18:37:11 | 0 |
| 42 | 1 | 2 | adf | 2013-11-19 18:37:05 | 0 |
+----+------+------+-----------+---------------------+--------+
from/to is the ID of the user's, message – obviously, the message, timestamp and read flag.
When user open's his profile I want him to see the list of dialogs he participated with last message in this dialog.
To find a conversation between 2 people I wrote this code, it's simple (Message model):
def self.conversation(from, to)
where(from: [from, to], to: [from, to])
end
So, I can now sort the messages and get the last one. But it's not cool to fire a lot of queries for each dialog.
How could I achieve the result I'm looking for with less queries?
UPDATE:
Ok, looks like it's not really clear, what I'm trying to achieve.
For example, 4 users – Kitty, Dandy, Beggy and Brucy used that chat.
When Brucy entered in dialogs, she shall see
Beggy: hello brucy haw ar u! | <--- the last message from beggy
-------
Dandy: Hi brucy! | <---- the last message from dandy
--------
Kitty: Hi Kitty, my name is Brucy! | <–– this last message is from current user
So, three separated dialogs. Then, Brucy can enter anyone dialog to continue private conversation.
And I can't figured out how could I fetch this records without firing a query for each dialog between users.
This answer is a bit late, but there doesn't seem to be a great way to do this, in Rails 3.2.x at least.
However, here is the solution I came up with
(as I had the same problem on my website).
#sender_ids =
Message.where(recipient_id: current_user.id)
.order("created_at DESC")
.select("DISTINCT owner_id")
.paginate(per_page: 10, page: params[:page])
sql_queries =
#sender_ids.map do |user|
user_id = user.owner_id
"(SELECT * FROM messages WHERE owner_id = #{user_id} "\
"AND recipient_id = #{current_user.id} ORDER BY id DESC "\
"LIMIT 1)"
end.join(" UNION ALL ")
#messages = Message.find_by_sql(sql_queries)
ActiveRecord::Associations::Preloader.new(#messages, :owner).run
This gets the last 10 unique people you sent messages to.
For each of those people, it creates a UNION ALL query to get the last message sent to each of those 10 unique people. With 50, 000 rows, the query completes in about ~20ms. And of course to get assocations to preload, you have to use .includes will not work when using .find_by_sql
def self.conversation(from, to)
order("timestamp asc").last
end
Edit:
This railscast will be helpful..
http://railscasts.com/episodes/316-private-pub?view=asciicast
EDIT2:
def self.conversation(from, to)
select(:from, :to, :message).where(from: [from, to], to: [from, to]).group(:from, :to, :country).order("timestamp DESC").limit(1)
end

SQL - Update table changing one column value based on another table

Sorry if the title is not as descriptive as it should be but it is kind of difficult to explain in one sentence what I am trying to do ;).
I have one table that links parent objects with its respective childs. And I have another table with all the objects (parents and childs) with its respectives images. However, the image is just set for the parents objects. I would like to update this last table and set the childs image the same image that is already set for its parent. Besides, as there is more than one image for each object, I would like to set one in particular, which I can know based on an attribute column.
My tables look something like:
RelationTable
child-id
parent-id
ImageTable
object-id
attribute-id
image-url
And here goes an example in order to clarify things:
RelationsTable
child-id | parent-id
3 | 1
4 | 1
5 | 2
ImageTable
object-id | attribute-id | image-url
1 | goodimage | image1.jpg
1 | badimage | image1b.jpg
2 | goodimage | image2.jpg
2 | badimage | image2b.jpg
3 | goodimage | no
3 | badimage | no
4 | goodimage | no
4 | badimage | no
5 | goodimage | no
5 | badimage | no
So, I would like to set the images of objects 3, 4 and 5 (child ones) to its respective parent images, but to the 'correct' ones, that is the images with 'goodimage' as attribute-id.
At the end it should look like:
1 | goodimage | image1.jpg
1 | badimage | image1b.jpg
2 | goodimage | image2.jpg
2 | badimage | image2b.jpg
3 | goodimage | image1.jpg
3 | badimage | no
4 | goodimage | image1.jpg
4 | badimage | no
5 | goodimage | image2.jpg
5 | badimage | no
Actually, I don't care if 'badimage' is set as well, but the important one is 'goodimage'.
I've been trying something like:
UPDATE ImageTable
SET image = (SELECT image-url FROM ImageTable WHERE ImageTable.object-id = RelationTable.parent-id AND ImageTable.attribute-id = 'goodimage')
WHERE ImageTable.object-id = RelationTable.child-id AND ImageTable.attribute-id = 'goodimage'
but it's not working since it is not correct SQL syntax. I don't know if I should use a variable (never used one) or if this can be done with just one SQL sentence.
Any help would be much appreciated.
something like this?
NOTES:
This could be merged into one
non-subquery statement but I was
lazy.
I did not test, expect typos
;WITH goodlist AS
(
SELECT child-id, image-url
FROM relationstable
left join imagetable on relationstable.child-id = relationstable.child-id and attribute-id = "goodimage"
)
UPATE imagetable
set imagetable.image-url =
(SELECT top 1 image-url from goodlist where child-id = imagetable.object-id)
WHERE imagetable.image-url = "no"
First of all, if the relation between parent and child is 1:n, why don't you just add that information to the same table? e.g. fieldname "parentid" (if it's empty/null it's only a parent). Benefit: you don't need an additional table and can easily create a hierarchy if needed.
And for the image, I guess you want to display this somewhere in your code, but it should be easier to just modify the SELECT to get the image-url of the parent is no image is given for the child. Benefit: no need to update the table when you get new entries and it would also be possible to for some child entries to have their own image (if needed).
edit: just noticed your new comment about the open-source project part, of course this makes it hard to change the database design. But maybe it's still easier to move some of the problems to the programming side (unless this is handled by the open-source software as well).