Implementing Two Level Aggregate in PostgreSQL using Sequelize Function for NodeJS App - sql

Hello guys please bear with me here. I'm using PostgreSQL, Sequelize, Express, and NodeJS to create a backend. I'm wondering if these lines of raw query code can be implemented using Sequelize Model findAll function.
First of all, what I am trying to do here is to calculate the total score of these students. Here are some tables and their relations.
Student Level
| student_id | name | level_id | | level_id | level_name |
|:----------:|:----------:|:--------:| |:--------:|:----------:|
| 1 | John | 1 | > | 1 | Rookie |
| 2 | Jane | 2 | | 2 | Expert |
v
StudentQuiz
| quiz_id | student_id | score |
|:----------:|:----------:|:--------:|
| 1 | 1 | 40 |
| 1 | 1 | 100 |
| 2 | 1 | 80 |
| 1 | 2 | 100 |
| 2 | 2 | 100 |
If I run line of codes below.
SELECT table2.student_id,
s.canvasser_name,
l.level_name,
table2.total_score
FROM (SELECT table1.student_id,
sum(table1.max_score) total_score
FROM (SELECT sq.student_id,
max(sq.score) max_score
FROM public.StudentQuiz sq
GROUP BY sq.quiz_id, sq.student_id) table1
GROUP BY table1.student_id) table2
INNER JOIN public.Student s
ON s.student_id = table2.student_id
INNER JOIN public.Level l
ON l.level_id = s.level_id
ORDER BY table2.total_score DESC
LIMIT 10;
I will get something like this.
| student_id | name | level | total_score |
|:----------:|:----------:|:--------:|:--------------:|
| 1 | John | Rookie | 180 |
| 2 | Jane | Expert | 200 |
Please note that I'm selecting the highest score if more than one quiz with the same id found.
Anyway, I want to implement it using sequelize built in function. What I've been trying to do is something like this.
const result = await StudentQuiz.findAll({
attributes: ['studentId', [sequelize.fn('sum', sequelize.fn('max', sequelize.col('score'))), 'totalPrice'], 'quizId'],
group: 'studentId',
include: [
{
model: Student,
include: [{
model: Level
}],
},
],
offset: 0,
limit: 10
});
The code above throws an error message which is "aggregate function calls cannot be nested".
Any kind of help will be appreciated. Thank you.
P.S. I know i can use sequelize.query() function to use the first code block shown, but that's not the point.

Sequelize is not intended to work with complex aggregations using models. Its primary goal is to provide CRUD operations with models.
To use models in such scenario you can use a model definition to get a schema, a table name and fields to build a query dynamically not knowing exact field names.

I find the solution without using any raw query, though I need to get two tables, that is StudentQuiz and Student that coupled by Level. Here is my answer.
// Finding maximum score and group it based on studentId and quizId
const maxScoreList = await StudentQuiz.findAll({
attributes: ['studentId', 'quizId', [sequelize.fn('max', sequelize.col('score')), 'maxScore']],
group: ['studentId', 'quizId'],
order: ['studentId', 'quizId'],
raw: true
});
// Calculating total score for the same student for each quiz recorded
const scoreArray = [maxScoreList.shift()];
let index = 0;
const unfilteredStudentId = maxScoreList.map((item) => {
if (scoreArray[index].studentId !== item.studentId) {
scoreArray.push(item);
index += 1;
}
scoreArray[index].maxScore += item.maxScore;
return item.studentId;
});
// Filtering studentId that show up more than one time
const extractedStudentId = [...new Set(unfilteredStudentId)];
// Finding student based on studentId inside extractedStudentId array
const student = await Student.findAll({
where: { id: extractedStudentId },
attributes: ['id', 'canvasserId', 'canvasserName', 'canvasserImageUrl'],
include: {
model: Level,
attributes: [['level_name', 'level'], ['icon_url', 'level_image_url']]
},
order: ['id'],
raw: true,
nest: true
});
// Combining total score list to student list
const rankList = student.map((item, idx) => ({ ...item, totalScore: scoreArray[idx] }));
With this much complexity, I agree that using raw query by far the best approach for this case.

Related

Condition with SQL

I've come to see you for a question. Is there a condition in SQL that allows you to do that:
IF(sup = void) {
}
Database
id | name | lastname | city | mail | number | picture | ...
1 | kiwi | kiwi | USA | kiwi#gmail.com | 0000000000 | default.img | (vide)
SELECT * FROM your_table WHERE sup IS NULL
https://www.w3schools.com/sql/sql_null_values.asp
Update after reading your comment.
$test = $db->query("SELECT * FROM ressource_view WHERE ID = 1")
Will give you the result of your query. Be careful as there could be multiple rows returned.
To fetch the first row
$row = $result->fetch_array()
And then to check if the sup column of your row is null you can use:
if(is_null($row['sup']))
{
}
Or this will have the same effect
if($row['sup'] === NULL)
{
}
But best to tag your question with PHP, MySQL. Your problem seems to be more on the PHP side and someone else could provide a better answer.

How to get collection users with collection tasks inside?

User has_many tasks
Task belongs_to user
(Task has key user_id)
I`m trying make sql query
User.find_by_sql("
SELECT
users.id,
users.name,
tasks # <-- how can I get collection tasks inside each users?
FROM users
JOIN tasks
ON tasks.user_id = users.id
")
I would like to get next response
[
{
id: 1,
name: Jon,
tasks: [ # <- collection, please
{ id: 1, user_id: 1, title: ... },
{ id: 2, user_id: 1, title: ... }
]
},
{
id: 2,
name: Sofia,
tasks: [ # <- collection, please
{ id: 3, user_id: 2, title: ... },
{ id: 4, user_id: 2, title: ... }
]
]
If use ActiveRecord, it be User.includes(:tasks)
How can I get collection tasks inside each users? (sql)
Is it possible?
To select rows from the joined table just specify the name of the table and the column:
SELECT
users.id,
users.name,
tasks.id AS task_id,
tasks.name AS task_name
FROM
users
LEFT OUTER JOIN
tasks ON tasks.user_id = users.id
Since SQL is tabular the result is a list of rows and columns:
id | name | task_id | task_name
----+----------+---------+-----------
2 | Xzbdulia | 1 | Cleaning
2 | Xzbdulia | 2 | Laundry
2 | Xzbdulia | 3 | Coding
3 | Elly | 4 | Cleaning
3 | Elly | 5 | Laundry
3 | Elly | 6 | Coding
4 | Lewis | 7 | Cleaning
4 | Lewis | 8 | Laundry
4 | Lewis | 9 | Coding
5 | Hang | 10 | Cleaning
5 | Hang | 11 | Laundry
5 | Hang | 12 | Coding
If you really want to rough it and do this as a raw sql query and handle processing the results yourself you can do:
sql = <<~SQL
SELECT
users.id,
users.name,
tasks.id AS task_id,
tasks.name AS task_name
FROM
users
LEFT OUTER JOIN
tasks ON tasks.user_id = users.id;
SQL
results = User.connection.execute(sql).each_with_object({}) do |row, memo|
id = row["id"]
memo[id] ||= {
id: id,
name: row["name"],
tasks: []
}
memo[id][:tasks].push({
id: row["task_id"],
name: row["task_name"]
}) unless row["task_id"].nil?
end.values
[{:id=>2, :name=>"Xzbdulia", :tasks=>[{:id=>1, :name=>"Cleaning"}, {:id=>2, :name=>"Laundry"}, {:id=>3, :name=>"Coding"}]}, {:id=>3, :name=>"Elly", :tasks=>[{:id=>4, :name=>"Cleaning"}, {:id=>5, :name=>"Laundry"}, {:id=>6, :name=>"Coding"}]}, {:id=>4, :name=>"Lewis", :tasks=>[{:id=>7, :name=>"Cleaning"}, {:id=>8, :name=>"Laundry"}, {:id=>9, :name=>"Coding"}]}, {:id=>5, :name=>"Hang", :tasks=>[{:id=>10, :name=>"Cleaning"}, {:id=>11, :name=>"Laundry"}, {:id=>12, :name=>"Coding"}]}]
But whats the point when you can just use ActiveRecord and ActiveModel::Serializers::JSON and do this in two lines:
#users = User.includes(:tasks).select(:id, :name)
.as_json(only: [:id, :name], include: { tasks: { only: [:id, :name]} })
Yeah it does two queries ands selects a few more columns on tasks but I doubt there will be any really noticeable performance difference plus its a lot more maintainable. There are better things to spend your time on like tests.

Unique values in Sequelize

I use Sequelize ORM for Node.js. I need to get lines with unique values in certain column. Example table:
id | name | group
-----------------
1 | One | 2
2 | Two | 1
3 | Three| 2
4 | Four | 3
5 | Five | 1
Query for column group and result:
id | name | group
-----------------
1 | One | 2
2 | Two | 1
4 | Four | 3
Lines One, Two and Four was the first who had unique group values. How to make it in Sequelize?
A Sequelize raw query is one way of getting out the rows that you want:
/*
var sql =
SELECT r.id,
r.name,
r.groupnum
FROM s04.row r
JOIN
(SELECT min(id) AS id,
groupnum
FROM s04.row
GROUP BY groupnum) s
ON r.id = s.id
*/
return sq.query(sql, { type: sq.QueryTypes.SELECT});
The resulting promise will resolve to a JSON array:
[
{
"id": 1,
"name": "One",
"groupnum": 2
}
...
]
If you then needed to work with these rows as Instances you can call build on each element of the array:
Model.build({ /* attributes-hash */ }, { isNewRecord: false })
See here for an example. If I find a way of doing this via Sequelize function calls (aggregate, find*, etc) that isn't too hideous I'll also post that here as a separate answer.

Access SQL Max-Function

I have a question concerning MS Access queries involving these tables:
tblMIDProcessMain ={ Process_ID,Process_Title,...}
tblMIDProcessVersion = { ProcessVersion_ID, ProcessVersion_FK_Process, ProcessVersion_VersionNo, ProcessVersion_FK_Status, ...}
tblMIDProcessVersionStatus = { ProcessVersionStatus_ID,ProcessVersionStatus_Value }
The tables store different versions of a process description. The "ProcessVersion_VersionNo" field contains an integer providing the version number. Now I would like to get for each process the highest version number thus the current version. If I do the following it kind of works:
SELECT tblMIDProcessMain.Process_Titel
, Max(tblMIDProcessVersion.ProcessVersion_VersionNo) AS CurrentVersion
FROM tblMIDProcessMain
INNER JOIN tblMIDProcessVersion
ON tblMIDProcessMain.Process_ID = tblMIDProcessVersion.ProcessVersion_FK_Process
GROUP BY tblMIDProcessMain.Process_Titel;
The query returns a recordset with each existing process_title and the respective max number of the version field. But as soon as I add other fields like "ProcessVersion_FK_Status" in the Select statement the query stops working.
Any help would be appreciated. Thanks in advance.
Jon
Edit:
To clarify things a little I added a simplified example
Parent-Table:
Process_ID | Process_Title
----------------------------------
1 | "MyProcess"
2 | "YourProcess"
Child-Table:
Version_ID | Version_FK_ProcessID | Version_No | Version_Status
---------------------------------------------------------------------------
1 | 1 | 1 | "New"
2 | 2 | 1 | "Discarded"
3 | 2 | 2 | "Reviewed"
4 | 2 | 3 | "Released"
Intended Result:
Title | Max_Version_No | Status
--------------------------------------------------------
MyProcess | 1 | "New"
YourProcess | 3 | "Released"
Given the example tables you updated your post with, this should work:
select process_title as Title
, max_version.max_version_no
, c.version_status as status
from (parenttable p
inner join (select max(version_id) as max_version_no, version_fk_process_id from childtable group by version_fk_process_id) max_version
on p.process_id = max_version.version_fk_process_id)
inner join childtable c
on max_version.max_version_no = c.version_id and max_version.version_fk_process_id = c.version_fk_process_id
I assume you are adding the new field to the 'Group By" clause? If not, then you either must include in the 'Group By', or you must use one of the operators like "Max" or "First" etc.

connecting three tables in one query

I have the following tables
mixes
mid | date | info
1 | 2009-07-01 | no info yet
music-review
mid | song | buy
1 | Example - Example | http://example.com
2 | Exam - Exam | http://example.com
tracklist
tid | mid | mrid
1 | 1 | 1
2 | 1 | 2
is it possible to have an SQL query where you can link these all into one?
so my results would turn out like:
date | info | tracklist
2009-07-01 | no info yet | Example - Example http://example.com, Exam - Exam http://example.com
or however this result would be returned... or would this need to be a two sql querys where i get the MID from the mixes and then do a query to get the tracklist from that?
For MySQL:
SELECT mixes.date, mixes.info,
GROUP_CONCAT(DISTINCT music-review.song + ' ' + music-review.buy
ORDER BY music-review.mid ASC SEPARATOR ', ')
FROM mixes
JOIN tracklist ON tracklist.mid = mixes.mid
JOIN music-review ON music-review.mrid = tracklist.mrid
GROUP BY mixes.date, mixes.info
this works as adapted from mherren:
SELECT mixes.`date`, mixes.info,
CONCAT(GROUP_CONCAT(DISTINCT `music-review`.song , ' ' , `music-review`.`mid`
ORDER BY `tracklist`.`tid` ASC SEPARATOR ', ')) as `tracklist`
FROM mixes
JOIN tracklist ON tracklist.`mid` = mixes.`mid`
JOIN `music-review` ON tracklist.`mrid` = `music-review`.`mid`
WHERE `mixes`.`date`='2009-07-01'
GROUP BY mixes.`date`, mixes.info;
it fixes a blurb issue i was getting but, one thing is that group_concat has a max limit set at 1024 this can be altered tho by
SET GLOBAL group_concat_max_len=4096
I left so many comments, I thought it would be more helpful to suggest a revision to your architecture as an answer. However, I do think mherren has already competently addressed your actual concern, so while votes are appreciated, I don't think this should be considered as the the right "answer".
I think you need to reconsider how you have arranged the data. Specifically, you have a table for "music-review" that seems out of place while at the same time you refer to "mixes" and "tracklists" which seems a bit redundant. I imagine you want a one-to-many relationship where "mixes" refers to the information about the mix, like when it was created, the user who created it, etc. While "tracklist" is the list of songs within the "mix". What if you tried this:
#song-info
song_id | artist | title | online-store
1 | The Airheads | Asthma Attack! | example.com/?id=123
2 | The Boners | Bite the Boner | example.com/?id=456
3 | Cats in Heat | Catching a Cold | example.com/?id=789
4 | Dirty Djangos | Dig these Digits | example.com/?id=147
#mixes
mix_id | date | info
1 | 2009-07-01 | no info yet
2 | 2009-07-02 | no info yet
#mix_tracklist
mix_id | song_id
1 | 1
1 | 2
2 | 2
2 | 3
Now you can have a list of available mixes, and if a user selects a mix, another query for the actual songs.
Trying to put all of the song data into one column should only be done if the results require that info right away or if there is a condition within the query itself that is conditional to the results of that sub-query. If you simply want to output a list of mixes with the track list for each one, you are better of doing the query for each mix based on the mix index. So in the case of php outputting HTML, you would go with:
$mixes = mysql_query("SELECT * FROM mixes WHERE date > '$last_week'");
while($mix = mysql_fetch_assoc($mixes)) {
$mix_id = $mix['mix_id'];
$mix_date = date('m/d/Y', strtotime($mix['mix_id']));
$mix_info = $mix['mix_id'];
echo <<<EOT
<h2 class="mix_number">$mix_number</h2>
<p class="mix_date">$mix_date</p>
<p class="mix_info">$mix_info</p>
<h3>Track Listing:</h3>
<ul class="tracklist">
EOT;
$tracks = mysql_query("SELECT song.artist artist,
song.title title,
song.online-store url
song.song_id
FROM song-info song
JOIN mix_tracklist tracks ON (tracks.song_id = song.song_id)
WHERE tracks.mix_id = '$mix_id'
ORDER_BY song_id);
while ($track = mysql_fetch_assoc($tracks)) {
$artist = $track['artist'];
$song_name = $track['title'];
$song_url = $track['url'];
echo <<<EOT
<li>$artist – $song_name</li>
EOT;
}
echo "</ul>";
}