Postgres: Convert Object Graphs to JSON - sql

Is there a procedure that converts Postgres Object Graphs to a nested JSON tree?
Context:
An author has many books.
A book has many review.
The query below gets all reviews of all books of all authors:
select author.id, book.title, review.text
from author
left join book on author.id = book.author_id
left join review on book.id = review.book_id;
With my test data set, this returns:
id title text
1 Oathbringer obath
1 Oathbringer oath
1 The Way of Kings a
1 The Way of Kings bye
1 The Way of Kings hi
2 The Eye of the World w
3 The Fellowship of the Ring x
The tabular format is inconvenient, so I convert it to json:
select json_agg(sub2.*) as data from (
select sub.id, sub.title, json_agg(sub.text) as review from (
select author.id, book.title, review.text
from author
left join book on author.id = book.author_id
left join review on book.id = review.book_id
) as sub group by sub.id, sub.title
) as sub2;
The data is fetched as
[
{
"id":1,
"title":"Oathbringer",
"review":[
"obath",
"oath"
]
},
{
"id":1,
"title":"The Way of Kings",
"review":[
"a",
"bye",
"hi"
]
},
{
"id":2,
"title":"The Eye of the World",
"review":[
"w"
]
},
{
"id":3,
"title":"The Fellowship of the Ring",
"review":[
"x"
]
}
]
Unfortunately, the query is not correct because books for an author are not getting aggregated. Attempting to correct this, I wrote this query:
select sub2.id, json_agg((sub2.title, sub2.review)) as data from (
select sub.id, sub.title, json_agg(sub.text) as review from (
select author.id, book.title, review.text
from author
left join book on author.id = book.author_id
left join review on book.id = review.book_id
) as sub group by sub.id, sub.title
) as sub2 group by sub2.id;
But the json keys are messed up ('f1' instead of 'title', 'f2' instead of 'review').
Questions:
How would the correct query be written?
Can you modify json_agg(sub2.*) to explicitly list the columns to aggregate? The most obvious way, e.g.json_agg((sub2.title, sub2.review)) creates json objects with nonsensical keys 'f1', 'f2', etc.
Is there a better way to convert object graphs to JSON? Correctly nesting queries, and grouping and aggregating fields is tricky to get right. How would you write a program that generates a SQL query returning JSON given an object graph?

Related

SQL query for nested one-to-many relations, as aggregated arrays?

I'm working on a kanban board using Postgres as my database. Normally I would use MongoDB for something like this, but I'm trying to practice and improve my understanding of relational databases.
So I have boards, which can have many lanes, and lanes can have many cards. Boards, lanes, and cards are 3 tables in the database. Lanes have a boardId which is a board.id value, cards have a laneId which is a lane.id value.
I want to query for a board by its id, but also have it include an array of its lanes. Then, I want each of the lanes to include an array of its cards. Like this:
{
id:123
title: 'Board title',
lanes: [{id: 0, title: 'Lane title', cards: [{id: 0, text: 'Card text'}]}, {id: 1, title: 'Lane title', cards: [{id: 1, text: 'Card text'}, {id: 2, text: 'Card text'}]}]
}
I have the first part down with a query that gets a board, then creates an array of lanes. Not sure if this is the 'right' way to do it, but here it is, with an example looking for a board with id '123':
select "boards"."id" as "boardId", "boards"."title" as "boardTitle", ARRAY_AGG(json_build_object('id', lanes.id, 'title', lanes.title)) as lanes from "boards" inner join "lanes" on "boards"."id" = "lanes"."boardId" and "boards"."id" = 123 group by "boards"."id"
But I'm not sure how I would get the cards to be included as a cards array for each element in the lanes array. My guess is that I could add another join like "cards" on "lanes"."id" = "cards"."laneId"... but then I don't know how I would include the cards for each lane in the json_build_object.
It would be best to accept json from the database in order to put rows inside of arrays.
I would build this up using CTEs to make it clear as possible:
with agg_lanes as (
select lane_id as id, jsonb_agg(cards) as "cards"
from cards
group by lane_id
), agg_boards as (
select b.id, b.title, jsonb_agg(a) as "lanes"
from agg_lanes a
join lanes l on l.id = a.id
join boards b on b.id = l.board_id
group by b.id, b.title
)
select to_json(a)
from agg_boards a
;
Working fiddle here.

SQL query to return nested array of objects in JSON for SQLite

I have 2 simple tables in a SQLite db and a nodejs, express api endpoint that should get results by student and have the subjects as a nested array of objects.
Tables:
Student(id, name) and Subject(id, name, studentId)
This is what I need to result to look like:
{
"id": 1,
"name": "Student name",
"subjects":
[{
"id": 1,
"name": "Subject 1"
},
{
"id": 2,
"name": "Subject 2"
}]
}
How can I write a query to get this result?
If your version of sqlite was built with support for the JSON1 extension, it's easy to generate the JSON from the query itself:
SELECT json_object('id', id, 'name', name
, 'subjects'
, (SELECT json_group_array(json_object('id', subj.id, 'name', subj.name))
FROM subject AS subj
WHERE subj.studentid = stu.id)) AS record
FROM student AS stu
WHERE id = 1;
record
---------------------------------------------------------------------------------------------------
{"id":1,"name":"Student Name","subjects":[{"id":1,"name":"Subject 1"},{"id":2,"name":"Subject 2"}]}
It seems that all you need is a LEFT JOIN statement:
SELECT subject.id, subject.name, student.id, student.name
FROM subject
LEFT JOIN student ON subject.studentId = student.id
ORDER BY student.id;
Then just parse the rows of the response into the object structure you require.

complex couchbase query using metadata & group by

I am new to Couchbase and kind a stuck with the following problem.
This query works just fine in the Couchbase Query Editor:
SELECT
p.countryCode,
SUM(c.total) AS total
FROM bucket p
USE KEYS (
SELECT RAW "p::" || ca.token
FROM bucket ca USE INDEX (idx_cr)
WHERE ca._class = 'backend.db.p.ContactsDo'
AND ca.total IS NOT MISSING
AND ca.date IS NOT MISSING
AND ca.token IS NOT MISSING
AND ca.id = 288
ORDER BY ca.total DESC, ca.date ASC
LIMIT 20 OFFSET 0
)
LEFT OUTER JOIN bucket finished_contacts
ON KEYS ["finishedContacts::" || p.token]
GROUP BY p.countryCode ORDER BY total DESC
I get this:
[
{
"countryCode": "en",
"total": 145
},
{
"countryCode": "at",
"total": 133
},
{
"countryCode": "de",
"total": 53
},
{
"countryCode": "fr",
"total": 6
}
]
Now, using this query in a spring-boot application i end up with this error:
Unable to retrieve enough metadata for N1QL to entity mapping, have you selected _ID and _CAS?
adding metadata,
SELECT
meta(p).id AS _ID,
meta(p).cas AS _CAS,
p.countryCode,
SUM(c.total) AS total
FROM bucket p
trying to map it to the following object:
data class CountryIntermediateRankDo(
#Id
#Field
val id: String,
#Field
#NotNull
val countryCode: String,
#Field
#NotNull
val total: Long
)
results in:
Unable to execute query due to the following n1ql errors:
{“msg”:“Expression must be a group key or aggregate: (meta(p).id)“,”code”:4210}
Using Map as return value results in:
org.springframework.data.couchbase.core.CouchbaseQueryExecutionException: Query returning a primitive type are expected to return exactly 1 result, got 0
Clearly i missed something important here in terms of how to write proper Couchbase queries. I am stuck between needing metadata and getting this key/aggregate error that relates to the GROUP BY clause. I'd be very thankful for any help.
When you have a GROUP BY query, everything in the SELECT clause should be either a field used for grouping or a group aggregate. You need to add the new fields into the GROUP by statement, sort of like this:
SELECT
_ID,
_CAS,
p.countryCode,
SUM(p.c.total) AS total
FROM testBucket p
USE KEYS ["foo", "bar"]
LEFT OUTER JOIN testBucket finished_contacts
ON KEYS ["finishedContacts::" || p.token]
GROUP BY p.countryCode, meta(p).id AS _ID, meta(p).cas AS _CAS
ORDER BY total DESC
(I had to make some changes to your query to work with it effectively. You'll need to retrofit the advice to your specific case.)
If you need more detailed advice, let me suggest the N1QL forum https://forums.couchbase.com/c/n1ql . StackOverflow is great for one-and-done questions, but the forum is better for extended interactions.

SQL - Returning combinations of ID from tables if combination not found in another

I have four tables of interest: users, models, questions and labels.
"Labels" contains rows that describes a user's answer to a question on a model. e.g username TEXT, mid INT, qid INT, answer TEXT
I am interested in finding out what model-question pairs the user is still required to provide. A user is asked to provide an answer for every combination of model and questions that appear in their respective tables. So for a given username, I can have rows of model ids and question ids.
Users:
Bacon, XXXX, XXXX, XXXX, ...
Mark, XXXX, XXXX, XXXX, ...
Models:
1, climateOne, XXXX, ....
2, climateTwo, XXXX, ....
3, climateTwo, XXXX, ....
Questions:
1, "Is this a question?"
2, "And another?"
labels:
Bacon, 1, 2, "Yes is was..."
Bacon, 3, 2, "Another what?!"
So the result of asking "what model question pairs has 'Bacon' not completed" would be the return of:
(1,1)(2,1)(2,2)(3,1)
To get all possible combinations of models and questions, use a cross join:
SELECT models.id,
questions.id
FROM models
CROSS JOIN questions
Then filter out those that have been completed:
...
WHERE (models.id, questions.id) NOT IN (SELECT mid, qid
FROM labels);
Thanks go to CL. for the point in the right direction. This code worked for me as I was using python's sqlite3 package. I have included the username specification too. However, I have no clue whether it is efficient or not :)
SELECT m.mid, q.qid
FROM models m CROSS JOIN questions q
WHERE NOT EXISTS(
SELECT 1
FROM labels l
WHERE l.username = ? AND l.mid = m.mid AND l.qid = q.qid
)
Thanks all!

Postgres - Querying table with many to many relationship

My schema is composed like this:
channel -> channels_categories -> category
A channel can have many categories, and categories can belong to many channels.
When i query for channels, i would like to get the channels categories with them.
Anyway if i use JOIN i will simply get the channel duplicated for each category.
[
{
"channel_name": "Channel1",
"category_name": "Category1",
"category_id": "1"
},
{
"channel_name": "Channel1",
"category_name": "Category2"
"category_id": "2"
}
]
The ideal result format (JSON) would be something like:
{
channel_name: 'Channel1',
categories: [{/**category**/}, ....]
}
Is there a way i can achieve this result format just with SQL?
This is completely untested but I've looked at the manual for JSON functions and it looks like the following might work:
select jsonb_build_object(
'channel_name', channel.name,
'categories', jsonb_agg(
jsonb_build_object(
'category_id', category.id,
'category_name', category.name
)
)
)
from channel
join channels_categories on channel.id = channel_id
join category on category.id = category_id
group by channel.id
This assumes that channel has a primary key called id.
I just noticed that you really want the result in the JSON format, this answer is about combining and merging attributes into JSON lists.
You can use string_agg and concat to combine the category_name and category_id to lists like this:
select channel_name,
concat('[', string_agg(category_name, ','), ']') as category_names,
concat('[', string_agg(cast(category.id as text), ','), ']') as category_ids
from
channel inner join channel_category on channel.id = channel_category.channel_id
inner join category on channel_category.category_id = category.id
group by channel_name;
Which produces results like that:
| channel_name | category_names | category_ids |
| channel1 | [category1,category2]| [1,2] |
For a complete JSON result melpomene's answer seems to fit your needs way better than this.