I am new to SQL previously we were using MongoDB and now we have shifted to Postgres. basically, we have 2 tables of Questions and Options. Each question can have multiple option so I design the database just like below table.
Table : dialogues
dialogue_id
dialogue_text
...
Table : options
option_id
option_name
dialogue_id ( which is FK for table question)
Now I am trying to get all the questions with their options. I have tried inner join like this
SELECT options.option_name, dialogues.*
FROM options INNER JOIN dialogues ON dialogues.dialogue_id = options.dialogue_id)
But it's not what I wanted.
Just to give you an example in Mongo we have used this query
const aggregateQuery = [
{
$lookup: {
from: "options", // The collection you want to join to
localField: "_id", // The field in orders you want to join on
foreignField: "question_id", // The field in products you want to join on
as: "options" // What you want to call the array that stores the joined documents from products
}
},
}
]
const questionList = await questionModel.aggregate(aggregateQuery)
In result I wanted all the dialogue along with field called "options" which contains all the relevant options from table "options".
let me share the final JSON that I get from mongo.
[
{
"_id": "yyyyy",
"question": "What is your gender?",
"options": [
{
"_id": "xxxxx",
"option": "Male",
"question_id": "yyyyy",
"created_at": "2020-07-04T05:57:00.293Z",
"updated_at": "2020-07-04T05:57:00.293Z"
},
{
"_id": "xxxx",
"option": "Female",
"question_id": "yyyyy",
"created_at": "2020-07-04T05:57:00.293Z",
"updated_at": "2020-07-04T05:57:00.293Z"
}
]
}
]
can anybody help ?
This may be your solution.
select jsonb_agg(to_jsonb(t)) from
(
select
d.dialogue_id "_id",
d.dialogue_text "question",
(
select
jsonb_agg(jsonb_build_object('_id', o.option_id, 'option', o.option_name))
from options o
where o.dialogue_id = d.dialogue_id
) "options"
from dialogues d
) t;
Here is a JOIN/GROUP BY version
select jsonb_agg(to_jsonb(t)) from
(
select
d.dialogue_id "_id",
d.dialogue_text "question",
jsonb_agg(jsonb_build_object('_id', o.option_id, 'option', o.option_name)) "options"
from dialogues d inner join options o using (option_id)
group by d.dialogue_id, d.dialogue_text
) t;
Related
I have 3 tables:
"tags"
id serial
name text
"questions"
id serial
content jsonb
"questions_tags" (many-to-many)
question_id integer
tag_id integer
I can select all data from questions:
SELECT * FROM questions;
I can select tags of specific question:
SELECT t.*
FROM tags t
JOIN questions_tags qt ON t.id = qt.tag_id
JOIN questions q ON qt.question_id = q.id
WHERE q.id = 157;
I need to select all questions and add to this selection tags for every question.
I'm using nodejs and node-postgres and I need to get something like:
[
{
id: 1,
content: { someProp: 'someValue' },
tags: [
{ id: 58, name: 'sometag58' },
{ id: 216, name: 'sometag216' }
]
},
...
]
How can I do this without ORM, only with the help of vanilla SQL?
I would let PostgreSQL handle it for me:
with tagjson as (
select q.id, q.content, jsonb_agg(to_jsonb(tags)) as tags
from questions q
left join questions_tags qt on qt.question_id = q.id
left join tags on tags.id = qt.tag_id
group by q.id, q.content
)
select jsonb_pretty(jsonb_agg(to_jsonb(tagjson))) as result
from tagjson;
Result:
[
{
"id": 1,
"tags": [
{
"id": 58,
"name": "sometag58"
},
{
"id": 216,
"name": "sometag216"
}
],
"content": "some question 1 content"
}
]
db<>fiddle here
EDIT TO ADD:
Based on a conversation in the comments, the final query should be modified to:
select * from tagjson;
Or the CTE could be promoted to be a stand-alone query.
This allowed the conventional node syntax while preserving the aggregated tags array under the tags key:
const questions: Question[] = result.rows;
I have a Postgres DB that stores data in a table that has the following columns "date | uid | id | value | data".
Using knex on a node.js server I currently get from this query:
data = await db(tableName).select(["date", "uid", "id", "value", "value as market_cap", "data"]).whereIn("id", ["index", "market_cap", "market"]);
the following result:
"data": [
{
"date": "2020-11-07T21:43:11.709Z",
"uid": "nmvdqy0kh87sd8a",
"id": "index",
"value": "999.9999999999999",
"market_cap": "999.9999999999999",
"data": null
},
{
"date": "2020-11-07T21:43:11.709Z",
"uid": "nmvdqy0kh87sd8b",
"id": "market_cap",
"value": "10125616413",
"market_cap": "10125616413",
"data": null
},
{
"date": "2020-11-07T21:43:11.709Z",
"uid": "nmvdqy0kh87sd8c",
"id": "market",
"value": null,
"market_cap": null,
"data": {
"1": [],
"2": []
}
},
...
];
The date pairs are all exactly the same. Data stored under id "market_cap" is actually stored as "value" and data stored under id "market" is actually stored as "data".
Now, what I actually need is:
"data": [
{
"date": "2020-11-07T21:43:11.709Z",
"value": "999.9999999999999",
"market_cap": "10125616413",
"data": {
"1": [],
"2": []
}
},
...
];
Is there a way to obtain this data structure directly from the database instead of transforming the data on the server? Bonus points it you provide the knex query / SQL query. Thank you!
You can accomplish this with a self-join on date and selecting on the rows with certain IDs. Left join ensures a result for each date even if they're missing a data type.
select mv."date", mv.value, mc.value as market_cap, md.data
from market_snapshots mv
left join market_snapshots mc on mv."date" = mc."date" and mc.id = 'market_cap'
left join market_snapshots md on mv."date" = md."date" and md.id = 'market'
where mv.id = 'index';
Try it.
In Knex it would be something like...
knex.select(['mv.date', 'mv.value', 'mc.value as market_cap', 'md.data'])
.from({ mv: 'market_snapshots' })
.leftJoin({ mc: 'market_snapshots' }, function() {
this.on('mv.date', '=', 'mc.date').andOn(knex.raw('mc.id = ?', 'market_cap'))
})
.leftJoin({ md: 'market_snapshots' }, function() {
this.on('mv.date', '=', 'md.date').andOn(knex.raw('md.id = ?', 'market'))
})
.where('mv.id', 'index')
I am constructing an interface between a PostgreSQL system and a SQL Server system and am attempting to "flatten" the structure of the JSON data to facilitate this. I'm very experienced in SQL Server but I'm new to both PostgreSQL and JSON.
The JSON contains essentially two types of structure: those of type "text" or "textarea" where the value I want is in an object named value (the first two cases below) and those of type "select" where the value object points to an id object in a lower-level options array (the third case below).
{
"baseGroupId": {
"fields": [
{
"id": "1f53",
"name": "Location",
"type": "text",
"options": [],
"value": "Over the rainbow"
},
{
"id": "b547",
"name": "Description",
"type": "textarea",
"options": [],
"value": "A place of wonderful discovery"
},
{
"id": "c12f",
"name": "Assessment",
"type": "select",
"options": [
{
"id": "e5fd",
"name": "0"
},
{
"id": "e970",
"name": "1"
},
{
"id": "0ff4",
"name": "2"
},
{
"id": "2db3",
"name": "3"
},
{
"id": "241f",
"name": "4"
},
{
"id": "3f52",
"name": "5"
}
],
"value": "241f"
}
]
}
}
Those with a sharp eye will see that the value of the last value object "241f" can also be seen within the options array against one of the id objects. When nested like this I need to extract the value of the corresponding name, in this case "4".
The JSON-formatted information is in table customfield field textvalue. It's datatype is text but I'm coercing it to json. I was originally getting array set errors when trying to apply the criteria in a WHERE clause and then I read about using a LATERAL subquery instead. It now runs but returns all the options, not just the one matching the value.
I'm afraid I couldn't get an SQL Fiddle working to reproduce my results, but I would really appreciate an examination of my query to see if the problem can be spotted.
with cte_custombundledfields as
(
select
textvalue
, cfname
, json_array_elements(textvalue::json -> 'baseGroupId'->'fields') ->> 'name' as name
, json_array_elements(textvalue::json -> 'baseGroupId'->'fields') ->> 'value' as value
, json_array_elements(textvalue::json -> 'baseGroupId'->'fields') ->> 'type' as type
from
customfield
)
, cte_custombundledfieldsoptions as
(
select *
, json_array_elements(json_array_elements(textvalue::json -> 'baseGroupId'->'fields') -> 'options') ->> 'name' as value2
from
cte_custombundledfields x
, LATERAL json_array_elements(x.textvalue::json -> 'baseGroupId'->'fields') y
, LATERAL json_array_elements(y -> 'options') z
where
type = 'select'
and z ->> 'id' = x.value
)
select *
from
cte_custombundledfieldsoptions
I posted a much-simplified rewrite of this question which was answered by Bergi.
How do I query a string from JSON based on another string within the JSON in PostgreSQL?
Each row in my table has a field that is an array, and I'd like to get a field from the first array entry.
For example, if my row is
[
{
"user_dim": {
"user_id": "123",
"user_properties": [
{
"key": "content_group",
"value": {
"value": {
"string_value": "my_group"
}
}
}
]
},
"event_dim": [
{
"name": "main_menu_item_selected",
"timestamp_micros": "1517584420597000"
},
{
"name": "screen_view",
"timestamp_micros": "1517584420679001"
}
]
}
]
I'd like to get
user_id: 123, content_group: my_group, timestamp_1517584420597000
As Elliott mentioned - BigQuery Standard SQL has way much better support for ARRAYs than legacy SQL. And in general, BigQuery team recommend using Standard SQL
So, below is for BigQuery Standard SQL (including handling wildcard stuff)
#standardSQL
SELECT
user_dim.user_id AS user_id,
(SELECT value.value.string_value
FROM UNNEST(user_dim.user_properties)
WHERE key = 'content_group' LIMIT 1
) content_group,
(SELECT event.timestamp_micros
FROM UNNEST(event_dim) event
WHERE name = 'main_menu_item_selected'
) ts
FROM `project.dataset.app_events_*`
WHERE _TABLE_SUFFIX BETWEEN '20180129' AND '20180202'
with result (for the dummy example from your question)
Row user_id content_group ts
1 123 my_group 1517584420597000
Following is the query:
SELECT [C].[Id],
[C].[Name],
[Module].[Id],
[Module].[Name]
FROM [dbo].[Category] [C]
INNER JOIN [CategoryModule] [CM]
ON [C].[Id] = [CM].[CategoryId]
CROSS APPLY (SELECT [M].[Id], [M].[Name] FROM [Module] [M]
WHERE [M].[Id] = [CM].[ModuleId]) [Module]
WHERE [C].[Id]IN (1,2)
FOR JSON AUTO
Output:
[
{"Id":1,"Name":"Book","Module":[{"Id":1,"Name":"Unit"}]},
{"Id":2,"Name":"Business","Module":[{"Id":1,"Name":"Unit"}]},
{"Id":1,"Name":"Book","Module":[{"Id":2,"Name":"App"}]}
]
Need:
[
{"Id":1,"Name":"Book","Module":[{"Id":1,"Name":"Unit"},{"Id":2,"Name":"App"}]},
{"Id":2,"Name":"Business","Module":[{"Id":1,"Name":"Unit"}]}
]
It generate separate object for same master table entry.
Hi i think it's possible use FOR JSON AUTO, OPENJSON and JSON_MODIFY
I had the same problem but i finally find a solution
In your case try this
SELECT
JSON_MODIFY (
(SELECT value FROM OPENJSON((SELECT * FROM Category WHERE Id = C.Id FOR JSON AUTO),'$')),
'$.Module',
(SELECT M.Id, M.Name FROM CategoryModule CM INNER JOIN Module M ON CM.ModuleId= M.Id WHERE CM.CategoryId= C.Id FOR JSON AUTO)
) AS Category
FROM Category C
FOR JSON AUTO
The Ouput is
[{
"Category": {
"Id": 1,
"Name": "Book",
"Module": [{
"Id": 1,
"Name": "Unit"
}, {
"Id": 2,
"Name": "App"
}]
}
}, {
"Category": {
"Id": 2,
"Name": "Business",
"Module": [{
"Id": 1,
"Name": "Unit"
}]
}
}]