How to query a nested json with varying key value pairs - sql

My previous question has been answered, thanks to #Erwin Brandstetter for the help:
Query individual values in a nested json record
I have a follow-up:
Aurora Postgres - PostgreSQL 13.1. My jsonb column value looks like this:
'{
"usertype": [
{
"type": "staff",
"status": "active",
"permissions": {
"1": "add user",
"2": "add account"
}
},
{
"type": "customer",
"status": "suspended",
"permissions": {
"1": "add",
"2": "edit",
"3": "view",
"4": "all"
}
}
]
}'
I would like to produce a table style output where each permission item i shown as a column. It should show the value if not null else it will be NULL.
type | status | perm1 | perm2 | perm3 | perm4 | perm5 | perm6
----------+-----------+---------+------------+-------+-------+-------+-------
staff | active | adduser | addaccount | null | null | null | null
customer | suspended | add | edit | view | all | null | null
In other words, I would like a way to find out the max permissions count and show that many column in the select query.

An SQL query has to return a fixed number of columns. The return type has to be known at call time (at the latest). Number, names and data types of columns in the returned row(s) are fixed by then. There is no way to get a truly dynamic number of result columns in SQL. You'd have to use two steps (two round trips to the DB server):
Determine the list or result columns.
Send a query to produce that result.
Notably, that leaves a time window for race conditions under concurrent write load.
Typically, it's simpler to just return an array or a list or a document type (like JSON) for a variable number of values. Or a set of rows.
If there is a low, well-known maximum of possible values, say 6, like in your added example, just over-provision:
SELECT id
, js_line_item ->> 'type' AS type
, js_line_item ->> 'status' AS status
, js_line_item #>> '{permissions, 1}' AS perm1
, js_line_item #>> '{permissions, 2}' AS perm2
-- , ...
, js_line_item #>> '{permissions, 6}' AS perm6
FROM newtable n
LEFT JOIN LATERAL jsonb_array_elements(n.column1 -> 'usertype') AS js_line_item ON true;
LEFT JOIN to retain rows without any permissions.

Related

Join Postgresql jsonb object with an item from an array in values

I have two tables in my postgresql database which I want to join:
Table 1) platforms:
ID (INT) | data (JSONB)
------------+---------------------------------------------------------
1 | {"identity": "1", "platformName": "Teradata" }
2 | {"identity": "2", "platformName": "MSSQL" }
Table 2) users:
ID (INT) | data (JSONB)
------------+-----------+---------------
12 | { "role": "developer", "identity": "12", "accessRights": {"platforms": ["1"]} }
13 | { "role": "admin", "identity": "13", "accessRights": {"platforms": ["1", "2"]}" }
I need to get the list of platforms along with the list of users who has access to them. Something like this:
Platform ID | data (JSONB)
------------+-----------+---------------
1 | [{"role": "developer", "identity": "12"}]
2 | [{"role": "developer", "identity": "12"}, {"role": "admin", "identity": "13"}]
I thought maybe something like this can help:
SELECT p.id, u.id, u.role
FROM users u
INNER JOIN platforms p ON (u.data->>'accessRights'->'platforms')::text::int = p.id
GROUP BY p.id
But I can't make it work. So is there anyway to get the result I need?
A simple join is not enough because you need to aggregate information from the users.data JSON for each platform.
select p.id as platform_id, u.*
from platforms p
cross join lateral (
select jsonb_agg(u.data - 'accessRights') as data
from users u
where u.data -> 'accessRights' -> 'platforms' ? p.id::text
) as u
Note this only works because you stored the platform IDs in the array as strings, not as (JSON) integers, because the ? operator only works on strings.
Online example

Left join add matched rows in child table to an array in parent row (as JSON format)

I have the following two tables:
+----------------------------+
| Parent Table |
+----------------------------+
| uuid (PK) |
| caseId |
| param |
+----------------------------+
+----------------------------+
| Child Table |
+----------------------------+
| uuid (PK) |
| parentUuid (FK) |
+----------------------------+
My goal is to do a (left?) join and get all matching rows in the child table based on the FK in an array on the parent row and not inside the parent row itself on matching column names (see further down on desired output).
Examples of values in tables:
Parent table:
1. uuid: "10dd617-083-e5b5-044b-d427de84651", caseId: 1, param: "test1"
2. uuid: "5481da7-8b7-22db-d326-b6a0a858ae2f", caseId: 1, param: "test1"
3. uuid: "857dec3-aa3-1141-b8bf-d3a8a3ad28a7", caseId: 2, param: "test1"
Child table:
1. uuid: 7eafab9f-5265-4ba6-bb69-90300149a87d, parentUuid: 10dd617-083-e5b5-044b-d427de84651
2. uuid: f1afb366-2a6b-4cfc-917b-0794af7ade85, parentUuid: 10dd617-083-e5b5-044b-d427de84651
What my desired output should look like:
Something like this query (with pseudo-ish SQL code):
SELECT *
FROM Parent_table
WHERE caseId = '1'
LEFT JOIN Child_table ON Child_table.parentUuid = Parent_table.uuid
Desired output (in JSON)
[
{
"uuid": "10dd617-083-e5b5-044b-d427de84651",
"caseId": "1",
// DESIRED FORMAT HERE
"childRows": [
{
"uuid": "7eafab9f-5265-4ba6-bb69-90300149a87d",
"parentUuid": "10dd617-083-e5b5-044b-d427de84651"
},
{
"uuid": "f1afb366-2a6b-4cfc-917b-0794af7ade85",
"parentUuid": "10dd617-083-e5b5-044b-d427de84651"
}
]
},
{
"uuid": "5481da7-8b7-22db-d326-b6a0a858ae2f",
"caseId": "1"
}
]
You can use nested FOR JSON clauses to achieve this.
SELECT
p.uuid,
p.caseId,
childRows = (
SELECT
c.uuid,
c.parentUuid
FROM Child_table c
WHERE c.parentUuid = p.uuid
FOR JSON PATH
)
FROM Parent_table p
WHERE p.caseId = '1'
FOR JSON PATH;
SQL does not support rows inside rows as you actually want, instead you have to return the entire result set (either as a join or 2 separate datasets) from SQL server then create the objects in your backend. If you are using .net and EF/Linq this is as simple as getting all the parents with an include to also get the children. Other backends will do this in other ways.

PostgreSQL: Sorting the rows based on value of a JSON in an array of JSON

A table says products have a JSONB column called identifiers that stores an array of JSON objects.
Sample data in products
id | name | identifiers
-----|-------------|---------------------------------------------------------------------------------------------------------------
1 | umbrella | [{"id": "productID-umbrella-123", "domain": "ecommerce.com"}, {"id": "amzn-123", "domain": "amzn.com"}]
2 | ball | [{"id": "amzn-234", "domain": "amzn.com"}]
3 | bat | [{"id": "productID-bat-234", "domain": "ecommerce.com"}]
Now, I have to write a query that sorts the elements in the table based on the "id" value for the domain "amzn.com"
Expected result
id | name | identifiers
----- |--------------|---------------------------------------------------------------------------------------------------------------
3 | bat | [{"id": "productID-bat-234", "domain": "ecommerce.com"}]
1 | umbrella | [{"id": "productID-umbrella-123", "domain": "ecommerce.com"}, {"id": "amzn-123", "domain": "amzn.com"}]
2 | ball | [{"id": "amzn-234", "domain": "amzn.com"}]
ids of amzn.com are "amzn-123" and "amzn-234".
When sorted by ids of amzn.com "amzn-123" appears first, followed by "amzn-234"
Ordering the table by values of "id" for the domain "amzn.com",
record with id 3 appears first since the id for amzn.com is NULL,
followed by a record with id 1 and 2, which has a valid id that is sorted.
I am genuinely clueless as to how I could write a query for this use case.
If it were a JSONB and not an array of JSON I would have tried.
Is it possible to write a query for such a use case in PostgreSQL?
If yes, please at least give me a pseudo code or the rough query.
As you don't know the position in the array, you will need to iterate over all array elements to find the amazon ID.
Once you have the ID, you can use it with an order by. Using nulls first puts those products at the top that don't have an amazon ID.
select p.*, a.amazon_id
from products p
left join lateral (
select item ->> 'id' as amazon_id
from jsonb_array_elements(p.identifiers) as x(item)
where x.item ->> 'domain' = 'amzn.com'
limit 1 --<< safe guard in case there is more than one amazon id
) a on true --<< we don't really need a join condition
order by a.amazon_id nulls first;
Online example
With Postgres 12 this would be a bit shorter:
select p.*
from products p
order by jsonb_path_query_first(identifiers, '$[*] ? (#.domain == "amzn.com").id') nulls first
After few tweaks, this is the query that finally made it,
select p.*, amzn -> 'id' AS amzn_id
from products p left join lateral JSONB_ARRAY_ELEMENTS(p.identifiers) amzn ON amzn->>'domain' = 'amzn.com'
order by amzn_id nulls first

Postgres get multiple rows into a single json object

I have a users table with columns like id, name, email, etc. I want to retrieve information of some users in the following format in a single json object:
{
"userId_1" : {"name" : "A", "email": "A#gmail.com"},
"userId_2" : {"name" : "B", "email": "B#gmail.com"}
}
Wherein the users unique id is the key and a json containing his information is its corresponding value.
I am able to get this information in two separate rows using json_build_object but I would want it get it in a single row in the form of one single json object.
You can use json aggregation functions:
select jsonb_object_agg(id, to_jsonb(t) - 'id') res
from mytable t
jsonb_object_agg() aggregates key/value pairs into a single object. The key is the id of each row, and the values is a jsonb object made of all columns of the table except id.
Demo on DB Fiddle
Sample data:
id | name | email
:------- | :--- | :----------
userid_1 | A | A#gmail.com
userid_2 | B | B#gmail.com
Results:
| res |
| :----------------------------------------------------------------------------------------------------- |
| {"userid_1": {"name": "A", "email": "A#gmail.com"}, "userid_2": {"name": "B", "email": "B#gmail.com"}} |
try -
select row_to_json(col) from T
link below might help https://hashrocket.com/blog/posts/faster-json-generation-with-postgresql
Try this:
SELECT json_object(array_agg(id), array_agg(json::text)) FROM (
SELECT id, json_build_object('name', name, 'email', email) as json
FROM users_table
) some_alias_name
If your id is not of text type then you have to cast it to text too.

JOOQ: Result.getValues() returns list with null entry

I query a many-to-many relationship using JOOQ and need to map the multiple results for one entry into a single object. To do so I fetch the results into groups using a unique identifier and extract the needed data from each grouped result. For the many-to-many data, I get all available values, for normal data I only use the data of the first entry:
Map<String, Result<Record2<String, String>> groupedResults = create.select(LOC.NAME, GROUP.GROUP_)
.from(LOC)
.leftOuterJoin(LOC2GROUP)
.on(Tables.LOC2GROUP.LOC_ID.eq(LOC.LOC_ID))
.leftOuterJoin(GROUP)
.on(LOC2GROUP.GROUP_ID.eq(GROUP.GROUP_ID))
.fetch().intoGroups(LOC.NAME);
Collection<Loc> ret = new ArrayList<Loc>();
for (Result<Record2<String, String>> result : groupedResults.values()) {
Loc loc = new Loc(result.getValue(0, LOCATION.NAME), result.getValues(Tables.GROUP.GROUP_));
ret.add(loc);
}
Now, I while each entry can have multiple groups, it does not need to have any, the following would be a valid entry simply without any groups set:
|name |group |
|simple|{null}|
Oddly, I noticed that Result.getValues() returns a List that contains null in such a situation, rather than an empty list.
Is this intended and, if so, is there a better workaround than removing null entries after fetching?
You're using OUTER JOIN in your query. This means that your result set may indeed containt values from the LOC table, but no corresponding values from the GROUP table. The following is some valid example output from your query:
+------+--------+
| name | group |
+------+--------+
| a | x |
| a | y |
| b | x |
| c | {null} | <-- there's a value in LOC, but no corresponding value in GROUP
+------+--------+
Now, when you call jOOQ's intoGroups() on that result, jOOQ simply sees 3 groups: a, b, and c, which gives you the following Map (using JSON as pseudo-notation):
{
"a" : [ { "name" : "a", "group" : "x" },
{ "name" : "a", "group" : "y" } ],
"b" : [ { "name" : "b", "group" : "x" } ],
"c" : [ { "name" : "c", "group" : null } ]
}
Just because there happens to be a null value in one of the columns doesn't mean that this null value should have any semantics when grouping. The fact that this one column (group) is the only column you're interested in doesn't change the fact that the grouping results are records, not single values.
I hope this makes sense?
jOOQ currently doesn't support a method that returns the data structure that you're really looking for:
{
"a" : [ "x", "y" ],
"b" : [ "x" ],
"c" : [ ]
}
But it should be rather easy to write such a utility method yourself...
PostgreSQL or HSQLDB solution:
Or, if you're using PostgreSQL or HSQLDB, you can use array_agg() instead:
Result<Record2<String, String[]>> result =
create.select(LOC.NAME, arrayAgg(GROUP.GROUP_))
.from(LOC)
.leftOuterJoin(LOC2GROUP)
.on(Tables.LOC2GROUP.LOC_ID.eq(LOC.LOC_ID))
.leftOuterJoin(GROUP)
.on(LOC2GROUP.GROUP_ID.eq(GROUP.GROUP_ID))
.groupBy(LOC.NAME)
.fetch();
DSL.arrayAgg() will be part of jOOQ 3.5, but you can create that jOOQ function yourself:
#Support({SQLDialect.POSTGRES})
protected static <T> Field<T[]> arrayAgg(Field<T> field) {
return DSL.field("array_agg({0})", field.getDataType().getArrayDataType(), field);
}