I have the following structure
task -> hasMany links -> hasOne user
what I would like to see returned is
task.id | [
{ linkType: 'foo', userid:1, taskid:1,
user : { name: 'jon', lastname: 'smith'},
... ]
being a sql noob, I have managed to get so far as this
select task.id, json_agg(links.*) as links from task
left outer join links on task.id = links.taskid
join "user" on "user".id = links.userid
group by task.id ;
which gives me
task.id | [{ linkType: 'foo', userid:1, taskid:1}, ... ]
but obviously missing the user
I'm kinda stuck now on how to add the user property to each of the link array items
I've read several documents but they always seem to stop at the first join
the schema design is
create table task (id int GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY)
create table "user" (id int GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, username text)
create table links (taskid integer references task (id), userid integer references "user" (id) );
Your example does not reference the table structure you provided, because "user" has a name and a lastname column in the former and a single username column in the latter, but you did provide enough information to give an answer. I am using the two-column user, because it allows to show how to construct nested JSON objects.
SELECT task.id,
json_agg(
json_build_object(
'linkType', 'foo',
'taskid', links.taskid,
'userid', links.userid,
'user', json_build_object('name', u.name, 'lastname', u.lastname)
)
) AS links
FROM task
LEFT OUTER JOIN links ON task.id = links.taskid
JOIN "user" u ON u.id = links.userid
GROUP BY task.id;
The general problem of building json objects with the appropriate key values is discussed in this excellent DBA.SE answer. I have adopted what is listed there as solution number 3 because I think it is the most flexible and the most readable. Your tastes might differ.
Related
I'm quite new to SQL and have two Postgresql tables :
CREATE TABLE project (
id uuid DEFAULT uuid_generate_v4 (),
name VARCHAR(100) NOT NULL,
creator_id uuid NOT NULL
);
CREATE TABLE task (
id uuid DEFAULT uuid_generate_v4 (),
name VARCHAR(100) NOT NULL,
project_id uuid NOT NULL,
);
I'm running a pretty simple join on it :
SELECT project.*, task.name as task_name
FROM project
INNER JOIN task ON task.project_id = $1
WHERE project.id =
Result is :
[
{
id: '5936d843-aca0-4453-ad24-a7b3a6b90393',
name: 'Test project',
creator_id: '2e0e73af-e824-46a2-89ee-c08cf9c5de7a',
task_name: 'Test task'
},
{
id: '5936d843-aca0-4453-ad24-a7b3a6b90393',
name: 'Test project',
creator_id: '2e0e73af-e824-46a2-89ee-c08cf9c5de7a',
task_name: 'Test task 2'
}
]
My question is, is that possible to merge those rows on id to have a result looking more like this :
[
{
id: '5936d843-aca0-4453-ad24-a7b3a6b90393',
name: 'Test project',
creator_id: '2e0e73af-e824-46a2-89ee-c08cf9c5de7a',
tasks: [
{
task_name: 'Test task'
},
{
task_name: 'Test task 2'
}
}
]
I know there is a few thing that can help me achieve that, like using COALESCE, json_build_object or json_agg. But this makes me build "complex" queries for something that looks pretty simple, so do you know if there's a simpler way to do this, or should I just take the first result and process it with my language of choice (here javascript) to merge as needed ?
You need to group by project and aggregate project tasks.
SELECT p.*,
jsonb_agg(jsonb_build_object('task_name', t.name)) tasks
FROM project p
INNER JOIN task t ON t.project_id = p.id
WHERE p.id = $1 -- or whatever your selection condition is
group by p.id, p.name, p.creator_id;
If project.id is primary key then
group by p.id, p.name, p.creator_id
can be simplified as
group by p.id;
I assume that the expected JSON array result is shaped in the logic tier or by something like an ORM. If you wish that (btw much better) the query itself returns the result as a JSON array, then
select jsonb_agg(to_josnb(t.*)) from
(
... the query above ...
) t;
Here are three tables
Table users {
id uuid [pk, default: `gen_random_uuid()`]
auth_id uuid [ref: - auths.id]
org_id uuid [ref: - orgs.id]
access_group text [default: 'DEFAULT']
created_at int [default: `now()::int`]
updated_at int
age int
status text
}
Table op_user_access_groups {
op_id uuid [pk, ref: > ops.id]
access_group text [pk, default: 'DEFAULT']
}
Table op_users {
op_id uuid [pk, ref: > ops.id]
user_id uuid [pk, ref: > users.id]
access boolean [default: false]
}
The table users has user info and he/she belongs to certain organization (org_id)
The table op_user_access_groups has the information regarding an operator having access to what all access_groups. The op_id belongs to the org_id
The table op_users has information about users (user_id) that can be accessed by op_id irrespective of which group a user belongs to.
I want to create a view such that if I do
select * from <that view> where op_id = ?
I should get the users the operator has access to.
Any help is appreciated :)
You should use a JOIN query to create such a view. Something like this:
CREATE VIEW v AS
SELECT * FROM users
JOIN op_users ON users.id=op_users.op_id
JOIN op_user_access_groups ON op_user_access_groups.access_group=users.access_group
WHERE op_users.access = true
It's not exactly clear how your data model works from your question, but creating a view with the JOIN that answers your question is the correct response here.
Question
When dealing with a one-to-many or many-to-many SQL relationship in Golang, what is the best (efficient, recommended, "Go-like") way of mapping the rows to a struct?
Taking the example setup below I have tried to detail some approaches with Pros and Cons of each but was wondering what the community recommends.
Requirements
Works with PostgreSQL (can be generic but not include MySQL/Oracle specific features)
Efficiency - No brute forcing every combination
No ORM - Ideally using only database/sql and jmoiron/sqlx
Example
For sake of clarity I have removed error handling
Models
type Tag struct {
ID int
Name string
}
type Item struct {
ID int
Tags []Tag
}
Database
CREATE TABLE item (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY
);
CREATE TABLE tag (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
name VARCHAR(160),
item_id INT REFERENCES item(id)
);
Approach 1 - Select all Items, then select tags per item
var items []Item
sqlxdb.Select(&items, "SELECT * FROM item")
for i, item := range items {
var tags []Tag
sqlxdb.Select(&tags, "SELECT * FROM tag WHERE item_id = $1", item.ID)
items[i].Tags = tags
}
Pros
Simple
Easy to understand
Cons
Inefficient with the number of database queries increasing proportional with number of items
Approach 2 - Construct SQL join and loop through rows manually
var itemTags = make(map[int][]Tag)
var items = []Item{}
rows, _ := sqlxdb.Queryx("SELECT i.id, t.id, t.name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
for rows.Next() {
var (
itemID int
tagID int
tagName string
)
rows.Scan(&itemID, &tagID, &tagName)
if tags, ok := itemTags[itemID]; ok {
itemTags[itemID] = append(tags, Tag{ID: tagID, Name: tagName,})
} else {
itemTags[itemID] = []Tag{Tag{ID: tagID, Name: tagName,}}
}
}
for itemID, tags := range itemTags {
items = append(Item{
ID: itemID,
Tags: tags,
})
}
Pros
A single database call and cursor that can be looped through without eating too much memory
Cons
Complicated and harder to develop with multiple joins and many attributes on the struct
Not too performant; more memory usage and processing time vs. more network calls
Failed approach 3 - sqlx struct scanning
Despite failing I want to include this approach as I find it to be my current aim of efficiency paired with development simplicity. My hope was by explicitly setting the db tag on each struct field sqlx could do some advanced struct scanning
var items []Item
sqlxdb.Select(&items, "SELECT i.id AS item_id, t.id AS tag_id, t.name AS tag_name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
Unfortunately this errors out as missing destination name tag_id in *[]Item leading me to believe the StructScan is not advanced enough to recursively loop through rows (no criticism - it is a complicated scenario)
Possible approach 4 - PostgreSQL array aggregators and GROUP BY
While I am sure this will not work I have included this untested option to see if it could be improved upon so it may work.
var items = []Item{}
sqlxdb.Select(&items, "SELECT i.id as item_id, array_agg(t.*) as tags FROM item AS i JOIN tag AS t ON t.item_id = i.id GROUP BY i.id")
When I have some time I will try and run some experiments here.
the sql in postgres :
create schema temp;
set search_path = temp;
create table item
(
id INT generated by default as identity primary key
);
create table tag
(
id INT generated by default as identity primary key,
name VARCHAR(160),
item_id INT references item (id)
);
create view item_tags as
select id,
(
select
array_to_json(array_agg(row_to_json(taglist.*))) as array_to_json
from (
select tag.name, tag.id
from tag
where item_id = item.id
) taglist ) as tags
from item ;
-- golang query this maybe
select row_to_json(row)
from (
select * from item_tags
) row;
then golang query this sql:
select row_to_json(row)
from (
select * from item_tags
) row;
and unmarshall to go struct:
pro:
postgres manage the relation of data. add / update data with sql functions.
golang manage business model and logic.
it's easy way.
.
I can suggest another approach which I have used before.
You make a json of the tags in this case in the query and return it.
Pros: You have 1 call to the db, which aggregates the data, and all you have to do is parse the json into an array.
Cons: It's a bit ugly. Feel free to bash me for it.
type jointItem struct {
Item
ParsedTags string
Tags []Tag `gorm:"-"`
}
var jointItems []*jointItem
db.Raw(`SELECT
items.*,
(SELECT CONCAT(
'[',
GROUP_CONCAT(
JSON_OBJECT('id', id,
'name', name
)
),
']'
)) as parsed_tags
FROM items`).Scan(&jointItems)
for _, o := range jointItems {
var tempTags []Tag
if err := json.Unmarshall(o.ParsedTags, &tempTags) ; err != nil {
// do something
}
o.Tags = tempTags
}
Edit: code might behave weirdly so I find it better to use a temporary tags array when moving instead of using the same struct.
You can use carta.Map() from https://github.com/jackskj/carta
It tracks has-many relationships automatically.
Let's say I create two tables using the following SQL,
such that post has many comment:
CREATE TABLE IF NOT EXISTS post (
id SERIAL PRIMARY KEY,
title VARCHAR NOT NULL,
text VARCHAR NOT NULL
)
CREATE TABLE IF NOT EXISTS comment (
id SERIAL PRIMARY KEY,
text VARCHAR NOT NULL,
post_id SERIAL REFERENCES post (id)
)
I would like to be able to query these tables so as to serve a response that
looks like this:
{
"post" : [
{ id: 100,
title: "foo",
text: "foo foo",
comment: [1000,1001,1002] },
{ id: 101,
title: "bar",
text: "bar bar",
comment: [1003] }
],
"comment": [
{ id: 1000,
text: "bla blah foo",
post: 100 },
{ id: 1001,
text: "bla foo foo",
post: 100 },
{ id: 1002,
text: "foo foo foo",
post: 100 },
{ id: 1003,
text: "bla blah bar",
post: 101 },
]
}
Doing this naively would involve to SELECT statements,
the first along the lines of
SELECT DISTINCT ON(post.id), post.title, post.text, comment.id
FROM post, comment
WHERE post.id = comment.post_id
... and the second something along the lines of
SELECT DISTINCT ON(comment.id), comment.text, post.id
FROM post, comment
WHERE post.id = comment.post_id
However, I cannot help but think that there is a way to do this involving
only one SELECT statement - is this possible?
Notes:
I am using Postgres, but I do not require a Postgres-specific solution. Any standard SQL solution should do.
The queries above are illustrative only, they do not give we exactly what is necessary at the moment.
It looks like what the naive solution here does is perform the same join on the same two tables, just doing a distinct on a different table each time. This definitely leaves room for improvement.
It appears that ActiveModel Serializers in Rails already do this - if someone familair with them would like to chime in how they work under the hood, that would be great.
You need two queries to get the form you laid out:
SELECT p.id, p.title, p.text, array_agg(c.id) AS comments
FROM post p
JOIN comment c ON c.post_id = p.id
WHERE p.id = ???
GROUP BY p.id;
Or faster, if you really want to retrieve all or most of your posts:
SELECT p.id, p.title, p.text, c.comments
FROM post p
JOIN (
SELECT post_id, array_agg(c.id) AS comments
FROM comment
GROUP BY 1
) c ON c.post_id = p.id
GROUP BY 1;
Plus:
SELECT id, text, post_id
FROM comment
WHERE post_id = ??;
Single query
SQL can only send one result type per query. For a single query, you would have to combine both tables, listing columns for post redundantly. That conflicts with the desired response in your question. You have to give up one of the two conflicting requirements.
SELECT p.id, p.title, p.text AS p_text, c.id, c.text AS c_text
FROM post p
JOIN comment c ON c.post_id = p.id
WHERE p.id = ???
Aside: The column comment.post_id should be integer, not serial! Also, column names are probably just for a quick show case. You wouldn't use the non-descriptive text as column name, which also conflicts with a basic data type.
Compare this related case:
Foreign key of serial type - ensure always populated manually
However, I cannot help but think that there is a way to do this involving only one SELECT statement - is this possible?
Technically: yes. If you really want your data in json anyway, you could use PostgreSQL (9.2+) to generate it with the json functions, like:
SELECT row_to_json(sq)
FROM (
SELECT array_to_json(ARRAY(
SELECT row_to_json(p)
FROM (
SELECT *, ARRAY(SELECT id FROM comment WHERE post_id = post.id) AS comment
FROM post
) AS p
)) AS post,
array_to_json(ARRAY(
SELECT row_to_json(comment)
FROM comment
)) AS comment
) sq;
But I'm not sure it's worth it -- usually not a good idea to dump all your data without limit / pagination.
SQLFiddle
SQL 2008 | .NET 4.0 | NHibernate 3.1 | NHibernate.Castle 3.1 | Castle.Core 2.5.2
So I have a linking table with metadata, like the author of this question NHibernate Mapping a Many to Many with Data on Join Table
Initially, I mapped just like the answer to this question as it seemed the most parsimonious way to handle it. However, after turning on show_sql and observing what was going on, the ID lookups ended up yielding N+1 queries where N is the number of associations.
Observe this example database that is analogous to my actual data, defined in sql-like syntax
CREATE TABLE [User]
(
Id int PRIMARY KEY
)
CREATE TABLE UserPref
(
Id int PRIMARY KEY,
Name varchar(32)
)
CREATE TABLE UserPrefAssociation
(
UserId int,
PrefId int,
Value varchar(32)
)
I hacked the following code together with this User one-to-many object mapping IList<UserPrefAssociation> Preferences { get; set; }
public IDictionary<string, string> GeneratePrefDict()
{
return Preferences
.ToDictionary(i => i.UserPref.Name, i => i.Value);
}
Sure, this works great, but as mentioned before, each i.UserPref.Name, is an additional query to SQL.
After playing in SQL, I have found the query that accomplishes what I want. My question then becomes how can I do this with NHibernate?
SELECT UserPref.Name, UserPrefAssociation.Value
FROM [User]
INNER JOIN UserPrefAssociation ON [User].Id = UserPrefAssociation.UserId
INNER JOIN UserPref ON UserPrefAssociation.UserPrefId = UserPref.Id
WHERE [User].Id = 1
~~~~SOLVED~~~~~
using NHibernate.Linq;
...
public IDictionary<string, string> GeneratePrefDict(ISession s)
{
return
(from entry in s.Query<User_UserPref>()
where entry.User == this
select new
{
key = entry.UserPref.Name,
value = entry.Value
})
.ToDictionary(i => i.key, i => i.value);
}
Generates this SQL
NHibernate: select userpref1_.Name as col_0_0_, user_userp0_.Value as col_1_0_ f
rom User_UserPref user_userp0_ left outer join UserPref userpref1_ on user_userp
0_.UserPrefId=userpref1_.Id where user_userp0_.UserId=#p0;#p0 = 1 [Type: Int32 (
0)]
Which is better than N+1 queries, and solves my issue.
I think you can achieve what you are wanting with Futures and QueryOver. Take a look at the following article:
Fighting cartesian product (x-join) when using NHibernate 3.0.0
If you can't visualize how to accomplish what you need from the above I can tailor that example more to your needs.