I have a GORM query with a preload that works just fine because I'm binding it to a struct called "companies" which is also the name of the corresponding database table:
var companies []Company
db.Preload("Subsidiaries").Joins("LEFT JOIN company_prod ON company_products.company_id = companies.id").Where("company_products.product_id = ?", ID).Find(&companies)
Now I want to do something similar, but bind the result to a struct that does not have a name that refers to the "companies" table:
var companiesFull []CompanyFull
db.Preload("Subsidiaries").Joins("LEFT JOIN company_prod ON company_products.company_id = companies.id").Where("company_products.product_id = ?", ID).Find(&companies)
I've simplified the second call for better understanding, the real call has more JOINs and returns more data, so it can't be bound to the "companies" struct.
I'm getting an error though:
column company_subsidiaries.company_full_id does not exist
The corresponding SQL query:
SELECT * FROM "company_subsidiaries" WHERE "company_subsidiaries"."company_full_id" IN (2,1)
There is no "company_subsidiaries.company_full_id", the correct query should be:
SELECT * FROM "company_subsidiaries" WHERE "company_subsidiaries"."company_id" IN (2,1)
The condition obviously gets generated from the name of the struct the result is being bound to. Is there any way to specify a custom name for this case?
I'm aware of the Tabler interface technique, however it doesn't work for Preload I believe (tried it, it changes the table name of the main query, but not the preload).
Updated: More info about the DB schema and structs
DB schema
TABLE companies
ID Primary key
OTHER FIELDS
TABLE products
ID Primary key
OTHER FIELDS
TABLE subsidiaries
ID Primary key
OTHER FIELDS
TABLE company_products
ID Primary key
Company_id Foreign key (companies.id)
Product_id Foreign key (products.id)
TABLE company_subsidiaries
ID Primary key
Company_id Foreign key (companies.id)
Subsidiary_id Foreign key (subsidiaries.id)
Structs
type Company struct {
Products []*Product `json:"products" gorm:"many2many:company_products;"`
ID int `json:"ID,omitempty"`
}
type CompanyFull struct {
Products []*Product `json:"products" gorm:"many2many:company_products;"`
Subsidiaries []*Subsidiary `json:"subsidiaries" gorm:"many2many:company_products;"`
ID int `json:"ID,omitempty"`
}
type Product struct {
Name string `json:"name"`
ID int `json:"ID,omitempty"`
}
type Subsidiary struct {
Name string `json:"name"`
ID int `json:"ID,omitempty"`
}
Generated SQL (by GORM)
SELECT * FROM "company_subsidiaries" WHERE "company_subsidiaries"."company_full_id" IN (2,1)
SELECT * FROM "subsidiaries" WHERE "subsidiaries"."id" IN (NULL)
SELECT companies.*, company_products.*, FROM "companies" LEFT JOIN company_products ON company_products.company_id = companies.id WHERE company_products.product_id = 1
Seems like the way to go in this case may be to customize the relationship in your CompanyFull model. Using joinForeignKey the following code works.
type CompanyFull struct {
Products []*Product `json:"products" gorm:"many2many:company_products;joinForeignKey:ID"`
Subsidiaries []*Subsidiary `json:"subsidiaries" gorm:"many2many:company_subsidiaries;joinForeignKey:ID"`
ID int `json:"ID,omitempty"`
}
func (CompanyFull) TableName() string {
return "companies"
}
func main(){
...
result := db.Preload("Subsidiaries").Joins("LEFT JOIN company_products ON company_products.company_id = companies.id").Where("company_products.product_id = ?", ID).Find(&companies)
if result.Error != nil {
log.Println(result.Error)
} else {
log.Printf("%#v", companies)
}
For more info regarding customizing the foreign keys used in relationships, take a look at the docs https://gorm.io/docs/many_to_many.html#Override-Foreign-Key
Related
I'm not entirely sure the best way to phrase this problem, but hopefully my description and the code will show what I mean.
I'm building an API that uses a SQL db. One of the record types, Order can contain a Key, but the key ID is null until the Order is finalized. I have a function that queries the DB for all orders, joined with the Key DB so that the key details are populated if the ID is not null.
How do I scan in Orders where some have keyId null and some do not?
Order struct:
type orderDb struct {
id int
certificate certificateDb
location string
status string
knownRevoked bool
err sql.NullString // stored as json object
expires sql.NullInt32
dnsIdentifiers commaJoinedStrings // will be a comma separated list from storage
authorizations commaJoinedStrings // will be a comma separated list from storage
finalize string
finalizedKey *keyDb
certificateUrl sql.NullString
pem sql.NullString
validFrom sql.NullInt32
validTo sql.NullInt32
createdAt int
updatedAt int
}
keyDb struct
type keyDb struct {
id int
name string
description string
algorithmValue string
pem string
apiKey string
apiKeyViaUrl bool
createdAt int
updatedAt int
}
Partial SQL query & scan
query := `
SELECT
ao.id, ao.status, ao.known_revoked, ao.error, ao.dns_identifiers, ao.valid_from,
ao.valid_to, ao.created_at, ao.updated_at,
pk.id, pk.name,
c.id, c.name, c.subject,
aa.id, aa.name, aa.is_staging
FROM
acme_orders ao
LEFT JOIN private_keys pk on (ao.finalized_key_id = pk.id)
LEFT JOIN certificates c on (ao.certificate_id = c.id)
LEFT JOIN acme_accounts aa on (c.acme_account_id = aa.id)
err = rows.Scan(
&oneOrder.id,
&oneOrder.status,
&oneOrder.knownRevoked,
&oneOrder.err,
&oneOrder.dnsIdentifiers,
&oneOrder.validFrom,
&oneOrder.validTo,
&oneOrder.createdAt,
&oneOrder.updatedAt,
&oneOrder.finalizedKey.id,
&oneOrder.finalizedKey.name,
Essentially sometimes key id and name are null because the key is null. How do I set finalizedKey to null when this is the case, but scan in the values when the key isn't null?
I don't really want to do a separate query for keys because the slice of Orders could have 20, 50, or 100+ records and I don't want to do 101 queries to return a slice of 100 Orders.
Here are three tables
Table users {
id uuid [pk, default: `gen_random_uuid()`]
auth_id uuid [ref: - auths.id]
org_id uuid [ref: - orgs.id]
access_group text [default: 'DEFAULT']
created_at int [default: `now()::int`]
updated_at int
age int
status text
}
Table op_user_access_groups {
op_id uuid [pk, ref: > ops.id]
access_group text [pk, default: 'DEFAULT']
}
Table op_users {
op_id uuid [pk, ref: > ops.id]
user_id uuid [pk, ref: > users.id]
access boolean [default: false]
}
The table users has user info and he/she belongs to certain organization (org_id)
The table op_user_access_groups has the information regarding an operator having access to what all access_groups. The op_id belongs to the org_id
The table op_users has information about users (user_id) that can be accessed by op_id irrespective of which group a user belongs to.
I want to create a view such that if I do
select * from <that view> where op_id = ?
I should get the users the operator has access to.
Any help is appreciated :)
You should use a JOIN query to create such a view. Something like this:
CREATE VIEW v AS
SELECT * FROM users
JOIN op_users ON users.id=op_users.op_id
JOIN op_user_access_groups ON op_user_access_groups.access_group=users.access_group
WHERE op_users.access = true
It's not exactly clear how your data model works from your question, but creating a view with the JOIN that answers your question is the correct response here.
Question
When dealing with a one-to-many or many-to-many SQL relationship in Golang, what is the best (efficient, recommended, "Go-like") way of mapping the rows to a struct?
Taking the example setup below I have tried to detail some approaches with Pros and Cons of each but was wondering what the community recommends.
Requirements
Works with PostgreSQL (can be generic but not include MySQL/Oracle specific features)
Efficiency - No brute forcing every combination
No ORM - Ideally using only database/sql and jmoiron/sqlx
Example
For sake of clarity I have removed error handling
Models
type Tag struct {
ID int
Name string
}
type Item struct {
ID int
Tags []Tag
}
Database
CREATE TABLE item (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY
);
CREATE TABLE tag (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
name VARCHAR(160),
item_id INT REFERENCES item(id)
);
Approach 1 - Select all Items, then select tags per item
var items []Item
sqlxdb.Select(&items, "SELECT * FROM item")
for i, item := range items {
var tags []Tag
sqlxdb.Select(&tags, "SELECT * FROM tag WHERE item_id = $1", item.ID)
items[i].Tags = tags
}
Pros
Simple
Easy to understand
Cons
Inefficient with the number of database queries increasing proportional with number of items
Approach 2 - Construct SQL join and loop through rows manually
var itemTags = make(map[int][]Tag)
var items = []Item{}
rows, _ := sqlxdb.Queryx("SELECT i.id, t.id, t.name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
for rows.Next() {
var (
itemID int
tagID int
tagName string
)
rows.Scan(&itemID, &tagID, &tagName)
if tags, ok := itemTags[itemID]; ok {
itemTags[itemID] = append(tags, Tag{ID: tagID, Name: tagName,})
} else {
itemTags[itemID] = []Tag{Tag{ID: tagID, Name: tagName,}}
}
}
for itemID, tags := range itemTags {
items = append(Item{
ID: itemID,
Tags: tags,
})
}
Pros
A single database call and cursor that can be looped through without eating too much memory
Cons
Complicated and harder to develop with multiple joins and many attributes on the struct
Not too performant; more memory usage and processing time vs. more network calls
Failed approach 3 - sqlx struct scanning
Despite failing I want to include this approach as I find it to be my current aim of efficiency paired with development simplicity. My hope was by explicitly setting the db tag on each struct field sqlx could do some advanced struct scanning
var items []Item
sqlxdb.Select(&items, "SELECT i.id AS item_id, t.id AS tag_id, t.name AS tag_name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
Unfortunately this errors out as missing destination name tag_id in *[]Item leading me to believe the StructScan is not advanced enough to recursively loop through rows (no criticism - it is a complicated scenario)
Possible approach 4 - PostgreSQL array aggregators and GROUP BY
While I am sure this will not work I have included this untested option to see if it could be improved upon so it may work.
var items = []Item{}
sqlxdb.Select(&items, "SELECT i.id as item_id, array_agg(t.*) as tags FROM item AS i JOIN tag AS t ON t.item_id = i.id GROUP BY i.id")
When I have some time I will try and run some experiments here.
the sql in postgres :
create schema temp;
set search_path = temp;
create table item
(
id INT generated by default as identity primary key
);
create table tag
(
id INT generated by default as identity primary key,
name VARCHAR(160),
item_id INT references item (id)
);
create view item_tags as
select id,
(
select
array_to_json(array_agg(row_to_json(taglist.*))) as array_to_json
from (
select tag.name, tag.id
from tag
where item_id = item.id
) taglist ) as tags
from item ;
-- golang query this maybe
select row_to_json(row)
from (
select * from item_tags
) row;
then golang query this sql:
select row_to_json(row)
from (
select * from item_tags
) row;
and unmarshall to go struct:
pro:
postgres manage the relation of data. add / update data with sql functions.
golang manage business model and logic.
it's easy way.
.
I can suggest another approach which I have used before.
You make a json of the tags in this case in the query and return it.
Pros: You have 1 call to the db, which aggregates the data, and all you have to do is parse the json into an array.
Cons: It's a bit ugly. Feel free to bash me for it.
type jointItem struct {
Item
ParsedTags string
Tags []Tag `gorm:"-"`
}
var jointItems []*jointItem
db.Raw(`SELECT
items.*,
(SELECT CONCAT(
'[',
GROUP_CONCAT(
JSON_OBJECT('id', id,
'name', name
)
),
']'
)) as parsed_tags
FROM items`).Scan(&jointItems)
for _, o := range jointItems {
var tempTags []Tag
if err := json.Unmarshall(o.ParsedTags, &tempTags) ; err != nil {
// do something
}
o.Tags = tempTags
}
Edit: code might behave weirdly so I find it better to use a temporary tags array when moving instead of using the same struct.
You can use carta.Map() from https://github.com/jackskj/carta
It tracks has-many relationships automatically.
I have three tables. I want to display all data from cms_planner table together with Topic Name from cms_topic table. To achieve that, I need to go through the cms_subject table.
I want to use belongsToMany but I already have cms_subject table that holds the foreign key for cms_planner and the foreign key for cms_topic. The table name does not represent pivot key.
I also want to use hasManyThrough but it doesn't work. I'm thinking to inverse the hasManyThrough.
How can I achieve that?
1. CmsPlanner
i. planner_id
ii. subject_id
iii. date_start
2. CmsSubject
i. subject_id
ii. topic_id
3. CmsTopic
i. topic_id
ii. topic name
In CmsPlanner model
public function subject(){
return $this->hasManyThrough(
'App\CmsTopic',
'App\CmsSubject',
'topic_id', 'topic_id', 'planner_id');}
In CmsPlanner controller
CmsPlanner::with('subject')->get();
Add this relation on CmsSubject
public function cmsTopic()
{
return $this->belongsTo('App\Models\CmsTopic', 'topic_id', 'topic_id');
}
then add following relation on CmsPlanner
public function cmsSubject()
{
return $this->belongsTo('App\Models\CmsSubject', 'subject_id', 'subject_id');
}
to get data
$cms_planner = CmsPlanner::with('CmsSubject')->where('id', $planner_id)->get();
'user' => $this->business->user
Using ObjectContext. I'm wanting to do this by passing an SQL query via the ExecuteStoreCommand since I don't fancy retrieving all relevant entities just for the sake of deleting them after.
The Category table is as so:
CatID | CatName | ParentID
Where CatID is the primary key to the ParentID FK
I am wishing to delete a category and also all those that
are under it. Can be 2+ levels deep of sub cats, so different ParentID's
Thought I could do it as below and just set "cascade" on delete option
for the foreign key in the database, but it won't let me and it does not appear to want to
cascade delete down by using the CatID - ParentID relationship and the query gets
stopped by this very FK constraint.
public RedirectToRouteResult DelCat(int CatID)
{
if (CatID != 0)
{
_db.ExecuteStoreCommand("DELETE FROM Categories WHERE CatID={0}", CatID);
_db.SaveChanges();
}
return RedirectToAction("CatManage");
}
Recursive CTE allCategories produces list of all categories in hierarchy. Delete part, obviously, deletes them all.
; with allCategories as (
select CatID
from Categories
where CatID = #CatID_to_delete
union all
select Categories.CatID
from allCategories
inner join Categories
on allCategories.CatID = Categories.ParentID
)
delete Categories
from Categories
inner join allCategories
on Categories.CatID = allCategories.CatID
Try it with select * from allCategories, though, to check first.
There is TEST # Sql Fiddle.
Why not just send two statements in your batch?
DELETE Categories WHERE ParentID = {0};
DELETE Categories WHERE CatID = {0};
If the framework you're using "won't let you" do that, then do this right: put your logic in a stored procedure, and call the stored procedure.