I have Product table which is connected with the other two tables Categorie & AttributeValue using many to many relationships. I am using GORM as an ORM. go struct for those tables are like bellow.
type Product struct {
ProductID int `gorm:"column:product_id;primary_key" json:"product_id"`
Name string `gorm:"column:name" json:"name"`
Categories []Category `gorm:"many2many:product_category;foreignkey:product_id;association_foreignkey:category_id;association_jointable_foreignkey:category_id;jointable_foreignkey:product_id;"`
AttributeValues []AttributeValue `gorm:"many2many:product_attribute;foreignkey:product_id;association_foreignkey:attribute_value_id;association_jointable_foreignkey:attribute_value_id;jointable_foreignkey:product_id;"`
}
type Category struct {
CategoryID int `gorm:"column:category_id;primary_key" json:"category_id"`
Name string `gorm:"column:name" json:"name"`
Products []Product `gorm:"many2many:product_category;foreignkey:category_id;association_foreignkey:product_id;association_jointable_foreignkey:product_id;jointable_foreignkey:category_id;"`
}
type AttributeValue struct {
AttributeValueID int `gorm:"column:attribute_value_id;primary_key" json:"attribute_value_id"`
AttributeID int `gorm:"column:attribute_id" json:"attribute_id"`
Value string `gorm:"column:value" json:"value"`
Products []Product `gorm:"many2many:product_attribute;foreignkey:attribute_value_id;association_foreignkey:product_id;association_jointable_foreignkey:product_id;jointable_foreignkey:attribute_value_id;"`
}
If I want to query Product table by category I can do it like bellow which will return all products in a category with category_id 3.
cat := model.Category{}
s.db.First(&cat, "category_id = ?", 3)
products := []*model.Product{}
s.db.Model(&cat).Related(&products, "Products")
If I want to query the Product table by both Category & AttributeValue how can I do that? Suppose I want to find all the products that are in category with category_id 3 and has AttributeValue with attribute_value_id 2?
I found some ways to query products based on category and AttributeValue. The best way I got is like bellow
products := []*model.Product{}
s.db.Joins("INNER JOIN product_attribute ON product_attribute.product_id = " +
"product.product_id AND product_attribute.attribute_value_id in (?)", 2).
Joins("INNER JOIN product_category ON product_category.product_id = " +
"product.product_id AND product_category.category_id in (?)", 3).
Find(&products)
After executing this, products slice will be populated with all the products that are in the category with category_id 3 and has AttributeValue with attribute_value_id 2. We can pass slice of string if we need to find products in multiple Category & AttributeValue.
Related
I have a GORM query with a preload that works just fine because I'm binding it to a struct called "companies" which is also the name of the corresponding database table:
var companies []Company
db.Preload("Subsidiaries").Joins("LEFT JOIN company_prod ON company_products.company_id = companies.id").Where("company_products.product_id = ?", ID).Find(&companies)
Now I want to do something similar, but bind the result to a struct that does not have a name that refers to the "companies" table:
var companiesFull []CompanyFull
db.Preload("Subsidiaries").Joins("LEFT JOIN company_prod ON company_products.company_id = companies.id").Where("company_products.product_id = ?", ID).Find(&companies)
I've simplified the second call for better understanding, the real call has more JOINs and returns more data, so it can't be bound to the "companies" struct.
I'm getting an error though:
column company_subsidiaries.company_full_id does not exist
The corresponding SQL query:
SELECT * FROM "company_subsidiaries" WHERE "company_subsidiaries"."company_full_id" IN (2,1)
There is no "company_subsidiaries.company_full_id", the correct query should be:
SELECT * FROM "company_subsidiaries" WHERE "company_subsidiaries"."company_id" IN (2,1)
The condition obviously gets generated from the name of the struct the result is being bound to. Is there any way to specify a custom name for this case?
I'm aware of the Tabler interface technique, however it doesn't work for Preload I believe (tried it, it changes the table name of the main query, but not the preload).
Updated: More info about the DB schema and structs
DB schema
TABLE companies
ID Primary key
OTHER FIELDS
TABLE products
ID Primary key
OTHER FIELDS
TABLE subsidiaries
ID Primary key
OTHER FIELDS
TABLE company_products
ID Primary key
Company_id Foreign key (companies.id)
Product_id Foreign key (products.id)
TABLE company_subsidiaries
ID Primary key
Company_id Foreign key (companies.id)
Subsidiary_id Foreign key (subsidiaries.id)
Structs
type Company struct {
Products []*Product `json:"products" gorm:"many2many:company_products;"`
ID int `json:"ID,omitempty"`
}
type CompanyFull struct {
Products []*Product `json:"products" gorm:"many2many:company_products;"`
Subsidiaries []*Subsidiary `json:"subsidiaries" gorm:"many2many:company_products;"`
ID int `json:"ID,omitempty"`
}
type Product struct {
Name string `json:"name"`
ID int `json:"ID,omitempty"`
}
type Subsidiary struct {
Name string `json:"name"`
ID int `json:"ID,omitempty"`
}
Generated SQL (by GORM)
SELECT * FROM "company_subsidiaries" WHERE "company_subsidiaries"."company_full_id" IN (2,1)
SELECT * FROM "subsidiaries" WHERE "subsidiaries"."id" IN (NULL)
SELECT companies.*, company_products.*, FROM "companies" LEFT JOIN company_products ON company_products.company_id = companies.id WHERE company_products.product_id = 1
Seems like the way to go in this case may be to customize the relationship in your CompanyFull model. Using joinForeignKey the following code works.
type CompanyFull struct {
Products []*Product `json:"products" gorm:"many2many:company_products;joinForeignKey:ID"`
Subsidiaries []*Subsidiary `json:"subsidiaries" gorm:"many2many:company_subsidiaries;joinForeignKey:ID"`
ID int `json:"ID,omitempty"`
}
func (CompanyFull) TableName() string {
return "companies"
}
func main(){
...
result := db.Preload("Subsidiaries").Joins("LEFT JOIN company_products ON company_products.company_id = companies.id").Where("company_products.product_id = ?", ID).Find(&companies)
if result.Error != nil {
log.Println(result.Error)
} else {
log.Printf("%#v", companies)
}
For more info regarding customizing the foreign keys used in relationships, take a look at the docs https://gorm.io/docs/many_to_many.html#Override-Foreign-Key
Question
When dealing with a one-to-many or many-to-many SQL relationship in Golang, what is the best (efficient, recommended, "Go-like") way of mapping the rows to a struct?
Taking the example setup below I have tried to detail some approaches with Pros and Cons of each but was wondering what the community recommends.
Requirements
Works with PostgreSQL (can be generic but not include MySQL/Oracle specific features)
Efficiency - No brute forcing every combination
No ORM - Ideally using only database/sql and jmoiron/sqlx
Example
For sake of clarity I have removed error handling
Models
type Tag struct {
ID int
Name string
}
type Item struct {
ID int
Tags []Tag
}
Database
CREATE TABLE item (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY
);
CREATE TABLE tag (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
name VARCHAR(160),
item_id INT REFERENCES item(id)
);
Approach 1 - Select all Items, then select tags per item
var items []Item
sqlxdb.Select(&items, "SELECT * FROM item")
for i, item := range items {
var tags []Tag
sqlxdb.Select(&tags, "SELECT * FROM tag WHERE item_id = $1", item.ID)
items[i].Tags = tags
}
Pros
Simple
Easy to understand
Cons
Inefficient with the number of database queries increasing proportional with number of items
Approach 2 - Construct SQL join and loop through rows manually
var itemTags = make(map[int][]Tag)
var items = []Item{}
rows, _ := sqlxdb.Queryx("SELECT i.id, t.id, t.name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
for rows.Next() {
var (
itemID int
tagID int
tagName string
)
rows.Scan(&itemID, &tagID, &tagName)
if tags, ok := itemTags[itemID]; ok {
itemTags[itemID] = append(tags, Tag{ID: tagID, Name: tagName,})
} else {
itemTags[itemID] = []Tag{Tag{ID: tagID, Name: tagName,}}
}
}
for itemID, tags := range itemTags {
items = append(Item{
ID: itemID,
Tags: tags,
})
}
Pros
A single database call and cursor that can be looped through without eating too much memory
Cons
Complicated and harder to develop with multiple joins and many attributes on the struct
Not too performant; more memory usage and processing time vs. more network calls
Failed approach 3 - sqlx struct scanning
Despite failing I want to include this approach as I find it to be my current aim of efficiency paired with development simplicity. My hope was by explicitly setting the db tag on each struct field sqlx could do some advanced struct scanning
var items []Item
sqlxdb.Select(&items, "SELECT i.id AS item_id, t.id AS tag_id, t.name AS tag_name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
Unfortunately this errors out as missing destination name tag_id in *[]Item leading me to believe the StructScan is not advanced enough to recursively loop through rows (no criticism - it is a complicated scenario)
Possible approach 4 - PostgreSQL array aggregators and GROUP BY
While I am sure this will not work I have included this untested option to see if it could be improved upon so it may work.
var items = []Item{}
sqlxdb.Select(&items, "SELECT i.id as item_id, array_agg(t.*) as tags FROM item AS i JOIN tag AS t ON t.item_id = i.id GROUP BY i.id")
When I have some time I will try and run some experiments here.
the sql in postgres :
create schema temp;
set search_path = temp;
create table item
(
id INT generated by default as identity primary key
);
create table tag
(
id INT generated by default as identity primary key,
name VARCHAR(160),
item_id INT references item (id)
);
create view item_tags as
select id,
(
select
array_to_json(array_agg(row_to_json(taglist.*))) as array_to_json
from (
select tag.name, tag.id
from tag
where item_id = item.id
) taglist ) as tags
from item ;
-- golang query this maybe
select row_to_json(row)
from (
select * from item_tags
) row;
then golang query this sql:
select row_to_json(row)
from (
select * from item_tags
) row;
and unmarshall to go struct:
pro:
postgres manage the relation of data. add / update data with sql functions.
golang manage business model and logic.
it's easy way.
.
I can suggest another approach which I have used before.
You make a json of the tags in this case in the query and return it.
Pros: You have 1 call to the db, which aggregates the data, and all you have to do is parse the json into an array.
Cons: It's a bit ugly. Feel free to bash me for it.
type jointItem struct {
Item
ParsedTags string
Tags []Tag `gorm:"-"`
}
var jointItems []*jointItem
db.Raw(`SELECT
items.*,
(SELECT CONCAT(
'[',
GROUP_CONCAT(
JSON_OBJECT('id', id,
'name', name
)
),
']'
)) as parsed_tags
FROM items`).Scan(&jointItems)
for _, o := range jointItems {
var tempTags []Tag
if err := json.Unmarshall(o.ParsedTags, &tempTags) ; err != nil {
// do something
}
o.Tags = tempTags
}
Edit: code might behave weirdly so I find it better to use a temporary tags array when moving instead of using the same struct.
You can use carta.Map() from https://github.com/jackskj/carta
It tracks has-many relationships automatically.
I have a database table "Category" which can have a self-reference. If self-reference is null, then it's a top-level category (no parent). If it's pointing to another category, it's a subcategory of the given category.
Now I'm trying to setup query which returns all subcategories with given parent category id. Something like following (Android Room syntax):
"SELECT * FROM category WHERE parent_id = :parentId"
This query returns nothing if parentId == null, but I would like it to return all categories where parentId == null (i.e. top-level categories). With a query
"SELECT * FROM category WHERE parent_id IS NULL"
I can get parent categories successfully, but obviously, it's not working for subcategories.
I tried also the following query
"SELECT * FROM category WHERE (parent_id IS NULL OR parent_id = :parentId)
but it fails of course, since it always returns top-level categories, even though the intention is to get only certain set of subcategories.
My question is, how I can set up my query so that it works as expected, i.e. returns all top-level categories when parentId == null, otherwise it returns all subcategories of given category id.
The SQL standard requires this handling of NULL values.
But SQLite has an extension: you can use IS also with non-NULL values:
SELECT * FROM category WHERE parent_id IS :parentId
I think you just need a slightly more complicated condition:
SELECT c.*
FROM category c
WHERE (c.parent_id IS NULL AND :parentId IS NULL) OR
(parent_id = :parentId);
Using ObjectContext. I'm wanting to do this by passing an SQL query via the ExecuteStoreCommand since I don't fancy retrieving all relevant entities just for the sake of deleting them after.
The Category table is as so:
CatID | CatName | ParentID
Where CatID is the primary key to the ParentID FK
I am wishing to delete a category and also all those that
are under it. Can be 2+ levels deep of sub cats, so different ParentID's
Thought I could do it as below and just set "cascade" on delete option
for the foreign key in the database, but it won't let me and it does not appear to want to
cascade delete down by using the CatID - ParentID relationship and the query gets
stopped by this very FK constraint.
public RedirectToRouteResult DelCat(int CatID)
{
if (CatID != 0)
{
_db.ExecuteStoreCommand("DELETE FROM Categories WHERE CatID={0}", CatID);
_db.SaveChanges();
}
return RedirectToAction("CatManage");
}
Recursive CTE allCategories produces list of all categories in hierarchy. Delete part, obviously, deletes them all.
; with allCategories as (
select CatID
from Categories
where CatID = #CatID_to_delete
union all
select Categories.CatID
from allCategories
inner join Categories
on allCategories.CatID = Categories.ParentID
)
delete Categories
from Categories
inner join allCategories
on Categories.CatID = allCategories.CatID
Try it with select * from allCategories, though, to check first.
There is TEST # Sql Fiddle.
Why not just send two statements in your batch?
DELETE Categories WHERE ParentID = {0};
DELETE Categories WHERE CatID = {0};
If the framework you're using "won't let you" do that, then do this right: put your logic in a stored procedure, and call the stored procedure.
I have two tables: 'Category' and 'Product'. In Category I have catid and cat name. In product table I have item-id, item-name, catid.
What I need to do is show a result that will have item name and category name. But the category name will be multiple. So that one product can have multiple categories on it.
You might want to create three tables, because of a join table to would allow each line to have multiple lines corresponding in the other table:
Category : catid catname
Product : itemid itemname
CategoryProduct : catid itemid
So a product can have 0, 1 or more Categories.
Example content for the join table, for two products having the same two categories:
catid itemid in CategoryProduct
1 3
1 4
2 3
2 4
If your schema has a category ID in the product table, then no, one produce can't have multiple categories. If you have another M:N table to link products to categories, you should update your question.
In reply to your comment:
The category id of the product table
is like 2,3,4 so 1 product can have
multiple category
If you're allowed to change the table structure, by all means follow KLE's advice. That's the sane and maintainable approach.
If you can't change the table structure, you can query the categories with a hack like this:
select *
from product p
inner join category c
on ',' + p.catid + ','
like '%,' + cast(c.catid as varchar) + ',%'