I'm new using go and gorm. I'm trying to insert many values in one SQL query.
I wrote this query to add multiple conversations to a user:
relationUserConversation := make([][]uint, len(users))
for i, v := range users {
relationUserConversation[i] = []uint{conversation.ID, v}
}
result = r.db.Debug().Exec(
"INSERT INTO `user_has_conversations` (`user_has_conversations`.`conversation_id`, `user_has_conversations`.`user_id`) VALUES ?",
relationUserConversation, // If i do this it works relationUserConversation[0], relationUserConversation[1]
// The issue is because the query has this value "VALUES ((35,1),(35,2))", but should be to work (35,1),(35,2)
)
I also tried to add it directly with the conversation that would be what I would like to do, but I'm having issue trying to add the relation with the many to many because instead of creating the relation between the user and the conversation it tries to add the user.
My conversation model:
type Conversation struct {
ID uint `gorm:"primarykey"`
Users []*User `gorm:"many2many:user_has_conversations;"`
Messages []ConversationMessage
}
Would be great if i could create a new conversation with the related users in one query instead of creating first the conversation and after the relation to the users.
Below is a minimum working example using the Gorm Appends method (see documentation here) to create a many to many association between two (or more) models. Hopefully you can adapt this to your use case.
package main
import (
"fmt"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
)
type User struct {
gorm.Model
Name string
Conversations []Conversation `gorm:"many2many:user_conversations;"`
}
type Conversation struct {
gorm.Model
Name string
Users []*User `gorm:"many2many:user_conversations;"`
}
func main() {
db, err := gorm.Open(sqlite.Open("many2many.db"), &gorm.Config{})
if err != nil {
panic("failed to connect database")
}
// Migrate the schema
err = db.AutoMigrate(&User{}, &Conversation{})
if err != nil {
fmt.Print(err)
}
userOne := User{
Name: "User One",
}
userTwo := User{
Name: "User Two",
}
// Create users
db.Create(&userOne)
db.Create(&userTwo)
conversation := Conversation{
Name: "Conversation One",
}
// Create conversation
db.Create(&conversation)
// Append users
err = db.Model(&conversation).Association("Users").Append([]User{userOne, userTwo})
if err != nil {
fmt.Print(err)
}
for _, convUser := range conversation.Users {
fmt.Println("Hello I am in the conversation: " + convUser.Name)
}
// Clean up database
db.Delete(&userOne)
db.Delete(&userTwo)
db.Delete(&conversation)
}
Number of queries
If you enable Debug() on Gorm:
err = db.Debug().Model(&conversation).Association("Users").Append([]User{userOne, userTwo})
It shows this:
[0.144ms] [rows:2] INSERT INTO `user_conversations`
(`conversation_id`,`user_id`) VALUES (8,15),(8,16) ON CONFLICT DO NOTHING
The Values part is correct (what you were trying to do manually) and achieved using the ORM.
Related
I apologize for the title, I don't exactly know how to word it. But essentially, this is a graph-type query but I know RavenDB's graph functionality will be going away so this probably needs to be solved with Javascript.
Here is the scenario:
I have a bunch of documents of different types, call them A, B, C, D. Each of these particular types of documents have some common properties. The one that I'm interested in right now is "Owner". The owner field is an ID which points to one of two other document types; it can be a Group or a User.
The Group document has a 'Members' field which contains an ID which either points to a User or another Group. Something like this
It's worth noting that the documents in play have custom IDs that begin with their entity type. For example Users and Groups begin with user: and group: respectively. Example IDs look like this: user:john#castleblack.com or group:the-nights-watch. This comes into play later.
What I want to be able to do is the following type of query:
"Given that I have either a group id or a user id, return all documents of type a, b, or c where the group/user id is equal to or is a descendant of the document's owner."
In other words, I need to be able to return all documents that are owned by a particular user or group either explicitly or implicitly through a hierarchy.
I've considered solving this a couple different ways with no luck. Here are the two approaches I've tried:
Using a function within a query
With Dejan's help in an email thread, I was able to devise a function that would walk it's way down the ownership graph. What this attempted to do was build a flat array of IDs which represented explicit and implicit owners (i.e. root + descendants):
declare function hierarchy(doc, owners){
owners = owners || [];
while(doc != null) {
let ownerId = id(doc)
if(ownerId.startsWith('user:')) {
owners.push(ownerId);
} else if(ownerId.startsWith('group:')) {
owners.push(ownerId);
doc.Members.forEach(m => {
let owner = load(m, 'Users') || load(m, 'Groups');
owners = hierarchy(owner, owners);
});
}
}
return owners;
}
I had two issues with this. 1. I don't actually know how to use this in a query lol. I tried to use it as part of the where clause but apparently that's not allowed:
from #all_docs as d
where hierarchy(d) = 'group:my-group-d'
// error: method hierarchy not allowed
Or if I tried anything in the select statement, I got an error that I have exceeded the number of allowed statements.
As a custom index
I tried the same idea through a custom index. Essentially, I tried to create an index that would produce an array of IDs using roughly the same function above, so that I could just query where my id was in that array
map('#all_docs', function(doc) {
function hierarchy(n, graph) {
while(n != null) {
let ownerId = id(n);
if(ownerId.startsWith('user:')) {
graph.push(ownerId);
return graph;
} else if(ownerId.startsWith('group:')){
graph.push(ownerId);
n.Members.forEach(g => {
let owner = load(g, 'Groups') || load(g, 'Users');
hierarchy(owner, graph);
});
return graph;
}
}
}
function distinct(value, index, self){ return self.indexOf(value) === index; }
let ownerGraph = []
if(doc.Owner) {
let owner = load(doc.Owner, 'Groups') || load(doc.Owner, 'Users');
ownerGraph = hierarchy(owner, ownerGraph).filter(distinct);
}
return { Owners: ownerGraph };
})
// error: recursion is not allowed by the javascript host
The problem with this is that I'm getting an error that recursion is not allowed.
So I'm stumped now. Am I going about this wrong? I feel like this could be a subquery of sorts or a filter by function, but I'm not sure how to do that either. Am I going to have to do this in two separate queries (i.e. two round-trips), one to get the IDs and the other to get the docs?
Update 1
I've revised my attempt at the index to the following and I'm not getting the recursion error anymore, but assuming my queries are correct, it's not returning anything
// Entity/ByOwnerGraph
map('#all_docs', function(doc) {
function walkGraph(ownerId) {
let owners = []
let idsToProcess = [ownerId]
while(idsToProcess.length > 0) {
let current = idsToProcess.shift();
if(current.startsWith('user:')){
owners.push(current);
} else if(current.startsWith('group:')) {
owners.push(current);
let group = load(current, 'Groups')
if(!group) { continue; }
idsToProcess.concat(group.Members)
}
}
return owners;
}
let owners = [];
if(doc.Owner) {
owners.concat(walkGraph(doc.Owner))
}
return { Owners: owners };
})
// query (no results)
from index Entity/ByOwnerGraph as x
where x.Owners = "group:my-group-id"
// alternate query (no results)
from index Entity/ByOwnerGraph as x
where x.Owners ALL IN ("group:my-group-id")
I still can't use this approach in a query either as I get the same error that there are too many statements.
So basically I have 3 models: User, Profile, and Post.
They are related like so: User has one Profile. Profile has many Post
They look like this:
type User struct {
Base // holds this object's uuid, createdAt, updatedAt
Role string `json:"role"`
Username string `json:"username" gorm:"unique"`
Password string `json:"password"`
Profile Profile `gorm:"constraint:OnDelete:CASCADE;"`
}
type Profile struct {
Base // holds this object's uuid, createdAt, updatedAt
UserId string `json:"user_id"`
Name string `json:"name"`
Bio string `json:"bio"`
Age uint8 `json:"age"`
Posts []Post `gorm:"constraint:OnDelete:CASCADE;"`
}
type Post struct {
Base // holds this object's uuid, createdAt, updatedAt
ProfileId string `json:"profile_id"`
Caption string `json:"caption"`
Likes uint32 `json:"num_likes" gorm:"default:0"`
}
What I want to happen is when I delete the user, I want the profile to be deleted and all the posts that are related to it. My only other experience with relational databases are Django where this is automatic.
What actually happens is when I delete the user, the profile gets deleted but the posts remain in the database.
This is how I am deleting the user:
...
base := models.Base{Id: id}
if err := configs.Database.Select(clause.Associations).Delete(&models.User{Base: base}).Error; err != nil {
return c.Status(400).JSON(err.Error())
}
...
I've already looked at this but its not very helpful. How could I accomplish this?
Thank you!
Based on the issue link you posted, and other related issues, it might not be possible to use clause.Associations for relations nested deeper than one level. In your case, the Profile related to your User is deleted, but not the Posts related to the Profile.
One way to delete all wanted associations is to use a delete hook. Either BeforeDelete or AfterDelete, depending on your setup and how strong your relationships are. For example:
func (u *User) BeforeDelete(tx *gorm.DB) (err error) {
if err := tx.Joins("profiles p ON p.id = posts.profile_id").Joins("users u ON u.id = p.user_id").Where("u.id = ?", u.Base.Id).Delete(&Post{}).Error; err != nil {
return err
}
return tx.Joins("users u ON u.id = profiles.user_id").Where("u.id = ?", u.Base.Id).Delete(&Profile{}).Error
}
This way, when you execute configs.Database.Delete(&models.User{Base: base}), it will execute the hook first, then this query.
Another way would be to execute all the queries from the hook function separately:
base := models.Base{Id: id}
if err := configs.Database.Joins("profiles p ON p.id = posts.profile_id").Joins("users u ON u.id = p.user_id").Where("u.id = ?", base.Id).Delete(&Post{}).Error; err != nil {
return c.Status(400).JSON(err.Error())
}
if err := configs.Database.Joins("users u ON u.id = profiles.user_id").Where("u.id = ?", base.Id).Delete(&Profile{}).Error; err != nil {
return c.Status(400).JSON(err.Error())
}
if err := configs.Database.Delete(&models.User{Base: base}).Error; err != nil {
return c.Status(400).JSON(err.Error())
}
Let’s say you have a basic API (GET/POST/PATCH/DELETE) backed by an SQL database.
The PATCH call should only update the fields in the JSON payload that the user sends, without touching any of the other fields.
Imagine the table (let's call it sample) has id, string_a and string_b columns, and the struct which corresponds to it looks like:
type Sample struct {
ID int `json:"id"`
StringA string `json:"stringA"`
StringB string `json:"stringB"`
}
Let's say the user passes in { "stringA": "patched value" } as payload. The json will be unmarshalled to something that looks like:
&Sample{
ID: 0,
StringA: "patched value",
StringB: "",
}
For a project using database/sql, you’d write the query to patch the row something like:
// `id` is from the URL params
query := `UPDATE sample SET string_a=$1, string_b=$2 WHERE id=$3`
row := db.QueryRow(query, sample.StringA, sample.StringB, id)
...
That query would update the string_a column as expected, but it’d also update the string_b column to "", which is undesired behavior in this case. In essence, I’ve just created a PUT instead of a PATCH.
My immediate thought was - OK, that’s fine, let’s use strings.Builder to build out the query and only add a SET statement for those that have a non-nil/empty value.
However, in that case, if a user wanted to make string_a empty, how would they accomplish that?
Eg. the user makes a PATCH call with { "stringA": "" } as payload. That would get unmarshalled to something like:
&Sample{
ID: 0,
StringA: "",
StringB: "",
}
The “query builder” I was theorizing about would look at that and say “ok, those are all nil/empty values, don’t add them to the query” and no columns would be updated, which again, is undesired behavior.
I’m not sure how to write my API and the SQL queries it runs in a way that satisfies both cases. Any thoughts?
I think reasonable solution for smaller queries is to build UPDATE query and list of bound parameters dynamically while processing payload with logic that recognizes what was updated and what was left empty.
From my own experience this is clear and readable (if repetitive you can always iterate over struct members that share same logic or employ reflection and look at struct tags hints, etc.). Every (my) attempt to write universal solution for this ended up as very convoluted overkill supporting all sorts of corner-cases and behavioral differences between endpoints.
func patchSample(s Sample) {
var query strings.Builder
params := make([]interface{}, 0, 2)
// TODO Check if patch makes sense (e.g. id is non-zero, at least one patched value provided, etc.
query.WriteString("UPDATE sample SET")
if s.StringA != "" {
query.WriteString(" stringA = ?")
params = append(params, s.StringA)
}
if s.StringB != "" {
query.WriteString(" stringB = ?")
params = append(params, s.StringB)
}
query.WriteString(" WHERE id = ?")
params = append(params, s.ID)
fmt.Println(query.String(), params)
//_, err := db.Exec(query.String(), params...)
}
func main() {
patchSample(Sample{1, "Foo", ""})
patchSample(Sample{2, "", "Bar"})
patchSample(Sample{3, "Foo", "Bar"})
}
EDIT: In case "" is valid value for patching then it needs to be distinguishable from the default empty value. One way how to solve that for string is to use pointer which will default to nil if value is not present in payload:
type Sample struct {
ID int `json:"id"`
StringA *string `json:"stringA"`
StringB *string `json:"stringB"`
}
and then modify condition(s) to check if field was sent like this:
if s.StringA != nil {
query.WriteString(" stringA = ?")
params = append(params, *s.StringA)
}
See full example in playground: https://go.dev/play/p/RI7OsNEYrk6
For what it's worth, I solved the issue by:
Converting the request payload to a generic map[string]interface{}.
Implementing a query builder that loops through the map's keys to create a query.
Part of the reason I went this route is it fit all my requirements, and I didn't particularly like having *strings or *ints laying around.
Here is what the query builder looks like:
func patchQueryBuilder(id string, patch map[string]interface{}) (string, []interface{}, error) {
var query strings.Builder
params := make([]interface{}, 0)
query.WriteString("UPDATE some_table SET")
for k, v := range patch {
switch k {
case "someString":
if someString, ok := v.(string); ok {
query.WriteString(fmt.Sprintf(" some_string=$%d,", len(params)+1))
params = append(params, someString)
} else {
return "", []interface{}{}, fmt.Errorf("could not process some_string")
}
case "someBool":
if someBool, ok := v.(bool); ok {
query.WriteString(fmt.Sprintf(" some_bool=$%d,", len(params)+1))
params = append(params, someBool)
} else {
return "", []interface{}{}, fmt.Errorf("could not process some_bool")
}
}
}
if len(params) > 0 {
// Remove trailing comma to avoid syntax errors
queryString := fmt.Sprintf("%s WHERE id=$%d RETURNING *", strings.TrimSuffix(query.String(), ","), len(params)+1)
params = append(params, id)
return queryString, params, nil
} else {
return "", []interface{}{}, nil
}
}
Note that I'm using PostgreSQL, so I needed to provide numbered parameters to the query, eg $1, which is what params is used for. It's also returned from the function so that it can be used as follows:
// Build the patch query based on the payload
query, params, err := patchQueryBuilder(id, patch)
if err != nil {
return nil, err
}
// Use the query/params and get output
row := tx.QueryRowContext(ctx, query, params...)
well i am trying to make a select function which will work like this,
From main function i will call this select function with necessary variables. and the select function will run the query in the database and give me the reselt.
Now for go lang if we want to do a select query for the results we need a struct where we can get the results. Since the function will be called from main func and the variables can not be predicted we cant declare a struct before hand so i am stuck here. can anyone give me a solution?
so basically what i am trying to do is we can simply call the query like
-- select name,phone from users where userid=1 ---
so from the main func we will get the values like column name(name,phone), table name (users) and the condition (userid=1)
we will pass these info to the select func and it will run the query and give us back the results.
no matter what is the query is it should work like it. can anyone give me an sample or idea how to work with this
func select() {
//DB CONNECTION HERE
type User struct {
Name string `json:"name"`
Age string `json:"age"`
Email string `json:"email"`
Phone string `json:"phone"`
Address string `json:"address"`
}
results, err := db.Query("SELECT Name, Age, Email, Phone, Address FROM `users` where personId=12 ")
if err != nil {
panic(err.Error())
}
for results.Next() {
var user User
var email User
var age User
var phone User
var address User
err = results.Scan(&user.Name, &age.Age, &email.Email, &phone.Phone, &address.Address)
if err != nil {
panic(err.Error())
}
fmt.Println(user.Name)
fmt.Println(age.Age)
fmt.Println(email.Email)
fmt.Println(phone.Phone)
fmt.Println(address.Address)
}
Now as zou can see this is the code where we know the query so we made a struct according to it but what about the problem i described above?
I am handling user auth data posted to my Go backend through an HTML form. I am building on some boilerplate to learn Go better.
My problem is what the following func returns:
func (ctrl UserController) Signin(c *gin.Context) {
var signinForm forms.SigninForm
user, err := userModel.Signin(signinForm)
if err := c.ShouldBindWith(&signinForm, binding.Form); err != nil {
c.JSON(406, gin.H{"message": "Invalid signin form", "form": signinForm})
c.Abort()
return
}
if err == nil {
session := sessions.Default(c)
session.Set("user_id", user.ID)
session.Set("user_email", user.Email)
session.Set("user_name", user.Name)
session.Save()
c.JSON(200, gin.H{"message": "User signed in", "user": user})
} else {
c.JSON(406, gin.H{"message": "Invalid signin details", "error": err.Error()})
}
}
The first if statement validates the input, and that works fine (error if the email isn't in proper email format, no error if it is). However, if input is properly validated, the else clause of the second statement is triggered, and the following JSON is returned:
{
error: "sql: no rows in result set",
message: "Invalid signin details"
}
It is probably useful to also post the relevant code in my User model:
//User ...
type User struct {
ID int `db:"id, primarykey, autoincrement" json:"id"`
Email string `db:"email" json:"email"`
Password string `db:"password" json:"-"`
Name string `db:"name" json:"name"`
UpdatedAt int64 `db:"updated_at" json:"updated_at"`
CreatedAt int64 `db:"created_at" json:"created_at"`
}
//UserModel ...
type UserModel struct{}
//Signin ...
func (m UserModel) Signin(form forms.SigninForm) (user User, err error) {
err = db.GetDB().SelectOne(&user, "SELECT id, email, password, name, updated_at, created_at FROM public.user WHERE email=LOWER($1) LIMIT 1", form.Email)
if err != nil {
return user, err
}
bytePassword := []byte(form.Password)
byteHashedPassword := []byte(user.Password)
err = bcrypt.CompareHashAndPassword(byteHashedPassword, bytePassword)
if err != nil {
return user, errors.New("Invalid password")
}
return user, nil
}
How do I resolve the sql: no rows in result set error?
You should change the order of operations in your code.
At first you need to get data from request with if err := c.ShouldBindWith(&signinForm, binding.Form); err != nil { and after that you need to try get data from database with user, err := userModel.Signin(signinForm)
There are good reasons for this. Ideally it tries to read paths by the separator, meaning /path/abcd/ and /path/abcd are different. abcd IS the resource in the latter while the resource lies somewhere within abcd in the former. With that in mind, then it will not be able to route properly to /path/abcd if you also have a path /path as well. In order to remove that ambiguity as to which handler to use, you need to mention the handler for the more specific path ie, /path/abcd before the more generic one /path.