how to query for multiples rows and parse to json? - sql

I have this code:
func GetAll(c *gin.Context) {
var veiculos model.Veiculo
rows, err := db.Query("SELECT * FROM vei_veiculo")
if err != nil {
fmt.Println("[GetAll] erro ao abrir o query db inteiro")
}
defer rows.Close()
for rows.Next() {
err := rows.Scan(&veiculos)
if err != nil {
fmt.Println("[GetAll] erro ao scanear uma linha'")
}
}
fmt.Println(veiculos)}
My struct name is at model.Veiculo and I want to print it all once. It seems there is an error on scan the query rows. What did I do wrong?

Assuming you are using database/sql, the function signature is func (rs *Rows) Scan(dest ...interface{}) error.
You need to be doing scanning into each member of the struct, something more like:
err := rows.Scan(&veiculos.ID, &veiculos.Name, &veiculos.Description)

To scan each row to its own struct, I recommend an approach like so:
package main
import (
"database/sql"
"fmt"
"log"
_ "github.com/go-sql-driver/mysql"
)
type Person struct {
FirstName string
LastName string
Email string
}
func main() {
db, err := sql.Open("mysql", "dsn...")
if err != nil {
log.Fatalln(err)
}
defer db.Close()
people := []Person{}
rows, err := db.Query("SELECT * FROM people")
if err != nil {
log.Fatalln(err)
}
for rows.Next() {
person := Person{}
if err := rows.Scan(&person.FirstName, &person.LastName, &person.Email); err != nil {
log.Fatalln(err)
}
people = append(people, person)
}
fmt.Printf("%+v\n", people)
}
Alternatively, the library https://github.com/jmoiron/sqlx exists as one of the solutions to marshaling SQL query data to structs. I personally prefer this method, as it behaves more like an Unmarshal function:
package main
import (
"fmt"
"log"
_ "github.com/go-sql-driver/mysql"
"github.com/jmoiron/sqlx"
)
type Person struct {
FirstName string `db:"first_name"`
LastName string `db:"last_name"`
Email string `db:"email"`
}
func main() {
db, err := sqlx.Connect("mysql", "dsn...")
if err != nil {
log.Fatalln(err)
}
people := []Person{}
if err := db.Select(&people, "SELECT * FROM people"); err != nil {
log.Fataln(err)
}
fmt.Printf("%+v\n", people)
}
Following either of these approaches allows you to easily use the json.Marshal
method with the people variable, and viola!
Good luck!

Related

Can I return rows result from db without scan it first in Golang func?

I have mini project using Golang, my plan is make a base function which it will be called from Model to execute sql query, then return the rows result without Scan it first. I'm using this way to prevent forget write defer rows.Close() and the code for execute the Query in model more simple. I had tried this way, but when try to print the result, I got nil without any error. here my screenshoot. The result exists when the query executed and the rows result scanned are in same function. Maybe I miss something? This is my first question, sorry it's too long. Thank you
The base model where the SQL query will be executed and return the result
package model
import "database/sql"
import "hb-backend-v1/config/database"
import "fmt"
func Query(query string) (*sql.Rows, error){
connect, err := database.Connect()
if err != nil{
fmt.Println("Connection Failed")
return nil, err
}
fmt.Println("Connection Success")
defer connect.Close()
rows, err := connect.Query(query)
defer rows.Close()
if err != nil{
return nil, err
}
return rows, nil
}
This is where the base model will be called and give the result
package product
import "database/sql"
import _"fmt"
import "hb-backend=v1/model"
type Hasil struct{
Id_alamat_store int
Id_tk int
Alamat string
Id_wil int
Latitude sql.NullString
Longitude sql.NullString
}
func ProductList() ([]Hasil, error){
rows, err := model.Query("SELECT * FROM alamat_store")
if err != nil{
return nil, err
}
var result []Hasil
for rows.Next(){
var each = Hasil{}
var err = rows.Scan(&each.Id_alamat_store, &each.Id_tk, &each.Alamat, &each.Id_wil, &each.Latitude, &each.Longitude)
if err != nil{
return nil, err
}
result = append(result, each)
}
return result, nil
}
Both connection and rows will be closed once Query exits, after those two are closed you can't use rows anymore.
One approach to get around that would be to pass a closure to Query and have Query execute it before closing the two resources:
func Query(query string, scan func(*sql.Rows) error) error {
connect, err := database.Connect()
if err != nil{
return err
}
defer connect.Close()
rows, err := connect.Query(query)
if err != nil{
return err
}
defer rows.Close()
return scan(rows)
}
func ProductList() ([]Hasil, error) {
var result []Hasil
err := model.Query("SELECT * FROM alamat_store", func(rows *sql.Rows) error {
for rows.Next() {
var each = Hasil{}
var err = rows.Scan(&each.Id_alamat_store, &each.Id_tk, &each.Alamat, &each.Id_wil, &each.Latitude, &each.Longitude)
if err != nil {
return err
}
result = append(result, each)
}
return nil
})
if err != nil {
return nil, err
}
return result, nil
}

VS Code how to format a few lines of SQL

I have a main.go file with a sql query saved as a string.
defer db.Close()
id := 1
// format this query
row := db.QueryRow("SELECT f_name, l_name from users WHERE id = ?", id)
var fname, lname string
err = row.Scan(&fname, &lname)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%v %v\n", fname, lname)
Is there a way to format that line?
I have also had the same problem. For me, the best solution is to move the SQL query to an embed file and format it there. This also has the advantage of making more complex or larger SQL queries more readable.
For *.sql files can than be a plugin used. For example Language PL/SQL.
Here is an example:
Excerpt of a repository (sqlite3,postgres,...)
package repository
import (
"context"
"database/sql"
_ "embed"
)
//go:embed dml/selectUsers.sql
var selectUsers string
type Repository struct{
sqlDB *sql.DB
}
func (r *Repository) GetUsers()([]*User,error){
tx, err := s.sqlDb.BeginTx(ctx, &sql.TxOptions{ReadOnly: false})
if err != nil {
return nil, fmt.Errorf("failed to begin new transaction: %w", err)
}
defer tx.Close()
selectedUsers, err := r.selectUsers(tx, selectUsers)
if err != nil {
return nil, fmt.Errorf("failed to select users: %w", err)
}
err = tx.Commit()
if err != nil {
return nil, fmt.Errorf("failed to commit transaction: %w", err)
}
return selectedUsers, nil
}
func (r *Repository) selectUsers(tx *sql.Tx, query string, args ...interface{}) ([]*User,error) {
stmt, err := tx.Prepare(insertCompany)
if err != nil {
return nil, fmt.Errorf("failed to prepare statement: %v", err)
}
defer stmt.Close()
rows, err := stmt.Query(args...)
if err != nil {
return nil, fmt.Errorf("failed to query statement: %v", err)
}
selectedUsers := make([]*User, 0)
for rows.Next() {
selectedUser := new(User)
err = rows.Scan(
&selectedUser.ID,
&selectedUser.Surename,
&selectedUser.Familyname,
)
if err != nil {
return nil, fmt.Errorf("failed to scan row: %v", err)
}
selectedUsers = append(selectedUsers, selectedUser)
}
return selectedUsers, nil
}
dml/selectUsers.sql
SELECT
id,
surename,
familyname
FROM
users;

How to handle nullable Postgres JSONB data and parse it as JSON

DB Records
---------------------------------------------------------
| id | test_json |
---------------------------------------------------------
| 1 | NULL |
---------------------------------------------------------
| 2 | { "firstName": "Hello", "lastName": "World" } |
---------------------------------------------------------
I have JSONB column in postgres which can be NULL. I want to read this records DB in golang and send it to the client.
I'm getting below error on SQL scan:
sql: Scan error on column index 2, name "test_json": unsupported Scan, storing driver.Value type []uint8 into type *models.TestJSONNullable
exit status 1
I'm using echo web server.
package models
import (
"fmt"
"github.com/lib/pq"
"encoding/json"
)
type TestJson struct {
First_name *string `json:"firstName"`
Last_name *string `json:"lastName"`
}
type TestJSONNullable struct {
Valid bool
}
func (i *TestJSONNullable) UnmarshalJSON(data []byte) error {
if string(data) == "null" {
i.Valid = false
return nil
}
// The key isn't set to null
var temp *TestJson
if err := json.Unmarshal(data, &temp); err != nil {
return err
}
i.Valid = true
return nil
}
type Test01 struct {
Id string `json:"id"`
Test_json *TestJSONNullable `json:"testJson"`
}
func (db *DB) TestRecords () ([]*Test01, error) {
rows, err := db.Query("SELECT id, test_json FROM table_1 where success = true")
if err != nil {
log.Fatal(err)
return nil, err
}
defer rows.Close()
recs := []*Test01{}
for rows.Next() {
r := new(Test01)
err := rows.Scan(&r.Id, &r.Test_json)
if err != nil {
log.Fatal(err)
return nil, err
}
recs = append(recs, r)
}
if err = rows.Err(); err != nil {
log.Fatal(err)
return nil, err
}
return recs, nil
}
Here's another solution: You can implement a nullable type for raw JSON data, similar to sql.NullString, and use it as a scan destination. In this scenario, you will first check whether the value is null, and then unmarshal it only if it's not null. For instance, the NullRawMessage type from github.com/soroushj/sqlt is a nullable json.RawMessage which can be used for this purpose.
Here's an example:
package main
import (
"database/sql"
"log"
_ "github.com/lib/pq"
"github.com/soroushj/sqlt"
)
func main() {
db, err := sql.Open("postgres", "dbname=dbname user=user password=password sslmode=disable")
if err != nil {
log.Fatal(err)
}
row := db.QueryRow(`SELECT test_json FROM my_table WHERE id = $1`, 1)
testJSON := sqlt.NullRawMessage{}
err = row.Scan(&testJSON)
if err != nil {
log.Fatal(err)
}
if testJSON.Valid {
// test_json is not null
// Unmarshal testJSON.RawMessage
} else {
// test_json is null
}
}
After doing some research i found the solution.
type TestJSONMap map[string]interface{}
func (t TestJSONMap) Value() (driver.Value, error) {
j, err := json.Marshal(t)
return j, err
}
func (p *TestJSONMap) Scan(val interface{}) error {
value, ok := val.([]byte)
if !ok {
return errors.New("Type assertion .([]byte) failed.")
}
var i interface{}
err := json.Unmarshal(value, &i)
if err != nil {
return err
}
*p, ok = i.(map[string]interface{})
if !ok {
return errors.New("Type assertion .(map[string]interface{}) failed.")
}
return nil
}
type Test01 struct {
Id string `json:"id"`
Test_json *TestJSONMap `json:"testJson"`
}
Got help from https://coussej.github.io/2016/02/16/Handling-JSONB-in-Go-Structs/
Instead of using struct to store values, You can use map.
type TestJson struct {
First_name *string `json:"firstName"`
Last_name *string `json:"lastName"`
}
You can use interface as
var TestJson interface{}
err := json.Unmarshal(b, &TestJson)
On the other side you can also use Dynamic creation of structs as well.
Something like
m := map[string]interface{}{
"key": "value",
}
And rather than having TestJSONNullable it would be better to have Switch case while saving data.
switch v := TestJson.(type) {
case int:
case float64:
case string:
default:
// i isn't one of the types above
}
Look into this for more details--> https://godoc.org/encoding/json#Unmarshal

How to avoid using for next method in getting a row count

In php, I can print rowcount where postid matches with the code below without passing the result in a while loop.
$status_query = "SELECT count(*) as postCount FROM postData WHERE postid=1";
$status_result = mysqli_query($con,$status_query);
$status_row = mysqli_fetch_array($status_result);
$postCount = $status_row['postCount'];
echo $postCount;
Now am re-writing the code to golang for the same row count. I leverage Stackoverflow solution found here.
source link
With that Stackoverflow solution link above, the golang code below is working great as I can get the row count.
rows1, err := db.Query("SELECT COUNT(*) as postCount FROM postData WHERE postid=1")
if err != nil {
log.Fatal(err)
}
defer rows1.Close()
var postCount int
for rows1.Next() {
if err := rows1.Scan(& postCount); err != nil {
log.Fatal(err)
}
}
fmt.Printf("Number of rows are %s\n", postCount)
Here is what I want modify:
The above code passed the row count result within
for rows1. Next(){
// result here.
}
My question is:
Please how do I avoid this for rows.next() function and just get my result straight since am retrieving the row count based on postid.
In php code above, I can get the result straight without passing it in a while loop.
In golang am thinking of something like the code below
rows1, err := db.Query("SELECT COUNT(*) as postCount FROM postData WHERE postid=1")
if err != nil {
log.Fatal(err)
}
defer rows1.Close()
var status_row = rows1.Next()
var postCount =rows1.Scan(& postCount)
fmt.Printf("Number of rows are %s\n", postCount)
Does anyone has a better way of getting this rowcount to display straight without passing the result within for rows1.Next() method
Here is the overall working code before I seek modification in the coding illustrations above.
package main
import "database/sql"
import _ "github.com/go-sql-driver/mysql"
import "net/http"
import "fmt"
import "log"
var db *sql.DB
var err error
func getRecordPage1(res http.ResponseWriter, req *http.Request) {
if req.Method != "POST" {
http.ServeFile(res, req, "getjsonRecord.html")
return
}
//The commented section below is the code I want to modify to avoid use of for rows.next function....
/*
rows1, err := db.Query("SELECT COUNT(*) as postCount FROM like_unlike WHERE postid=1")
if err != nil {
log.Fatal(err)
}
defer rows1.Close()
var postCount int
for rows1.Next() {
if err := rows1.Scan(& postCount); err != nil {
log.Fatal(err)
}
}
fmt.Printf("Number of rows are %s\n", postCount)
}
*/
func homePage(res http.ResponseWriter, req *http.Request) {
http.ServeFile(res, req, "index.html")
}
func main() {
db, err = sql.Open("mysql", "root:#/golang44")
if err != nil {
panic(err.Error())
}
defer db.Close()
err = db.Ping()
if err != nil {
panic(err.Error())
}
http.HandleFunc("/getjsonRecord", getRecordPage1)
http.HandleFunc("/", homePage)
fmt.Println("Listening on 127.0.0.1:8088")
http.ListenAndServe(":8088", nil)
}
Generally, if you know you are getting one row, use DB.QueryRow
It allows you to chain the query and scan together, so your example would look like:
var postCount int
err := db.QueryRow("SELECT COUNT(*) as postCount FROM postData WHERE postid=1").Scan(&postCount)
if err != nil {
log.Fatal(err)
}

Can I get EXPLAIN ANALYZE output through the lib/pq Go SQL driver?

I'd like to be able to evaluate my queries inside my app, which is in Go and using the github.com/lib/pq driver. Unfortunately, neither the [lib/pq docs][1] nor the [database/sql][2] docs seem to say anything about this, and nothing in the database/sql interfaces suggests this is possible.
Has anyone found a way to get this output?
Typical EXPLAIN ANALYZE returns several rows, so you can do it with simple sql.Query. Here is an example:
package main
import (
"database/sql"
"fmt"
_ "github.com/lib/pq"
"log"
)
func main() {
db, err := sql.Open("postgres", "user=test dbname=test sslmode=disable")
if err != nil {
log.Fatal(err)
}
defer db.Close()
rows, err := db.Query("EXPLAIN ANALYZE SELECT * FROM accounts ORDER BY slug")
if err != nil {
log.Fatal(err)
}
for rows.Next() {
var s string
if err := rows.Scan(&s); err != nil {
log.Fatal(err)
}
fmt.Println(s)
}
if err := rows.Err(); err != nil {
log.Fatal(err)
}
}