I need to execute some things after all create executions fail.
It seems that callbacks can be satisfied, but there is a case that if it is an operation in a transaction, it may not actually be executed. I need to do it after rollback Treat accordingly. So the question is, how do I intercept rollback?
You can use manual transaction in a function like this.
func CreateAnimals(db *gorm.DB) error {
// Note the use of tx as the database handle once you are within a transaction
tx := db.Begin()
defer func() {
if r := recover(); r != nil {
tx.Rollback()
}
}()
if err := tx.Error; err != nil {
return err
}
if err := tx.Create(&Animal{Name: "Giraffe"}).Error; err != nil {
tx.Rollback()
return err
}
if err := tx.Create(&Animal{Name: "Lion"}).Error; err != nil {
tx.Rollback()
return err
}
return tx.Commit().Error
}
If CreateAnimals fails then you can do your desired job.
Related
I am currently in the process of creating a little Go App. Right now I am working on the DB part. The Library I use is this one: https://github.com/jackc/pgx
The problem I have is that every time I try to execute the database read, it tells me that my 'conn is busy'. I read about using a pgxpool instead of a single connection, but it still does not work. What am I doing wrong?
func (postgre *PostgreClient) read(query string) (pgx.Row, error) {
client, err := postgre.client.Acquire(context.TODO())
transaction, err := client.BeginTx(context.TODO(), pgx.TxOptions{})
if err != nil {
return nil, err
}
defer transaction.Rollback(context.TODO())
rows := transaction.QueryRow(context.TODO(), query)
if err != nil {
return nil, err
}
err = transaction.Commit(context.TODO())
return rows, err
}
Thanks in advance.
You have to scan the row before you commit the transaction.
If you want the handling of the transaction to remain within the function you can pass an interface that does the scanning also inside the function.
For example:
// implemented by *sql.Row & *sql.Rows
type Row interface {
Scan(dst ...interface{}) error
}
// implemented by your "models"
type RowScanner interface {
ScanRow(r Row) error
}
type User struct {
Id int
Email string
}
func (u *User) ScanRow(r Row) error {
return r.Scan(
&u.Id,
&u.Email,
)
}
func (postgre *PostgreClient) read(query string, rs RowScanner) (err error) {
conn, err := postgre.client.Acquire(context.TODO())
if err != nil {
return err
}
defer conn.Release()
tx, err := conn.BeginTx(context.TODO(), pgx.TxOptions{})
if err != nil {
return err
}
defer func() {
if err != nil {
tx.Rollback(context.TODO())
} else {
tx.Commit(context.TODO())
}
}()
row := tx.QueryRow(context.TODO(), query)
if err != nil {
return nil, err
}
return rs.ScanRow(row)
}
u := new(User)
if err := pg.read("select id, email from users limit 1", u); err != nil {
panic(err)
}
For scanning a list of models:
type UserList []*User
func (ul *UserList) ScanRow(r Row) error {
u := new(User)
if err := u.ScanRow(r); err != nil {
return err
}
*ul = append(*ul, u)
return nil
}
func (postgre *PostgreClient) list(query string, rs RowScanner) (err error) {
conn, err := postgre.client.Acquire(context.TODO())
if err != nil {
return err
}
defer conn.Release()
tx, err := conn.BeginTx(context.TODO(), pgx.TxOptions{})
if err != nil {
return err
}
defer func() {
if err != nil {
tx.Rollback(context.TODO())
} else {
tx.Commit(context.TODO())
}
}()
rows, err := tx.Query(context.TODO(), query)
if err != nil {
return err
}
defer rows.Close()
for rows.Next() {
if err := rs.ScanRow(rows); err != nil {
return err
}
}
return rows.Err()
}
ul := new(UserList)
if err := pg.list("select id, email from users", ul); err != nil {
panic(err)
}
I have a test suite that pollute my database using a seed read from a YAML file.
I'm wondering is there a way to clean my database (delete all records used for the test suite) after running my tests.
// Open db and returns pointer and closer func
func prepareMySQLDB(t *testing.T) (db *sql.DB, closer func() error) {
db, err := sql.Open("mysql", "user:pass#/database")
if err != nil {
t.Fatalf("open mysql connection: %s", err)
}
return db, db.Close
}
// Pollute my database
func polluteDb(db *sql.DB, t *testing.T) {
seed, err := os.Open("seed.yml")
if err != nil {
t.Fatalf("failed to open seed file: %s", err)
}
defer seed.Close()
p := polluter.New(polluter.MySQLEngine(db))
if err := p.Pollute(seed); err != nil {
t.Fatalf("failed to pollute: %s", err)
}
}
func TestAllUsers(t *testing.T) {
t.Parallel()
db, closeDb := prepareMySQLDB(t)
defer closeDb()
polluteDb(db, t)
users, err := AllUsersD(db)
if err != nil {
t.Fatal("AllUsers() failed")
}
got := users[0].Email
if got != "myemail#gmail.com" {
t.Errorf("AllUsers().Email = %s; want myemail#gmail.com", got)
}
got1 := len(users)
if got1 != 1 {
t.Errorf("len(AllUsers()) = %d; want 1", got1)
}
}
// Test I'm interested in
func TestAddUser(t *testing.T) {
t.Parallel()
db, closeDb := prepareMySQLDB(t)
defer closeDb()
polluteDb(db, t)
user, err := AddUser(...)
if err != nil {
t.Fatal("AddUser() failed")
}
//how can I clean my database after this?
}
Should I retrieve the last ID inserted in TestAddUser() and just delete that line manually or there's any other way to save my database state and retrieve it after?
As I said I'm new to Go so any other comments on my code or what so ever are strongly appreciated.
The best way is usually to use a transaction, then ROLLBACK, so they are never committed in the first place.
The github.com/DATA-DOG/go-txdb package can help a lot with that.
Final code:
import (
"database/sql"
"os"
"testing"
txdb "github.com/DATA-DOG/go-txdb"
"github.com/romanyx/polluter"
)
//mostly sql tests
func init() {
txdb.Register("txdb", "mysql", "root:root#/betell_rest")
}
func TestAddUser(t *testing.T) {
db, err := sql.Open("txdb", "root:root#/betell_rest")
if err != nil {
t.Fatal(err)
}
defer db.Close()
users, _ := AllUsers(db)
userscount := len(users)
err = AddUser(db, "bla#gmail.com", "pass")
if err != nil {
t.Fatal("AddUser() failed")
}
users, _ = AllUsers(db)
if (userscount + 1) != len(users) {
t.Fatal("AddUser() failed to write in database")
}
}
Note: Also you can pass db into your polluter so you don't affect your database at all.
In code which tries to be database agnostic, I would like to perform some database specific queries, so I need to know name of Database Driver in Go language:
db,err := sql.Open(dbstr, dbconnstr)
if err != nil {
log.Fatal(err)
}
errp := db.Ping()
if errp != nil {
log.Fatal(errp)
}
log.Printf("%s\n", db.Driver())
How I can determine name of database driver I'm using?
Give your database string in url format like postgres://postgres#localhost:5432/db_name?sslmode=disable.
And then find the database type you are using Parse function of url package. Based on the database type, run db specific queries.
func New(url string) (Driver, error) {
u, err := neturl.Parse(url)
if err != nil {
return nil, err
}
switch u.Scheme {
case "postgres":
d := &postgres.Driver{}
if err := d.Initialize(url); err != nil {
return nil, err
}
return d, nil
case "mysql":
d := &mysql.Driver{}
if err := d.Initialize(url); err != nil {
return nil, err
}
return d, nil
case "bash":
d := &bash.Driver{}
if err := d.Initialize(url); err != nil {
return nil, err
}
return d, nil
case "cassandra":
d := &cassandra.Driver{}
if err := d.Initialize(url); err != nil {
return nil, err
}
return d, nil
case "sqlite3":
d := &sqlite3.Driver{}
if err := d.Initialize(url); err != nil {
return nil, err
}
return d, nil
default:
return nil, errors.New(fmt.Sprintf("Driver '%s' not found.", u.Scheme))
}
}
You should already know the name of the database driver because its represented by the parameter you identified with the dbstr variable.
db, err := sql.Open("postgres", "user= ... ")
if err != nil {
log.Fatal(err)
}
db.Driver() correctly returns the underlying driver in use, but you are formatting it as string (because of %s). If you change %s with %T you will see that it correctly prints out the type:
log.Printf("%T\n", db.Driver())
For example, if you use github.com/lib/pq, the output is *pq.drv. This is the same of using the reflect package:
log.Printf("%s\n", reflect.TypeOf(db.Driver()))
It may be impractical to use that value for performing conditional executions. Moreover, the Driver interface doesn't specify any way to get the specific driver information, except the Open() function.
If you have specific needs, you may want to either use the driver name passed when you open the connection, or create specific drivers that delegate to the original ones and handle your custom logic.
I want to execute a shell script from Go.
The shell script takes standard input and echoes the result.
I want to supply this input from GO and use the result.
What I am doing is:
cmd := exec.Command("python","add.py")
in, _ := cmd.StdinPipe()
But how do I read from in?
Here is some code writing to a process, and reading from it:
package main
import (
"bufio"
"fmt"
"os/exec"
)
func main() {
// What we want to calculate
calcs := make([]string, 2)
calcs[0] = "3*3"
calcs[1] = "6+6"
// To store the results
results := make([]string, 2)
cmd := exec.Command("/usr/bin/bc")
in, err := cmd.StdinPipe()
if err != nil {
panic(err)
}
defer in.Close()
out, err := cmd.StdoutPipe()
if err != nil {
panic(err)
}
defer out.Close()
// We want to read line by line
bufOut := bufio.NewReader(out)
// Start the process
if err = cmd.Start(); err != nil {
panic(err)
}
// Write the operations to the process
for _, calc := range calcs {
_, err := in.Write([]byte(calc + "\n"))
if err != nil {
panic(err)
}
}
// Read the results from the process
for i := 0; i < len(results); i++ {
result, _, err := bufOut.ReadLine()
if err != nil {
panic(err)
}
results[i] = string(result)
}
// See what was calculated
for _, result := range results {
fmt.Println(result)
}
}
You might want to read/write from/to the process in different goroutines.
Using the database/sql and driver packages and Tx, it is not possible it appears to detect whether a transaction has been committed or rolled-back without attempting another and receiving an error as a result, and then examining the error to determine the type of error. I would like to be able to determine from the Tx object whether committed or not. Sure, I can define and set another variable in the function that uses Tx, but I have quite a number of them, and it is times 2 every time (variable and assignment). I also have a deferred function to do a Rollback if needed, and it needs to be passed the bool variable.
Would it be acceptable to set the Tx variable to nil after a Commit or Rollback, and will the GC recover any memory, or is that a no-no, or is there a better alternative?
You want to make sure that Begin(), Commit(), and Rollback() appear within the same function. It makes transactions easier to track, and lets you ensure they are closed properly by using a defer.
Here is an example of this, which does a Commit or Rollback depending on whether an error is returned:
func (s Service) DoSomething() (err error) {
tx, err := s.db.Begin()
if err != nil {
return
}
defer func() {
if err != nil {
tx.Rollback()
return
}
err = tx.Commit()
}()
if _, err = tx.Exec(...); err != nil {
return
}
if _, err = tx.Exec(...); err != nil {
return
}
// ...
return
}
This can get a bit repetitive. Another way of doing this is by wrapping your transactions using a transaction handler:
func Transact(db *sql.DB, txFunc func(*sql.Tx) error) (err error) {
tx, err := db.Begin()
if err != nil {
return
}
defer func() {
if p := recover(); p != nil {
tx.Rollback()
panic(p) // re-throw panic after Rollback
} else if err != nil {
tx.Rollback() // err is non-nil; don't change it
} else {
err = tx.Commit() // err is nil; if Commit returns error update err
}
}()
err = txFunc(tx)
return err
}
Using the transaction hander above, I can do this:
func (s Service) DoSomething() error {
return Transact(s.db, func (tx *sql.Tx) error {
if _, err := tx.Exec(...); err != nil {
return err
}
if _, err := tx.Exec(...); err != nil {
return err
}
return nil
})
}
This keeps my transactions succinct and ensures by transactions are properly handled.
In my transaction handler I use recover() to catch panics to ensure a Rollback happens right away. I re-throw the panic to allow my code to catch it if a panic is expected. Under normal circumstances a panic should not occur. Errors should be returned instead.
If we did not handle panics the transaction would be rolled back eventually. A non-commited transaction gets rolled back by the database when the client disconnects or when the transaction gets garbage collected. However, waiting for the transaction to resolve on its own could cause other (undefined) issues. So it's better to resolve it as quickly as possible.
One thing that may not be immediately clear is that defer can change the return value within a closure if the return variable is captured. In the transaction handler the transaction is committed when err (the return value) is nil. The call to Commit can also return an error, so we set its return to err with err = tx.Commit(). We do not do the same with Rollback because err is non-nil and we do not want to overwrite the existing error.