How to do transaction Management - sql

Question

I have found a lot of golang tutorials on how to handle transactions, clean architecture, but none has satisfied my needs.
I have a go web application with a series of rest API each of one served by a three layers structure:
API
Service
Dao
DB
This is the main :
func main() {
DB = db.New()
dao = dao.New(DB)
service := service.New(dao)
server := &http.Server(....)
server.ListenAndServe()
}
Now, what I would realize is a transaction manager in the service layer.
Something like this:
type Service interface {
TransactionalExecute(fn func() error)
}
type ServiceImpl struct {
dao DaoImpl
}
func (service *ServiceImpl) TransactionalExecute(fn func() error) {
service.dao.TxBegin()
err := fn()
service.dao.TxEnd(err)
}
Dao should be like this:
type Dao interface {
TxBegin()
TxEnd(err error)
}
type DaoImpl struct {
DB db
tx *sql.Tx
}
func (dao *DaoImpl) TxBegin() {
dao.tx = dao.db.Begin()
}
func (dao *DaoImpl) TxEnd(err error) {
if p:= recover(); p != nil {
dao.tx.Rollback()
panic(p)
} else if err != nil {
dao.tx.Rollback()
} else {
dao.tx.Commit()
}
}
This kind of POC has two problems :
fn func() error parameter passed in the transactionalExecute() method must use the dao.Tx variable of the dao instance
This approach is not thread-safe : I am currently using gorilla mux and each http request will start a goroutine using a single instance of service, dao and DB. These instances are shared between multiple threads and is not safe. Anyway I was thinking about the use of mutex to access to the dao.tx variable but I am concerned about performances.
Any suggestion? Or different approaches to the problem?

Answer 1

Transactions should not really be part of DAO layer, it's a completely separate concern. A transaction is simply a single connection that executes all statements atomically. DAO, on the other hand, is a collection of database functions which use connections/transactions.
You can implement transactions and DAO nicely as two separate interfaces. Transaction can be an interface that is implemented both by sql.DB (connection pool from the standard library) and sql.Tx (transaction).
// Transaction represents both database connection pool and a single transaction/connection
type Transaction interface {
// ExecContext from database/sql library
ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
// PrepareContext from database/sql library
PrepareContext(ctx context.Context, query string) (*sql.Stmt, error)
// QueryContext from database/sql library
QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
// QueryRowContext from database/sql library
QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row
// QueryxContext from sqlx library
QueryxContext(ctx context.Context, query string, args ...interface{}) (*sqlx.Rows, error)
// QueryRowxContext from sqlx library
QueryRowxContext(ctx context.Context, query string, args ...interface{}) *sqlx.Row
// PreparexContext from sqlx library
PreparexContext(ctx context.Context, query string) (*sqlx.Stmt, error)
// GetContext from sqlx library
GetContext(ctx context.Context, dest interface{}, query string, args ...interface{}) error
// SelectContext from sqlx library
SelectContext(ctx context.Context, dest interface{}, query string, args ...interface{}) error
}
We use both database/sql and sqlx extended library for easier data scanning, you can just drop sqlx functions if you do not need them. Then a helper function for easier work:
// A TxFn is a function that will be called with an initialized `Transaction` object
// that can be used for executing statements and queries against a database.
type TxFn func(Transaction) error
// WithTransaction creates a new transaction and handles rollback/commit based on the
// error object returned by the `TxFn` or when it panics.
func WithTransaction(ctx context.Context, db *sqlx.DB, fn TxFn) error {
logger := ctxval.GetLogger(ctx)
tx, err := db.BeginTxx(ctx, &sql.TxOptions{
Isolation: sql.LevelDefault,
ReadOnly: false,
})
if err != nil {
logger.Error().Err(err).Msg("Cannot begin database transaction")
return fmt.Errorf("cannot begin transaction: %w", err)
}
defer func() {
if p := recover(); p != nil {
logger.Trace().Msg("Rolling database transaction back")
err := tx.Rollback()
if err != nil {
logger.Error().Err(err).Msg("Cannot rollback database transaction")
return
}
panic(p)
} else if err != nil {
logger.Trace().Msg("Rolling database transaction back")
err := tx.Rollback()
if err != nil {
logger.Error().Err(err).Msg("Cannot rollback database transaction")
return
}
} else {
logger.Trace().Msg("Committing database transaction")
err = tx.Commit()
if err != nil {
logger.Error().Err(err).Msg("Cannot rollback database transaction")
return
}
}
}()
logger.Trace().Msg("Starting database transaction")
err = fn(tx)
return err
}
Then you need a DAO interface, we have an initializer as a function that is then initialized by individual implementations, it takes context and transaction and returns DAO implementation:
var GetAccountDao func(ctx context.Context, tx Transaction) (AccountDao, error)
type AccountDao interface {
GetById(ctx context.Context, id uint64) (*models.Account, error)
GetByAccountNumber(ctx context.Context, number string) (*models.Account, error)
GetByOrgId(ctx context.Context, orgId string) (*models.Account, error)
List(ctx context.Context, limit, offset uint64) ([]*models.Account, error)
}
Implementation looks something like this:
const (
getAccountById = `SELECT * FROM accounts WHERE id = $1 LIMIT 1`
)
type accountDaoSqlx struct {
name string
getById *sqlx.Stmt
}
func getAccountDao(ctx context.Context, tx dao.Transaction) (dao.AccountDao, error) {
var err error
daoImpl := accountDaoSqlx{}
daoImpl.name = "account"
daoImpl.getById, err = tx.PreparexContext(ctx, getAccountById)
if err != nil {
return nil, NewPrepareStatementError(ctx, &daoImpl, getAccountById, err)
}
return &daoImpl, nil
}
func init() {
dao.GetAccountDao = getAccountDao
}
func (dao *accountDaoSqlx) GetById(ctx context.Context, id uint64) (*models.Account, error) {
query := getAccountById
stmt := dao.getById
result := &models.Account{}
err := stmt.GetContext(ctx, result, id)
if err != nil {
return nil, NewGetError(ctx, dao, query, err)
}
return result, nil
}
// etc
And example use in a "service" package (without and with transaction):
// import sqlx implementation which calls init()
_ "internal/dao/sqlx"
"internal/dao"
// etc
func CreatePubkey(w http.ResponseWriter, r *http.Request) {
payload := &payloads.PubkeyRequest{}
if err := render.Bind(r, payload); err != nil {
renderError(w, r, payloads.NewInvalidRequestError(r.Context(), err))
return
}
// example with transaction
err := dao.WithTransaction(r.Context(), db.DB, func(tx dao.Transaction) error {
pkDao, err := dao.GetPubkeyDao(r.Context(), tx)
if err != nil {
return payloads.NewInitializeDAOError(r.Context(), "pubkey DAO", err)
}
err = pkDao.Create(r.Context(), payload.Pubkey)
if err != nil {
return payloads.NewDAOError(r.Context(), "create pubkey", err)
}
// ... cut ...
return nil
})
// ignore errors etc
}
func ListPubkeys(w http.ResponseWriter, r *http.Request) {
// example without transaction
pubkeyDao, err := dao.GetPubkeyDao(r.Context(), db.DB)
if err != nil {
renderError(w, r, payloads.NewInitializeDAOError(r.Context(), "pubkey DAO", err))
return
}
pubkeys, err := pubkeyDao.List(r.Context(), 100, 0)
if err != nil {
renderError(w, r, payloads.NewDAOError(r.Context(), "list pubkeys", err))
return
}
if err := render.RenderList(w, r, payloads.NewPubkeyListResponse(pubkeys)); err != nil {
renderError(w, r, payloads.NewRenderError(r.Context(), "list pubkeys", err))
return
}
}
This is incomplete code I did cut details. Note I have both these interfaces in our dao package, these might be either separate or the same package.
One important thing to know: you must not share DAO implementation just like that (as in your snippet). The standard library sql.DB interface is actually a connection pool and it is thread safe (as documented), you can call as many Query/Exec/Etc function as you want, the driver will use one or more connection as needed.
Therefore the correct approach is to create new DAO type everytime a new request comes in, the connection pool will automatically acquire existing or new connection. More about this in the Go documentation.

Answer 2

you can modify the package dao to look like the following, which will make it thread-safe as func NewDao will be called in handler. you make this work you need to pass db *sql.DB from package to handler.
type Dao interface {
TxEnd(err error)
}
type DaoImpl struct {
tx *sql.Tx
db *sql.DB
txStared bool
}
func NewDao(db *sql.DB, txStared bool) Dao {
if txStared {
tx, _ := db.Begin()
return &DaoImpl{tx: tx, db: db, txStared: txStared}
}
return &DaoImpl{tx: nil, db: db, txStared: txStared}
}
func (dao *DaoImpl) TxEnd(err error) {
if !dao.txStared {
return
}
if p:= recover(); p != nil {
dao.tx.Rollback()
panic(p)
} else if err != nil {
dao.tx.Rollback()
} else {
dao.tx.Commit()
}
}
}