Go validator with sql null types? - sql

I am having problems getting the golang validator to work with SQL null types. Here's an example of what I tried:
package main
import (
"database/sql"
"database/sql/driver"
"log"
"gopkg.in/go-playground/validator.v9"
)
// NullInt64
type NullInt64 struct {
sql.NullInt64
Set bool
}
func MakeNullInt64(valid bool, val int64) NullInt64 {
n := NullInt64{}
n.Set = true
n.Valid = valid
if valid {
n.Int64 = val
}
return n
}
func (n *NullInt64) Value() (driver.Value, error) {
if !n.NullInt64.Valid {
return nil, nil
}
return n.NullInt64.Int64, nil
}
type Thing struct {
N2 NullInt64 `validate:"min=10"`
N3 int64 `validate:"min=10"`
N4 *int64 `validate:"min=10"`
}
func main() {
validate := validator.New()
n := int64(6)
number := MakeNullInt64(true, n)
thing := Thing{number, n, &n}
e := validate.Struct(thing)
log.Println(e)
}
When I run this code, I only get this output:
Key: 'Thing.N3' Error:Field validation for 'N3' failed on the 'min'
tag
Key: 'Thing.N4' Error:Field validation for 'N4' failed on the
'min' tag
The problem is that I want it to also show that Thing.N2 failed for the same reasons as Thing.N3 and Thing.N4.
I tried introducing the func (n *NullInt64) Value() method because it was mentioned in the documentation. But I think I misunderstood something. Can anyone tell me what I did wrong?

UPDATE
There is an Example specifically for that. You may check it out. My other proposed solution should still work though.
Since the value you are trying to validate is Int64 inside sql.NullInt64, the easiest way would be to remove the validate tag and just register a Struct Level validation using:
validate.RegisterStructValidation(NullInt64StructLevelValidation, NullInt64{})
while NullInt64StructLevelValidation is a StructLevelFunc that looks like this:
func NullInt64StructLevelValidation(sl validator.StructLevel) {
ni := sl.Current().Interface().(NullInt64)
if ni.NullInt64.Int64 < 10 {
sl.ReportError(ni.NullInt64.Int64, "Int64", "", "min", "")
}
}
Note #1: this line thing := Thing{number,&number,n,&n} has one argument too many. I assume you meant thing := Thing{number, n, &n}
Note #2: Go tooling including gofmt is considered to be one of the most powerful features of the language. Please consider using it/them.
EDIT #1:
I don't think implementing Valuer interface is of any value in this context.

Related

Get pointers to all fields of a struct dynamically using reflection

I'm trying to build a simple orm layer for golang.
Which would take a struct and generate the cols [] which can then be passed to sql function
rows.Scan(cols...) which takes pointers of fields in the struct corresponding to each of the columns it has found in the result set
Here is my example struct
type ExampleStruct struct {
ID int64 `sql:"id"`
aID string `sql:"a_id"`
UserID int64 `sql:"user_id"`
And this is my generic ORM function
func GetSqlColumnToFieldMap(model *ExampleStruct) map[string]interface{} {
typeOfModel := reflect.TypeOf(*model)
ValueOfModel := reflect.ValueOf(*model)
columnToDataPointerMap := make(map[string]interface{})
for i := 0; i < ValueOfModel.NumField(); i++ {
sql_column := typeOfModel.Field(i).Tag.Get("sql")
structValue := ValueOfModel.Field(i)
columnToDataPointerMap[sql_column] = structValue.Addr()
}
return columnToDataPointerMap
}
Once this method works fine i can use the map it generates to create an ordered list of sql pointers according to the column_names i get in rows() object
However i get below error on the .Addr() method call
panic: reflect.Value.Addr of unaddressable value [recovered]
panic: reflect.Value.Addr of unaddressable value
Is it not possible to do this ?
Also in an ideal scenario i would want the method to take an interface instead of *ExampleStruct so that it can be reused across different db models.
The error says the value whose address you want to get is unaddressable. This is because even though you pass a pointer to GetSqlColumnToFieldMap(), you immediately dereference it and work with a non-pointer value later on.
This value is wrapped in an interface{} when passed to reflect.ValueOf(), and values wrappped in interfaces are not addressable.
You must not dereference the pointer, but instead use Type.Elem() and Value.Elem() to get the element type and pointed value.
Something like this:
func GetSqlColumnToFieldMap(model *ExampleStruct) map[string]interface{} {
t := reflect.TypeOf(model).Elem()
v := reflect.ValueOf(model).Elem()
columnToDataPointerMap := make(map[string]interface{})
for i := 0; i < v.NumField(); i++ {
sql_column := t.Field(i).Tag.Get("sql")
structValue := v.Field(i)
columnToDataPointerMap[sql_column] = structValue.Addr()
}
return columnToDataPointerMap
}
With this simple change it works! And it doesn't depend on the parameter type, you may change it to interface{} and pass any struct pointers.
func GetSqlColumnToFieldMap(model interface{}) map[string]interface{} {
// ...
}
Testing it:
type ExampleStruct struct {
ID int64 `sql:"id"`
AID string `sql:"a_id"`
UserID int64 `sql:"user_id"`
}
type Point struct {
X int `sql:"x"`
Y int `sql:"y"`
}
func main() {
fmt.Println(GetSqlColumnToFieldMap(&ExampleStruct{}))
fmt.Println(GetSqlColumnToFieldMap(&Point{}))
}
Output (try it on the Go Playground):
map[a_id:<*string Value> id:<*int64 Value> user_id:<*int64 Value>]
map[x:<*int Value> y:<*int Value>]
Note that Value.Addr() returns the address wrapped in a reflect.Value. To "unwrap" the pointer, use Value.Interface():
func GetSqlColumnToFieldMap(model interface{}) map[string]interface{} {
t := reflect.TypeOf(model).Elem()
v := reflect.ValueOf(model).Elem()
m := make(map[string]interface{})
for i := 0; i < v.NumField(); i++ {
colName := t.Field(i).Tag.Get("sql")
field := v.Field(i)
m[colName] = field.Addr().Interface()
}
return m
}
This will output (try it on the Go Playground):
map[a_id:0xc00007e008 id:0xc00007e000 user_id:0xc00007e018]
map[x:0xc000018060 y:0xc000018068]
For an in-depth introduction to reflection, please read blog post: The Laws of Reflection

How to implement BDD practices with standard Go testing package?

I want to write tests first, then write code that makes the tests pass.
I can write tests functions like this:
func TestCheckPassword(t *testing.T) {
isCorrect := CheckPasswordHash("test", "$2a$14$rz.gZgh9CHhXQEfLfuSeRuRrR5uraTqLChRW7/Il62KNOQI9vjO2S")
if isCorrect != true {
t.Errorf("Password is wrong")
}
}
But I'd like to have more descriptive information for each test function.
For example, I am thinking about creating auth module for my app.
Now, in plain English, I can easily describe my requirements for this module:
It should accept a non-empty string as input.
String must be from 6 to 48 characters long.
Function should return true if password string fits provided hash string and false if not.
What's the way to put this information that is understandable by a non-tech business person into tests besides putting them into comments?
In Go, a common way of writing tests to perform related checks is to create a slice of test cases (which is referred to as the "table" and the method as "table-driven tests"), which we simply loop over and execute one-by-one.
A test case may have arbitrary properties, which is usually modeled by an anonymous struct.
If you want to provide a description for test cases, you can add an additional field to the struct describing a test case. This will serve both as documentation of the test case and as (part of the) output in case the test case would fail.
For simplicity, let's test the following simple Abs() function:
func Abs(x int) int {
if x < 0 {
return -x
}
return x
}
The implementation seems to be right and complete. If we'd want to write tests for this, normally we would add 2 test cases to cover the 2 possible branches: test when x is negative (x < 0), and when x is non-negative. In reality, it's often handy and recommended to also test the special 0 input and the corner cases: the min and max values of the input.
If we think about it, this Abs() function won't even give a correct result when called with the minimum value of int32, because that is -2147483648, and its absolute value is 2147483648 which doesn't fit into int32 because max value of int32 is: 2147483647. So the above implementation will overflow and incorrectly give the negative min value as the absolute of the negative min.
The test function that lists cases for each possible branches plus includes 0 and the corner cases, with descriptions:
func TestAbs(t *testing.T) {
cases := []struct {
desc string // Description of the test case
x int32 // Input value
exp int32 // Expected output value
}{
{
desc: "Abs of positive numbers is the same",
x: 1,
exp: 1,
},
{
desc: "Abs of 0 is 0",
x: 0,
exp: 0,
},
{
desc: "Abs of negative numbers is -x",
x: -1,
exp: 1,
},
{
desc: "Corner case testing MaxInt32",
x: math.MaxInt32,
exp: math.MaxInt32,
},
{
desc: "Corner case testing MinInt32, which overflows",
x: math.MinInt32,
exp: math.MinInt32,
},
}
for _, c := range cases {
got := Abs(c.x)
if got != c.exp {
t.Errorf("Expected: %d, got: %d, test case: %s", c.exp, got, c.desc)
}
}
}
In Go, the idiomatic way to write these kinds of tests is:
func TestCheckPassword(t *testing.T) {
tcs := []struct {
pw string
hash string
want bool
}{
{"test", "$2a$14$rz.gZgh9CHhXQEfLfuSeRuRrR5uraTqLChRW7/Il62KNOQI9vjO2S", true},
{"foo", "$2a$14$rz.gZgh9CHhXQEfLfuSeRuRrR5uraTqLChRW7/Il62KNOQI9vjO2S", false},
{"", "$2a$14$rz.gZgh9CHhXQEfLfuSeRuRrR5uraTqLChRW7/Il62KNOQI9vjO2S", false},
}
for _, tc := range tests {
got := CheckPasswordHash(tc.pw, tc.hash)
if got != tc.want {
t.Errorf("CheckPasswordHash(%q, %q) = %v, want %v", tc.pw, tc.hash, got, want)
}
}
}
This is called "table-driven testing". You create a table of inputs and expected outputs, you iterate over that table and call your function and if the expected output does not match what you want, you write an error message describing the failure.
If what you want isn't as simple as comparing a return against a golden value - for example, you want to check that either an error, or a value is returned, or that a well-formed hash+salt is returned, but don't care what salt is used (as that's not part of the API), you'd write additional code for that - in the end, you simply write down what properties the result should have, add some if's to check that and provide a descriptive error message if the result is not as expected. So, say:
func Hash(pw string) (hash string, err error) {
// Validate input, create salt, hash thing…
}
func TestHash(t *testing.T) {
tcs := []struct{
pw string
wantError bool
}{
{"", true},
{"foo", true},
{"foobar", false},
{"foobarbaz", true},
}
for _, tc := range tcs {
got, err := Hash(tc.pw)
if err != nil {
if !tc.wantError {
t.Errorf("Hash(%q) = %q, %v, want _, nil", tc.pw, got, err)
}
continue
}
if len(got) != 52 {
t.Errorf("Hash(%q) = %q, want 52 character string", tc.pw, got)
}
if !CheckPasswordHash(tc.pw, got) {
t.Errorf("CheckPasswordHash(Hash(%q)) = false, want true", tc.pw)
}
}
}
If you want a test suite with descriptive texts and contexts (like rspec for ruby) you should check out ginko: https://onsi.github.io/ginkgo/

Generate a tree of structs with testing/quick, respecting invariants

I have a tree of structs which I'd like to test using testing/quick, but constraining it to within my invariants.
This example code works:
var rnd = rand.New(rand.NewSource(time.Now().UnixNano()))
type X struct {
HasChildren bool
Children []*X
}
func TestSomething(t *testing.T) {
x, _ := quick.Value(reflect.TypeOf(X{}), rnd)
_ = x
// test some stuff here
}
But we hold HasChildren = true whenever len(Children) > 0 as an invariant, so it'd be better to ensure that whatever quick.Value() generates respects that (rather than finding "bugs" that don't actually exist).
I figured I could define a Generate function which uses quick.Value() to populate all the variable members:
func (X) Generate(rand *rand.Rand, size int) reflect.Value {
x := X{}
throwaway, _ := quick.Value(reflect.TypeOf([]*X{}), rand)
x.Children = throwaway.Interface().([]*X)
if len(x.Children) > 0 {
x.HasChildren = true
} else {
x.HasChildren = false
}
return reflect.ValueOf(x)
}
But this is panicking:
panic: value method main.X.Generate called using nil *X pointer [recovered]
And when I change Children from []*X to []X, it dies with a stack overflow.
The documentation is very thin on examples, and I'm finding almost nothing in web searches either.
How can this be done?
Looking at the testing/quick source code it seems that you can't create recursive custom generators and at the same time reuse the quick library facilities to generate the array part of the struct, because the size parameter, that is designed to limit the number of recursive calls, cannot be passed back into quick.Value(...)
https://golang.org/src/testing/quick/quick.go (see around line 50)
in your case this lead to an infinite tree that quickly "explodes" with 1..50 leafs at each level (that's the reason for the stack overflow).
If the function quick.sizedValue() had been public we could have used it to accomplish your task, but unfortunately this is not the case.
BTW since HasChildren is an invariant, can't you simply make it a struct method?
type X struct {
Children []*X
}
func (me *X) HasChildren() bool {
return len(me.Children) > 0
}
func main() {
.... generate X ....
if x.HasChildren() {
.....
}
}

Extending existing type testing.T for adding extra checks

I realize I could pass the testing.T type to the functions that need to deal with the testing interfaces. But how about extending the struct and using it? I know I can type extend but this doesn't work unfortunately:
package test
import "testing"
type testingT testing.T
func (t *testingT) assert(val int) {
if val == 0 {
t.Errorf("%d is zero", val)
}
}
func TestSomething(t *testingT) {
t.assert(0)
}
One solution is embedding as you can see in eduncan911's answer. It's great if you want to add other fields to the new struct.
However, this is not a requirement. Creating a new type having testing.T as its underlying type (just as you did) is just enough:
type myT testing.T
func (t *myT) assert(val int) {
if val == 0 {
t.Errorf("%d is zero", val)
}
}
Using it
In TestSomething() we can't just call t.assert(0) because assert() is not a method of testing.T but our myT type. So we need a value of type myT on which we can call myT.assert().
We can't use type assertion to convert a testing.T to our myT type though, because type assertion requires the expression to be of interface type! And t is not an interface type, it's a concrete type: *testing.T.
The solution is really simple, we can use simple type conversion because myT and testing.T has the same underlying type:
my := myT(*t) // Conversion valid, but not yet enough, read on...
The above works, and we can call:
my.assert(0)
But when running tests, it won't fail! Why is that? It's because the above *t dereferencing, conversion and assignment will make a copy, and my will be of type myT but will be a copy of *t, and so when we call myT.assert(), it gets a pointer, but a pointer to the copy. And when myT.assert() calls t.Errorf(), it will register the error on a copy, so the testing framework will not see this Errorf() call.
The "final" solution is simple: use type conversion, but don't convert *t, but t itself (which is a pointer). Of course *t cannot be converted to myT, only to *myT. And this type needs to be parenthesized (to avoid ambiguity):
func TestSomething(t *testing.T) {
my := (*myT)(t)
my.assert(0)
}
And now we're happy, everything works as expected.
Note: You can also call myT.assert() in one line if you don't want to call multiple methods of myT:
func TestSomething(t *testing.T) {
(*myT)(t).assert(0)
}
You can use embedding which would allow you to add your owner methods:
package test
import "testing"
type testingT struct {
*testing.T
}
func (t *testingT) assert(val int) {
if val == 0 {
t.Errorf("%d is zero", val)
}
}
func TestSomething(t *testing.T) {
t2 := &testingT{t}
t2.assert(0)
}
Prepare custom testing struct:
type T testing.T
func (t *T) AssertEqualsInt(expected, actual int) {
if actual != expected {
// Correct point for file and line where happened error
t.Helper()
t.Errorf("Values not equal expected: %d, got: %d", expected, actual)
}
}
func (t *T) AssertEqualsString(expected, actual string) {
if actual != expected {
t.Helper()
t.Errorf("Values not equal expected: %s, got: %s", expected, actual)
}
}
Than use it in testing:
func TestFooBarBaz(t *testing.T) {
(*T)(t).AssertEqualsInt(0, 10)
(*T)(t).AssertEqualsString("foo", "bar")
}

how to deal with the "fmt" golang library package for CLI testing

Disclaimer: I wish you a merry XMas and I hope my question does not disturb you!
sample.go:
package main
import(
"fmt"
"os"
)
type sample struct {
value int64
}
func (s sample) useful() {
if s.value == 0 {
fmt.Println("Error: something is wrong!")
os.Exit(1)
} else {
fmt.Println("May the force be with you!")
}
}
func main() {
s := sample{42}
s.useful()
s.value = 0
s.useful()
}
// output:
// May the force be with you!
// Error: something is wrong!
// exit status 1
I did a lot of research on how to use interfaces in golang testing. But so far I was not able to wrap my head around this completely. At least I can not see how interfaces help me when I need to "mock" (apologies for using this word) golang std. library packages like "fmt".
I came up with two scenarios:
use os/exec to test the command line interface
wrap fmt package so I have control and am able to check the output strings
I do not like both scenarios:
I experience going through the actual command line a convoluted and not-performant (see below). Might have portability issues, too.
I believe this is the way to go but I fear that wrapping the fmt package might be a lot of work (at least wrapping the time package for testing turned out a non-trivial task (https://github.com/finklabs/ttime)).
Actual Question here: Is there another (better/simpler/idiomatic) way?
Note: I want to do this in pure golang, I am not interested in the next testing framework.
cli_test.go:
package main
import(
"os/exec"
"testing"
)
func TestCli(t *testing.T) {
out, err := exec.Command("go run sample.go").Output()
if err != nil {
t.Fatal(err)
}
if string(out) != "May the force be with you!\nError: this is broken and not useful!\nexit status 1" {
t.Fatal("There is something wrong with the CLI")
}
}
Chapter 11 of Kerningham's Book gives a good solution to this question.
The trick is to change the calls to fmt.Printline() to calls to
fmt.Fprint(out, ...) where out is initialised to os.Stdout
This can be overwritten in the test harness to new(bytes.Buffer) allowing the
test to capture the output.
See https://github.com/adonovan/gopl.io/blob/master/ch11/echo/echo.go and
https://github.com/adonovan/gopl.io/blob/master/ch11/echo/echo_test.go
edited by OP...
sample.go:
package main
import(
"fmt"
"os"
"io"
)
var out io.Writer = os.Stdout // modified during testing
var exit func(code int) = os.Exit
type sample struct {
value int64
}
func (s sample) useful() {
if s.value == 0 {
fmt.Fprint(out, "Error: something is wrong!\n")
exit(1)
} else {
fmt.Fprint(out, "May the force be with you!\n")
}
}
func main() {
s := sample{42}
s.useful()
s.value = 0
s.useful()
}
// output:
// May the force be with you!
// Error: this is broken and not useful!
// exit status 1
cli_test.go:
package main
import(
"bytes"
"testing"
)
func TestUsefulPositive(t *testing.T) {
bak := out
out = new(bytes.Buffer)
defer func() { out = bak }()
s := sample{42}
s.useful()
if out.(*bytes.Buffer).String() != "May the force be with you!\n" {
t.Fatal("There is something wrong with the CLI")
}
}
func TestUsefulNegative(t *testing.T) {
bak := out
out = new(bytes.Buffer)
defer func() { out = bak }()
code := 0
osexit := exit
exit = func(c int) { code = c }
defer func() { exit = osexit }()
s := sample{0}
s.useful()
if out.(*bytes.Buffer).String() != "Error: something is wrong!\n" {
t.Fatal("There is something wrong with the CLI")
}
if code != 1 {
t.Fatal("Wrong exit code!")
}
}
Am I missing something here or are you talking of testable examples?
Basically, it works like this: In a *_test.go file, you need to adhere to the convention Example[[T][_M]] where T is a placeholder for the type and M a placeholder for the method you want to display the testable example as example code in the Godoc. If the function is just called Example(), the code will be shown as a package example.
Below the last line of the code of your example, you can put a comment like this
// Output:
// Foo
Now go test will make sure that the testable example function either exactly puts out everything below // Output: (including whitespace) or it will make the test fail.
Here is an actual example for an testable example
func ExampleMongoStore_Get() {
sessionId := "ExampleGetSession"
data, err := ms.Get(sessionId)
if err == sessionmw.ErrSessionNotFound {
fmt.Printf("Session '%s' not found\n", sessionId)
data = make(map[string]interface{})
data["foo"] = "bar"
ms.Save(sessionId, data)
}
loaded, _ := ms.Get(sessionId)
fmt.Printf("Loaded value '%s' for key '%s' in session '%s'",
loaded["foo"],
"foo", sessionId)
// Output:
// Session 'ExampleGetSession' not found
// Loaded value 'bar' for key 'foo' in session 'ExampleGetSession'
}
Edit: Have a look at the output of above example at godoc.org