Serializing Models in Golang - serialization

I'm trying to separate my code into models and serializers with the idea that there be defined serializers that handles all json responsibilities, i.e. separation of concerns. I also want to be able to call a model object obj.Serialize() to get the serializer struct obj that I can then marshal. Therefore, I've come up with the following design. To avoid circular import I had to use interfaces in my serializers which leads to using getters in my models. I've read that getters/setters aren't idiomatic go code and I would prefer not to have "boilerplate" getter code all over my models. Is there a better solution to what I want to accomplish, keeping in mind I want separation of concerns and obj.Serialize()?
src/
models/
a.go
serializers/
a.go
models/a.go
import "../serializers"
type A struct {
name string
age int // do not marshal me
}
func (a *A) Name() string {
return a.name
}
// Serialize converts A to ASerializer
func (a *A) Serialize() interface{} {
s := serializers.ASerializer{}
s.SetAttrs(a)
return s
}
serializers/a.go
// AInterface used to get Post attributes
type AInterface interface {
Name() string
}
// ASerializer holds json fields and values
type ASerializer struct {
Name `json:"full_name"`
}
// SetAttrs sets attributes for PostSerializer
func (s *ASerializer) SetAttrs(a AInterface) {
s.Name = a.Name()
}

It looks like you are actually trying to translate between your internal structs and json. We can start by taking advantage of the json library.
If you want certain libraries to handle your struct fields in certain ways, there are tags. This example shows how json tags tell json to never marshal the field age into json, and to only add the field jobTitle if it is not empty, and that the field jobTitle is actually called title in json. This renaming feature is very useful when structs in go contain capitalized (exported) fields, but the json api you're connecting to uses lowercase keys.
type A struct {
Name string
Age int `json:"-"`// do not marshal me
location string // unexported (private) fields are not included in the json marshal output
JobTitle string `json:"title,omitempty"` // in our json, this field is called "title", but we only want to write the key if the field is not empty.
}
If you need to precompute a field, or simply add a field in your json output of a struct that isn't a member of that struct, we can do that with some magic. When json objects are decoded again into golang structs, fields that don't fit (after checking renamed fields and capitalization differences) are simply ignored.
// AntiRecursionMyStruct avoids infinite recursion in MashalJSON. Only intended for the json package to use.
type AntiRecursionMyStruct MyStruct
// MarshalJSON implements the json.Marshaller interface. This lets us marshal this struct into json however we want. In this case, we add a field and then cast it to another type that doesn't implement the json.Marshaller interface, and thereby letting the json library marshal it for us.
func (t MyStruct) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
AntiRecursionMyStruct
Kind string // the field we want to add, in this case a text representation of the golang type used to generate the struct
}{
AntiRecursionMyStruct: AntiRecursionMyStruct(t),
Kind: fmt.Sprintf("%T", MyStruct{}),
})
}
Keep in mind that json will only include your exported (capitalized) struct members. I've made this misstake multiple times.
As a general rule, if something seems too complicated, there's probably a better way to do it.

Related

How to handle conflicting function names when implementing multiple interfaces?

I have an interface defined in C# that implements IEnumerable. The implementation of the interface will be done in C++/WinRT as it needs direct access to native code. When I attempt to implement this interface using C++/WinRT, the generated header/implementation contains two 'First()' functions (one from IIterable, and one from IBindableIterable) with different return types. Obviously this isn't going to compile.
Is there some way to "rename" one (or both) of the conflicting functions in the IDL file? C++/CX had a work around that allowed you to use a different function name and then 'bind' it back to the interface name.
Simplified example code below:
Interface:
public interface IUInt32Array : IEnumerable<uint> {}
IDL:
[default_interface]
runtimeclass UInt32Array : IUInt32Array
{
UInt32Array(UInt32 size);
}
IDL Generated Header:
struct UInt32Array : UInt32ArrayT<UInt32Array>
{
UInt32Array(uint32_t size);
Windows::Foundation::Collections::IIterator<uint32_t> First(); // <-- Problem
Windows::UI::Xaml::Interop::IBindableIterator First(); // <-- Problem
}
A solution for this specific problem is to use a combination of 'auto' as the declared return type for the First() function implementation, and to return a type with conversion operators for the two different return types.
Here is an example showing how this was solved in the CppWinRT source code. The linked source code is for the base_collections_vector.h header, specifically see the convertible_observable_vector::First() function (replicated below).
auto First() {
struct result {
container_type* container;
operator wfc::IIterator<T>() {
return static_cast<base_type*>(container)->First();
}
operator wfc::IIterator<Windows::Foundation::IInspectable>() {
return make<iterator>(container);
}
};
return result{ this };
}
Notice here that the function itself is defined as returning auto, which allows us to return an intermediate type. This intermediate type then implements conversion operators for converting to the type expected by the caller. This works for this particular problem as the generated CppWinRT source code immediately assigns the result of the call to a value of the expected type, thus immediately causing the invocation of the conversion operators which in turn return the final correct iterator type.
Thanks to Kenny Kerr who pointed me at both the example and a write-up explaining the above.

Understanding polymorphism in Go

I guess I got stuck in thinking about a polymorphism solution to my following problem:
Let's say I have a BaseTX struct with fields for a transaction. Now I have two special types of transactions: RewardTX struct and AllowanceTX struct.
RewardTX struct has at this moment only the composition of BaseTX struct.
AllowanceTX struct has a composition of BaseTX struct and an AddField.
I have also a function logicAndSaveTX(), which has some logic on fields from BaseTX but at the end is serializing the whole object using json.Marshal() and saving the byte[] somewhere.
type TXapi interface {
logicAndSaveTX()
}
type BaseTX struct {
Field1 string
Field2 string
}
type RewardTX struct {
BaseTX
}
type AllowanceTX struct {
BaseTX
AddField string
}
func (tx BaseTX) logicAndSaveTX() {
// logic on BaseTX fields; simplified:
tx.Field1 = "overwritten"
tx.Field2 = "logic done"
// here would be marshal to json and save; simplified to print object:
fmt.Printf("saved this object: %+v \n", tx)
}
func SaveTX(tx TXapi) {
tx.logicAndSaveTX()
}
func main() {
rewardTX := RewardTX{BaseTX : BaseTX{Field1: "Base info1", Field2: "Base info2"}}
SaveTX(rewardTX) // should print rewardTX with fields from BaseTX
allowanceTX := AllowanceTX{BaseTX : BaseTX{Field1: "Base info1", Field2: "Base info2"}, AddField: "additional field"}
SaveTX(allowanceTX) // would like to print allowanceTX with fields from BaseTX + AdditionalField >>> instead only printing fields from BaseTX
}
https://play.golang.org/p/0Vu_YXktRIk
I try to figure out how to implement the structures and the function to operate on both kinds of transactions but at the end serializing both structures properly. My problem is, that the AddField is not being seen in my current implementation.
Maybe I have got some brain fail here--I would really like to implement this the "proper Go way". :)
Go is not object-oriented. The only form of polymorphism in Go is interfaces.
Coming from other, object-oriented languages can be difficult, because you have to get rid of a lot of ideas you might try to carry over - things like, for example, "base" classes/types. Just remove "base" from your design thinking; you're trying to turn composition into inheritance, and that's only going to get you into trouble.
In this case, maybe you have a legitimate case for composition here; you have some common shared fields used by multiple types, but it's not a "base" type. It's maybe "metadata" or something - I can't say what to call it given that your example is pretty abstract, but you get the idea.
So maybe you have:
type TXapi interface {
logicAndSaveTX()
}
type Metadata struct {
Field1 string
Field2 string
}
type RewardTX struct {
Metadata
}
func (tx RewardTX) logicAndSaveTX() {
// logic on BaseTX fields; simplified:
tx.Field1 = "overwritten"
tx.Field2 = "logic done"
// here would be marshal to json and save; simplified to print object:
fmt.Printf("saved this object: %+v \n", tx)
}
type AllowanceTX struct {
Metadata
AddField string
}
func (tx AllowanceTX) logicAndSaveTX() {
// logic on BaseTX fields; simplified:
tx.Field1 = "overwritten"
tx.Field2 = "logic done"
tx.AddField = "more stuff"
// here would be marshal to json and save; simplified to print object:
fmt.Printf("saved this object: %+v \n", tx)
}
If the handling of the metadata (or whatever) fields is identical in all uses, maybe you give that type its own logicTX method to fill those fields, which can be called by the logicAndSaveTX of the structs that embed it.
The key here is to think of the behavior (methods) on a type to be scoped to that type, instead of thinking of it as somehow being able to operate on "child types". Child types don't exist, and there is no way for a type that is embedded in another type to operate on its container.
Also point to be noted here aht Go only support run time polymorphism through interfaces. Compile time polymorphism is not possible in Golang.
Source: - https://golangbyexample.com/oop-polymorphism-in-go-complete-guide/

JSON marshall nullable type value in database/sql Go

I am trying to figure out how one can properly marhsall nullable type (string, int, time) properly to JSON in Go. I know that database/sql provide sql.NullTime, sql.NullInt, etc but when you marshall these values, you get something like
{"first_name": {
"Value": "",
"Valid": false,
}}
What I really want is
{"first_name": null}
I understand that you can implement your own MarshalJSON to do this (I wrote about it here http://dennissuratna.com/marshalling-nullable-string-db-value-to-json-in-go/)
BUT I am wondering if anyone knows a better way to do this. I want to know other people know a less tedious way to do this.
May be late, but when searching for the same problem google report this page at a high rank, so : my solution is to define special type that surcharge NullInt64 for exemple and has and export JSON
Example for NullInt64 :
// NullInt64 is the same as sql.NullInt64
// But when exported to JSON it is null or the int64 val that is exported
// not a JSON containing Value and Valid properties
type NullInt64 struct {
sql.NullInt64
}
// NullInt64 MarshalJSON interface redefinition
func (r NullInt64) MarshalJSON() ([]byte, error) {
if r.Valid {
return json.Marshal(r.Int64)
} else {
return json.Marshal(nil)
}
}
And then replace all sql.NullInt64 by NullInt64.
You can easily do the same for sql.NullString.
Hope it helps
Create a type that embeds (e.g.) sql.NullInt and implements the json.Marshaler interface.
If you really want to have null field, you will have to resort to a custom marshaller (or maybe, just maybe, use *string fields in struct and assign nil instead of an empty string).
However, if you look at the original goal of JSON (JavaScript Object Notation), you will notice that in JavaScript there is hardly any difference between:
var obj = JSON.parse('{"first_name": null }');
alert(obj.first_name)
and:
var obj = JSON.parse('{}');
alert(obj.first_name)
In other words: assigning null to a field has the same effect as not specifying it all. And most JSON parsers work like this.
Not specifying empty fields is supported by the Go JSON marshaller:
type MyType struct {
firstname string `json:omitempty`
}
In the end, it depends on what you want to do with your JSON :)

Constructors in Go

I have a struct and I would like it to be initialised with some sensible default values.
Typically, the thing to do here is to use a constructor but since go isn't really OOP in the traditional sense these aren't true objects and it has no constructors.
I have noticed the init method but that is at the package level. Is there something else similar that can be used at the struct level?
If not what is the accepted best practice for this type of thing in Go?
There are some equivalents of constructors for when the zero values can't make sensible default values or for when some parameter is necessary for the struct initialization.
Supposing you have a struct like this :
type Thing struct {
Name string
Num int
}
then, if the zero values aren't fitting, you would typically construct an instance with a NewThing function returning a pointer :
func NewThing(someParameter string) *Thing {
p := new(Thing)
p.Name = someParameter
p.Num = 33 // <- a very sensible default value
return p
}
When your struct is simple enough, you can use this condensed construct :
func NewThing(someParameter string) *Thing {
return &Thing{someParameter, 33}
}
If you don't want to return a pointer, then a practice is to call the function makeThing instead of NewThing :
func makeThing(name string) Thing {
return Thing{name, 33}
}
Reference : Allocation with new in Effective Go.
There are actually two accepted best practices:
Make the zero value of your struct a sensible default. (While this looks strange to most people coming from "traditional" oop it often works and is really convenient).
Provide a function func New() YourTyp or if you have several such types in your package functions func NewYourType1() YourType1 and so on.
Document if a zero value of your type is usable or not (in which case it has to be set up by one of the New... functions. (For the "traditionalist" oops: Someone who does not read the documentation won't be able to use your types properly, even if he cannot create objects in undefined states.)
Go has objects. Objects can have constructors (although not automatic constructors). And finally, Go is an OOP language (data types have methods attached, but admittedly there are endless definitions of what OOP is.)
Nevertheless, the accepted best practice is to write zero or more constructors for your types.
As #dystroy posted his answer before I finished this answer, let me just add an alternative version of his example constructor, which I would probably write instead as:
func NewThing(someParameter string) *Thing {
return &Thing{someParameter, 33} // <- 33: a very sensible default value
}
The reason I want to show you this version is that pretty often "inline" literals can be used instead of a "constructor" call.
a := NewThing("foo")
b := &Thing{"foo", 33}
Now *a == *b.
There are no default constructors in Go, but you can declare methods for any type. You could make it a habit to declare a method called "Init". Not sure if how this relates to best practices, but it helps keep names short without loosing clarity.
package main
import "fmt"
type Thing struct {
Name string
Num int
}
func (t *Thing) Init(name string, num int) {
t.Name = name
t.Num = num
}
func main() {
t := new(Thing)
t.Init("Hello", 5)
fmt.Printf("%s: %d\n", t.Name, t.Num)
}
The result is:
Hello: 5
I like the explanation from this blog post:
The function New is a Go convention for packages that create a core type or different types for use by the application developer. Look at how New is defined and implemented in log.go, bufio.go and cypto.go:
log.go
// New creates a new Logger. The out variable sets the
// destination to which log data will be written.
// The prefix appears at the beginning of each generated log line.
// The flag argument defines the logging properties.
func New(out io.Writer, prefix string, flag int) * Logger {
return &Logger{out: out, prefix: prefix, flag: flag}
}
bufio.go
// NewReader returns a new Reader whose buffer has the default size.
func NewReader(rd io.Reader) * Reader {
return NewReaderSize(rd, defaultBufSize)
}
crypto.go
// New returns a new hash.Hash calculating the given hash function. New panics
// if the hash function is not linked into the binary.
func (h Hash) New() hash.Hash {
if h > 0 && h < maxHash {
f := hashes[h]
if f != nil {
return f()
}
}
panic("crypto: requested hash function is unavailable")
}
Since each package acts as a namespace, every package can have their own version of New. In bufio.go multiple types can be created, so there is no standalone New function. Here you will find functions like NewReader and NewWriter.
In Go, a constructor can be implemented using a function that returns a pointer to a modified structure.
type Colors struct {
R byte
G byte
B byte
}
// Constructor
func NewColors (r, g, b byte) *Colors {
return &Color{R:r, G:g, B:b}
}
For weak dependencies and better abstraction, the constructor does not return a pointer to a structure, but an interface that this structure implements.
type Painter interface {
paintMethod1() byte
paintMethod2(byte) byte
}
type Colors struct {
R byte
G byte
B byte
}
// Constructor return intreface
func NewColors(r, g, b byte) Painter {
return &Color{R: r, G: g, B: b}
}
func (c *Colors) paintMethod1() byte {
return c.R
}
func (c *Colors) paintMethod2(b byte) byte {
return c.B = b
}
another way is;
package person
type Person struct {
Name string
Old int
}
func New(name string, old int) *Person {
// set only specific field value with field key
return &Person{
Name: name,
}
}
If you want to force the factory function usage, name your struct (your class) with the first character in lowercase. Then, it won't be possible to instantiate directly the struct, the factory method will be required.
This visibility based on first character lower/upper case work also for struct field and for the function/method. If you don't want to allow external access, use lower case.
Golang is not OOP language in its official documents.
All fields of Golang struct has a determined value(not like c/c++), so constructor function is not so necessary as cpp.
If you need assign some fields some special values, use factory functions.
Golang's community suggest New.. pattern names.
I am new to go. I have a pattern taken from other languages, that have constructors. And will work in go.
Create an init method.
Make the init method an (object) once routine. It only runs the first time it is called (per object).
func (d *my_struct) Init (){
//once
if !d.is_inited {
d.is_inited = true
d.value1 = 7
d.value2 = 6
}
}
Call init at the top of every method of this class.
This pattern is also useful, when you need late initialisation (constructor is too early).
Advantages: it hides all the complexity in the class, clients don't need to do anything.
Disadvantages: you must remember to call Init at the top of every method of the class.

How can I simplify my deserialization framework?

I have a Serialization interface which is designed to encapsulate the differences between XML/JSON/binary serialization for my application. It looks something like this:
interface Serialization {
bool isObject();
int opApply(int delegate(string member, Serialization value) del); //iterate object
...
int toInt(); //this part is ugly, but without template member overloading, I
long toLong(); //figure out any way to apply generics here, so all basic types
... //have a toType primitive
string toString();
}
class JSONSerialization : Serialization {
private JSON json;
...
long toLong() {
enforce(json.type == JSON_TYPE.NUMBER, SerializationException.IncorrectType);
return cast(long)json.toNumber();
}
...
}
So, what I then set up is a set of templates for registering type deserializers and calling them:
...
registerTypeDeserializer!Vec3(delegate Vec3(Serialization s) {
return Vec3(s[0].toFloat, s[1].toFloat, s[2].toFloat);
});
...
auto v = parseJSON("some file").deserialize!Vec3;
...
registerTypeDeserializer!Light(delegate Light(Serialization s) {
return new Light(s["intensity"].toFloat, s["position"].deserialize!Vec3);
});
This works well for structs and simple classes, and with the new parameter identifier tuple and parameter default value tuple I should even be able to add automatic deserializer generation. However, I don't really like the inconsistency between basic and user defined types, and more importantly, complex types have to rely on global state to acquire references:
static MaterialLibrary materials;
registerTypeDeserializer!Model(delegate Model(Serialization s) {
return new Model(materials.borrow(s["material"].toString), ...);
});
That's where it really falls apart. Because I can't (without a proliferation of register deserializer functions) pass other parameters to the deserializer, I'm having difficulty avoiding ugly global factories. I've thought about eliminating the deserialize template, and requiring a deserialize function (which could accept multiple parameters) for each user defined type, but that seems like a lot of work for e.g. POD structs.
So, how can I simplify this design, and hopefully avoid tons of boilerplate deserializers, while still allowing me to inject object factories appropriately, instead of assigning them globally?
Basic types can be read using readf \ formattedRead, so you can create a wrapper function that uses this formattedRead it possible, otherwise it uses a static function from the desired type to read the value. Something like this:
auto _readFrom(T)(string s){
static if(__traits(compiles,(readf("",cast(T*)(null))))){
T result;
formattedRead(s,"%s",&result);
return result;
}else{
return T.readFrom(s);
}
}