I am trying to figure out how one can properly marhsall nullable type (string, int, time) properly to JSON in Go. I know that database/sql provide sql.NullTime, sql.NullInt, etc but when you marshall these values, you get something like
{"first_name": {
"Value": "",
"Valid": false,
}}
What I really want is
{"first_name": null}
I understand that you can implement your own MarshalJSON to do this (I wrote about it here http://dennissuratna.com/marshalling-nullable-string-db-value-to-json-in-go/)
BUT I am wondering if anyone knows a better way to do this. I want to know other people know a less tedious way to do this.
May be late, but when searching for the same problem google report this page at a high rank, so : my solution is to define special type that surcharge NullInt64 for exemple and has and export JSON
Example for NullInt64 :
// NullInt64 is the same as sql.NullInt64
// But when exported to JSON it is null or the int64 val that is exported
// not a JSON containing Value and Valid properties
type NullInt64 struct {
sql.NullInt64
}
// NullInt64 MarshalJSON interface redefinition
func (r NullInt64) MarshalJSON() ([]byte, error) {
if r.Valid {
return json.Marshal(r.Int64)
} else {
return json.Marshal(nil)
}
}
And then replace all sql.NullInt64 by NullInt64.
You can easily do the same for sql.NullString.
Hope it helps
Create a type that embeds (e.g.) sql.NullInt and implements the json.Marshaler interface.
If you really want to have null field, you will have to resort to a custom marshaller (or maybe, just maybe, use *string fields in struct and assign nil instead of an empty string).
However, if you look at the original goal of JSON (JavaScript Object Notation), you will notice that in JavaScript there is hardly any difference between:
var obj = JSON.parse('{"first_name": null }');
alert(obj.first_name)
and:
var obj = JSON.parse('{}');
alert(obj.first_name)
In other words: assigning null to a field has the same effect as not specifying it all. And most JSON parsers work like this.
Not specifying empty fields is supported by the Go JSON marshaller:
type MyType struct {
firstname string `json:omitempty`
}
In the end, it depends on what you want to do with your JSON :)
Related
This question already has answers here:
Cant serialize when using enum as key in Hashmap
(2 answers)
Closed 9 months ago.
I need to serialize and deserialize a HashMap, h, with enum Foo as a key to and from JSON. Foo's variants contain data (here simplified to u32, but actually are enums themselves):
use serde::{Serialize, Deserialize};
use serde_json;
use std::collections::HashMap;
#[derive(Serialize, Deserialize)]
enum Foo {
A(u32),
B(u32),
}
// Tried several different things here! Just deriving the relevant traits doesn't work.
struct Bar {
h: HashMap<Foo, i32>, // The i32 value type is arbitrary
}
fn main() {
let mut bar = Bar { h: HashMap::new() };
bar.h.insert(Foo::A(0), 1);
// I want to be able to do this
let bar_string = serde_json::to_string(&bar).unwrap();
let bar_deser: Bar = serde_json::from_str(&bar_string).unwrap();
}
Since the JSON specification requires keys to be strings I know I need to customise the way serialization and deserialization are done for Foo when it is a key in h. I've tried the following:
Custom implementations of Serialize and Deserialize (e.g. in the accepted answer here)
Serde attributes e.g.: #[serde(into = "String", try_from = "String")] + implementing Into<String> for Foo and TryFrom<String> for Foo (described in the answers here)
Using the 'serde_with' crate (also described in the answers here).
Unfortunately none of these have worked - all eventually failed with different panics after compiling successfully.
What is a good way to achieve what I want in serde, if there is one? If not, I'd be very grateful for any workaround suggestions.
Bonus question: why doesn't serde/serde_json provide a default serialization to a String + deserialization of an enum like Foo when it is used as a key in a HashMap?
Here is a working solution with serde_as from the serde_with helper crate:
use serde::{Deserialize, Serialize};
use serde_json;
use std::collections::HashMap;
#[derive(Serialize, Deserialize, Hash, Eq, PartialEq, Debug)]
enum Foo {
A(u32),
B(u32),
}
#[serde_with::serde_as]
#[derive(Serialize, Deserialize, Debug)]
struct Bar {
#[serde_as(as = "HashMap<serde_with::json::JsonString, _>")]
h: HashMap<Foo, i32>,
}
fn main() {
let mut bar = Bar { h: HashMap::new() };
bar.h.insert(Foo::A(0), 1);
let bar_string = serde_json::to_string(&bar).unwrap();
let bar_deser: Bar = serde_json::from_str(&bar_string).unwrap();
println!("{:?}", bar);
println!("{}", bar_string);
println!("{:?}", bar_deser);
}
Bar { h: {A(0): 1} }
{"h":{"{\"A\":0}":1}}
Bar { h: {A(0): 1} }
Bonus question: why doesn't serde/serde_json provide a default serialization to a String + deserialization of an enum like Foo when it is used as a key in a HashMap?
If you noticed, most of those problems become runtime errors.
That is because serde actually does have a representation of a map that works with all Rust values. So serde itself has no power over this problem.
The problem comes when serde_json tries to feed the map into a json Serializer. And yes, this serializer could indeed convert any key into a JSON string and back if that was implemented.
I think it's more of a design decision why this isn't done. If JSON only supports string keys, so should the JSON serializer. If the user wants to use other types as keys, he should convert them to strings first as seen in the code example above.
Kotlin provides Array.isArrayOf() for checking if an array is of a certain type.
It's used like this
if(object.isArrayOf<String>())
And defined like this
/**
* Checks if array can contain element of type [T].
*/
#Suppress("REIFIED_TYPE_PARAMETER_NO_INLINE")
public fun <reified T : Any> Array<*>.isArrayOf(): Boolean =
T::class.java.isAssignableFrom(this::class.java.componentType)
But it's only for Array. I need to check ArrayList.
I thought to change the signature like so.
#Suppress("REIFIED_TYPE_PARAMETER_NO_INLINE")
public fun <reified T : Any> ArrayList<*>.isArrayListOf(): Boolean =
T::class.java.isAssignableFrom(this::class.java.componentType)
but class.java.componentType is specific to Array
How can I check what type of ArrayList I have?
I should clarify, I only care if its one of 3 types, so I don't need a completely open-ended way of checking.
If you want to check the type of a list you can do:
when (list.firstOrNull()) {
is String -> { /*do something*/ }
is Int -> { /*do another thing*/ }
else -> { /*do something else*/ }
}
And if you need to use the list of a certain type you can use:
list.filterInstance</*the type you need*/>()
Hope this works for you.
You can't. Arrays are the only generic type for which this is possible (because they aren't really generic in the same sense, Kotlin just hides it).
The only thing you can do is look at its contents, but of course
that won't work for empty lists;
if a list contains e.g. a String, it could be ArrayList<String>, ArrayList<CharSequence>, ArrayList<Any>, etc.
For this purpose:
I need to direct it into the appropriate Bundle method. bundle.putStringArrayList(), bundle.putIntegerArrayList(), ect
neither should be a problem, I believe.
If the list is of one type then you can convert the list to array using: toTypedArray() and after you can then check the type using: isArrayOf
But this would be inefficient since you are converting the list to array, better if you can just directly guess or retrieved the first item of the list.
I'm trying to separate my code into models and serializers with the idea that there be defined serializers that handles all json responsibilities, i.e. separation of concerns. I also want to be able to call a model object obj.Serialize() to get the serializer struct obj that I can then marshal. Therefore, I've come up with the following design. To avoid circular import I had to use interfaces in my serializers which leads to using getters in my models. I've read that getters/setters aren't idiomatic go code and I would prefer not to have "boilerplate" getter code all over my models. Is there a better solution to what I want to accomplish, keeping in mind I want separation of concerns and obj.Serialize()?
src/
models/
a.go
serializers/
a.go
models/a.go
import "../serializers"
type A struct {
name string
age int // do not marshal me
}
func (a *A) Name() string {
return a.name
}
// Serialize converts A to ASerializer
func (a *A) Serialize() interface{} {
s := serializers.ASerializer{}
s.SetAttrs(a)
return s
}
serializers/a.go
// AInterface used to get Post attributes
type AInterface interface {
Name() string
}
// ASerializer holds json fields and values
type ASerializer struct {
Name `json:"full_name"`
}
// SetAttrs sets attributes for PostSerializer
func (s *ASerializer) SetAttrs(a AInterface) {
s.Name = a.Name()
}
It looks like you are actually trying to translate between your internal structs and json. We can start by taking advantage of the json library.
If you want certain libraries to handle your struct fields in certain ways, there are tags. This example shows how json tags tell json to never marshal the field age into json, and to only add the field jobTitle if it is not empty, and that the field jobTitle is actually called title in json. This renaming feature is very useful when structs in go contain capitalized (exported) fields, but the json api you're connecting to uses lowercase keys.
type A struct {
Name string
Age int `json:"-"`// do not marshal me
location string // unexported (private) fields are not included in the json marshal output
JobTitle string `json:"title,omitempty"` // in our json, this field is called "title", but we only want to write the key if the field is not empty.
}
If you need to precompute a field, or simply add a field in your json output of a struct that isn't a member of that struct, we can do that with some magic. When json objects are decoded again into golang structs, fields that don't fit (after checking renamed fields and capitalization differences) are simply ignored.
// AntiRecursionMyStruct avoids infinite recursion in MashalJSON. Only intended for the json package to use.
type AntiRecursionMyStruct MyStruct
// MarshalJSON implements the json.Marshaller interface. This lets us marshal this struct into json however we want. In this case, we add a field and then cast it to another type that doesn't implement the json.Marshaller interface, and thereby letting the json library marshal it for us.
func (t MyStruct) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
AntiRecursionMyStruct
Kind string // the field we want to add, in this case a text representation of the golang type used to generate the struct
}{
AntiRecursionMyStruct: AntiRecursionMyStruct(t),
Kind: fmt.Sprintf("%T", MyStruct{}),
})
}
Keep in mind that json will only include your exported (capitalized) struct members. I've made this misstake multiple times.
As a general rule, if something seems too complicated, there's probably a better way to do it.
I have received a JavaScript object in response to a remote HTTP request. I have a kotlin model (trait) that defines the various fields I expect on the object (the nullable ones are optional).
First, I want to do an is check to make sure my object is in fact of the expected type. I initially tried payload is MyModel but that doesn't work due to the way the is operator is written in kotlin.js.
Second, I want to cast to MyModel so I can get auto-complete, etc. on the object while I work with it. Normally, the is alone would be enough but since that doesn't work I need something for this problem as well.
I would like to avoid manually populating my object from a dynamic. I wouldn't mind doing this so much if I could use by Delegates.mapVal(...) but that requires a Map<String, Any?> and I don't know how to get my dynamic/Any? payload into a Map<String, Any?>.
1) We don't have structure check for is in performance reasons.
I don't sure that we need generic solution for this, but anyway I created issue about it, feel free to vote or star it to get updates.
2) is enough if you use smart cast, like:
if (payload is MyModel) {
// call MyModel members on payload
}
But don't forget about (1) :)
3) You can write something like:
class MapDynamic<out V>(val d: dynamic) {
public fun get(thisRef: Any, desc: PropertyMetadata): V {
return d[desc.name]
}
}
class Foo(data: dynamic) {
val field: Int by MapDynamic(data)
}
fun main(args : Array<String>) {
val f = Foo(object { val field = 123 })
println(f.field)
}
But it looks too verbose, but You can add additional logic for e.g. when data don't have requested field. And if You don't need custom logic I think cast is enough.
For the second part, the cast, you can do:
fun responseHandler(payload: dynamic) {
val myModel = payload as MyModel
}
or
fun responseHandler(payload: dynamic) {
val myModel: MyModel = payload
}
This will throw an NPE if payload is null, but it won't actually validate that the payload matches MyModel. In particular, you may end up with null fields/properties that shouldn't be if the payload was missing those fields/properties.
I'm experimenting with PetaPoco to convert a table into POCOs.
In my table, I've got a column named TheEnum. The values in this column are strings that represent the following enum:
public enum MyEnum
{
Fred,
Wilma
}
PetaPoco chokes when it tries to convert the string "Fred" into a MyEnum value.
It does this in the GetConverter method, in the line:
Convert.ChangeType( src, dstType, null );
Here, src is "Fred" (a string), and dstType is typeof(MyEnum).
The exception is an InvalidCastException, saying Invalid cast from 'System.String' to 'MyEnum'
Am I missing something? Is there something I need to register first?
I've got around the problem by adding the following into the GetConverter method:
if (dstType.IsEnum && srcType == typeof(string))
{
converter = delegate( object src )
{
return Enum.Parse( dstType, (string)src ) ;
} ;
}
Obviously, I don't want to run this delegate on every row as it'll slow things down tremendously. I could register this enum and its values into a dictionary to speed things up, but it seems to me that something like this would likely already be in the product.
So, my question is, do I need to do anything special to register my enums with PetaPoco?
Update 23rd February 2012
I submitted a patch a while ago but it hasn't been pulled in yet. If you want to use it, look at the patch and merge into your own code, or get just the code from here.
I'm using 4.0.3 and PetaPoco automatically converts enums to integers and back. However, I wanted to convert my enums to strings and back. Taking advantage of Steve Dunn's EnumMapper and PetaPoco's IMapper, I came up with this. Thanks guys.
Note that it does not handle Nullable<TEnum> or null values in the DB. To use it, set PetaPoco.Database.Mapper = new MyMapper();
class MyMapper : PetaPoco.IMapper
{
static EnumMapper enumMapper = new EnumMapper();
public void GetTableInfo(Type t, PetaPoco.TableInfo ti)
{
// pass-through implementation
}
public bool MapPropertyToColumn(System.Reflection.PropertyInfo pi, ref string columnName, ref bool resultColumn)
{
// pass-through implementation
return true;
}
public Func<object, object> GetFromDbConverter(System.Reflection.PropertyInfo pi, Type SourceType)
{
if (pi.PropertyType.IsEnum)
{
return dbObj =>
{
string dbString = dbObj.ToString();
return enumMapper.EnumFromString(pi.PropertyType, dbString);
};
}
return null;
}
public Func<object, object> GetToDbConverter(Type SourceType)
{
if (SourceType.IsEnum)
{
return enumVal =>
{
string enumString = enumMapper.StringFromEnum(enumVal);
return enumString;
};
}
return null;
}
}
You're right, handling enums is not built into PetaPoco and usually I just suggest doing exactly what you've done.
Note that this won't slow things down for requests that don't use the enum type. PetaPoco generates code to map responses to pocos so the delegate will only be called when really needed. In other words, the GetConverter will only be called the first time a particular poco type is used, and the delegate will only be called when an enum needs conversion. Not sure on the speed of Enum.Parse, but yes you could cache in a dictionary if it's too slow.
If you are using PetaPoco's T4 generation and you want enums in your generated type, you can use the PropertyType override in Database.tt:
tables["App"]["Type"].PropertyType = "Full.Namespace.To.AppType";
I you want to store the value of the enum instead of the index number (1,2,4 for example) you can locate the update function in PetaPoco class because the code is "managed" etc, when you add it as nuget package it will store the .cs file to your project. If we would have the enum variable Color = {red, yellow, blue}
Instead of:
// Store the parameter in the command
AddParam(cmd, pc.GetValue(poco), pc.PropertyInfo);
change to:
//enum?
if (i.Value.PropertyInfo.PropertyType.IsEnum)
{
AddParam(cmd, i.Value.GetValue(poco).ToString(), i.Value.PropertyInfo);
}
else
{
// Store the parameter in the command
AddParam(cmd, i.Value.GetValue(poco), i.Value.PropertyInfo);
}
It would store "yellow" instead of 2