I know how to serialize in F# using mutable objects, but is there a way to serialize/deserialize using record types using either XmlSerializer or the DataContractSerializer? looks like there is a way to do this for a discriminated union using the KnownType attribute, but i am looking for a way to use non-mutable records without default constructor...
Beginning with F# 3.0, serialization of record types is now supported by applying the CliMutableAttribute to the type. Example:
[<CLIMutable>]
type MyRecord = { Name : string; Age : int }
This example is taken from http://blogs.msdn.com/b/fsharpteam/archive/2012/07/19/more-about-fsharp-3.0-language-features.aspx, which includes a discussion of this feature and three other new features in F# 3.0: triple-quoted strings, automatic properties, and unused variable warnings.
The sample code for reading data from Freebase by Jomo Fisher uses DataContractJsonSerializer to load data into immutable F# records. The declaration of the record that he uses looks like this:
[<DataContract>]
type Result<'TResult> = { // '
[<field: DataMember(Name="code") >]
Code:string
[<field: DataMember(Name="result") >]
Result:'TResult // '
[<field: DataMember(Name="message") >]
Message:string }
The key point here is that the the DataMember attribute is attached to the underlying field that's actually used to store the data and not to the read-only property that the F# compiler generates (using the field: modifier on the attribute).
I'm not 100% sure if this is going to work with other types of serialization (probably not), but it may be a useful pointer to start with...
EDIT I'm not sure if I'm missing something here, but the following basic example works fine for me:
module Demo
#r "System.Runtime.Serialization.dll"
open System.IO
open System.Text
open System.Xml
open System.Runtime.Serialization
type Test =
{ Result : string[]
Title : string }
do
let sb = new StringBuilder()
let value = { Result = [| "Hello"; "World" |]; Title = "Hacking" }
let xmlSerializer = DataContractSerializer(typeof<Test>);
xmlSerializer.WriteObject(new XmlTextWriter(new StringWriter(sb)), value)
let sr = sb.ToString()
printfn "%A" sr
let xmlSerializer = DataContractSerializer(typeof<Test>);
let reader = new XmlTextReader(new StringReader(sr))
let obj = xmlSerializer.ReadObject(reader) :?> Test
printfn "Reading: %A" obj
EDIT 2 If you want to generate cleaner XML then you can add attributes like this:
[<XmlRoot("test")>]
type Test =
{ [<XmlArrayAttribute("results")>]
[<XmlArrayItem(typeof<string>, ElementName = "string")>]
Result : string[]
[<XmlArrayAttribute("title")>]
Title : string }
It doesn't use XmlSerializer or the DataContractSerializer, but Json.NET 6.0 includes nice F# support.
It looks like this:
type TestTarget =
{ a: string
b: int }
[<TestFixture>]
type JsonTests() =
[<Test>]
member x.``can serialize``() =
let objectUnderTest = { TestTarget.a = "isa"; b = 9 }
let jsonResult: string = Newtonsoft.Json.JsonConvert.SerializeObject(objectUnderTest)
printfn "json is:\n%s" jsonResult
let xmlResult = Newtonsoft.Json.JsonConvert.DeserializeXmlNode(jsonResult, "root")
printfn "xml is:\n%s" (xmlResult.OuterXml)
let jsonRoundtrip = Newtonsoft.Json.JsonConvert.DeserializeObject<TestTarget>(jsonResult)
printfn "json roundtrip: %A" jsonRoundtrip
let xmlAsJson = Newtonsoft.Json.JsonConvert.SerializeXmlNode(xmlResult, Newtonsoft.Json.Formatting.Indented, true)
printfn "object -> json -> xml -> json:\n%A" xmlAsJson
let xmlRoundtrip = Newtonsoft.Json.JsonConvert.DeserializeObject<TestTarget>(xmlAsJson)
printfn "xml roundtrip:\n%A" xmlRoundtrip
Assert.That(true, Is.False)
()
json is:
{"a":"isa","b":9}
xml is:
<root><a>isa</a><b>9</b></root>
json roundtrip: {a = "isa";
b = 9;}
object -> json -> xml -> json:
"{
"a": "isa",
"b": "9"
}"
xml roundtrip:
{a = "isa";
b = 9;}
You can use this series of annotations on the properties of classes to format the XML:
[XmlRoot("root")]
[XmlElement("some-element")]
[XmlAttribute("some-attribute")]
[XmlArrayAttribute("collections")]
[XmlArrayItem(typeof(SomeClass), ElementName = "item")]
I use the attributes on my c# classes, but deserialize in F# (c# classes are ina referenced lib).
in f#:
use file = new FileStream(filePath, FileMode.Open)
let serializer= XmlSerializer(typeof<SomeClass>)
let docs = serializer.Deserialize file :?> SomeClass
Related
Following a question where the answer provided a working solution to serialize / deserialize discriminated unions (IgnoreMissingMember setting doesn't seem to work with FSharpLu.Json deserializer)
I have now a practical case where this fails (although it works in simpler cases).
here is the test code:
open System.Collections.Generic
open Microsoft.FSharpLu.Json
open Newtonsoft.Json
open Newtonsoft.Json.Serialization
// set up the serialization / deserialization based on answer from:
// https://stackoverflow.com/questions/62364229/ignoremissingmember-setting-doesnt-seem-to-work-with-fsharplu-json-deserializer/62364913#62364913
let settings =
JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
Converters = [| CompactUnionJsonConverter(true, true) |]
)
let serialize object =
JsonConvert.SerializeObject(object, settings)
let deserialize<'a> object =
JsonConvert.DeserializeObject<'a>(object, settings)
// define the type used
type BookSide =
| Bid
| Ask
type BookEntry =
{
S : float
P : float
}
type BookSideData =
Dictionary<int, BookEntry>
type BookData =
{
Data: Dictionary<BookSide, BookSideData>
}
static member empty =
{
Data = Dictionary<BookSide, BookSideData>(dict [ (BookSide.Bid, BookSideData()); (BookSide.Ask, BookSideData()) ])
}
// make some sample data
let bookEntry = { S=3.; P=5. }
let bookData = BookData.empty
bookData.Data.[BookSide.Bid].Add(1, bookEntry)
// serialize. This part works
let s = serialize bookData
// deserialize. This part fails
deserialize<BookData> s
the serialized test data will look like this:
{"Data":{"Bid":{"1":{"S":3.0,"P":5.0}},"Ask":{}}}
but deserializing will crash like this:
Could not convert string 'Bid' to dictionary key type 'FSI_0023+BookSide'. Create a TypeConverter to convert from the string to the key type object.
although the serialization / deserialization of the DU through FSharpLu which has a DU converter.
The reason I am trying to find some automated solution, vs writing a custom TypeConverter (besides the fact I've never done it) is that I have a lot of types I do not control to go through.
Here is a fiddle:
https://dotnetfiddle.net/Sx0k4x
Your basic problem is that you are using BookSide as a dictionary key -- but this is an f# union which makes it a complex key -- one not immediately convertible to and from a string. Unfortunately Json.NET does not support complex dictionary keys out of the box as is stated in its Serialization Guide:
When serializing a dictionary, the keys of the dictionary are converted to strings and used as the JSON object property names. The string written for a key can be customized by either overriding ToString() for the key type or by implementing a TypeConverter. A TypeConverter will also support converting a custom string back again when deserializing a dictionary.
There are two basic approaches to handling this issue:
Implement a TypeConverter as is shown in, e.g., Not ableTo Serialize Dictionary with Complex key using Json.net.
Serialize the dictionary as an array of key/value pair objects e.g. as is shown in Serialize dictionary as array (of key value pairs).
Since your data model includes dictionaries with a variety of keys (DU, strings and ints) the second solution would appear to be the only possibility. The following DictionaryConverter should have the necessary logic:
let inline isNull (x:^T when ^T : not struct) = obj.ReferenceEquals (x, null)
type Type with
member t.BaseTypesAndSelf() =
t |> Seq.unfold (fun state -> if isNull state then None else Some(state, state.BaseType))
member t.DictionaryKeyValueTypes() =
t.BaseTypesAndSelf()
|> Seq.filter (fun i -> i.IsGenericType && i.GetGenericTypeDefinition() = typedefof<Dictionary<_,_>>)
|> Seq.map (fun i -> i.GetGenericArguments())
type JsonReader with
member r.ReadAndAssert() =
if not (r.Read()) then raise (JsonReaderException("Unexpected end of JSON stream."))
r
member r.MoveToContentAndAssert() =
if r.TokenType = JsonToken.None then r.ReadAndAssert() |> ignore
while r.TokenType = JsonToken.Comment do r.ReadAndAssert() |> ignore
r
type internal DictionaryReadOnlySurrogate<'TKey, 'TValue>(i : IDictionary<'TKey, 'TValue>) =
interface IReadOnlyDictionary<'TKey, 'TValue> with
member this.ContainsKey(key) = i.ContainsKey(key)
member this.TryGetValue(key, value) = i.TryGetValue(key, &value)
member this.Item with get(index) = i.[index]
member this.Keys = i.Keys :> IEnumerable<'TKey>
member this.Values = i.Values :> IEnumerable<'TValue>
member this.Count = i.Count
member this.GetEnumerator() = i.GetEnumerator()
member this.GetEnumerator() = i.GetEnumerator() :> IEnumerator
type DictionaryConverter () =
// ReadJson adapted from this answer https://stackoverflow.com/a/28633769/3744182
// To https://stackoverflow.com/questions/28451990/newtonsoft-json-deserialize-dictionary-as-key-value-list-from-datacontractjsonse
// By https://stackoverflow.com/users/3744182/dbc
inherit JsonConverter()
override this.CanConvert(t) = (t.DictionaryKeyValueTypes().Count() = 1) // If ever implemented for IReadOnlyDictionary<'TKey, 'TValue> then reject DictionaryReadOnlySurrogate<'TKey, 'TValue>
member private this.ReadJsonGeneric<'TKey, 'TValue> (reader : JsonReader, t : Type, existingValue : obj, serializer : JsonSerializer) : obj =
let contract = serializer.ContractResolver.ResolveContract(t)
let dict = if (existingValue :? IDictionary<'TKey, 'TValue>) then existingValue :?> IDictionary<'TKey, 'TValue> else contract.DefaultCreator.Invoke() :?> IDictionary<'TKey, 'TValue>
match reader.MoveToContentAndAssert().TokenType with
| JsonToken.StartArray ->
let l = serializer.Deserialize<List<KeyValuePair<'TKey, 'TValue>>>(reader)
for p in l do dict.Add(p)
dict :> obj
| JsonToken.StartObject ->
serializer.Populate(reader, dict)
dict :> obj
| JsonToken.Null -> null // Or throw an exception if you prefer
| _ -> raise (JsonSerializationException(String.Format("Unexpected token {0}", reader.TokenType)))
override this.ReadJson(reader, t, existingValue, serializer) =
let keyValueTypes = t.DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
let m = typeof<DictionaryConverter>.GetMethod("ReadJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
m.MakeGenericMethod(keyValueTypes).Invoke(this, [| reader; t; existingValue; serializer |])
member private this.WriteJsonGeneric<'TKey, 'TValue> (writer : JsonWriter, value : obj, serializer : JsonSerializer) =
let dict = value :?> IDictionary<'TKey, 'TValue>
let keyContract = serializer.ContractResolver.ResolveContract(typeof<'Key>)
// Wrap the value in an enumerator or read-only surrogate to prevent infinite recursion.
match keyContract with
| :? JsonPrimitiveContract -> serializer.Serialize(writer, new DictionaryReadOnlySurrogate<'TKey, 'TValue>(dict))
| _ -> serializer.Serialize(writer, seq { yield! dict })
()
override this.WriteJson(writer, value, serializer) =
let keyValueTypes = value.GetType().DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
let m = typeof<DictionaryConverter>.GetMethod("WriteJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
m.MakeGenericMethod(keyValueTypes).Invoke(this, [| writer; value; serializer |])
()
Which you would add to settings as follows:
let settings =
JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
Converters = [| CompactUnionJsonConverter(true, true); DictionaryConverter() |]
)
And generates the following JSON for your bookData:
{
"Data": [
{
"Key": "Bid",
"Value": [
{
"Key": 1,
"Value": {
"S": 3.0,
"P": 5.0
}
}
]
},
{
"Key": "Ask",
"Value": []
}
]
}
Notes:
The converter works for all Dictionary<TKey, TValue> types (and subtypes).
The converter detects whether the dictionary keys will be serialized using a primitive contract, and if so, serializes the dictionary compactly as a JSON object. If not the dictionary is serialized as an array. You can observe this in the JSON shown above: the Dictionary<BookSide, BookSideData> dictionary is serialized as a JSON array, and the Dictionary<int, BookEntry> dictionary is serialized as a JSON object.
During deserialization the converter detects whether the incoming JSON value is an array or object, and adapts as required.
The converter is only implemented for the mutable .Net Dictionary<TKey, TValue> type. The logic would require some slight modification to deserialize the immutable Map<'Key,'Value> type.
Demo fiddle here.
A quick sample to understand my situation:
static Interpreter ParseInterpreter = new Interpreter();
...
var func = ParseInterpreter.Parse("ctx.SomeProp", new Parameter("ctx", typeof(???1)).Compile<Func<???2, object>>;
...
var token = JToken.Parse(s);
dynamic dToken = token;
var obj = func(dToken);
In other words, is there a way to pass some dynamic parameter to Parse method and then get a functor which accepts such parameters?
You can directly use the Lambda class returned by the Parse method, and not call the Compile function:
var interpreter = new Interpreter()
string expression = "ctx.SomeProp";
Lambda parsedExpression = interpreter.Parse(expression, new Parameter("ctx", typeof(object)));
var token = JToken.Parse(s);
var result = parsedExpression.Invoke(token);
I have not tested exactly your code but for example I have a test like this that works correctly:
dynamic dyn = new ExpandoObject();
dyn.Foo = "bar";
var interpreter = new Interpreter()
.SetVariable("dyn", (object)dyn);
Assert.AreEqual(dyn.Foo, interpreter.Eval("dyn.Foo"));
Consider that this only works on .NET 4.x, on .NET Standard/Core dynamics are not supported.
I post a request to get some json data and try to convert the data to Model
var newDrugs = Array<DxyDrugInfo>()
newDrugs = self.parseDrugJsonToModels(data) ?? []
private func parseDrugJsonToModels(data: NSData) -> Array<DxyDrugInfo>?{
let json = JSON(data: data)
if let drugsArray = json["data"].arrayObject as? Array<[String : AnyObject]>, success = json["success"].bool where success {
var sortedArray = Array<DxyDrugInfo>()
for drugDic in drugsArray {
let drugInfo = DxyDrugInfo()
drugInfo.setValuesForKeysWithDictionary(drugDic)
sortedArray.append(drugInfo)
}
return sortedArray
}
return nil
}
DxyDrugInfo is a Objective-C Model Class
My question is the sortedArray's type is Array of DxyDrugInfo, and it changed to
when assigned to newDrugs.
I want to convert it to DxyDrugInfo and I also want to what the type is.
Thank you
That type you're looking at, #lvalue [DxyDrugInfo], is "array of DxyDrugInfo". You're already done! The square brackets around a type are shorthand for "array", that is:
[DxyDrugInfo] == Array<DxyDrugInfo>
and #lvalue just means that newDrugs is a variable you can assign to. (It's the opposite of "r-value", which is a value you could assign to an "l-value".)
I want to convert my class to a Map so I'm using Serialization package. From the example it looks simple:
var address = new Address();
address.street = 'N 34th';
address.city = 'Seattle';
var serialization = new Serialization()
..addRuleFor(Address);
Map output = serialization.write(address);
I expect to see an output like {'street' : 'N 34th', 'city' : 'Seattle'} but instead it just output something I-don't-know-what-that-is
{"roots":[{"__Ref":true,"rule":3,"object":0}],"data":[[],[],[],[["Seattle","N 34th"]]],"rules":"{\"roots\":[{\"__Ref\":true,\"rule\":1,\"object\":0}],\"data\":[[],[[{\"__Ref\":true,\"rule\":4,\"object\":0},{\"__Ref\":true,\"rule\":3,\"object\":0},{\"__Ref\":true,\"rule\":5,\"object\":0},{\"__Ref\":true,\"rule\":6,\"object\":0}]],[[],[],[\"city\",\"street\"]],[[]],[[]],[[]],[[{\"__Ref\":true,\"rule\":2,\"object\":0},{\"__Ref\":true,\"rule\":2,\"object\":1},\"\",{\"__Ref\":true,\"rule\":2,\"object\":2},{\"__Ref\":true,\"rule\":7,\"object\":0}]],[\"Address\"]],\"rules\":null}"}
Serialization is not supposed to create human-readable output. Maybe JSON output is more what you look for:
import dart:convert;
{
var address = new Address();
..address.street = 'N 34th';
..address.city = 'Seattle';
var encoded = JSON.encode(address, mirrorJson);
}
Map mirrorJson(o) {
Map map = new Map();
InstanceMirror im = reflect(o);
ClassMirror cm = im.type;
var decls = cm.declarations.values.where((dm) => dm is VariableMirror);
decls.forEach((dm) {
var key = MirrorSystem.getName(dm.simpleName);
var val = im.getField(dm.simpleName).reflectee;
map[key] = val;
});
return map;
}
The new Address() creates a full prototype object which is what you are seeing. That being said, they could have done something to avoid part of those, but if you want to restore the object just the way it is, that's necessary.
To see the full content of an object you use the for() instruction in this way:
for(obj in idx) alert(obj[idx]);
You'll see that you get loads of data this way. Without the new Address() it would probably not be that bad.
Serialization won't help you here...
You might give a try to JsonObject library, and maybe go through this in depth explanation how to do what you are trying to do using mirrors.
I am dynamically creating a Lambda expression (based on user input but at the moment using dummy values for a proof-of-concept) for a type which I will only know at runtime. I therefore need to pass the T portion of the Func<T,TResult> as a dynamic type, since I won't know the type until runtime (TResult will always be a bool).
It seems that I cannot pass in a Type variable or use typeof with generics. Basically I'm trying to do something like this:
// (f => f.Baz == 1)
Type theType = Type.GetType("Foo");
ParameterExpression pe = Expression.Parameter(theType, "f");
Expression left = Expression.Property(pe, theType.GetProperty("Baz"));
Expression right = Expression.Constant(1);
Expression expr = Expression.Equal(left, right);
// This works fine but uses a hard-coded type, which I won't know until runtime:
// var lambda = Expression.Lambda<Func<Foo,bool>>(expr, new ParameterExpression[] { pe }).Compile();
var lambda = Expression.Lambda<Func<theType, bool>>(expr, new ParameterExpression[] { pe }).Compile();
However, I cannot use the variable theType as the T portion of the Func. How can I fix this?
No you can't.
For example, in C#, you can't:
Type t = typeof(int);
List<t> list = new List<t>();
or
object list = new List<t>();
Unless you use reflection, but then you have to put the list in an object, and you can use it only through reflection.
So if you want you can save your Func<> in an object (or a dynamic) but nothing more.
What you COULD do is always return Func<object, bool> and cast the object to the desidered type IN the lambda function (so use a Expression.Convert(pe, theType));
Or you could use the dynamic:
// lambda == Func<Foo, bool>
dynamic lamdba = Expression.Lambda(expr, new ParameterExpression[] { pe }).Compile();
bool res = lambda(myvalue);
or
// lambda == Func<Foo, bool>
Delegate lamdba = Expression.Lambda(expr, new ParameterExpression[] { pe }).Compile();
bool res = (bool)lambda2.DynamicInvoke(t);
To be taken "not as real" some benchmarks (in StopWatch.Ticks, look at them only for proportions) (Release Mode + Start Without Debugging + some useless cycles so that they are "hot"):
236384685 dynamic
56773593 Func<object, bool> + cast
10556024247 DynamicInvoke
as a note, Func<Foo, bool> has the same speed, so there isn't any speed lost in the extra cast.
You can see the code here http://ideone.com/qhnVP3