Writing NEAR smart contracts in Rust, my compiler seems to require objects sent through the API to have the Serialize trait, and objects stored in the blockchain to have BorshSerialize and BorshDeserialize. Is that really the case, or am I doing something wrong? I would have thought that BorshSerialize would provide Serialize, and that you could use one library for both purposes.
I would have thought that BorshSerialize would provide Serialize
That is not the case. BorshSerialize is specific to Borsh whereas Serialize comes from serde and neither implies the other. The reason why return type requires Serialize is because we want it to be serialized as json so that it is easy to handle on the frontend.
Your observation is correct. Serialize is serde::Serialize, with serde_json gives you serde Serialize. BorshSerailize is serialize with borsh, commonly used serialization format implemented by NEAR to store data on chain. Depending on your use case you might want to use them both or separately. Separately: you create "model" that store on chain that has only BorshSerialize, create "view" that returns by API that has only Serialize.
Related
I'm currently struggling with the experimental KXS-properties serialization backend, mainly because of two reasons:
I can't find any documentation for it (I think there is none)
KXS-properties only includes a serializer / deserializer, but no encoder / decoder
The endpoint provided by the framework is essentially Map<String, Any>, but the map is flat and the keys already have the usual dot-separated properties syntax. So the step that I have to take is to encode the map to a single string that is printable to a .properties file AND decode a single string from a .properties file into the map. I'm generally following the Properties Format Spec from https://docs.oracle.com/javase/10/docs/api/java/util/Properties.html#load(java.io.Reader), it's not as easy as one might think.
The problem is that I can't use java.util.Properties right away because KXS is multiplatform and it would kinda kill the purpose of it when I'd restrict it to JVM because I use java.util.Properties. If I were to use it, the solution would be pretty simple, like this: https://gist.github.com/RaphaelTarita/748e02c06574b20c25ab96c87235096d
So I'm trying to implement my own encoder / decoder, following the rough structure of kotlinx.serialization.json.Json.kt. Although it's pretty tedious, it went well so far, but now I've stumbled upon a new problem:
As far as I know (I am not sure because there is no documentation), the map only contains primitives (or primitive-equivalents, as Kotlin does not really have primitives). I suspect this because when you write your own KSerializers for the KXS frontend, you can specify to encode to any primitive by invoking the encodeXXX() functions of the Encoder interface. Now the problem is: When I try to decode to the map that should contain primitives, how do I even know which primitives are expected by the model class?
I've once written my own serializer / deserializer in Java to learn about the topic, but in that implementation, the backend was a lot more tightly coupled to the frontend, so that I could query the expected primitive type from the model class in the backend. But in my situation, I don't have access to the model class and I have no clue how to retrieve the expected types.
As you can see, I've tried multiple approaches, but none of them worked right away. If you can help me to get any of these to work, that would be very much appreciated
Thank you!
The way it works in kotlinx.serialization is that there are serializers that describe classes and structures etc. as well as code that writes/read properties as well as the struct. It is then the job of the format to map those operations to/from a data format.
The intended purpose of kotlinx.serialization.Properties is to support serializing a Kotlin class to/from a java.util.Properties like structure. It is fairly simple in setup in that every nested property is serialized by prepending the property name to the name (the dotted properties syntax).
Unfortunately it is indeed the case that this deserializing from this format requires knowing the types expected. It doesn't just read from string. However, it is possible to determine the structure. You can use the descriptor property of the serializer to introspect the expectations.
From my perspective this format is a bit more simple than it should be. It is a good example of a custom format though. A key distinction between formats is whether they are intended to just provide a storage format, or whether the output is intended (be able to) to represent a well designed api. The latter ones need to be more complex.
I see that Sorm already supports org.joda.time.DateTime. Is there a possibility to add support for other types?
For example, my case class has a java.nio.charset.Charset or Locale field, which I would like to convert to a string. Suppose I have functions to accomplish the conversion from the custom type to/from a SQL type, how can I tell Sorm to use it?
SORM's support for a certain datatype is quite more complex than just ability to convert to and from an SQL type. Values of some types may span several columns (e.g. Tuple, Range), others may require intermediate tables (Seq, Set, Map) and all of them require an individual approach to translating query clauses. All that would have resulted in a quite complex ad-hoc type-mapping API if one was to be exposed.
But the thing is the above is really not the reason why such an API is not exposed and most probably not to ever be. You see, SORM's philosophy is essentially all about pure immutable data model and the cleanest way to design such one is to use standard Scala's immutable datatypes and case classes.
So the clean way for you to design your application with SORM would be to convert those stateful Java's classes to immutable values in your application. For instance you could implement a custom case class Charset (...) in your model, register it with SORM's Instance and have your conversion functions work between this type and the Java's one in your application. Besides that, you could implement this Charset as an Enumeration, which seems to be the most appropriate.
Concerning your argument about the Joda Time types support, it's there mostly because some data types were needed to represent the SQL's timestamps. See this logic as reverse to what you were thinking of.
I know for a matter of fact that Type cannot be used when passing in to a WCF service. Does anyone have a complete list?
I'm not sure anyone bothered compiling a list, and i'm not sure there is any use in compiling one. Instead, there are requirements that a type must meet in order to be used in WCF contracts. Mainly, it has to be serializable.
I think it is the programmer's responsibility to verify that the types used in contracts are all serializable, and to make sure all custom types are serializing and deserializing properly.
Anything that you want to use in a WCF service needs to be serializable first, and secondly, it needs to be able to be expressed using XML schema. Also, WCF is interoprable by nature, so anything that's too specific to .NET (like exceptions, the .NET Type and so forth) should be avoided.
Anything non-serializable is out from the get go, and anything that cannot be expressed in XML schema can't be used either. This includes interfaces - you can only use concrete classes - and it also exludes generic types, since XML schema doesn't know how to handle generic types.
You're quite okay as long as you stick to the basic types (int, string, datetime etc.) and anything that is directly composed from those types.
Anything not marked Serializable, for starters.
I have a WCF service method that expects an object and then retrieves its properties using reflection.
On the client side I create an anonymous type object
var obj = new {FirstName="John", LastName="Doe"}
and pass it to the method. I'm getting an exception:
Type '<>f__AnonymousType0`2[System.String,System.String]' cannot be serialized.
Consider marking it with the DataContractAttribute attribute, and marking all
of its members you want serialized with the DataMemberAttribute attribute.
See the Microsoft .NET Framework documentation for other supported types.
I can't mark the type or its members with serialization attributes because there is really no type nor properties declared explicitly. Is there a way to overcome that problem?
The answers suggesting that you can't use an anonymous type over WCF are wrong.
If you are using the default DataContractSerializer to serialize your types on the channel, then yes, the answers are correct. This is because the DataContractSerializer supports the following scenarios:
Types serialized using the Serializable attribute
Types serialized using XML Serialization
Types serialized using the DataContract attribute
Plain-Old-C#-Object (POCO) Serialization
Respectively, they fail with anonymous types because of the following:
You can't apply attributes to anonymous types.
XML Serialization requires a default parameterless constructor, which anonymous types do not have.
Same as 1.
Same as 2.
However, you are not forced to use the DataContractSerializer to serialize your messages in WCF. You can create a custom message formatter which you can use to perform the serialization yourself.
You have a problem if the types you are sending out as the results of requests are anonymous types. When you get the results back, it's going to have a definite name within a namespace (not in a .NET sense, but in a SOA sense) and you are going to have to handle the mapping of that concrete type back to the anonymous type. However, because you don't have access to the actual anonymous type or ways of constructing it in code (at least in a dynamic way), you have no choice but to pass it along as an object if it's passed back to you, which makes it kind of worthless, since everyone will have to use bad practices such as dynamic (not a bad practice in itself, but to get around these limitations in this case, yes), or cast-by-example.
So in the end I will say that while it certainly is possible to serialize anonymous types and send them over the wire, the work invovled is usually not worth it.
Don't do this.
It's an attempt to be clever. Don't. Just declare the datatype you need, and use it. If you need a more loosely-defined datatype, just use a key-value mapping of some sort.
It will take you 5 minutes to write something that can handle this for good. Using any technique like this will cost you hours of debugging at some future point down the road.
You could serialize the object into a JSON string and send it through WCF, like so:
//in WCF Server
dynamic mysentclass = new { FirstName = "John", LastName = "Doe" };
string jsonstring = JsonConvert.SerializeObject(mysentclass, Newtonsoft.Json.Formatting.Indented);
//send the string through WCF
//in WCF client
dynamic myreceivedclass = JsonConvert.DeserializeObject<dynamic>(jsonstring);
MessageBox.Show(myreceivedclass.FirstName.ToString() + " " + myreceivedclass.LastName.ToString());
The example uses Json.Net, which can be obtained here:
http://www.nuget.org/packages/Newtonsoft.Json/
You could also use System.Web.Script.Serialization.JavaScriptSerializer (in System.Web.Extensions.dll), which is not as powerful as Json.Net, but would suffice for simple objects.
No, there is not. While there are tricks and techniques to return objects of an anonymous type from a method, you cannot send them over WCF.
WCF needs to know all of its concrete types that will be sent across, since you're not really just calling a .NET method. Instead, the message call gets converted into a serialized message, and therefore, any "thing" being passed over a WCF call must be serializable - no exceptions.
You've already got the answer. It can't be done.
In fact, you can't pass an instance of an anonymous type from one method to another within your program. You certainly can't pass them between programs.
As said before, the objects must be deserializable and so you will have to define the structure beforehand. However, you can use inheritance to defining them, and hence lower the pain. WCF provides the KnownType attribute to allow a Service Operation to receive an object of the base class and deserialize it into an object of a derived class. So you will still only have one (or a few) Service Operations that can handle all your scenarios.
how does Serialization of objects works? How object got deserialized and a instance is created from serialized date without a call to any constructor?
I've kept this answer language agnostic since a language wasn't given.
When the object is serialized, all the require information to rebuild it is encoded in way which can be retrieved. This typically includes the type of the object, as well as the value of all the instance variables.
When the object is deserialized, an area in memory of the correct size is allocated and is populated using the serialized information such that the new object is identical to the serialized one.
The running program can then refer to this new object in memory without having to actually call the constructor.
There are lots of little details which this doesn't explain, but this is the general idea of serialization/deserialization.
Are you talking about Java? If so, serialization is an extralingual object creation mechanism. It's a backdoor that uses native code to create the object without calling any constructors. Therefore, when designing a class for serializability, you need to make sure that a class created through deserialization maintains the same invariants (key fields being initialized) as you would through the constructor path. A third way to create objects in Java is through cloning, and similar issues apply.
Cloning and serialization don't interact well with the use of final fields if you need to set the value of that field to something different than what is returned by clone or the deserialization process.
Josh Bloch's "Effective Java" has some chapters that explain these issues in more depth.
(this answer may apply to other languages too, but I've only used serialization in Java)
Regarding .NET: this isn't a definitive or textbook answer, and I might be all-out wrong...
.NET Serialization needs to be seperated out into Binary vs. others (XML or an XML derivitave typically). Binary serialization is mostly a black-box to me, but it allows the object to be serialized and restored in their current state. XML serialization typically only serialized the public fields/properties of an object, unless overriden by adding a custom ISerializable implementation.
In the case of XML serialization I believe .NET uses Reflection to determine which fields and properties get converted to their equivalent Elements. Adding an [XMLSerializable] attribute will implement a default behavior which can be adjusted by applying other attributes at the field level (such as [XMLAttribute]).
The metadata (which Reflection depends on) stores all the object members as well as their attributes and addresses, which allows the serializer to determine how it should build the output.