How to customize data serialization based on content in WCF? - wcf

Trying to serialize a union-like data-type. There is an enum field indicating the type of data stored in the union, and a variety of possible field types.
The desired result is DataContractSerializer produced XML which contains just the enum, and the relevant field.
Possible solutions, none of which have been attempted yet, are:
Use a custom serializer and mark the union properties with a custom attribute, similar to this question. The custom serializer would strip out the members not required.
Use ISerializationSurrogate and serialize a different object which just contains the relevant data.
Don't use separate fields in the union, use one object field (this could be used as part of the implementation of the ISerializationSurrogate approach).
Other... ?
For example:
[DataContract]
public class WCFTestUnion
{
public enum EUnionType
{
[EnumMember]
Bool,
[EnumMember]
String,
[EnumMember]
Dictionary,
[EnumMember]
Invalid
};
EUnionType unionType = EUnionType.Invalid;
bool boolValue = true;
string stringValue = "Hello";
IDictionary<object, object> dictionaryValue = null;
// Could use custom attribute here ?
[DataMember]
public bool BoolValue
{
get { return this.boolValue; }
set { this.boolValue = value; }
}
// Could use custom attribute here ?
[DataMember]
public string StringValue
{
get { return this.stringValue; }
set { this.stringValue = value; }
}
// Could use custom attribute here ?
[DataMember]
public IDictionary<object, object> DictionaryValue
{
get { return this.dictionaryValue; }
set { this.dictionaryValue = value; }
}
[DataMember]
public EUnionType UnionType
{
get { return this.unionType; }
set { this.unionType = value; }
}
} // Ends class WCFTestUnion
Test
class TestSerializeUnion
{
internal static void Test()
{
Console.WriteLine("===TestSerializeUnion.Test()===");
WCFTestUnion u = new WCFTestUnion();
u.UnionType = WCFTestUnion.EUnionType.Dictionary;
u.DictionaryValue = new Dictionary<object, object>();
u.DictionaryValue[1] = "one";
u.DictionaryValue["two"] = 2;
System.Runtime.Serialization.DataContractSerializer serialize = new System.Runtime.Serialization.DataContractSerializer(typeof(WCFTestUnion));
System.IO.Stream stream = new System.IO.MemoryStream();
serialize.WriteObject(stream, u);
stream.Seek(0, System.IO.SeekOrigin.Begin);
byte[] buffer = new byte[stream.Length];
int length = checked((int)stream.Length);
int read = stream.Read(buffer, 0, length);
while (read < stream.Length)
{
read += stream.Read(buffer, 0, length - read);
}
string xml = Encoding.Default.GetString(buffer);
System.Xml.XmlDocument doc = new System.Xml.XmlDocument();
doc.LoadXml(xml);
System.Xml.XmlTextWriter xmlwriter = new System.Xml.XmlTextWriter(Console.Out);
xmlwriter.Formatting = System.Xml.Formatting.Indented;
doc.WriteContentTo(xmlwriter);
xmlwriter.Flush();
Console.WriteLine();
}
} // Ends class TestSerializeUnion
Output:
<WCFTestUnion xmlns="http://schemas.datacontract.org/2004/07/WCFTestServiceContracts" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<BoolValue>true</BoolValue>
<DictionaryValue xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
<a:KeyValueOfanyTypeanyType>
<a:Key i:type="b:int" xmlns:b="http://www.w3.org/2001/XMLSchema">1</a:Key>
<a:Value i:type="b:string" xmlns:b="http://www.w3.org/2001/XMLSchema">one</a:Value>
</a:KeyValueOfanyTypeanyType>
<a:KeyValueOfanyTypeanyType>
<a:Key i:type="b:string" xmlns:b="http://www.w3.org/2001/XMLSchema">two</a:Key>
<a:Value i:type="b:int" xmlns:b="http://www.w3.org/2001/XMLSchema">2</a:Value>
</a:KeyValueOfanyTypeanyType>
</DictionaryValue>
<StringValue>Hello </StringValue>
<UnionType>Dictionary</UnionType>
</WCFTestUnion>
Desired Output (only field being used is serialized, along with enum):
<WCFTestUnion xmlns="http://schemas.datacontract.org/2004/07/WCFTestServiceContracts" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<DictionaryValue xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
<a:KeyValueOfanyTypeanyType>
<a:Key i:type="b:int" xmlns:b="http://www.w3.org/2001/XMLSchema">1</a:Key>
<a:Value i:type="b:string" xmlns:b="http://www.w3.org/2001/XMLSchema">one</a:Value>
</a:KeyValueOfanyTypeanyType>
<a:KeyValueOfanyTypeanyType>
<a:Key i:type="b:string" xmlns:b="http://www.w3.org/2001/XMLSchema">two</a:Key>
<a:Value i:type="b:int" xmlns:b="http://www.w3.org/2001/XMLSchema">2</a:Value>
</a:KeyValueOfanyTypeanyType>
</DictionaryValue>
<UnionType>Dictionary</UnionType>
</WCFTestUnion>

You do have several options here. What you use depends on the complexity of this scenario (where else you have to do something like this, how often and in what ways you have to serialize this data, performance, etc.) Take a look at these options, ask away if you have more questions, but mostly, I recommend you just play and experiment with multiple strategies from the list below before picking one or a hybrid solution.
Use a data contract resolver. Provides a mechanism for dynamically mapping types to and from wire representations during serialization and deserialization, giving you flexibility to support far more types than you can out-of-the-box.
Use IObjectReference. You can have a class which implements and returns a reference to a different object after it has been deserialized.
Use a data contract surrogate. This is different from the serialization surrogates you're referring to, but also similar. I think these might work out nicely for you

Related

HTTP end point property string starts with "is" will get omit [duplicate]

This might be a duplicate. But I cannot find a solution to my Problem.
I have a class
public class MyResponse implements Serializable {
private boolean isSuccess;
public boolean isSuccess() {
return isSuccess;
}
public void setSuccess(boolean isSuccess) {
this.isSuccess = isSuccess;
}
}
Getters and setters are generated by Eclipse.
In another class, I set the value to true, and write it as a JSON string.
System.out.println(new ObjectMapper().writeValueAsString(myResponse));
In JSON, the key is coming as {"success": true}.
I want the key as isSuccess itself. Is Jackson using the setter method while serializing? How do I make the key the field name itself?
This is a slightly late answer, but may be useful for anyone else coming to this page.
A simple solution to changing the name that Jackson will use for when serializing to JSON is to use the #JsonProperty annotation, so your example would become:
public class MyResponse implements Serializable {
private boolean isSuccess;
#JsonProperty(value="isSuccess")
public boolean isSuccess() {
return isSuccess;
}
public void setSuccess(boolean isSuccess) {
this.isSuccess = isSuccess;
}
}
This would then be serialised to JSON as {"isSuccess":true}, but has the advantage of not having to modify your getter method name.
Note that in this case you could also write the annotation as #JsonProperty("isSuccess") as it only has the single value element
I recently ran into this issue and this is what I found. Jackson will inspect any class that you pass to it for getters and setters, and use those methods for serialization and deserialization. What follows "get", "is" and "set" in those methods will be used as the key for the JSON field ("isValid" for getIsValid and setIsValid).
public class JacksonExample {
private boolean isValid = false;
public boolean getIsValid() {
return isValid;
}
public void setIsValid(boolean isValid) {
this.isValid = isValid;
}
}
Similarly "isSuccess" will become "success", unless renamed to "isIsSuccess" or "getIsSuccess"
Read more here: http://www.citrine.io/blog/2015/5/20/jackson-json-processor
Using both annotations below, forces the output JSON to include is_xxx:
#get:JsonProperty("is_something")
#param:JsonProperty("is_something")
When you are using Kotlin and data classes:
data class Dto(
#get:JsonProperty("isSuccess") val isSuccess: Boolean
)
You might need to add #param:JsonProperty("isSuccess") if you are going to deserialize JSON as well.
EDIT: If you are using swagger-annotations to generate documentation, the property will be marked as readOnly when using #get:JsonProperty. In order to solve this, you can do:
#JsonAutoDetect(isGetterVisibility = JsonAutoDetect.Visibility.NONE)
data class Dto(
#field:JsonProperty(value = "isSuccess") val isSuccess: Boolean
)
You can configure your ObjectMapper as follows:
mapper.setPropertyNamingStrategy(new PropertyNamingStrategy() {
#Override
public String nameForGetterMethod(MapperConfig<?> config, AnnotatedMethod method, String defaultName)
{
if(method.hasReturnType() && (method.getRawReturnType() == Boolean.class || method.getRawReturnType() == boolean.class)
&& method.getName().startsWith("is")) {
return method.getName();
}
return super.nameForGetterMethod(config, method, defaultName);
}
});
I didn't want to mess with some custom naming strategies, nor re-creating some accessors.
The less code, the happier I am.
This did the trick for us :
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
#JsonIgnoreProperties({"success", "deleted"}) // <- Prevents serialization duplicates
public class MyResponse {
private String id;
private #JsonProperty("isSuccess") boolean isSuccess; // <- Forces field name
private #JsonProperty("isDeleted") boolean isDeleted;
}
Building upon Utkarsh's answer..
Getter names minus get/is is used as the JSON name.
public class Example{
private String radcliffe;
public getHarryPotter(){
return radcliffe;
}
}
is stored as { "harryPotter" : "whateverYouGaveHere" }
For Deserialization, Jackson checks against both the setter and the field name.
For the Json String { "word1" : "example" }, both the below are valid.
public class Example{
private String word1;
public setword2( String pqr){
this.word1 = pqr;
}
}
public class Example2{
private String word2;
public setWord1(String pqr){
this.word2 = pqr ;
}
}
A more interesting question is which order Jackson considers for deserialization. If i try to deserialize { "word1" : "myName" } with
public class Example3{
private String word1;
private String word2;
public setWord1( String parameter){
this.word2 = parameter ;
}
}
I did not test the above case, but it would be interesting to see the values of word1 & word2 ...
Note: I used drastically different names to emphasize which fields are required to be same.
You can change primitive boolean to java.lang.Boolean (+ use #JsonPropery)
#JsonProperty("isA")
private Boolean isA = false;
public Boolean getA() {
return this.isA;
}
public void setA(Boolean a) {
this.isA = a;
}
Worked excellent for me.
If you are interested in handling 3rd party classes not under your control (like #edmundpie mentioned in a comment) then you add Mixin classes to your ObjectMapper where the property/field names should match the ones from your 3rd party class:
public class MyStack32270422 {
public static void main(String[] args) {
ObjectMapper om3rdParty = new ObjectMapper();
om3rdParty .addMixIn(My3rdPartyResponse.class, MixinMyResponse.class);
// add further mixins if required
String jsonString = om3rdParty.writeValueAsString(new My3rdPartyResponse());
System.out.println(jsonString);
}
}
class MixinMyResponse {
// add all jackson annotations here you want to be used when handling My3rdPartyResponse classes
#JsonProperty("isSuccess")
private boolean isSuccess;
}
class My3rdPartyResponse{
private boolean isSuccess = true;
// getter and setter here if desired
}
Basically you add all your Jackson annotations to your Mixin classes as if you would own the class. In my opinion quite a nice solution as you don't have to mess around with checking method names starting with "is.." and so on.
there is another method for this problem.
just define a new sub-class extends PropertyNamingStrategy and pass it to ObjectMapper instance.
here is a code snippet may be help more:
mapper.setPropertyNamingStrategy(new PropertyNamingStrategy() {
#Override
public String nameForGetterMethod(MapperConfig<?> config, AnnotatedMethod method, String defaultName) {
String input = defaultName;
if(method.getName().startsWith("is")){
input = method.getName();
}
//copy from LowerCaseWithUnderscoresStrategy
if (input == null) return input; // garbage in, garbage out
int length = input.length();
StringBuilder result = new StringBuilder(length * 2);
int resultLength = 0;
boolean wasPrevTranslated = false;
for (int i = 0; i < length; i++)
{
char c = input.charAt(i);
if (i > 0 || c != '_') // skip first starting underscore
{
if (Character.isUpperCase(c))
{
if (!wasPrevTranslated && resultLength > 0 && result.charAt(resultLength - 1) != '_')
{
result.append('_');
resultLength++;
}
c = Character.toLowerCase(c);
wasPrevTranslated = true;
}
else
{
wasPrevTranslated = false;
}
result.append(c);
resultLength++;
}
}
return resultLength > 0 ? result.toString() : input;
}
});
The accepted answer won't work for my case.
In my case, the class is not owned by me. The problematic class comes from 3rd party dependencies, so I can't just add #JsonProperty annotation in it.
To solve it, inspired by #burak answer above, I created a custom PropertyNamingStrategy as follow:
mapper.setPropertyNamingStrategy(new PropertyNamingStrategy() {
#Override
public String nameForSetterMethod(MapperConfig<?> config, AnnotatedMethod method, String defaultName)
{
if (method.getParameterCount() == 1 &&
(method.getRawParameterType(0) == Boolean.class || method.getRawParameterType(0) == boolean.class) &&
method.getName().startsWith("set")) {
Class<?> containingClass = method.getDeclaringClass();
String potentialFieldName = "is" + method.getName().substring(3);
try {
containingClass.getDeclaredField(potentialFieldName);
return potentialFieldName;
} catch (NoSuchFieldException e) {
// do nothing and fall through
}
}
return super.nameForSetterMethod(config, method, defaultName);
}
#Override
public String nameForGetterMethod(MapperConfig<?> config, AnnotatedMethod method, String defaultName)
{
if(method.hasReturnType() && (method.getRawReturnType() == Boolean.class || method.getRawReturnType() == boolean.class)
&& method.getName().startsWith("is")) {
Class<?> containingClass = method.getDeclaringClass();
String potentialFieldName = method.getName();
try {
containingClass.getDeclaredField(potentialFieldName);
return potentialFieldName;
} catch (NoSuchFieldException e) {
// do nothing and fall through
}
}
return super.nameForGetterMethod(config, method, defaultName);
}
});
Basically what this does is, before serializing and deserializing, it checks in the target/source class which property name is present in the class, whether it is isEnabled or enabled property.
Based on that, the mapper will serialize and deserialize to the property name that is exist.

Serialization of Keys with Circular References using Jackson

I am trying to serialize a HashMap from Objects to Strings, but the specific Object has a reference to the current class leading to an infinite recursion, which doesn't seem to be solved with the usual JsonIdentifyInfo annotation. Here's an example:
public class CircularKey {
public void start() throws IOException {
ObjectMapper mapper = new ObjectMapper();
Cat cat = new Cat();
// Encode
String json = mapper.writeValueAsString(cat);
System.out.println(json);
// Decode
Cat cat2 = mapper.readValue(json, Cat.class);
System.out.println(mapper.writeValueAsString(cat2));
}
}
#JsonIdentityInfo(generator = ObjectIdGenerators.IntSequenceGenerator.class, property = "#id")
#JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "#class")
class Mouse {
int id;
#JsonProperty
Cat cat;
}
#JsonIdentityInfo(generator = ObjectIdGenerators.IntSequenceGenerator.class, property = "#id")
#JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "#class")
class Cat {
int id;
#JsonSerialize(keyUsing = MouseMapKeySerializer.class)
#JsonDeserialize(keyUsing = MouseMapKeyDeserializer.class)
#JsonProperty
HashMap<Mouse, String> status = new HashMap<Mouse, String>();
public Cat() {
Mouse m = new Mouse();
m.cat = this;
status.put(m, "mike");
}
}
Here's the serializer/deserializer for the key:
class MouseMapKeySerializer extends JsonSerializer<Mouse> {
static ObjectMapper mapper = new ObjectMapper();
#Override
public void serialize(Mouse value, JsonGenerator generator,
SerializerProvider provider) throws IOException,
JsonProcessingException {
String json = mapper.writeValueAsString(value);
generator.writeFieldName(json);
}
}
class MouseMapKeyDeserializer extends KeyDeserializer {
static ObjectMapper mapper = new ObjectMapper();
#Override
public Mouse deserializeKey(String c, DeserializationContext ctx)
throws IOException, JsonProcessingException {
return mapper.readValue(c, Mouse.class);
}
}
If I switch the map to HashMap (String,Object) it works but I cannot change the original mapping. Any ideas?
It looks like you found your answer at http://jackson-users.ning.com/forum/topics/serializing-hashmap-with-object-key-and-recursion. This doesn't seem to be possible because:
Complex keys are tricky, and it is not a use case I ever considered. Then again, there is nothing specifically preventing use of standard components; main concern was just the limitations than JSON has (must be String-value, JsonParser/JsonGenerator expose keys as different tokens).
There is no explicit support for either polymorphic types or object ids for Object keys. Standard serializers/deserializers are mostly for relatively simple types that can be easily and reliably converted to/from Strings; numbers, Dates, UUIDs.
So: unlike with value handlers, where modular design (with separation of TypeSerializer/JsonSerializer) makes sense, I think what you need to do is to have custom (de)serializers that handle all aspects. You should be able to use code from existing value (de)serializers, type (de)serializers, but not classes themselves.
Your use case does sound interesting, but for better or worse, it is pushing the envelope quite a bit. :-)

Store and retrieve string arrays in HBase

I've read this answer (How to store complex objects into hadoop Hbase?) regarding the storing of string arrays with HBase.
There it is said to use the ArrayWritable Class to serialize the array. With WritableUtils.toByteArray(Writable ... writable) I'll get a byte[] which I can store in HBase.
When I now try to retrieve the rows again, I get a byte[] which I have somehow to transform back again into an ArrayWritable.
But I don't find a way to do this. Maybe you know an answer or am I doing fundamentally wrong serializing my String[]?
You may apply the following method to get back the ArrayWritable (taken from my earlier answer, see here) .
public static <T extends Writable> T asWritable(byte[] bytes, Class<T> clazz)
throws IOException {
T result = null;
DataInputStream dataIn = null;
try {
result = clazz.newInstance();
ByteArrayInputStream in = new ByteArrayInputStream(bytes);
dataIn = new DataInputStream(in);
result.readFields(dataIn);
}
catch (InstantiationException e) {
// should not happen
assert false;
}
catch (IllegalAccessException e) {
// should not happen
assert false;
}
finally {
IOUtils.closeQuietly(dataIn);
}
return result;
}
This method just deserializes the byte array to the correct object type, based on the provided class type token.
E.g:
Let's assume you have a custom ArrayWritable:
public class TextArrayWritable extends ArrayWritable {
public TextArrayWritable() {
super(Text.class);
}
}
Now you issue a single HBase get:
...
Get get = new Get(row);
Result result = htable.get(get);
byte[] value = result.getValue(family, qualifier);
TextArrayWritable tawReturned = asWritable(value, TextArrayWritable.class);
Text[] texts = (Text[]) tawReturned.toArray();
for (Text t : texts) {
System.out.print(t + " ");
}
...
Note:
You may have already found the readCompressedStringArray() and writeCompressedStringArray() methods in WritableUtils
which seem to be suitable if you have your own String array-backed Writable class.
Before using them, I'd warn you that these can cause serious performance hit due to
the overhead caused by the gzip compression/decompression.

Jackson vector serialization exception

I have the following code with a simple class and a method for writing and then reading:
ObjectMapper mapper = new ObjectMapper();
try{
DataStore testOut = new DataStore();
DataStore.Checklist ch1 = testOut.addChecklist();
ch1.SetTitle("Checklist1");
String output = mapper.writeValueAsString(testOut);
JsonNode rootNode = mapper.readValue(output, JsonNode.class);
Map<String,Object> userData = mapper.readValue(output, Map.class);
}
public class DataStore {
public static class Checklist
{
public Checklist()
{
}
private String _title;
public String GetTitle()
{
return _title;
}
public void SetTitle(String title)
{
_title = title;
}
}
//Checklists
private Vector<Checklist> _checklists = new Vector<Checklist>();
public Checklist addChecklist()
{
Checklist ch = new Checklist();
ch.SetTitle("New Checklist");
_checklists.add(ch);
return ch;
}
public Vector<Checklist> getChecklists()
{
return _checklists;
}
public void setChecklists(Vector<Checklist> checklists)
{
_checklists = checklists;
}
}
The line:
String output = mapper.writeValueAsString(testOut);
causes an exception that has had me baffled for hours and about to abandon using this at all.
Any hints are appreciated.
Here is the exception:
No serializer found for class DataStore$Checklist and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS) ) (through reference chain: DataStore["checklists"]->java.util.Vector[0])
There are multiple ways to do it, but I will start with what you are doing wrong: your naming of getter and setter method is wrong -- in Java one uses "camel-case", so you should be using "getTitle". Because of this, properties are not found.
Besides renaming methods to use Java-style names, there are alternatives:
You can use annotation JsonProperty("title") for GetTitle(), so that property is recognized
If you don't want the wrapper object, you could alternatively just add #JsonValue for GetTitle(), in which case value used for the whole object would be return value of that method.
The answer seems to be: You can't do that with Json. I've seen comments in the Gson tutorial as well, that state that some serialization just doesn't work. I downloaded XStream and spat it out with XML in a few minutes of work and a lot less construction around what I really wanted to persist. In the process, I was able to delete a lot of code.

Data member default values, how to figure out whether something was really sent?

By default, WCF deserializes missing elements into default values like null, 0 or false. The problem with this approach is that if it's a basic type like number 0 I'm not sure whether it means the real value sent by an external system or a default value generated by WCF.
So my question is: Is it possible to find out at run-time whether the default value means "I didn't send anything".
This is crucial because we can't update and overwrite existing data in the database with the default values just because the external system didn't send a particular element this time (data corruption).
Microsoft's short answer is "It is up to the receiving endpoint to appropriately interpret a missing element."
Data member default values
http://msdn.microsoft.com/en-us/library/aa347792.aspx
Can somebody please clarify what's that supposed to mean?
Thanks
If you define your data members as properties, you can use whether the setter was called or not to decide whether some value was sent. The code below shows one data contract which knows whether it deserialized its fields.
public class Post_51ca1ead_2f0a_4912_a451_374daab0101b
{
[DataContract(Name = "Person", Namespace = "")]
public class Person
{
string name;
int age;
bool nameWasSent;
bool ageWasSent;
[DataMember]
public string Name
{
get
{
return this.name;
}
set
{
this.nameWasSent = true;
this.name = value;
}
}
[DataMember]
public int Age
{
get
{
return this.age;
}
set
{
this.ageWasSent = true;
this.age = value;
}
}
[OnDeserializing]
void OnDeserializing(StreamingContext ctx)
{
this.ageWasSent = false;
this.nameWasSent = false;
}
public override string ToString()
{
return string.Format("Person[Name={0},Age={1}]",
nameWasSent ? name : "UNSPECIFIED",
ageWasSent ? age.ToString() : "UNSPECIFIED");
}
}
public static void Test()
{
MemoryStream ms = new MemoryStream();
DataContractSerializer dcs = new DataContractSerializer(typeof(Person));
dcs.WriteObject(ms, new Person { Name = "John", Age = 30 });
Console.WriteLine(Encoding.UTF8.GetString(ms.ToArray()));
string noAge = "<Person><Name>John</Name></Person>";
ms = new MemoryStream(Encoding.UTF8.GetBytes(noAge));
object p = dcs.ReadObject(ms);
Console.WriteLine("No age: {0}", p);
string noName = "<Person><Age>45</Age></Person>";
ms = new MemoryStream(Encoding.UTF8.GetBytes(noName));
p = dcs.ReadObject(ms);
Console.WriteLine("No name: {0}", p);
}
}