The JSON response from my ASP.NET Core 3.1 API controller is missing properties. This happens when a property uses a derived type; any properties defined in the derived type but not in the base/interface will not be serialized to JSON. It seems there is some lack of support for polymorphism in the response, as if serialization is based on a property's defined type instead of its runtime type. How can I change this behavior to ensure that all public properties are included in the JSON response?
Example:
My .NET Core Web API Controller returns this object that has a property with an interface type.
// controller returns this object
public class Result
{
public IResultProperty ResultProperty { get; set; } // property uses an interface type
}
public interface IResultProperty
{ }
Here is a derived type that defines a new public property named Value.
public class StringResultProperty : IResultProperty
{
public string Value { get; set; }
}
If I return the derived type from my controller like this:
return new MainResult {
ResultProperty = new StringResultProperty { Value = "Hi there!" }
};
then the actual response includes an empty object (the Value property is missing):
I want the response to be:
{
"ResultProperty": { "Value": "Hi there!" }
}
While the other answers are good and solves the problem, if all you want is the general behavior to be like pre netcore3, you can use the Microsoft.AspNetCore.Mvc.NewtonsoftJson NuGet package and in Startup.cs do:
services.AddControllers().AddNewtonsoftJson()
More info here. This way, you don't need to create any extra json-converters.
I ended up creating a custom JsonConverter (System.Text.Json.Serialization namespace) which forces JsonSerializer to serialize to the object's runtime type. See the Solution section below. It's lengthy but it works well and does not require me to sacrifice object oriented principles in my API's design. (If you need something quicker and can use Newtonsoft then check out the top voted answer instead.)
Some background: Microsoft has a System.Text.Json serialization guide with a section titled Serialize properties of derived classes with good information relevant to my question. In particular it explains why properties of derived types are not serialized:
This behavior is intended to help prevent accidental exposure of data
in a derived runtime-created type.
If that is not a concern for you then the behavior can be overridden in the call to JsonSerializer.Serialize by either explicitly specifying the derived type or by specifying object, for example:
// by specifying the derived type
jsonString = JsonSerializer.Serialize(objToSerialize, objToSerialize.GetType(), serializeOptions);
// or specifying 'object' works too
jsonString = JsonSerializer.Serialize<object>(objToSerialize, serializeOptions);
To accomplish this with ASP.NET Core you need to hook into the serialization process. I did this with a custom JsonConverter that calls JsonSerializer.Serialize one of the ways shown above. I also implemented support for deserialization which, while not explicitly asked for in the original question, is almost always needed anyway. (Oddly, supporting only serialization and not deserialization proved to be tricky anyway.)
Solution
I created a base class, DerivedTypeJsonConverter, which contains all of the serialization & deserialization logic. For each of your base types, you would create a corresponding converter class for it that derives from DerivedTypeJsonConverter. This is explained in the numbered directions below.
This solution follows the "type name handling" convention from Json.NET which introduces support for polymorphism to JSON. It works by including an additional $type property in the derived type's JSON (ex: "$type":"StringResultProperty") that tells the converter what the object's true type is. (One difference: in Json.NET, $type's value is a fully qualified type + assembly name, whereas my $type is a custom string which helps future-proof against namespace/assembly/class name changes.) API callers are expected to include $type properties in their JSON requests for derived types. The serialization logic solves my original problem by ensuring that all of the object's public properties are serialized, and for consistency the $type property is also serialized.
Directions:
1) Copy the DerivedTypeJsonConverter class below into your project.
using System;
using System.Collections.Generic;
using System.Dynamic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
public abstract class DerivedTypeJsonConverter<TBase> : JsonConverter<TBase>
{
protected abstract string TypeToName(Type type);
protected abstract Type NameToType(string typeName);
private const string TypePropertyName = "$type";
public override bool CanConvert(Type objectType)
{
return typeof(TBase) == objectType;
}
public override TBase Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
// get the $type value by parsing the JSON string into a JsonDocument
JsonDocument jsonDocument = JsonDocument.ParseValue(ref reader);
jsonDocument.RootElement.TryGetProperty(TypePropertyName, out JsonElement typeNameElement);
string typeName = (typeNameElement.ValueKind == JsonValueKind.String) ? typeNameElement.GetString() : null;
if (string.IsNullOrWhiteSpace(typeName)) throw new InvalidOperationException($"Missing or invalid value for {TypePropertyName} (base type {typeof(TBase).FullName}).");
// get the JSON text that was read by the JsonDocument
string json;
using (var stream = new MemoryStream())
using (var writer = new Utf8JsonWriter(stream, new JsonWriterOptions { Encoder = options.Encoder })) {
jsonDocument.WriteTo(writer);
writer.Flush();
json = Encoding.UTF8.GetString(stream.ToArray());
}
// deserialize the JSON to the type specified by $type
try {
return (TBase)JsonSerializer.Deserialize(json, NameToType(typeName), options);
}
catch (Exception ex) {
throw new InvalidOperationException("Invalid JSON in request.", ex);
}
}
public override void Write(Utf8JsonWriter writer, TBase value, JsonSerializerOptions options)
{
// create an ExpandoObject from the value to serialize so we can dynamically add a $type property to it
ExpandoObject expando = ToExpandoObject(value);
expando.TryAdd(TypePropertyName, TypeToName(value.GetType()));
// serialize the expando
JsonSerializer.Serialize(writer, expando, options);
}
private static ExpandoObject ToExpandoObject(object obj)
{
var expando = new ExpandoObject();
if (obj != null) {
// copy all public properties
foreach (PropertyInfo property in obj.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance).Where(p => p.CanRead)) {
expando.TryAdd(property.Name, property.GetValue(obj));
}
}
return expando;
}
}
2) For each of your base types, create a class that derives from DerivedTypeJsonConverter. Implement the 2 abstract methods which are for mapping $type strings to actual types. Here is an example for my IResultProperty interface that you can follow.
public class ResultPropertyJsonConverter : DerivedTypeJsonConverter<IResultProperty>
{
protected override Type NameToType(string typeName)
{
return typeName switch
{
// map string values to types
nameof(StringResultProperty) => typeof(StringResultProperty)
// TODO: Create a case for each derived type
};
}
protected override string TypeToName(Type type)
{
// map types to string values
if (type == typeof(StringResultProperty)) return nameof(StringResultProperty);
// TODO: Create a condition for each derived type
}
}
3) Register the converters in Startup.cs.
services.AddControllers()
.AddJsonOptions(options => {
options.JsonSerializerOptions.Converters.Add(new ResultPropertyJsonConverter());
// TODO: Add each converter
});
4) In requests to the API, objects of derived types will need to include a $type property. Example JSON: { "Value":"Hi!", "$type":"StringResultProperty" }
Full gist here
The documentation shows how to serialize as the derived class when calling the serializer directly. The same technique can also be used in a custom converter that we then can tag our classes with.
First, create a custom converter
public class AsRuntimeTypeConverter<T> : JsonConverter<T>
{
public override T Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
return JsonSerializer.Deserialize<T>(ref reader, options);
}
public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
{
JsonSerializer.Serialize(writer, value, value?.GetType() ?? typeof(object), options);
}
}
Then mark the relevant classes to be used with the new converter
[JsonConverter(typeof(AsRuntimeTypeConverter<MyBaseClass>))]
public class MyBaseClass
{
...
Alternately, the converter can be registered in startup.cs instead
services
.AddControllers(options =>
.AddJsonOptions(options =>
{
options.JsonSerializerOptions.Converters.Add(new AsRuntimeTypeConverter<MyBaseClass>());
}));
I had a similar issue, where I was returning an enumerable of type TAnimal (but the object instances were of derived types such as Dog, Cat, etc.):
[HttpGet]
public IEnumerable<TAnimal> GetAnimals()
{
IEnumerable<TAnimal> list = GetListOfAnimals();
return list;
}
This only included properties defined in TAnimal.
However, in ASP .NET Core 3.1 at least, I found that I could just cast the object instances to object, and the JSON serializer then included all the properties from the derived classes:
[HttpGet]
public IEnumerable<object> GetAnimals()
{
IEnumerable<TAnimal> list = GetListOfAnimals();
return list.Select(a => (object)a);
}
(Note that the signature of the GetAnimals method must also changed, but that doesn't usually matter much in a web API context). If you need to provide type information for Swagger or whatever, you can annotate the method:
[HttpGet]
[Produces(MediaTypeNames.Application.Json, Type = typeof(TAnimal[]))]
public IEnumerable<object> GetAnimals()
{
...
}
Casting to object is a simple solution if you only have a 1-layer-deep object hierarchy to worry about.
This is the expected result. You're upcasting when you do that, so what will be serialized is the upcasted object, not the actual derived type. If you need stuff from the derived type, then that has to be the type of the property. You may want to use generics for this reason. In other words:
public class Result<TResultProperty>
where TResultProperty : IResultProperty
{
public TResultProperty ResultProperty { get; set; } // property uses an interface type
}
Then:
return new Result<StringResultProperty> {
ResultProperty = new StringResultProperty { Value = "Hi there!" }
};
I solved it by writing this extension:
public static class JsonSerializationExtensions
{
public static string ToJson<T>(this IEnumerable<T> enumerable, bool includeDerivedTypesProperties = true)
where T : class
{
var jsonOptions = new JsonSerializerOptions()
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};
if (includeDerivedTypeProperties)
{
var collection = enumerable.Select(e => e as object).ToList();
return JsonSerializer.Serialize<object>(collection, jsonOptions);
}
else
{
return JsonSerializer.Serialize(enumerable, jsonOptions);
}
}
}
I was also struggling with this in a .NET Core 3.1 API, where I wanted the result to include $type attribute.
As suggested, install the correct package and then 'AddNewtonsoftJson'.
I wanted the $type field to be added to show the derived type handling, to get that
services.AddControllers().AddNewtonsoftJson(options =>
{
options.SerializerSettings.TypeNameHandling = Newtonsoft.Json.TypeNameHandling.All;
});
Not knocking Newtonsoft, but I found an easier way to resolve this with the built handlers.
[OperationContract]
[WebInvoke(Method = "GET", UriTemplate = "/emps", BodyStyle = WebMessageBodyStyle.Wrapped, RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json)]
List<emp> GetEmps();
//[DataContract(Namespace = "foo")] <<< comment/removed this line
public class emp
{
public string userId { get; set; }
public string firstName { get; set; }
}
public class dept
{
public string deptId{ get; set; }
public string deptName{ get; set; }
}
In my case dept objects where working fine, but emp ones were not - they came across as empty.
I have a simple data contract:
[DataContract]
public class MyData
{
[DataMember]
public string AwesomeData { get; set; }
}
And service contract:
[ServiceContract]
public interface IMyDataService
{
[OperationContract]
MyData GetAwesomeData();
}
In the server-side assembly, I create a derived class for doing processing:
public class MyDataWithInnerds: MyData
{
public MyDataWithInnerds(object intializationStuff)
{
AwesomeData = Hypermaxulate(initializationStuff);
}
}
and the service implementation:
public class MyDataService: IMyDataService
{
public MyData GetAwesomeData()
{
return new MyDataWithInnerds(HupnerRayvakManager.GetInitializationStuff());
}
}
MyDataWithInnerds is just the functional implementation of the purely data contract MyData.
Is there anyway without decorating MyData and referencing MyDataWithInnerds to tell the serializer to serialize MyDataWithInnerds as MyData? The assembly with MyData is also given to clients. I don't want the server-side only MyDataWithInnerds to be referenced.
I don't need to de-serialize it to MyDataWithInnerds. I'm guessing I'm going to need to clone the MyDataWithInnerds instance to a MyData instance so it doesn't have any type information associated with MyDataWithInnerds, but it would be really efficient to not have to do that and just let the serializer know "I know this is a MyDataWithInnerds, but you can just treat it like the base MyData"
Thanks,
Mike
EDIT
I was able to add the following to my config file on the server:
<system.runtime.serialization>
<dataContractSerializer>
<declaredTypes>
<add type="MyClientLib.MyData, MyClientLib">
<knownType type="MyServerLib.MyDataWithInnerds, MyServerLib"/>
</add>
</declaredTypes>
</dataContractSerializer>
</system.runtime.serialization>
And this makes the serialization work properly.
It just leaves two side questions:
In my JSON serialization, I end up with a "__type" member that contains the derived class type. Can I remove that?
Is there a declarative way to do, on the server objects (not MyData), what I have done in configuration (adding KnownType's)?
Thanks2,
Mike
In the end, I created a copy constructor for my base class and instead of returning the derived class (as the base class), I return new base(derived).
[DataContract]
public class MyData
{
[DataMember]
public string AwesomeData { get; set; }
public MyData(MyData obj)
{
AwesomeData = obj.AwesomeData;
}
}
The service implementation then becomes:
public class MyDataService: IMyDataService
{
public MyData GetAwesomeData()
{
MyDataWithInnerds data =
new MyDataWithInnerds(HupnerRayvakManager.GetInitializationStuff());
return new MyData(data);
}
}
In the simplistic example it looks like overkill, but in the actual application, it works out decently -- without having extra configuration and not emitting extraneous serialized data.
This is one of the classes in Interface file.
[DataContract]
public class ClassX
{
public ClassX()
{
ClassXParameters = new List<ClassXParameter>();
}
public void Add(string name, string value)
{
ClassXParameters.Add(new ClassXParameter() { Name = name, Value = value });
}
[DataMember]
public List<ClassXParameter> ClassXParameters { get; set; }
}
[DataContract]
public class ClassXParameter
{
[DataMember]
public string Name { get; set; }
[DataMember]
public string Value { get; set; }
}
on the client I'm trying to do something like this
ClassX classx = new ClassX();
classx.Add("testname", "testvalue");
But this .Add method is not even visible.
currently I'm doing
ClassX classx = new ClassX();
List<ClassXParameter> params = new List<ClassXParameter()>;
params.add(new ClassXParameter() {Name="testname", Value="testvalue"});
classx.ClassXParameters = params;
Is there anyway I can do what I'm trying to do?
Note: I am not sure why some of the text above are in bold.
If you autogenerate the client code from scratch, it will generate a new class, which contains those members and properties that are marked with DataContract.
If you have methods that you want available on the client, you can accomplish this by putting the DataContract types in an own assembly, which you reference from both the server and the client. When you generate the service reference you have to choose the option to reuse existing classes instead of generating new ones.
Often it is suitable to put data validation rules in the data contract classes property setters. Reusing the data contract assembly in the client will cause the data validation to occur directly on the client, without the need for a roundtrip. It also causes the error in a place where it is much easier to spot than if it is reported as deserialization error.
Data Contracts are for data only. Any methods will not be visible on the client.
The bold was because of the "-----".
What's the practical way of serializing an instance of subclass by using DataContractSerializer?
For example, here are the data types:
[DataContract]
public class Car
{
public Car()
{
Wheels = new Collection<Wheel>();
}
[DataMember]
public Collection<Wheel> Wheels { get; set; }
}
[DataContract]
public abstract class Wheel
{
[DataMember]
public string Name { get; set; }
}
[DataContract]
public class MichelinWheel : Wheel
{
[DataMember]
public string Wheel1Test { get; set; }
}
[DataContract]
public class BridgeStoneWheel : Wheel
{
[DataMember]
public string Wheel2Test { get; set; }
}
Then here is the code that creates a car with two differen wheels:
Car car = new Car();
MichelinWheel w1 = new MichelinWheel { Name = "o1", Wheel1Test = "o1 test" };
BridgeStoneWheel w2 = new BridgeStoneWheel { Name = "o2", Wheel2Test = "o2 test" };
car.Wheels.Add(w1);
car.Wheels.Add(w2);
Now if I try to serialize the car by using DataContractSerializer, I will get an exception that says MichelinWheel is not expected. And I have to modify the Wheel class like this to make it work:
[DataContract]
[KnownType(typeof(MichelinWheel))]
[KnownType(typeof(BridgeStoneWheel))]
public abstract class Wheel
{
[DataMember]
public string Name { get; set; }
}
But this approach is not practical, because I am not able to list all kinds of wheels before they are created. And changing the Wheel class every time after a new brand of wheel is created is also not practical, because they might by created in third-party code.
So, what is the practical approach of serializing an instance of a subclass when using DataContractSerializer?
Thanks
Check this article using DataContractResolver from WCF 4. You can also use KnownTypeAttribute with passing name of a method that will use reflection to get all types. Anyway service requires that all types are known before it starts.
There are several ways to make known types available to the service.
The simplest you have outlined above, but obviously this requires you to recompile when new types are added, and depending on your configuration can make it awkward to avoid circular dependencies.
You can also configure the KnownTypes:
through the service configuration file (service restart only required),
add them as service known types provided through a static method on the service interface which you could get through reflection as Ladislav Mrnka has indicated (you could probably reflect over all loaded assemblies and return all types that have the DataContact attribute on them as known types, but I couldn't find an example of that.)
implement your own way of getting them (perhaps through some bespoke configuration elements in the config file or just through a text file)
I'm using two class NiceCustomer & RoughCustomer which implment the interface ICustomer.
The ICustomer has four properties. They are:
Property Id() As Integer
Property Name() As String
Property IsNiceCustomer() As Boolean
ReadOnly Property AddressFullText() As String
I don't know how to map the interface ICustomer, to the database.
I get an error like this in the inner exception.
An association refers to an unmapped class: ICustomer
I'm using Fluent and NHibernate.
You can map directly to interfaces in NHibernate, by plugging in an EmptyInterceptor during the configuration stage. The job of this interceptor would be to provide implementations to the interfaces you are defining in your mapping files.
public class ProxyInterceptor : EmptyInterceptor
{
public ProxyInterceptor(ITypeHandler typeHandler) {
// TypeHandler is a custom class that defines all Interface/Poco relationships
// Should be written to match your system
}
// Swaps Interfaces for Implementations
public override object Instantiate(string clazz, EntityMode entityMode, object id)
{
var handler = TypeHandler.GetByInterface(clazz);
if (handler == null || !handler.Interface.IsInterface) return base.Instantiate(clazz, entityMode, id);
var poco = handler.Poco;
if (poco == null) return base.Instantiate(clazz, entityMode, id);
// Return Poco for Interface
var instance = FormatterServices.GetUninitializedObject(poco);
SessionFactory.GetClassMetadata(clazz).SetIdentifier(instance, id, entityMode);
return instance;
}
}
After this, all relationships and mappings can be defined as interfaces.
public Parent : IParent {
public int ID { get; set; }
public string Name { get; set; }
public IChild Child { get; set; }
}
public Child : IChild {
public int ID { get; set; }
public string Name { get; set; }
}
public class ParentMap : ClassMap<IParent>
{
public ParentMap()
{
Id(x => x.ID).GeneratedBy.Identity().UnsavedValue(0);
Map(x => x.Name)
}
}
...
This type of technique is great if you want to achieve true decoupling of your ORM, placing all configuration/mappings in a seperate project and only referencing interfaces. Your domain layer is then not being polluted with ORM, and you can then replace it at a later stage if you need to.
how are you querying? If you're using HQL you need to import the interface's namespace with an HBM file with this line:
<import class="name.space.ICustomer, Customers" />
If you're using Criteria you should just be able to query for ICustomer and it'll return both customer types.
If you're mapping a class that has a customer on it either through a HasMany, HasManyToMany or References then you need to use the generic form:
References<NiceCustomer>(f=>f.Customer)
If you want it to cope with either, you'll need to make them subclasses
Subclassmap<NiceCustomer>
In which case I think you'll need the base class Customer and use that for the generic type parameter in the outer class:
References<Customer>(f=>f.Customer)
Regardless, you shouldn't change your domain model to cope with this, it should still have an ICustomer on the outer class.
I'm not sure if the 1.0RTM has the Generic form working for References but a quick scan of the changes should show the change, which I think is a two line addition.
It is not possible to map an interface in nhibernate. If your goal is to be able to query using a common type to retrieve both types of customers you can use a polymorphic query. Simply have both your classes implement the interface and map the classes normally. See this reference:
https://www.hibernate.org/hib_docs/nhibernate/html/queryhql.html (section 11.6)