Allow Invalid Date in ASP.NET Core API Service - serialization

We have a API service that we are updating and we converted some date objects from strings to DateTime objects. In the old code we tested the string if it would parse to a data time or not. If it was a bad formatted string, it would assign DateTime.Min and continue on. Now customers are sending in bad dates and it blows up since the serialization happens outside our code (MVC Controller). I am trying to find some way that when serializing a DateTime object, if it can not parse it, it just returns DateTime.Min instead of blowing up the call.
Here is the response from the API Call.
{
"date": [
"Could not convert string to DateTime: Invalid Date. Path 'date', line 3, position 24."
]
}
===== UPDATE =====
I finally found somewhere that recommended a custom JsonConverter. I finally got something that works, but there is little out there so if there is something I could do better I am all ears.
Custom Converter
public class DateConverter : JsonConverter
{
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
try
{
return DateTime.Parse(reader.Value.ToString());
}
catch (Exception ex)
{
return DateTime.MinValue;
}
}
public override bool CanConvert(Type objectType)
{
return objectType == typeof(DateTime);
}
}
DTO Class
public class Request
{
[JsonConverter(typeof(SafeDateConverter))]
public DateTime Date { get; set; }
}

Another approach introduce another property on deserialized class of type DateTime? and leave original string property as it is.
public class Request
{
public string Date { get; set; }
private DateTime? _parsedDate;
[JsonIgnore]
public DateTime? ParsedDate
{
get { return _parsedDate; }
set
{
if (DateTime.TryParse(value, out DateTime parsed)
{
_parsedDate = parsed;
return;
}
_parsed = null;
}
}
}
But having custom serializer looks better, because you need change nothing in the code which already uses deserialized object.
Suggestion:
Do not use try ... catch for serializing bad formatted dates, there is DateTime.TryParse method which will do this without throwing exceptions.
And if it is not late, you can use Nullable<DateTime> instead of having DateTime.Min as "not existing" value.

Related

Derived type's properties missing in JSON response from ASP.NET Core API

The JSON response from my ASP.NET Core 3.1 API controller is missing properties. This happens when a property uses a derived type; any properties defined in the derived type but not in the base/interface will not be serialized to JSON. It seems there is some lack of support for polymorphism in the response, as if serialization is based on a property's defined type instead of its runtime type. How can I change this behavior to ensure that all public properties are included in the JSON response?
Example:
My .NET Core Web API Controller returns this object that has a property with an interface type.
// controller returns this object
public class Result
{
public IResultProperty ResultProperty { get; set; } // property uses an interface type
}
public interface IResultProperty
{ }
Here is a derived type that defines a new public property named Value.
public class StringResultProperty : IResultProperty
{
public string Value { get; set; }
}
If I return the derived type from my controller like this:
return new MainResult {
ResultProperty = new StringResultProperty { Value = "Hi there!" }
};
then the actual response includes an empty object (the Value property is missing):
I want the response to be:
{
"ResultProperty": { "Value": "Hi there!" }
}
While the other answers are good and solves the problem, if all you want is the general behavior to be like pre netcore3, you can use the Microsoft.AspNetCore.Mvc.NewtonsoftJson NuGet package and in Startup.cs do:
services.AddControllers().AddNewtonsoftJson()
More info here. This way, you don't need to create any extra json-converters.
I ended up creating a custom JsonConverter (System.Text.Json.Serialization namespace) which forces JsonSerializer to serialize to the object's runtime type. See the Solution section below. It's lengthy but it works well and does not require me to sacrifice object oriented principles in my API's design. (If you need something quicker and can use Newtonsoft then check out the top voted answer instead.)
Some background: Microsoft has a System.Text.Json serialization guide with a section titled Serialize properties of derived classes with good information relevant to my question. In particular it explains why properties of derived types are not serialized:
This behavior is intended to help prevent accidental exposure of data
in a derived runtime-created type.
If that is not a concern for you then the behavior can be overridden in the call to JsonSerializer.Serialize by either explicitly specifying the derived type or by specifying object, for example:
// by specifying the derived type
jsonString = JsonSerializer.Serialize(objToSerialize, objToSerialize.GetType(), serializeOptions);
// or specifying 'object' works too
jsonString = JsonSerializer.Serialize<object>(objToSerialize, serializeOptions);
To accomplish this with ASP.NET Core you need to hook into the serialization process. I did this with a custom JsonConverter that calls JsonSerializer.Serialize one of the ways shown above. I also implemented support for deserialization which, while not explicitly asked for in the original question, is almost always needed anyway. (Oddly, supporting only serialization and not deserialization proved to be tricky anyway.)
Solution
I created a base class, DerivedTypeJsonConverter, which contains all of the serialization & deserialization logic. For each of your base types, you would create a corresponding converter class for it that derives from DerivedTypeJsonConverter. This is explained in the numbered directions below.
This solution follows the "type name handling" convention from Json.NET which introduces support for polymorphism to JSON. It works by including an additional $type property in the derived type's JSON (ex: "$type":"StringResultProperty") that tells the converter what the object's true type is. (One difference: in Json.NET, $type's value is a fully qualified type + assembly name, whereas my $type is a custom string which helps future-proof against namespace/assembly/class name changes.) API callers are expected to include $type properties in their JSON requests for derived types. The serialization logic solves my original problem by ensuring that all of the object's public properties are serialized, and for consistency the $type property is also serialized.
Directions:
1) Copy the DerivedTypeJsonConverter class below into your project.
using System;
using System.Collections.Generic;
using System.Dynamic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
public abstract class DerivedTypeJsonConverter<TBase> : JsonConverter<TBase>
{
protected abstract string TypeToName(Type type);
protected abstract Type NameToType(string typeName);
private const string TypePropertyName = "$type";
public override bool CanConvert(Type objectType)
{
return typeof(TBase) == objectType;
}
public override TBase Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
// get the $type value by parsing the JSON string into a JsonDocument
JsonDocument jsonDocument = JsonDocument.ParseValue(ref reader);
jsonDocument.RootElement.TryGetProperty(TypePropertyName, out JsonElement typeNameElement);
string typeName = (typeNameElement.ValueKind == JsonValueKind.String) ? typeNameElement.GetString() : null;
if (string.IsNullOrWhiteSpace(typeName)) throw new InvalidOperationException($"Missing or invalid value for {TypePropertyName} (base type {typeof(TBase).FullName}).");
// get the JSON text that was read by the JsonDocument
string json;
using (var stream = new MemoryStream())
using (var writer = new Utf8JsonWriter(stream, new JsonWriterOptions { Encoder = options.Encoder })) {
jsonDocument.WriteTo(writer);
writer.Flush();
json = Encoding.UTF8.GetString(stream.ToArray());
}
// deserialize the JSON to the type specified by $type
try {
return (TBase)JsonSerializer.Deserialize(json, NameToType(typeName), options);
}
catch (Exception ex) {
throw new InvalidOperationException("Invalid JSON in request.", ex);
}
}
public override void Write(Utf8JsonWriter writer, TBase value, JsonSerializerOptions options)
{
// create an ExpandoObject from the value to serialize so we can dynamically add a $type property to it
ExpandoObject expando = ToExpandoObject(value);
expando.TryAdd(TypePropertyName, TypeToName(value.GetType()));
// serialize the expando
JsonSerializer.Serialize(writer, expando, options);
}
private static ExpandoObject ToExpandoObject(object obj)
{
var expando = new ExpandoObject();
if (obj != null) {
// copy all public properties
foreach (PropertyInfo property in obj.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance).Where(p => p.CanRead)) {
expando.TryAdd(property.Name, property.GetValue(obj));
}
}
return expando;
}
}
2) For each of your base types, create a class that derives from DerivedTypeJsonConverter. Implement the 2 abstract methods which are for mapping $type strings to actual types. Here is an example for my IResultProperty interface that you can follow.
public class ResultPropertyJsonConverter : DerivedTypeJsonConverter<IResultProperty>
{
protected override Type NameToType(string typeName)
{
return typeName switch
{
// map string values to types
nameof(StringResultProperty) => typeof(StringResultProperty)
// TODO: Create a case for each derived type
};
}
protected override string TypeToName(Type type)
{
// map types to string values
if (type == typeof(StringResultProperty)) return nameof(StringResultProperty);
// TODO: Create a condition for each derived type
}
}
3) Register the converters in Startup.cs.
services.AddControllers()
.AddJsonOptions(options => {
options.JsonSerializerOptions.Converters.Add(new ResultPropertyJsonConverter());
// TODO: Add each converter
});
4) In requests to the API, objects of derived types will need to include a $type property. Example JSON: { "Value":"Hi!", "$type":"StringResultProperty" }
Full gist here
The documentation shows how to serialize as the derived class when calling the serializer directly. The same technique can also be used in a custom converter that we then can tag our classes with.
First, create a custom converter
public class AsRuntimeTypeConverter<T> : JsonConverter<T>
{
public override T Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
return JsonSerializer.Deserialize<T>(ref reader, options);
}
public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
{
JsonSerializer.Serialize(writer, value, value?.GetType() ?? typeof(object), options);
}
}
Then mark the relevant classes to be used with the new converter
[JsonConverter(typeof(AsRuntimeTypeConverter<MyBaseClass>))]
public class MyBaseClass
{
...
Alternately, the converter can be registered in startup.cs instead
services
.AddControllers(options =>
.AddJsonOptions(options =>
{
options.JsonSerializerOptions.Converters.Add(new AsRuntimeTypeConverter<MyBaseClass>());
}));
I had a similar issue, where I was returning an enumerable of type TAnimal (but the object instances were of derived types such as Dog, Cat, etc.):
[HttpGet]
public IEnumerable<TAnimal> GetAnimals()
{
IEnumerable<TAnimal> list = GetListOfAnimals();
return list;
}
This only included properties defined in TAnimal.
However, in ASP .NET Core 3.1 at least, I found that I could just cast the object instances to object, and the JSON serializer then included all the properties from the derived classes:
[HttpGet]
public IEnumerable<object> GetAnimals()
{
IEnumerable<TAnimal> list = GetListOfAnimals();
return list.Select(a => (object)a);
}
(Note that the signature of the GetAnimals method must also changed, but that doesn't usually matter much in a web API context). If you need to provide type information for Swagger or whatever, you can annotate the method:
[HttpGet]
[Produces(MediaTypeNames.Application.Json, Type = typeof(TAnimal[]))]
public IEnumerable<object> GetAnimals()
{
...
}
Casting to object is a simple solution if you only have a 1-layer-deep object hierarchy to worry about.
This is the expected result. You're upcasting when you do that, so what will be serialized is the upcasted object, not the actual derived type. If you need stuff from the derived type, then that has to be the type of the property. You may want to use generics for this reason. In other words:
public class Result<TResultProperty>
where TResultProperty : IResultProperty
{
public TResultProperty ResultProperty { get; set; } // property uses an interface type
}
Then:
return new Result<StringResultProperty> {
ResultProperty = new StringResultProperty { Value = "Hi there!" }
};
I solved it by writing this extension:
public static class JsonSerializationExtensions
{
public static string ToJson<T>(this IEnumerable<T> enumerable, bool includeDerivedTypesProperties = true)
where T : class
{
var jsonOptions = new JsonSerializerOptions()
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};
if (includeDerivedTypeProperties)
{
var collection = enumerable.Select(e => e as object).ToList();
return JsonSerializer.Serialize<object>(collection, jsonOptions);
}
else
{
return JsonSerializer.Serialize(enumerable, jsonOptions);
}
}
}
I was also struggling with this in a .NET Core 3.1 API, where I wanted the result to include $type attribute.
As suggested, install the correct package and then 'AddNewtonsoftJson'.
I wanted the $type field to be added to show the derived type handling, to get that
services.AddControllers().AddNewtonsoftJson(options =>
{
options.SerializerSettings.TypeNameHandling = Newtonsoft.Json.TypeNameHandling.All;
});
Not knocking Newtonsoft, but I found an easier way to resolve this with the built handlers.
[OperationContract]
[WebInvoke(Method = "GET", UriTemplate = "/emps", BodyStyle = WebMessageBodyStyle.Wrapped, RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json)]
List<emp> GetEmps();
//[DataContract(Namespace = "foo")] <<< comment/removed this line
public class emp
{
public string userId { get; set; }
public string firstName { get; set; }
}
public class dept
{
public string deptId{ get; set; }
public string deptName{ get; set; }
}
In my case dept objects where working fine, but emp ones were not - they came across as empty.

Using [FromQuery] to parse enum values

I'm using the [FromQuery] attribute to parse a Get requests arguments into a complex object. For example:
[HttpGet("some-get-request")]
public IActionResult DoSomething([FromQuery] SomeArguments someArgs)
{
}
One of the properties of the SomeArguments object is an enum.
public enum SomeEnum { EnumValue01, EnumValue02 }
public class SomeArguments
{
[FromQuery(Name = "enum_arg")]
public SomeEnum EnumArgument { get; set; }
}
And I call the endpoint with something like:
http://localhost:1234/api/some-controller/some-get-request?enum_arg=EnumValue01
And this all works great. However, I want to be able to use a different enum value in the URL than in my C# enum value. For example, I want to call using a URL such as
http://localhost:1234/api/some-controller/some-get-request?enum_arg=eval01
How can I do this?
I thought I could use the [FromQuery] attribute, like I can with properties, but that doesnt seem to be possible:
'FromQuery' is not valid on this declaration type. It is only valid on 'property, indexer, parameter'
You can use EnumMemberAttribute in conjunction with StringEnumConverter to achieve your goal. Define SomeEnum as following
[JsonConverter(typeof(StringEnumConverter))]
public enum SomeEnum
{
[EnumMember(Value = "eval01")]
EnumValue01,
[EnumMember(Value = "eval02")]
EnumValue02
}
At this point it will work as you need only when Newtonsoft json serializer is used. For example, when controller expects POST request and parameter is marked as [FromBody]. In your case it won't work yet because during binding of [FromQuery] parameter json serializer is not used. To solve this one create custom model binder
public class JsonModelBinder : IModelBinder
{
public Task BindModelAsync(ModelBindingContext bindingContext)
{
string rawData = bindingContext.ValueProvider.GetValue(bindingContext.ModelName).FirstValue;
rawData = JsonConvert.SerializeObject(rawData); //turns value to valid json
try
{
SomeEnum result = JsonConvert.DeserializeObject<SomeEnum>(rawData); //manually deserializing value
bindingContext.Result = ModelBindingResult.Success(result);
}
catch (JsonSerializationException ex)
{
//do nothing since "failed" result is set by default
}
return Task.CompletedTask;
}
}
Update SomeEnum definition to use JsonModelBinder
[JsonConverter(typeof(StringEnumConverter))]
[ModelBinder(typeof(JsonModelBinder))]
public enum SomeEnum

custom serializer for just one property in Json.NET

UPDATE
Found the issue -- was inheriting from wrong class, needed to be JsonConverter.
I have a class that has a Location property of type System.Data.Entity.Spatial.DbGeography. The default Json.NET serializer puts out JSON text like this:
...
"PlaceType": 0,
"Location": {
"Geography": {
"CoordinateSystemId": 4326,
"WellKnownText": "POINT (-88.00000 44.00000)"
}
},
"AddedDT": null,
...
I want it to put out text like this:
...
"PlaceType": 0,
"Location": [-88.00000,44.00000],
"AddedDT": null,
...
...so it seems to me what I should do would be to override whatever converter is currently being used on the DbGeography type.
The examples I've seen so far that use CustomCreationConverters and ContractResolvers seem to address how you'd replace the serializer for the main class being serialized, not for a type that's only a property of that class. The examples that involve annotating the class that's being overridden don't work for me because I don't define DbGeography in my code and it's effectively a sealed class because it has no constructor and can only be instantiated by internal factory methods.
Is there a way to apply a JsonConverter to a type fluently? If so, what would the converter look like? Do I just override the WriteJson() method?
You can add a custom serializer to a single attribute like this:
public class Comment
{
public string Author { get; set; }
[JsonConverter(typeof(NiceDateConverter))]
public DateTime Date { get; set; }
public string Text { get; set; }
}
public class NiceDateConverter : JsonConverter
{
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var date = (DateTime) value;
var niceLookingDate = date.ToString("MMMM dd, yyyy 'at' H:mm tt");
writer.WriteValue(niceLookingDate);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
throw new NotImplementedException("Unnecessary because CanRead is false. The type will skip the converter.");
}
public override bool CanRead
{
get { return false; }
}
public override bool CanConvert(Type objectType)
{
return objectType == typeof(DateTime);
}
}
Then, when you serialize your object with JsonConvert.SerializeObject(), the custom serializer will be used for the Date property.
Turns out I just needed to inherit from JsonConverter instead of CustomCreationConverter, and everything else I was trying to change was OK all along.
I'm still not sure if there's a way to apply the JsonConverter fluently, but there is another way to apply the JsonConverter without referencing Json.NET in your domain/core project or marking up your domain classes with references to a peripheral library:
var jsonSerializer = new JsonSerializer();
jsonSerializer.Converters.Add(new DbGeographyConverter());
jsonSerializer.Serialize(jsonWriter, place);
Use the JsonConverterAttribute on the property and define a custom converter-
for example, we have a property that comes in as a unix value (long int) and we serialize it to a .Net DateTime:
[JsonConverter(typeof(UnixTimeJsonConverter))]
public DateTime Requested { get; set; }

How to serialize a "union-like" field in C# with Json.NET

I am attempting to generate a JSON file that will be used within the Dojo javascript framework and would like to return a position attribute to be used in a dojo.place() call. The position parameter can be either a number or a string.
Using the StructLayoutwould not seem to work as-is since the serializer would try to emit both the String and Integer types. I'm looking at creating a custom ContractResolver that overrides the CreatePrimitiveContract to return a custom JsonConverter class. However, looking a the API, it appears that the JsonConverter is created based on type, and not a specific object value.
How can I handle this case in C# using the Json.NET serializer?
Presumably the solution would involve two properties with custom setters to null out the other property when one is set in conjunction with some sort of custom Json.Net class to inspect the values of the properties and only serialize the non-null one.
** Hypothetical Example **
// C# struct (or class)
[StructLayout(LayoutKind.Explicit)]
struct DojoPosition {
[JsonProperty(PropertyName="position")]
[FieldOffset(0)]
public String StrPos;
[JsonProperty(PropertyName="position")]
[FieldOffset(0)]
public Int32 IntPos;
}
// Serialization output
DojoPosition pos;
pos.StrPos = "only";
var output = JsonConvert.SerializeObject(pos);
// Output is: { "position": "only" }
pos.IntPos = 3;
var output = JsonConvert.SerializeObject(pos);
// Output is: { "position": 3 }
I just had a similiar problem.
For simple manipulation of a contract look there: Overriding the serialization behaviour in Json.Net
For resolving a JsonPrimitiveContract override the CreateContract method.
Here is an example based on our solution:
public class JsonDotNetContractResolver : DefaultContractResolver
{
protected override JsonContract CreateContract(Type objectType)
{
if (typeof(DojoPosition).IsAssignableFrom(objectType))
{
return new JsonPrimitiveContract(objectType.GetGenericArguments()[1])
{
CreatedType = typeof(object), // Not sure this will work for you, or is necessary...
IsReference = false,
Converter = DojoPositionConverter,
};
}
return base.CreateContract(objectType);
}
private class DojoPositionConverter : JsonConverter
{
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var dp = (DojoPosition) value;
if(string.IsNullOrEmpty(dp.StrPos))
serializer.Serialize(writer,dp.IntPos);
else
serializer.Serialize(writer,dp.StrPos);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
//...
}
public override bool CanConvert(Type objectType)
{
//....
}
}
}
How to determine the type to deserialize from the reader is your homework ;)

is it possible to use sqlite manifest typing features in nhibernate?

this was posted on hibernate.org forums and nhusers list without much luck, so I thought I would try here.
Put simply, suppose I have a class:
class A
{
public virtual object SomeValue { get; set; }
}
the type of SomeValue is basically in the set of .NET IConvertible types (primitives like bool, byte, char, int16, double, float etc.), plus byte[] and string.
I am trying to create a nhibernate mapping for A to reflect this - so that I can basically set SomeValue to an arbitrary object (of one of the types above) and retrieve it later on on. My applogic will then reflect on it to find the type and behave accordingly.
So far I have tried Creating an implementation of IUserType to try and handle this. However I don't know what to return for the SqlType[] SqlTypes. I considered new SqlType(DbType.Object) but when I try to generate a schema from this I get a System.ArgumentException: Dialect does not support DbType.Object
If I try another data type then I get various cast exceptions when trying to convert the type. For instance if i use a DbType.Binary, and set someValue to an int32, upon attempting to commit I get System.InvalidCastException: Unable to cast object of type 'System.Int32' to type 'System.Byte[]'.
Is there a way to achieve this?
Attached code below for a non-working implementation of IUserType (based on http://intellect.dk/post/Implementing-custom-types-in-nHibernate.aspx )
public class DataElementType : IUserType
{
SqlType baseType = new SqlType(DbType.Binary);
public SqlType[] SqlTypes
{
get
{
return new[] { baseType };
}
}
public System.Type ReturnedType
{
get { return typeof(object); }
}
public new bool Equals(object x, object y)
{
if (x == null)
return false;
else
return x.Equals(y);
}
public int GetHashCode(object x)
{
return x.GetHashCode();
}
public object NullSafeGet(IDataReader rs, string[] names, object owner)
{
return rs[names[0]];
}
public void NullSafeSet(IDbCommand cmd, object value, int index)
{
var param = new SQLiteParameter(baseType.DbType, value);
cmd.Parameters.Insert(index, param);
}
public object DeepCopy(object value)
{
if (value == null) return null;
return value;
}
public bool IsMutable
{
get { return false; }
}
public object Replace(object original, object target, object owner)
{
return original;
}
public object Assemble(object cached, object owner)
{
return cached;
}
public object Disassemble(object value)
{
return value;
}
}
It turns out to get around the problem with SqlType(DbType.Object) being unsupported by the Dialect, we make it supported by subclassing the SQLiteDialect with explicit support:
public class SQLiteDialectWithManifestTyping : SQLiteDialect
{
public SQLiteDialectWithManifestTyping() : base()
{
base.RegisterColumnType(DbType.Object, "NONE");
}
}
To use this dialect in Fluent, call Dialect() on your SQLiteConfiguration object. In NHibernate, set the configuration property dialect appropriatelly (see section 3.5.1 of the ref manual).
Then we can apply the above DataElementType implementation for our mappings (need to change the SqlTypes definition to this:
public SqlType[] SqlTypes
{
get
{
return new[] { new SqlType(DbType.Object) };
}
}
Notes:
It is not perfect. There is a tendency to upcast all discrete numbers to Int64 and floats to double.
There is no implicit way to store large unsigned values (e.g. values of ulong >= long.MaxValue) but this is a general sqlite problem (and possible a general ado.net problem?).
Due to the loss of compile time checking it is probably desireable to put some runtime checks in the NullSafeSet method to ensure the value is a primitive type. Attempting to store general objects seems to just cause the objects ToString() method to be called.