Is there an issue with the custom object JSON serialization implementation of System.IdentityModel.Tokens.Jwt for .Net 6? - asp.net-core

We migrated from .Net Core (dotnet core) 3.1 to .Net 6. We were using the System.IdentityModel.Tokens.Jwt to create a payload and generate a security token with that payload.
Our application has yet to be migrated from Newtonsoft.Json to System.Text.Json due to a lot of nonstandard property serialization that is currently favoring the former. The custom claim value contains an object that was previously serialized properly by adhering to the camelCase contract resolver that was specified in the Startup.cs configuration with regards to JSON serialization.
We upgraded from version 5.5.0 of System.IdentityModel.Tokens.Jwt to version 6.16.0 and the serialization is behaving differently.
We are using a mixture of IdentityModel well-known claims along with a custom claim. The custom claim is the only one that is an object and also the only one behaving this way. All other claims are primitive types and are written to the token as specified and expected.
This is an example of the code that is not working:
var payload = new JwtPayload()
{
{JwtRegisteredClaimNames.Iss, issuer},
{JwtRegisteredClaimNames.Iat, now},
{JwtRegisteredClaimNames.Nbf, now},
{JwtRegisteredClaimNames.Exp, exp},
{JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString("N")},
{"role", user.UserType},
{"customClaim", customClaimObjectInstance }
};
var jwt = new JwtSecurityToken(_jwtHeader, payload);
/* Token below contains a valid base64 encoded JWT Token with
the customClaim property containing pascal values that match
the properties of the C# Poco object and not at all following
either default convention or even any JsonProperty description
attributes such as [JsonProperty("name")] that may decorate each
property of the custom object */
var token = _jwtSecurityTokenHandler.WriteToken(jwt);
My first hunch was such that it may be related to a conflict with default library of System.Text.Json. I proceeded to troubleshoot by adding the [JsonPropertyName("name")] attribute to some of the properties but did not succeed. I expected that if System.Text.Json was being used that at least those description attributes would be respected or consulted during the serialization of the claim object.
I also tried serializing the value with Newtonsoft JsonConverter.Serialize function and use the serialized value as the value of the claim key-value-pair. However, the stringified object quotes were escaped and found plenty of escaping characters ("****") all over the value which was undesired.

After some time searching online and trying to come up with the right keywords to search google and GitHub and I finally got to what I, for now, consider a workaround and not a long-term solution.
The clue was provided by this open issue on Github. I simply, to my interpretation, forced the use of the Newtonsoft serializing and deserializing delegates by specifying the following lines before instantiation of payload variable posted in the question:
JsonExtensions.Serializer = JsonConvert.SerializeObject;
JsonExtensions.Deserializer = JsonConvert.DeserializeObject;
This was the first indication of potentially System.Text.Json being forced from deep within a library. It may also be an indication that the time has come to prioritize the migration to System.Text.Json from Newtonsoft.Json.
I hope this workaround helps somebody else get to this short-term patch, and not spend as much as I did.
If I find anymore concrete ideas or clues about this matter, I will update this answer.
The code below works
/* This was the key to achieving the prior result, however
temporary it may be. */
JsonExtensions.Serializer = JsonConvert.SerializeObject;
JsonExtensions.Deserializer = JsonConvert.DeserializeObject;
var payload = new JwtPayload()
{
{JwtRegisteredClaimNames.Iss, issuer},
{JwtRegisteredClaimNames.Iat, now},
{JwtRegisteredClaimNames.Nbf, now},
{JwtRegisteredClaimNames.Exp, exp},
{JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString("N")},
{"role", user.UserType},
{ "customClaim", customClaimObjectInstance}
};
var jwt = new JwtSecurityToken(_jwtHeader, payload);
var token = _jwtSecurityTokenHandler.WriteToken(jwt);
I am thankful for the issue on github, but more importantly to the solution suggested here.

Related

API Versioning in .NET Core - AssumeDefaultVersionWhenUnspecified

I'm looking into adding versioning to an existing API we have. We are embedding the version in the URL.
The requirement for the versioning is that it should be easy to add a new version, but not everything changes and we don't want to go through all controllers and add new version attributes when we get a new version. Is there any way to tell Microsoft.AspNetCore.Mvc.Versioning that a particular method should be available regardless of versions?
I have tried decorating my method with both with the version:apiVersion and with just a basic route.
[Route("v{version:apiVersion}/customers/{customerId}/estates/{estateId:int}/meters/{meterNumber}-{installationId}/consumption/detail")
[Route("customers/{customerId}/estates/{estateId:int}/meters/{meterNumber}-{installationId}/consumption/detail")]
My configuration is as such (the version number 3 is just for testing purposes):
services.AddApiVersioning(config =>
{
config.DefaultApiVersion = new ApiVersion(3, 0);
config.AssumeDefaultVersionWhenUnspecified = true;
config.ReportApiVersions = true;
});
When calling the endpoint with this configuration I get "The HTTP resource that matches the request URI 'http://{{host}}/api/customers/{customer}/estates/{estate}/meters/{meter}-{installation}/consumption/detail' is not supported
As soon as I add the [ApiVersion("3.0")] attribute everything works. But my thought was that if I currently run on version 2, but changes to other parts of the API warrant a new version and the default version of the API, I don't want to have to go this controller and "bump" the version. It should just continue to respond, unless I specify something specific.
If I change the AssumeDefaultVersionWhenUnspecified to false, I get an "An API version is required, but was not specified." But I expected that it would just grab the route without the version?
I have read about the limitations here: https://github.com/Microsoft/aspnet-api-versioning/wiki/Known-Limitations#url-path-segment-routing-with-a-default-api-version
But it doesn't seem to work.
Setting AssumeDefaultVersionWhenUnspecified = true is a highly abused feature. This is only meant to facilitate backward compatibility. Once you opt into API Versioning, all API controllers have some implicit or explicit API version. Assuming a version provides a mechanism to handle the "original", unnamed version that would otherwise break existing clients that don't know to include an API version in requests.
"Is there any way to tell Microsoft.AspNetCore.Mvc.Versioning that a particular method should be available regardless of versions?"
Yes, but I'm not entirely sure that is what you're looking for. An API can be version-neutral, which means it will accept any and all API versions, including none at all. This is useful for certain types of APIs such as a /ping health check or DELETE, which typically do not change over time. Such an API can be decorated with [ApiVersionNeutral]. This can be for all APIs on a controller or a specific action. Note that once you choose this path, "there can be only one."
There is not a complete picture of your setup. It appears that you are adding API versioning to an existing set of APIs. One of the primary purposes of DefaultApiVersion is to set what the default API version should be when no other information is available. If you applied no API versions whatsoever - for example, using attributes, then this would be the implicit API version of all APIs. This prevents you from having to retrofit all of your existing controllers and code. What's less obvious is that once you apply any explicit API version, then the implicit rule is ignored. This ensures that implicit versions can be sunset. The default version often does not matter. It's simply a way for you to indicate what the initial API version originally was. It's most relevant when grandfathering in existing APIs.
Given your configuration:
// implicitly 3.0 because DefaultApiVersion = 3.0 and there
// are no explicit versions from attributes or conventions
[Route("customers/{customerId}/estates/{estateId:int}/meters/{meterNumber}-{installationId}/consumption/detail")]
public class CustomerController : ControllerBase
{
}
Grandfathered controller with implicit versioning
// explicitly 2.0 due to attributes; DefaultApiVersion is ignored
[ApiVersion("2.0")]
[Route("v{version:apiVersion}/customers/{customerId}/estates/{estateId:int}/meters/{meterNumber}-{installationId}/consumption/detail")
[Route("customers/{customerId}/estates/{estateId:int}/meters/{meterNumber}-{installationId}/consumption/detail")]
public class CustomerController : ControllerBase
{
}
Controller with explicit versioning
With the exception of supporting the backward compatible, original API version, all APIs versions are explicit and discrete; this is by design. Fuzzy matching will not work in a predictable way so it's important to have exact matches.
You seem to be describing API version interleaving - e.g. multiple API versions implemented on a single controller implementation. I presume it might look like:
[ApiVersion("2.0")]
[ApiVersion("3.0")]
[Route("v{version:apiVersion}/customers/{customerId}/estates/{estateId:int}/meters/{meterNumber}-{installationId}/consumption/detail")
[Route("customers/{customerId}/estates/{estateId:int}/meters/{meterNumber}-{installationId}/consumption/detail")]
public class CustomerController : ControllerBase
{
// implicitly maps to 2.0 from the controller, but does not
// match 3.0 because there is an explicit 3.0 mapping
[HttpGet]
public IActionResult Get(
string customerId,
int estateId,
string meterNumber,
string installationId ) => Ok();
// explicitly maps to 3.0 only
[MapToApiVersion("3.0")]
[HttpGet]
public IActionResult GetV3(
string customerId,
int estateId,
string meterNumber,
string installationId ) => Ok();
// explicitly maps to 3.0 only
// a 2.0 request produces 405
[MapToApiVersion("3.0")]
[HttpPut]
public IActionResult Put(
string customerId,
int estateId,
string meterNumber,
string installationId
[FromBody] ComsumptionDetail detail ) => Ok();
}
Controller with version interleaving
The DefaultApiVersion can only apply a single, implicit API version. This means that no interleaving with it can ever occur. Once you start explicitly adding versions, you must also start including the implicit version because there is no other way to indicate that the implicit version should no longer be matched.
In terms of route definitions and API version mapping, there are several methods that can be used to minimize code churn. The VersionByNamespaceConvention is a built-in convention you can apply which will derive an API version from the .NET namespace of the controller type. This can make it really easy to add, remove, and organize controllers with their API versions. They can also use interleaving because the mapping is additive, but that may be confusing to maintain. You can define your own conventions as well.
Double-route registration is a consequence of versioning by URL segment. It is the least RESTful method of versioning because it violates the Uniform Interface constraint. No other method of versioning has this issue. I advise against floating the default route and API version mapping because you are very likely to break clients at some point. If you're only using double-routes for backward compatibility of your original API version, then you should be fine. When you eventually create new controllers for future API versions, you'll only have one route with the API version in the template.

WebAPI - How do I get XML to output honour namespaces in the response content?

I am building an API using the WebAPI in ASP.NET MVC 4.0. I have built model classes based on sample XML supplied by my Business Analyst using the new, super smart Paste XML as Classes feature.
The problem is, when my client Accepts application/xml, the serialized response doesn't look like the original XML.
I have manually deserialized and serialized some XML (roundtrip) using the XMLSerializer, and although its better, closer to the original, it still lacks some namespace prefixes.
How can I ensure the output is exactly on spec.?
Firstly, you'll need to ensure the WebAPI is using the XmlSerializer to format your WebAPI responses, or at least use the XmlSerializer just for this resource/API.
In the WebApiConfig.cs file you'll find the route registration stuff and also some commented-out code.
Add under that chunk, add the following:
var xmlSerializer = new XmlSerializer(typeof(FruitXmlModel));
config.Formatters.XmlFormatter.SetSerializer<FruitXmlModel>(xmlSerializer);
This will specify the old XmlSerializer be used when serializing the FruitXmlModel CLR type. Note that you'll probably need to reference the XML serialization assemblies.
Next, you'll need to add this code to the class in your model that represents the element in your XML that begins the new namespace.
...
[XmlNamespaceDeclarations]
public XmlSerializerNamespaces Namespaces = new XmlSerializerNamespaces();
public FruitXmlModel() // ctor for one of my models
{
this.Namespaces.Add("banana", "http://www.fruitschema.org/banana");
}
...
If you've used the Paste XML as Classes feature, then this class should already be annotated with the correct attributes XmlTypeAttribute with the correct namespace set.
All being well, this simple change will provide the WebAPI and the XmlSerializer all that's needed to produce the properly prefixed XML output.
I wish you good luck, Luke, hope to meet again soon.

JSON.NET JsonIgnore DeserializeObject

I created a Web API using VS 2012. I have a method with a custom object parameter that I am passing JSON to via Fiddler for testing:
[HttpPost, HttpPut]
public HttpResponseMessage UpsertProject(Projects p)
{
...
}
My Projects object has about a dozen properties marked as JsonIgnore. My assumption was that when my object was serialized into Json those properties would be ignored...which is true. However, when I debug my method I'm noticing that all the object properties marked with JsonIgnore are set to null even if the Json that I pass in from Fiddler is setting them. I also try to get data as Json and deserialize it into a new instance of the object but that also does not set the properties that are marked JsonIngore. I knew JsonIgnore would work for serializing but didn't think it would prevent properties from being set when deserializing. What's frustrating is I know that ScriptIgnore doesn't behave this way, but I want to use JSON.net to handle my serializing/deserializing. I've also created a windows app and tested the same serializing/deserializing functionality and it works in it. So I'm wondering if this is a Web API limitation with the JsonIgnore attribute?
If it works the way you want in the Windows application but not in the Web API, that tells me that the JSON serializer settings are different between the two. What settings are you using in the Windows app that makes it work? You can take those settings and apply them to the Web API serializer in the Register method of the WebApiConfig class (in the App_Start folder of your Web API project). For example:
JsonSerializerSettings jsonSettings = config.Formatters.JsonFormatter.SerializerSettings;
jsonSettings.NullValueHandling = NullValueHandling.Ignore;
jsonSettings.DefaultValueHandling = DefaultValueHandling.IgnoreAndPopulate;
...
EDIT
OK, so if I understand you correctly, based on your most recent comments, you want everything to be deserialized if possible, but you only want two specific properties to be serialized and those apparently do not have null or default values. In that case, there are two approaches you can take:
Set the properties that you don't want serialized to null or zero or false (the default value) just before serializing. Because you have DefaultValueHandling set to Ignore, this will cause those properties not to be serialized.
Create several boolean ShouldSerializeXXX() methods in your class where XXX is the name of each property you don't want serialized. These methods should return false. See the first answer of this question for an example.
Don't use JsonIgnore because, as you have seen, this will cause the property to be completely ignored by Json.Net, both for serializing and deserializing.

asp.net web api having trouble sending objects

I've just switched to using the new web api for MVC 4, and I am having great difficulty in deserializing the response from my webservice.
This is my server code:
IEnumerable<Fish> myList = GetFish();
return Request.CreateResponse(HttpStatusCode.OK, myList,"application/xml");
This is serialised as:
<ArrayOfFish xmlns:i=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns=\"http://schemas.datacontract.org/2004/07/MySolution.NameSpace.Resources\" />
As there are no results returned.
When I try to parse this using the XmlMediaTypeFormatter, I get an error saying it wasn't expecting what it got.
My code to deserialise it hasn't changed from before I started using web api, and is simply:
return content.ReadAsAsync<T>(formatters).Result;
formatters is a List of formatters, containing only the XmlMediaTypeFormatter
T is a generic list of Fish
If I omit the "application/xml" type then it appears to send in JSon (as the result is []) however the client expects xml, and will not use a JSon serialiser on it for some reason, even if I explicitly put the type as text/json.
It should be fairly simple to deserialise objects as it's exactly what I was doing before, I've just changed my server very slightly to use CreateResponse to make a HttpResponseMessage instead of using HttpResponseMessage<T> directly, because the generic version isn't supported anymore. I can't find a single client/server example online of someone decoding the result into objects which is frustrating.
Any ideas?
An XmlSerializer can't serialize an interface (IEnumerable). Convert to a List before returning.
return Request.CreateResponse(HttpStatusCode.OK, myList.ToList(),"application/xml");
simlar question with references here - XmlSerializer won't serialize IEnumerable
I ended up discovering that the serialization method web api uses is very slightly different to the former, standard way. I added this to my global.asax and it resolved the issue:
GlobalConfiguration.Configuration.Formatters.Insert(0, new XmlMediaTypeFormatter() { UseXmlSerializer = true });

EF4 POCO WCF Serialization problems (no lazy loading, proxy/no proxy, circular references, etc)

OK, I want to make sure I cover my situation and everything I've tried thoroughly. I'm pretty sure what I need/want can be done, but I haven't quite found the perfect combination for success.
I'm utilizing Entity Framework 4 RTM and its POCO support. I'm looking to query for an entity (Config) that contains a many-to-many relationship with another entity (App). I turn off lazy loading and disable proxy creation for the context and explicitly load the navigation property (either through .Include() or .LoadProperty()). However, when the navigation property is loaded (that is, Apps is loaded for a given Config), the App objects that were loaded already contain references to the Configs that have been brought to memory. This creates a circular reference.
Now I know the DataContractSerializer that WCF uses can handle circular references, by setting the preserveObjectReferences parameter to true. I've tried this with a couple of different attribute implementations I've found online. It is needed to prevent the "the object graph contains circular references and cannot be serialized" error. However, it doesn't prevent the serialization of the entire graph, back and forth between Config and App.
If I invoke it via WcfTestClient.exe, I get a stackoverflow (ha!) exception from the client and I'm hosed. I get different results from different invocation environments (C# unit test with a local reference to the web service appears to work ok though I still can drill back and forth between Configs and Apps endlessly, but calling it from a coldfusion environment only returns the first Config in the list and errors out on the others.) My main goal is to have a serialized representation of the graph I explicitly load from EF (ie: list of Configs, each with their Apps, but no App back to Config navigation.)
NOTE: I've also tried using the ProxyDataContractResolver technique and keeping the proxy creation enabled from my context. This blows up complaining about unknown types encountered. I read that the ProxyDataContractResolver didn't fully work in Beta2, but should work in RTM.
For some reference, here is roughly how I'm querying the data in the service:
var repo = BootStrapper.AppCtx["AppMeta.ConfigRepository"] as IRepository<Config>;
repo.DisableLazyLoading();
repo.DisableProxyCreation();
//var temp2 = repo.Include(cfg => cfg.Apps).Where(cfg => cfg.Environment.Equals(environment)).ToArray();
var temp2 = repo.FindAll(cfg => cfg.Environment.Equals(environment)).ToArray();
foreach (var cfg in temp2)
{
repo.LoadProperty(cfg, c => c.Apps);
}
return temp2;
I think the crux of my problem is when loading up navigation properties for POCO objects from Entity Framework 4, it prepopulates navigation properties for objects already in memory. This in turn hoses up the WCF serialization, despite every effort made to properly handle circular references.
I know it's a lot of information, but it's really standing in my way of going forward with EF4/POCO in our system. I've found several articles and blogs touching upon these subjects, but for the life of me, I cannot resolve this issue. Feel free to simply ask questions and help me brainstorm this situation.
PS: For the sake of being thorough, I am injecting the WCF services using the HEAD build of Spring.NET for the fix to Spring.ServiceModel.Activation.ServiceHostFactory. However I don't think this is the source of the problem.
EDIT: The ProxyDataContractResolver class works correctly if I don't have the circular references. (i.e.: I make the setter of App.Configs to be private, which prevents serialization of the property.) It blows up, it appears, when it hits Configs via the App object -- they don't seem to be recognized as the same type as the top level Configs.
EDIT2: It appears that either EF or WCF doesn't recognize that the entities are indeed equal. i.e.: 'Config' is the same as a particular 'Config.Apps[x].Configs[y]'. The entity keys are properly set in the CSDL for each model and I've overridden the Equals() function to compare entities based on their 'Id' property. This fits the symptoms as no circular reference error is thrown, yet it is indeed a circular reference (and blows up WcfTestClient.exe) AND the ProxyDataContractResolver blows up when it hits the 'Config.Apps[x].Configs[y]' level of Configs. (It doesn't know how to map a Config proxy. The ProxyDataContractResolver works otherwise. It's like it knows how to handle the initial round of entities, but the second level it considers as different entities.)
Wow, I can be wordy. Sorry folks!
You might want to check out my blog post on this specific scenario - please email me if it doesn't help fix your current predicament! I've included a sample solution as well.
Please drop me some feedback either way, I'd really like to hear from more people on this particular issue - especially with the implementation problems on the client end of things.
hrmm, I may not have fully understood the issue, but everytime I run into circular references with WCF the answer is to change [DataContract] on the offending classes to [DataContract(IsReference = true)].
This is a huge help compared to all the drek of messing with the contract resolvers that was needed pre WCF 3.5 SP1.
Hope this helps.
Faced the same issue today and used Value Injecter to solve it. It's as simple as:
var dynamicProxyMember = _repository.FindOne<Member>(m=>m.Id = 1);
var member = new Member().InjectFrom(dynamicProxyMember) as Member;
We couldnt afford disabling ProxyCreation
Try setting myContext.ContextOptions.ProxyCreationEnabled = false;
If the problem is solved (like mine) then you've not followed the steps mentioned in: http://msdn.microsoft.com/en-us/library/ee705457.aspx
This solved the problem for me.
You can use the ApplyDataContractResolverAttribute and a ProxyDataContractResolver along with the CyclicReferencesAwareAttribute. At first this produces error like this one - as if there is no DataContractResolver specified at all:
Type 'System.Data.Entity.DynamicProxies.Whatever_E6......A9' with data contract name 'Whatever_E6......A9:http://schemas.datacontract.org/2004/07/System.Data.Entity.DynamicProxies' is not expected. Consider using a DataContractResolver or add any types not known statically to the list of known types - for example, by using the KnownTypeAttribute attribute or by adding them to the list of known types passed to DataContractSerializer.
It will work with one simple change.
In the ApplyCyclicDataContractSerializerOperationBehavior, the constructors for the DataContractSerializer must also pass in the DataContractResolver. This is left out of all the versions I have seen online.
Example:
public class ApplyCyclicDataContractSerializerOperationBehavior : DataContractSerializerOperationBehavior
{
private readonly Int32 _maxItemsInObjectGraph;
private readonly bool _ignoreExtensionDataObject;
public ApplyCyclicDataContractSerializerOperationBehavior(OperationDescription operationDescription, Int32 maxItemsInObjectGraph, bool ignoreExtensionDataObject, bool preserveObjectReferences)
: base(operationDescription)
{
_maxItemsInObjectGraph = maxItemsInObjectGraph;
_ignoreExtensionDataObject = ignoreExtensionDataObject;
}
public override XmlObjectSerializer CreateSerializer(Type type, String name, String ns, IList<Type> knownTypes)
{
return new DataContractSerializer(type, name, ns, knownTypes,
_maxItemsInObjectGraph,
_ignoreExtensionDataObject,
true,
null /*dataContractSurrogate*/,
DataContractResolver); // <-----------------------------
}
public override XmlObjectSerializer CreateSerializer(Type type, XmlDictionaryString name, XmlDictionaryString ns, IList<Type> knownTypes)
{
return new DataContractSerializer(type, name, ns, knownTypes,
_maxItemsInObjectGraph,
_ignoreExtensionDataObject,
true,
null /*dataContractSurrogate*/,
DataContractResolver); // <-----------------------------
}
}