Service fabric remoting serializer - serialization

In experimenting with Service Fabric remoting I have some data types that are not serialized correctly. This is causing me many issues.
From the documentation it appears that everything needs to be decorated with [DataContract]. After using this on some test types it does appear that they serialize correctly.
However I frankly don't want to have to decorate everything. That would be a huge step backwards for me. I would prefer to use custom serialization all the way around.
This documentation seems to suggest that it is possible to register a custom serializer however it appears to only be for stateful services. I am primarily using remoting with stateless services.

The current remoting stack requires that your types use DataContract. Supposedly the team is close to releasing a new remoting stack in the near future that contains the ability to plug in custom serialization and a lot of improvements on the performance side but this is not available yet.
In the meantime, a workaround (not a very nice one mind you) is to make all of your proxies receive string or byte[] or something like that and take care of serialization/deserialization manually using something like JSON.Net. Personally I'd bite the bullet and make your types Data Contract Serializable until the new remoting bits are available.

With the release of Service Fabric V2 Remoting, this is now possible. See here for further details. Below
Here is an implementation of MessagePack remoting serializer I have used, but in your case the JSON example in the docs would probably suffice.
public class MessagePackMessageFactory : IServiceRemotingMessageBodyFactory
{
public IServiceRemotingRequestMessageBody CreateRequest(string interfaceName, string methodName, int numberOfParameters)
{
return new MessagePackRemotingRequestMessageBody(numberOfParameters);
}
public IServiceRemotingResponseMessageBody CreateResponse(string interfaceName, string methodName)
{
return new MessagePackServiceRemotingResponseMessageBody();
}
}
[MessagePackObject]
public class MessagePackRemotingRequestMessageBody : IServiceRemotingRequestMessageBody
{
[Key(0)]
public object Value;
public MessagePackRemotingRequestMessageBody()
{
}
public MessagePackRemotingRequestMessageBody(int parameterInfos)
{
}
public void SetParameter(int position, string paramName, object parameter)
{
Value = parameter;
}
public object GetParameter(int position, string paramName, Type paramType)
{
return Value;
}
}
[MessagePackObject]
public class MessagePackServiceRemotingResponseMessageBody : IServiceRemotingResponseMessageBody
{
[Key(0)]
public object Response;
public object Get(Type paramType)
{
// ignore paramType?
return Response;
}
public void Set(object response)
{
Response = response;
}
}
public class ServiceRemotingResponseMessagePackMessageBodySerializer : IServiceRemotingResponseMessageBodySerializer
{
public OutgoingMessageBody Serialize(IServiceRemotingResponseMessageBody responseMessageBody)
{
if (!(responseMessageBody is MessagePackServiceRemotingResponseMessageBody body))
{
return new OutgoingMessageBody(new[] { new ArraySegment<byte>(new byte[0]) });
}
var bytes = MessagePackSerializer.Serialize(body, ServiceFabricResolver.Instance);
return new OutgoingMessageBody(new[] { new ArraySegment<byte>(bytes) });
}
public IServiceRemotingResponseMessageBody Deserialize(IncomingMessageBody messageBody)
{
using (var stream = messageBody.GetReceivedBuffer())
{
if (stream.Length == 0)
{
return new MessagePackServiceRemotingResponseMessageBody();
}
var body = MessagePackSerializer.Deserialize<MessagePackServiceRemotingResponseMessageBody>(stream, ServiceFabricResolver.Instance);
return body;
}
}
}
public class ServiceRemotingMessagePackSerializationProvider : IServiceRemotingMessageSerializationProvider
{
public IServiceRemotingRequestMessageBodySerializer CreateRequestMessageSerializer(Type serviceInterfaceType,
IEnumerable<Type> requestBodyTypes)
{
return new ServiceRemotingRequestMessagePackMessageBodySerializer();
}
public IServiceRemotingResponseMessageBodySerializer CreateResponseMessageSerializer(Type serviceInterfaceType, IEnumerable<Type> responseBodyTypes)
{
return new ServiceRemotingResponseMessagePackMessageBodySerializer();
}
public IServiceRemotingMessageBodyFactory CreateMessageBodyFactory()
{
return new MessagePackMessageFactory();
}
}
public class ServiceRemotingRequestMessagePackMessageBodySerializer : IServiceRemotingRequestMessageBodySerializer
{
public OutgoingMessageBody Serialize(IServiceRemotingRequestMessageBody serviceRemotingRequestMessageBody)
{
if (serviceRemotingRequestMessageBody == null) return null;
if (!(serviceRemotingRequestMessageBody is MessagePackRemotingRequestMessageBody body))
{
return new OutgoingMessageBody(new[] { new ArraySegment<byte>(new byte[0]) });
}
var bytes = MessagePackSerializer.Serialize(body, ServiceFabricResolver.Instance);
return new OutgoingMessageBody(new[] { new ArraySegment<byte>(bytes) });
}
public IServiceRemotingRequestMessageBody Deserialize(IncomingMessageBody messageBody)
{
using (var stream = messageBody.GetReceivedBuffer())
{
if (stream.Length == 0)
{
return new MessagePackRemotingRequestMessageBody();
}
var body = MessagePackSerializer.Deserialize<MessagePackRemotingRequestMessageBody>(stream, ServiceFabricResolver.Instance);
return body;
}
}
}

Related

How to change api return result in asp.net core 2.2?

My requirement is when the return type of the action is void or Task, I'd like to return my custom ApiResult instead. I tried the middleware mechanism, but the response I observed has null for both ContentLength and ContentType, while what I want is a json representation of an empty instance of ApiResult.
Where should I make this conversion then?
There are multiple filter in .net core, and you could try Result filters.
For void or Task, it will return EmptyResult in OnResultExecutionAsync.
Try to implement your own ResultFilter like
public class ResponseFilter : IAsyncResultFilter
{
public async Task OnResultExecutionAsync(ResultExecutingContext context, ResultExecutionDelegate next)
{
// do something before the action executes
if (context.Result is EmptyResult)
{
context.Result = new JsonResult(new ApiResult());
}
var resultContext = await next();
// do something after the action executes; resultContext.Result will be set
}
}
public class ApiResult
{
public int Code { get; set; }
public object Result { get; set; }
}
And register it in Startup.cs
services.AddScoped<ResponseFilter>();
services.AddMvc(c =>
{
c.Filters.Add(typeof(ResponseFilter));
}).SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
All you have to do is to check the return type and on the basis of the return you can perform whatever operations you want.
Here is the abstract demo:
You have a method:
public Action SomeActionMethod()
{
var obj = new object();
return (Action)obj;
}
Now in your code you can use the following code to get the name of the method:
MethodBase b = p.GetType().GetMethods().FirstOrDefault();
var methodName = ((b as MethodInfo).ReturnType.Name);
Where p in the above code is the class which contains the methods whose return type you want to know.
after having the methodname you can decide on variable methodName what to return.
Hope it helps.

Web API will not use ISerializable implementation

I thought I had jumped through the necessary hoops to get my JsonMediaTypeFormatter working with custom ISerializable implementations, complete with passing unit tests. But I'm unable to get it to work when I pass in values via Swagger UI.
My key questions are:
What am I doing wrong with my unit test causing it to serialize/deserialize different from what Web API is doing?
What do I need to change to get this working with Web API's serializing/deserialization and Swagger/Swashbuckle?
Class being serialized: (Notice that serializing and then deserializing drops off the time component and only keeps the date component. The helps for testing/observing purposes.)
public class Pet : ISerializable
{
public DateTime Dob { get; set; }
public Pet()
{
Dob = DateTime.Parse("1500-12-25 07:59:59");
}
public Pet(SerializationInfo info, StreamingContext context)
{
Dob = DateTime.Parse(info.GetString("Dob"));
}
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("Dob", Dob.Date.ToString());
}
}
Web API Method: (always returns null)
public class TestController : ApiController
{
[Route("~/api/Pet")]
public string Get([FromUri] Pet data)
{
return data.Dob.ToString();
}
}
Passing Unit Test: (and serialization helpers from MSDN docs)
[TestFixture]
public class SerializationTests
{
[Test]
public void PetTest()
{
var date = new DateTime(2017, 1, 20, 5, 0, 0);
var foo = new Pet { Dob = date };
var jsonFormatter = new JsonMediaTypeFormatter { SerializerSettings = new JsonSerializerSettings { ContractResolver = new DefaultContractResolver { IgnoreSerializableInterface = false } } };
var serialized = SerializationHelpers.Serialize(jsonFormatter, foo);
Console.WriteLine(serialized);
var deserialized = SerializationHelpers.Deserialize<Pet>(jsonFormatter, serialized);
Assert.That(foo.Dob, Is.Not.EqualTo(date.Date));
Assert.That(deserialized.Dob, Is.EqualTo(date.Date));
}
}
public static class SerializationHelpers
{
public static string Serialize<T>(MediaTypeFormatter formatter, T value)
{
// Create a dummy HTTP Content.
Stream stream = new MemoryStream();
var content = new StreamContent(stream);
// Serialize the object.
formatter.WriteToStreamAsync(typeof(T), value, stream, content, null).Wait();
// Read the serialized string.
stream.Position = 0;
return content.ReadAsStringAsync().Result;
}
public static T Deserialize<T>(MediaTypeFormatter formatter, string str) where T : class
{
// Write the serialized string to a memory stream.
Stream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(str);
writer.Flush();
stream.Position = 0;
// Deserialize to an object of type T
return formatter.ReadFromStreamAsync(typeof(T), stream, null, null).Result as T;
}
}
WebApiConfig.cs
public static class WebApiConfig
{
public static void Register(HttpConfiguration config)
{
// Web API configuration and services
config.Formatters.Clear();
var jsonFormatter = new JsonMediaTypeFormatter { SerializerSettings = new JsonSerializerSettings { ContractResolver = new DefaultContractResolver { IgnoreSerializableInterface = false } } };
config.Formatters.Add(jsonFormatter);
// Web API routes
config.MapHttpAttributeRoutes();
config.Routes.MapHttpRoute(
name: "DefaultApi",
routeTemplate: "api/{controller}/{id}",
defaults: new { id = RouteParameter.Optional }
);
}
}
A few other notes:
When I run the passing unit test, the Console.WriteLine output is:
{"Dob":"1/20/2017 12:00:00 AM"}
which is exactly what I want/expect.
My Swagger UI looks like this using the default Swashbuckle settings from Nuget. Note that value of the date is what is set in the default constructor, showing that my ISerializable implementation is ignored.
NOTE:
I have changed the question to remove all generics from the picture. This problem is fundamentally about ISerializable implementations now and not about Generics.
WebAPI api does not know how to deserialize this generic object. I see a similar question here in SO but did not personally try/test it. Hope it helps: Generic Web Api method
Rather than having a generic method, you can create a generic controller. So your code above will look something like below.
public abstract class MyClass{ }
public class PersonDto: MyClass{}
public class TestController<T> : ApiController where T: MyClass
{
public string Get([FromUri] T data)
{
...
}
}

Why is Akka.Persistance not replaying my Journal entries

I am writing an implementation of the Akka.Persistence for Service Fabric, and I don't seem to be able to get the snapshotting working. When it attempts to recover state it gets the latest snapshot but it does not replay the events since the latest snapshot. Its not clear to me If I have simply not wired up the components correctly or if my Implementation of the persistence library is incorrect.
My actor is a simple counter, my state is just the current count.
I expect that the Recover should get called first and then the Recover would get called for each journal entry between the last snapshot and the highest sequence number. There is a function ReplayMessagesAsync(...) in the journal that looks like it should do this but it does not get called.
The code for my counter is below, the rest of my code is: Code
using Akka.Actor;
using Akka.Persistence;
using Akka.Persistence.ServiceFabric.Journal;
using Akka.Persistence.ServiceFabric.Snapshot;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace AkkaPersistence.Actors
{
public class Counter : ReceivePersistentActor
{
public class GetCount { }
private int counter;
private CounterState State = new CounterState();
private int _msgsSinceLastSnapshot = 0;
public Counter()
{
Recover<Evt>(evt =>
{
State.Update(evt);
});
Recover<SnapshotOffer>(offer => {
var snapshotEntry = offer.Snapshot as SnapshotEntry;
if (snapshotEntry != null)
{
State = (CounterState)snapshotEntry.Snapshot;
}
});
Command<string>(str => Persist(str, s =>
{
++counter;
var evt = new Evt(s);
State.Update(evt);
if (++_msgsSinceLastSnapshot % 10 == 0)
{
//time to save a snapshot
SaveSnapshot(State.Copy());
}
}));
Command<GetCount>(get => Sender.Tell(State.Count));
Command<SaveSnapshotSuccess>(success =>
{
ServiceEventSource.Current.Message($"Saved snapshot");
DeleteMessages(success.Metadata.SequenceNr);
});
Command<SaveSnapshotFailure>(failure => {
// handle snapshot save failure...
ServiceEventSource.Current.Message($"Snapshot failure");
});
}
public override string PersistenceId
{
get
{
return "counter";
}
}
}
internal class CounterState
{
private long count = 0L;
public long Count
{
get { return count; }
set { count = value; }
}
public CounterState(long count)
{
this.Count = count;
}
public CounterState() : this(0)
{
}
public CounterState Copy()
{
return new CounterState(count);
}
public void Update(Evt evt)
{
++Count;
}
}
public class Evt
{
public Evt(string data)
{
Data = data;
}
public string Data { get; }
}
public class Cmd
{
public Cmd(string data)
{
Data = data;
}
public string Data { get; }
}
}
there were a couple of things I had wrong:
1) I needed to return what was passed in, not my SnapshotEntry which is an implementation detail of my persistence mechanism.
2) A simple miss as I translated from saving strings to attempting to save objects as part of the Journal.
3) Finally there was one more issue, that was the underlying issue, and that was, that the serialization was failing with child objects. In this piece of code I did not want to have to include the type of child object so instead I added a custom serializer (the Wire serializer) for the Journal as well as the already existing SnapshotSerializer and it is now working.

GWT with Serialization

This is my client side code to get the string "get-image-data" through RPC calls and getting byte[] from the server.
CommandMessage msg = new CommandMessage(itemId, "get-image-data");
cmain.ivClient.execute(msg, new AsyncCallback<ResponseMessage>() {
#Override
public void onFailure(Throwable caught) {
}
#Override
public void onSuccess(ResponseMessage result) {
if (result.result) {
result.data is byte[].
}
}
});
From the server side I got the length of the data is 241336.
But I could not get the value in onSuccess method. It is always goes to onFailure method.
And I got log on Apache:
com.google.gwt.user.client.rpc.SerializationException: Type '[B' was
not included in the set of types which can be serialized by this
SerializationPolicy or its Class object could not be loaded.
How can I do serialisation in GWT?
1) Create a pojo which implements Serializable interface
Let this pojo has all the data you want in the response of RPC service, in this case image-data
2) Pass this pojo in the response for your RPC service.
The below tutorial has enough information for creating RPC service
http://www.gwtproject.org/doc/latest/tutorial/RPC.html
The objects you transfer to and from the server has to implement IsSerializable.
All your custom Objects within the Object you are transferring also needs to implement IsSerializable.
Your objects cannot have final fields and needs an no argument constructor.
You need getters and setters.
A common serialize object in GWT:
public class MyClass implements IsSerializable {
private String txt;
private MyOtherClass myOtherClass; // Also implements IsSerializable
public MyClass() {
}
public String getTxt() {
return this.txt;
}
public void setTxt(String txt) {
return this.txt = txt;
}
public String getMyOtherClass() {
return this.myOtherClass;
}
public void setMyOtherClass(MyOtherClass myOtherClass) {
return this.myOtherClass = myOtherClass;
}
}

Duplex services / Singleton class with background thread

I'm wondering if anyone can help me. I have a wcf service running over TCP which will make use of a duplex service. currently this service calls a business object which in turn does some processing. While this processing is happening on a background thread I wish the UI to be updated at certain points. I've attached my code below. TestStatus should be broken up into six parts and the service should update the windows forms UI each time this changes.
The class Scenariocomponent is a singleton (following).
public void BeginProcessingPendingTestCases()
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessPendingTestCases));
}
public void BeginProcessingPendingTestCases()
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessPendingTestCases));
}
private void ProcessPendingTestCases(object state)
{
while (this.IsProcessingScenarios)
{
ProcessNextPendingTestCase();
}
}
private void ProcessNextPendingTestCase()
{
while (this.ServiceStatus == Components.ServiceStatus.Paused)
{
//Wait.
}
var testOperation = this.PendingTestCases.Dequeue();
if (testOperation.OperationStatus == TestStatus.Pending)
{
throw new NotImplementedException(); //TODO : Handle test.
if (testOperation.OperationStatus != TestStatus.Failed)
{
testOperation.OperationStatus = TestStatus.Processed;
}
this.CompletedTestCases.Enqueue(testOperation);
}
}
Initially I was using MSMQ to update the UI as it worked sufficiently however this is no longer acceptable due to client restrictions.
My Service is as follows:
public class TestHarnessService : ITestHarnessService
{
public bool Ping()
{
return true;
}
public bool IsProcessingScenarios()
{
return ScenarioComponent.Instance.IsProcessingScenarios;
}
public void BeginProcessingScenarios(string xmlDocument, Uri webServiceUri)
{
var doc = new XmlDocument();
doc.LoadXml(xmlDocument);
var scenarios = ScenarioComponent.Deserialize(doc);
ScenarioComponent.Instance.EnqueueScenarioCollection(scenarios, webServiceUri);
ScenarioComponent.Instance.BeginProcessingPendingTestCases();
}
public void ValidateScenarioDocument(string xmlDocument)
{
var doc = new XmlDocument();
doc.LoadXml(xmlDocument);
ScenarioComponent.ValidateScenarioSchema(doc);
}
ITestOperationCallBack Callback
{
get
{
return OperationContext.Current.GetCallbackChannel<ITestOperationCallBack>();
}
}
Now I need the UI to update each time a testoperation changes or completes but I am unsure how to accomplish this. Any feedback would be greatly appreciated.
Thank you!
Instead of using WinForms, you could use WPF and binding, which would handle the updating for you.