Rebus with RabbitMQ accept requests from Python - rabbitmq

I am setting up a .NET core service that is reading from RabbitMQ using Rebus. It seems that the request placed in RabbitMQ needs to have the .NET object namespace information. Is there a way to work around this. For example if I had a service written in Python placing items on the queue would it be possible to read and process these requests. It seems every time I test and try to send something besides the .NET object I get an exception.
System.Collections.Generic.KeyNotFoundException: Could not find the key 'rbs2-content-type' - have the following keys only: 'rbs2-msg-id'

It depends on which serializer, you're using in the receiving end.
By default, Rebus will use its built-in JSON serializer with a fairly "helpful" setting, meaning that all .NET types names are included. This enables serialization of complex objects, including abstract/interface references, etc.
This serializer requires a few special headers to be present, though, e.g. the rbs2-content-type header, which it uses to verify that the incoming message presents itself as JSON (most likely by having application/json; charset=utf-8 as its content type).
If you want to enable deserialization of messages from other platforms, I suggest you provide the necessary headers on the messages (which – at least with Rebus' built-in serializer – also includes a .NET type name of the type to try to deserialize into).
Another option is to install a custom serializer, which is a fairly easy thing to do – you can get started by registering your serializer like this:
Configure.With(...)
.(...)
.Serialization(s => s.Register(c => new YourCrazySerializer()))
.Start();
which you then implement somewhat like this:
public class YourCrazySerializer : ISerializer
{
public async Task<TransportMessage> Serialize(Message message)
{
var headers = message.Headers.Clone();
// turn into byte[] here
//
// possibly add headers
return new TransportMessage(headers, bytes);
}
public async Task<Message> Deserialize(TransportMessage transportMessage)
{
var headers = transportMessage.Headers.Clone();
// turn into object here
//
// possibly remove headers
return new Message(headers);
}
}
As you can see, it's pretty easy to modify Rebus to accept messages from other systems.

Related

ServiceStack: Reinstate pipeline when invoking a Service manually?

As a follow-up to this question, I wanted to understand how my invoking of a Service manually can be improved. This became longer than I wanted, but I feel the background info is needed.
When doing a pub/sub (broadcast), the normal sequence and flow in the Messaging API isn't used, and I instead get a callback when a pub/sub message is received, using IRedisClient, IRedisSubscription:
_subscription.OnMessage = (channel, msg) =>
{
onMessageReceived(ParseJsonMsgToPoco(msg));
};
The Action onMessageReceived will then, in turn, invoke a normal .NET/C# Event, like so:
protected override void OnMessageReceived(MyRequest request)
{
OnMyEvent?.Invoke(this, new RequestEventArgs(request));
}
This works, I get my request and all that, however, I would like it to be streamlined into the other flow, the flow in the Messaging API, meaning, the request finds its way into a Service class implementation, and that all normal boilerplate and dependency injection takes place as it would have using Messaging API.
So, in my Event handler, I manually invoke the Service:
private void Instance_OnMyEvent(object sender, RequestEventArgs e)
{
using (var myRequestService = HostContext.ResolveService<MyRequestService>(new BasicRequest()))
{
myRequestService.Any(e.Request);
}
}
and the MyRequestService is indeed found and Any called, and dependency injection works for the Service.
Question 1:
Methods such as OnBeforeExecute, OnAfterExecute etc, are not called, unless I manually call them, like: myRequestService.OnBeforeExecute(e) etc. What parts of the pipeline is lost? Can it be reinstated in some easy way, so I don't have to call each of them, in order, manually?
Question 2:
I think I am messing up the DI system when I do this:
using (var myRequestService = HostContext.ResolveService<MyRequestService>(new BasicRequest()))
{
myRequestService.OnBeforeExecute(e.Request);
myRequestService.Any(e.Request);
myRequestService.OnAfterExecute(e.Request);
}
The effect I see is that the injected dependencies that I have registered with container.AddScoped, isn't scoped, but seems static. I see this because I have a Guid inside the injected class, and that Guid is always the same in this case, when it should be different for each request.
container.AddScoped<IRedisCache, RedisCache>();
and the OnBeforeExecute (in a descendant to Service) is like:
public override void OnBeforeExecute(object requestDto)
{
base.OnBeforeExecute(requestDto);
IRedisCache cache = TryResolve<IRedisCache>();
cache?.SetGuid(Guid.NewGuid());
}
So, the IRedisCache Guid should be different each time, but it isn't. This however works fine when I use the Messaging API "from start to finish". It seems that if I call the TryResolve in the AppHostBase descendant, the AddScoped is ignored, and an instance is placed in the container, and then never removed.
What parts of the pipeline is lost?
None of the request pipeline is executed:
myRequestService.Any(e.Request);
Is physically only invoking the Any C# method of your MyRequestService class, it doesn't (nor cannot) do anything else.
The recommended way for invoking other Services during a Service Request is to use the Service Gateway.
But if you want to invoke a Service outside of a HTTP Request you can use the RPC Gateway for executing non-trusted services as it invokes the full Request Pipeline & converts HTTP Error responses into Typed Error Responses:
HostContext.AppHost.RpcGateway.ExecuteAsync()
For executing internal/trusted Services outside of a Service Request you can use HostContext.AppHost.ExecuteMessage as used by ServiceStack MQ which applies Message Request Request/Response Filters, Service Action Filters & Events.
I have registered with container.AddScoped
Do not use Request Scoped dependencies outside of a HTTP Request, use Singleton if the dependencies are ThreadSafe, otherwise register them as Transient. If you need to pass per-request storage pass them in IRequest.Items.

How to set up handlers in RedMQ from events raised in my domain

Just getting my head around message queues and Redis MQ, excellent framework.
I understand that you have to use .RegisterHandler(...) to determine which handler will process the type of message/event that is in the message queue.
So if I have EventA, EventB etc should I have one Service which handles each of those Events, like :
public class DomainService : Service {
public object Any(EventA eventA) {...}
public object Any(EventB eventA) {...}
}
So these should be only queue/redis list created?
Also, what If I want a chain of events to happen, so for example a message of type EventA also has a handler that sends an Email providing handlers earlier on the chain are successful?
ServiceStack has no distinction between services created for MQ's, REST, HTML or SOAP services, they're the same thing. i.e. they each accept a Request DTO and optionally return a Response DTO and the same service can handle calls from any endpoint or format, e.g HTML, REST, SOAP or MQ.
Refer to ServiceStack's Architecture diagram to see how MQ fits in.
Limitations
The only things you need to keep in mind are:
Like SOAP, MQ's only support 1 Verb so your methods need to be named Post or Any
Only Action Filters are executed (i.e. not Global or Attribute filters)
You get MqRequest and MqResponse stubs in place of IHttpRequest, IHttpResponse. You can still use .Items to pass data through the request pipeline but any HTTP actions like setting cookies or HTTP Headers are benign
Configuring a Redis MQ Host
The MQ Host itself is completely decoupled from the rest of the ServiceStack framework, who doesn't know the MQ exists until you pass the message into ServiceStack yourself, which is commonly done inside your registered handler, e.g:
var redisFactory = new PooledRedisClientManager("localhost:6379");
var mqHost = new RedisMqServer(redisFactory, retryCount:2);
mqHost.RegisterHandler<Hello>(m => {
return this.ServiceController.ExecuteMessage(m);
});
//shorter version:
//mqHost.RegisterHandler<Hello>(ServiceController.ExecuteMessage);
mqHost.Start(); //Starts listening for messages
In your RegisterHandler<T> you specify the type of Request you want it to listen for.
By default you can only Register a single handler for each message and in ServiceStack a Request is tied to a known Service implementation, in the case of MQ's it's looking for a method signature first matching: Post(Hello) and if that doesn't exist it looks for the fallback Any(Hello).
Can add multiple handlers per message yourself
If you want to invoke multiple handlers then you would just maintain your own List<Handler> and just go through and execute them all when a request comes in.
Calling different services
If you want to call a different service, just translate it to a different Request DTO and pass that to the ServiceController instead.
When a MQ Request is sent by anyone, e.g:
mqClient.Publish(new Hello { Name = "Client" });
Your handler is invoked with an instance of type IMessage where the Request DTO is contained in the Body property. At that point you can choose to discard the message, validate it or alter it.
MQ Requests are the same as any other Service requests
In most cases you would typically just forward the message on to the ServiceController to process, the implementation of which is:
public object ExecuteMessage<T>(IMessage<T> mqMsg)
{
return Execute(mqMsg.Body, new MqRequestContext(this.Resolver, mqMsg));
}
The implementation just extracts the Request DTO from the mqMsg.Body and processes that message as a normal service being passed a C# Request DTO from that point on, with a MqRequestContext that contains the MQ IHttpRequest, IHttpResponse stubs.

List<dynamic> on NServiceBus Message

I'm using NServiceBus to power the back-end of an application with a
JavaScript client. Therefore, we have lots of cases where our commands
really want dynamic lists (object arrays in JS, List in C#). e.g.:
JSON:
{
"listProperty" : [
{
"propertyA" : "value",
"propertyB" : "valueB"
},
{
"propertyQ" : "valueQ"
}
]
}
C#:
class TheCommand : IMessage {
public List<dynamic> ListProperty {get; set;}
}
Is there a way to do this with NServiceBus?
----- Experimentation Results -----
Some experimentation with the XML Serializer from NServiceBus shows that
it does not understand the dynamics:
<TheCommand>
<ListProperty>
<Object>
</Object>
</ListProperty>
</TheCommand>
They simply serialize (and deserialize) as Objects, with no properties.
(This a bit of a cross-post with the yahoo group as well.)
With our JS clients we use a set of REST style services that handle serialization and passing messages to NSB. We pass variable length arrays all the time.
In reference to dynamic structures all together, NSB requires a message to be formatted specifically to activate its handlers. It may be a good idea to implement your own serializer. You must implement NServiceBus.Serialization.IMessageSerializer. Beyond that you must also be able to tell NSB to use that serializer and you may have your own custom configuration that goes along with it. Take a look at the class ConfigureXmlSerializer.
I also just saw that in the 3.0 branch there is a JSON/BSON serializer implemented. I've never used it, but it may meet your needs.

Request/Response pattern in SOA implementation

In some enterprise-like project (.NET, WCF) i saw that all service contracts accept a single Request parameter and always return Response:
[DataContract]
public class CustomerRequest : RequestBase {
[DataMember]
public long Id { get; set; }
}
[DataContract]
public class CustomerResponse : ResponseBase {
[DataMember]
public CustomerInfo Customer { get; set; }
}
where RequestBase/ResponseBase contain common stuff like ErrorCode, Context, etc. Bodies of both service methods and proxies are wrapped in try/catch, so the only way to check for errors is looking at ResponseBase.ErrorCode (which is enumeration).
I want to know how this technique is called and why it's better compared to passing what's needed as method parameters and using standard WCF context passing/faults mechanisms?
The pattern you are talking about is based on Contract First development. It is, however not necessary that you use the Error block pattern in WCF, you can still throw faultexceptions back to the client, instead of using the Error Xml block. The Error block has been used for a very long time and therefore, a lot of people are accustom to its use. Also, other platform developers (java for example) are not as familiar with faultExceptions, even though it is an industry standard.
http://docs.oasis-open.org/wsrf/wsrf-ws_base_faults-1.2-spec-os.pdf
The Request / Response pattern is very valuable in SOA (Service Oriented Architecture), and I would recommend using it rather than creating methods that take in parameters and pass back a value or object. You will see the benefits when you start creating your messages. As stated previously, they evolved from Contract First Development, where one would create the messages first using XSDs and generate your classes based on the XSDs. This process was used in classic web services to ensure all of your datatypes would serialize properly in SOAP. With the advent of WCF, the datacontractserializer is more intelligent and knows how to serialize types that would previously not serialize properly(e.g., ArrayLists, List, and so on).
The benefits of Request-Response Pattern are:
You can inherit all of your request and responses from base objects where you can maintain consistency for common properties (error block for example).
Web Services should by nature require as little documentation as possible. This pattern allows just that. Take for instance a method like public BusScheduleResponse GetBusScheduleByDateRange(BusDateRangeRequest request); The client will know by default what to pass in and what they are getting back, as well, when they build the request, they can see what is required and what is optional. Say this request has properties like Carriers [Flag Enum] (Required), StartDate(Required), EndDate(Required), PriceRange (optional), MinSeatsAvailable(Option), etc... you get the point.
When the user received the response, it can contain a lot more data than just the usual return object. Error block, Tracking information, whatever, use your imagination.
In the BusScheduleResponse Example, This could return Multiple Arrays of bus schedule information for multiple Carriers.
Hope this helps.
One word of caution. Don't get confused and think I am talking about generating your own [MessageContract]s. Your Requests and Responses are DataContracts. I just want to make sure I am not confusing you. No one should create their own MessageContracts in WCF, unless they have a really good reason to do so.

Type 'System.Web.HttpRequest' cannot be serialized

I am trying to design an Picture Upload feature into a web site.
I am using ASP.NET 3.5, C#, and WCF.
I have been asked to accomplish the following:
1) Make the Uploader a Web Service
2) Return progress updates to the user as files are uploaded.
3) Log other relevant user-selected options in the database.
So, I have started off by creating a WCF web client with the
below service contract:
IService.UploadPictures(HttpRequest request);
private UploadServiceClient upload;
protected void Page_Load(object sender, EventArgs e)
{
upload = new UploadServiceClient();
upload.UploadPictures(Request.Files);
}
When I compile, I get the below error:
Type 'System.Web.HttpRequest' cannot
be serialized. Consider marking it
with the DataContractAttribute, and
marking all of its members you want
serialized with the
DataMemberAttribute attribute.
So, I went back into my service contract and
changed [OperationContract] to [DataContract]
but the change produced the same error.
Can somebody kindly tell me what I am doing wrong
and provide examples as to how to best move forward?
Thanks for your time.
You cannot use something like a HttpRequest as a WCF parameter. The error messages says it all - the HttpRequest is not serializable, and in order to work with WCF, types have to be serializable.
Also, you need to remember: you're not just passing an object instance to a method here - what you're really doing is having the WCF runtime serialize your request (the method name to call plus all the parameters passed in) into a message (think: e-mail or xml file), sending it to the server, deserialising there and building up a new copy of the given datatype (as defined in your DataContract), and doing something with it.
Your WCF service could well be self-hosted, e.g. running in a NT Service or console app - no HttpRequest available in those circumstances!
You need to definitely rearchitect your solution - you need to either check into WCF streaming to upload files to WCF (google for it - you'll find plenty of hits) or you'll need to find another way to pass the relevant info (e.g. list of filenames) to the WCF service without use of a HttpRequest object.
Marc
You are submitting a request as a parameter to a request. This is not what you want to do. I'm assuming that "Request.Files" is an array of files. This is what you want to upload. So something like:
IService.UploadPictures(List<SomeFileType> request);