Simplified... We are using NServiceBus for updating our storage.
In our sagas we first read data from our storage and updates the data and puts it back again to storage.The NServicebus instance is selfhosted in a windows service. Calls to storage are separated in its own assembly ('assembly1').
Now we will also need synchronous read from our storage through WCF. In some cases there will be the same reads that were needed when updating in sagas.
I have my opinion quite clear but maybe I am wrong and therefore I am asking this question...
Should we set up a separate WCF service that is using a copy of 'assembly1'?
Or, should the WCF instance host nservicebus?
Or, is there even a better way to do it?
It is in a way two endpoints, WCF for the synchronous calls and the windows service that hosts nservicebus (which already exists) right now.
I see no reason to separate into two distinct endpoints in your question or comments. It sounds like you are describing a single logical service, and my default position would be to host each logical service in a single process. This is usually the simplest approach, as it makes deployment and troubleshooting easier.
Edit
Not sure if this is helpful, but my current client runs NSB in an IIS-hosted WCF endpoint. So commands are handled via NSB messages, while queries are still exposed via WCF. To date we have had no problems hosting the two together in a single process.
Generally speaking, a saga should only update its own state (the Data property) and send messages to other endpoints. It should not update other state or make RPC calls (like to WCF).
Before giving more specific recommendations, it would be best to understand more about the specific responsibilities of your saga and the data being updated by 'assembly1'.
Related
I've read other questions on StackOverflow regarding using NSB on SF and also the sample on github (outdated) and I'm still not sure how to configure NServiceBus properly for this platform.
I'm looking to set up a send only publish/subscribe workflow. What I can't determine through my research is how to set this up so that only one instance of a particular service responds to the message.
For example: 3 services running on the standard 5 nodes (so pretend 5 instances of each of the 3 services).
Existing load balancer routes an http request to a specific instance of Service A.
Service A publishes the "OrderComplete" event
Services B and C both subscribe to the event.
How can I make sure that only one instance of Services B and C respond instead of all 5 instances of Service B and all 5 instances of Service C?
All the services are currently Stateless services.
I was thinking of using the AzureServiceBus or AzureStorageQueue transport.
Stateless approach is fine. You do not need to go into stateful services with a single partition unless you want to leverage reliable collections for your services. But let's look at both options
Going with Stateless services
It's ok to have multiple instances of your services. Yes, they all will create subscriptions. I'd argue that is exactly what you want - competing consumers. More service instances you have, more throughput you'll get, i.e. handling more messages.
What I can't determine through my research is how to set this up so that only one instance of a particular service responds to the message.
This will happen automatically due to the nature of the competing consumer transport (both ASB and ASQ).
Going with Stateful services
With stateful services you need to be very careful. Yes, you could go with a single partition per service, hence having a single primary replica handling your messages. But then, arguably, you're wasting your cluster resources by not utilizing them for concurrent processing of many messages. If you decide to partition your service, then you won't be able to use reliable collections as replicas of services do not share reliable collections among themselves. Should you choose to use partitioned stateful services w/o reliable collections, well, then you better to utilize stateless counterpart.
Note: NSB will provide support for running with stateful services to take advantage of the reliable collections for persistence needs, but even then partitioning is something that would need to be through through to align with business needs. If you do not have a need like that, I'd suggest to stick to stateless services and Azure Storage persistence.
In the NSB/SF sample on github there is a Stateful service that handles a command. What is important is that in the application it has an PartitionCount=1. The same goes for all other solutions with NSB I have seen, only one partition or instance of each service that handles messages. Otherwise you would end up with one subscription per instance for each message as you describe.
Perhaps you could adopt the Distributor to achieve load balancing between multiple instances of the same service, but afaik Distributor only works with MSMQ so you will have to rewrite it to work with SF and Azure Service Bus.
If you stick with single instances, it should work fine for you. You would still get some benefit from SF as it ensures your services are up and running, but load balancing between multiple instances will require some work for you.
does someone know if it possible to use one WCF Data Service as data source of another WCF Data Service? If so, how?
So the short answer is yes. Actually I have consumed one WCF service in another (HttpBinding coming to a service on computer, then that service had a NamedPipesBinding service to communicate with multiple desktop apps, but it did some data transformation in the middle). That would not be an issue at all, you would set up a proxy/client just like you would in a desktop client, and handle everything in your new service as if it was just passing information along, you could even create a shared library for the DataContracts and such.
HOWEVER I would not suggest the leapfrog method in your implementation. Depending on how many customers you are potentially opening the door too, you may be introducing a bottlekneck, if you have a singleton service, or overload your existing service in the case of many connections from the new one. Since you have a SQL server, why would you not have a WCF service on your web/app server (public) that connected to it and provided the data you need? I'm only thinking this because your situation can become exponentially complicated when you start trying to pass credentials for authentication and authorization between the two, depending on your security settings. Another thing to consider is the complexity in debugging this new service and the old one, and a client at the same time, as if it wasn't a pain just to do server and client, since you are opening it to a public facing port, there are different things to set up, and debugging everything on the same machine is not the same as a public facing application server.
Sorry if this goes against what you were hoping to hear. I'm just saying that it is possible, but not suggested (at least by me) in your particular case.
I'm thinking of adding a queue function in a product based on a bunch of WCF services. I've read some about MSMQ, first I thought that was what I needed but I'm not sure and are considering to just put the queue in a database table. I wonder if somone here got some feedback on which way to go.
Basicly I'm planning to have a facade WCF service called over http. The facade service should only write all incoming messages to a queue to give a fast response to the calling system. The messages in the queue should then be processed by another component, either a WCF service or a Windows service depending om my choice of queue.
The product is running in a load balanced enviroment with 2 to n web servers.
The options I'm considering and the questions I got are:
To let the facade WCF write to a MSMQ and then have anothther WCF service reading from this queue to do the processing of the messages. What I don't feel confident about for this alternative from what I've read is how this will work in a load balanced enviroment.
1A. Where should the MSMQ(s) be placed? One on each web server? One on a separate server? Mulitple on a separate server? (not considering need of redundance and that data in rare cases could be lost and re-sent)
1B. How it the design affected if I want the system redundant? I'd like to be alble to lose a server (it never comes up online again) holding the MSMQ without losing the data in that queue. From what I've read about MSMQ that leaves me to the only option of placing the MSMQ on a windows cluster. Is that correct? (I'd like to avoid using a windows cluster fo this).
The second design alternative is to let the facade WCF service write the queue to a database. Then have two or more Windows services to do the processing of the queue. I don't have any questions on this alternative. If you wonder why I don't pick this one as it seems simpler to me then it is because I'd like to build this not introducing any windows services to the solution, that I beleive the MSMQ got functionality I don't want to code myself and I'm also curious about using MSMQ as I've never used it before.
Best Regards
Håkan
OK, so you're not using WCF with MSMQ integration, you're using WCF to create MSMQ messages as an end-product. That simplifies things to "how do I load balance MSMQ?"
The arrangement you use is based on what works best for you.
You could have multiple webservers sending messages to a remote queue on a central machine.
Instead you could have a webservers putting messages in local queues with a central machine polling the queues for new arrivals.
You don't need to cluster MSMQ to make it resilient. You can instead make your code resilient so that it copes with lost messages using dead letter queues, transactional queues, journaling, and so on. Hardware clustering is the easy option :-)
Load-balancing MSMQ - a brief
discussion
Oil and water - MSMQ transactional
messages and load balancing
After reading some more on the subjet I haver decided to not use MSMQ. It seems like I really got no reason to go down this road. I need this to be non-transactional and as I understand it none of the journaling or dead letter techniques will help me with my redundancy requirement.
All my components will be online most of the time (maybe a couple of hours per year when they got access problems).
The MSQM will only add complexity to the exciting solution, another technique and maybe another server to keep track of.
To get full redundance to prevent data loss in MSMQ I will need a windows cluster or implement send/recieve to multiple identical queues. I don't want to do either of those.
All this lead me to front my recieving application with a WCF facade accepting http calls writing to a database queue. This database is already protected from data loss. The queue will be polled by muliple active instances of a Windows Servce containing all the heavy business logic. With low process priority these services could be hosted on the already existing nodes used by the load balaced web application. If I got time to use MSMQ or if I needed it for another reason in my application I might change my decision.
I am to design a webservice using WCF that yields methods that don't require a session (like static calls, eg: giving back some information about the webservice itsself) and other methods that require a session.
furthermore, the session based methods are using Workflows that are supposed to be able to be changed at runtime.
my current design would look like this:
there is a singleton service that runs in IIS that handles all the per call methods which also works as a host for the session based services. that way the singleton class knows about all the sessions and can halt the running workflows to exchange them.
is this a good/possible design choice?
is this a common scenario that uses a common design?
would be happy about any reading hints as the msdn help wasn't such a help to me.
thanks for your answers
-redoced
Using a singleton WCF service class is almost never a good idea - unless you really have just one single (physical) resource which you want to protect from concurrent access, it doesn't really make sense.
Because: either it's not multi-threading capable, but in this case, it becomes a huge bottleneck - requests are handled strictly sequentially - one after another. Not very good for performance.
Or then you need to make the whole service multi-threading aware - and let me tell you, making this properly, safely, and efficiently isn't for the faint of heart. It's really really hard to get this right, and make it perform well.
I really don't see any need for this, at all.
leave you "static" message calls (as you call them) be per-call services - those are easy to program, work well, perform well, never cause any multithreading issues
those few service calls (hopefully!) that do require a session - put them on their own service endpoint(s), make them per-session, use the "SessionId" in your session to identify them. It's a bit more work than per-call - but still nowhere near as complicated and error-prone as multi-threaded programming
Resources for WCF sessions:
MSDN article on Using Sessions in WCF
WCF Sessions blog post
Resources for WCF durable services (that persist their state between calls):
Blog post on "durable services"
WCF Durable Services
Orcas durable services
Screencast by Mike Taulty on durable WCF Services
Resources for WCF Workflow Services:
Foundataions: Workflow Services
Channel9 webcast: Intro to Workflow Services
Webcast: Intro to workflow services in WCF 4.0
I have found myself responsible for carrying on the development of a system which I did not originally design and can't ask the original designers why certain design decisions were taken, as they are no longer here. I am a junior developer on design issues so didn't really know what to ask when I started on the project which was my first SOA / WCF project.
The system has 7 WCF services, will grow to 9, each self-hosted in a seperate console app/windows service. All of them are single instance and single threaded. All services have the same OperationContract: they expose a Register() and Send() method. When client services want to connect to another service, they first call Register(), then if successful they do all the rest of their communication with Send(). We have a DataContract that has an enum MessageType and a Content propety which can contain other DataContract "payloads." What the service does with the message is determined by the enum MessageType...everything comes through the Send() method and then gets routed to a switch statement...I suspect this is unusual
Register() and Send() are actually OneWay and Async...ALL results from services are returned to client services by a WCF CallbackContract. I believe that the reson for using CallbackContracts is to facilitate the Publish-Subscribe model we are using. The problem is not all of our communication fits publish-subscribe and using CallbackContracts means we have to include source details in returned result messages so clients can work out what the returned results were originally for...again clients have a switch statements to work out what to do with messages arriving from services based on the MessageType (and other embedded details).
In terms of topology: the services form "nodes" in a graph. Each service has hardcoded a list of other services it must connect to when it starts, and wont allow client services to "Register" with it until is has made all of the connections it needs. As an example, we have a LoggingService and a DataAccessService. The DataAccessSevice is a client of the LoggingService and so the DataAccess service will attempt to Register with the LoggingService when it starts. Until it can successfully Register the DataAccess service will not allow any clients to Register with it. The result is that when the system is fired up as a whole the services start up in a cascadeing manner. I don't see this as an issue, but is this unusual?
To make matters more complex, one of the systems requirements is that services or "nodes" do not need to be directly registered with one another in order to send messages to one another, but can communicate via indirect links. For example, say we have 3 services A, B and C connected in a chain, A can send a message to C via B...using 2 hops.
I was actually tasked with this and wrote the routing system, it was fun, but the lead left before I could ask why it was really needed. As far as I can see, there is no reason why services cannot just connect direct to the other services they need. Whats more I had to write a reliability system on top of everything as the requirement was to have reliable messaging across nodes in the system, wheras with simple point-to-point links WCF reliabily does the job.
Prior to this project I had only worked on winforms desktop apps for 3 years, do didn't know any better. My suspicions are things are overcomplicated with this project: I guess to summarise, my questions are:
1) Is this idea of a graph topology with messages hopping over indirect links unusual? Why not just connect services directly to the services that they need to access (which in reality is what we do anyway...I dont think we have any messages hopping)?
2) Is exposing just 2 methods in the OperationContract and using the a MessageType enum to determine what the message is for/what to do with it unusual? Shouldnt a WCF service expose lots of methods with specific purposes instead and the client chooses what methods it wants to call?
3) Is doing all communication back to a client via CallbackContracts unusual. Surely sync or asyc request-response is simpler.
4) Is the idea of a service not allowing client services to connect to it (Register) until it has connected to all of its services (to which it is a client) a sound design? I think this is the only design aspect I agree with, I mean the DataAccessService should not accept clients until it has a connection with the logging service.
I have so many WCF questions, more will come in later threads. Thanks in advance.
Well, the whole things seems a bit odd, agreed.
All of them are single instance and
single threaded.
That's definitely going to come back and cause massive performance headaches - guaranteed. I don't understand why anyone would want to write a singleton WCF service to begin with (except for a few edge cases, where it does make sense), and if you do have a singleton WCF service, to get any decent performance, it must be multi-threaded (which is tricky programming, and is why I almost always advise against it).
All services have the same
OperationContract: they expose a
Register() and Send() method.
That's rather odd, too. So anyone calling will first .Register(), and then call .Send() with different parameters several times?? Funny design, really.... The SOA assumption is that you design your services to be the model of a set of functionality you want to expose to the outside world, e.g. your CustomerService might have methods like GetCustomerByID, GetAllCustomersByCountry, etc. methods - depending on what you need.
Having just a single Send() method with parameters which define what is being done seems a bit.... unusual and not very intuitive / clear.
Is this idea of a graph topology with
messages hopping over indirect links
unusual?
Not necessarily. It can make sense to expose just a single interface to the outside world, and then use some internal backend services to do the actual work. .NET 4 will actually introduce a RoutingService in WCF which makes these kind of scenarios easier. I don't think this is a big no-no.
Is doing all communication back to a
client via CallbackContracts unusual.
Yes, unusual, fragile, messy - if you can ever do without it - go for it. If you have mostly simple calls, like GetCustomerByID - make those a standard Request/Response call - the client requests something (by supplying a Customer ID) and gets back a Customer object as a return value. Much much simpler!
If you do have long-running service calls, that might take minutes or more to complete - then you might consider One-Way calls which just deposit a request into a queue, and that request gets handled later on. Typically, here, you can either deposit the answer into a response queue which the client then checks, or you can have two additional service methods which give you the status of a request (is it done yet?) and a second method to retrieve the result(s) of that request.
Hope that helps to get you started !
All services have the same OperationContract: they expose a Register() and Send() method.
Your design seems unusual at some parts specially exposing only two operations. I haven't worked with WCF, we use Java. But based on my understanding the whole purpose of Web Services is to expose Operations that your partners can utilise.
Having only two Operations looks like odd design to me. You generally expose your API using WSDL. In this case the WSDL would add nothing of value to the partners, unless you have lot of documentation. Generally the operation name should be self-explanatory. Right now your system cannot be used by partners without having internal knowledge.
Is doing all communication back to a client via CallbackContracts unusual. Surely sync or asyc request-response is simpler.
Agree with you. Async should only be used for long running processes. Async adds the overhead of correlation.