I have a few servers that are within different DMZs. Each of these servers will be running a few NServiceBus services and/or a web application which sends messages. I'll need to be able to send messages across the different sites a lot of the time. Opening the required ports poses a security problem our network admin is not willing to compromise on.
I've had a look at the NServiceBus Gateway. ( http://docs.particular.net/nservicebus/gateway/ ) I do think that this is going to be a good solution. However my I'm not exactly sure on the best way to implement it.
Consider a network consisting of multiple services where multiple messages will need to be sent across the sites. Is it best to create an additional service on the same server which handles all messages coming from within that network or to let each service manage it's cross site sending?
Thanks
The nservicebus gateway is a viable solution. Although you must consider the following: the gateway like everything in NSB is also oppinionated. It only allows send semantics over it. You also need to carefully design your SOA service boundaries. Cross site communication should connect logical services which are physically seperate on mutiple sites. Communication going over the gateway should have significant business importance but not be very chatty (because cross site is very expensive operation).
The gateway does require you to open ports to at least allow incoming http communication.
Related
I am sure that this is answered dozens of times, but I am at a loss as to what keywords to search for and thus I hope that someone can at least tell me where I should be looking given an explanation of my scenario.
I need two services (one can be just a client if that is easier) to talk to each other, but the client will be on a private network whereas the server will be on the internet. I want to be able to push jobs to the client, but the server obviously does not have an IP to hit the client. I'd rather not poll from the client every X seconds. I have read various topics all circling this issue and so I am going to throw out a few terms that I think are relevant, but I am not sure which to use or exactly how.
Comet, SignalR, WebSockets, XSockets, Publisher/Subscriber Pattern...
I have looked at each of these and I am not sure which is the right way to go. The client can certainly "subscribe" to the server on startup, so that should not be an issue. But the client should be either a console app, windows service, or WCF service. It seems Comet and SignalR are more for ASP.NET apps, where the client is JS in a browser. I just need "server(client)" to server connection where the client is behind firewalls.
Which of these terms (or none of them) is a good way to handle server -> client push notifications?
Pub/Sub architecture pattern with something like Azure Service Bus should help you create the solution you desire. This does require that service and the client are aware of the bus. For the plumbing of the client and the services use the WCF which has built in bindings to facilitate the use of this pattern.
Azure: How to Use Service Bus Topics/Subscriptions.
Azure SB has a counter part that works on-premises as well. There are other popular message bus tech (NServiceBus, MasTransit, etc.)
You can have a look at node.js together with socket.io.
This will give everything you need.
socket.io uses web sockets, and if the browser does not support web sockets, it gracefully falls back to other communication mechanism like xhr, flash, polling, aso.
Background
My group are complete noobs with MassTransit and messaging in general. I understand the simple demos found online, but I'm confused on how to set things up for non-trivial scenarios. (many producers, many consumers, with consumers communicating back to producers)
We currently make 3rd party web service calls directly from web code via synchronous calls. Some of them are notoriously slow and unreliable to the point of browser timeouts and YSODs that aren't directly our code's fault. We want to replace these sync calls with messages and eventual consistency for retries and poison queue.
We also want to replace various scheduled/batch tasks with messaging to get closer to real time processing instead of waiting for next batch to run.
Our website runs on a farm of 6 IIS servers behind a hardware load balancer. There are 2 additional "application" servers that run the scheduled tasks. I figure we will put our new worker services on the app servers or maybe even all 8 servers.
Questions
So... The "common gotchas" section of the MT docs say that each application needs it's own address. My question is around what exactly is the definition of application in this case.
I have 6 web servers running the website. Does each of these need a unique address or can they all just be "rabbitmq://localhost/MyApp/Website". What if IIS is configured for multiple worker processes? Do each of those also need a different rabbit address?
Same question goes for my 2 application servers. If I'm running the same worker on both boxes does it need different addresses? Some stuff says if you want competing consumers to share an address, but if you want "event" type messages to be delivered to everyone they need to be different addresses.
What if you need both event (broadcast) and command (consumed once) messages sent to a worker cluster? (Multiple instances of the same workers to handle more load.)
What if I have consumers hosted in the web application directly? (I'm not sure this is a good idea to start with.)
What about request/response messages? I assume the responses should go back to the originating web server. Otherwise the MT request call will never unblock or at best timeout.
Each instance of an IServiceBus needs it's own RecieveFrom address. And yeah, if there are multiple worker processes, each should have it's own queue. You can use temporary queue for this though in web apps.
For competing consumers, each process/IServiceBus that is one of the consumes should be an exact copy. If there's an event that doesn't need to be competing, then it needs to have it's own process.
(Ha! see what I did there?)
I have a system whereby a server pushes information from a central DB out to many client DBs (cross-domain via internet), and periodically they call services on the server. This has to withstand intermittent connections, ie queue messages.
I've created a development version using duplex MSMQ that I'm trying to apply transport security. From the reading I've done, it appears that:
MSMQ uses AD Windows Security, which is irrelevant cross-domain.
Due to the nature of duplex, each client is effectively a server as well. That means I need to pay $1200 every time I install the system with another client if I want to use SSL.
Are these facts correct? Am I really the only person who needs to secure services that are queued AND cross-domain AND duplex?
"MSMQ uses AD Windows Security, which is irrelevant cross-domain."
No, MSMQ uses Windows security which includes local accounts and, if available, domain accounts. MSMQ also uses certificates, if available.
"Due to the nature of duplex, each client is effectively a server as
well."
MSMQ doesn't use a client-server model. All MSMQ machines are effectively peers, sending messages between each other. For the $1,200 payment, are you referring to the certificate needed by the web service for sending MSMQ over HTTPS?
This is the first time I've seen anyone want to push secure messages over HTTPS to multiple destinations.
You may, in fact, be the only person in the world right now who wants to do this.
Let me embellish.
Not many companies are using MSMQ (in the grand scheme of things).
Of those that are, the vast majority are using only private queues, a small minority only use public queues.
Of those that are, only a handful are using it across the internet.
Of those that are, perhaps one is using it to exchange messages in both directions (that would be yours).
But that aside, it seems to me your main challenge will be using MSMQ as a secure transport layer over the internet. Although I have never had to do this, here are a couple of articles:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms701477(v=vs.85).aspx
http://msdn.microsoft.com/en-us/magazine/cc164041.aspx
Sorry couldn't be of more help.
After developing mini project with WCF duplex (Chat Service | Sms Service), I got a Point that maybe not be correct!!
I believed Duplex theory is good and useful but there is a lot problem about using Wcf Duplex. (like reliable session, Time-out exceptions, Client address-Management on server side, proxy management on Client Side)
am I think wrong ? am I miss something?
For more Information I Used wsDualHttpBinding not tcpBinding.
If you need bidirectional communication and you want to use WCF, duplex channels are the way to go. You just need to design your application correctly and correctly handle all problems you have described. If you feel that these problems are overhead and make things even worse you can always use network programming directly (sockets) or handle bidirectional communication by yourselves exposing separate service on server and another on client (where first call from client inform server about clients address) - this scenario will suffer from the same communication problems as WsDualHttpBinding.
WsDualHttpBinding itself is special kind of duplex communication. I personally don't like it because people very often misuse it. The problem is that this binding uses two separate connections - one from client to server and second from server to client. That is big difference to net.tcp where only connection initiated from client to server is used. Obviously using WsDualHttpBinding over internet (= you don't have control over client machines) becomes much more complicated because each client must configure its firewall (in computer, on home internet gateway, etc.) to allow connection on some port. Also if you want to run more then one instance of application on the same client machine, each instance must use its own port.
Just a technology update, now that .NET 4.0 is out.
I write an application that communicates to the server through what is basically a message bus (instead of method calls). This is based on the internal architecture of the application (which is multi threaded, passing the messages around).
There are a limited number of messages to go from the client to the server, quite a lot more from the server to the client. Most of those can be handled via a separate specialized mechanism, but at the end we talk of possibly 10-100 small messages per second going from the server to the client.
The client is supposed to operate under "internet conditions". THis means possibly home end users behind standard NAT devices (i.e. typical DSL routers) - a firewalled secure and thus "open" network can not be assumed.
I want to have as little latency and as little overhad for the communication as possible.
What is the technologally best way to handle the message bus callback? I Have no problem regularly calling to the server for message delivery if something needs to be sent...
...but what are my options to handle the messagtes from the server to the client?
WsDualHttp does work how? Especially under a NAT scenario?
Just as a note: polling is most likely out - the main problem here is that I would have a significant overhead OR a significant delay, both aren ot really wanted. Technically I would love some sort of streaming appraoch, where the server can write messags to a stream while he generates them and they get sent to the client as they come. Not esure this is doable with WCF, though (if not, I may acutally decide to handle the whole message part outside of WCF and just do control / login / setup / destruction via WCF).
For bidirectional communications, your best bet is NetTcpBinding, rather than the http bindings, if they're available.
This has the advantage of only requiring that the client can initiate a connection with the server.
I would go with Windows Azure Service Bus. See my answer in the following question:
WCF, 4.0, Bidirectional
Take a look at Windows AppFabric, good place to start is Here. It fundamentally wraps up WCF and WF into an application server, with WCF activation supported through WAS. Its where I would host this type of app. It offerd full duplex connection orientated, p2p or sessions between client and server. Don't confuse the Windows appfabric with Azure appfabric, (formely called Azure Service Bus).
As regards bindings above, both NetTcpBinding and WsDualHttp offer callbacks, but the ws binding you get a lot for your cash, especially if it's a mixed programming environment and you have to flatten the wsdl to make interop work. I also think that WsDual is easier on routers traversal, although I understand talking to friends, that Windows AppFabric mitigates this, with new Relay Services, (which i've not seen, and I think have now been renamed).
Hope that helps.