To make an OO system as decoupled as possible, I'm thinking of the following approach:
1) we run an RMI/directory like service where objects can register and discover each other. They talk to this service through an interface
2) we run a messaging service to which objects can publish messages, and register subscription callbacks. Again, this happens through interfaces
3) when object A wants to invoke a method on object B, it discovers the target object's unique identity through #1 above, and publishes a message on the message service for object B
4) message services invokes B's callback to give it the message
5) B processes the request and sends the response for A on message service
6) A's callback is called and it gets the response.
I feel this system is as decoupled as practically possible, but it has the following problems:
1) communication is typically asynchronous
2) hence it's non real time
3) the system as a whole is less efficient.
Are there any other practical problems where this design obviously won't be applicable ? What are your thoughts on this design in general ?
Books
Enterprise Integration Patterns
It appears he's talking about using a Message Oriented Middleware
Here are some things to consider
Security
What will prevent another rogue service from registering as a key component in your system. You will need way to validate and verify that services are who they say they are. This can be done through a PKI system. There are scenarios that you might not need to do this, if your system is hosted entirely on your intranet. IF that is the case Social Engineering and Rogue Employees will be your biggest threat.
Contract
What kind of contract will your clients have with the services? Will messages all be serialized as XML and sent as a TextMessage? If you use a pure byte message you'll have to be careful about byte order if your services are to run on multiple platforms.
Synchronization
Most developers are not able to comprehend and utilize asynchronous messages correctly. Where possible it might be in the best interest of your design to create a way to invoke "synchronous" messages. I've done this in the past by creating a sendMessageAndWait() method with a timeout and a return object. Within the method you can create a temporary topic id to receive the response, register a listener for it, then use locks to wait for a message to be returned on your temporary topic.
Unsolicited Messages
What happens if you want to allow your service(s) to send unsolicited messages to your clients? A critical event happened in Service A and it must notify your clients or possibly a Watch Dog service. Allow for your design to register for a common communication channel for services to communicate with clients without clients initiating the conversation.
Failover
What happens if a critical service processing your credit cards goes down? You'll need to implement a Failover and Watch Dog service to ensure that your key infrastructure is always up and running. You could register a list of services within your registry then your register could give out the primary service, falling back to a secondary service if your primary stops communicating. Or if your Message Oriented Middleware can handle Round Robin messaging you might be able to register all the services on the same topic. Think about creating a way to know when a service has died. Since most messages are Asynchronous it will be difficult to determine when a service has gone offline. This can be done with a Heartbeat and Watch Dog.
I've created this type of system a few times in my past for large systems that needed to communicate. If you and other developers understand the pros and cons of such a system it can be very powerful and flexible.
The biggest piece of advice I can give is to build a toolkit for your other developers so they don't have to think about how to register a service, or implement failover, or respond to messages from a client. These are the sorts of things that will kill your system and have others say it is too complicated. Making it painless for them will allow your system to work the way you need it with flexibility and decoupling while not burdening your developers with understanding enterprise design patterns.
This is not a Ivory Tower Architect/Architecture. It would be if he said, "This is how it will do done, now go do it and don't bother me about it because I know I'm right." If you really wanted to reference a Anti-Pattern it could be Kitchen Sink, maybe. Nah now that I think about it, it isn't Kitchen Sink either.
If you can find one please post it as a comment.
Anti-Patterns
Coupling is simply a balance between efficiency and re-usability. If you wish the modules of your system to be as reusable as possible then that will undoubtedly come at a cost.
Personally I think it best to define some key assumptions which may tighten coupling, but bring increased efficiency.
There are design patterns which never see the light of day just because the benefit they provide is not worth the cost in complexity.
What's the simplest thing that could possibly work? Do modularize into reasonable size routines, but avoid interfaces, services, messages and all of this unless you are going to have multiple implementations or multiple hardware resources to divide a job.
Make it simple, then refactor those parts that turned out to matter.
Related
I want to create a "Notifications Microservice" that will handle different type of notifications (Google Chat, Email, etc).
For this task, we will create a microservice that contains the logic on how to process these messages, and we'll be using Rabbit MQ to manage the queue.
Now, the question that I have, is if it is possible (or if it is a bad practice) to expose two endpoints in the microservice like this:
registerNotification('channel', $data)
processNotification(Rabbit Message)
So I only have to implement the communication with RabbitMQ in one service, and other services will just register messages using this same service instead of directly talking to RabbitMQ.
This way for each channel I could validate in the service that I have everything that I need before enqueuing a message.
Is this a good approach?
I'd suggest splitting your question into two separate ones. As usual, it depends ... there's pros and cons to either one. Below my points without claiming completeness. Assessing those really depends on your specific needs in the end.
1) Is it a good practice to use a Notification / Event Gateway in front of a message queue (here RabbitMQ)?
Pros:
enforce strong guarantees on message structure / correctness
provide advanced authentication / authorization mechanisms if required
provide convenience if languages in your stack lack first-class client support
abstract away / encapsulate technology choices & deployment considerations from services (publishers)
eliminate routing logic for messages from individual services (though, using available routing topologies in RabbitMQ, it's hard to see any added value here)
Cons:
availability becomes a critical concern for your gateway, e.g. assuming you can guarantee an uptime of four nines per service, you are already down to three nines for the composed system by adding this dependency
added operational complexity
added latency
An alternative consideration here might be to use a library to achieve some of the pros above. Though, this approach also comes with its own cons.
2) Is it a good practice to run both message publishers and consumers in one service?
Pros:
quick (shortcut?)
initially less deployed instances (until you have to scale up)
Cons:
operational requirements for producers and consumers (workers!) are typically very different
harder (and more expensive) to scale the system adequately and fine grained
(performance) metrics become difficult to interpret
consumers might impact producer latency negatively as everything is competing for the same resources
loss of flexibility on the consumer side (quick, low risk deployments)
harder to guarantee availability of producers
I hope this helps to better evaluate your architecture based on your own needs / priorities.
I'm trying to understand how to persist data in a Twisted application. Let's say I've decided to write a Twisted server that:
Accepts inbound SMTP requests
Sends the message to a 3rd party system for modification
Relays the modified message to its destination
A typical Twisted tutorial would have you build this app using Deferreds and callbacks, roughly:
A Factory handles inbound requests
Each time a full email is received a call is sent to the remote message processor, returning a deferred
Add an errback that substitutes the original message if anything goes wrong in the modify call.
Add a callback to send the message on to the recipient, which again returns a deferred.
A real server would add/include additional call/errbacks to retry or notify the sender or whatnot. Again for simplicity, assume we consider this an acceptable amount of effort and just log errors.
Of course, this persists NO data in the event of a crash/restart/something else. I get that a solution involves a 3rd party persistent datastore (RabbitMQ is often mentioned) and could probably come up with a dozen random ways to achieve the outcome.
However, I imagine there are a few approaches that work best in a Twisted app. What do they look like? How do they store (and restore in the event of a crash) the in-process messages?
If you found this question, you probably already know that Twisted is event-based. It sounds simple, but the "hardest" part of the answer is to get the persistence platform generating the events we need when we need them. Naturally, you can persist the data in a DB or a message queue, but some platforms don't naturally generate events. For example:
ZeroMQ has (or at least had) no callback for new data. It's also relatively poor at persistence.
In other cases, events are easy but reliability is a problem:
pgSQL can be configured to generate events using triggers, but they're one-time things so you can't resume incomplete events
The light at the end of the tunnel seems to be something like RabbitMQ.
RabbitMQ can persist the message in a database to survive a crash
We can use acknowledgements on both legs (incoming SMTP to RabbitMQ and RabbitMQ to outgoing SMTP) to ensure the application is reliable. Importantly, RabbitMQ supports acknowledgements.
Finally, several of the RabbitMQ clients provide full asynchronous support (see for example pika, txampq, and puka)
It's enough for our purposes that the RabbitMQ client provides us an event-based interface.
At a more theoretical level, however, this need not be the case. In fact, despite the "notification" issue above, ZeroMQ has an event-based client. Even if our software is elegantly event-based, we will run into systems that aren't. In these cases, we have no choice but to fall back on polling. In principle, if not in practice, we just query the message provider for messages. When we exhaust the current queue (and immediately if there are no messages), we use a callLater command to check again in the future. It may feel anti-pattern, but it's (to the best of my knowledge anyway) the right way to handle this particular case.
On my team at work, we use the IBM MQ technology a lot for cross-application communication. I've seen lately on Hacker News and other places about other MQ technologies like RabbitMQ. I have a basic understanding of what it is (a commonly checked area to put and get messages), but what I want to know what exactly is it good at? How will I know where I want to use it and when? Why not just stick with more rudimentary forms of interprocess messaging?
All the explanations so far are accurate and to the point - but might be missing something: one of the main benefits of message queueing: resilience.
Imagine this: you need to communicate with two or three other systems. A common approach these days will be web services which is fine if you need an answers right away.
However: web services can be down and not available - what do you do then? Putting your message into a message queue (which has a component on your machine/server, too) typically will work in this scenario - your message just doesn't get delivered and thus processed right now - but it will later on, when the other side of the service comes back online.
So in many cases, using message queues to connect disparate systems is a more reliable, more robust way of sending messages back and forth. It doesn't work well for everything (if you want to know the current stock price for MSFT, putting that request into a queue might not be the best of ideas) - but in lots of cases, like putting an order into your supplier's message queue, it works really well and can help ease some of the reliability issues with other technologies.
MQ stands for messaging queue.
It's an abstraction layer that allows multiple processes (likely on different machines) to communicate via various models (e.g., point-to-point, publish subscribe, etc.). Depending on the implementation, it can be configured for things like guaranteed reliability, error reporting, security, discovery, performance, etc.
You can do all this manually with sockets, but it's very difficult.
For example: Suppose you want to processes to communicate, but one of them can die in the middle and later get reconnected. How would you ensure that interim messages were not lost? MQ solutions can do that for you.
Message queueuing systems are supposed to give you several bonuses. Among most important ones are monitoring and transactional behavior.
Transactional design is important if you want to be immune to failures, such as power failure. Imagine that you want to notify a bank system of ATM money withdrawal, and it has to be done exactly once per request, no matter what servers failed temporarily in the middle. MQ systems would allow you to coordinate transactions across multiple database, MQ and other systems.
Needless to say, such systems are very slow compared to named pipes, TCP or other non-transactional tools. If high performance is required, you would not allow your messages to be written thru disk. Instead, it will complicate your design - to achieve exotic reliable AND fast communication, which pushes the designer into really non-trivial tricks.
MQ systems normally allow users to watch the queue contents, write plugins, clear queus, etc.
MQ simply stands for Message Queue.
You would use one when you need to reliably send a inter-process/cross-platform/cross-application message that isn't time dependent.
The Message Queue receives the message, places it in the proper queue, and waits for the application to retrieve the message when ready.
reference: web services can be down and not available - what do you do then?
As an extension to that; what if your local network and your local pc is down as well?? While you wait for the system to recover the dependent deployed systems elsewhere waiting for that data needs to see an alternative data stream.
Otherwise, that might not be good enough 'real time' response for today's and very soon in the future Internet of Things (IOT) requirements.
if you want true parallel, non volatile storage of various FIFO streams(at least at some point along the signal chain) use an FPGA and FRAM memory. FRAM runs at clock speed and FPGA devices can be reprogrammed on the fly adding and taking away however many independent parallel data streams are needed(within established constraints of course).
I have found myself responsible for carrying on the development of a system which I did not originally design and can't ask the original designers why certain design decisions were taken, as they are no longer here. I am a junior developer on design issues so didn't really know what to ask when I started on the project which was my first SOA / WCF project.
The system has 7 WCF services, will grow to 9, each self-hosted in a seperate console app/windows service. All of them are single instance and single threaded. All services have the same OperationContract: they expose a Register() and Send() method. When client services want to connect to another service, they first call Register(), then if successful they do all the rest of their communication with Send(). We have a DataContract that has an enum MessageType and a Content propety which can contain other DataContract "payloads." What the service does with the message is determined by the enum MessageType...everything comes through the Send() method and then gets routed to a switch statement...I suspect this is unusual
Register() and Send() are actually OneWay and Async...ALL results from services are returned to client services by a WCF CallbackContract. I believe that the reson for using CallbackContracts is to facilitate the Publish-Subscribe model we are using. The problem is not all of our communication fits publish-subscribe and using CallbackContracts means we have to include source details in returned result messages so clients can work out what the returned results were originally for...again clients have a switch statements to work out what to do with messages arriving from services based on the MessageType (and other embedded details).
In terms of topology: the services form "nodes" in a graph. Each service has hardcoded a list of other services it must connect to when it starts, and wont allow client services to "Register" with it until is has made all of the connections it needs. As an example, we have a LoggingService and a DataAccessService. The DataAccessSevice is a client of the LoggingService and so the DataAccess service will attempt to Register with the LoggingService when it starts. Until it can successfully Register the DataAccess service will not allow any clients to Register with it. The result is that when the system is fired up as a whole the services start up in a cascadeing manner. I don't see this as an issue, but is this unusual?
To make matters more complex, one of the systems requirements is that services or "nodes" do not need to be directly registered with one another in order to send messages to one another, but can communicate via indirect links. For example, say we have 3 services A, B and C connected in a chain, A can send a message to C via B...using 2 hops.
I was actually tasked with this and wrote the routing system, it was fun, but the lead left before I could ask why it was really needed. As far as I can see, there is no reason why services cannot just connect direct to the other services they need. Whats more I had to write a reliability system on top of everything as the requirement was to have reliable messaging across nodes in the system, wheras with simple point-to-point links WCF reliabily does the job.
Prior to this project I had only worked on winforms desktop apps for 3 years, do didn't know any better. My suspicions are things are overcomplicated with this project: I guess to summarise, my questions are:
1) Is this idea of a graph topology with messages hopping over indirect links unusual? Why not just connect services directly to the services that they need to access (which in reality is what we do anyway...I dont think we have any messages hopping)?
2) Is exposing just 2 methods in the OperationContract and using the a MessageType enum to determine what the message is for/what to do with it unusual? Shouldnt a WCF service expose lots of methods with specific purposes instead and the client chooses what methods it wants to call?
3) Is doing all communication back to a client via CallbackContracts unusual. Surely sync or asyc request-response is simpler.
4) Is the idea of a service not allowing client services to connect to it (Register) until it has connected to all of its services (to which it is a client) a sound design? I think this is the only design aspect I agree with, I mean the DataAccessService should not accept clients until it has a connection with the logging service.
I have so many WCF questions, more will come in later threads. Thanks in advance.
Well, the whole things seems a bit odd, agreed.
All of them are single instance and
single threaded.
That's definitely going to come back and cause massive performance headaches - guaranteed. I don't understand why anyone would want to write a singleton WCF service to begin with (except for a few edge cases, where it does make sense), and if you do have a singleton WCF service, to get any decent performance, it must be multi-threaded (which is tricky programming, and is why I almost always advise against it).
All services have the same
OperationContract: they expose a
Register() and Send() method.
That's rather odd, too. So anyone calling will first .Register(), and then call .Send() with different parameters several times?? Funny design, really.... The SOA assumption is that you design your services to be the model of a set of functionality you want to expose to the outside world, e.g. your CustomerService might have methods like GetCustomerByID, GetAllCustomersByCountry, etc. methods - depending on what you need.
Having just a single Send() method with parameters which define what is being done seems a bit.... unusual and not very intuitive / clear.
Is this idea of a graph topology with
messages hopping over indirect links
unusual?
Not necessarily. It can make sense to expose just a single interface to the outside world, and then use some internal backend services to do the actual work. .NET 4 will actually introduce a RoutingService in WCF which makes these kind of scenarios easier. I don't think this is a big no-no.
Is doing all communication back to a
client via CallbackContracts unusual.
Yes, unusual, fragile, messy - if you can ever do without it - go for it. If you have mostly simple calls, like GetCustomerByID - make those a standard Request/Response call - the client requests something (by supplying a Customer ID) and gets back a Customer object as a return value. Much much simpler!
If you do have long-running service calls, that might take minutes or more to complete - then you might consider One-Way calls which just deposit a request into a queue, and that request gets handled later on. Typically, here, you can either deposit the answer into a response queue which the client then checks, or you can have two additional service methods which give you the status of a request (is it done yet?) and a second method to retrieve the result(s) of that request.
Hope that helps to get you started !
All services have the same OperationContract: they expose a Register() and Send() method.
Your design seems unusual at some parts specially exposing only two operations. I haven't worked with WCF, we use Java. But based on my understanding the whole purpose of Web Services is to expose Operations that your partners can utilise.
Having only two Operations looks like odd design to me. You generally expose your API using WSDL. In this case the WSDL would add nothing of value to the partners, unless you have lot of documentation. Generally the operation name should be self-explanatory. Right now your system cannot be used by partners without having internal knowledge.
Is doing all communication back to a client via CallbackContracts unusual. Surely sync or asyc request-response is simpler.
Agree with you. Async should only be used for long running processes. Async adds the overhead of correlation.
We can use polling to find out about updates from some source, for example, clients connected to a webserver. WCF provides a nifty feature in the way of Duplex contracts, in which, I can maintain a connection to a client, and make invocations on that connection at will.
Some peeps in the office were discussing the merits of both solutions, and I wanted to get feedback on when each strategy is best used.
I would use an event-based mechanism instead of polling. In WCF, you can do this easily by following the Publish-Subscribe framework that Juval Lowy provides at his website, IDesign.net.
Depends partly on how many users you have.
Say you have 1,000,000 users you will have problems maintaining that many sessions.
But if your system can respond to 1000 poll requests a second then each client can poll every 1000 seconds.
I think Shiraz nailed this one, but I wanted to say two more things.
I've had trouble with Duplex
contracts. You have to have all of
your ducks in a row with regards to
the callback channel... you have to
check it to make sure it's open,
etc. The IDesign.net stuff would be
a minimum amount of plumbing code
you'll have to include.
If it makes sense for your solution
(this is only appropriate in certain
situations), the MSMQ binding allows
a client to send data to a service
in an async manner (like Duplex),
but the service isn't "polling" for
messages... it gets notified when
one enters the queue through some
under-the-covers plumbing.
This sort of forces you to turn the
communication around (client becomes
server, server becomes client), but
if the majority of the communication
is one-way, this would provide a lot
of benefits. The other advantage
here is obviously the queued
communication - the server can be
down and not miss any messages...
it'll pick 'em up when it comes back
online.
Something to think about.