When should I go for Staged Event Driven Architecture ? Do you see any issues in implementing SEDA in weblogic by having multiple Queues/MDBs as stages ? Any insights on disadvantages using SEDA ?
Do you have any specific concerns you want to address? WebLogic JMS is very high-performing so this should not be a problem.
As you have probably learned at this point, message replication between distributed topic members can cause duplicate messages in some cases. There is a new feature in WebLogic 10.3.4 that you should look at for this called 'Partitioned Distributed Topics'.
Here are some links for reference:
http://www.oracle.com/technetwork/middleware/weblogic/learnmore/weblogic-javaee6-webcasts-358613.html
http://www.youtube.com/OracleWebLogic
Related
I've spent quite a bit of time trying to figure out whether I should use the RabbitMQ federation plugin or shovel.
Basically I have two microservices. I want one of them to send a message to another. Each microservice has a different rabbitMQ cluster, so I need to use Federation/shovel.
I read this post When to use RabbitMQ shovels and when Federation plugin? and still couldn't figure it out / make a decision.
I want to satisfy the following:
Loose coupling
Microservices don't know about each other -- I.e the first microservice emits a message saying "i'm done doing x". And the second microservice just listens to that 'event' and acts accordingly..
In the future I 'might' want to add more microservices, each with their own rabbitMQ cluster / vhost.
Based on this information - what do you recommend, shovel or federation?
Why not just have one cluster for everything? RabbitMQ is build for handling 10k+ exchanges and queues, actually there is no upper limit except memory or disk space. Setting up a cluster for each microservice is too much work and creates unnecessary overhead. Using vhost should also not be used for this, but for each business area.
I'm only using shovels and I use them to transfer messages from my production environment to test, so I can test with real data. It's very easy to setup with scripts. And yes, you should only do this with scripts. Using the UI is too slow.
I know this doesn't answer your question directly, but I hope it has given you some food for thought.
Happy messaging!
We have to choose the best way of implementing RabbitMQ Queue.
We have two approaches
1. Create a Queue and Bind using #Bean and Queue class in Spring.
2. Create a Queue in RabbitMQ web console itself.
We need to know which is the best way the Programming way or Console way and Why?
IMHO, the better way is using the web console. Queue is an infrastructure and will be used by many applications. You should not provide full control of the infrastructure to applications. It should be maintained by the admin.
Also please consider the following aspects.
Security
Ease of use
Threats
We have multiple web and windows applications which were deployed to different servers that we are planning to integrate using NservierBus to let all apps can pub/sub message between them, I think we using pub/sub pattern and using MSMQ transport will be good for it. but one thing I am not clear if it is a way to avoid hard code to set sub endpoint to MSMQ QueueName#ServerName which has server name in it directly if pub is on another server. on 6-pre I saw idea to set endpoint name then using routing to delegate to transport-level address, is that a solution to do that? or only gateway is the solution? is a broker a good idea? what is the best practice for this scenario?
When using pub/sub, the subscriber currently needs to know the location of the queue of the publisher. The subscriber then sends a subscription-message to that queue, every single time it starts up. It cannot know if it subscribed already and if it subscribed for all the messages, since you might have added/configured some new ones.
The publisher reads these subscriptions messages and stores the subscription in storage. NServiceBus does this for you, so there's no need to write code for this. The only thing you need is configuration in the subscriber as to where the (queue of the) publisher is.
I wrote a tutorial myself which you can find here : http://dennis.bloggingabout.net/2015/10/28/nservicebus-publish-subscribe-tutorial/
That being said, you should take special care related to issues regarding websites that publish messages. More information on that can be found here : http://docs.particular.net/nservicebus/hosting/publishing-from-web-applications
In a scale out situation with MSMQ, you can also use the distributor : http://docs.particular.net/nservicebus/scalability-and-ha/distributor/
As a final note: It depends on the situation, but I would not worry too much about knowing locations of endpoints (or their queues). I would most likely not use pub/sub just for this 'technical issue'. But again, it completely depends on the situation. I can understand that rich-clients which spawn randomly might want this. But there are other solutions as well, with a more centralized storage and an API that is accessed by all the rich clients.
I am putting together a queue based distributed system, all standard stuff. We are using the latest version of RabbitMQ to provide our messaging transport tier.
I have some questions regarding achieving high availability (for my applications and not actually RabbitMQ) that I couldn't answer by reading the documentation. Would appreciate some advice, it's very likely my lack of understanding of Rabbit/AMQP that is causing the problem :)
Problem: I have a message producer (called the primary). There is one and only 1 message producer. There is a secondary producer (called the backup) which should take over from the primary should it fail.
How could I achieve this using existing RabbitMQ capabilities?
Thoughts: Use an "exclusive" queue, to which the primary will be connected to. The backup will attempt to connect to to this queue. When the primary fails, the backup will gain connectivity to the queue and establish control over the process.
What is the correct pattern I should be using to achieve this? I couldn't find any documentation on competing producers etc, would appreciate your advice! How do others do this?
Kind regards
TM
If you want to have only one producer at a time - you can't afford it with RabbitMQ mechanism (unless you'll get some plugin but I don't know such of a kind). You can gain control on producers number on application level.
P.S.:
Looks like you don't get AMQP idea well, producers publish messages to exchanges, while consuming get them from queue. The broker (RabbitMQ) route messages from exchange to on or more queues (in fact, it can also route messages to other exchange, but that's another story).
I have been working on a CQRS project (my first) for over the last 9 months which has been a heavy learning curve. I am currently using JOliver's excellent EventStore in my write model and using PostGresSql for my read model.
Both my read and write databases are on the same machine which means that when a change is made to the write database, in the same synchronous call a change is made to the read model.
As I was learning CQRS I felt this was the best way to go as I had no experience with message queue/service bus frameworks such as MassTransit, NServiceBus etc.
I am now at a point with most of my architecture in place to introduce a message queue framework.
Today, I came across Redis MQ which is part of ServiceStack and as we are already using ServiceStack for our Rest based HTTP clients, this seems like the right way to go.
My question is more about understanding what I need to know (or if I have any misunderstandings) to implement Redis MQ and whether Redis MQ is the right choice?
Now from what I understand, I would use Redis MQ as a durable queue between the write and read database. Once my event store has recorded that something has happened in my domain then it will publish to Redis MQ. The services listening for events/messages would receive the event/message from Redis MQ and once it has processed it (i.e. update or write to the read model), a notification/response goes back to the event store to tell the event store that the message has been received and processed by the listener/subscriber.
Does this sound correct?
Also would the Redis MQ architecture give me everything that NSB, RavenDB, MassTransit etc offer?
Also, I will be deploying to windows 2008 and 2003 server. Is Redis stable for these OSs?
I think the ServiceStack implementation of message queueing in Redis is more appropriate for job-queue scenarios - it pushes a message onto the end of a Redis list and then uses Redis pub-sub to notify listening subscribers that there is a message to pull from the queue. Any consumers would be competing for messages.
For event sourcing, you may be more interested in a type of fanout or topic based messaging topology as offered by RabbitMQ, not that that precludes you from building that sort of thing using Redis data structures yourself.
Now from what I understand, I would use Redis MQ as a durable queue
between the write and read database.
Yes this is correct.
Once my event store has recorded that something has happened in my
domain then it will publish to Redis MQ.
Yes and this can be done in several ways. It can either happen as part of the transaction which persists to the event store or you can have an out of band process which continuously publishes events from the event store.
a notification/response goes back to the event store to tell the event
store that the message has been received and processed by the
listener/subscriber.
The response back to the event publisher is usually omitted. This truly decouples the publishers from subscribers. You make the assumption that once the message is published, all interested subscribers will handle it. If something happens, an error should be logged.
Also would the Redis MQ architecture give me everything that NSB,
RavenDB, MassTransit etc offer?
I don't have experience running Redis MQ, but I do know that Redis supports pub/sub which is one of the value propositions of NSB and MassTransit (as opposed to say bare-bones MSMQ). What MT and NSB offer beyond pub/sub are sagas and it doesn't seem like Redis MQ support those out of the box at least. You may not ever have a need for sagas so this should not automatically be a deterrent. RavenDB is not a message queue so it doesn't apply here.
Also, I will be deploying to windows 2008 and 2003 server. Is Redis
stable for these OSs?
I've run Redis on 2008 R2 and it has been stable so I would think Redis MQ would be stable as well.
You may be interested in a little side project of mine on GitHub which is a queue and persistence implementation for NServiceBus using Redis. https://github.com/mackie1001/NServicebus.Redis
I'd not call it production ready and I want to port it to NSB 4 and do some thorough testing but the meat of it is done.