According to this architecture doc, stackoverflow uses L1 (in memory) and L2 (redis) cache, then use Pub/Sub message to sync between instances and invalidate local caches.
I am trying to do the same thing for my site by using the stackexchange.redis library. However, when I put subscriber in startup code (global.asax), some times it won't respond to the messages I publish and the behavior is bit random. The testing code is something I directly copied from the doc.
sub.Subscribe("messages", (channel, message) => {
Console.WriteLine((string)message);
});
I am wondering if stackoverflow ever sees the same problem or maybe other people has some better ideas?
Related
Recently, I have implemented RabbitMQ for a couple of use cases. Sending mails is one of them (which is quite common in practice)
My Problem Statement:
A web service(say service A) needs to publish 1000 messages in the queue (which will be picked by some mail sending engine). But unfortunately, after publishing 500 messages to the queue, my app crashes.
Now, if I hit the same service again then the 500 messages that were already pushed in the first go will be pushed again. Though the mails duplication isn't a big deal for now, but is definitely not desired. How to deal with this one. Any thoughts ?
Solutions that I came up with:
Using the batch feature - but it is not supported by AsyncRabbitTemplate so I'm restrained from using that.
Using the database. But that's definitely cumbersome. I won't use this one as well.
If you can identify the duplicates, you can use the Idempotent Receiver enterprise integration pattern on the consumer side.
Spring Integration has an implementation.
However, it's not clear why you are using the async template since that is for send and receive operations. This application sounds like it only needs to send the requests, not wait for a reply.
It's also not clear how batching can help since the crash could occur on the consumer side after it has processed half of the batch.
In either case, you need to track where you got to before the crash.
I'm trying to understand how to persist data in a Twisted application. Let's say I've decided to write a Twisted server that:
Accepts inbound SMTP requests
Sends the message to a 3rd party system for modification
Relays the modified message to its destination
A typical Twisted tutorial would have you build this app using Deferreds and callbacks, roughly:
A Factory handles inbound requests
Each time a full email is received a call is sent to the remote message processor, returning a deferred
Add an errback that substitutes the original message if anything goes wrong in the modify call.
Add a callback to send the message on to the recipient, which again returns a deferred.
A real server would add/include additional call/errbacks to retry or notify the sender or whatnot. Again for simplicity, assume we consider this an acceptable amount of effort and just log errors.
Of course, this persists NO data in the event of a crash/restart/something else. I get that a solution involves a 3rd party persistent datastore (RabbitMQ is often mentioned) and could probably come up with a dozen random ways to achieve the outcome.
However, I imagine there are a few approaches that work best in a Twisted app. What do they look like? How do they store (and restore in the event of a crash) the in-process messages?
If you found this question, you probably already know that Twisted is event-based. It sounds simple, but the "hardest" part of the answer is to get the persistence platform generating the events we need when we need them. Naturally, you can persist the data in a DB or a message queue, but some platforms don't naturally generate events. For example:
ZeroMQ has (or at least had) no callback for new data. It's also relatively poor at persistence.
In other cases, events are easy but reliability is a problem:
pgSQL can be configured to generate events using triggers, but they're one-time things so you can't resume incomplete events
The light at the end of the tunnel seems to be something like RabbitMQ.
RabbitMQ can persist the message in a database to survive a crash
We can use acknowledgements on both legs (incoming SMTP to RabbitMQ and RabbitMQ to outgoing SMTP) to ensure the application is reliable. Importantly, RabbitMQ supports acknowledgements.
Finally, several of the RabbitMQ clients provide full asynchronous support (see for example pika, txampq, and puka)
It's enough for our purposes that the RabbitMQ client provides us an event-based interface.
At a more theoretical level, however, this need not be the case. In fact, despite the "notification" issue above, ZeroMQ has an event-based client. Even if our software is elegantly event-based, we will run into systems that aren't. In these cases, we have no choice but to fall back on polling. In principle, if not in practice, we just query the message provider for messages. When we exhaust the current queue (and immediately if there are no messages), we use a callLater command to check again in the future. It may feel anti-pattern, but it's (to the best of my knowledge anyway) the right way to handle this particular case.
I have a producer sending durable messages to a RabbitMQ exchange. If the RabbitMQ memory or disk exceeds the watermark threshold, RabbitMQ will block my producer. The documentation says that it stops reading from the socket, and also pauses heartbeats.
What I would like is a way to know in my producer code that I have been blocked. Currently, even with a heartbeat enabled, everything just pauses forever. I'd like to receive some sort of exception so that I know I've been blocked and I can warn the user and/or take some other action, but I can't find any way to do this. I am using both the Java and C# clients and would need this functionality in both. Any advice? Thanks.
Sorry to tell you but with RabbitMQ (at least with 2.8.6) this isn't possible :-(
had a similar problem, which centred around trying to establish a channel when the connection was blocked. The result was the same as what you're experiencing.
I did some investigation into the actual core of the RabbitMQ C# .Net Library and discovered the root cause of the problem is that it goes into an infinite blocking state.
You can see more details on the RabbitMQ mailing list here:
http://rabbitmq.1065348.n5.nabble.com/Net-Client-locks-trying-to-create-a-channel-on-a-blocked-connection-td21588.html
One suggestion (which we didn't implement) was to do the work inside of a thread and have some other component manage the timeout and kill the thread if it is exceeded. We just accepted the risk :-(
The Rabbitmq uses a blocking rpc call that listens for a reply indefinitely.
If you look the Java client api, what it does is:
AMQChannel.BlockingRpcContinuation k = new AMQChannel.SimpleBlockingRpcContinuation();
k.getReply(-1);
Now -1 passed in the argument blocks until a reply is received.
The good thing is you could pass in your timeout in order to make it return.
The bad thing is you will have to update the client jars.
If you are OK with doing that, you could pass in a timeout wherever a blocking call like above is made.
The code would look something like:
try {
return k.getReply(200);
} catch (TimeoutException e) {
throw new MyCustomRuntimeorTimeoutException("RabbitTimeout ex",e);
}
And in your code you could handle this exception and perform your logic in this event.
Some related classes that might require this fix would be:
com.rabbitmq.client.impl.AMQChannel
com.rabbitmq.client.impl.ChannelN
com.rabbitmq.client.impl.AMQConnection
FYI: I have tried this and it works.
I was trying to queue messages to the same consumer using stomp-js on a node server.
Producer:
producer.send({'JMSXGroupID':JMSXGroupID, 'destination':confMgr.getConfig("jmsqueue.destination"), 'body':JSON.stringify(msg), 'persistent':'true'}, false);
Consumer:
client.on('message', function(message) {
client.ack(message.headers['message-id']);
})
I was sending two messages using the same JMSXGroupID and it seems that the the client processess both the messages in parallel rather than processing message1 and ack'ing it and going ahead to process message2 and then ack'ing message2. I tried using 'correlation-id' and it doesn't seem to work either. Can anyone suggest a better way?
Thank you in advance,
Chandra.
I guess you are using this stomp-js lib (correct me if I'm wrong): https://github.com/benjaminws/stomp-js
Message groups are supported by ActiveMQ using Stomp, so you are most likely getting the messages in order. Processing them in order requires you to somehow process each message synchronously on the client, which is rather simple when you can controll how many threads that the listener will run in. This might not be as easy with java script. which is not
From what I can see, the lib you are using is not the most well documented, the only setting you could tweak that might (I have not tried it!), is to alter the prefetch size to one.
var headers = {
destination: '/queue/test_stomp',
ack: 'client',
'activemq.prefetchSize': '1'
};
It might be the case that this lib still starts eagerly directly to fetch the next message, but you might want to test it.
On the other hand, you might as well want to re design the application to be sequence independent, since you are running node.js and java script. It's always better to have a sequence independence with messaging, since you are able to optimize performance a lot better and can avoid synchronous behaviours.
I don't know what you did try to achieve with correlation id, but that header is used to correlate a request with a reply, which is not the case here.
It's a really common pattern and I'm finding it a nightmare to implement!
--
The WcfIntegration sample is almost what I'm looking for in that it receives messages via a WCF endpoint. However, it receives messages back on itself. I want a separate subscriber.
So what I'm trying to do is merge the WcfIntegration and pub/sub samples.
The real need is for a website to call a class library, which then calls the WCF endpoint of the publisher.
A subscriber then picks receives a message that the publisher publishes, and does whatever with it.
--
Rob
You shouldn't need the interface IEventMessageService; NSB handles that for you.
In my local example, I mapped the messages to myself. For you this would be:
<UnicastBusConfig><MessageEndpointMappings>
<add Messages="MyMessages" Endpoint="MyPublisherInputQueue"/>
</MessageEndpointMappings></UnicastBusConfig>
You also need to move your Publish to the actual handler. Right now, it is only happening once (at startup):
public void Handle(EventMessage message)
{
bus.Publish(message);
bus.Return((int)ErrorCodes.None);
}
Make sure your infrastructure is primed, i.e. the queue is transactional, and MSDTC is running.
I'm willing to bet that since there is no mapping, once the service is called, it doesn't know where to put the messages (it calls Bus.Send() internally). I didn't look at the subscribers since that didn't seem to be the issue.
I've finally got it to work here:-
http://code.google.com/p/nservicebus-wcf-pubsub/downloads/list
Anyone is welcome to improve the code.