Is there an easy way to subscribe to the default error queue in EasyNetQ? - rabbitmq

In my test application I can see messages that were processed with an exception being automatically inserted into the default EasyNetQ_Default_Error_Queue, which is great. I can then successfully dump or requeue these messages using the Hosepipe, which also works fine, but requires dropping down to the command line and calling against both Hosepipe and the RabbitMQ API to purge the queue of retried messages.
So I'm thinking the easiest approach for my application is to simply subscribe to the error queue, so I can re-process them using the same infrastructure. But in EastNetQ, the error queue seems to be special. We need to subscribe using a proper type and routing ID, so I'm not sure what these values should be for the error queue:
bus.Subscribe<WhatShouldThisBe>("and-this", ReprocessErrorMessage);
Can I use the simple API to subscribe to the error queue, or do I need to dig into the advanced API?
If the type of my original message was TestMessage, then I'd like to be able to do something like this:
bus.Subscribe<ErrorMessage<TestMessage>>("???", ReprocessErrorMessage);
where ErrorMessage is a class provided by EasyNetQ to wrap all errors. Is this possible?

You can't use the simple API to subscribe to the error queue because it doesn't follow EasyNetQ queue type naming conventions - maybe that's something that should be fixed ;)
But the Advanced API works fine. You won't get the original message back, but it's easy to get the JSON representation which you could de-serialize yourself quite easily (using Newtonsoft.JSON). Here's an example of what your subscription code should look like:
[Test]
[Explicit("Requires a RabbitMQ server on localhost")]
public void Should_be_able_to_subscribe_to_error_messages()
{
var errorQueueName = new Conventions().ErrorQueueNamingConvention();
var queue = Queue.DeclareDurable(errorQueueName);
var autoResetEvent = new AutoResetEvent(false);
bus.Advanced.Subscribe<SystemMessages.Error>(queue, (message, info) =>
{
var error = message.Body;
Console.Out.WriteLine("error.DateTime = {0}", error.DateTime);
Console.Out.WriteLine("error.Exception = {0}", error.Exception);
Console.Out.WriteLine("error.Message = {0}", error.Message);
Console.Out.WriteLine("error.RoutingKey = {0}", error.RoutingKey);
autoResetEvent.Set();
return Task.Factory.StartNew(() => { });
});
autoResetEvent.WaitOne(1000);
}
I had to fix a small bug in the error message writing code in EasyNetQ before this worked, so please get a version >= 0.9.2.73 before trying it out. You can see the code example here

Code that works:
(I took a guess)
The screwyness with the 'foo' is because if I just pass that function HandleErrorMessage2 into the Consume call, it can't figure out that it returns a void and not a Task, so can't figure out which overload to use. (VS 2012)
Assigning to a var makes it happy.
You will want to catch the return value of the call to be able to unsubscribe by disposing the object.
Also note that Someone used a System Object name (Queue) instead of making it a EasyNetQueue or something, so you have to add the using clarification for the compiler, or fully specify it.
using Queue = EasyNetQ.Topology.Queue;
private const string QueueName = "EasyNetQ_Default_Error_Queue";
public static void Should_be_able_to_subscribe_to_error_messages(IBus bus)
{
Action <IMessage<Error>, MessageReceivedInfo> foo = HandleErrorMessage2;
IQueue queue = new Queue(QueueName,false);
bus.Advanced.Consume<Error>(queue, foo);
}
private static void HandleErrorMessage2(IMessage<Error> msg, MessageReceivedInfo info)
{
}

Related

How to expect some messages and ignore others with akka.net testkit?

I have a test for an actor that might respond with some unexpected messages but eventually it must respond with a particular known message.
So in essence I want an assertion that will within in some timespan ignore other messages but expect a known message, like so:
[TestMethod]
[TestCategory("Integration")]
public async Task Should_fetch_fund_shareclass_and_details_from_test_service()
{
var testIsins = new HashSet<string> {"isin1", "isin2", "isin3"};
var props = Props.Create(() => new DataFetchSupervisor());
var actor = Sys.ActorOf(props, "fetchSupervisor");
Within(TimeSpan.FromSeconds(30), () =>
{
actor.Tell(new StartDataFetch(testIsins));
//ignore unexpected messages here
var fetchComplteMsg = ExpectMsg<DataFetchComplete>();
});
}
So now this will fail because I get some other messages before DataFetchComplete message.
As always, thanks in advance for any help.
Akka.TestKit class has a number of different versions of ExcpectMsg or equivalent checkers. The one you're looking for is probably a FishForMessage. It takes a predicate and will ignore all incoming messages as long as they will fail to pass predicate's condition. Once a passing message is found this check will complete and your code may be continued.
You can also use the TestKit.IgnoreMessages method, which will accept a delegate function specifying which types of messages you wish to ignore while waiting for your target message.

Recursive Bus.Send() with-in a Handler (Transactions, Threading, Tasks)

I have a handler similar to the following, which essentially responds to a command and sends a whole bunch of commands to a different queue.
public void Handle(ISomeCommand message)
{
int i=0;
while (i < 10000)
{
var command = Bus.CreateInstance<IAnotherCommand>();
command.Id = i;
Bus.Send("target.queue#d1555", command);
i++;
}
}
The issue with this block is, until the loop is fully completed none of the messages appear in the target queue or in the outgoing queue. Can someone help me understand this behavior?
Also if I use Tasks to send messages within the Handler as below, messages appear immediately. So two questions on this,
What's the explanation on Task based Sends to go through immediately?
Are there are any ramifications on using Tasks with in message handlers?
public void Handle(ISomeCommand message)
{
int i=0;
while (i < 10000)
{
System.Threading.ThreadPool.QueueUserWorkItem((args) =>
{
var command = Bus.CreateInstance<IAnotherCommand>();
command.Id = i;
Bus.Send("target.queue#d1555", command);
i++;
});
}
}
Your time is much appreciated!
First question: Picking a message from a queue, running all the registered message handlers for it AND any other transactional action(like writing new messages or writes against a database) is performed in ONE transaction. Either it all completes or none of it. So what you are seeing is the expected behaviour: picking a message from the queue, handling ISomeCommand and writing 10000 new IAnotherCommand is either done completely or none of it. To avoid this behaviour you can do one of the following:
Configure your NServiceBus endpoint to not be transactional
public class EndpointConfig : IConfigureThisEndpoint, AsA_Publisher,IWantCustomInitialization
{
public void Init()
{
Configure.With()
.DefaultBuilder()
.XmlSerializer()
.MsmqTransport()
.IsTransactional(false)
.UnicastBus();
}
}
Wrap the sending of IAnotherCommand in a transaction scope that suppresses the ambient transaction.
public void Handle(ISomeCommand message)
{
using (new TransactionScope(TransactionScopeOption.Suppress))
{
int i=0;
while (i < 10000)
{
var command = Bus.CreateInstance();
command.Id = i;
Bus.Send("target.queue#d1555", command);
i++;
}
}
}
Issue the Bus.Send on another thread, by either starting a new thread yourself, using System.Threading.ThreadPool.QueueUserWorkItem or the Task classes. This works because an ambient transaction is not automatically carried over to a new thread.
Second question: The ramifications of using Tasks, or any of the other methods I mentioned, is that you have no transactional quarantee for the whole thing.
How do you handle the case where you have generated 5000 IAnotherMessage and the power suddenly goes out?
If you use 2) or 3) the original ISomeMessage will not complete and will be retried automatically by NServiceBus when you start up the endpoint again. End result: 5000 + 10000 IAnotherCommands.
If you use 1) you will lose IAnotherMessage completely and end up with only 5000 IAnotherCommands.
Using the recommended transactional way, the initial 5000 IAnotherCommands would be discarded, the original ISomeMessage comes back on the queue and is retried when the endpoint starts up again. Net result: 10000 IAnotherCommands.
If memory serves NServiceBus wraps the calls to the message handlers in a TransactionScope if the transaction option is used and TransactionScope needs some help to be cross-thread friendly:
TransactionScope and multi-threading
If you are trying to reduce overhead you can also bundle your messages. The signature for the send is Bus.Send(IMessage[]messages). If you can guarantee that you don't blow up the size limit for MSMQ, then you could Send() all the messages at once. If the size limit is an issue, then you can chunk them up or use the Databus.

Memory leak using WCF GetCallbackChannel over named pipe

We have a simple wpf application that connects to a service running on the local machine. We use a named pipe for the connection and then register a callback so that later the service can send updates to the client.
The problem is that with each call of the callback we get a build up of memory in the client application.
This is how the client connects to the service.
const string url = "net.pipe://localhost/radal";
_channelFactory = new DuplexChannelFactory<IRadalService>(this, new NetNamedPipeBinding(),url);
and then in a threadpool thread we loop doing the following until we are connected
var service = _channelFactory.CreateChannel();
service.Register();
service.Register looks like this on the server side
public void Register()
{
_callback = OperationContext.Current.GetCallbackChannel<IRadalCallback>();
OperationContext.Current.Channel.Faulted += (sender, args) => Dispose();
OperationContext.Current.Channel.Closed += (sender, args) => Dispose();
}
This callback is stored and when new data arrives we invoke the following on the server side.
void Sensors_OnSensorReading(object sender, SensorReadingEventArgs e)
{
_callback.OnReadingReceived(e.SensorId, e.Count);
}
Where the parameters are an int and a double. On the client this is handled as follows.
public void OnReadingReceived(int sensorId, double count)
{
_events.Publish(new SensorReadingEvent(sensorId, count));
}
But we have found that commenting out _event.Publish... makes no difference to the memory usage. Does anyone see any logical reason why this might be leaking memory. We have used a profiler to track the problem to this point but cannot find what type of object is building up.
Well I can partially answer this now. The problem is partially caused by us trying to be clever and getting the connection to be opened on another thread and then passing it back to the main gui thread. The solution was to not use a thread but instead use a dispatch timer. It does have the downside that the initial data load is now on the GUI thread but we are not loading all that much anyway.
However this was not the entire solution (actually we don't have an entire solution). Once we moved over to a better profiler we found out that the objects building up were timeout handlers so we disabled that feature. That's OK for us as we are running against the localhost always but I can imagine for people working with remote services it would be an issue.

Retry mechanism on WCF operation call when channel in fautled state

I'm trying to find an elegant way to retry an operation when a WCF channel is in faulted state. I've tried using the Policy Injection AB to reconnect and retry the operation when a faulted state exception occurs on first call, but the PolicyInjection.Wrap method doesn't seem to like wrapping the TransparentProxy objects (proxy returned from ChannelFactory.CreateChannel).
Is there any other mechanism I could try or how could I try get the PIAB solution working correctly - any links, examples, etc. would be greatly appreciated.
Here is the code I was using that was failing:
var channelFactory = new ChannelFactory(endpointConfigurationName);
var proxy = channelFactory.CreateChannel(...);
proxy = PolicyInjection.Wrap<IService>(proxy);
Thank you.
I would rather use callback functions, something like this:
private SomeServiceClient proxy;
//This method invokes a service method and recreates the proxy if it's in a faulted state
private void TryInvoke(Action<SomeServiceClient> action)
{
try
{
action(this.proxy);
}
catch (FaultException fe)
{
if (proxy.State == CommunicationState.Faulted)
{
this.proxy.Abort();
this.proxy = new SomeServiceClient();
//Probably, there is a better way than recursion
TryInvoke(action);
}
}
}
//Any real method
private void Connect(Action<UserModel> callback)
{
TryInvoke(sc => callback(sc.Connect()));
}
And in your code you should call
ServiceProxy.Instance.Connect(user => MessageBox.Show(user.Name));
instead of
var user = ServiceProxy.Instance.Connect();
MessageBox.Show(user.Name);
Although my code uses proxy-class approach, you can write a similar code with Channels.
Thank you so much for your reply. What I ended up doing was creating a decorator type class that implemented the interface of my service, which then just wrapped the transparent proxy generated by the ChannelFactory. I was then able to use the Policy Injection Application Block to create a layer on top of this that would inject code into each operation call that would try the operation, and if a CommunicationObjectFaultedException occurred, would abort the channel, recreate it and retry the operation. It's working great now - although it works great, the only downside though is the wrapper class mentioned having to implement every service operation, but this was the only way I could use the PIAB as this made sense for me for in case I did find a way in future, it was easy enough to change just using interfaces.

Persisted properties - asynchronously

In classic ASP.NET I’d persist data extracted from a web service in base class property as follows:
private string m_stringData;
public string _stringData
{ get {
if (m_stringData==null)
{
//fetch data from my web service
m_stringData = ws.FetchData()
}
return m_stringData;
}
}
This way I could simply make reference to _stringData and know that I’d always get the data I was after (maybe sometimes I’d use Session state as a store instead of a private member variable).
In Silverlight with a WCF I might choose to use Isolated Storage as my persistance mechanism, but the service call can't be done like this, because a WCF service has to be called asynchronously.
How can I both invoke the service call and retrieve the response in one method?
Thanks,
Mark
In your method, invoke the service call asynchronously and register a callback that sets a flag. After you have invoked the method, enter a busy/wait loop checking the flag periodically until the flag is set indicating that the data has been returned. The callback should set the backing field for your method and you should be able to return it as soon as you detect the flag has been set indicating success. You'll also need to be concerned about failure. If it's possible to get multiple calls to your method from different threads, you'll also need to use some locking to make your code thread-safe.
EDIT
Actually, the busy/wait loop is probably not the way to go if the web service supports BeginGetData/EndGetData semantics. I had a look at some of my code where I do something similar and I use WaitOne to simply wait on the async result and then retrieve it. If your web service doesn't support this then throw a Thread.Sleep -- say for 50-100ms -- in your wait loop to give time for other processes to execute.
Example from my code:
IAsyncResult asyncResult = null;
try
{
asyncResult = _webService.BeginGetData( searchCriteria, null, null );
if (asyncResult.AsyncWaitHandle.WaitOne( _timeOut, false ))
{
result = _webService.EndGetData( asyncResult );
}
}
catch (WebException e)
{
...log the error, clean up...
}
Thanks for your help tvanfosson. I followed your code and have also found a pseudo similar solution that meets my needs exactly using a lambda expression:
private string m_stringData;
public string _stringData{
get
{
//if we don't have a list of departments, fetch from WCF
if (m_stringData == null)
{
StringServiceClient client = new StringServiceClient();
client.GetStringCompleted +=
(sender, e) =>
{
m_stringData = e.Result;
};
client.GetStringAsync();
}
return m_stringData;
}
}
EDIT
Oops... actually this doesn't work either :-(
I ended up making the calls Asynchronously and altering my programming logic to use MVVM pattern and more binding.