I am experimenting with a new NServiceBus project using Azure Storage Queues for message transport and JSON serialization. I have noticed that when I run a message through the queue that is missing NServiceBus headers, for example an empty JSON message: { } It will throw the following warning message:
2020-02-06 17:46:35.587 WARN NServiceBus.Transport.AzureStorageQueues.MessagePump Azure Storage Queue transport failed pushing a message through pipeline
System.ArgumentNullException: Value cannot be null.
Parameter name: nativeMessageId
at NServiceBus.Transport.IncomingMessage..ctor(String nativeMessageId, Dictionary`2 headers, Byte[] body)
at NServiceBus.Transport.ErrorContext..ctor(Exception exception, Dictionary`2 headers, String transportMessageId, Byte[] body, TransportTransaction transportTransaction, Int32 immediateProcessingFailures)
at NServiceBus.Transport.AzureStorageQueues.ReceiveStrategy.CreateErrorContext(MessageRetrieved retrieved, MessageWrapper message, Exception ex, Byte[] body)
at NServiceBus.Transport.AzureStorageQueues.AtLeastOnceReceiveStrategy.<Receive>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.Transport.AzureStorageQueues.MessagePump.<InnerReceive>d__7.MoveNext()
After which point it appears to stop processing the message but leaves it in the queue. Then, after waiting the configured message invisible period, the message becomes visible in the queue again, and NServiceBus will repeat the 'warning and stop processing' process indefinitely. Is there any way to alter the way NServiceBus handles this scenario so that it will throw the message to the configured error queue when it is unable to parse header information and not attempt to process the message indefinitely?
NServiceBus Storage Queues transport is making an assumption that the messages arrive with the correct envelope. If that envelope is not found, you will get the exception you see above. For messages that are not constructed by NServiceBus or with a custom envelope, please see the documentation here. In short, you need a custom envelope unwrapper.
What the custom unrapper (callback) is responsible for is to deserialize the message and construct a MessageWrapper that NServiceBus is expecting to work with.
var transport = endpointConfiguration.UseTransport<AzureStorageQueueTransport>();
transport.UnwrapMessagesWith(cloudQueueMessage =>
{
using (var stream = new MemoryStream(cloudQueueMessage.AsBytes))
using (var streamReader = new StreamReader(stream))
using (var textReader = new JsonTextReader(streamReader))
{
//try deserialize to a NServiceBus envelope first
var wrapper = jsonSerializer.Deserialize<MessageWrapper>(textReader);
if (wrapper.Id != null)
{
//this was a envelope message
return wrapper;
}
//this was a native message just return the body as is with no headers
return new MessageWrapper
{
Id = cloudQueueMessage.Id,
Headers = new Dictionary<string, string>(),
Body = cloudQueueMessage.AsBytes
};
}
});
Related
Using SpringBoot.
I have created an TopicExchange which accepts messages and directs them to two queues based on a routingKey present in the message.
Messages are sent via :
rabbitTemplate.convertAndSend('in-out-topic', 'inbound.queue.route.key', payload)
Messages are received:
#RabbitListener(queues = "inbound-queue")
def onInboundMessage(def message) {
try {
log.debug("Received inbound message: ${message.messageId} on inbound queue listener", message)
} catch (Exception ex) {
log.error("Inbound message exception: ${ex.getMessage()}")
return;
}
return message.payload
}
But when my listener (consumer) receives a message I get the following exception:
org.springframework.amqp.AmqpException: Cannot determine ReplyTo message property value: Request message does not contain reply-to property, and no default response Exchange was set.
Should I create a dummy response exchange via RabbitMQ dashboard?
Hardcode a non existent replyTo property?
Configure the existing topicExchange or Queues somehow?
I just want the message being removed from the corresponding queue when consumed by my message listener.
Your problem is in the end of method, here:
return message.payload
If you really are not going to send reply and we indeed see that by expectations via convertAndSend(), then you shouldn’t return anything from the #RabbitListener method. Otherwise, as you are experiencing, the return from such a method is treated as an attempt to send a reply.
See more info in the Reference Manual: https://docs.spring.io/spring-amqp/docs/2.0.3.RELEASE/reference/html/_reference.html#async-annotation-driven. Pay attention to the Reply Management paragraph.
I am doing the following steps:
MessageListener receives the message from queue Q1
Validate the message
If validation fails, call channel.basicReject() and move it to dead letter queue
Else, lets say, email server fails. I call channel.basicReject() with requeue true and throw an exception. It goes to retry template and after maxAttempts, is recovered(RepublishMessageRecoverer) and goes to dead letter queue.
But it does not remove the message from Q1.
public void onMessage(Message message, Channel channel) throws Exception {
try {
validateMessage();
processMessage(message);
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
catch (DataValidationException ex){
channel.basicReject(message.getMessageProperties().getDeliveryTag(),false);
}
catch(DownstreamAppException ex) {
channel.basicReject(message.getMessageProperties().getDeliveryTag(),true);
throw ex;
}
}
void validMessage() {
..
throw new DataValidationException();
}
void processMessage() {
...
throw new DownstreamAppException();
}
I do not want to requeue messages that failed validation, but want to requeue those that were not processed because of some downstream app failure for retries.
A couple of questions:
1. If I don't throw exception in catch of DownstreamAppException, message does not go throw retryTemplate and recoverer. Is it because requeuing a rejected message is a new message?
Why is the message not removed from Q1 ? and how can I fix it ?
Thanks
You are responsible for acking when using manual acks (regardless of retry). If your code never acks, the message will (eventually) be requeued; but you don't have access to the channel in the recoverer.
The real question is why are you using manual acks and a ChannelAwareMessageListener? Your use case is straightforward. Using AUTO ackmode, the container will ack the message on success and reject it on any exception.
Since the recoverer republishes the message, that is considered success and the message will be ack'd by the container.
To selectively retry/requeue, you will need a customized error handler see this answer for more information.
I am using IErrorHandler to catch the exceptions for handling and logging purpose in my WCF service. I catch exception in ProvideFault(.....) method and from here I send Fault Exception to the client.
Below is the sample code for it..
public void ProvideFault(Exception error, MessageVersion version, ref Message fault)
{
FaultException<MyResponse> fe = new FaultException<MyResponse>(
new ErrorResponse
{
MessageCode = "ERR001",
MessageDetail = "Error Occured at server Side"
});
MessageFault faultMsg = fe.CreateMessageFault();
fault = Message.CreateMessage(version, faultMsg, fe.Action);
}
Where MyResponse is the custom type.
Now due to some requirement change, I donot want to send FaultMessage from service. Can I send the MyResponse object from ProvideFault method? Something like as..
MyResponse response = new MyResponse();
response.MessageCode = "ERR001";
response.MessageDetail = "Error Occured at server Side";
Message.CreateMessage(version, response, response.Action);
So this can be handled at the client for all scenarios. The reason for this change is that all the client may not handling FaultException.
I do not want to use try catch in my service and send response from catch, instead I want to use IErrorHandler for this.
Any suggestions would be highly appreciated.
We are implementing a saga that calls out to other services with NServiceBus. I'm not quite clear about how NServiceBus deals with exceptions inside a saga.
Inside the saga we have a handler, and that handler calls an external service that should only be called once the original message handler completes succesfully. Is it okay to do:
public void Handle(IFooMessage message)
{
var message = Bus.CreateInstance<ExternalService.IBarMessage>();
Bus.Send(message);
// something bad happens here, exception is thrown
}
or will the message be sent to ExternalService multiple times? Someone here has suggested changing it to:
// handler in the saga
public void Handle(IFooMessage message)
{
// Do something
var message = Bus.CreateInstance<ISendBarMessage>();
Bus.SendLocal(message);
// something bad happens, exception is thrown
}
// a service-level handler
public void Handle(ISendBarMessage message)
{
var message = Bus.CreateInstance<ExternalService.IBarMessage>();
Bus.Send(message);
}
I've done an experiment and from what I can tell the first method seems fine, but I can't find any documentation other than http://docs.particular.net/nservicebus/errors/ which says:
When an exception bubbles through to the NServiceBus infrastructure, it rolls back the transaction on a transactional endpoint, causing the message to be returned to the queue, and any messages that user code tried to send or publish to be undone as well.
Any help to clarify this point would be much appreciated.
As long as you're doing messaging from your saga and not doing any web service calls, then you're safe - no need to do SendLocal.
I'm using the WebChannelFactory<> to create a channel and interact with a WCF REST Service.
When there is an error, I want to retrieve the response from the channel to read the error message from the body of the response. But I cannot figure out how to get the response stream.
Here is my code:
using (var cf = new WebChannelFactory<T>(new Uri(url)))
{
var channel = cf.CreateChannel();
using (new OperationContextScope(channel as IContextChannel))
{
WebOperationContext.Current.OutgoingRequest.Headers
.Add("x-st-authtoken", HttpUtility.UrlDecode(Constants.General_AuthorizedToken));
WebOperationContext.Current.OutgoingRequest.Headers
.Add("x-st-tesskey", HttpUtility.UrlDecode(Constants.General_SessionKey));
try
{
a(channel);
}
catch (Exception ex)
{
throw new Exception("Status: " + ((int)WebOperationContext.Current.IncomingResponse.StatusCode).ToString());
}
}
}
In the catch statement, I want to include the data from the Response body...
It seems like an obvious thing, but I can't seem to find any information on the internet or anything.
Is there any specific reason for you to use ChannelFactory to interact with the REST service. I think it is easier to use HttpWebRequest object to invoke the REST service and there you can get the respone stream when an error is throw on the server.
Also check out RestSharp API through which you can achieve your task to read the response stream.
I believe it will throw a WebException.
so if you explicitly catch that type you can get use the WebException.Response property (which is an HttpWebReponse) on the exception and you can get the content from its stream.