In the last release of my app, I added a command that tells it to wait when something arrives in the Service Broker queue
WAITFOR (RECEIVE CONVERT(int, message_body) AS Message FROM MyQueue)
The DBAs tell me that since the addition, the log sizes have gone through the roof. Could this be correct? Or should I be looking elsewhere?
I haven't tested this in service broker but I assume the same ACID compliance mechanisms would be in play. It would depend on if it's leaving a transaction open or not in your code. If it is leaving a transaction open and not committing it, the log will continue to grow until something closes it and only at that point will it finally mark the old areas for re-use.
I haven't rolled service broker in prod yet but the testing/reading I did did not include any WAITFOR.
Instead, the Server Broker MVPs like Denny Cherry would typically keep querying the queue instead of doing a WAITFOR.
Can you post some of the other code and also tell us why you're using WAITFOR? Maybe there's something I'm not getting that would be a good use case scenario. Thanks!
Related
Currently I have a queue that stores merge queries which are run once it is read off the queue. This all works well, and currently if there is an error with the merge the queue will disable and I have to manually remove the message (or fix the merge, as it were).
I was wondering whether it was possible to simply move the poisoned message to a table? The queues run important (and different) merges that must continually run to ensure data is updated. It is not beneficial to me for the queue to, say, become disabled over night and gain a huge backlog.
Is there any way for me to simply push the bad message into a table? I have attempted this myself however I wound up having a TRY...CATCH inside a TRANSACTION, which performs a rollback on the error anyway (thus invoking the 5 rollbacks to disable rule). Most solutions online mention only manually removing the message.
Any suggestions? Is this just a bad idea? If so, why?
Thanks.
The disable-after-5-rollbacks can be switched off by setting POISON_MESSAGE_HANDLING status to OFF in the CREATE/ALTER QUEUE statement. You can then use TRY...CATCH to manually deal with transactions that fail.
Like you I don't find this feature very useful, so almost always turn it off in my applications and deal with problem messages in whatever way seems best.
We're running NServiceBus for a web application to handle situations where the user do "batch like" actions. Like fire a command that affects 1000 entities..
It works well, but during moderate load we get some deadlocks, this isn't a problem, just retry the message.. right? :)
The problem occurs when the next message arrives and tries to open a connection. The connection is then "corrupt".
We get the following error:
System.Data.SqlClient.SqlException (0x80131904): New request is not allowed to start because it should come with valid transaction descriptor
I've searched the web and I think our problem is a reported NH "bug":
A workaround should be to disable connection pooling. But I don't like that, since performce will degrade..
We're running NServiceBus 2.6, NHibernate 3.3.
Does anyone have any experience with this? Can a upgrade of NServiceBus help?
I’ve seen this in the past, if your design warrants, try breaking the transaction into two, if you flow the message transaction all the way to your database operations, any failures will have a cascading effect and it will impact (ideally it shouldn’t) any subsequent messages as well.
Instead of updating the 1000 entities in the command could you publishing an event to say that the command has been completed and then have several subscribers acting on this event to update effect entities. It sounds to me that a command that updates a 1000 entities should be split into a number of smaller commands. Take a look a the sagas to see how you can handle long running business process. For example, you might have something like, process started, step 1 completed, step 2 completed , process completed etc...
What is the intended usage of ForwardRecievedMessagesTo?
I read some where that it is to support auditing. Is there any harm in using it as a solution to ensure that messages have been processed and if not reprocessing them? lets say a message was sent to queue_A#server_A and also forwarded to q_All#server_All and before the message was handled, machine_A died irrecoverably. In such a case, I could have a handler pick up messages from q_All#sever_All and check against a database table if the message has been processed. If not reprocess(publish or send) the message or save it in a database table.
Also, what is the performance implication of using forwardreceivedmessageto? How is it different from journalling?
Yes, I am trying to not use msmq clustering.
The feature is there to support auditing. If your machine dies during processing then the messages will backup at the sending machine and would continue to flow after the machine recovered. This means you must size the disk on the sending machine appropriately. You could leverage auditing to accomplish this and the overhead would be minimal. The implication would be the time it would take to complete the distributed transaction to the other machine where your audit queue lives which should be very small.
This is probably a very simple answer, but I'm not seeing an obvious solution in the MassTransit docs or forums.
When you have some messages that have been moved over to the error queue in RabbitMQ, what's the best mechanism for getting them back into the processing queue? Also, is there any built-in logging of why they got moved over there in the first place?
Enable logging with the right plugin (NLog, log4net, etc) and failures should be in the log, assuming the right log level is enabled.
There is no great way to move messages back. Dru has worked on a busdriver tool https://github.com/MassTransit/MassTransit/tree/master/src/Tools/BusDriver. This, I believe, will allow you move items from one queue to another - but it's not a tool I've used. I have historically written tools that are related to business processes to move items back to the proper queue for processing that ops will manage.
I am concerned with my NServiceBus solution.
I have a "MessageHub" that publishes some very important messages. But sometimes it loses track of its subscriptions and just discards the message because it thinks no one is listening.
I have tried turning on "NServiceBus.Integration" to store the subscriptions. But despite that, I still have issues with bad start up order where it thinks nothing is listening.
Is there a way to debug this process? Try to figure out why it is getting confused?
I don't even know a way to look at what subscriptions it "thinks" it has...
I went with NServiceBus because it is not supposed to lose data ever. Now I am losing large chucks. I know it is a config issue, but it is causing much grief.
What is probably happening in your case is that you are using MSMQ for subscription storage. Even though it's possible for subscriptions to endure for a while, using MSMQ to store things long term is always going to be volatile.
For durable subscriptions storage (which survive "forever") you should be using SQL server as your subscription storage.
Note: You can always view your current subscriptions whether you are using sql or msmq to store them. In SQL just look in the subscriptions table and for msmq look in the publisher's subscription queue.
UPDATE
Since version 3 I have been using RavenDb which is the default.
In my experiance, to get the subscriptions assigned correctly, one should first start the EventHandler projects and then when they are all idle, start the CommandHandlers (Publishers).
You can see what messages are being Subscribed to using Service Bus MQ Manager, it has a dialog listing all "messages" and their subscribers/publishers. A side project of mine, its free and open sourced.
http://blog.halan.se/page/Service-Bus-MQ-Manager.aspx