My machine is Windows 7 ultimate (64 bit). I have installed MSMQ and checked that it is working fine (ran some sample codes for MSMQ).
When i try to create a WCF Service using MsmqIntegrationBinding class, i get the below exception:
"An error occurred while opening the queue:The queue does not exist or you do not have sufficient permissions to perform the operation. (-1072824317, 0xc00e0003). The message cannot be sent or received from the queue. Ensure that MSMQ is installed and running. Also ensure that the queue is available to open with the required access mode and authorization."
I am running the visual studio in Administrator mode and explicitly grant permission to myself via a URL ACL using:
netsh http add urlacl url=http://+:80/ user=DOMAIN\user
Below is the code:
public static void Main()
{
Uri baseAddress = new Uri(#"msmq.formatname:DIRECT=OS:AJITDELL2\private$\Orders");
using (ServiceHost serviceHost = new ServiceHost(typeof(OrderProcessorService), baseAddress))
{
MsmqIntegrationBinding serviceBinding = new MsmqIntegrationBinding();
serviceBinding.Security.Transport.MsmqAuthenticationMode = MsmqAuthenticationMode.None;
serviceBinding.Security.Transport.MsmqProtectionLevel = System.Net.Security.ProtectionLevel.None;
//serviceBinding.SerializationFormat = MsmqMessageSerializationFormat.Binary;
serviceHost.AddServiceEndpoint(typeof(IOrderProcessor), serviceBinding, baseAddress);
serviceHost.Open();
// The service can now be accessed.
Console.WriteLine("The service is ready.");
Console.WriteLine("The service is running in the following account: {0}", WindowsIdentity.GetCurrent().Name);
Console.WriteLine("Press <ENTER> to terminate service.");
Console.WriteLine();
Console.ReadLine();
// Close the ServiceHostBase to shutdown the service.
serviceHost.Close();
}
}
Can you please help?
Make sure you have created the "Orders" queue in MSMQ.
In Windows Server 2008, you can do so from the Server Manager (right click on My Computer and select Manage), then Features -> Message Queuing -> Private Queues. Right click on Private Queues and add your "Orders" queue there.
You may also want to check Nicholas Allen's article: Diagnosing Common Queue Errors. It suggests that your error can only be: "that the queue does not exist, or perhaps you've specified the queue name incorrectly". All the other error cases would have thrown a different exception.
Related
I have a .net micro-service receiving messages using RabbitMQ client, I need to test the following:
1- consumer is successfully connected to rabbitMq host.
2- consumer is listening to queue.
3- consumer is receiving messages successfully.
To achieve the above, I have created a sample application that sends messages and I am debugging consumer to be sure that it is receiving messages.
How can I automate this test? hence include it in my micro-service CI.
I am thinking to include my sample app in my CI so I can fire a message then run a consumer unit test that waits a specific time then passes if the message received, but this seems like a wrong practice to me because the test will not start until a few seconds the message is fired.
Another way I am thinking of is firing the sample application from the unit test itself, but if the sample app fails to work that would make it the service fault.
Is there any best practices for integration testing of micro-services connecting through RabbitMQ?
I have built many such tests. I have thrown up some basic code on
GitHub here with .NET Core 2.0.
You will need a RabbitMQ cluster for these automated tests. Each test starts by eliminating the queue to ensure that no messages already exist. Pre existing messages from another test will break the current test.
I have a simple helper to delete the queue. In my applications, they always declare their own queues, but if that is not your case then you'll have to create the queue again and any bindings to any exchanges.
public class QueueDestroyer
{
public static void DeleteQueue(string queueName, string virtualHost)
{
var connectionFactory = new ConnectionFactory();
connectionFactory.HostName = "localhost";
connectionFactory.UserName = "guest";
connectionFactory.Password = "guest";
connectionFactory.VirtualHost = virtualHost;
var connection = connectionFactory.CreateConnection();
var channel = connection.CreateModel();
channel.QueueDelete(queueName);
connection.Close();
}
}
I have created a very simple consumer example that represents your microservice. It runs in a Task until cancellation.
public class Consumer
{
private IMessageProcessor _messageProcessor;
private Task _consumerTask;
public Consumer(IMessageProcessor messageProcessor)
{
_messageProcessor = messageProcessor;
}
public void Consume(CancellationToken token, string queueName)
{
_consumerTask = Task.Run(() =>
{
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: queueName,
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
_messageProcessor.ProcessMessage(message);
};
channel.BasicConsume(queue: queueName,
autoAck: false,
consumer: consumer);
while (!token.IsCancellationRequested)
Thread.Sleep(1000);
}
}
});
}
public void WaitForCompletion()
{
_consumerTask.Wait();
}
}
The consumer has an IMessageProcessor interface that will do the work of processing the message. In my integration test I created a fake. You would probably use your preferred mocking framework for this.
The test publisher publishes a message to the queue.
public class TestPublisher
{
public void Publish(string queueName, string message)
{
var factory = new ConnectionFactory() { HostName = "localhost", UserName="guest", Password="guest" };
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
var body = Encoding.UTF8.GetBytes(message);
channel.BasicPublish(exchange: "",
routingKey: queueName,
basicProperties: null,
body: body);
}
}
}
My example test looks like this:
[Fact]
public void If_SendMessageToQueue_ThenConsumerReceiv4es()
{
// ARRANGE
QueueDestroyer.DeleteQueue("queueX", "/");
var cts = new CancellationTokenSource();
var fake = new FakeProcessor();
var myMicroService = new Consumer(fake);
// ACT
myMicroService.Consume(cts.Token, "queueX");
var producer = new TestPublisher();
producer.Publish("queueX", "hello");
Thread.Sleep(1000); // make sure the consumer will have received the message
cts.Cancel();
// ASSERT
Assert.Equal(1, fake.Messages.Count);
Assert.Equal("hello", fake.Messages[0]);
}
My fake is this:
public class FakeProcessor : IMessageProcessor
{
public List<string> Messages { get; set; }
public FakeProcessor()
{
Messages = new List<string>();
}
public void ProcessMessage(string message)
{
Messages.Add(message);
}
}
Additional advice is:
If you can append randomized text to your queue and exchange names on each test run then do so to avoid concurrent tests interfering with each other
I have some helpers in the code for declaring queues, exchanges and bindings also, if your applications don't do that.
Write a connection killer class that will force close connections and check your applications still work and can recover. I have code for that, but not in .NET Core. Just ask me for it and I can modify it to run in .NET Core.
In general, I think you should avoid including other microservices in your integration tests. If you send a message from one service to another and expect a message back for example, then create a fake consumer that can mock the expected behaviour. If you receive messages from other services then create fake publishers in your integration test project.
I was successfully doing such kind of test. You need test instance of RabbitMQ, test exchange to send messages to and test queue to connect to receive messages.
Do not mock everything!
But, with test consumer, producer and test instance of rabbitMQ there is no actual production code in that test.
use test rabbitMQ instance and real aplication
In order to have meaniningfull test I would use test RabbitMQ instance, exchange and queue, but leave real application (producer and consumer).
I would implement following scenario
when test application does something that test message to rabbitMQ
then number of received messages in rabbitMQ is increased then
application does something that it should do upon receiving messages
Steps 1 and 3 are application-specific. Your application sends messages to rabbitMQ based on some external event (HTTP message received? timer event?). You could reproduce such condition in your test, so application will send message (to test rabbitMQ instance).
Same story for verifying application action upon receiving message. Application should do something observable upon receiving messages.
If application makes HTTP call- then you can mock that HTTP endpoint and verify received messages. If application saves messages to the database- you could pool database to look for your message.
use rabbitMQ monitoring API
Step 2 can be implemented using RabbitMQ monitoring API (there are methods to see number of messages received and consumed from queue https://www.rabbitmq.com/monitoring.html#rabbitmq-metrics)
consider using spring boot to have health checks
If you are java-based and then using Spring Boot will significantly simpify your problem. You will automatically get health check for your rabbitMQ connection!
See https://spring.io/guides/gs/messaging-rabbitmq/ for tutorial how to connect to RabbitMQ using Spring boot.
Spring boot application exposes health information (using HTTP endpoint /health) for every attached external resource (database, messaging, jms, etc)
See https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html#_auto_configured_healthindicators for details.
If connection to rabbitMQ is down then health check (done by org.springframework.boot.actuate.amqp.RabbitHealthIndicator) will return HTTP code 4xx and meaninfull json message in JSON body.
You do not have to do anything particular to have that health check- just using org.springframework.boot:spring-boot-starter-amqp as maven/gradle dependency is enough.
CI test- from src/test directory
I have written such test (that connect to external test instance of RabbitMQ) using integration tests, in src/test directory. If using Spring Boot it is easiest to do that using test profile, and having details of connection to test RabbitMQ instance in application-test.properties (production could use production profile, and application-production.properties file with production instance of RabbitMQ).
In simplest case (just verify connection to rabbitMQ) all you need is to start application normally and validate /health endpoint.
In this case I would do following CI steps
one that builds (gradle build)
one that run unit tests (tests without any external dependenices)
one that run integration tests
CI test- external
Above described approach could also be done for application deployed to test environment (and connected to test rabbitMQ instance). As soon as application starts, you can check /health endpoint to make sure it is connected to rabbitMQ instance.
If you make your application send message to rabbitMQ, then you could observe rabbbitMQ metrics (using rabbitMQ monitoring API) and observe external effects of message being consumed by application.
For such test you need to start and deploy your application from CI befor starting tests.
for that scenario I would do following CI steps
step that that builds app
steps that run all tests in src/test directory (unit, integration)
step that deploys app to test environment, or starts dockerized application
step that runs external tests
for dockerized environment, step that stops docker containers
Consider dockerized enevironment
For external test you could run your application along with test RabbitMQ instance in Docker. You will need two docker containers.
one with application
one with rabbitMQ . There is official docker image for rabbitmq https://hub.docker.com/_/rabbitmq/ and it is really easy to use
To run those two images, it is most reasonable to write docker-compose file.
I am experimenting with using NServiceBus with MSMQ as transport. A very simple example:
static void Main(string[] args)
{
var endpointConfiguration = new EndpointConfiguration("myappqueue");
endpointConfiguration.UseTransport<MsmqTransport>();
var endpointInstance = Endpoint.Start(endpointConfiguration).Result;
Console.ReadKey();
endpointInstance.Stop();
}
I have added the Windows feature Message Queue in and created a private queue called myappqueue.
When I run the application and get to the line with Endpoint.Start, I get this error:
Faults forwarding requires an error queue to be specified using 'EndpointConfiguration.SendFailedMessagesTo()'
What am I missing? This configuration is not mentioned in the samples on Particular's documentation site.
When an endpoint is created and operational, messages can fail processing. In that case, NServiceBus needs to forward failed messages to the designated error queue which you need to specify. EndpointConfiguration.SendFailedMessagesTo() is the API to use to configure what error queue to use.
You mind find this documentaiton helpful when configuring your endpoint for error handling. And since you're new to NServiceBus, tutorials can be helpful as well.
I have a self-hosted WCF service (runs inside a windows service). This service listens for messages on an MSMQ. The service is PerCall, and Transactional running on Windows 2008 R2, .NET 4.0, MSMQ 5.0.
Once every couple of weeks the service will stop processing messages. The windows service remains running but the WCF servicehost itself stops. The servicehost faults with the following exception:
Timestamp: 3/21/2015 5:37:06 PM Message: HandlingInstanceID:
a26ffd8b-d3b4-4b89-9055-4c376d586268 An exception of type
'System.ServiceModel.MsmqException' occurred and was caught.
--------------------------------------------------------------------------------- 03/21/2015 13:37:06 Type : System.ServiceModel.MsmqException,
System.ServiceModel, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089 Message : An error occurred while
receiving a message from the queue: The transaction's operation
sequence is incorrect. (-1072824239, 0xc00e0051). Ensure that MSMQ is
installed and running. Make sure the queue is available to receive
from. Source : System.ServiceModel Help link : ErrorCode :
-1072824239 Data : System.Collections.ListDictionaryInternal TargetSite : Boolean TryReceive(System.TimeSpan,
System.ServiceModel.Channels.Message ByRef) dynatrace_invocationCount
: 0 Stack Trace : at
System.ServiceModel.Channels.MsmqInputChannelBase.TryReceive(TimeSpan
timeout, Message& message) at
System.ServiceModel.Dispatcher.InputChannelBinder.TryReceive(TimeSpan
timeout, RequestContext& requestContext) at
System.ServiceModel.Dispatcher.ErrorHandlingReceiver.TryReceive(TimeSpan
timeout, RequestContext& requestContext)
Searching for the particular exception ("The transaction's operation sequence is incorrect") doesn't yield a lot of info. And most suggestions for how to remedy a faulted services is to restart the servicehost within the faulted event.
I can do that but I hoping that there is a known fixable cause for this exception and/or whether there is a cleaner way to handle it.
We have faced this issue in our product and we opened a ticket with Microsoft, at the end they admits its a bug in .NET Framework and it will be fixed soon.
The issue was reported on windows server 2008 and 2012 but never on 2016 or windows 10.
So we did two solution, recommended all customers to upgrade to Windows 2016, and we added a code to handle the on fault for the service host to restart the service (You can simulate the same error by restarting the MSMQ service while the WCF service host is open.
The code to restore the service is as below:
first you add an event handler for your host to handle "Faulted" event:
SH.Faulted += new EventHandler(SH_Faulted);
//SH is the ServiceHost
Then inside the event handler
private static void SH_Faulted(object sender, EventArgs e)
{
if (SH.State != CommunicationState.Opened)
{
int intSleep = 15 * 1000;
//Abort the host
SH.Abort();
//Remove the event
SH.Faulted -= new EventHandler(SH_Faulted);
//I sleep to make sure that the MSMQ have enough time to recover, better make it optional.
System.Threading.Thread.Sleep(intSleep);
try
{
ReConnectCounter++;
LogEvent(string.Format("Service '{0}' faulted restarting service count # {1}", serviceName, ReConnectCounter));
//Restart the service again here
}
catch (Exception ex)
{
//failed.. .you can retry if you like
}
}
}
Eventually the error will happen again, but your service will continue working fine, till Microsoft solves the issue or you upgrade to 2016
Updated:
After further investigation, and help from Microsoft we found the root cause of the issue, which is the order of the timeout between the below:
MachineLeveDTCTimeOut(20 minutes) >=
DefaultTimeOut(15 minutes) >=
WCF service transactionTimeout >
receiveTimeout()
So by adding the below it should fix this issue:
<system.transactions>
<defaultSettings timeout="00:05:00"/>
</system.transactions>
More detailed article:
https://blogs.msdn.microsoft.com/asiatech/2013/02/18/wcfmsmq-intermittent-mq_error_transaction_sequence-error/
We have the same problem in our production environment. Unfortunately, there is an issue opened with Microsoft about it, but it's marked "Closed as Deferred" since 2013. The following workaround is mentioned by EasySR20:
If you set the service's receiveTimeout a few seconds less than the
service's transactionTimeout this will prevent the exception from
happening and taking down the service host. These are both settings
that can be set in the server's app.config file.
I haven't confirmed this resolves the issue, but it's one option.
We have implemented the service fault restart option instead.
I have a problem with WCF NetNamedPipeBinding. When I run my server and client code through Visual Studio 2008 on a Windows XP machine everything works fine. But as soon as I deploy my server as a Windows Service and install my client app in Windows Server 2008 I get a TimeoutException on the client end whenever I try to use any of the contract methods. It seems that I can successfully create the client and open it, but can't call any of the methods.
Service initialisation code:
Uri baseAddress = new Uri("http://localhost:8500/xNet/xNetService");
string address = "net.pipe://localhost/xNet/xNetService";
_xNetAPIServiceHost = new ServiceHost(typeof(xNetService), baseAddress);
NetNamedPipeBinding binding = new NetNamedPipeBinding(NetNamedPipeSecurityMode.None);
_xNetAPIServiceHost.AddServiceEndpoint(typeof(IServiceAPI), binding, address);
// Add a mex endpoint
ServiceMetadataBehavior smb = new ServiceMetadataBehavior();
smb.HttpGetEnabled = true;
smb.HttpGetUrl = new Uri("http://localhost:8501/xNet/xNetService/mex");
_xNetAPIServiceHost.Description.Behaviors.Add(smb);
_xNetAPIServiceHost.Open();
Client initialisation code:
string address = "net.pipe://localhost/xNet/xNetService";
NetNamedPipeBinding binding = new NetNamedPipeBinding(NetNamedPipeSecurityMode.None);
_serviceClient = new ServiceAPIClient(binding, new EndpointAddress(address));
_serviceClient.Open();
The Windows service runs as "Local System Account". I'm at a loss as to what the problem is. I don't know if it's a security account problem, or if the named pipe is even open? I would assume since I can successfully create and open the client side it would appear it at least found the named pipe. I just can't call any of the service methods without a TimeoutException.
After trying out various bindings and going back to basics I noticed that the sample programs worked, but mine didn't work unless I was using Visual Studio to debug. I decided at that point that it must be something going on with my own code. To simplify debugging I turned off all security in the binding.
I started commenting out most of the statements in my service's OnStart method in order to determine what could be going on. I commented out everything except for the code that initialises the ServiceHost. Magically, my client could now successfully communicate with the service. I then started uncommenting each line of code in the OnStart method until my client suddenly started giving me a TimeoutException again.
My service class, say "MyAPI", implements the contract "IMyAPI". As well as using "MyAPI" class as the WCF service, I was also using an instance of the "MyAPI" class internally in my service to do various things ("internal" methods). In my OnStart method I first created an instance of the "MyAPI" class and then created the ServiceHost:
MyAPI API = new MyAPI();
ServiceHost service = new ServiceHost(typeof(MyAPI));
I was not getting any errors or exceptions, so it appeared everything is fine, but really I couldn't connect to the service using a client. As soon as I changed the order of the above statements, the client started working again:
ServiceHost service = new ServiceHost(typeof(MyAPI));
MyAPI API = new MyAPI();
I'm not sure WHY this is occuring, all I know is that I can use my API internally and as a service without any client connection issues. Maybe someone out there will provide some light on the reasons behind this, or maybe my code is not properly designed.
I have a WCF host that listen to a topic and process incoming messages. the code look this:
using (ServiceHost host = new ServiceHost(MessagingServiceType))
{
host.Open();
}
and the MessagingServiceType looks like that:
[ServiceBehavior(AddressFilterMode = AddressFilterMode.Any)]
public class MessagingServiceType : IMessagingService
{
/// <summary>
/// service instance
/// </summary>
private readonly MessagingService service;
/// <summary>
/// Initializes a new instance of the <see cref="MessagingServiceType"/> class.
/// </summary>
public MessagingServiceType()
{
// creating new messaging service
service = new Singleton<MessagingService>();
}
/// <summary>
/// Sends the message.
/// </summary>
/// <param name="messageContent">Content of the message.</param>
public void SendMessage(string messageContent)
{
Message msg = Message.CreateMessage(MessageVersion.Default, string.Empty, messageContent);
service.MessageReceived(msg);
}
}
The issue is that when i'm running it on a seperate test application, everything works fine and all the service receive all messages. However, when i take the exact same code and put it into my REAL application, no messages are being received.
my question is very simple: how can i "debug" this service to see what's wrong with it and why messages are not being processed? is there anyway to compare between the two?
Thanks
Sure your service won't receive anything.... just look at your code:
using (ServiceHost host = new ServiceHost(MessagingServiceType))
{
host.Open();
}
What exactly happens when you reach the "}" ?? The object in the "using" clause - your ServiceHost - will be freed / disposed of! ==> before you know it, your service is gone again.....
You need to do something like:
ServiceHost host = new ServiceHost(MessagingServiceType);
host.Open();
Console.ReadLine(); // wait for a ENTER press
host.Close();
No big debugging needed........ :-)
Several options:
You can attach your Visual Studio debugger into your service host
You can to create some message inspector to check received messages
You can use a packet sniffer, but this seems to be an overkill.
Just make sure your service is up and running (no errors thrown from your host) and if you copied all relevant setup information from your development .config file.
Visual Studio offers the ability to attach to remote processes for the purposes of debugging. It will require you to have the Visual Studio Remote Debugging Monitor running on the remote computer.
From visual studio, you can go to Debug / Attach To Process. From the dialog that comes up, you'll see a Qualifier tag that allows you to put in the remote server name. Once you put it in, you can attach to the process for your WCF service. If your service is hosted through IIS, you probably want to attach to the w3wp.exe process.
This will allow you to set breakpoints in your code in the service methods, and when the remote service executes those methods the breakpoints will be hit. It's important that your code be totally in sync with the code on the remote server, otherwise, your lines will be off as you step through the debugger.
Here's a link that also has some tips:
http://geekswithblogs.net/TimH/archive/2006/08/08/87355.aspx
Configuring Tracing and using the Service Trace Viewer can often show you where the issue is.
Once, of course, you make sure the server stays running as per marc_s' answer.