The setup
I have a WCF service hosted in IIS/AppFabric running on Windows Server 2012R2.
The service is bound to a local transactional MSMQ queue via netMsmqBinding. My operations are decorated with TransactionScopeRequired = true.
The service operations recieves calls from a BizTalk server, handles them and send responses back to a remote queue (on the same BizTalk Server), also via a netMsmqBinding.
<endpoint name="Outbound" address="net.msmq://int01test.mydomain.com/private/queue.name" binding="netMsmqBinding" bindingConfiguration="QueueBindingConfigurationOutbound" contract="My.Outbound.Contract" />
<netMsmqBinding>
<binding name="QueueBindingConfigurationOutbound">
<security>
<transport msmqAuthenticationMode="WindowsDomain" msmqProtectionLevel="Sign" />
</security>
</binding>
</netMsmqBinding>
In the testing environment this works as intended.
Physical setup in testing environment:
Server int01test.mydomain.com hosts a BizTalk server and my inbound queue. This runs under service account mydomain\inttestuser.
Server app01test.mydomain.com hosts my application (IIS/AppFabric), my database (SQL server) and my outbound queue. This runs under service account mydomain\apptestuser.
The problem
When this solution is promoted to the acceptance testing environment, calls are still handled, but the responses are blocked with error message:
System.ServiceModel.EndpointNotFoundException: An error occurred while
opening the queue:Unrecognized error -1072824317 (0xc00e0003). The
message cannot be sent or received from the queue. Ensure that MSMQ is
installed and running. Also ensure that the queue is available to open
with the required access mode and authorization. --->
System.ServiceModel.MsmqException: An error occurred while opening the
queue:Unrecognized error -1072824317 (0xc00e0003). The message cannot
be sent or received from the queue. Ensure that MSMQ is installed and
running. Also ensure that the queue is available to open with the
required access mode and authorization.
Differences
In the testing environment, my service and my database is running on a single server instance. (The BizTalk Server and it's queue, the target of my outbound messages, is on another server though)
In the acceptance testing environment, my solution is deployed on two load balanced servers and the database is on a separate cluster.
There are also more strict external firewall rules to mimic the production environment.
Even the BizTalk server is clustered, though we communicate machine-to-machine rather than cluster-to-cluster right now.
So setup in QA Environment is:
Server int01qa.mydomain.com (clustered with int02qa.mydomain.com) hosts a BizTalk server and my inbound queue. This runs under service account mydomain\intqauser.
Server app01qa.mydomain.com (clustered with app02qa.mydomain.com) hosts my application (IIS/AppFabric) and my outbound queue. This runs under service account mydomain\appqauser.
Server db01qa.mydomain.com hosts my database.
What we've already tried
We have disabled authentication on the remote queue.
We have granted full control to the account which my service is running under as well as to "everyone".
We have, successfully, sent msmq messages manually between the two servers.
I have configured my service to send responses to a local private queue, same error.
The problem turned out to be that MSMQ couldn't find a certificate for the app pool user. That is, the
0xc00e0003, MQ_ERROR_QUEUE_NOT_FOUND
was really caused by a
0xC00E002F, MQ_ERROR_NO_INTERNAL_USER_CERT
Changing security settings to
<transport msmqAuthenticationMode="None" msmqProtectionLevel="None" />
enabled messages to be sent.
The real solution, of course, is not to disable security but to ensure that the app pool users cerificate is installed in msmq.
We came across this issue and didn't want to disable authentication. We tried a number of different approaches, but it was something to do with the User Certificate not existing we think.
We went to the App Pool of the client application (which calls the WCF endpoint via MSMQ) and changed the Load Profile property to True. The call then worked. As an aside, changing it back to false continued to work - presumably because it had already sorted the certificate issue out.
Related
We have a logging service that works by pushing into an MSMQ queue and then a persistence service that pulls out of that queue to drop into a database. Both services are WCF. Recently, though the logging service has continued to work without issue, the persistence service has stopped working, and it throws the following error when trying to open:
An error occurred while opening the queue:Generic error code. (-1072824319, 0xc00e0001). The message cannot be sent or received from the queue. Ensure that MSMQ is installed and running. Also ensure that the queue is available to open with the required access mode and authorization.
Obviously, this error is less than helpful. I've tried switching the persistence service to use the same app pool as the logging service, creating a new app for the service, restarting MSMQ, and restarting the servers - but no dice. I'm out of ideas, so any suggestions would be greatly appreciated.
Service Binding
Here's the binding on the services:
<msmqIntegrationBinding>
<binding exactlyOnce="true" durable="true" serializationFormat="Xml" maxReceivedMessageSize="2147483647" closeTimeout="00:00:30" sendTimeout="00:00:30" receiveTimeout="00:00:30" timeToLive="24:00:00" receiveRetryCount="1" maxRetryCycles="1" retryCycleDelay="00:10:00" receiveErrorHandling="Move">
<security mode="Transport"/>
</binding>
</msmqIntegrationBinding>
It hasn't changed since deployment many moons ago, but it looks like the service hasn't been working for just over a month.
EDIT: Well, reinstalling MSMQ on all servers didn't seem to help (in fact, just seemed to mess up my queue access even more). Groovy.
EDIT2: Even if I switch to netMsmqBinding in place of msmqIntegrationBinding, I get the same error.
EDIT3: If I create the queue local to the server instead of on another server, it works. So something about going to the other server is the issue. Restarting DTC doesn't resolve.
EDIT4: I can use a queue on another server, just not the MSMQ server. So it must be something with the connection to the MSMQ server (or the MSMQ server itself).
EDIT5: Ports 1801, 135, 2103, and 2105 are all open between the service server and the queue server.
EDIT6: DTCPing comes back with successes.
I have my binding specified like this in the Service hosted in windows server 2008.
<bindings>
<wsHttpBinding>
<binding name="transactionalWsHttpBinding"
transactionFlow="true" />
</wsHttpBinding>
</bindings>
I have not installed the WSAtomicTransaction roles in the server where the service is hosted.
when I am trying to consume the above hosted service from a different machine, I am getting the following error message.
The flowed transaction could not be unmarshaled. The following exception occurred: The MSDTC transaction manager's WS-AtomicTransaction protocol service 'Version10' is disabled and cannot unmarshal incoming transactions.
should I install the ws-atomic feature ? If I am trying to install the WSAtomic transaction feature in Server, I am forced to add a certificate, should I implement the certificate based authentication?
This worked for me. You may try this.
Open component services in server machine and expand component services and then expand computers expand my computer and expand distributed transaction coordinator right click local DTC open properties:
Go to Security tab, enable Network DTC Access and then enable Allow Inbound in Transaction manager communication leave other settings.
Now open Component Services in Client machine follow above and go to local DTC properties open security tab , enable Network DTC Access and then enable Allow Outbound in Transaction manager communication leave other settings.
I have several websites that push messages to a remote MSMQ (so they can then be published to a list of subscribers). Because the websites don't consume messages -- it's just a send-and-forget set up -- I only have the following config in my web.config for the sites:
<UnicastBusConfig>
<MessageEndpointMappings>
<add Messages="MyNamespace.Messages" Endpoint="PublishingInputQueue#remoteserver" />
</MessageEndpointMappings>
</UnicastBusConfig>
I'm trying to chase down a bug where intermittently messages appear to be successfully sent via Bus.Send(..) (no exceptions are thrown), but the messages never make it to the destination queue.
Is there any other configuration I can do to help diagnose this issue? Is there a way to set up an error queue for messages that fail with UnicastBus? Is it as simple as establishing an Error queue in the MsmqTransportConfig (which I currently don't have in my config?)
Update
A little more context...
As mentioned in the comments, I'm seeing a bunch of messages apparently piled up in the outgoing message queue of the web server(s). (I can't view the actual messages, but can see a count > 0 when clicking on "Outgoing Queues" in Computer Management.)
The publisher that created the destination queue (on remoteserver) has .IsTransactional(true) in the NServiceBus startup code.
The clients that push to that queue via UnicastBus has .IsTransactional(false) in its NServiceBus startup code.
This doesn't seem to cause an issue for other client websites that push to this destination queue. HOWEVER:
The webservers that don't seem to fail are Windows Server 2008
The webservers that do fail are Windows Server 2003
No clue if that makes a difference or not
The domain user the application pools run under has full permissions to the destination queue.
For completeness sake:
It turned out that the Active Directory objects used for the MSMQ service on the servers were out of sync. I believe it occurred when our DevOps team created a single server instance, set it all up, and then cloned it for the other server instances in the cluster. This initial server worked correctly (explaining the intermittent success), but the cloned instances had an out-of-sync AD object for MSMQ. Deleting the objects to allow for recreation fixed the issue.
I have a WPF app that uses WCF (duplex netMsmqBinding) to talk to a self-hosted service app in our domain.
I'm now trying to move this WPF app to the big wide world out there, to talk to the WCF service over the internet (well, make them talk to each other).
I've installed MSMQ and created the services on both, and checked queue and firewall permissions. Based on the reading I've found, what I needed to do then is:
1) Enable MSMQ's HTTP addon.
2) specify the binding's useActiveDirectory=false and queueTransferProtocol="Srmp" to ensure that it uses DIRECT to find the private queue.
However I did that and the WPF app receives the error
An error occurred while opening the queue:Unrecognized error
-1072824215 (0xc00e0069). The message cannot be sent or received from the queue. Ensure that MSMQ is installed and running. Also ensure that
the queue is available to open with the required access mode and
authorization.
Which translates to MQ_ERROR_REMOTE_MACHINE_NOT_AVAILABLE.
I can browse to the remote server ok, and the rest services work fine, so I must be missing something with HTTP or the fact that it's self hosting. Any ideas?
MSMQ over HTTP is a push technology (client -> web server).
You cannot open a remote queue and pull messages from it.
I want to test out the possibility of queuing message on remote clients who may or may not be connected, those clients when connected will push the messages sent to an msmq over the internet that is hosted in IIS 6.
Now, I setup MSMQ on the win server2003 hosting IIS. After I did this "MSMQ" shows up in the IIS default web site.
Ok, then I added a new transactional private queue through computer management-> message queuing.
From there all I want to do is see messages stack up, I'll deal with those after this works.
Now, I made a client app that has the following code:
using (var contract = new HttpMsmqBridgeProxy())
{
var valueToSend = 2456;
contract.TestFunction(valueToSend);
Console.WriteLine("value sent: " + valueToSend + "\r\n");
}
Here's the app.config of this client:
<configuration>
<system.serviceModel>
<client>
<endpoint
address="net.msmq://**.**.***.228/private/MarksTestHttpQueue"
binding="netMsmqBinding"
bindingConfiguration="srmpBinding"
contract="HttpMsmqBridgeLibrary.IHttpMsmqBridgeContract">
</endpoint>
</client>
<bindings>
<netMsmqBinding>
<binding name="srmpBinding"
queueTransferProtocol="Srmp">
<security mode="None"/>
</binding>
</netMsmqBinding>
</bindings>
</system.serviceModel>
</configuration>
The IP is my public facing IP that works, I can host a wcf service or webpage just fine. I followed this guide somewhat for using srmpBinding.
http://msdn.microsoft.com/en-us/library/aa395217.aspx
So, in short what happens when I run the app is it succeeds, tells me it was sent, I go into Message Queue of my client and see that a new queue has shown up in Outgoing folder called:
Direct:http://..*.228/msmq/private$/MarksTestHttpQueue
there is no outgoing messages waiting in this queue so I assume the message was sent.
When I look at my msmq now on the winserver2003 there are no arrived queued messages waiting.
ETA: I can send messages to a non-transactional queue using the classic MessageQueue implimintation:
var queue = new MessageQueue("FormatName:DIRECT=http://**.**.***.228/msmq/private$/nonTransQueue");
System.Messaging.Message Msg;
Msg = new System.Messaging.Message();
Msg.Formatter = new ActiveXMessageFormatter();
Msg.Body = "Testing";
queue.Send(Msg);
The messages show up (after altering the mapping file in the system32/msmq/mapping directory) just fine. I'm wondering if because it's IIS6 I won't be able to send using the net.msmq binding.
You are correct in that your WCF service hosted in IIS6 won't be able to process the messages. This is because IIS6 doesn't use WAS which instantiates processes for non-http requests. But I think that this comes after everything you're doing in the workflow. I would expect
you run your client, pushing the message to the remote queue
the message appears in the remote queue
your WCF service does not pickup the message because it's hosted in IIS6, so you are left with a message in the remote queue.
I don't believe that IIS is involved at all up until the point where it wouldn't be working anyway.
A simple test for this is to self host your service on the server, e.g. run it in a console app. It will be able to accept MSMQ messages just as IIS7 would, and will remove that as a potential problem from your rig.
You might also want to test whether you can push a message directly from the client to a transactional queue on the server. If you're having problems sending messages to transactional queues on other machines then you can possibly check the MSDTC log. I don't envy having to delve into there.