We have a logging service that works by pushing into an MSMQ queue and then a persistence service that pulls out of that queue to drop into a database. Both services are WCF. Recently, though the logging service has continued to work without issue, the persistence service has stopped working, and it throws the following error when trying to open:
An error occurred while opening the queue:Generic error code. (-1072824319, 0xc00e0001). The message cannot be sent or received from the queue. Ensure that MSMQ is installed and running. Also ensure that the queue is available to open with the required access mode and authorization.
Obviously, this error is less than helpful. I've tried switching the persistence service to use the same app pool as the logging service, creating a new app for the service, restarting MSMQ, and restarting the servers - but no dice. I'm out of ideas, so any suggestions would be greatly appreciated.
Service Binding
Here's the binding on the services:
<msmqIntegrationBinding>
<binding exactlyOnce="true" durable="true" serializationFormat="Xml" maxReceivedMessageSize="2147483647" closeTimeout="00:00:30" sendTimeout="00:00:30" receiveTimeout="00:00:30" timeToLive="24:00:00" receiveRetryCount="1" maxRetryCycles="1" retryCycleDelay="00:10:00" receiveErrorHandling="Move">
<security mode="Transport"/>
</binding>
</msmqIntegrationBinding>
It hasn't changed since deployment many moons ago, but it looks like the service hasn't been working for just over a month.
EDIT: Well, reinstalling MSMQ on all servers didn't seem to help (in fact, just seemed to mess up my queue access even more). Groovy.
EDIT2: Even if I switch to netMsmqBinding in place of msmqIntegrationBinding, I get the same error.
EDIT3: If I create the queue local to the server instead of on another server, it works. So something about going to the other server is the issue. Restarting DTC doesn't resolve.
EDIT4: I can use a queue on another server, just not the MSMQ server. So it must be something with the connection to the MSMQ server (or the MSMQ server itself).
EDIT5: Ports 1801, 135, 2103, and 2105 are all open between the service server and the queue server.
EDIT6: DTCPing comes back with successes.
Related
I host a WCF Service on IIS and have the following binding in web.config:
<bindings>
<wsHttpBinding>
<binding name="transactionalBinding"
transactionFlow = "true"
sendTimeout = "00:00:01"
receiveTimeout = "00:00:01"
openTimeout = "00:00:01"
closeTimeout = "00:00:01">
<security mode="Transport">
<transport clientCredentialType="None" proxyCredentialType="None" realm=""/>
</security>
</binding>
</wsHttpBinding>
</bindings>
In my service method I sleep for 10 seconds. I do not get a timeout exception when calling my service method from a client.
Is there any meaning in defining timeouts in server side bindings?
I do not get a timeout exception when calling my service method from a client.
TL;DR: because WCF timeouts by default are one minute so naturally a server operation that only takes 10 seconds isn't going to timeout. The timeouts you have specified on the server would only affect transmission not execution of your method. (you aren't calling anything else)
You are specifying the timeouts in the server config. What you need to do is specify the timeouts in the client's config file, specifically SendTimeout. Essentially whatever end is making the call, needs to specify the operation timeout. Probably not relevant in your case but if your "server" in turn made another WCF call to another service, you would want your own timeout there too.
MSDN:
SendTimeout – used to initialize the OperationTimeout, which governs the whole process of sending a message, including receiving a reply message for a request/reply service operation. This timeout also applies when sending reply messages from a callback contract method.
Generally, WCF client and server configs should match one another and unless you are using Add Service Reference/Refresh Service Reference each time the server contracts and/or config change, the client won't know about it. By the way, avoid the latter because it duplicates your model and can lead to runtime errors if they are out of sync. Not to mention service contracts can get out of sync.
A passing thought
And this brings up one of the problems of WCF configuration via config files, they are subject to runtime errors impossible to find at compile time.
A better practice is to do away with config files completely and do programatic configuration via a common assembly that both your client and server use. Specify bindings in code along with your timeouts.
That way both server and client are always in sync with regards to WCF configuration.
With both client and server agreeing on timeouts would have addressed some issues.
Tell me more
WCF the Manual Way… the Right Way
The setup
I have a WCF service hosted in IIS/AppFabric running on Windows Server 2012R2.
The service is bound to a local transactional MSMQ queue via netMsmqBinding. My operations are decorated with TransactionScopeRequired = true.
The service operations recieves calls from a BizTalk server, handles them and send responses back to a remote queue (on the same BizTalk Server), also via a netMsmqBinding.
<endpoint name="Outbound" address="net.msmq://int01test.mydomain.com/private/queue.name" binding="netMsmqBinding" bindingConfiguration="QueueBindingConfigurationOutbound" contract="My.Outbound.Contract" />
<netMsmqBinding>
<binding name="QueueBindingConfigurationOutbound">
<security>
<transport msmqAuthenticationMode="WindowsDomain" msmqProtectionLevel="Sign" />
</security>
</binding>
</netMsmqBinding>
In the testing environment this works as intended.
Physical setup in testing environment:
Server int01test.mydomain.com hosts a BizTalk server and my inbound queue. This runs under service account mydomain\inttestuser.
Server app01test.mydomain.com hosts my application (IIS/AppFabric), my database (SQL server) and my outbound queue. This runs under service account mydomain\apptestuser.
The problem
When this solution is promoted to the acceptance testing environment, calls are still handled, but the responses are blocked with error message:
System.ServiceModel.EndpointNotFoundException: An error occurred while
opening the queue:Unrecognized error -1072824317 (0xc00e0003). The
message cannot be sent or received from the queue. Ensure that MSMQ is
installed and running. Also ensure that the queue is available to open
with the required access mode and authorization. --->
System.ServiceModel.MsmqException: An error occurred while opening the
queue:Unrecognized error -1072824317 (0xc00e0003). The message cannot
be sent or received from the queue. Ensure that MSMQ is installed and
running. Also ensure that the queue is available to open with the
required access mode and authorization.
Differences
In the testing environment, my service and my database is running on a single server instance. (The BizTalk Server and it's queue, the target of my outbound messages, is on another server though)
In the acceptance testing environment, my solution is deployed on two load balanced servers and the database is on a separate cluster.
There are also more strict external firewall rules to mimic the production environment.
Even the BizTalk server is clustered, though we communicate machine-to-machine rather than cluster-to-cluster right now.
So setup in QA Environment is:
Server int01qa.mydomain.com (clustered with int02qa.mydomain.com) hosts a BizTalk server and my inbound queue. This runs under service account mydomain\intqauser.
Server app01qa.mydomain.com (clustered with app02qa.mydomain.com) hosts my application (IIS/AppFabric) and my outbound queue. This runs under service account mydomain\appqauser.
Server db01qa.mydomain.com hosts my database.
What we've already tried
We have disabled authentication on the remote queue.
We have granted full control to the account which my service is running under as well as to "everyone".
We have, successfully, sent msmq messages manually between the two servers.
I have configured my service to send responses to a local private queue, same error.
The problem turned out to be that MSMQ couldn't find a certificate for the app pool user. That is, the
0xc00e0003, MQ_ERROR_QUEUE_NOT_FOUND
was really caused by a
0xC00E002F, MQ_ERROR_NO_INTERNAL_USER_CERT
Changing security settings to
<transport msmqAuthenticationMode="None" msmqProtectionLevel="None" />
enabled messages to be sent.
The real solution, of course, is not to disable security but to ensure that the app pool users cerificate is installed in msmq.
We came across this issue and didn't want to disable authentication. We tried a number of different approaches, but it was something to do with the User Certificate not existing we think.
We went to the App Pool of the client application (which calls the WCF endpoint via MSMQ) and changed the Load Profile property to True. The call then worked. As an aside, changing it back to false continued to work - presumably because it had already sorted the certificate issue out.
Perhaps I am barking up the wrong tree - but I have a set of services (WebAPI and WCF) that use WebSphere MQ to interact with other systems.
This works without issue - until I now need to find a way of listening for messages on one of the queues.
Is this even possible, or do I need to go down the windows Service route?
You could write a Windows service that is continually calling MQ Get on the queue, and invokes a WCF service to process the message. Or you could write a trigger program (a console application) that MQ will launch for you when a message arrives, that invokes the WCF service.
I might be just better at googling than you are, but I seem to have found the answer here.
Seems you want to load the IBM binding configuration in you app.config
<extensions>
<bindingElementExtensions>
<add name="IBM.XMS.WCF.SoapJmsIbmTransportChannel"
type="IBM.XMS.WCF.SoapJmsIbmTransportBindingElementConfig, IBM.XMS.WCF, Version=7.5.0.0, Culture=neutral, PublicKeyToken=8c7c0be90afcd8ba"/>
</bindingElementExtensions>
</extensions>
Then you can add a WebSphere WCF binding config.
<bindings>
<customBinding>
<binding name="CustomBinding_WMQ">
<textMessageEncoding messageVersion="Soap11" />
<IBM.XMS.WCF.SoapJmsIbmTransportChannel />
</binding>
</customBinding>
</bindings>
Your problem can be broken down into two distinct elements:
How to integrate MQ with a WCF-supported transport
How to expose a WCF endpoint over this transport
To address the first issue, you should look at the MQ-MSMQ bridge which ships with Host Integration Server up to version 2009 (not R2), which allows you to have messages delivered to MQSeries queues forwarded to local MSMQs in windows. Although this feature is deprecated it's probably the easiest way if you have a MSDN license.
Another way of addressing this issue is to use BizTalk server which ships with a MQSeries adapter, though unless you're using BizTalk currently in your enterprise I would avoid.
The last way you could do this is to program directly against the MQSeries .NET client libraries or via the XMS client.
If you manage to solve the first issue then solving the second one is easy enough. You can expose one way WCF service operations over msmq transport by using the netMsmqBinding (for WCF on both ends), or msmqIntegrationBinding for clients using System.Messaging or native msmq COM libraries.
This in-effect acts as a listener service, with messages being handled by the service operation.
how to get connect with ibm websphere mq by using c#.net
Perhaps you could use the above answer and within that queue consumer app create a "Service Reference" to your WCF service.
I want to test out the possibility of queuing message on remote clients who may or may not be connected, those clients when connected will push the messages sent to an msmq over the internet that is hosted in IIS 6.
Now, I setup MSMQ on the win server2003 hosting IIS. After I did this "MSMQ" shows up in the IIS default web site.
Ok, then I added a new transactional private queue through computer management-> message queuing.
From there all I want to do is see messages stack up, I'll deal with those after this works.
Now, I made a client app that has the following code:
using (var contract = new HttpMsmqBridgeProxy())
{
var valueToSend = 2456;
contract.TestFunction(valueToSend);
Console.WriteLine("value sent: " + valueToSend + "\r\n");
}
Here's the app.config of this client:
<configuration>
<system.serviceModel>
<client>
<endpoint
address="net.msmq://**.**.***.228/private/MarksTestHttpQueue"
binding="netMsmqBinding"
bindingConfiguration="srmpBinding"
contract="HttpMsmqBridgeLibrary.IHttpMsmqBridgeContract">
</endpoint>
</client>
<bindings>
<netMsmqBinding>
<binding name="srmpBinding"
queueTransferProtocol="Srmp">
<security mode="None"/>
</binding>
</netMsmqBinding>
</bindings>
</system.serviceModel>
</configuration>
The IP is my public facing IP that works, I can host a wcf service or webpage just fine. I followed this guide somewhat for using srmpBinding.
http://msdn.microsoft.com/en-us/library/aa395217.aspx
So, in short what happens when I run the app is it succeeds, tells me it was sent, I go into Message Queue of my client and see that a new queue has shown up in Outgoing folder called:
Direct:http://..*.228/msmq/private$/MarksTestHttpQueue
there is no outgoing messages waiting in this queue so I assume the message was sent.
When I look at my msmq now on the winserver2003 there are no arrived queued messages waiting.
ETA: I can send messages to a non-transactional queue using the classic MessageQueue implimintation:
var queue = new MessageQueue("FormatName:DIRECT=http://**.**.***.228/msmq/private$/nonTransQueue");
System.Messaging.Message Msg;
Msg = new System.Messaging.Message();
Msg.Formatter = new ActiveXMessageFormatter();
Msg.Body = "Testing";
queue.Send(Msg);
The messages show up (after altering the mapping file in the system32/msmq/mapping directory) just fine. I'm wondering if because it's IIS6 I won't be able to send using the net.msmq binding.
You are correct in that your WCF service hosted in IIS6 won't be able to process the messages. This is because IIS6 doesn't use WAS which instantiates processes for non-http requests. But I think that this comes after everything you're doing in the workflow. I would expect
you run your client, pushing the message to the remote queue
the message appears in the remote queue
your WCF service does not pickup the message because it's hosted in IIS6, so you are left with a message in the remote queue.
I don't believe that IIS is involved at all up until the point where it wouldn't be working anyway.
A simple test for this is to self host your service on the server, e.g. run it in a console app. It will be able to accept MSMQ messages just as IIS7 would, and will remove that as a potential problem from your rig.
You might also want to test whether you can push a message directly from the client to a transactional queue on the server. If you're having problems sending messages to transactional queues on other machines then you can possibly check the MSDTC log. I don't envy having to delve into there.
I'm trying to get IIS6 to work reliably with a WCF service I have hosted in a separate Windows Service application on the same machine. Users connect to IIS via some HTTP exposed services, which is working fine, and then IIS needs to get some information from the Windows service to put in the HTTP response. I also need a callback channel between the Windows Service and IIS.
After a lot of effort I got it working with a netTcpBinding and everything would be rosey for 5 or 10 minutes but after that IIS would report the WCF channel as being faulted and then clam up and stop processing any requests until the worker process got recycled and the whole thing repeated.
I've been trying to swap to a netNamedPipeBinding but IIS refuses or is refused access to the pipe with an "There was no endpoint listening at net.pipe://localhost/mypipename" error. I can connect to the pipe fine from a console app.
So my question is has anyone got either of those two bindings working with IIS as a client or have any other approaches?
We are using IIS 7 hosting about 20 services with the net.tcp and net.pipe bindings and it's working fine.
Your problem with the pipe looks like a misconfiguration to me. If it helps, this is how we have them configured:
Server:
<endpoint address ="" binding="fooBinding"
contract="Bla.IBlaAPI"
bindingConfiguration="BlaAPI.BindingConfig">
Binding config:
<binding name="BlaAPI.BindingConfig"
receiveTimeout = "10:50:00"
sendTimeout = "10:50:00"
maxReceivedMessageSize="2147483647"
maxBufferSize="2147483647"
maxBufferPoolSize="2147483647"
transactionFlow="false">
<readerQuotas maxDepth="32"
maxStringContentLength="2147483647"
maxArrayLength="2147483647"
maxBytesPerRead="8192"
maxNameTableCharCount="2147483647" />
<security mode="None"/>
</binding>
Note that we are using long timeouts and have really high quotas for message size and etc. because we are passing some big chunks of data through this service. You can adjust for your own needs. We have the security set to "none" because the service is only being contacted from the local machine which is secured. Again, your mileage may vary.
Client:
<endpoint name="Bla.Bindings.BlaAPI" address="net.pipe://localhost/bla/IBlaAPI.svc"
behaviorConfiguration="BlaAPI.ServiceBehavior"
binding="netNamedPipeBinding" bindingConfiguration="BlaAPI.BindingConfig"
contract="Bla.IBlaAPI" />
About the Faulted state problem, please note that if an unhandled exception occurs during execution of the service code, the service instance will remain in Faulted state until it is closed properly. To avoid this, either handle exceptions at service top-level or use, for example, Enterprise Library Excexption Handling blocks.
Re NetNamedPipeBinding and "There was no endpoint listening at net.pipe://localhost/mypipename"
Is your web application impersonating its users? The above error is what you get if you try to access a WCF service via the named pipe binding, in a security context whose logon token has membership of NETWORK USERS. The WCF client-side channel stack doesn't distinguish between access denied errors and "not found" errors, when it attempts to read the shared memory object created by the service to publish the name of the pipe in use. (See http://blogs.charteris.com/blogs/chrisdi/archive/2008/05.aspx etc)
Impersonation tokens in an IIS application will always have NETWORK USERS membership.
Can you show me the code you use to dispose of the wcf client proxy?
Never use 'using' on a wcf proxy, as it will not dispose correctly every time. This can possibly lead to the faulted state.