I have a WCF service that does some document conversions and returns the document to the caller. When developing locally on my local dev server, the service is hosted on ASP.NET Development server, a console application invokes the operation and executes within seconds.
When I host the service in IIS via a .svc file, two of the documents work correctly, the third one bombs out, it begins to construct the word document using the OpenXml Sdk, but then just dies. I think this has something to do with IIS, but I cannot put my finger on it.
There are a total of three types of documents I generate. In a nutshell this is how it works
SQL 2005 DB/IBM DB2 -> WCF Service written by other developer to expose data. This service only has one endpoint using basicHttpBinding
My Service invokes his service, gets the relevant data, uses the Open Xml Sdk to generate a Microsoft Word Document, saves it on a server and returns the path to the user.
The word documents are no bigger than 100KB.
I am also using basicHttpBinding although I have tried wsHttpBinding with the same results.
What is amazing is how fast it is locally, and even more that two of the documents generate just fine, its the third document type that refuses to work.
To the error message:
An error occured while receiving the HTTP Response to http://myservername.mydomain.inc/MyService/Service.Svc. This could be due to the service endpoint binding not using the HTTP Protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the server shutting down). See server logs for more details.
I have spent the last 2 days trying to figure out what is going on, I have tried everything, including changing the maxReceivedMessageSize, maxBufferSize, maxBufferPoolSize, etc etc to large values, I even included:
<httpRuntime maxRequestLength="2097151" executionTimeout="120"/>
To see maybe if IIS was choking because of that.
Programatically the service does nothing special, it just constructs the word documents from the data using the Open Xml Sdk and like I said, locally all 3 documents work when invoked via a console app running locally on the asp.net dev server, i.e. http://localhost:3332/myService.svc
When I host it on IIS and I try to get a Windows Forms application to invoke it, I get the error.
I know you will ask for logs, so yes I have logging enabled on my Host.
And there is no error in the logs, I am logging everything.
Basically I invoke two service operations written by another developer.
MyOperation calls -> HisOperation1 and then HisOperation2, both of those calls give me complex types. I am going to look at his code tomorrow, because he is using LINQ2SQL and there may be some funny business going on there. He is using a variety of collections etc, but the fact that I can run the exact same document, lets call it "Document 3" within seconds when the service is being hosted locally on ASP WebDev Server is what is most odd, why would it run on scaled down Cassini and blow up on IIS?
From the log it seems, after calling HisOperation1 and HisOperation2 the service just goes into la-la land dies, there is a application pool (w3wp.exe) error in the Windows Event Log.
Faulting application w3wp.exe, version 6.0.3790.1830, stamp 42435be1, faulting module kernel32.dll, version 5.2.3790.3311, stamp 49c5225e, debug? 0, fault address 0x00015dfa.
It's classified as .NET 2.0 Runtime error.
Any help is appreciated, the lack of sleep is getting to me.
Help me Obi-Wan Kenobi, you're my only hope.
I had this message appearing:
An error occured while receiving the HTTP Response to http://myservername.mydomain.inc/MyService/Service.Svc. This could be due to the service endpoint binding not using the HTTP Protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the server shutting down). See server logs for more details.
And the problem was that the object that I was trying to transfer was not [Serializable]. The object I was trying to transfer was DataTable.
I believe word documents you were trying to transfer are also non serializable so that might be the problem.
Yes, we'd want logs, or at least some idea of what you're logging. I assume you have both message and transport logging on at the WCF level.
One thing to look at is permissions. When you run under Cassini the web server is running as the currently logged in user. This hides any SQL or CAS permission problems (as, lets be honest, your account is usually a local administrator). As soon as you publish to IIS you are now running under the application pool user, which is, by default, a lot more limited.
Try turning on IIS debug dumps and following the steps in KB919789
Fyi, I changed IIS 6 to work in IIS 5.0 Isolation mode and everything works. Odd.
I had the same error when using an IEnumerable<T> DataMember in my WCF service. Turned out that in some cases I was returning an IQueryable<T> as an IEnumerable<T>, so all I had to do was add .ToList<T>() to my LINQ statements.
I changed the IEnumerable<T> to IList<T> to prevent making the same mistake again.
Related
We have some WCF services implemented in an IIS application, communicating over net.tcp on the default port (808), using the Microsoft Net.Tcp Port Sharing Service, throwing an error on production servers. When I instantiate a connection to the first of the services, I get back an exception:
The server at <URL> rejected the session-establishment request. All the other services respond fine.
But it runs fine on our test servers.
I initially thought there was something wrong with the particular service that was failing, but I tried rearranging the list of services into a different order, and it SEEMS to always be the first service that I hit that fails. (I say SEEMS because it think once in the early iterations of testing, I saw it happen on the second service that it hit. But I haven't been able to reproduce that.)
I've looked at application startup delays, and that doesn't seem to be the problem, because I can come back and run the test again as soon as it finishes - a delay of only a minute or two - and get the same error. Also, in the lower level environments, there is a start up delay of probably 30 seconds to a minute, but the result still comes back as expected.
I've tried accessing the services over http from INetManager, and I get intermittent failures on all the services - a particular service will return a yellow screen of death on on invocation, then come up with the expected link to the WSDL on the next one seconds later.
I'm completely at a loss to explain this behavior, or how to resolve it. I've googled the error message, and not found anything helpful. It may be a configuration issue - the production servers are newly provisioned VM's, and we may not have the config exactly right (whereas all the lower level environments have been running this and other similar apps for some time), but I have not idea what to look for. I've looked at the properties of the app pool that the app is running on and compared it to the lower level environments without finding any differences.
If somebody can point me in the right direction, you would have my undying gratitude.
Things I can find:
http://go4answers.webhost4life.com/Example/connect-busy-wcf-service-host-while-725.aspx:
MaxConcurrentSessions (default = 10) [Per-channel] The maximum number of sessions that a service can accept at one time. Only comes into play with session-based bindings (wsHttp or netTcp)"
http://blogs.infosupport.com/unable-to-generate-a-wcf-proxy-using-svcutil-but-retreiving-the-wsdl-works/
So in the end the trick is to add the additional right on the c:\windows\temp folder for your App Pool Identity [for the service to be able to generate metadata] to solve the problem.
Also, are timeouts or other limits configured and being hit? Give tracing a look and access the service using WcfTestClient and see if you can find underlying errors.
Continuing the discussion started in another post: I used windows service to host a WCF mail sender. The service itself is quite simple: it uses a DataContract to get the basic System.Net.Mail.MailMessage class properties (supports attachment - please see the mentioned post, code by edosoft) and the System.Net.Mail.SmtpClient class to actually send the e-mail. The .NET framework used was 3.5SP1 (with VS2008).
Please note that I already had the code working inside an assembly; the exercise was just to expose the functionality by using a WCF service. I used basicHttpBinding for the endpoint binding, no additional configurations.
When I tested the service in my desktop environment (Windows XP) I noticed the following:
The e-mails were only sent if I restarted the windows service in the first 10-15 seconds after calling the WCF service, otherwise nothing would happen;
No exception/error messages whatsoever in the service trace/log files;
No exception/error messages when calling the SmtpClient.Send(MailMessage msg) method or anywhere else when running the service code.
I played with different bindings and configurations for a while, even tried to send the message from a different thread - no changes. I actually came accross with a forum question related to this problem: the same method called from within an ASMX webservice would only work is an IISRESET was performed. I was able to replicate this issue by creating an ASMX webservice using the same sending code used in the WCF. Hosting the WCF service under IIS was also of no help (same problem).
I decided then to deploy the service in a Windows 2008 environment (windows service hosted) - see, no changes in the original, simple code. Same configuration, same everything. It worked right away, no delays or any other problem I could identify. Does anyone have any idea about this issue? I'll try it in a Windows 2003 environment now, but I'd like to know if it's actually a known bug/limitation, or if it's somehow related to the ASMX bug related (as it appears to be). Does anyone have seen it before?
Thanks,
I have a Problem which confuses me a little bit, resp. where I don't have any idea about what it could be.
The System I'm using is Windows Vista, IIS 7.0, VS2008, Windows Software Factory, Entity Framework, WCF. The Binding for all Webservices is wshttpbinding.
I'm using a Webservice hosted in IIS. This Webservice uses/calls another Webservice (also installed in the IIS). If I use a client calling the first Webservice (which calls the second Webservice) it works fine for about 4-10 Times. And then (it is repeatable to get this Problem, but sometimes it happens after 4, sometimes after 10 Time, but it always will happen), the Service and the IIS gets stuck.
Stuck means, that this Webservice isn't callable anymore and generates an timeout after 1 minute.
Even increasing Timeout doesn't change anything.
If i try to restart the IIS I get an timeout error (and this is really confusing me. It seems that the Webservice has "crashed" somehow and blocks the Restart of the IIS). So the IIS is also "stuck" (it is not really stuck, but I can't restart it). Only if I kill the w3wp.exe IIS is restartable and the Webservice will work again (until i again call this service several times).
The logfiles (i'm no expert in things like logging or where to find/enable such logs, so to say : i'm a newbie) like http-logging, Event Viewer or WCF-Message Logging don't show any hints upon the source of the problem.
I don't have this problem when I'm using a Webservice which doesn't call another Service.
Calling a Webservice is done by Service Reference (I'm using no Proxy-Classes), but I think this should be no Problem.
I have no idea of what is happening, nor how to solve this Problem.
Regards
Rene
Edit. : I hope my posting is more readable now :-)
insert System.Diagnostics.Debugger.Break() into your web service code. When that point is reached, you will be able to step through the service logic. This may help you diagnose the cause of the deadlock.
Another alternative is to turn on WCF Tracing, and diagnose that way.
My application has 50 service endpoints (such as /mysite/myService.svc). It's hosted in IIS. Intermittently (once every two or three days) a service stops responding. It's never the same service that hangs. While a service is hung, some of the other services work fine and some other are also hung.
All clients (from different computers) get this error:
ServiceModel.CommunicationException
Message: An error occurred while receiving the HTTP response to
https://server/mysite/myservice1.svc.
This could be due to the service endpoint binding not using the HTTP
protocol. This could also be due to an HTTP request context being
aborted by the server (possibly due to the service shutting down).
See server logs for more details.
No exceptions are raised by the server when the client attempts to call the service that is hung. All I have is that error on the client side.
I have to manually recycle the application pool to fix the problem.
Do you know what could be the cause? How can I investigate this issue? I'm willing to take a memory dump of the worker process when a service is hung but I would not know what to search for in the dump.
Update (Aug 13 2009): I have almost ruled out the idea that the server runs out of connections (see comment in Shiraz Bhaiji's answer). I might have a new lead: I log all server-side exceptions in a log file. So in theory, when this occurs on the client, no exceptions are raised on the server; otherwise I'd have proof of that in my logs. But what if an error does occur on the server but is happening at a low level where exceptions are not routed to my exception handling code? I have posted this question about scenarios where low level exceptions cannot be handled. I'll keep you informed of the progress of my investigation.
Sounds like you are running out of connections.
By default WCF has a timeout and therefore holds a connection open for 10 mins.
When you recycle the app pool all connections are closed, and therefore things work again.
To fix it check your code to make sure that you close connections / dispose of proxies.
To resolve this, we set establishSecurityContext to False on the binding.
I have not come across this particular issue but would suggest to turn on tracing/message logging for the WCF service in the config for the service and/or the client app (if you have control over that). I've done this in the last few days for a service that I needed to troubleshoot.
The MSDN link here is a good starting point.
Also see the table in this post for the varying levels of trace detail you can configure. There are several levels which can go from exception only logging to full message details. It is quite quick to set this up in the app.config file.
To parse the log file output use the SvcTraceViewer.exe that comes with the Windows SDK, which if you have it installed should be located in this folder: C:\Program Files\Microsoft SDKs\Windows\v6.0\Bin
I have a WCF test service and a test client in the same solution. The service is configured to run on localhost (Ie, "http://localhost:8731/Design_Time_Addresses/MyService/Service") I run the client app and it correctly invokes the service and gets back the correct answer. I've verified via logs that it's definitely running the service that's local to my machine.
However, I can't debug into the service when running the client. WCF is supposed to allow just stepping through, but no breakpoints on the service get hit, and stepping in to the service call doesn't work either. Has anyone ever seen this?
I've checked all the obvious stuff like "is the PDB file being generated?" -- yes, it is. If I run the service project by itself, then hit it with some test client, breakpoints get hit correctly. So debugging on the service works. I can even explicitly "attach to remote process" and debug the service that way. But WCF will not automatically step from the client to the service.
I've had this happen as well. Curiously it seems to happen after I make changes to the build configuration (from x64 to x86). Here's how I've been able to fix it:
Right-click the solution, select Properties.
Under "Startup Project" switch from "Single startup project" to "Multiple startup projects".
Set Action to "Start" for the WCF project.
Even though I'm hosting the project in the local IIS server (not the VS Development Server) this solves it, which is a little puzzling. I suspect that setting a project to "Start" also causes VS to attach to the process for debugging.
http://msdn.microsoft.com/en-us/library/bb157685.aspx
For Visual Studio multi-configurations:
In the Property Pages of your solution, ensure that the "Configuration" for your web-service is set for "Debug", not "Release".
Are you using the automatic WCF service hosting feature in Visual Studio? You can check this by opening the project properties of your WCF service project and navigating to the WCF Options tab. Is the Start WCF Service Host when debugging another project in the same solution checkbox checked? If it is, then this explains the problem.
What's happening is that when you run your client, a separate process is created to automatically host your WCF service. Because it is a separate process, you will not be able to "step into" it when debugging your client. You'll have to run two separate debug sessions, one for your service and one for your client.
An easy way to do this is to put a call to System.Diagnostics.Debugger.Break() in the startup logic for your service. When your service is automatically started by the WCF service host, you'll be prompted to debug it, allowing you to open a second instance of Visual Studio for its debug session.
Also when using VS debug and attaching it to the w3wp.exe process and you have specific port bindings other than http:80:* you must have the debug > web in VS set to the site and port http is running on.
Yes, I got the same problem, WCF uses another processs.
At last, I found a way, i start a new instance(debug->start a new instance) of WCF service first, then start the client side, the breakpoints in the WCF service hits!
One other reason for not stepping into WCF service may be the size of your data that is passed to the WCF methods.
For ex., I was passing an array of dobules of size 1000, and I was getting a bad request error (400) with seeing how the method is run. I made the size 500 and now it debugs the WCF.
In my case, I had published it using Release configuration. You need to select Debug here.
I had similar problem. WCF hosted by WCF Service Host, requests tested by WCF Test Client. My problem was caused by space in build configuration name. As soon as I used configuration name without spaces, debbuging worked.
I guess the real problem was the build output folder ending with folder named by the configuration name.
#msulis - yes, thanks, this remedied it for me. I had also changed recently, the target CPU from any to x86. I couldn't reply directly to you message for some reason so apologize if this comes up as an "answer".