We have a new Win 2012 server hosting a legacy .NET 3.5 WCF Service (with 22 endpoints and maybe a couple of hundred operations) and we are having some performance issues and want to control the number of connections.
We have attached perfmon Web Service/Current Connections counter and see this value repeatedly rise to a steady 24 connections (with occasional spikes to 25).
We have tried managing the number of connections using system.ServiceModel\behaviors\serviceBehaviors\behavior\serviceThrottling. We have tried default values, low numbers, recommended and high numbers:
<!-- default -->
<serviceThrottling />
<!-- low -->
<serviceThrottling maxConcurrentCalls="5" maxConcurrentSessions="10" maxConcurrentInstances="15"/>
<!-- recommended (2 processors) -->
<serviceThrottling maxConcurrentCalls="32" maxConcurrentSessions="200" maxConcurrentInstances="232"/>
<!-- high-->
<serviceThrottling maxConcurrentCalls="2000" maxConcurrentSessions="2000" maxConcurrentInstances="2000"/>
Where else might this connection limit be getting set?
Have we added the correct counter in perfmon?
Update
We've added another client, and doubled the number of requests that the WS server is processing. So this has proved that the throttling is occuring upstream. The upstream client is a .NET3.5 ASP.NET website. The connection to the WCF service is via a wsHttpBinding.
We have already added system.net\connectionManagement\maxconnection and this hasn't had any impact.
What other settings might throttle outgoing ASP.NET connections to a WCF service?
please try adding this section and test the performance again:
<system.net>
<connectionManagement>
<add address="*" maxconnection="250" />
</connectionManagement>
</system.net>
the default value is 2.
Related
I have a web.config that looks like so, and is added as a link to a number of applications in my solution:
<configuration>
<!-- For more info to customize the asp.net core module see https://go.microsoft.com/fwlink/?linkid=838655 -->
<system.webServer>
<!-- As connection proxy request contains double escape sequence, we need to enable it. background Azure ARM Apis are enabled it. -->
<security>
<requestFiltering allowDoubleEscaping="true" />
</security>
<handlers>
<add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified"/>
</handlers>
<aspNetCore processPath="%LAUNCHER_PATH%" arguments="%LAUNCHER_ARGS%" stdoutLogEnabled="false" stdoutLogFile="\\?\%home%\LogFiles\stdout"/>
</system.webServer>
</configuration>
One of the services has potential for long running single requests, and so I want to increase the IIS request timeout for that service, which can be achieved by adding to the aspNetCore configuration like so:
<aspNetCore processPath="%LAUNCHER_PATH%" arguments="%LAUNCHER_ARGS%" stdoutLogEnabled="false" stdoutLogFile="\\?\%home%\LogFiles\stdout" requestTimeout="00:10:00"/>
However, given this web.config is otherwise shared and I don't want to alter the behaviour of my other apps, I'd like configure this via an application setting from my arm templates.
When I attempt something like the following, I get an error that %REQUEST_TIMEOUT% is not a valid time, and deploying fails to startup the webapp.
How can I enable app settings to insert into the request timeout field?
Azure App Service doesn't allow increase requestTimeout, the limit is 230 seconds - see Why does my request time out after 230 seconds?.
I would recommend looking at the Polling (link2) - it is the standard REST-pattern to process long running op.
Other resources:
How can increase Azure App Service 230 sec request time out
Increase azure web app request timeout
If you are using Azure DevOps, some options during deployment:
"XML variable substitution" can be enabled during deployment if you are using Azure App Service deploy task.
Add powershell task to change the request timeout with the value from a pipeline variable.
Note: If the requestTimeout is set to say 10 mins in web.config, the request will continue to run to completion even after 230 seconds. But the client will be terminated by the app service at 230 seconds and will receive an error.
I have a orchestration in BizTalk which is collect to data via web services from SAP.
My process is as below.
We have a SOAP service on receive port and when we get an request from SOAP we transform it to SAP RFC File format and send it to SAP. When we try to get a response from SAP we get an error when response data is big. If response message size is so big our service get a timeout error. Otherwise there is a no problem if the message size is not big.
I tried to increase timeout duration on BizTalk management console but still fails. Whatever I did, the timeout duration is always in 1 minutes.
After Adding below XML config tags to machine.config file I get an error as below figure.
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config
<configuration>
<system.transactions>
<machineSettings maxTimeout="00:20:00" />
</system.transactions>
</configuration>
Below image is SAP Send Port
SAP Send Port Details
In the detail as you can see my time out duration about 10 hour but in SOAP UI I get an timeout error after 1 minute.
Below image is Receive Port
Also you can find biztalk event viewer error as below.
A response message sent to adapter "SOAP" on receive port "WebPort_SAP/MusteriFaturaT/ABC_SAP_Fatura_T_FaturaOrch_InvoiceReceivePort" with URI "/SAP/MusteriFaturaT/ABC_SAP_Fatura_T_FaturaOrch_InvoiceReceivePort.asmx" is suspended.
Error details: The original request has timed out. The response arrived after the timeout interval and it cannot be delivered to the client.
And SOAPUI response screen is blank as below
On WCF Custom send port, change the value of time out (close, open, send, receive) to increase timeout. See the link for more details
You can also use SOAPUI tool to test your web service outside BizTalk to see the response time and response size.
Problem is about SOAP UI timeout issue. When i change the timeout property of SOAP our timeout duration was increase to 110 second. 110 second is default soap timeout duration. For fix this issue you have to add below config options to web.config file.
<system.web>
<httpRuntime executionTimeout="43200" />
</system.web>
Also you have to add below configs to client app.config file.
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name="IBRAHIM_SAP_Fatura_T_FaturaOrch_InvoiceReceivePortSoap" maxReceivedMessageSize="2147483647" receiveTimeout="03:00:00"
sendTimeout="02:00:00" transferMode="Streamed"/>
</basicHttpBinding>
</bindings>
</system.serviceModel>
UPDATE:
Now we have the error it is indicating that it is the client that timed out rather than BizTalk, in this case SoapUI. As per the answer from this question SoapUI: ConnectException: Connection timed out:
Default socket timeout is set to 60000 milliseconds. You can change it:
File -> Preferences -> HTTP Settings -> Socket Timeout
When you are connecting with another client, you will also have to check/set the timeouts in that.
ORIGINAL POST:
If you are using BizTalk 10 or above check the response timeout on the host is set to a value greater than a minute.
I had an issue with Timeouts as well an this was one setting I changed (in this case to the maximum).
You can set up a specific Host for large transactions where the Response timeout in minutes is changed and only those ports that need it use it.
However even after I changed that to the maximum value I encountered a timeout after 15 minutes.
For that I had to disable Use Transaction, only do this if you still encounter issues and you are retrieving data and not inserting/changing data.
The above images are from my blog post about the issue BizTalk Server Negative Acknowledgement issues in SAP and WCF.
As to setting the system.transaction setting in the config, that is rather dangerous to do as it will affect everything, not just that port.
You have to increase your timeout in the machine.config file
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config
<configuration>
<system.transactions>
<machineSettings maxTimeout="00:20:00" />
</system.transactions>
</configuration>
You must also set the property allowExeDefinition to "MachineToApplication" from "MachineOnly"
I have a Windows forms app which uses WCF services. Our application sends messages using one of our WCF services to specific users running our client, so our callback “http:” string is dynamically constructed each time a message is sent to a user. It includes the server IP address and port (126.221.97.105:701) onto which the current user is logged, the user’s id (56281), and the client GUID (7392d27a-e4a0-42e2-89a3-adc332e28934). So, a typical callback “http:” string looks like this:
http://xxx.xxx.xx.xxx:701/CmesCns/CALLBACK/56281/7392d27a-e4a0-42e2-89a3-adc332e28934
We have an http namespace (http://+:701/) on our client and the group “Everyone” is tied to this namespace with all of the access permissions checked (GenericAll, GenericExecute, GenericRead, and GenericWrite). We use “http namespace” to create our namespaces.
Our application has been in production (on Windows Server 2003) for a few years and everything is working fine.
We have recently converted our application to run in the Windows 2008 server environment. The “Target Framework” in each of our projects is set to the “.NET Framework 4.0”. Our application works fine on my Windows 7 developer workstation. That is, I am able to receive messages from our WCF service, but when I place our application onto our Windows 2008 server and I attempt to run the application, I receive the following error message:
"There was no endpoint listening at http://xxx.xxx.xx.xxx:701/CmesCns/CALLBACK/56281/7392d27a-e4a0-42e2-89a3-adc332e28934
that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details.”
The http namespace (http://+:701/) exists on my developer workstation and on my Windows 2008 server. The group “Everyone” is tied the namespace on my Windows 7 box and on my Windows 2008 server, and all of the access permissions are checked (GenericAll, GenericExecute, GenericRead, and GenericWrite).
We have been searching the web for an answer but have not discovered anything. Would anybody have any ideas on why this would work on our Windows 7 workstations, but not on our Windows 2008 servers?
Any help is greatly appreciated!
Kevin
When you host a WCF service in IIS you don't specify an absolute url in the address. You should use a relative url to the .svc file. The base url will be determined by the web site where it is hosted.
<service name="WebService.Receptor">
<endpoint
address="/WS.svc"
binding="wsHttpBinding"
contract="IMyContract"
/>
and on the client, depending on how your IIS is configured you should obviously specify the full address:
<client>
<endpoint
name="Receptor"
address="http://MyServer:8000/WS.svc"
binding="wsHttpBinding"
contract="IMyContract"
/>
This assumes that you have configured a site in IIS that listens on the 8000 port and that you have hosted your WCF application inside this site.
if it does not help please follow these links, hope it would be useful.
Stack overflow link
Multiple Endpoint
Typically, this error is because there is no endpoint on the server that matches what the client is requesting (the address, the service, or the authentication is different).
However, in my case, I had the exact same error, and it was not due to any of these things.
When I enabled the tracing on IIS and reviewed the svclog trace with SvcTraceViewer.exe (included in Visual Studio), the actual internal error was "Maximum request length exceeded."
My client was uploading an image via the service. And I guess the image was too big.
To enable tracing I added this to the configuration section:
<system.diagnostics>
<sources>
<source name="System.ServiceModel"
switchValue="All"
propagateActivity="true">
<listeners>
<add name="traceListener"
type="System.Diagnostics.XmlWriterTraceListener"
initializeData= "c:\log\Traces.svclog" />
</listeners>
</source>
</sources>
To solve the error, I increased the message request length in the web config and the error went away.
To do this, in the system.websection in the web.config I added the line:
<httpRuntime maxRequestLength="32768" />
Then I added this section inside the configuration section
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="32000000" />
</requestFiltering>
</security>
</system.webServer>
So I recommend you enable tracing and then review the trace for the exact error.
We have a WCF net.tcp service in high traffic site.
http://www.artedelcorpo.com/
It works very well for some time, then it stops and returns a timeout error.
When I restart IIS, it runs again.
Why?
Is there a limit on connections?
There is a default setting of 10 connections for the NetTcp Binding. You can increase this in the <binding> section of your config. The same is true of the timeouts - the default is 1 minute, but you can also adjust the close timeout, open timeout, receive timeout and send timeout.
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding name="MyNetTcpBinding" closeTimeout="00:05:00"
openTimeout="00:05:00" receiveTimeout="00:05:00"
sendTimeout="00:05:00" maxConnections="100" />
<netTcpBinding>
</bindings>
</system.serviceModel>
The above sample will set the timeouts to 5 minutes and the max connections to 100.
See <netTcpBinding> for more detail/information.
Increasing network timeouts buys you time for the request to be processed. This can help you survive a short burst of high traffic, but it doesn't change your system throughput.
If your hardware still has some headroom in terms of memory and CPU when these problems occur, you should look at changing your concurrency throttles. This will allow more requests to be processed in parallel. The default throttle values in .NET 3.5 are actually quite conservative and this might be a quick fix (depending on your service architecture).
Here's some good examples of configuring WCF service throttles.
If your hardware is maxed out and you are still getting timeouts, its probably time to cluster the service and add another node.
I have a WCF service that accepts a complex type and returns some data. I want to use Fiddler to see what the incoming requests to the service looks like. The client is .net console app which uses a Service reference proxy. Is this possible with Fiddler. I'm new to this tool and have only used it in the past to post data with the request builder.
You need to add this in your web.config
<system.net>
<defaultProxy>
<proxy bypassonlocal="False" usesystemdefault="True" proxyaddress="http://127.0.0.1:8888" />
</defaultProxy>
</system.net>
then Start Fiddler on the WEBSERVER machine.
Click Tools | Fiddler Options => Connections => adjust the port as 8888.(allow remote if you need that)
Ok, then from file menu, capture the traffic.
That's all, but don't forget to remove the web.config lines after closing the fiddler, because if you don't it will make an error.
Reference : http://fiddler2.com/documentation/Configure-Fiddler/Tasks/UseFiddlerAsReverseProxy
Fiddler listens to outbound requests rather than inbound requests so you're not going to be able to monitor all the requests coming in to your service by using Fiddler.
The best you're going to get with Fiddler is the ability to see all of the requests as they are generated by your Console App (assuming that the app generates web requests rather than using some other pipeline).
If you want a tool that is more powerful (but more difficult to use) that will allow you to monitor ALL incoming requests, you should check out WireShark.
Edit
I stand corrected. Thanks to Eric Law for posting the directions to configuring Fiddler to be a reverse proxy!
Just had this problem, what worked for me was to use localhost.fiddler:
<endpoint address="http://localhost.fiddler/test/test.svc"
binding="basicHttpBinding"
bindingConfiguration="customBinding"
contract="test"
name="customBinding"/>
Consolidating the caveats mentioned in comments/answers for several use cases.
Mostly, see http://docs.telerik.com/fiddler/Configure-Fiddler/Tasks/ConfigureDotNETApp
Start Fiddler before your app
In a console app, you might not need to specify the proxyaddress:
<proxy bypassonlocal="False" usesystemdefault="True" />
In a web application / something hosted in IIS, you need to add the proxyaddress:
<proxy bypassonlocal="False" usesystemdefault="True" proxyaddress="http://127.0.0.1:8888" />
When .NET makes a request (through a service client or HttpWebRequest, etc) it will always bypass the Fiddler proxy for URLs containing localhost, so you must use an alias like the machine name or make up something in your 'hosts' file (which is why something like localhost.fiddler or http://HOSTNAME works)
If you specify the proxyaddress, you must remove it from your config if Fiddler isn't on, or any requests your app makes will throw an exception like:
No connection could be made because the target machine actively refused it 127.0.0.1:8888
Don't forget to use config transformations to remove the proxy section in production
So simple, all you need is to change the address in the config client: instead of 'localhost' change to the machine name or IP
This is straightforward if you have control over the client that is sending the communications. All you need to do is set the HttpProxy on the client-side service class.
I did this, for example, to trace a web service client running on a smartphone. I set the proxy on that client-side connection to the IP/port of Fiddler, which was running on a PC on the network. The smartphone app then sent all of its outgoing communication to the web service, through Fiddler.
This worked perfectly.
If your client is a WCF client, then see this Q&A for how to set the proxy.
Even if you don't have the ability to modify the code of the client-side app, you may be able to set the proxy administratively, depending on the webservices stack your client uses.
Standard WCF Tracing/Diagnostics
If for some reason you are unable to get Fiddler to work, or would rather log the requests another way, another option is to use the standard WCF tracing functionality. This will produce a file that has a nice viewer.
Docs
See https://learn.microsoft.com/en-us/dotnet/framework/wcf/samples/tracing-and-message-logging
Configuration
Add the following to your config, make sure c:\logs exists, rebuild, and make requests:
<system.serviceModel>
<diagnostics>
<!-- Enable Message Logging here. -->
<!-- log all messages received or sent at the transport or service model levels -->
<messageLogging logEntireMessage="true"
maxMessagesToLog="300"
logMessagesAtServiceLevel="true"
logMalformedMessages="true"
logMessagesAtTransportLevel="true" />
</diagnostics>
</system.serviceModel>
<system.diagnostics>
<sources>
<source name="System.ServiceModel" switchValue="Information,ActivityTracing"
propagateActivity="true">
<listeners>
<add name="xml" />
</listeners>
</source>
<source name="System.ServiceModel.MessageLogging">
<listeners>
<add name="xml" />
</listeners>
</source>
</sources>
<sharedListeners>
<add initializeData="C:\logs\TracingAndLogging-client.svclog" type="System.Diagnostics.XmlWriterTraceListener"
name="xml" />
</sharedListeners>
<trace autoflush="true" />
</system.diagnostics>
I have used wire shark tool for monitoring service calls from silver light app in browser to service. try the link gives clear info
It enables you to monitor the whole request and response contents.
I just tried the first answer from Brad Rem and came to this setting in the web.config under BasicHttpBinding:
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding bypassProxyOnLocal="False" useDefaultWebProxy="false" proxyAddress="http://127.0.0.1:8888" ...
...
</basicHttpBinding>
</bindings>
...
<system.serviceModel>
Hope this helps someone.
You can use the Free version of HTTP Debugger.
It is not a proxy and you needn't make any changes in web.config.
Also, it can show both; incoming and outgoing HTTP requests.
HTTP Debugger Free
Use fiddler a Reverse Proxy is the final solution for me.
First, configure fiddler as reverse proxy with REGDIT, like the doc said: https://docs.telerik.com/fiddler/configure-fiddler/tasks/usefiddlerasreverseproxy#configure-fiddler-as-reverse-proxy
1)Click Tools > Fiddler Options. Ensure Allow remote clients to connect is checked
2)Create a new DWORD named ReverseProxyForPort inside HKEY_CURRENT_USER\SOFTWARE\Microsoft\Fiddler2.
3)Set the DWORD to the local port where Fiddler will re-route inbound traffic.
4)Restart Fiddler.
Second, change the client to call service through proxy
for example , here is my client app.config:
<client>
<endpoint address="http://localhost:61236/WeatherForecastService.svc"
binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_IWeatherForecastService"
contract="ServiceReference1.IWeatherForecastService" name="BasicHttpBinding_IWeatherForecastService" />
</client>
change the client to use proxy endpoint address.
WeatherForecastServiceClient client = new WeatherForecastServiceClient("BasicHttpBinding_IWeatherForecastService", "http://localhost:8888/WeatherForecastService.svc");
var data = client.GetData(1000);
client.Close();
Change the localhost in the URL to localhost.fiddler, this small change worked for me.
Also if anyone testing the service from WCF Test Client don't forget to edit the URL in the config endpoint
Right click on the config file
Click Edit with Svc Config Editor
Click on Endpoints and edit the endpoint to localhost.fiddler
Check Start a new proxy while calling method