WCF SOAP Error for Larger Requests - wcf

Question

I need to download a few hundred thousand entities from a 3rd party hosted web service. The web service allows me to pull down up to 2,000 entities per web service call, but in practice I have only been able to pull down 50-75 entities at a time because of intermittent errors I receive when I call more records. I'm trying to determine if the problem is due to a setting I need to adjust on my side or if the problem is with the 3rd party web service provider. Here's the error I receive:
An error occurred while receiving the HTTP response to
https://some.vendor.com/API/SomeService.svc. This could be due to the
service endpoint binding not using the HTTP protocol. This could also
be due to an HTTP request context being aborted by the server
(possibly due to the service shutting down). See server logs for more
details.
The following is a copy of my application configuration settings for binding the web service:
<system.serviceModel>
<bindings>
<customBinding>
<binding name="SomeService">
<security defaultAlgorithmSuite="Default" authenticationMode="UserNameOverTransport"
requireDerivedKeys="true" securityHeaderLayout="Strict" includeTimestamp="true"
keyEntropyMode="CombinedEntropy" messageSecurityVersion="WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10">
<localClientSettings cacheCookies="true" detectReplays="false"
replayCacheSize="900000" maxClockSkew="00:05:00" maxCookieCachingTime="Infinite"
replayWindow="00:05:00" sessionKeyRenewalInterval="10:00:00"
sessionKeyRolloverInterval="00:05:00" reconnectTransportOnFailure="true"
timestampValidityDuration="00:05:00" cookieRenewalThresholdPercentage="60" />
<localServiceSettings detectReplays="false" issuedCookieLifetime="10:00:00"
maxStatefulNegotiations="128" replayCacheSize="900000" maxClockSkew="00:05:00"
negotiationTimeout="00:01:00" replayWindow="00:05:00" inactivityTimeout="00:02:00"
sessionKeyRenewalInterval="15:00:00" sessionKeyRolloverInterval="00:05:00"
reconnectTransportOnFailure="true" maxPendingSessions="128"
maxCachedCookies="1000" timestampValidityDuration="00:05:00" />
<secureConversationBootstrap />
</security>
<textMessageEncoding maxReadPoolSize="64" maxWritePoolSize="16"
messageVersion="Soap11" writeEncoding="utf-8">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
</textMessageEncoding>
<httpsTransport manualAddressing="false" maxBufferPoolSize="2147483647"
maxReceivedMessageSize="2147483647" allowCookies="false" authenticationScheme="Anonymous"
bypassProxyOnLocal="false" decompressionEnabled="true" hostNameComparisonMode="StrongWildcard"
keepAliveEnabled="true" maxBufferSize="2147483647" proxyAuthenticationScheme="Anonymous"
realm="" transferMode="Streamed" unsafeConnectionNtlmAuthentication="false"
useDefaultWebProxy="true" requireClientCertificate="false" />
</binding>
</customBinding>
</bindings>
<client>
<endpoint address="https://secure.Some.com/API/1.2/Service.svc"
binding="customBinding" bindingConfiguration="SomeService"
contract="SomeService.SomeService" name="SomeService" />
</client>
</system.serviceModel>
As you can see in the application configuration binding, I increased the maxBufferPoolSize, maxReceivedMessageSize, and maxBufferSize to 2147483647. I also changed the transferMode to Streamed.

Answer 1

There is another important setting that can be set by adding a behavior:
<behaviors>
<endpointBehaviors>
<behavior name="SomeBehavior">
<dataContractSerializer maxItemsInObjectGraph="2147483647" />
</behavior>
</endpointBehaviors>
</behaviors>
Then, update your endpoint:
<client>
<endpoint address="https://secure.Some.com/API/1.2/Service.svc"
binding="customBinding" bindingConfiguration="SomeService"
contract="SomeService.SomeService" name="SomeService"
**behaviorConfiguration="SomeBehavior"**/>
</client>