The formatter threw an exception while trying to deserialize the message: Error in deserializing body of request message for operation 'PostTransaction'. The maximum string content length quota (8192) has been exceeded while reading XML data. This quota may be increased by changing the MaxStringContentLength property on the XmlDictionaryReaderQuotas object used when creating the XML reader. Line 52
You need add this argument to your config, like
<bindings> <webHttpBinding> <binding> <readerQuotas maxStringContentLength="65535"/> </binding> </webHttpBinding> </bindings>
Exaple from this answer: Fixing maximum length quota on XmlDictionaryReaderQuotas for WCF 4.0 REST
Related
In my streamed WCF application I get System.OutOfMemoryExceptions in my service for large messages (querying >7GB MSSQL tables using a datareader, messages might even exceed 7GB) while small messages work just fine. I can observe the memory usage grow constantly during the execution of DataReaderToExcelXml (see below). The weird thing is, that it usually grows fast to 2GB, stays at 2GB +-1GB for a while (1-5 minutes) and then again raises very fast to ~6.5GB which leads to the exception (machine has 8GB memory). At this point it looks to me, as if the Stream isn't passed through anymore but buffered.
I already enabled trace logging, but it seems to stop with the exception. The DataReaderToExcelXml function call is the last visible event in the trace log.
In the WCF message contract I ensured that the message only contains the Stream. On client side, the returned Stream is simply read, written to a filestream and disposed. However, I can never observe the client code being executed or the file being written when I get the exception.
I already tried setting the maxBufferPoolSize to zero, both on the client and server side, as described here https://stackoverflow.com/a/595871/4166885. No success.
Stream writer function on WCF-service side:
Public Shared Function DataReaderToExcelXml(ByRef dr As SqlDataReader) As Stream
Dim ms As New MemoryStream
Dim tw As New IO.StreamWriter(ms)
For Each row As DbDataRecord In dr
'Embed row in ExcelXml, detailed function omitted
tw.write(row.toString()) 'row.toString is just a simplification
End While
tw.Flush()
dr.Close()
ms.Seek(0, SeekOrigin.Begin)
Return ms
End Function
Web.config bindings
<bindings>
<basicHttpBinding>
<binding receiveTimeout="24.00:00:00" sendTimeout="24.00:00:00"
maxBufferPoolSize="9223372036854775807" maxReceivedMessageSize="9223372036854775807"
messageEncoding="Mtom" transferMode="Streamed" bypassProxyOnLocal="True">
<readerQuotas maxStringContentLength="2147483647" maxArrayLength="2147483647"
maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="Transport">
<transport clientCredentialType="None" />
</security>
</binding>
</basicHttpBinding>
</bindings>
App.config bindings
<binding name="BasicHttpBinding_IFileService" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="24.00:00:00" sendTimeout="24.00:00:00"
allowCookies="false" bypassProxyOnLocal="true" hostNameComparisonMode="StrongWildcard"
maxBufferSize="2147483647" maxBufferPoolSize="2147483647"
maxReceivedMessageSize="8589934592" messageEncoding="Mtom"
textEncoding="utf-8" transferMode="Streamed">
<readerQuotas maxDepth="32" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="4096" maxNameTableCharCount="2147483647" />
<security mode="Transport">
<transport clientCredentialType="None" proxyCredentialType="None"
realm="" />
<message clientCredentialType="UserName" algorithmSuite="Default" />
</security>
</binding>
edit: Using .NET memory profiler on my service I found out, that the increasing amount of memory can be traced back to the type byte[]. Personally, that's not telling anything to me, but maybe it's useful information.
edit2: While inspecting the System.OutOfMemoryException I just noticed that it occurs when ms.Capacity = ms.length = 1073741824 = 1GB. So just before the memory stream would double its capacity. I'm still not sure, why w3wp consumes such large amounts of memory in the first place, but now it's clear that the memory stream triggers the Exception.
Well, reading your question, I think that the source of your error, is the key. Where is exactly the error? In transport, or in your server?
If the Exception is raised in DataReaderToExcel, maybe you could consider another ideas (I think it isn't a WCF issue):
By now, I think you would have the same problem in a Windows forms app.
You need memory, so you can fill it only with your SqlDataReader, you have to query by blocks, based on ranges.
We will start computing the "TotalRecords":
SELECT COUNT(*)
FROM YOUR_TABLE
WHERE WATHEVER_YOU_WANT
For example, querying blocks of 10.000 records, you have TotalRecords/10.000 pages (querys) +1. (Example: 30.001 rows, 10.000 row per block = 4 queries )
Iterate building them so:
select top **10.000** * from
(SELECT TOP (100) PERCENT ROW_NUMBER() OVER (ORDER BY YOUR_TABLE_FIELD_1 DESC) ROW_PAGINATED, YOUR_TABLE_FIELD_1, YOUR_TABLE_FIELD_2 , ... , YOUR_TABLE_FIELD_N
FROM YOUR_TABLE
WHERE WATHEVER_YOU_WANT
ORDER BY YOUR_TABLE_FIELD_1 DESC
) YOUR_ALIAS
WHERE YOUR_ALIAS.ROW_PAGINATED BETWEEN min_records_per_page AND max_records_per_page
)
Where min_records_per_page AND max_records_per_page have this values:
Query 1:
min_records_per_page= 1
max_records_per_page = 10000
Query 2:
min_records_per_page= 10001
max_records_per_page = 20000
...
Query N:
min_records_per_page= (N-1)* +1
max_records_per_page = TotalRecords
On each iteration, you will map each datarow into a ExcelXml class.
Doing so, you will avoid to consume all the memory, assuming you will be Disposing the objects you use. You can use the Function SetProcessWorkingSetSize each 2 iterations, for example.
To free memory after a significant number of processed rows:
Private Declare Auto Function SetProcessWorkingSetSize Lib "kernel32.dll" (ByVal procHandle As IntPtr, ByVal min As Int32, ByVal max As Int32) As Boolean
Dim Mem As Process
Mem = Process.GetCurrentProcess()
SetProcessWorkingSetSize(Mem.Handle, -1, -1)
Then, now you have an array with mapped objects. Part 1 finished.
Part 2. Your client WCF may receive this data. Tell us then if you have problems once the data to be sent are ready. Then we can explore communication parameters, and then we would say "increase that parameter, or add this one".
Hope it helps
when try a simple wcf service call at client, the string data member received at service side has its whitespaces at the beginning removed if we using binding as below.
binding:
<binding name="WSHttpBinding_CustomerService"
messageEncoding="Mtom" >
<security mode="None" />
</binding>
the question is: why is the front whitespace removed at service side when using this binding?
however,
1. if we use "Text" instead of "Mtom", no whitespace removed at service side.
2. if we remove
<security mode="None" />
and keep "Mtom", also no whitespace removed at service side.
client side ---
string customerName = " before and after ";
double totalAmount = customerAccountService.AddBalance(customerName , 100);
service side ---
the customerName becomes "before and after "
Microsoft has confirmed this as a bug:
http://connect.microsoft.com/wcf/feedback/details/619894/strings-stripped-of-leading-whitespace-when-using-mtom-message-encoding
I get an exception when there are too many objects returned:
The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://.../Contract:GetBlaBlaResult. The InnerException message was 'Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota. '. Please see InnerException for more details.
I’ve looked it up and added under behaviors in the server side:
<behaviors>
<serviceBehaviors>
<behavior name="MyServiceBehavior">
<dataContractSerializer maxItemsInObjectGraph="2147483646"/>
...
And in the client side:
<behaviors>
<endpointBehaviors>
<behavior name="maxItems">
<dataContractSerializer maxItemsInObjectGraph="2147483646" />
</behavior>
</endpointBehaviors>
</behaviors>
But it seems that it is ignored and the exception keeps arising.
The thing is I use XmlSerializer (for attributes in the elements):
[ServiceContract(Namespace = "http://BlaBla.com/webservices/BlaBlaService")]
[XmlSerializerFormat(SupportFaults = true)]
public interface IBlaBlaServices
{
[OperationContract]
BlaBlaResponse GetBlaBla(BlaBlaRequestMessage searchBlaBlaRequest);
}
and the exception, as you can see, refers to the dataContractSerializer (and so is the configuration I’ve added above).
Is it the XmlSerializer that mess up the things?
Can somebody advise please?
Thanks a lot :)
Just to make sure, are you applying the MyServiceBehavior behavior to the service definition in the service config and the maxItems behavior to the endpoint definition in the client config (the behaviorConfiguration attribute)?
This one has me puzzled and am hoping it's a silly oversight on my part. When I call the following web service, if a null parameter is encountered then all following parameters are null (or zero for the numerics).
[OperationContract, ProtoBehavior]
[ServiceKnownType(typeof(ServiceFault))]
[FaultContract(typeof(ServiceFault))]
PersonProfileMessagePart Create(ObjectId person, ObjectId industry, int yearsInIndustry, ObjectId occupation, string occupationFreeText,
int yearsInOccupation, string maritalStatus, string educationLevel,
List<ChildMessagePart> children, FinancialProfileMessagePart financialProfile,
List<ObjectId> traits);
For example, if person and yearsInIndustry are present but not industry, all parameters including yearsInIndustry default to null/zero. Is there an additional attribute I need to apply?
{edit}
My bad. Marc, it is WCF with a custom binding.
<customBinding>
<binding name="ServiceFedBinding">
<security authenticationMode="SecureConversation">
<secureConversationBootstrap authenticationMode="IssuedToken">
<issuedTokenParameters tokenType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV1.1">
<issuer address="http://links-dev:91/security/authentication/securetokenservice.svc/" binding="wsHttpBinding">
<identity>
<dns value="LINKS"/>
</identity>
</issuer>
<issuerMetadata address="http://links-dev:91/security/authentication/securetokenservice.svc/mex"/>
</issuedTokenParameters>
</secureConversationBootstrap>
</security>
Thanks in advance.
Mike
my WCF service it's used by a Silverlight application to retrieve data. I've no problem,
[OperationContract]
MyCollectionClass GetList(int sessID, string name);
[CollectionDataContract]
public class MyCollectionClass : List<MyClass>{ }
[DataContract]
public class MyClass {
[DataMember]
public string Prop1 { get; set; }
[DataMember]
public string Prop2 { get; set; }
}
But.. when MyCollectionClass have a less then 3000+ "record" it works. When the number of records is greater the WCF service seems to work, but on the completed event of the Silverlight app an exception occurs: "Service Not Found".
I've found that could be related to service configuration and i've tryied to use both:
maxBufferSize="2147483647"
maxReceivedMessageSize="2147483647"
on WCF and Client configuration. Also added:
readerQuotas:
maxArrayLength="2000000"
maxStringContentLength="2000000"/>
(also changed the values found) But seems to not working.
I think that the problem is that the message exceed the max number of byte per "message", but I do not understand why data is not spanned on different message. Any tips is appreciated.
Giorgio
I had the same problem, in my case was just serializing an string and no prob there, BUT in your case you are serializing a big bunch of objects, there's a default limit for that, I remember I saw a post about that (just a setting in the config --> maxItemsInObjectGraph) to high up that number of serialized objects,
Links
http://silverlight.net/forums/t/17674.aspx
http://forums.asp.net/t/1330713.aspx
Settings:
HTH
Braulio
Try to enable wcf service logging on server side. This might help: http://msdn.microsoft.com/en-us/library/ms730064.aspx
I use these 2 statement right after InitializeComponent:
binding.MaxReceivedMessageSize = 5000000
binding.MaxBufferSize = 5000000
You can change the numbers to what you want but I had to do this in order to recieve a large amount of data on the Silverlgiht client. My binding object is defines as:
Private binding As New BasicHttpBinding
This is in vb.net. Works like a charm after I include these items.
I would take a look at http://smehrozalam.wordpress.com/2009/01/29/retrieving-huge-amount-of-data-from-wcf-service-in-silverlight-application/.
I use silverlight 4 and vs 2010, and I had the same problem, and I resolved modifing the web.config file.
My original web.config file had:
<system.serviceModel>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
</system.serviceModel>
And I changed it by:
<system.serviceModel>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
<behaviors>
<serviceBehaviors>
<behavior>
<serviceMetadata httpGetEnabled="true"/>
<serviceDebug includeExceptionDetailInFaults="true"/>
<dataContractSerializer maxItemsInObjectGraph="2147483647"/> <!--this very is important: it is the size of the buffer-->
</behavior>
</serviceBehaviors>
</behaviors>
</system.serviceModel>
Could I suggest that you decrease the number of records you are returning? Not as a work around but as a usability suggestion. I cannot imagine any user coping with being shown 3000+ records. If you will be aggregating values from the data set rather aggregate them server side, it will boost your app's performance greatly... I have encountered this a couple of times in my apps, and almost always it worked out better to change the design than the options...