This one has me puzzled and am hoping it's a silly oversight on my part. When I call the following web service, if a null parameter is encountered then all following parameters are null (or zero for the numerics).
[OperationContract, ProtoBehavior]
[ServiceKnownType(typeof(ServiceFault))]
[FaultContract(typeof(ServiceFault))]
PersonProfileMessagePart Create(ObjectId person, ObjectId industry, int yearsInIndustry, ObjectId occupation, string occupationFreeText,
int yearsInOccupation, string maritalStatus, string educationLevel,
List<ChildMessagePart> children, FinancialProfileMessagePart financialProfile,
List<ObjectId> traits);
For example, if person and yearsInIndustry are present but not industry, all parameters including yearsInIndustry default to null/zero. Is there an additional attribute I need to apply?
{edit}
My bad. Marc, it is WCF with a custom binding.
<customBinding>
<binding name="ServiceFedBinding">
<security authenticationMode="SecureConversation">
<secureConversationBootstrap authenticationMode="IssuedToken">
<issuedTokenParameters tokenType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV1.1">
<issuer address="http://links-dev:91/security/authentication/securetokenservice.svc/" binding="wsHttpBinding">
<identity>
<dns value="LINKS"/>
</identity>
</issuer>
<issuerMetadata address="http://links-dev:91/security/authentication/securetokenservice.svc/mex"/>
</issuedTokenParameters>
</secureConversationBootstrap>
</security>
Thanks in advance.
Mike
Related
In my streamed WCF application I get System.OutOfMemoryExceptions in my service for large messages (querying >7GB MSSQL tables using a datareader, messages might even exceed 7GB) while small messages work just fine. I can observe the memory usage grow constantly during the execution of DataReaderToExcelXml (see below). The weird thing is, that it usually grows fast to 2GB, stays at 2GB +-1GB for a while (1-5 minutes) and then again raises very fast to ~6.5GB which leads to the exception (machine has 8GB memory). At this point it looks to me, as if the Stream isn't passed through anymore but buffered.
I already enabled trace logging, but it seems to stop with the exception. The DataReaderToExcelXml function call is the last visible event in the trace log.
In the WCF message contract I ensured that the message only contains the Stream. On client side, the returned Stream is simply read, written to a filestream and disposed. However, I can never observe the client code being executed or the file being written when I get the exception.
I already tried setting the maxBufferPoolSize to zero, both on the client and server side, as described here https://stackoverflow.com/a/595871/4166885. No success.
Stream writer function on WCF-service side:
Public Shared Function DataReaderToExcelXml(ByRef dr As SqlDataReader) As Stream
Dim ms As New MemoryStream
Dim tw As New IO.StreamWriter(ms)
For Each row As DbDataRecord In dr
'Embed row in ExcelXml, detailed function omitted
tw.write(row.toString()) 'row.toString is just a simplification
End While
tw.Flush()
dr.Close()
ms.Seek(0, SeekOrigin.Begin)
Return ms
End Function
Web.config bindings
<bindings>
<basicHttpBinding>
<binding receiveTimeout="24.00:00:00" sendTimeout="24.00:00:00"
maxBufferPoolSize="9223372036854775807" maxReceivedMessageSize="9223372036854775807"
messageEncoding="Mtom" transferMode="Streamed" bypassProxyOnLocal="True">
<readerQuotas maxStringContentLength="2147483647" maxArrayLength="2147483647"
maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="Transport">
<transport clientCredentialType="None" />
</security>
</binding>
</basicHttpBinding>
</bindings>
App.config bindings
<binding name="BasicHttpBinding_IFileService" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="24.00:00:00" sendTimeout="24.00:00:00"
allowCookies="false" bypassProxyOnLocal="true" hostNameComparisonMode="StrongWildcard"
maxBufferSize="2147483647" maxBufferPoolSize="2147483647"
maxReceivedMessageSize="8589934592" messageEncoding="Mtom"
textEncoding="utf-8" transferMode="Streamed">
<readerQuotas maxDepth="32" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="4096" maxNameTableCharCount="2147483647" />
<security mode="Transport">
<transport clientCredentialType="None" proxyCredentialType="None"
realm="" />
<message clientCredentialType="UserName" algorithmSuite="Default" />
</security>
</binding>
edit: Using .NET memory profiler on my service I found out, that the increasing amount of memory can be traced back to the type byte[]. Personally, that's not telling anything to me, but maybe it's useful information.
edit2: While inspecting the System.OutOfMemoryException I just noticed that it occurs when ms.Capacity = ms.length = 1073741824 = 1GB. So just before the memory stream would double its capacity. I'm still not sure, why w3wp consumes such large amounts of memory in the first place, but now it's clear that the memory stream triggers the Exception.
Well, reading your question, I think that the source of your error, is the key. Where is exactly the error? In transport, or in your server?
If the Exception is raised in DataReaderToExcel, maybe you could consider another ideas (I think it isn't a WCF issue):
By now, I think you would have the same problem in a Windows forms app.
You need memory, so you can fill it only with your SqlDataReader, you have to query by blocks, based on ranges.
We will start computing the "TotalRecords":
SELECT COUNT(*)
FROM YOUR_TABLE
WHERE WATHEVER_YOU_WANT
For example, querying blocks of 10.000 records, you have TotalRecords/10.000 pages (querys) +1. (Example: 30.001 rows, 10.000 row per block = 4 queries )
Iterate building them so:
select top **10.000** * from
(SELECT TOP (100) PERCENT ROW_NUMBER() OVER (ORDER BY YOUR_TABLE_FIELD_1 DESC) ROW_PAGINATED, YOUR_TABLE_FIELD_1, YOUR_TABLE_FIELD_2 , ... , YOUR_TABLE_FIELD_N
FROM YOUR_TABLE
WHERE WATHEVER_YOU_WANT
ORDER BY YOUR_TABLE_FIELD_1 DESC
) YOUR_ALIAS
WHERE YOUR_ALIAS.ROW_PAGINATED BETWEEN min_records_per_page AND max_records_per_page
)
Where min_records_per_page AND max_records_per_page have this values:
Query 1:
min_records_per_page= 1
max_records_per_page = 10000
Query 2:
min_records_per_page= 10001
max_records_per_page = 20000
...
Query N:
min_records_per_page= (N-1)* +1
max_records_per_page = TotalRecords
On each iteration, you will map each datarow into a ExcelXml class.
Doing so, you will avoid to consume all the memory, assuming you will be Disposing the objects you use. You can use the Function SetProcessWorkingSetSize each 2 iterations, for example.
To free memory after a significant number of processed rows:
Private Declare Auto Function SetProcessWorkingSetSize Lib "kernel32.dll" (ByVal procHandle As IntPtr, ByVal min As Int32, ByVal max As Int32) As Boolean
Dim Mem As Process
Mem = Process.GetCurrentProcess()
SetProcessWorkingSetSize(Mem.Handle, -1, -1)
Then, now you have an array with mapped objects. Part 1 finished.
Part 2. Your client WCF may receive this data. Tell us then if you have problems once the data to be sent are ready. Then we can explore communication parameters, and then we would say "increase that parameter, or add this one".
Hope it helps
The formatter threw an exception while trying to deserialize the message: Error in deserializing body of request message for operation 'PostTransaction'. The maximum string content length quota (8192) has been exceeded while reading XML data. This quota may be increased by changing the MaxStringContentLength property on the XmlDictionaryReaderQuotas object used when creating the XML reader. Line 52
You need add this argument to your config, like
<bindings> <webHttpBinding> <binding> <readerQuotas maxStringContentLength="65535"/> </binding> </webHttpBinding> </bindings>
Exaple from this answer: Fixing maximum length quota on XmlDictionaryReaderQuotas for WCF 4.0 REST
when try a simple wcf service call at client, the string data member received at service side has its whitespaces at the beginning removed if we using binding as below.
binding:
<binding name="WSHttpBinding_CustomerService"
messageEncoding="Mtom" >
<security mode="None" />
</binding>
the question is: why is the front whitespace removed at service side when using this binding?
however,
1. if we use "Text" instead of "Mtom", no whitespace removed at service side.
2. if we remove
<security mode="None" />
and keep "Mtom", also no whitespace removed at service side.
client side ---
string customerName = " before and after ";
double totalAmount = customerAccountService.AddBalance(customerName , 100);
service side ---
the customerName becomes "before and after "
Microsoft has confirmed this as a bug:
http://connect.microsoft.com/wcf/feedback/details/619894/strings-stripped-of-leading-whitespace-when-using-mtom-message-encoding
I have a very strange situation. I have a large set of records to return as a List from a WCF service. If I return the set as a DataTable, everything works fine. There are about 19,000 records in the set. If I return the set as a List (where T is a DataContract) it errors out and closes connection upon returning any set longer than 10922 records. I would think it was a problem with my data except another person has reported the exact same problem with a limit of 10922 records. Has anyone else encountered this problem, and if so how did you solve it?
We encountered the same problem.
From the service trace log we could retrieve the following exception:
Error while trying to serialize parameter []. Maximum numberError while trying to serialize parameter []. Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota.
After changing the dataContractSerializer key with parameter maxItemsInObjectGraph everything was running smoothly even with millions of records (on condition that you changed the maxReceivedMessageSize accordingly).
These changes have to be made in the web.config and the app.config in the following way:
web.config:
<behaviors>
<serviceBehaviors>
<behavior name="WasteWatcher.TestService.ServiceImplementation.TestService_Behavior">
<serviceDebug includeExceptionDetailInFaults="true" />
<serviceMetadata httpGetEnabled="true" />
<dataContractSerializer maxItemsInObjectGraph="2147483647"/>
</behavior>
</serviceBehaviors>
</behaviors>
app.config:
<behaviors>
<endpointBehaviors>
<behavior name="SerializerBehavior">
<dataContractSerializer maxItemsInObjectGraph="2147483647" />
</behavior>
</endpointBehaviors>
</behaviors>
Don't forget to add the parameter behaviorConfiguration="SerializerBehavior" to the endpoint key:
<endpoint address="http://localhost:9542/TestService.Host/TestService.svc"
binding="customBinding" bindingConfiguration="DefaultEndpoint"
contract="WasteWatcher.TestService.Test.Client.TestServiceProxy.TestServiceContract"
name="DefaultEndpoint" behaviorConfiguration="SerializerBehavior">
<identity>
<dns value="localhost" />
</identity>
</endpoint>
Best regards
Markus Rohlof
Check your endpoints' maxReceivedMessageSize on both client and server.
We actually have this problem at work. We try to send a lot of data through a WCF web service. We get cutout at something like 20,000 records so we ended up breaking up the data and doing a number of web service calls. Can you do something similar? Break up the records into smaller chunks and then merge them on the other end?
my WCF service it's used by a Silverlight application to retrieve data. I've no problem,
[OperationContract]
MyCollectionClass GetList(int sessID, string name);
[CollectionDataContract]
public class MyCollectionClass : List<MyClass>{ }
[DataContract]
public class MyClass {
[DataMember]
public string Prop1 { get; set; }
[DataMember]
public string Prop2 { get; set; }
}
But.. when MyCollectionClass have a less then 3000+ "record" it works. When the number of records is greater the WCF service seems to work, but on the completed event of the Silverlight app an exception occurs: "Service Not Found".
I've found that could be related to service configuration and i've tryied to use both:
maxBufferSize="2147483647"
maxReceivedMessageSize="2147483647"
on WCF and Client configuration. Also added:
readerQuotas:
maxArrayLength="2000000"
maxStringContentLength="2000000"/>
(also changed the values found) But seems to not working.
I think that the problem is that the message exceed the max number of byte per "message", but I do not understand why data is not spanned on different message. Any tips is appreciated.
Giorgio
I had the same problem, in my case was just serializing an string and no prob there, BUT in your case you are serializing a big bunch of objects, there's a default limit for that, I remember I saw a post about that (just a setting in the config --> maxItemsInObjectGraph) to high up that number of serialized objects,
Links
http://silverlight.net/forums/t/17674.aspx
http://forums.asp.net/t/1330713.aspx
Settings:
HTH
Braulio
Try to enable wcf service logging on server side. This might help: http://msdn.microsoft.com/en-us/library/ms730064.aspx
I use these 2 statement right after InitializeComponent:
binding.MaxReceivedMessageSize = 5000000
binding.MaxBufferSize = 5000000
You can change the numbers to what you want but I had to do this in order to recieve a large amount of data on the Silverlgiht client. My binding object is defines as:
Private binding As New BasicHttpBinding
This is in vb.net. Works like a charm after I include these items.
I would take a look at http://smehrozalam.wordpress.com/2009/01/29/retrieving-huge-amount-of-data-from-wcf-service-in-silverlight-application/.
I use silverlight 4 and vs 2010, and I had the same problem, and I resolved modifing the web.config file.
My original web.config file had:
<system.serviceModel>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
</system.serviceModel>
And I changed it by:
<system.serviceModel>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
<behaviors>
<serviceBehaviors>
<behavior>
<serviceMetadata httpGetEnabled="true"/>
<serviceDebug includeExceptionDetailInFaults="true"/>
<dataContractSerializer maxItemsInObjectGraph="2147483647"/> <!--this very is important: it is the size of the buffer-->
</behavior>
</serviceBehaviors>
</behaviors>
</system.serviceModel>
Could I suggest that you decrease the number of records you are returning? Not as a work around but as a usability suggestion. I cannot imagine any user coping with being shown 3000+ records. If you will be aggregating values from the data set rather aggregate them server side, it will boost your app's performance greatly... I have encountered this a couple of times in my apps, and almost always it worked out better to change the design than the options...