As far as I know the limit on maxReceivedMessageSize property is limited by max int size which is 2147483647 bytes (2gb) I'm hoping this information is not correct as I'm typing it .
In my application I need to send files over this size limit, I need to be able to send more than 4-5 gb files. Is there any workaround for that? or should I follow another approach? I am really stuck here.
Even if my maxReceivedMessageSize limit is 2gb now my service throws out of memory exception at 1.5 gb both while uploading and downloading. Please help me out.
For sending large files through WCF the streaming transfer should be used. See http://msdn.microsoft.com/en-us/library/ms789010.aspx about how to make stream transfer
Related
I have a Duplex communication which sends Small as well as Large messages.
In very rare cases I have more data to be send, because of this reason I am in a confusion to change the TransportMode to Streamed.
So I set the buffer size to 1GB (or a Large Size).
maxBufferSize="1073741824"
Is this cause my Small message communication ?
As the setting says this is the maximum buffer size not the buffer size used for all requests. Although maxBufferSize is only used for defining the buffer size for headers when the message is streamed
I am connecting to the Tridion core service using a System.ServiceModel.WsHttpBinding. The service will only be used by authenticated users, and probably only by code which I control. I have to choose values for the following
MaxBufferPoolSize (default 524,288 bytes)
MaxReceivedMessageSize (default 65,536 bytes)
ReaderQuotas.MaxArrayLength (default 16384 bytes)
ReaderQuotas.MaxBytesPerRead (default 4096 bytes)
ReaderQuotas.MaxNameTableCharCount (default 16384 bytes)
ReaderQuotas.MaxStringContentLength (default 8192 bytes)
The code examples I have seen for using the core service invariably set at least some of these to values larger than the defaults, for example 4Mb. Is this because of known problems when other values, such as the defaults, are used?
MaxBufferPoolSize is there to allow you to prevent excessive garbage collections. Is this simply a matter of monitoring GCs and tuning based on that?
MaxReceivedMessageSize, MaxArrayLength and MaxBytesPerRead are there to defend against Dos attacks, so in my scenario, perhaps I can improve throughput by increasing these. Would a really large number help?
MaxNameTableCharCount seems to be there to prevent uncontrolled growth of something you might not want to grow uncontrolledly, so perhaps leaving the default would be a good thing.
The documentation on MaxStringContentLength doesn't specify what happens if you exceed the quota. Presumably ReadContentAsString will fail in some way, so perhaps this value should be large.
So - should I leave these values at their defaults? Will that cause me problems? Should I increase them to large values? Will that help with throughput etc., or is it more likely to cause other problems?
The general rule is to have these values as small as possible, just enough for you code to work. If you will take a look at default config that is shipped with CoreService.dll, it has some of the values increased.
For example, if you expect to get large XML lists (or search results) - you should increase MaxReceivedMessageSize. Keep in mind that you have control over the size of the list you will get using BaseColumns property of a filter.
If you prefer to use GetList, GetSystemWideList and GetSearchResults methods over their XML counterparts, you will probably have to increase ReaderQuotas.MaxArrayLength together with MaxReceivedMessageSize. But please note, that large arrays will be stored in memory.
I'm not sure you want to increase any of these values until you will hit the limit. WCF is quite good with pointing you to the parameter you have to adjust.
I'm afraid this is not really an answer to your questions... But, from my experience, I increased the values to more than the defaults suggested. I used 4MB as you already suggested. This was namely because I was experiencing error while communicating with the Core Service. They were related to the request/response sizes exceeding the allotted sizes.
Moreover, in the case of Core Service transactionality, I saw more of these exceptions. It seems that the sizes of request/responses increase quite a bit when using transactions. In my case, I was creating a batch of Components in one big transaction. If one Component failed to create, I would roll-back the whole transaction.
Hope this helps.
I've been experimenting with the Core Service recently and have seen an XmlReader exception occur when trying to open a large TBB (C# fragment) using the following code:
using(var client = new CoreService.CoreService2010Client())
{
var item = client.Read(tcmId,new ReadOptions());
//More code
}
System.Xml.XmlException: The maximum string content length quota (8192) has been exceeded while reading XML data. This quota may be
increased by changing the MaxStringContentLength property on the
XmlDictionaryReaderQuotas object used when creating the XML reader.
Line 1, position 9201.
As it says in the message, I had to up the ReaderQuotas.MaxStringContentLength to fix this. So if you're working with any Building Blocks that have content bigger than 8KB expect this error.
We have a WCF service that has a threshold of 30MB to send files over an http message, anything above that value gets transferred by file copy and the path sent back to the caller. Now we were requested to eliminate that file copy because customers complained it was too slow. So the decision was to remove any size limitation when sending binary content over WCF/HTTP.
My question is - how reliable is that? What type of issues will we encounter by pushing, say, a 2GB file over the wire in a single WCF message, if that is even possible?
Thanks!
If you set the MaxReceivedMessageSize in WCF to a high enough value on your WCF service, you can push a fairly large file through that service. The maximum is int64.MaxValue = 9,223,372,036,854,775,807, so you should be able to set a value to cover a 2GB message.
You might want to control the MaxBufferSize to ensure you're not trying to store too much into memory, and maybe consider switching to the more binary-efficient MTOM message encoding if you can. Note that the MaxReceivedMessageSize governs the size of the message after the binary file has been encoded, which means the original binary file size which can be sent over the service will be smaller than 2GB.
MSDN has a very nice article covering sending large amounts of data over WCF and what to look out for: Large Data and Streaming.
Edit: Turns out the max value allowed is actually Int64.MaxValue)
I'm getting the following communication exception for my wcf service making cal to another wcf service:
"The maximum message size quota for incoming messages (65536) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element."
I resolved this by increasing the size as below:
maxReceivedMessageSize="50000000"
But, here I want to know whether any side effects of increasing message size to such big level.
Yes - it might. The reason WCF keeps this limit low (64K) by default is this: imagine your server is busy responding to requests, say dozens or hundreds, and they all require the maximum message size.
Potentially, your server could have to allocate dozens or hundreds of message buffers at the same time - if you have 100 users and each requests 64K, that's 6.4 MByte - but if you have 200 users and each requests 5 MB - that's a gigabyte of RAM in the server - just for the message buffers, for one service.
So yes - putting a limit on the max message size does make sense and it helps manage your server's memory consumption (and thus performance). If you open it up too wide, an attacker might just do such an attack - flooding your server with bogus requests, each allocating as much memory as they can get, ultimately bringing your server down (Denial of Service attacks like that are quite common).
You need to increase the quota according to the web service. One side effect I can think of if you use very large values is that memory usage will increase but you can safely increase it to a suitable value without any adverse effects.
I have a wcf operation that sends byte array to client.
The maximum size of byte array I intend to send is 2mb. So I have set maxbuffersize and maxreceivedmessagesize to 2097152 (2 mb) on basichttpbinding with transfermode=buffered on the server.
Despite these settings, no buffer overflow exception is getting thrown if I transfer a 17mb file?
Thanks.
UPDATE:
My understanding of the buffering in WCF and the effect of the various values was wrong. Please check out this MSDN thread and this related blog post on the ins and outs of WCF buffer management.
However, I still cannot find a definitive answer on how to limit the buffers on the server. From what I understand, if you limit maxBufferPoolSize (total for the pool of all buffers) and maxBufferSize (max. size for a single buffer) on the server side, you should be able to achieve what you're trying to do.
So in your case, you should set maxBufferSize to 2mb on the server, and maxBufferPoolSize to 2mb or more, also on the server. On the client, set the maxReceivedMessageSize also to 2mb.