WCF tcp binding messages sizes - wcf

This subject is discussed a lot around here but no answer is really accurate.
i have a wcf service that i've created with nettcpbinding.
i wanted to examine the messages sizes/sent data rate/received data rate that are being transferred between the server and client.
The service is a duplex channel and the client code is auto-generated by adding a service reference.
Both applications (client/server) are written in C# .Net 4.0.
The nettcp binding that is using the default binary encoder and the configuration settings on the server side is as follows:
<bindings>
<netTcpBinding>
<binding name="largeMessage"
receiveTimeout="infinite"
sendTimeout="24:00:00"
openTimeout="00:00:10"
closeTimeout="00:00:10"
maxReceivedMessageSize="2147483647"
maxBufferSize="2147483647"
maxConnections="2000"
transactionFlow="false"
listenBacklog="2147483647"
maxBufferPoolSize="2147483647">
<readerQuotas maxDepth="128"
maxArrayLength="200000000"
maxStringContentLength="2147483647"/>
<security mode="None" />
<reliableSession enabled="false" ordered="true" inactivityTimeout="infinite" />
</binding>
</netTcpBinding>
</bindings>
The service is self hosted and is started as a console application by creating:
ServiceHost host = new ServiceHost(serviceObject, addresses);
i used wireshark to get a general statistics and the tcp packets that are transferred and i saw that i have a very high data rate.
So i wanted to examine each message size that is transferred between client and server.
For this i used several techniques for measuring messages sizes that are suggested in this forum but none of them give the exact same size as the wireshark does.
i already tested the following:
WCF - Measuring approximate message sizes programmatically
this links:
http://winterdom.com/2009/07/comparing-text-and-binary-serialization-in-wcf/
and did this in my code:
static byte[] SerializeToBin<T>(T obj)
{
Message msg = ObjectToMessage(obj);
MessageEncodingBindingElement mebe = new BinaryMessageEncodingBindingElement();
mebe.MessageVersion = MessageVersion.Soap12;
return Serialize(msg, mebe.CreateMessageEncoderFactory());
}
static byte[] Serialize(Message msg, MessageEncoderFactory factory)
{
MessageEncoder enc = factory.Encoder;
MemoryStream stream = new MemoryStream();
enc.WriteMessage(msg, stream);
return stream.ToArray();
}
static Message ObjectToMessage<T>(T obj)
{
DataContractSerializer ser = new DataContractSerializer(typeof(T));
return Message.CreateMessage(MessageVersion.Soap12, "", obj, ser);
}
it doesn't give accurate results.
Then i tried to add a message inspector and to listen to the Before/After - Send/Receive messages as suggested in this article
http://devlicio.us/blogs/derik_whittaker/archive/2011/02/03/how-to-intercept-a-wcf-message-to-track-message-size.aspx
and called this function:
private void DetermineAndLogMessageDiagnostics(ref Message message, eMessageDirection messageDirection)
{
long sizeInBytes = 0;
var mb = message.CreateBufferedCopy(int.MaxValue);
message = mb.CreateMessage();
var ms = new MemoryStream();
using (var memWriter = XmlDictionaryWriter.CreateBinaryWriter(ms))
{
mb.CreateMessage().WriteMessage(memWriter);
memWriter.Flush();
sizeInBytes = ms.Position;
}
//Recalculates the updated average and updates the variables
//sentBytes, averageSentbps, receivedBytes, averageReceivedbps
RecalculateAverage(bodySizeInBytes, messageDirection);
Logger.Info("Message '{0}' size: {1} bits / {2} bytes. ({3} KBits). Sent (Total: {4} Bytes, {5:0.00} bps). Received (Total: {6} Bytes, {7:0.00} bps)",
action,
sizeInBytes * 8,
sizeInBytes,
((double)sizeInBytes * 8) / 1024,
sentBytes,
averageSentbps,
receivedBytes,
averageReceivedbps
);
}
This also doesn't give accurate results.
When i say not accurate - i mean around 300 - 500 bytes differences. But the gap is not constant.
i didn't want to create a custom binding because i wanted to examine the netTcpBinding and didn't want to change any configuration.
Does anyone have an accurate solution for this matter?
Thank you for reading this long question description.
And thanks in advance for any suggested solution.

The only way where you'll get an accurate measurement of the message size is to use a custom message encoder. It can wrap the original encoder, and at that point (on the calls to WriteMessage and ReadMessage you can look at the message sizes.
You don't get accurate results by writing the message to a stream using the binary writer because the binary encoder used by the NetTcpBinding uses some dictionaries to further compress the message during transmission.

Related

How to change default message size using ClearUserNameBinding?

We are using ClearUserNameBindig in our WCF service.
When we tried to return a message with more than 3k records, we received this error:
The maximum message size quota for incoming messages (65536) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element.
We tried to modify web.config like that:
<bindings>
<clearUsernameBinding>
<binding name="myClearUsernameBinding"
maxReceivedMessageSize="20000000"
maxBufferSize="20000000"
maxBufferPoolSize="20000000" />
<readerQuotas maxDepth="32"
maxArrayLength="200000000"
maxStringContentLength="200000000"/>
</clearUsernameBinding>
</bindings>
But we received this error:
Unrecognized attribute 'maxReceivedMessageSize'.
How to change default message size using ClearUserNameBinding?
We found the solution following this steps:
http://sureshjakka.blogspot.com.ar/2010/03/changing-message-sizes-in.html
We modify the code of ClearUserNameBinding like this:
In AutoSecuredHttpTransportElement() constructor, initialize the values to maximum possible
public AutoSecuredHttpTransportElement()
{
MaxReceivedMessageSize = int.MaxValue;
MaxBufferSize = int.MaxValue;
MaxBufferPoolSize = long.MaxValue;
}
In CreateBindingElements() method create XMLDictionaryReaderQutotas object and set the same on TextMessageEncodingBindingElement. Here is the modified version of this method.
public override BindingElementCollection CreateBindingElements()
{
XmlDictionaryReaderQuotas rqMax = XmlDictionaryReaderQuotas.Max;
TextMessageEncodingBindingElement textBE = new TextMessageEncodingBindingElement();
textBE.MessageVersion = this.messageVersion;
rqMax.CopyTo(textBE.ReaderQuotas);
var res = new BindingElementCollection();
res.Add(textBE);
res.Add(SecurityBindingElement.CreateUserNameOverTransportBindingElement());
res.Add(new AutoSecuredHttpTransportElement());
return res;
}
Note: Make sure that you have "System.Runtime.Serialization" version 3.0.0.0 and above in your references. Because if you have version 2.0.0.0, you will get compile error as this version does not allow setting properties on ReaderQuotas.
Web.config:
<bindings>
<clearUsernameBinding>
<binding name="myClearUsernameBinding" />
</clearUsernameBinding>
</bindings>
Finally We update the references in server and client.

Deserializing an object graph from MSMQ that was enqueued using WCF and NetMsmqBinding

I've done some work using MSMQ with WCF and the NetMsmqBinding to generate messages and dequeue them as they arrive. I've also standardized my solution by using an object graph as the message body. This object contains meta data and an internal payload.
I'd like to construct an admin tool that can monitor queues and peek at the contents of messages. So far I've been unsuccessful figuring out how to deserialize the Message.Body back into the object graph using the System.Messaging libraries.
Any ideas?
Do you have any scope changing the WCF service bindings?
If you use MsmqIntegrationBinding rather than netMsmqBinding you have a range of formatter options you can specify in your binding. For example
<service name="MyQueueListenner">
<!-- Active X endpoint -->
<endpoint address="msmq.formatname:DIRECT=OS:.\private$\myQueue"
binding="msmqIntegrationBinding"
bindingConfiguration="ActiveXBinding"
contract="MyContract" />
<!-- .Net endpoint-->
<endpoint address="msmq.formatname:DIRECT=OS:.\private$\myOtherQueue"
binding="msmqIntegrationBinding"
bindingConfiguration="DotNetBinding"
contract="MyContract" />
</service>
...
<msmqIntegrationBinding>
<binding serializationFormat="ActiveX" name="ActiveXBinding" durable="false" exactlyOnce="false">
<security mode="None" />
</binding>
<binding serializationFormat="Xml" name="DotNetBinding" durable="false" exactlyOnce="false">
<security mode="None" />
</binding>
</msmqIntegrationBinding>
This allows you the full range of formatters providing the greatest range of interoperability with your System.Messaging based sniffer.
The full list of values is here:
http://msdn.microsoft.com/en-us/library/system.servicemodel.msmqintegration.msmqmessageserializationformat.aspx
i know this is old, but to serialize and back any object you can do the following:
//Sample
enter code here
public person class
{
public int Id {get;set;}
public string Name {get;set;}
public static person Desserialize(byte[] data)
{
person result = new person ();
using (MemoryStream m = new MemoryStream(data))
{
using (BinaryReader reader = new BinaryReader(m))
{
result.id = reader.ReadInt32();
result.Name = reader.ReadString();
}
}
return result;
}
public byte[] Serialize()
{
using (MemoryStream m = new MemoryStream())
{
using (BinaryWriter writer = new BinaryWriter(m))
{
writer.Write(id);
writer.Write(Name);
}
return m.ToArray();
}
}
// You can do
Byte[] w_byte = Person.serialize();
Person _Person = Person.Desiserile(w_byte);
}

How to use ServiceRoutes while defining maxReceivedMessageSize for non-custom bindings in WCF

Editing this to refocus on the actual issue. I've preserved the origional question at the bottom of the message but changing the title and content to reflect what was really happening.
I need to override the maxReceivedMessageSize for a WCF service added to an MVC3 project via the ServiceRoute mechanism. Specifing the binding in the web.config doesn't work. How does one do this.
Initial question is below this line but is misleading based on lots of false positives I was seeing.
Hi I have used some examples to add a file streaming upload service to my MVC3 project. If I use the default bindings (i.e., not defined in web.config) the service works as long as I don't exceed the 64k default size. When I try and define my own binding to increase the size I get a content-type mismatch in my trace and a HTTP415 Unsupported Media Type in the response. I'm trying to call this via fiddler via HTTP and am not using a WCF client.
Here is the error in the trace:
Content Type image/jpeg was sent to a service expecting multipart/related;type="application/xop+xml". The client and service bindings may be mismatched.
Here is the web.config service model section
<system.serviceModel>
<behaviors>
<endpointBehaviors>
<behavior name="NewBehavior0" />
</endpointBehaviors>
</behaviors>
<services>
<service name="AvyProViewer.FileService">
<endpoint address="UploadFile" binding="basicHttpBinding" bindingConfiguration=""
contract="AvyProViewer.FileService" />
</service>
</services>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true"/>
<bindings>
<basicHttpBinding>
<binding name="NewBinding0" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647"
messageEncoding="Mtom" transferMode="StreamedRequest">
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
</binding>
</basicHttpBinding>
</bindings>
Here is the service:
[ServiceContract]
[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
public class FileService
{
[OperationContract]
[WebInvoke(Method = "POST", UriTemplate = "UploadFile")]
public string UploadFile(Stream fileStream)
{
string path = HostingEnvironment.MapPath("~");
string fileName = Guid.NewGuid().ToString() + ".jpg";
FileStream fileToupload = new FileStream(path + "\\FileUpload\\" + fileName, FileMode.Create);
byte[] bytearray = new byte[10000];
int bytesRead, totalBytesRead = 0;
do
{
bytesRead = fileStream.Read(bytearray, 0, bytearray.Length);
totalBytesRead += bytesRead;
} while (bytesRead > 0);
fileToupload.Write(bytearray, 0, bytearray.Length);
fileToupload.Close();
fileToupload.Dispose();
return fileName;
}
}
And here is where I expose it in my MVC3 routes:
public static void RegisterRoutes(RouteCollection routes)
{
routes.IgnoreRoute("{resource}.axd/{*pathInfo}");
routes.Add(new ServiceRoute("FileService", new WebServiceHostFactory(), typeof(FileService)));
. . .
}
I think the issue is with the mtom declaration for messageEncoding in your binding. Try changing messageEncoding to Text.
Answer ended up being a combination of three different stack overflow posts. None by themselves solved the question but each provided crucial clues as to what was happing.
It seems that if you add a ServiceRoute the web.config binding information is ignored. This SO post clued me in to what seems to be undocumented behavior of this function: Unable to set maxReceivedMessageSize through web.config
I then used this post to determine how to programatically override the maxreceivedmesssagesize for the binding: Specifying a WCF binding when using ServiceRoute.
Unfortunately the code form #2 didn't work out of the box (not sure if the binding behavior for ServiceRoute has changed or what makes the difference). Turns out that if you specify a ServiceRoute its automatically created as a CustomBinding which can't be cast to the WebHTTPBinding type used in #2. So this post: How to set the MaxReceivedMessageSize programatically when using a WCF Client? helped me determine how to change the code in #2 to add this capability to a custom binding.

problem when sending images through wcf from silverlight

Silverlight uses WCF with basicHttpBinding
<basicHttpBinding>
<binding name="BasicHttpBinding_BugsService"
maxBufferSize="2147483647"
maxReceivedMessageSize="2147483647">
<security mode="None"/>
</binding>
</basicHttpBinding>
I send the image as an byte[], and it's working as long as the image has less than 20KB
but when it's bigger i get the error:
The remote server returned an error: NotFound
in the Reference.cs
public bool EndSave(System.IAsyncResult result)
{
object[] _args = new object[0];
bool _result = ((bool)(base.EndInvoke("Save", _args, result))); // error
return _result;
}
Please catch the exception on the wcf side in the interface implementation and post that.
The error may be related to this problem:
"WCF Message Size Issue"
Check the "maxReceivedMessageSize" property (representing the size in bytes) on the binding configuration. If your data exceeds that size, the message is discarded.

WCF - How to accept long strings as parameters

I have a simple web service, it takes 2 parameters one is a simple xml security token, the other is usually a long xml string. It works with short strings but longer strings give a 400 error message. maxMessageLength did nothing to allow for longer strings.
After the answer on quotas I just did all that in the web.config
<bindings>
<wsHttpBinding>
<binding name="WSHttpBinding_IPayroll" maxReceivedMessageSize="6553600">
<security mode="None"/>
<readerQuotas maxDepth="32"
maxStringContentLength="6553600"
maxArrayLength="16384"
maxBytesPerRead="4096"
maxNameTableCharCount="16384" />
</binding>
</wsHttpBinding>
</bindings>
You should remove the quotas limitations as well.
Here is how you can do it in code with Tcp binding.
I have added some code that shows removal of timeout problems because usually sending very big arguments causes timeout issues. So use the code wisely...
Of course, you can set these parameters in the config file as well.
NetTcpBinding binding = new NetTcpBinding(SecurityMode.None, true);
// Allow big arguments on messages. Allow ~500 MB message.
binding.MaxReceivedMessageSize = 500 * 1024 * 1024;
// Allow unlimited time to send/receive a message.
// It also prevents closing idle sessions.
// From MSDN: To prevent the service from aborting idle sessions prematurely increase the Receive timeout on the service endpoint's binding.’
binding.ReceiveTimeout = TimeSpan.MaxValue;
binding.SendTimeout = TimeSpan.MaxValue;
XmlDictionaryReaderQuotas quotas = new XmlDictionaryReaderQuotas();
// Remove quotas limitations
quotas.MaxArrayLength = int.MaxValue;
quotas.MaxBytesPerRead = int.MaxValue;
quotas.MaxDepth = int.MaxValue;
quotas.MaxNameTableCharCount = int.MaxValue;
quotas.MaxStringContentLength = int.MaxValue;
binding.ReaderQuotas = quotas;