I have an already running basicHttpBinding with transferMode=Buffered WCF service for sending small amount of data from client.
Now, as part of enhancement, need to upload multiple images with each image of size around 5-10 MB.
I am planning to implement streaming for uploading the large files (file by file).
My questions
1. Can the buffered and Stream TransportMode exists side by side?
2. Can I use the same configuration? (end-point, Binding, etc) for buffered and streamed
3. Do I need to have separate service contract and can it be part of the existing service contract?
4. Can I have sample working code?
5. I am open to adopt, if you have any other better suggestion.
You can have one contract with some operations that use streaming and some that don't.
However, Learning WCF book advises to have separate contract with streaming operations. This way you have separate endpoint with binding that has TransferMode=Streamed.
For sample working code - you only need to set
<basicHttpBinding>
<binding transferMode="StreamedRequest"/>
</basicHttpBinding>
in binding configuration and your contract operations must have only Stream as a parameter (or return type)
void Upload(Stream input)
Related
basically need to transfer large file between wcf service and java client,
can someone give directions please?
Basically I need to a create wcf service which needs to read blob content(actually a file content stored in db column) and pass it to a java web application(being a client to wcf).
File size may vary from 1kb to 20MB in size.
By now I have already researched/checked below options but still not able to finalize which one i should go with, which is feasible and which is not,
could someone guide me please.
pass file content as byte[]:
I understand it will increase data size passed to client as it will encode data into base 64 format and embed the base 64 encoding into soap message itself and hence makes communication slower and have performance issues.
But this works for sure, but I am not sure if it is advisable to go by this approach.
Share a NetworkDrive/FTPFolder accessible to both client and wcf service App:
Per this File needed by client will first get stored there by wcf and then client needs to use java I/O OR FTP options to read it.
This looks good from data size/bandwidth point of view, but has extra processing at both service and client side (as need to store/read via NetworkShared/FTP folder)
Streaming:
This one I am not sure will be feasible with a Java client, but my understanding is that streaming is supported for Non .net clients but how to go about it i am not sure???
I understand for streaming i need to use basichttp binding, but do i need to use DataContract or MessageContract or any will work, and then what is to be done at java client side, that i am not sure about.
Using MTOM approach for passing large data in soap requests:
This looks actually having support specifically designed to solve large data transfer in web service calls, but have to investigate further on this, as of now I don’t have much idea on this. Does anyone of you have some suggestion on this?
I understand question is bit lengthier but i had to put all 4 options i have tried and my concerns/findings with each, so that you all can suggest among these or new option may be, also you will know what i have already tried and so can direct me in more effective way.
I am in the same position as yourself and I can state from experience that option 1 is a poor choice for anything more than a couple of MB.
In my own system, times to upload increase exponentially, with 25MB files taking in excess of 30mins to upload.
I've run some timings and the bulk of this is in the transfer of the file from the .NET client to the Java Web Service. Our web service is a facade for a set of 3rd party services; using the built in client provided by the 3rd party (not viable in the business context) is significantly faster - less than 5mins for a 25MB file. Upload to our client application is also quick.
We have tried MTOM and, unless we implemented it incorrectly, didn't see huge improvements (under 10% speed increase).
Next port of call will be option 2 - file transfers are relatively quick so by uploading the file directly to one of the web service hosts I'm hoping this will speed things up dramatically - if I get some meaningful results I will add them to my post.
I need to streaming a large content with WCF and deferred execution (using NetTcpBinding), in other words, return a list of person (can be anything) from a database through a WCF service without consume all memory in the server side.
I have tried the solution in this post: Streaming large content with WCF and deferred execution
Using BasicHttpBinding it works like a charm, but when using NetTcpBinding....well...not working.
Can anyone help me with this??
Tks!
Project here: WCF Streaming IEnumerable
Here is guidance on transferring large data with streaming.
http://msdn.microsoft.com/en-us/library/ms733742.aspx
At the bottom of the page is a link to some samples, one of which shows using TCP.
http://msdn.microsoft.com/en-us/library/ms789010.aspx
If this doesn't help, then perhaps you could provide more detail on how it is failing (error message)?
I have added service reference to my web.config file but I am not sure about the transferMode property inside binding tag.
In the basicHttpBinding, which is the best transferMode for soap/xml response?
Basically there are four transfer mode. If you narrow down those to two, buffered and streamed, here is the criteria:
If you are transferring large files, mostly binary files, try using streamed. This method streams the data to the client instead of sending a big chunk of data. It helps your application to be more efficient in terms of memory consumption. Some of the advanced functionalities of WCF are not available with this transfer mode.
By default buffered is selected. This is suitable for normal messages with relatively small or medium size. The whole request or response will be buffered in memory and then flush to the client or server.
There is another method that needs a custom channel that send messages in multiple chunks.
Chunk Channel
I'm about to implement a FileService using WCF. It should be able to upload files by providing the filecontent itself and the filename. The current ServiceContract looks like the following
[ServiceContract]
public interface IFileService
{
[OperationContract]
[FaultContract(typeof(FaultException))]
byte[] LoadFile(string relativeFileNamePath);
[OperationContract]
[FaultContract(typeof(FaultException))]
void SaveFile(byte[] content, string relativeFileNamePath);
}
It works fine at the moment, but i want to be able to reduce the network payload of my application using this Fileservice. I need to provide many files as soon as the user openes a specific section of my application, but i might be able to cancel some of them as soon as the user navigates further through the application. As many of mine files are somewhere between 50 and 300 MB, it takes quite a few seconds to transfer the files (the application might run on very slow networks, it might take up a minute).
To clarify and to outline the difference to all those other WCF questions: The specific problem is that providing the data between client <-> server is the bottleneck, not the performance of the service itself. Is changing the interface to a streamed WCF service reasonable?
It is a good practice to use a stream if the file size is above a certain amount. At my work on the enterprise application we are writing, if it is bigger than 16kb then we stream it. If it is less than that, we buffer. Our file service is specially designed to handle this logic.
When you have the transfer mode of your service set to buffer, it will buffer on the client as well as on the service when you are transmitting your data. This means if you are sending a 300mb file, it will buffer all 300mb during the call on both ends before the call is complete. This will definitely create bottlenecks. For performance reasons, this should only be when you have small files that buffer quickly. Otherwise, a stream is the best way to go.
If the majority or all of your files are larger files I'd switch to using a stream.
Well, I know that in a duplex contract the service can send messages to the client, but I would like to know when that is really useful.
I have a common application that send request to the service to get data from the a database, insert data... etc. Also, I need to store files about 40MB in the database, so I need a good performance. For this reason, I would like to use the net.tcp binding with transfer mode streamed, but the problem is that a net.tcp duplex service can't use the streamed transfer mode.
So I think I have some options.
1.- study if I really need a duplex contract for this kind of application. Perhaps in a chat application, for example, it has more sense a duplex contract because the server perhaps need to notify to the client when a contact is connected... etc. But in a common client that access to a data base, is necessary a duple contract? what kind of operations would can need a duplex contract?
2.- Other option it's not to have a duplex contract, but implement a no duplex contract in the server and other single contract in the the client, so when a client connect to the service, the service receive the needed information to connect to the service of the client. But, is this a good way to avoid a duplex contract?
3.- Really for my application I need tcp instead of a duplex HTTP that allows a streamed transfer mode? What is the advantages of the tcp over the HTTP in terms of performance?
Thanks.
You need duplex if you want to implement callback pattern. Callback means that client does not know when some event happens in server.
If you do not know when event happens you have two options to implement:
Polling - send requests every X minutes to check if event happened. Server should either return event details (if it happened) or return flag saying that you need to continue calling. Server also can return recommended timeout in advanced scenarios.
Callback - client sends some form of description what server should do if event happened. This may be pointer to function in C, delegate in .NET or endpoint schema in WCF. Server remembers that info and makes call from their side when time came.
As you can see duplex/callback means that at some point server works as client (initiates communication) and this is a big game change.
WCF duplex communications may require special network configuration because in many cases network allows you to call external services (you work as client) but forbids external resources to call you (external service works as client). This is implemented for security purposes.
Returning to your questions:
You do not need duplex if you only need to download big amount of data. You may need it if you want to catch updates that happened in server and notify clients. Duplex should work for Chat because in chat there are many scenarios when client needs to be notified with changes introduced by others.
What you described is hand-made variant of duplex channel. You should use proved and tested duplex implementation made by MS If you want server to call your method. Otherwise your option is polling.
You are right, you need tcp + streamed transfer mode to deal with big amount of data. TCP uses binary serialization which is more compact comparing to text serialization + with TCP you do not need to send any HTTP headers or SOAP envelops. Disable security if you do not need it. It has a big performance impact.
Addressing each point:
1, 2. I think that for your scenario a duplex service is an overkill. As you say yourself a duplex service is usually handy when both the client and service need to keep notifying each other on a constant basis, what you're doing, getting lots of data in/out of a database doesn't seem to be a good case for using duplex communication. Regarding netTcpBinding not allowing Streaming with duplex, you can just return a byte array (byte[]) instead of a stream. 40 MB is a lot, but I don't think Streaming will necessarily have a significant performance gain over a duplex service which will return a byte array (up to you to test each setup and compare the results). So you have a few options here, don't stream and return a byte array (you can do this with your duplex service) or you can just forget about making your service duplex since there doesn't seem to be a strong case for you to make it duplex and just return a Stream:
[OperationContract]
Stream RetrieveFile(long _fileId);
[OperationContract]
long SaveFile(Stream _stream);
3. netTcpBinding has a considerable performance advantage over HTTP bindings, but it comes with a price, mostly because its TCP ports are sometimes blocked by internet firewalls, although you can use netTcpBinding over the internet, it's not recommended. Choosing a binding depends on what you're looking to do, if your clients are going to consume your service over the internet, then netTcpBinding is not a good idea (blocked TCP ports, firewalls etc.), but if your clients are consuming the service in the same network (LAN) then netTcpBinding is the most sensible choice. wsDualHttpBinding (doesn't support streaming :#) is a good choice if you want to stick to a duplex service (equivalent of PollingDuplexHttpBinding in Silverlight), or any other HTTP based bindings if you let go of the idea of a duplex service.
Some articles that may help you out, performance comparison of various WCF bindings:
http://blog.shutupandcode.net/?p=1085
http://tomasz.janczuk.org/2010/03/comparison-of-http-polling-duplex-and.html
And about Streaming large data with WCF over HTTP, according to the authors, both samples have been tested with up to 2GB of data:
http://garfoot.com/blog/2008/06/transferring-large-files-using-wcf/
http://www.codeproject.com/Articles/166763/WCF-Streaming-Upload-Download-Files-Over-HTTP
You shouldn't think that you must use netTcpBinding or you must use Streamed transfer for your service, netTcpBinding only becomes more performant than HTTP bindings after you enable throttling and configure some socket level properties. And streaming 40 MB will not have significant performance gains over buffered transfer. So you have a lot of options and a lot of trade-offs to make. There is no black and white and right or wrong, it's about how you customise your service to suit your needs best, most solutions will work. Your scenrio is a very common one and there are lots of stuff online about large data transfer in WCF, do more research ;)