How to handle files in an SOA architecture - file-upload

I have an enterprise service bus with a collection of services. There is an HTML5 user interface that allows customers to perform all sorts of selfcare actions. The UI uses JavaScripts that communicate with the ESB in JSON format through an API server.
Now I want to offer the customer the possibility to upload documents. I can base64 the document and send it as a JSON field, but that doesn't seem right. What is a good pattern I could use?

It does not sound right because it is not right.
In this cases you should a write the file to a write only public FTP given it a GUID name and then send a message to the service giving details about the files. On the service side once a message with the file details is received, store the metadata in your persistence layer and copy the files to a private FTP.
Something like this:
Claim Check Pattern

Related

Using WCF Message

I am still confused when it is appropriate to use the Message type in WCF like below
[ServiceContract]
public interface IMyService
{
[OperationContract]
Message GetData();
[OperationContract]
void PutData(Message m);
}
Why would you want to use it?
Can you use it for streaming?
Thanks
MSDN lists the follow reasons for using the message class directly:
When you need an alternative way of creating outgoing message contents (for example, creating a message directly from a file on disk) instead of serializing .NET Framework objects.
When you need an alternative way of using incoming message contents (for example, when you want to apply an XSLT transformation to the raw XML contents) instead of deserializing into .NET Framework objects.
When you need to deal with messages in a general way regardless of message contents (for example, when routing or forwarding messages when building a router, load-balancer, or a publish-subscribe system).
See Using the Message Class for more detailed information.
Edit to address the streaming question
I didn't find a definitive answer in my quick scan via google, but the article above states "All communication between clients and services ultimately results in Message instances being sent and received" - so I would assume it could be used directly in streaming.
While the reasons listed by Tim are valid, we use messages directly in our services to create one uber routing service.
we have one service that can take any method call you throw at it, Clients are generated by wsdls supplied from multiple sources.
This service would take the message, examine its content and route it accordingly.
So in my opinion if you want to get closer to the wire, or when you dont know the type of incoming messages, you can use the message in the signature directly.
Streaming is a separate concept than message signature, streaming is supported by wcf under very specific bindings and security mechanism and the method signature has to be very specific (i.e it should return/accept stream). Also in streaming the actual stream of data travels outside the scope of soap message.

Transfer XML between systems with Biztalk and WCF

I have system A which produces an xml-file and system B which takes the file. How can I implement this exchange using Biztalk with WCF?
You have two possible different solutions as I see it and it basically comes down to how typed you WCF service needs to be.
Do you need to transform you message into a new format? Or are you planning to use other features in BizTalk as content based routing etc? Are you at all interested in the content of the message while in BizTalk are you OK with just passing it thru?
If you are looking for a solution with the capabilities as described above you'll need to get the schema for the XML message you want to receive into WCF service and publish that service. Once the message is in BizTalk it's then typed and you can do what ever with it using BizTalk.
If you however just want to pass it thru you could just publish a service that received as message of type XML document and pass that thru. Here's a good post describing the a few different techniques to create a generic service accepting any XML as input.

WCF service authentication method

I'm building a WCF SOAP service at the moment. I will, of course, need some authentication on the service.
Reading this very helpful blog post says that to use the built-in authentication points requires that the endpoint use the wsHttp binding.
This would be fine if I could guarantee that users would be communicating with the service through a client based on the meta-data exposed by WCF (basically, something like a client written in C# with a web service reference). However, I can't guarantee this!
I will still need to allow users to communicate with just raw (unencrypted) XML.
So, questions:
Does the wsHttp binding still allow for raw XML input?
If not, would I be wiser to
Implement two separate authetication points? One for raw XML input and one for encrypted input
Or
Allow input from wsHttp to fall back on some in-method validation that would be shared with the raw XML input?
Is it wise to allow users to pass their credentials inside a raw XML request?
EDIT: It sounds like I miscommunicated or misunderstood something in my original post, so here I will clarify what I mean by "raw XML".
By raw XML, I mean just the SOAP packet and the accompanying HTTP headers - as I might send from soapUI or Fiddler. As I understand it, messages over the wsHttp binding are encrypted when a client is generated from the WSDL (for example, in C#).
If this is not the case, then how would I go about attaching the same sorts of credentials to a raw XML (for want of a better term) request as I do a request run through a client? Are they attached as HTTP headers? XML elements in the SOAP envelope?
wsHttp is a SOAP binding, which means that your content gets wrapped in a SOAP envelope, possibly with headers relating to the message and various WS-* specifications being used.
I would ask why you need to support raw XML? Most platforms today support SOAP messaging and the whole idea of SOAP is to provide interoperability between different platforms. On most platforms it is as easy to develop a SOAP client as a raw XML client. In most cases, it is simply a case of taking the WSDL and generating a client. If you want to use standard facilities like authentication and message encryption then this is a much better way to go.
There are currently no hooks to do interoperable authentication for raw XML. You will have to come up with your own mechanism to do this and it will be non-standard. For your web service users, this means it will be probably entail more development effort than if you just went with SOAP.

RESTful windows phone

How do I transfer an object in RESTful web service? it seems like RESTful only supports string to exchange data between client and service. thanks
All web services only support the transfer of a representation of objects. Given the right client, you can easily generate objects with the information passed. For example, using JSON and a javascript client with jQuery, you can easily call jQuery.parseJSON(stringFromRESTServer); (or any number of methods in other good js libraries) to get a js object.

Best practice for WCF services used both locally and remotely that process large files on the filesystem?

I'm creating a WCF service that may be used either locally or remotely, and processes files sometimes using third-party components applications that unfortunately require as input a path to actual file on the filesystem, not a .net Stream or anything like that. Is there a standard approach for this situation, in terms of what the parameters to contract operations should be etc.? Although I suppose this can't be vital since it ultimately has to perform acceptably in both the local and remote cases, I'd prefer if, in the local case, it didn't have to read the whole file from the filesystem, include the contents in the message, and rematerialize it again on the filesystem, but for remote use this is necessary. Is there a way to do this e.g. by having an FSRefDoc type which serializes differently depending on whether it's used locally or remotely?
edit: To clarify: The problem is that I want to send different pieces of information entirely in the two cases. If I'm controlling a local service, I can just send a path to the file on the local filesystem, but if it's a remote service, I have to send the file contents themselves. Of course I can send the contents in both cases, but that means I lose performance in the local case. Maybe I shouldn't be worried about this.
OK,
Following your update, I would consider the following.
1) Create a method that takes a path. Expose this via a named pipe binding and use this locally.
2) Create a method that takes a file (stream/byte array etc). Expose this using an appropriate binding (on a different end point) for non local computers (in a LAN scenario TCP is usually your best bet).
Then all you need to do is make sure you don't duplicate the same business logic. So in a nutshell- create 2 different service interfaces, 2 different end points and 2 different bindings.
Well, you really touch on two separate issues:
local vs. remote service availability
"normal" vs. streamed service (for large files)
In general, if your service works behind a corporate firewall on a LAN, you should use the NetTcpBinding since it's the fastest and most efficient. It's fast and efficient because it uses binary message encoding (vs. text message encoding over the internet).
If you must provide a service for the "outside" world, you should try to use a binding that's as interoperable as possible, and here your choices are basicHttpBinding (totally interoperable - "old" SOAP 1.1 protocols) which cannot be secured too much, and wsHttpBinding which offers a lot more flexibility and options, but is less widely supported.
Since you can easily create a single service with three endpoints, you can really create your service and then define these three endpoints: one for local clients using NetTcpBinding, one of the widest availability using basicHttpBinding, and optionally another one with wsHttpBinding.
That's one side of the story.
The other is: for your "normal" service calls, exchanging a few items of information (up to a few KB in size), you should use the normal default behavior of "buffered transfer" - the message is prepared completely in a buffer and sent as a whole.
However, for handling large files, you're better off using a streaming transfer mode - either "StreamedResponse" if you want clients to be able to download files from your server, or "StreamedRequest" if you want clients to be able to uplaod files, or just plain "Streamed" if you send files both ways.
So besides the three "regular" endpoints, you should have at least another endpoint for each binding that handles streaming exchange of data, i.e. upload/download of files.
This may seems like a lot of different endpoints - but that's really not a problem, your clients can connect to whatever endpoint(s) are appropriate for them - regular vs. streamed and internal/local (netTcpBinding) vs. external (basicHttpBinding) as they need - and in the end, you write the code only once!
Ah , the beauty of WCF! :-)
Marc
UPDATE:
OK, after your comment, this is what I would do:
create a ILocalService service contract with a single method GetFile that returns a path and file name
create an implementation for the service contract
host that service on an endpoint with netTcpBinding (since it's internal, local)
[ServiceContract]
interface ILocalService
{
[OperationContract]
string GetFile(......(whatever parameters you need here).....);
}
class LocalService : ILocalService
{
string GetFile(......(whatever parameters you need here).....)
{
// do stuff.....
return fileName;
}
}
and secondly:
create a second service contract IRemoteService with a single method GetFile which doesn't return a file name as string, but instead returns a stream
create an implementation for the service contract
host that service on an endpoint with basicHttpBinding for internet use
make sure to have transferMode="StreamedResponse" in your binding configuration, to enable streaming back the file
[ServiceContract]
interface IRemoteService
{
[OperationContract]
Stream GetFile(......(whatever parameters you need here).....);
}
class RemoteService : IRemoteService
{
Stream GetFile(......(whatever parameters you need here).....)
{
// do stuff.....
FileStream stream = new FileStream(....);
return stream;
}
}