I think I'm missing something conceptual and fundamental about WCF channels. I get how there's a channel stack of protocols on either side going from the top level TCP or HTTP down to the wire-level transport protocol.
What I don't get is what it means to "open" a channel and how that channel stays "open" and how that channel is "faulted".
What is happening on the client and service that makes a channel "open"? Open seems like a sate but it's hard for me to conceptualize state in a stateless service. Does that make sense?
Your service may be stateless, but many networking protocols are not.
From Understanding State Changes
State Machines and Channels
Objects that deal with communication, for
example sockets, usually present a state machine whose state
transitions relate to allocating network resources, making or
accepting connections, closing connections and terminating
communication. The channel state machine provides a uniform model of
the states of a communication object that abstracts the underlying
implementation of that object. The ICommunicationObject interface
provides a set of states, state transition methods and state
transition events. All channels, channel factories and channel
listeners implement the channel state machine.
The whole WCF Channel Model Overview is a good resource.
Related
what is the relation between those properties? which one of them govern the number of clients connect to net.tcp reliable service?
I tried to read on both of them but tit is not clear which control what in the throtlling of the service
Hope this is helpful.
MaxPendingChannels has to do with the number do number of clients that can connect to a service via reliable session.
When the sender creates a reliable session channel to a receiver, a handshake between them establishes a reliable session. After the reliable session is established, the channel is put in a pending channel queue for acceptance by the service. The MaxPendingChannels property indicates how many channels can be in this state.
MaxConnection behavior depends if it is set on the client or the server: On the client, it's a limit on the connections that are pooled, and on the server it's a limit on connections that haven't been accepted yet by the ServiceModel layer, ref
In my opinion, this property describes the same thing, the number of channels that clients can connect to at the same time. With one difference, the default Concurrencymode for the WCF service is concurrencymode.single, which limits the number of connections a customer can make. In this mode, maxconnection represents the maximum number of connections allowed to be pending dispatch on the server, and maxpendingchannels refers to the number of connections for a reliable session.
I am using On-Premise Service Bus 1.1 for communication between processes.
I need to perform request-response methods between end points and need to decide if I will use WCF or the bus (Service Bus Relay for WCF is not currently available for on premise).
WCF would be easiest to talk to via a generated client proxy, potential complexity with IIS host (or self host) and versioning of clients calling the service.
For Service Bus create two queues per remote service (i.e.
userService, userServiceResponse) and then use sessions. Flexible versioning with different commands. Management of these queues could become complex.
For my project everything is within the same subnet and if required WCF endpoints could talk directly to one another
To help me decide which technology to use, my questions are:
Where would WCF be used over request-response service bus?
Are there any libraries for Service Bus queues to implement
request-response messaging (or any robust code examples)?
If we have multiple publishers on a queue, how would we return a reply to a specific sender? Would we have multiple serviceReponse queues, or can a single return queue be used?
Service Bus messages can have a SessionID unique for that request where the service will receive the message, do something with it and reply with a message that has the same ID in the ReplyToSessionID. This allows the requesting party to receive based on the Session ID like this
MessageSession sessionReceiver = _queueClient.AcceptMessageSession(_mySessionID,TimeSpan.FromSeconds(5));
sessionReceiver.Peek();
I think the big question here is Sync vs Async whether you want the requesting party to sit back and wait for a response (WCF) or back later and check if the response is ready yet Service Bus but that is a business decision.
This link or this MSDN article might help you get started with Req/Rep for SB.
I don't think that deciding which technology should be used is a business decision. At first, it's a technical one.
I would not go with a product which is very operating system dependent, and worst, it's so premature. We can be creating coupling (OS x Bus) and stepping over a mined field.
But, this is only a personal opinion and might be biased as I'm not a Azure SB specialist.
I agree with #Tom, your decision is more related to sync/async model.
Some questions I usually answer before deciding on this subject:
Can we preview the rate of requests/minute and the amount of clients?
What is the nature of the service? Heavy processing logic or simple queries against a database?
I can list some others if you wish, but those two can easily help on the decision, forcing you to think broadly.
I'm a little confused as to what the difference is between a message broker e.g. RabbitMQ and Message-orientated Middleware. I can't find much info apart from what's on Wikipedia. When searching MOM I find info on AMQP which states is a protocol for MOM.. what does this mean? What is MOM then? I also have read that RabbitMQ implements the AMPQ protocol.. so why does that make a RabbitMQ a messsage broker? Are a message broker and MOM the same thing?
Hope some can unravel my confusion. thanks
An overview -
A protocol - A set of rules.
AMQP - AMQP is an open internet protocol for reliably sending and receiving messages.
MOM (message-oriented-middleware) - is an approach, an architecture for distributed system i.e. a middle layer for the whole distributed system, where there's lot of internal communication (a component is querying data, and then needs to send it to the other component, which will be doing some processing on the data) so components have to share info/data among them.
Message broker - is any system (in MOM) which handles messages (sending as well as receiving), or to be more precise which routes messages to the specific consumer/recipient. A Message Broker is typically built upon a MOM. The MOM provides the base communication among the applications, and things like message persistence and guaranteed delivery. "Message brokers are a building block of Message oriented middleware."
Rabbitmq - a message broker; a MOM implementation; an open-source implementation of AMQP; as per Wikipedia:
RabbitMQ is an open source message broker software (sometimes
called message-oriented middleware) that implements the Advanced
Message Queuing Protocol (AMQP).
As you asked:
When searching MOM I find info on AMQP which states is a protocol for MOM.. what does this mean?
MOM is about having a messaging middleware (middle layer) between (distributed) system components, and AMQP is protocol (set of rules) for reliably sending and receiving messages. So, a MOM implementation (i.e. Rabbitmq) may use AMQP.
What is MOM then?
Message-Oriented-Middleware - is an approach, an architecture for distributed system i.e. a middle layer for the whole distributed system, where there's lot of internal communication (a component is querying data, and then needs to send it to the other component, which will be doing some processing on the data) so components have to share info/data among them.
In short it's a way to design a system, for example: depending upon the overall requirements we need to develop a distributed system, with some internal communication. The biggest advantage of MOM architecture/decision is decoupling of the components i.e. if we're going to change the data query component it'll have no effect on the data processing components, as they're communicating via MOM (e.g. Rabbitmq Cluster) - the data processing component is getting the data in form messages, which then parses and processes them.
MOM at the end is just a design decision, that we use a middleware for gluing our system (distributed) components, a middleware for handling communication between them, in the form of messages (i.e. JSON). To implement a message-oriented-middleware we need more - set of specific rules i.e. how the messages will be published, consumed, how the acknowledgement will work, the lifetime of a message is until it is consumed, the persistence of a message, etc. AMQP is basically these set of rules i.e. a standard/protocol for implementing a MOM i.e. a messaging system using AMQP, means it confines itself by the stated rules. From Wikipedia:
AMQP mandates the behavior of the messaging provider and client to the
extent that implementations from different vendors are inter-operable,
in the same way as SMTP, HTTP, FTP, etc. have created inter-operable
systems.
I also have read that RabbitMQ implements the AMPQ protocol.. so why does that make a RabbitMQ a message broker?
Yes, Rabbitmq is a message broker (publisher -> exchange -> queue -> consumer). It's an open source AMQP implementation i.e. a messaging system/broker which confines to AMQP (the AMQP rules) - one can use Rabbitmq as the middleware, hence MOM.
AMQP - is just a set of rules i.e .how messages will be published, kept (in queues), consumed, delivery acknowledgement, etc.
Are a message broker and MOM the same thing?
In simple words, Yes. If we need to go with MOM design for our distributed system, we can simply use Rabbitmq (a message broker; an AMQP implementation) as the middleware.
"MOM" broadly means any technology that can deliver "messages" from one user-space application to another. A message is usually understood to be a discrete piece of information, as compared to a stream.
MOM products used to be quite large and complex: CORBA, JMS, TIBCO, WebsphereMQ, etc. and tried to do a lot more than simply deliver messages.
A broker is a particular set of routing and queuing patterns, and we usually use the term "broker" specifically in MOM (as compared to HTTP, email, XMPP, etc.) Routing means, one message goes to one peer, to one of many peers, to all of many peers, etc. Queuing means messages are held in memory or disk until they can be delivered (and in some cases, acknowledged).
AMQP used to specific those broker patters, so an application could rely on consistent behavior from any AMQP-compatible broker (thus RabbitMQ and OpenAMQ looked much the same to a client app, like two HTTP or two XMPP servers would look the same). AMQP/1.0 specifies just the connection between nodes, so you don't have guarantees of behavior. This makes AMQP/1.0 much easier for firms to implement, but doesn't deliver interoperability.
ZeroMQ is message-oriented middleware that defines, like AMQP/1.0, the connections between pieces rather than the behaviour of a central broker. However it's relatively easy to write MOM brokers using 0MQ, and we've done a few of these (like Majordomo).
Message brokers are one (quite popular) kind of MOM. Another kind of MOM would be brokerless MOM, like ZeroMQ. With broker based MOM, all messages go to one central place: broker, and get distributed from there. Broker less MOM usually allows for peer to peer messaging (but does not exclude option of central server as well) .
AMQP is broker based MOM protocol definition (at least all versions prior to 1.0, which drifts into more general MOM), and there are several different Message brokers implementing that protocol, RabbitMQ is just one of them.
I am novice in WCF and I have a project that needs to be migrated into WCF communication base with the client/server and server to server architecture.
My question is what is the right messaging function that I need for this project that insure the security of data across the network ,reliable connection and speed exchange of data.
I was able to find out the WCF has numerous messaging function.
Below is the architecture of my project:
Note: The clients should be simultaneously updated by both data processing and feed source servers. And clients also sends simultaneous requests to the servers while feeds are still being supplied by the feed source server.
I would be appreciate any suggestion or comments.
My first question is why are you putting the Connection Manager (CM) component in-between your clients and the services which they want to use? What is the job it does which means it needs to be right in the middle of everything?
This ultimately means that your CM component will have to handle potentially high volumes of bi-directional traffic across potentially different transport bindings and introduces a single failure point.
What if client A wants only to receive messages from the Feed Source (FS) component? Why should client A have to deal with an intermediary when it just wants to send a subscription notification to receive updates from the FS?
Equally, what if client B wants to send a message to the Data Processing (DP) component? Surely it should just be able to fire off a message to DP?
I think the majority of what you want to do with this architecture can be achieved with one-way messaging, in which case you should use netMsmqBinding (assuming you are in a pure wcf environment).
Well, I know that in a duplex contract the service can send messages to the client, but I would like to know when that is really useful.
I have a common application that send request to the service to get data from the a database, insert data... etc. Also, I need to store files about 40MB in the database, so I need a good performance. For this reason, I would like to use the net.tcp binding with transfer mode streamed, but the problem is that a net.tcp duplex service can't use the streamed transfer mode.
So I think I have some options.
1.- study if I really need a duplex contract for this kind of application. Perhaps in a chat application, for example, it has more sense a duplex contract because the server perhaps need to notify to the client when a contact is connected... etc. But in a common client that access to a data base, is necessary a duple contract? what kind of operations would can need a duplex contract?
2.- Other option it's not to have a duplex contract, but implement a no duplex contract in the server and other single contract in the the client, so when a client connect to the service, the service receive the needed information to connect to the service of the client. But, is this a good way to avoid a duplex contract?
3.- Really for my application I need tcp instead of a duplex HTTP that allows a streamed transfer mode? What is the advantages of the tcp over the HTTP in terms of performance?
Thanks.
You need duplex if you want to implement callback pattern. Callback means that client does not know when some event happens in server.
If you do not know when event happens you have two options to implement:
Polling - send requests every X minutes to check if event happened. Server should either return event details (if it happened) or return flag saying that you need to continue calling. Server also can return recommended timeout in advanced scenarios.
Callback - client sends some form of description what server should do if event happened. This may be pointer to function in C, delegate in .NET or endpoint schema in WCF. Server remembers that info and makes call from their side when time came.
As you can see duplex/callback means that at some point server works as client (initiates communication) and this is a big game change.
WCF duplex communications may require special network configuration because in many cases network allows you to call external services (you work as client) but forbids external resources to call you (external service works as client). This is implemented for security purposes.
Returning to your questions:
You do not need duplex if you only need to download big amount of data. You may need it if you want to catch updates that happened in server and notify clients. Duplex should work for Chat because in chat there are many scenarios when client needs to be notified with changes introduced by others.
What you described is hand-made variant of duplex channel. You should use proved and tested duplex implementation made by MS If you want server to call your method. Otherwise your option is polling.
You are right, you need tcp + streamed transfer mode to deal with big amount of data. TCP uses binary serialization which is more compact comparing to text serialization + with TCP you do not need to send any HTTP headers or SOAP envelops. Disable security if you do not need it. It has a big performance impact.
Addressing each point:
1, 2. I think that for your scenario a duplex service is an overkill. As you say yourself a duplex service is usually handy when both the client and service need to keep notifying each other on a constant basis, what you're doing, getting lots of data in/out of a database doesn't seem to be a good case for using duplex communication. Regarding netTcpBinding not allowing Streaming with duplex, you can just return a byte array (byte[]) instead of a stream. 40 MB is a lot, but I don't think Streaming will necessarily have a significant performance gain over a duplex service which will return a byte array (up to you to test each setup and compare the results). So you have a few options here, don't stream and return a byte array (you can do this with your duplex service) or you can just forget about making your service duplex since there doesn't seem to be a strong case for you to make it duplex and just return a Stream:
[OperationContract]
Stream RetrieveFile(long _fileId);
[OperationContract]
long SaveFile(Stream _stream);
3. netTcpBinding has a considerable performance advantage over HTTP bindings, but it comes with a price, mostly because its TCP ports are sometimes blocked by internet firewalls, although you can use netTcpBinding over the internet, it's not recommended. Choosing a binding depends on what you're looking to do, if your clients are going to consume your service over the internet, then netTcpBinding is not a good idea (blocked TCP ports, firewalls etc.), but if your clients are consuming the service in the same network (LAN) then netTcpBinding is the most sensible choice. wsDualHttpBinding (doesn't support streaming :#) is a good choice if you want to stick to a duplex service (equivalent of PollingDuplexHttpBinding in Silverlight), or any other HTTP based bindings if you let go of the idea of a duplex service.
Some articles that may help you out, performance comparison of various WCF bindings:
http://blog.shutupandcode.net/?p=1085
http://tomasz.janczuk.org/2010/03/comparison-of-http-polling-duplex-and.html
And about Streaming large data with WCF over HTTP, according to the authors, both samples have been tested with up to 2GB of data:
http://garfoot.com/blog/2008/06/transferring-large-files-using-wcf/
http://www.codeproject.com/Articles/166763/WCF-Streaming-Upload-Download-Files-Over-HTTP
You shouldn't think that you must use netTcpBinding or you must use Streamed transfer for your service, netTcpBinding only becomes more performant than HTTP bindings after you enable throttling and configure some socket level properties. And streaming 40 MB will not have significant performance gains over buffered transfer. So you have a lot of options and a lot of trade-offs to make. There is no black and white and right or wrong, it's about how you customise your service to suit your needs best, most solutions will work. Your scenrio is a very common one and there are lots of stuff online about large data transfer in WCF, do more research ;)