hi guys i'm making a client-server software and this is my first question
i'd like to ask: how to distinguish data that sended by tcp Connection?
Well, my points are:
-we can determine data that sended by tcpconnection.
for example, we have 3 Listviews in our form
the point of the first listview is for Biodata of client.
the point of second listview is for *The value obtained from the clients
n the point of third listview is for The picture obtained from the clients
in this case we have 3 main points that must be processed.
in fact, we only have 1 connection in our system.
Well, here I'm confused..
how to determine that data we received is for the first listview or second listview or third listview?
remember, the data of third listview is a picture that we received from tcpconnection
How do we do that with 1 connection in our system?
do i have to make 3 connection to control third listviews?
With socket communication, both the client and the server must use the same agreed-upon protocol so that they can understand each other. There are many standard protocols that have already been created, so for most tasks, creating your own protocol is unnecessary. However, if you must, you can always define your own protocol. The nature of your protocol will obviously depend completely on your specific needs, so it would be impossible to tell you what the protocol should be. However, generally speaking, the first thing your protocol must define is how to know where each complete message begins and ends. This is often accomplished by separating each message with a delimiter (e.g. new line, EOF, null). As Francois recommended, you could alternatively specify the length of the message at the beginning of each message. Within each message, you then will need a header section which, among other things, would specify the type (the format) of the data stored in the body of the message.
A simple implementation might be to send each message as a JSON or XML document. Doing so makes it very easy to define and modify the format of the message.
However, unless you really need to, I would recommend using one of the built-in communication frameworks that are provided with .NET. For simple tasks, often a simple asmx web service is sufficient. For more complex tasks, often WCF is a good choice. An asmx web service uses SOAP via HTTP via TCP/IP. WCF uses SOAP, but the lower level connection is configurable so it doesn't have to use TCP/IP, but it can easily do so.
Related
When is an API NOT RPC ?
After a long long discussion on twitter regarding API design.
I want to try to come to a clear answer when an API is not RPC based.
There seems to be quite a bit of confusion around this Is That REST API Really RPC? Roy Fielding Seems to Think So
That specific link is about REST and RPC. my question just aims to ask about RPC vs not RPC in general, not in the context of HTTP.
The TLDR; definition of when an API IS RPC is "when it hides the network from the developer"
Fair enough.
But back to when is it not.
The Twitter duscussion focused on the HttpClient, as not being RPC.
My argument is that the HttpClient does not really model the network.
There is nothing in there modelling latency.
There is also a very weak representation of network failures.
Most status codes in HTTP are not network related, e.g. NotFound, Payment Required etc.
Depending on language and SDK, the HttpClient may take a string or an URI.
In the case where it takes a string representing the URI, that also doesn't really communicate that there is a network involved, you would need to know what the string represents and whant an URI is.
If you had never seen HTTP before, how would you know by looking at that specific API?
There is an API consuming a string, and returning an object either containing a string or some int response codes.
How would you know?
Most developers of-course knows what HTTP is, but from a strict definition point of view, how would you define an API as not RPC?
Would a checked exception in Java "NetworkException" be enough to clearly communicate that there is a network involved?
If the functions you are calling have descriptive names like "SayHelloOverNetwork" would that be enough to make it not RPC?
And exactly when is a procedure a procedure in remote terms?
All network communication will result in some code being run on the receiving system.
If we take a person who have never been in touch with technology or protocols, and teach this person a programming language.
What would be the definition of "Not RPC" this person could employ to spot what is RPC and not?
I am being dense and possibly silly here, but I am trying to find the essence of "Not RPC", what exactly is required for this?
Does it for example need to be non blocking in order to play nice with unknown latency?
Or could a non RPC API be blocking?
IMO, if it is blocking, it hides that there will be latency involved, and thus falls into the "Latency is zero" fallacy, meaning it hides the network.
This all seems to be super obvious to everyone else but me, but no one have yet shown a concise answer on what the requirements for not being RPC is.
For me it looks like you're putting all eggs into one basket. Lets start from the very beginning:
API is application programming interface
it is a set of clearly defined methods of communication between various software components
The only definitive property of API is that it is defined/documented way of communication.
If one application/module/component A can use/call another application/module/component B somehow - it means that B provides API and A uses this API.
Usually there are two aspects of API which must be defined:
What exactly is passed into/returned from component (its your application logic)
How exactly data is transferred/serialized (this is technical implementation)
I'm not touching "what" part for obvious reasons.
Lets focus on different ways of "how":
push 4 byte integer to stack, change IP register and read 4 byte integer output from EAX register
connect to socket, write data serialized as byte array and read some response as byte array back
call 911 from any phone, say your address and expect several cars on your driveway
In all these cases you're using some API = you're communicating with other components in some predefined way.
RPC is remote procedure call
computer program causes a procedure (subroutine) to execute in another address space
The only definitive property of RPC is that data is passed across different/remote address spaces, some architecture allow different address spaces on single host, for example x86. As soon as different physical hosts usually do not share address space, any call across network is RPC, but not vice versa.
Note: It is rare, but possible to share memory space across different physical hosts connected in network - then such communication strictly speaking is not RPC, lets omit such cases.
Any RPC call automatically means that you're calling some API. By definition. So we can say that RPC is part of API's "how", it is transport level. As soon as RPC itself does not define actual mechanism, there are could be very different implementations, for example shared memory, DMA, TCP/IP, etc.
I suppose now you can answer your question when an API is not RPC based - When API says so. It is up to API developer to define whether it should/can be called via RPC or not, API can define multiple ways of calling it.
As antonym to RPC you can use "in-process".
So, phrase API IS RPC is "when it hides the network from the developer" is absolutely non-sense. API must define "how" section.
HTTP is hyper text transfer protocol
request–response protocol in the client–server computing model
The only definitive property of HTTP is that it describes protocol/format of request (input arguments) and response.
You can use HTTP in non-RPC API. For example I prefer to think about HTTP as file format. So we can say that HTTP is another part of API's "how". It defines serialization part of API, but does not dictate you transport level.
Note: Some RPC protocols actually define both transport and serialization.
So, HttpClient is the tool which allows you to invoke API and use HTTP encoded request/response, usually such library supposes RPC as transport level. None of these terms mandates network or any particular transport protocol. This is why http client should not declare any kind of network exceptions/errors, but it could throw HTTP errors as exceptions.
Note: Network exceptions could be thrown from TCP/IP RPC implementation for example, HTTP client library could proxy them to you. Unfortunately some libraries wrongly couple HTTP with TCP/IP too much and border between different responsibilities is crossed.
REST is representational state transfer
architectural style for distributed hypermedia systems
It is very wide term, it defines a lot of things from different aspects and at very different levels, most important:
HOW your API should be designed (usage of URI)
HOW your API should be implemented (stateless, HTTP verbs)
HOW your API should be called (client-server)
Client-server, usually assumes cross-process communication. Ie different address spaces, this is why we can say that REST mandates RPC as invoking mechanism to API + HTTP as serialization format.
Now, I suppose you will be able to understand these answers:
How would you know?
when you give client your REST API, it automatically defines some part of your API in terms of other protocols. Ie - to use REST API you must read/know HTTP protocol first. API must define what kind of RPC must be used.
HttpClient does not really model the network.
It should not do this. It works with HTTP semantics.
you would need to know what the string represents and whant an URI
There are two URIs actually:
URI is part of API's "What" section, it defines business object location. It has nothing to do with network or DNS system. You should not understand it.
URI could be part of TCP/IP RPC requirements, in this case it represents domain name/path. But some implementations can work with IP addresses, not URI.
If the functions you are calling have descriptive names like "SayHelloOverNetwork" would that be enough to make it not RPC?
As I wrote before - we can assume network as RPC always.
Does it for example need to be non blocking in order to play nice with unknown latency?
Its up to API developer:
API can contain functions which suppose asynchronous execution, but client can call them in blocking way
API can contain blocking functions, but client can call them asynchronously
An RPC is a network API with enough layers of abstraction on top. What's enough layers? That's a subjective thing.
Whether an API is an RPC or not doesn't, in my opinion, depend on method names or the names of exceptions/errors you need to handle. We're adults, we know that the call is done over the network. Naming it "SayHello" instead of "SayHelloOverNetwork" doesn't make a difference.
What does make a difference is all of those layers of abstraction - the less resource management and error handling you need to do, the more RPC-ish the code.
And about the specific HttpClient example - I'd say that today, developers doing high-level work consider HTTP to be a transport medium; an alternative to "plain old sockets".
So while a person who does not know what HTTP is might look at "a function that takes a string and returns an error code" as RPC, a modern developer would probably see it as "nitty gritty networking code". He would then say yuck, and put a few more layers of abstraction on top to make method calls from his business logic more RPC-ish. That way, the business logic would have to handle only the most extreme failures.
A) If the documentation told you how to do what you want by calling the API, then you are calling an RPC-like API, not a REST API.
B) If the API itself told you how to call it to accomplish what you want, then that API is not very RPC-like, and might be a REST API.
Programming in an object oriented or a procedural style is like case (A) -- you know what to call and how to call it to accomplish what you want before you write the code to call it. When [I assume Roy F.] says that an RPC-like API hides the network from the developer, he means that the developer can continue to program in this way whether or not his calls are remote -- he doesn't have to care about the network.
When you call a REST API, however, you have to program differently, because you have to let the API tell you what you can do and how you can do it. That's what it means to be "hypertext-driven".
Being hypertext-driven means that your stuff will continue to work when the guy on the other side of the network, who doesn't know or care about your program at all, completely changes what you can do and how you have to do it. Note that the lack of any contract between you and the system you are calling is the fundamental feature of the network that RPC hides.
Perhaps I'm going about this the wrong way, but here's my current "setup".
I have a silverlight client that uses Caliburn.Micro and a MEF container with a "LoadCatalog" class, to keep everything loosely coupled in an MVVM way.
I have a "common" dll where all the interfaces are kept.
All my views and viewmodels are separate projects, that only have a reference to the common dll.
The viewmodels use WCF (regular) to communicate to the backend. The frontend itself has a duplex connection to the backend.
Now here's where the question comes to mind. Whenever the backend thinks it's time to have a new screen appear at the frontend, it uses the callback channel to tell the frontend to load the next screen.
Does this seem like a good pattern to use? Or should I leave the management of what screen to load when to the frontend? I think it's nice to have this in the backend, but perhaps this is some kind of anti-pattern I'm not aware of, hence the question.
Now for argument sake, lets say I want to keep this in the backend.
What would be the best way to go about managing the collection of callback channels on the backend? If I enable SessionMode.Required on all the regular WCF endpoints, as well as the duplex channel, does this persist state together over multiple endpoints (regular+duplex)? Or will this persist state only within a single endpoint?
My guess (from the tests I have been able to do so far) is that I need to add some logic, like for example provide the frontend with a guid as soon as the callback connection is made. And then use that guid in the regular endpoint connections so the backend knows which "client" it is.
And would I "ever" be able to reliably collect all the channels and detect current state if I made a collection of the callback channels that I receive? I can intercept the callback channel now (just 1 instance atm, no collections made or anything, so single user) and use that to tell the frontend what to do. But sometimes when the client stops abrubtly (in other words when an error occurs) and I start the client again, it seems like the previous (faulted?) connection is still "re-used" or something, without luck so the communication flow stops after connection the duplex endpoint.
Does this make any sense?
Hope there's someone who has some experience in the matter that can shed some light on it for me. I'm no total newbie, but with regards to multiple connection and keeping them separated, I might need some pointers in the right direction.
Thanks!
Huron.
I managed to get this up and running.
When I create the duplex connection from the frontend to the backend I return a unique Guid. From now on I use this guid for all communication I do with the backend.
This makes the backend "recognize" the client.
In the backend I have a list of connections (grabbed the callback channel and stored it together with the Guid).
Just had to make sure to lock the list object whenever I iterate it or when I add/remove items from it, since it will be used from multiple threads by design.
The pattern to take control from the backend seems to work great so far.
Does NServiceBus 2.0 allow for defining serializer for given message type?
I want for all but one of my messaages to be serialized using XmlSerializer. The remaining one should be serialized using BinarySerializer.
Is it possible with NServiceBus 2.0?
I believe the serializer is specified on an endpoint basis, so all messages using that endpoint would use the same serializer.
However, if you follow the rote NServiceBus recommendation of one message type per endpoint/queue then you could effectively isolate one message type and use a different serializer for it.
I'm curious, however, what is special about the one message type that requires binary serialization?
Edit in response to comment
The Distributor info indirectly mentions this under Routing with the Distributor. Udi Dahan also frequently advises this in the NServiceBus Yahoo Group although it's difficult to provide links because the search there is poor.
Basically, the idea is that you wouldn't want high priority messages to get stuck behind lower-priority ones, and also that this provides you with the greatest flexibility to scale out certain message processing if necessary.
Because the MsmqTransportConfig only allows for one InputQueue to be specified, having one message type per queue also means that you only have one message handler per endpoint.
To address the image, you may still be able to encapsulate it in an XML-formatted message if you encode the byte array as a Base64-encoded string. It's not ideal, but if your images aren't too large, it may be easier to do this than to go to the trouble of using a different serializer on only one message type.
Another option is to store the image data out-of-band in a database or filesystem and then refer to it by an ID or path (respectively).
Not possible in Version 2. But it can be done using the pipeline in versions 5 and above http://docs.particular.net/samples/pipeline/multi-serializer/
So I've created a series of objects that interact with a piece of hardware over a serial port. There is a thread running monitoring the serial port, and if the state of the hardware changes it updates properties in my objects. I'm using observable collections, and INotifyPropertyChanged.
I've built a UI in WPF and it works great, showing me real time updating when the hardware changes and allows me to send changes to the hardware as well by changing these properties using bindings.
What I'm hoping is that I can run the UI on a different machine than what the hardware is hooked up to without a lot of wiring up of events. Possibly even allow multiple UI's to connect to the same service and interact with this hardware.
So far I understand I'm going to need to create a WCF service. I'm trying to figure out if I'll be able to pass a reference to an object created at the service to the client leaving events intact. So that the UI will really just be bound to a remote object.
Am I moving the right direction with WCF?
Also I see tons of examples for WCF in C#, are there any good practical use examples in VB that might be along the lines of what I'm trying to do?
No, WCF is a message based system - you pass around serialized (text/xml) messages. There's no "object references" that you can pass around.
The client has a proxy, which gives you the ability to "call" the service method. The WCF runtime then captures the parameters to that call, packages them up in a serialized message, and sends that message across the wire.
There is no direct connection between the client and the server - the client can't "reach over" to the service to get a remote object, nor can the service go back to the client to find out who called it or anything like that.
All that you want to send to the service must be part of either the message itself, or the headers that accompany the message.
Those messages must conform to the XML schema standard, which again means: only concrete non-generic types. You can't pass around interfaces, you cannot pass references - only concrete types made up of basic types such a string, int, datetime etc.
Update: maybe you need to check out the publish/subscribe (pub/sub for short) pattern - which you can also build using WCF. This would allow you data collection machine to publish its data on a regular basis or whenver it changes, and any number of subscribers could be notified of those changes.
Check out some of those articles - googling or binging for "WCF pub sub" will definitely turn out quite a few more!
Tom Hollander: Building a Pub/Sub Message Bus with WCF and MSMQ
Pub/sub sample with WCF net.tcp protocol in Silverlight 4
Pub/sub sample using HTTP polling duplex WCF channel in Microsoft Silverlight 3
WCF Router and Publish/Subscribe Sample Implementation
I have found myself responsible for carrying on the development of a system which I did not originally design and can't ask the original designers why certain design decisions were taken, as they are no longer here. I am a junior developer on design issues so didn't really know what to ask when I started on the project which was my first SOA / WCF project.
The system has 7 WCF services, will grow to 9, each self-hosted in a seperate console app/windows service. All of them are single instance and single threaded. All services have the same OperationContract: they expose a Register() and Send() method. When client services want to connect to another service, they first call Register(), then if successful they do all the rest of their communication with Send(). We have a DataContract that has an enum MessageType and a Content propety which can contain other DataContract "payloads." What the service does with the message is determined by the enum MessageType...everything comes through the Send() method and then gets routed to a switch statement...I suspect this is unusual
Register() and Send() are actually OneWay and Async...ALL results from services are returned to client services by a WCF CallbackContract. I believe that the reson for using CallbackContracts is to facilitate the Publish-Subscribe model we are using. The problem is not all of our communication fits publish-subscribe and using CallbackContracts means we have to include source details in returned result messages so clients can work out what the returned results were originally for...again clients have a switch statements to work out what to do with messages arriving from services based on the MessageType (and other embedded details).
In terms of topology: the services form "nodes" in a graph. Each service has hardcoded a list of other services it must connect to when it starts, and wont allow client services to "Register" with it until is has made all of the connections it needs. As an example, we have a LoggingService and a DataAccessService. The DataAccessSevice is a client of the LoggingService and so the DataAccess service will attempt to Register with the LoggingService when it starts. Until it can successfully Register the DataAccess service will not allow any clients to Register with it. The result is that when the system is fired up as a whole the services start up in a cascadeing manner. I don't see this as an issue, but is this unusual?
To make matters more complex, one of the systems requirements is that services or "nodes" do not need to be directly registered with one another in order to send messages to one another, but can communicate via indirect links. For example, say we have 3 services A, B and C connected in a chain, A can send a message to C via B...using 2 hops.
I was actually tasked with this and wrote the routing system, it was fun, but the lead left before I could ask why it was really needed. As far as I can see, there is no reason why services cannot just connect direct to the other services they need. Whats more I had to write a reliability system on top of everything as the requirement was to have reliable messaging across nodes in the system, wheras with simple point-to-point links WCF reliabily does the job.
Prior to this project I had only worked on winforms desktop apps for 3 years, do didn't know any better. My suspicions are things are overcomplicated with this project: I guess to summarise, my questions are:
1) Is this idea of a graph topology with messages hopping over indirect links unusual? Why not just connect services directly to the services that they need to access (which in reality is what we do anyway...I dont think we have any messages hopping)?
2) Is exposing just 2 methods in the OperationContract and using the a MessageType enum to determine what the message is for/what to do with it unusual? Shouldnt a WCF service expose lots of methods with specific purposes instead and the client chooses what methods it wants to call?
3) Is doing all communication back to a client via CallbackContracts unusual. Surely sync or asyc request-response is simpler.
4) Is the idea of a service not allowing client services to connect to it (Register) until it has connected to all of its services (to which it is a client) a sound design? I think this is the only design aspect I agree with, I mean the DataAccessService should not accept clients until it has a connection with the logging service.
I have so many WCF questions, more will come in later threads. Thanks in advance.
Well, the whole things seems a bit odd, agreed.
All of them are single instance and
single threaded.
That's definitely going to come back and cause massive performance headaches - guaranteed. I don't understand why anyone would want to write a singleton WCF service to begin with (except for a few edge cases, where it does make sense), and if you do have a singleton WCF service, to get any decent performance, it must be multi-threaded (which is tricky programming, and is why I almost always advise against it).
All services have the same
OperationContract: they expose a
Register() and Send() method.
That's rather odd, too. So anyone calling will first .Register(), and then call .Send() with different parameters several times?? Funny design, really.... The SOA assumption is that you design your services to be the model of a set of functionality you want to expose to the outside world, e.g. your CustomerService might have methods like GetCustomerByID, GetAllCustomersByCountry, etc. methods - depending on what you need.
Having just a single Send() method with parameters which define what is being done seems a bit.... unusual and not very intuitive / clear.
Is this idea of a graph topology with
messages hopping over indirect links
unusual?
Not necessarily. It can make sense to expose just a single interface to the outside world, and then use some internal backend services to do the actual work. .NET 4 will actually introduce a RoutingService in WCF which makes these kind of scenarios easier. I don't think this is a big no-no.
Is doing all communication back to a
client via CallbackContracts unusual.
Yes, unusual, fragile, messy - if you can ever do without it - go for it. If you have mostly simple calls, like GetCustomerByID - make those a standard Request/Response call - the client requests something (by supplying a Customer ID) and gets back a Customer object as a return value. Much much simpler!
If you do have long-running service calls, that might take minutes or more to complete - then you might consider One-Way calls which just deposit a request into a queue, and that request gets handled later on. Typically, here, you can either deposit the answer into a response queue which the client then checks, or you can have two additional service methods which give you the status of a request (is it done yet?) and a second method to retrieve the result(s) of that request.
Hope that helps to get you started !
All services have the same OperationContract: they expose a Register() and Send() method.
Your design seems unusual at some parts specially exposing only two operations. I haven't worked with WCF, we use Java. But based on my understanding the whole purpose of Web Services is to expose Operations that your partners can utilise.
Having only two Operations looks like odd design to me. You generally expose your API using WSDL. In this case the WSDL would add nothing of value to the partners, unless you have lot of documentation. Generally the operation name should be self-explanatory. Right now your system cannot be used by partners without having internal knowledge.
Is doing all communication back to a client via CallbackContracts unusual. Surely sync or asyc request-response is simpler.
Agree with you. Async should only be used for long running processes. Async adds the overhead of correlation.