Before you decide its a tl:dr (too long, didnt read) post try to read at least some, since Its a question broken down in a lot of small pieces. Some of which you can probably answer and help me.
Please try to help me as much as you can. These types of problems are very common on the internet and I think you will help me and much more people after me.
I am currently researching HTTP services and the protocol itself so that I can discover if it is useful to me.
I have some basic questions as well as some code that needs to be discussed.
First I would like to know how does the communication start? I have discovered that the client sends a message in which it requests a resource (is this correct?). Then what happens? I (as a server) have to reply with what?
Do I need to append a carriage return and a line feed after every response? Somewhere it says there even need to be two (\r\n\r\n).
How can an asynchronous writing be established? (I hope this question is understandable) My primary goal is to achieve a connection between a client and a server and then a continuous data stream from server to the client. Does the client need to reply for every message it gets?
I hope I made my questions clear, since I'm not an expert in these things (yet, I am very interested in it).
And for the programming part of my problem.
I have managed to put together a simple program in Qt in C++ (server side) and a simple client in Objective C (iOS). The client connects and I can read the request header. It is like this:
Data available, incoming: "GET / HTTP/1.1
Host: localhost:9990
Connection: close
User-Agent: CFStream%20test/1.0 CFNetwork/609 Darwin/12.2.0
Should I reply to this header manually? And if so, what?
The client side code looks like this (i know its not pseudo but i think its pretty self-explanatory):
- (void)setupStream
{
NSURL *url = [NSURL URLWithString:#"http://localhost:9990"];
CFHTTPMessageRef message = CFHTTPMessageCreateRequest(NULL, (CFStringRef)#"GET", (CFURLRef)url, kCFHTTPVersion1_1);
stream = CFReadStreamCreateForHTTPRequest(NULL, message);
CFRelease(message);
if (!CFReadStreamSetProperty(stream, kCFStreamPropertyHTTPShouldAutoredirect, kCFBooleanTrue))
{
NSLog(#"Some error.");
}
CFDictionaryRef proxySettings = CFNetworkCopySystemProxySettings();
CFReadStreamSetProperty(stream, kCFStreamPropertyHTTPProxy, proxySettings);
CFRelease(proxySettings);
if (!CFReadStreamOpen(stream))
{
CFRelease(stream);
NSLog(#"Error opening stream.");
}
CFStreamClientContext context = {0, self, NULL, NULL, NULL};
CFReadStreamSetClient(stream, kCFStreamEventHasBytesAvailable | kCFStreamEventErrorOccurred, readStreamCallback, &context);
CFReadStreamScheduleWithRunLoop(stream, CFRunLoopGetCurrent(), kCFRunLoopCommonModes);
NSLog(#"Done");
}
This is the setup stream method. The stream variable is a class variable of type CFReadStreamRef.
The callback looks like this:
static void readStreamCallback(CFReadStreamRef aStream, CFStreamEventType event, void *client)
{
ViewController *controller = (ViewController*)client;
[controller handleEvent:event forStream:aStream];
}
And the handle event like this:
- (void)handleEvent:(CFStreamEventType)event forStream:(CFReadStreamRef)aStream
{
if (aStream != stream)
{
return;
}
NSLog(#"Handle event callback");
switch (event)
{
case kCFStreamEventHasBytesAvailable:
NSLog(#"Work log");
UInt8 bytes[11];
CFIndex length;
length = CFReadStreamRead(stream, bytes, 11); //I know 11 bytes is hard coded, its in testing stage now. Feel free to suggest me how to do it better.
if (length == -1)
{
NSLog(#"Error, data length = -1");
return;
}
NSLog(#"Len: %li, data: %s", length, bytes);
break;
default:
NSLog(#"Other event");
break;
}
}
And thats practically all the client code that is worth mentioning. The Qt Server part (I will only post the important parts) is done like this: (this is a subclassed QTcpServer class). First the startServer(); is called:
bool Server::startServer()
{
if (!this->listen(QHostAddress::Any, 9990))
return false;
return true;
}
When there is a connection incoming the incomingConnection is fired off with the socket descriptor as a parameter:
void Server::incomingConnection(int handle)
{
qDebug("New client connected");
ServerClient *client = new ServerClient(handle, this); //The constructor takes in the socket descriptor needed to set up the socket and the parent (this)
client->setVectorLocation(clients.count()); //This is a int from a Qvector in which i append the clients, its not important for understanding right now.
connect(client, SIGNAL(clientDisconnected(int)), this, SLOT(clientDisconnected(int)), Qt::QueuedConnection); //When the client socket emits a disconnected signal the ServerClient class emits a client disconnected signal which the server uses to delete that client from the vector (thats why I use "setVectorLocation(int)") - not important right now
clients.push_back(client); //And then I append the client to the QVector - not important right now
}
The ClientServer class constructor just creates a new socket and connects the required methods:
ServerClient::ServerClient(int handle, QObject *parent) :
QObject(parent)
{
socket = new QTcpSocket(this); //Socket is a class variable
connect(socket, SIGNAL(disconnected()), this, SLOT(disconnected()));
connect(socket, SIGNAL(readyRead()), this, SLOT(readyRead()));
socket->setSocketDescriptor(handle);
}
Ready read just writes me the data incoming (it wont be much user later i think):
void ServerClient::readyRead()
{
qDebug() << "Data available, incoming: " << socket->readAll();
}
And finally the write data:
void ServerClient::writeData(QByteArray *data)
{
data->append("\r\n\r\n"); //I have read this must be appended to all outgoing data from a HTTP server
socket->write(*data);
socket->flush();
qDebug() << "Written data to client: " << *data;
}
This code however does not always work. Sometimes when I write message like "Message" the client recieves all the data and some things that shouldnt be there (the new line and a wierd symbol - can NSLog cause this?). Sometimes when I send "Hellow" the client only gets "Hel" and some other funky stuff.
What are the problems? What should I pay more attention about? Anything that will help me will be MUCH appreciated. And please dont paste in some links that contain a book with a few hundred pages, Im sure this can be solved just by explaining things to me.
THANKS A LOT!
Jan.
You asked many questions ... and that's a perfectly legitimate thing to do :)
I confess - it was too long, I didn't read :(
BUT ...
1) Yes, the HTTP protocol does expect na "CRLF" ("\r\n"). Many servers and many clients are "forgiving", but strictly speaking - yes, you need them.
REFERENCE: RFC 2616
2) Wanting to understand HTTP "internals" is also perfectly legitimate - I applaud you.
One good way is to read the RFC(s).
Another is to use a "telnet" client: http://blog.tonycode.com/tech-stuff/http-notes/making-http-requests-via-telnet
Yet another is to study requests and responses in FF Firebug
3) Socket programming is another issue - which explains why sometimes you might read "hello world", and other times you might just get "hel".
Strong recommendation: Beej's Guide to Network Programming
4) Finally, no way would I actually write a server in Qt with C++ (except maybe as a toy "science experiment", or for some really off-the-wall requirement)
I would definitely write server code in C# (for Windows servers), Java (for everything else) or a scripting language I felt comfortable with (Perl, Ruby/RoR, Python and Lua all come to mind).
IMHO .. and hope that helps!
Your questions pretty much amount to "how does HTTP work", and the full answer lies in the specification.
Related
The client I'm building is using Reactive Cocoa with Octokit and so far it has been going very well. However now I'm at a point where I want to fetch a collection of repositories and am having trouble wrapping my head around doing this the "RAC way"
// fire this when an authenticated client is set
[[RACAbleWithStart([GHDataStore sharedStore], client)
filter:^BOOL (OCTClient *client) {
return client != nil && client.authenticated;
}]
subscribeNext:^(OCTClient *client) {
[[[client fetchUserRepositories] deliverOn:RACScheduler.mainThreadScheduler]
subscribeNext:^(OCTRepository *fetchedRepo) {
NSLog(#" Received new repo: %#",fetchedRepo.name);
}
error:^(NSError *error) {
NSLog(#"Error fetching repos: %#",error.localizedDescription);
}];
} completed:^{
NSLog(#"Completed fetching repos");
}];
I originally assumed that -subscribeNext: would pass an NSArray, but now understand that it sends the message every "next" object returned, which in this case is an OCTRepository.
Now I could do something like this:
NSMutableArray *repos = [NSMutableArray array];
// most of that code above
subscribeNext:^(OCTRepository *fetchedRepo) {
[repos addObject:fetchedRepo];
}
// the rest of the code above
Sure, this works, but it doesn't seem to follow the functional principles that RAC enables. I'm really trying to stick to conventions here. Any light on capabilities of RAC/Octokit are greatly appreciated!
It largely depends on what you want to do with the repositories afterward. It seems like you want to do something once you have all the repositories, so I'll set up an example that does that.
// Watch for the client to change
RAC(self.repositories) = [[[[[RACAbleWithStart([GHDataStore sharedStore], client)
// Ignore clients that aren't authenticated
filter:^ BOOL (OCTClient *client) {
return client != nil && client.authenticated;
}]
// For each client, execute the block. Returns a signal that sends a signal
// to fetch the user repositories whenever a new client comes in. A signal of
// of signals is often used to do some work in response to some other work.
// Often times, you'd want to use `-flattenMap:`, but we're using `-map:` with
// `-switchToLatest` so the resultant signal will only send repositories for
// the most recent client.
map:^(OCTClient *client) {
// -collect will send a single value--an NSArray with all of the values
// that were send on the original signal.
return [[client fetchUserRepositories] collect];
}]
// Switch to the latest signal that was returned from the map block.
switchToLatest]
// Execute a block when an error occurs, but don't alter the values sent on
// the original signal.
doError:^(NSError *error) {
NSLog(#"Error fetching repos: %#",error.localizedDescription);
}]
deliverOn:RACScheduler.mainThreadScheduler];
Now self.repositories will change (and fire a KVO notification) whenever the repositories are updated from the client.
A couple things to note about this:
It's best to avoid subscribeNext: whenever possible. Using it steps outside of the functional paradigm (as do doNext: and doError:, but they're also helpful tools at times). In general, you want to think about how you can transform the signal into something that does what you want.
If you want to chain one or more pieces of work together, you often want to use flattenMap:. More generally, you want to start thinking about signals of signals--signals that send other signals that represent the other work.
You often want to wait as long as possible to move work back to the main thread.
When thinking through a problem, it's sometimes valuable to start by writing out each individual signal to think about a) what you have, b) what you want, and c) how to get from one to the other.
EDIT: Updated to address #JustinSpahrSummers' comment below.
There is a -collect operator that should do exactly what you're looking for.
// Collect all receiver's `next`s into a NSArray. nil values will be converted
// to NSNull.
//
// This corresponds to the `ToArray` method in Rx.
//
// Returns a signal which sends a single NSArray when the receiver completes
// successfully.
- (RACSignal *)collect;
Id like to use srp in my current project. But im kinda at a loss as to how i would implement that with openssl. I got the client side running but i dont know how to write the server side. I also couldnt find any documentation orexample implementations of use. What i want is to store the login information inside a database and then retrieve that data when needed. Im using poco for most of the network part so writing the client was rather easy and i sucessfully tested it against other servers. So i would be gratefull for hints as to how to implement the server side.
There is an example of how to do this in file ssl/ssltest.c in OpenSSL archive.
At a high level you register an authentication callback. This callback is called automatically as necessary during SSL_Accept() to authenticate your users.
From ssltest.c example callback looks like the following:
static int MS_CALLBACK ssl_srp_server_param_cb(SSL *s, int *ad, void *arg)
{
SRP_SERVER_ARG * p = (SRP_SERVER_ARG *) arg;
if (strcmp(p->expected_user, SSL_get_srp_username(s)) != 0)
{
fprintf(stderr, "User %s doesn't exist\n", SSL_get_srp_username(s));
return SSL3_AL_FATAL;
}
if (SSL_set_srp_server_param_pw(s,p->expected_user,p->pass,NULL)<0)
{
*ad = SSL_AD_INTERNAL_ERROR;
return SSL3_AL_FATAL;
}
return SSL_ERROR_NONE;
}
To register you call
SSL_CTX_set_srp_username_callback(s_ctx, ssl_srp_server_param_cb);
The arg parameter of callback function is a void pointer passed to your callback so that you can reference any necessary application context in your application from the callback.
To set arg parameter call SSL_CTX_set_srp_cb_arg(s_ctx, mypointer);
It is really this easy. Be sure SRP ciphers are being included in your cipher list.
There is a way to get feedback on autentication failures that can be used to implement countermeasure against high rate online dictionary attack.
To do this call SSL_CTX_set_info_callback(s_ctx,mynotifycallback) to register your callback. Had success using condition below to filter unrelated notifications.
void mynotifycallback(const SSL *s, int reason, int ret)
{
if (reason & SSL_CB_ALERT && ret & SSL3_AD_BAD_RECORD_MAC && SSL_get_srp_username((SSL *)s))
// authentication failure
}
I am trying to write a basic app that uses CoreMidi to receive midi events from a specific source. I understand that all midi events that come into a port call the proc that I connected via MidiInputPortCreate(). I also understand that when using MidiPortConnectSource() that you can send an identifier (connRefCon) to help know what the source is. But I'm not sure how to use it.
I figure that within my MidiReadProc that I can use the scrConnRefCon and an if statement to listen to a specific source, but I still dont know what *void I should pass to separate each source. Ideally my ReadProc will look something like this:
void SourceReadProc (const MIDIPacketList *pktlist,
void *readProcRefCon,
void *srcConnRefCon)
{
if (srcConnRefCon == mySourceChoice) {
// pass the pktlist to do something
}
};
Any help will be greatly appreciated.
GW
After a break I've come back to this project with a fresh perspective. When I call MIDIPortConnectSource and pass a unique connRefCon it's not apparently passing for each endpoint. Here's my code:
ItemCount count = MIDIGetNumberOfSources();
for (Itemcount i=0; i<count; i++) {
MIDIEndpointRef endpoint = MIDIGetSource(i);
MIDIObjectGetStringProperty(endpoint,kMIDIPropertyName, &midiEndpointSourceName);
NSLog(#"Source %lu: %#", i, midiEndpointSourceName);
MIDIPortConnectSource(midiSourcePort, endpoint, (void*)&i);
}
Then my read proc:
void SourceReadProc (const MIDIPacketList *pktlist,
void *readProcRefCon,
void *srcConnRefCon)
{
ItemCount *source = (ItemCount*) srcConnRefCon;
NSLog(#"source: %lu", *source);
}
I've hooked up two different midi sources and I can find them both just fine. My first code reports that there are two sources and tells me their names. But my read proc says that the sources is always the first source. I've tried three different data types when passing the connRefCon with no luck. I feel that my issue must be with the MIDIPortConnectSource.
Any help or even troubleshooting ideas would be great. I wish that CoreMIDI had functions to query what's connected to ports so I could check that, but alas, there's not.
The srcConnRefCon is useful if you've made multiple MIDIPortConnectSource() calls. Most commonly, it's a pointer to an object representing the source, but it could be anything. If you just want to disambiguate multiple sources, you could, say, use a string.
MIDIPortConnectSource(port, endpoint1, (void *)"endpoint1");
MIDIPortConnectSource(port, endpoint2, (void *)"endpoint2");
Then, in your SourceReadProc, you'd do something like this:
char *source = (char *)srcConnRefCon;
if (!strcmp(source, "endpoint1")) {
// Process packets from source 1
}
Make sure the allocation lifetime of whatever you pass in extends as long as the port is connected - otherwise you'll get a dangling pointer, which can be hell to debug.
I've had trouble finding simple guides/examples for basic sockets in Objective-C (using NSSocketPort with NSFileHandle or using CFSocket/CSNetwork). Can anyone recommend a guide or a useful example? I would appreciate this greatly! I have tried to use this, but it is incomplete. Thanks!
P.S. I have been stuck in this part of my project for awhile and am starting to get desperate for some help.
I know I am submitting answer to an very old question. In case you(visitor/stackoverflow {lover/users}) want to code your own asynchronous socket. All you need to do is detach native socket handle from connected CFSocketRef object.
void TCPClientCallBackHandler(CFSocketRef s, CFSocketCallBackType callbacktype,CFDataRef address, const void *data,void *info){
ClientSocket *obj_client_ptr=(__bridge ClientSocket*)info;
switch (callbacktype) {
case kCFSocketConnectCallBack :
if(data){
[obj_client_ptr StopClient];
}
else{ //detach socket started
CFSocketNativeHandle handle=CFSocketGetNative(s);
CFSocketSetSocketFlags(s, 0);
CFSocketInvalidate(s);
CFRelease(s);
s=nil;//detach socket ended
[obj_client_ptr ConfigureStream:handle];
// handle pass to CFStreamCreatePairWithSocket then bridge to NSStream
}
break;
}
}
if still not got it, then watch it on youtube:
https://www.youtube.com/watch?v=bJP4nysTmnI
The easiest way I've found to do this is the Cocoa Async Socket class:
https://github.com/robbiehanson/CocoaAsyncSocket
It's pretty straight forward to use, there's good html documentation included.
I'm finding mixed answers to my question out in the web. To elaborate on the question:
Should I instantiate a service client proxy once per asynchronous invocation, or once per Silverlight app?
Should I close the service client proxy explicitly (as I do in my ASP.NET MVC application calling WCF services synchronously)?
I've found plenty of bloggers and forum posters out contradicting each other. Can anyone point to any definitive sources or evidence to answer this once and for all?
I've been using Silverlight with WCF since V2 (working with V4 now), and here's what I've found. In general, it works very well to open one client and just use that one client for all communications. And if you're not using the DuplexHttBinding, it also works fine to do just the opposite, to open a new connection each time and then close it when you're done. And because of how Microsoft has architected the WCF client in Silverlight, you're not going to see much performance difference between keeping one client open all the time vs. creating a new client with each request. (But if you're creating a new client with each request, make darned sure you're closing it as well.)
Now, if you're using the DuplexHttBinding, i.e., if you want to call methods on the client from the server, it's of course important that you don't close the client with each request. That's just common sense. However, what none of the documentation tells you, but which I've found to be absolutely critical, is that if you're using the DuplexHttBinding, you should only ever have one instance of the client open at once. Otherwise, you're going to run into all sorts of nasty timeout problems that are going to be really, really hard to troubleshoot. Your life will be dramatically easier if you just have one connection.
The way that I've enforced this in my own code is to run all my connections through a single static DataConnectionManager class that throws an Assert if I try to open a second connection before closing the first. A few snippets from that class:
private static int clientsOpen;
public static int ClientsOpen
{
get
{
return clientsOpen;
}
set
{
clientsOpen = value;
Debug.Assert(clientsOpen <= 1, "Bad things seem to happen when there's more than one open client.");
}
}
public static RoomServiceClient GetRoomServiceClient()
{
ClientsCreated++;
ClientsOpen++;
Logger.LogDebugMessage("Clients created: {0}; Clients open: {1}", ClientsCreated, ClientsOpen);
return new RoomServiceClient(GetDuplexHttpBinding(), GetDuplexHttpEndpoint());
}
public static void TryClientClose(RoomServiceClient client, bool waitForPendingCalls, Action<Exception> callback)
{
if (client != null && client.State != CommunicationState.Closed)
{
client.CloseCompleted += (sender, e) =>
{
ClientsClosed++;
ClientsOpen--;
Logger.LogDebugMessage("Clients closed: {0}; Clients open: {1}", ClientsClosed, ClientsOpen);
if (e.Error != null)
{
Logger.LogDebugMessage(e.Error.Message);
client.Abort();
}
closingIntentionally = false;
if (callback != null)
{
callback(e.Error);
}
};
closingIntentionally = true;
if (waitForPendingCalls)
{
WaitForPendingCalls(() => client.CloseAsync());
}
else
{
client.CloseAsync();
}
}
else
{
if (callback != null)
{
callback(null);
}
}
}
The annoying part, of course, is if you only have one connection, you need to trap for when that connection closes unintentionally and try to reopen it. And then you need to reinitialize all the callbacks that your different classes were registered to handle. It's not really all that difficult, but it's annoying to make sure it's done right. And of course, automated testing of that part is difficult if not impossible . . .
You should open your client per call and close it immediately after. If you in doubt browse using IE to a SVC file and look at the example they have there.
WCF have configuration settings that tells it how long it should wait for a call to return, my thinking is that when it does not complete in the allowed time the AsyncClose will close it. Therefore call client.AsyncClose().