I have following scenario:
Server should be Daemon.
Other Apps should be clients.
Many clients should communicate with server to get their task done by server at a time.
These tasks are such as copyfile, deletefile etc.
My solution:
Server has 5 worker threads each containing named pipe. Each pipe's availability status is kept in Shared memory structure. When client wants to communicate with server, it checks which pipe is available from shared memory then opens that pipe & sends its message on that pipe, respective worker thread of server servers this client request. That worker thread sends request status (Success/failure) on that pipe so that client will become aware of last operation status.
As far as I know, pipes on Mac os x are unidirectional & they lack capability of creating unlimited instances like Windows.
What mechanism could be best suited for such kind of communication?
Thanks,
Vaibhav.
As far as I know, pipes on Mac os x are unidirectional & they lack capability of creating unlimited instances like Windows.
Pipes are one directional, but Unix sockets are not. This is probably what you are after if you want to directly port your code to OS X.
However, there are probably better ways to do what you want to do, including stuff like Distributed Objects which I admit I have never used. Even if you stick with a socket interface, I think one socket would be easier with a thread monitoring the socket and handing off work to worker threads as it arrives, using listen and accept. Better still, have an NSOperationQueue or a dispatch queue to put the work on, then the OS will handle the task of optimising the thread count.
Related
Currently, we plan to upgrade our product to use MQ(RabbitMQ or ActiveMQ) for message transfer between server and client. And now we are using a network lib(evpp) for doing so.
Because I don't use MQ before, so excpet for a lot of new features of MQ, I can't figure out the essential difference between them, and don't know exactly when and where should we use MQ or just use network library is fine.
And the purpose that we want to use MQ is that we want to solve the unreliability of communication, such as message loss or other problems caused by unstable network environment.
Hope there is someone familiar with both of them could release my confusion. Thanks for advance.
Message queuing systems (MQ, Qpid, RabbitMQ, Kafka, etc.) are higher-layer systems purpose-built for handling messages reliably and flexibly.
Network programming libraries/frameworks (ACE, asio, etc.) are helpful tools for building message queueing (and many other types of) systems.
Note that in the case of ACE, which encompasses much more than just networking, you can use a message queuing system like the above and drive it with a program that also uses ACE's classes for thread management, OS abstraction, event handling, etc.
Like in any network-programming, when a client sends a request to the server, the server responds with a response. But for this to happen the following conditions must be met
The server must be UP and running
The client should be able to make some sort of connection between them
The connection should not break while the server is sending the response to the client or vice-versa
But in case of a message queue, whatever the server wants to tell the client, the message is placed in a message-queue i.e., separate server/instance. The client listens to the message-queue and processes the message. On a positive acknowledgement from the client, the message is removed from the message queue. Obviously a connection has to made by the server to push a message to the message-queue instance. Even if the client is down, the message stays in the queue.
Will a web server (WS) (like apache2 or nginix (or container like tomcat(TC)) create a new process to handle incoming request. My concern is about servers that support high number of parallel users (say 20K+ parallel users).
I think load balancing happens on the other side of web server (if it is used to front Tomcat etc). So in theory, a single web server should be accepting all the (20K+)incoming request before it can distribute the load to other servers backing it.
So, the questions is: Does Web Server (WS) handle all these requests in a single process or it smartly spawns other process to help share the work (i know the "client - server" binding happens though - client_host:random_port plus server_host:fixed_port).
Reference: Prior to reading this article:Fronting Tomcat with Apache I was thinking it is a single process doing all the smart work. But in this article there is mentioning of MPM (Multi-Processing Module)
It combines the best from two worlds, having a set of child processes each having a set of separate threads. There are sites that are running 10K+ concurrent connections using this technology.
And as it goes, it is getting more sophisticated as threads also being spawned like mentioned above. (these are not the tomcat threads that serve each individual request by calling the service method, but these are threads on Apache WS to handle request and distribute them to nodes for processing).
If any one used MPM. Little further explanation of how all this works will be great.
Questions like -
(1) As child processes are spawned what is it exact role. Is the child process just for mediating the request to tomcat or any thing more. If so, then after the child process gets response from TC, does the child process forward the response to parent process or directly to the client (since it can know the client_host:random_port from parent process. I am not sure if this is allowed in theory, though the child process can not accept any new request as the fixed_port which can bind to only one process is already tied to parent process.
(2) What kind of load is shared to thread by the child or parent process. Again it must almost be same as in (1). But what I am not sure is that even in theory if a thread can directly send the request to client.
Apache historically use prefork model of processing. In this model each request == separate operation system (OS) process. It's calling "prefork" because Apache fork some spare processes and process request within. If number of preforked processes not enough - Apache fork new. Pros: process can execute other modules or processes and not care that they do; cons: each request = one process, too much memory used and OS fork also can be slow for your requests.
Other model of Apache - worker MPM. Almost same as prefork, but using not OS processes but OS threads. Thread - it's like lightweight process. One OS process can run many threads using one memory space. Worker MPM used much less memory and new threads created fast. Cons: modules need to support thread, crash of module can crash all threads of all OS process (but this it not important for you because you are using apache as reverse proxy only). Other cons: CPU switching context when switching between threads.
So yes, worker much better than prefork in your case, but...
But we have Nginx :) Nginx using other model (btw, Apache has event MPM too). In this case you has only one process (well, can be few processes, see below). How it works. New request rising special event, OS process waking up, receive request, prepare answer, write answer and gone sleep.
You can say "wow, but this is not multitasking" and will be right. But one big difference between this model and simple sequentially request processing. What happens if you need write big data to slow client? In synchronous way your process need to wait acknowledging about data receiving and only after - process new request. Nginx and Apache event model use asynchronous model. Nginx tell to OS to send some piece of data write this data to OS buffer and... gone sleep, or process new requests. When OS will send piece of data - special event will be sent to nginx. So, main difference - Nginx do not wait I/O (like connect, read, write), Nginx tell to OS that he want and OS send event to Nginx than this task ready (socket connected, data written or new data ready to read in local buffer). Also, modern OS can work asynchronously with HDD (read/write) and even can send files from HDD to tcp socket directly.
Sure, all math operations in this Nginx process will block this process and its stop to process new and existing requests. But when main workflow is work with network (reverse proxy, forward requests to FastCGI or other backend server) plus send static files (asynchronous too) - Nginx can serve thousands simultaneous requests in one OS process! Also, because one process of OS (and one thread) - CPU will execute it in one context.
How I told before - Nginx can start few OS processes and each of this process will be assigned by OS to separate CPU core. Almost no reasons to fork more Nginx OS processes (there is only one reason to do it: if you need to do some blocking operations, but simple reverse proxy with backend balancing - not this case)
So, pros: less CPU context switching, less memory (comparing with worker MPM too), fast connection processing. More pros: Nginx created as HTTP load balancer and have lot of options for it (and even more in commercial Nginx Plus). Cons: If you need some hard math inside OS process, this process will be blocked (but all you math in Tomcat, so Nginx only balancer).
PS: typo fix will come later, out of time. Also, my English bad, so fixes always welcome :)
PPS: Answer question about number of TC thread, asked in comments (was too long for post as comment):
Best way to know it - test it using stress loading tools. Because this number depend on application profile. Response time is not good enough to help answer. Because, for example, big difference between 200ms of 100% math (100% cpu bound) vs 50ms of math + 150ms of sleep waiting database answer.
If application is 100% CPU bound - probably one thread per one core, but in real cases all applications also spent some time in I/O (receive request, send answer to client).
If application work with I/O and need to wait for answers from other services (database, for example), this application spends some time in sleep state and CPU can process other tasks.
So best solution to create number of requests close to real load and run stress test increasing number of concurrent requests (and number of TC workers for sure). Find acceptable response time and fix this number of threads. Sure, need to check before that it is not database fault.
Sure, here I'm talking about dynamic content only, requests for static files from disk must be processed before tomcat (by Nginx, for example).
From what I understand IMAP requires a connection per each user. I'm writing an IMAP client (currently just gmail) that supports many (100s, 1000s maybe 10000s+) users at a time. Obviously cutting down the number of open connections would be great. I'm wondering if it's possible to use thread pooling on my side to connect to gmail via IMAP or if that simply isn't supported by the IMAP protocol.
IMAP typically uses SSL over TCP/IP. And a TCP/IP connection will need to be maintained per IMAP client connection, meaning that there will be many simultaneous open connections.
These multiple simultaneous connections can easily be maintained in a non-threaded (single thread) implementation without affecting the state of the TCP connections. You'll have to have some sort of a flow concept per IMAP TCP/IP connection, and store all of the flows in a container (a c++ STL map for instance) using the TCP/IP five-tuple (or socketFd) as a key. For each data packet received, lookup the flow and handle the packet accordingly. There is nothing about this approach that will affect the TCP nor IMAP connections.
Considering that this will work in a single-thread environment, adding a thread pool will only increase the throughput of the application, since you can handle data packets for several flows simultaneously (assuming its a multi-core CPU) You will just need to make sure that 2 threads dont handle data packets for the same flow at the same time, which could cause the packets to be handled out of order. An approach could be to have a group of flows per thread, maybe using IP pools or something similar.
Why can't the application server send messages directly to the application? Why do you need the C2DM service in the middle?
To send a message from the server side you have two possibilities:
The client polls for new messages in certain intervals. Downside: Not a real-time solution. If you poll too frequently it will drain battery, consume your quota (if you don't have an unlimited package). Generally you do a lot of unnecessary work and traffic as most polls will return no messages.
Stay connected all the time. Downside: hard to deliver technically as phones can close connections when going to sleep mode. (At least nothing guarantees that they won't). Also you are running a background application 24/7.
The current state of C2DM will give you:
The ability to get messages even when your application is not running as Android will start your application (the part of it you configured, not necessarily the whole UI) when a message arrives.
A central, shared channel to deliver such messages. If 10 applications need real-time notifications on your phone this is one single facility, not 10 applications running and polling in parallel.
The promise: As this is the sanctioned API by Google for push messaging you can expect it to be optimized in the future. One improvement can be carrier-level messaging to initiate a C2DM session. That would mean you can put 100% of the "smart" part of your phone asleep.
Because the application can't (or isn't supposed to) act as a server.
If you would like to send messages to your app directly, then your application would need to have some sort of server listening in some port. This is bad because:
connections are usually firewalled, you cant just listen in some port,
your device can be turned off or without connectivity (then you app sever would need to retry),
the app server would need to know the address of your device,
app would need to be running (at least the server module) all the time, this isn't battery friendly.
I have an application where I have potentially many processes must send a message to one process. I figured the most expedient way to do this would be to just send a UDP datagram.
When opening a socket, I need to specify which port to listen to. As I just want to fire and forget the UDP datagram, I'd like to just have the OS pick an available port. I'd rather not have to hunt for one, especially as many instances of my process may be running. Is there a way I can tell the OS to just pick an available port (or better yet, not even open a listening socket) to fire and forget my UDP datagram?
Yes. Specify 0 as the port. The OS will pick an available port for you.
Answering the "Problem" rather than the "Question"
If all the processes are operating on the same PC, you might want to look into shared files or some other means of communications other than a networking stack.
I'd suggest you explore the options in Boost's Interprocess library
http://www.boost.org/doc/libs/1_37_0/doc/html/interprocess.html
Quick guide here:
http://www.boost.org/doc/libs/1_37_0/doc/html/interprocess/quick_guide.html
Specifically, I suggest you use a named pipe or shared memory between your processes.