I'm creating a multiplayer game and I want players to send a request to Redis to handle their join request. The first backend game will then need to connect the player and cancel the request so that the request won't be executed on every backend server (no racing conditions). How can I achieve this without racing conditions?
What I think the process would be is to:
when a backend game receives the request, it should handle the request, checking if there's space for creating/loading a game.
if it can handle the request, check if the player is still in the join queue, and attempt to lock the request if it's still unlocked (failed to lock will cancel the operation for this specific game, but the request will still be readable by other games).
The locking process is basically: attempt to SETNX <lockName> <expiryEpoch> until acquire timeout. <lockName> is the common lock name (e.g. "games:requests:join:<playerId>") and <expiryEpoch> is when the lock will expire. More about locking
if successfully locked: send the player to the game, unlock/delete the lock, and remove the player from the join queue.
Is there anything I should know about when using this process? Are there any possible bugs/exploits I miss?
Related
The issue is we have some modern web applications that are integrated with a legacy system that was never designed to support multiple concurrent requests from a single user. Basically there are certain types of requests that the legacy system can only handle one-at-a-time from a single user. It can handle multiple concurrent requests coming from different users, but for technical reasons cannot handle multiple from a single user. In these situations, the user's first request will complete successfully, but any subsequent requests from that same user that come in while the first request is still executing will fail.
Because our apps are ajax enabled, multi-tab/multi-browser friendly, and just the fact that there are multiple apps - there are certain scenarios where a user could wind up having more than one of these types of requests being sent to the legacy system at the same time.
I'm trying to determine if something like RabbitMQ could be positioned in front of the legacy system and leveraged to single-thread requests per user/IP. The thinking being that the web apps would send all requests to MQ, and they'd stack into per-user queues and pass on to the legacy system one at a time.
I don't know if there would be concerns about the potential number of queues this could create - we have a user-base of approx 4,000.
And I know we could somewhat address this in the web apps individually, but since there are multiple apps it'd be duplicating logic across them, and you'd still have the potential for two different apps to fire off concurrent requests.
Any feedback would be appreciated. Thanks-
I'm not sure a unique queue per user will work as you would need to have a backend worker process listening for messages on that queue that would need to be dynamically created.
Below is one option but it does have a performance bottleneck potential as a single backend process would be handling all requests sequentially. You could use multiple worker processes but you wouldn't know if one had completed before the other causing a race condition if your app requires a specific sequence of actions.
You could simply put all transactions (from all users) into a single queue and have a backend process pull off of that queue and service the request. If there needs to be a response back to the user once the request was serviced, then the worker process could respond back to a separate queue with a correlationID that could be used to send the response date back to the correct user.
I've done this before with ExpressJS apps where the following flow would happen:
The user/process/ajax makes a request
Express takes the payload from the request object and sends it to a RabbitMQ queue with a unique correlationId (e.g. UUID).
Express then takes the response object and stores it in a responseStore object with the key being the correlationId
Meanwhile, a backend worker process pulls the item from the queue, does some work and then sends a message to a different response queue with the same correlationId
The ExpressJS application has a connection to the response queue and when it receives a message, it takes the correlationId from the response and looks for a response object stored with same correlationId in the responseStore. If it finds it, it takes the payload from the message and does something like response.send(payload) or response.json(payload)
To do this, you should also have a mechanism that stores the creation time of the response object in the responseStore along with the response object. Then have a separate process that will check the responseStore and clean up old response objects after a certain timeout in case there are issues with the backend process completing.
Look here for more info on RPC with RabbitMQ:
https://www.rabbitmq.com/tutorials/tutorial-six-javascript.html
Hope this helps.
I will be implementing a Java program that acts as a gemfire client. The program will continuosly process records that it receives on its port from a remote program. Each record will be processed using the static data cached with my program. The cache may get updated behind the scenes in my program when it is changed on the gemfire server. The processing of one record may take a few seconds. I run the risk of processing half the record with static data that was prevalent before the change and rest of the record with static data that has taken effect after the change. Is there a way I can tell gemfire to not apply the cache to the local client until I am done processing the ongoing record?
Regards,
Yash
Consider this approach: Use a Continuous query "Select *" instead of event registration. A CQ does not update the client region like a subscription does. Make your client region LOCAL. After receiving the CQ event on the client, execute your long running process and put the value that you received from the CQ into your client region. Decoupling client and server in this way will allow your client to run long-running processes.
Alternatively: if you must have the client cache proxied with the server as an absolute requirement, then keep the interest registration AND register a CQ. Ignore the event callback from the subscription but handle your long-running process using the event callback from the CQ.
The following is from page 683 at http://gemfire.docs.pivotal.io/pdf/pivotal-gemfire-ug.pdf
CQs do not update the client region. This is in contrast to other server-to-client messaging like the updates sent to satisfy interest registration and responses to get requests from the client's Pool.
When subscribing to real-time notifications, I go through the normal handshake, subscribe, connect flow.
Once the connection returns with events, I reconnect and wait for the next response to return. My question is:
If events are generated the first response and the next reconnect, could they be lost?
As an example: A synchronous application which processes returned response data after it returns and only reconnects once the data processing has finished could cause a significant delay between the response and the next reconnect. Are the cumulocity events generated during that delay buffered in the real-time queue for that particular client id or are they just lost?
Another possible example is when the client ID is no longer valid (this seems to happen every day at midnight), I have to resubscribe, causing a period of time during which no one is subscribed.
The client ID that you receive when handshaking is connected to a queue on the server side. That queue keeps all notifications that you are not able to receive until the next connect. It delivers them when you reconnect. (Try it out with Postman: After a connect returns, send a couple of events, then connect again. You will notice that you will get all events at once.)
However, as you noticed, the queue is not kept forever. If you are not able to reconnect within two hours (I believe), the queue is thrown away in order to not block server resources. This is what you noticed. In that case, you need to query the database to determine any missed events (e.g., poll any operations in pending state from devices).
Everytime the client/browser connects to
Mochiweb server, it creates new process of Loop, doesn't it? So, if I want
to transfer a message from one client to another (typical chat system) I
should use the self() of Loop to store all connected clients PIDs, shouldn't
I?
If something(or everything) is wrong so far plz explain me briefly how the
system works, where is server process and where is client process?
How to send a message to the Loop process of client using its PID? I mean where to
put the "receive" in the Loop?
Here's a good article about a Mochiweb Web Chat implemention. HTTP Clients don't have PID's as HTTP is a stateless protocol. You can use cookies to connect a request to a unique visitor of the chat room.
First, do your research right. Check out this article , and this one and then this last one.
Let the mochiweb processes bring chat data into your other application server (could be a gen_server, a worker in your OTP app with many supervisors, other distributed workers e.t.c). You should not depend on the PID of the mochiweb process. Have another way of uniquely identifying your users. Cookies, Session ids, Auth tokens e.t.c. Something managed only by your application. Let the mochiweb processes just deliver chat data to your servers as soon as its available. You could do some kind of queuing in mnesia where every user has a message queue into which other users post chat messages. Then the mochiweb processes just keep asking mnesia if there is a message available for the user at each connection. In summary, it will depend on the chat methodology: HTTP Long polling/COMET, REST/ Server push/Keep-alive connections blur blur blur....Just keep it fault tolerant and do not involve mochiweb processes in the chat engine, just let mochiweb be only transport and do your chat jungle behind it!
You can use several data structures to avoid using PIDs for identity. Take an example of a queue(). Imagine you have a replicated mnesia Database with RAM Table in which you have implemented a clearly uniquely identifiable queue() per user. A process (mochiweb process), holding a connection to the user only holds an identity of this users session. It then uses this identity to keep checking into his queue() in Mnesia at regular intervals (if you are intending to it this way -- keeping mochiweb processes alive as long as the users session). Then it means that no matter which Process PID a user is connected through, as long as the process has the users identity, then it can fetch (read) messages from his message queue(). This would consequently result in having the possibility of a user having multiple client sessions. The same process can use this identity to dump messages from this user into other users' queues().
I have a requirement where I need to send continuous updates to my clients. Client is browser in this case. We have some data which updates every sec, so once client connects to our server, we maintain a persistent connection and keep pushing data to the client.
I am looking for suggestions of this implementation at the server end. Basically what I need is this:
1. client connects to server. I maintain the socket and metadata about the socket. metadata contains what updates need to be send to this client
2. server process now waits for new client connections
3. One other process will have the list of all the sockets opened and will go through each of them and send the updates if required.
Can we do something like this in Apache module:
1. Apache process gets the new connection. It maintains the state for the connection. It keeps the state in some global memory and returns back to root process to signify that it is done so that it can accept the new connection
2. the Apache process though has returned the status to root process but it is also executing in parallel where it going through its global store and sending updates to the client, if any.
So can a Apache process do these things:
1. Have more than one connection associated with it
2. Asynchronously waiting for new connection and at the same time processing the previous connections?
This is a complicated and ineffecient model of updating. Your server will try to update clients that have closed down. And the server has to maintain all that client data and meta data (last update time, etc).
Usually, for continuous updates ajax is used in a polling model. The client has a javascript timer that when it fires, hits a service that provides updated data. The client continues to get updates at regular intervals without having to write an apache module.
Would this model work for your scenario?
More reasons to opt for poll instead of push
Periodic_Refresh
With a little patch to resume a SUSPENDED mpm_event connection, I've got an asynchronous Apache module working. With this you can do the improved polling:
javascript connects to Apache and asks for an update;
if there's no updates, then instead of answering immediately the module uses SUSPENDED;
some time later, after an update or a timeout happens, callback fires somewhere;
callback gives an update (or a "no updates" message) to the client and resumes the connection;
client goes to step 1, repeating the poll which with Keep-Alive will use the same connection.
That way the number of roundtrips between the client and the server can be decreased and the client receives the update immediately. (This is known as Comet's Reverse Ajax, AFAIK).