I have batches of 500 messages. And to send them I am using the external API which allows sending only 1 message at a time. Also, they have a rate limit of 10/seconds.
If there is a single instance then I can handle the rate limit by adding a delay between the API calls. But in my case number of instance are not fixed. It totally depends upon the traffic I receive.
Let's say I have 10 instances running and they are processing messages in batches of 500 messages. So, I have to process 5000 messages in total. But the rate limit is 10/seconds so if all the instance invokes the same API with the same credentials then the rate limit get exceeded after a single API call of every instance.
When It will try to send the second message then they will get the 429 error as the rate limit is exceeded.
Now, I want to make sure that combining all the 10 instances will only send the 10 messages per second. How can I implement it?
Any better suggestions or recommendations are appreciated!!
Related
Background
I have an application that send HTTP request to foreign servers. The application communicating with other services with strict rate limit policy. For example, 5 calls per second. Any call above the allowed rate will get 429 error code.
The application is deployed in the cloud and run by multiple instances. The tasks are coming from shared queue.
The allowed rate limit synced by Redis Rate Limit pattern.
My current implementation
Assuming that the rate limit is 5 per second: I split the time into multiple "window". Each window has maximum rate of 5. Before each call I checking if the counter is less then 5. If yes, fire the request. If no, wait for the next window (after a second).
The problem
In order to sync the application around the Redis, I need to Redis calls: INCR and EXPR. Let's say that each call can take around 250ms to be returned. So we have checking time of ~500ms. Having said that, in some cases you will check for old window because until you will get the answer the current second has been changed. In case that on the next second we will have another 5 quick calls - it will lead to 429 from the server.
Question
As you can see, this pattern not really ensuring that the rate of my application will be up to 5 calls\second.
How do you recommend to do it right?
my app listen requests with telegram messages URL inside and process messages in loop:
get entity for group
search +/- 2 messages from target message
get entity for each recieved message author (in fact find entity to each uniqual from_id)
I use client.get_messages() and client.get_entity() methods ans sleep 10-15 secs between each loop.
And after 2-3 hours without any alerts (floodwait to 10sec or 5 minuts) I getting floodwait error with insane timeout (~22 hours).
I not trying to send spam, in fact I don`t sent any messages at all .
Where I can find limits to use get_entity methods?
Or may be using this method is overkill and user info may be finded someother method
I suggest you to take a look at the Telethon documentation: in there there are a few tips that let you avoid the limits using get_entity combined with get_input_entity.
When dequeuing from RabbitMQ it seems that the entire queue of messages is taken into memory and then processed, a minimal example can be the tutorial at https://www.rabbitmq.com/tutorials/tutorial-two-dotnet.html.
Giving for example 1000 messages, they are taken all together in the event Received and the event handler manages 1000 messages: instead, I would to take 100 messages at time in order to divide the message consuming among 2 or more servers using a load balancer as HAProxy.
Is it possible or the only possibility is in the event handler make a manual round robin to other servers?
You have to use the prefecth to decide how many messages you want to download to the client.
instead, I would take 100 messages at a time in order to divide the message consuming among 2 or more servers.
it is what is explained here of "Fair dispatch" section, in your case you have to set the prefetch to 100 instead of 1 and attach more consumers to the same queue.
I'd like to know what Marketo means by 10 concurrent API calls. If for example 20 people use an API in the same time, is it going to crash ? And if I make the script sleep for X seconds if I get that limit response and try the API call again, will it work ?
Thanks,
Best Regards,
Martin
Maximum of 10 concurrent API calls means, that Marketo will process only 10 simultaneous API requests per subscription at maximum.
So, for example if you have a service that directly queries the API every time it is used, and this very service gets called 11 or more times in the same time, than Marketo will respond with an error message for the eleventh call and the rest. The first 10 calls should be processed fine. According to the docs, the error message the following requests will receive will have an error code of 615.
If your script is single threaded (like standard PHP) and you have more that 10 API calls, and your script is running in one instance, than you are fine, since the calls are performed one after another (so they are not concurrent). However, if your script can run in multiple instance you can hit the limit easily. In case a sleep won't help you, but you can always check the response code in your script and retry the call if it received an error. This retry process is often called Exponential Backoff. Here is a great article on this topic.
I've seen the 600 calls / 600 seconds rate limit mentioned by some (e.g. on quora).
What I want to know is whether I am allowed to do 600 batch requests in 600 secs (a batch request consists of up to 50 requests).
You should handle the rate limiting programmatically by checking for the following error message. You should then put in a time-wait loop before your next call if you encounter the error. One of my high traffic applications accounts watches for this error and will slow down.
From: https://developers.facebook.com/docs/bestpractices/
Rate limited (API_EC_TOO_MANY_CALLS) If your application is making too many calls, the API server might rate limit you automatically,
returning an "API_EC_TOO_MANY_CALLS" error. Generally, this should not
happen. If it does, it is because your application has been determined
to be making too many API calls. Iterate on your code so you're making
as few calls as possible in order to maintain the user experience
needed. You should also avoid complicated FQL queries. To understand
if your application is being throttled, go to Insights and click
"Throttling".
edit
As reported by Igy in the comment thread, each request in that batch counts as 1. For your example of 600 being the max limit, that means you can fire off 15 batch requests containing 50 calls each.
According to FB docs, each element in a batch counts as a separate call.
We currently limit the number of requests which can be in a
batch to 50, but each call within the batch is counted separately for
the purposes of calculating API call limits and resource limits. For
example, a batch of 10 API calls will count as 10 calls and each call
within the batch contributes to CPU resource limits in the same
manner.
Quoted from: https://developers.facebook.com/docs/reference/api/batch/
I don't have empirical evidence however.
David
From my experience, they count individual requests regardless the way they were made (in batch or not).
For example, if I'm trying to do 1 batch/sec containing 10 requests each, I soon get 'TOO MANY CALLS'.
If I'm doing 1 batch/10 sec, each batch containg 10 requests, I never see TOO MANY CALLS.
I personally do not see any reason to prefer batches over regular API calls.
I have quite a big and painful experience now with the Facebook API and I can state that :
If a batch request contains 50 requests, then it counts as 50 requests on Facebook
1 request != 1 call. Facebook has its own definition of what a call is. If your request is big, return a lot of data or consume a lot of cpu then it will count as several calls.
The most frequent graph API call I am doing contains a lot of nested fields and I have noticed that I reached the "600 calls / 600 seconds" after doing it only 200 times. So basically this call count for 3 in my case...
You have a lot of other rate limits but none of of them are properly documented...
Batch calls definitely are counted per item in the batch. One batch call with 50 items is the equivalent of 50 api calls using the graph.