I am trying to sync oneDrive Files (metadata and permissions) for a domain using MSGraph API using list, children and permission endpoints.
I am using batching for children and permission endpoints, sending 10-20 request urls in single batch requests concurrently for 10 users.
I am getting a lot of 429 errors by doing so. Though, I was also getting 429 errors on making single (non-batched) calls also.
According to the documentation related to throttling, they ask to
1. Reduce the number of operations per request
2. Reduce the frequency of calls.
So, my question is
Does a batch call of 10 get urls, count as 10 different operations and 10 different calls ?
Does a batch call of 10 get urls, count as 10 different operations and
10 different calls ?
Normally, N URLs will be treated as N+1 operations(even more). N operations from the batch URLs and one for the batch URL itself.
Pay attention to the docs:
JSON batching allows you to optimize your application by combining
multiple requests into a single JSON object.
Due to multiple requests have been combined to one request, the server side just need to send back one response too. But the underlying operation for each URL still need to be handle, so the workload on server side is still very high, just may reduce a little.
The answer lies somewhere in between.
Even though the documentation (cannot find the actual page at this moment) says you can combine up to 20 requests, I found out by experimenting that the limit is currently set to 15. So if you reduce the amount off calls in a single batch you should be good to go.
I'm not sure but it might also help to restrict the batches to a single user.
The throttling limit is set to 10000 items per 10 minutes per user resource, see this blog item
Related
Right now, I am working on an ASP.NET Core Web API that calls an external web service and uses the returned data in its own response. This is working fine.
However, I discovered that the external service is not as scalable as I would like to. Therefore, as discussed with the company providing this external service, the number of outgoing requests needs to be limited to one per second. I als use caching to reduce the number of outgoing requests but this has been shown to be not effective enough because (as logically) it only works when a lot of requests are the same so cache data can be reused.
I have been doing some investigation on rate limiting but the documented examples and types are far more complicated than what I need. This is not about partitions, tokens or concurrency. What I want is far more simple. Just 1 outgoing request per second and that´s all.
I am not that experienced when it comes to rate limiting. I just started reading the referred documentation and found out that there is a nice and professional package for it. But the examples are more complicated than what I need for the reasons explained. It is not about tokens or concurrency or so. It is the number of outgoing requests per second that needs to be limited.
Possibly, there is a way using the package System.Threading.RateLimiting in such a way that this is possible by applying the RateLimiter class correctly. Otherwise, I may need to write my own DelegateHandler implementation to do this. But there must be a straightforward way which people with experience in rate limiting can explain me.
So how to limit the number of outgoing web request per second?
In addition, what I want to prevent is a 429 or so in case of to many request. In such a situation, the process should just take more waiting time in order to complete so the number of outgoing requests is limited.
I'd like to know what Marketo means by 10 concurrent API calls. If for example 20 people use an API in the same time, is it going to crash ? And if I make the script sleep for X seconds if I get that limit response and try the API call again, will it work ?
Thanks,
Best Regards,
Martin
Maximum of 10 concurrent API calls means, that Marketo will process only 10 simultaneous API requests per subscription at maximum.
So, for example if you have a service that directly queries the API every time it is used, and this very service gets called 11 or more times in the same time, than Marketo will respond with an error message for the eleventh call and the rest. The first 10 calls should be processed fine. According to the docs, the error message the following requests will receive will have an error code of 615.
If your script is single threaded (like standard PHP) and you have more that 10 API calls, and your script is running in one instance, than you are fine, since the calls are performed one after another (so they are not concurrent). However, if your script can run in multiple instance you can hit the limit easily. In case a sleep won't help you, but you can always check the response code in your script and retry the call if it received an error. This retry process is often called Exponential Backoff. Here is a great article on this topic.
Our organisation is currently migrating to Apigee.
I currently have a problem very similar to this one, but due to the fact that I am a Stack Overflow newcomer and have low reputation I couldn't comment on it: Apigee - SpikeArrest behavior
So, in our organisation we have 6 MessageProcessors (MP) and I assume they are working in a strictly round-robin manner.
Please see this config (It is applied to the TARGET ENDPOINT of the ApiProxy):
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SpikeArrest async="false" continueOnError="false" enabled="true" name="spikearrest-1">
<DisplayName>SpikeArrest-1</DisplayName>
<FaultRules/>
<Properties/>
<Identifier ref="request.header.some-header-name"/>
<MessageWeight ref="request.header.weight"/>
<Rate>3pm</Rate>
</SpikeArrest>
I have a rate of 3pm, which means 1 hit each 20sec, calculated according to ApigeeDoc1.
The problem is that instead of 1 successful hit every 20sec I get 6 successful ones in the range of 20sec and then the SpikeArrest error, meaning it hit once each MP in a round robin manner.
This means I get 6 hit per 20 sec to my api backend instead of the desired 1 hit per 20sec.
Is there any way to sync the spikearrests across the MPs?
ConcurrentRatelimit doesn't seem to help.
SpikeArrest has no ability to be distributed across message processors. It is generally used for stopping large bursts of traffic, not controlling traffic at the levels you are suggesting (3 calls per minute). You generally put it in the Proxy Request Preflow and abort if the traffic is too high.
The closest you can get to 3 per minute using SpikeArrest with your round robin message processors is 1 per minute, which would result in 6 calls per minute. You can only specify SpikeArrests as "n per second" or "n per minute", which does get converted to "1 per 1/n second" or "1 per 1/n minute" as you mentioned above.
Do you really only support one call every 20 seconds on your backend? If you are trying to support one call every 20 seconds per user or app, then I suggest you try to accomplish this using the Quota policy. Quotas can share a counter across all message processors. You could also use quotas with all traffic (instead of per user or per app) by specifying a quota identifier that is a constant. You could allow 3 per minute, but they could all come in at the same time during that minute.
If you are just trying to protect against overtaxing your backend, the ConcurrentRateLimit policy is often used.
The last solution is to implement some custom code.
Update to address further questions:
Restating:
6 message processors handled round robin
want 4 apps to each be allowed 5 calls per second
want the rest of the apps to share 10 calls per second
To get the kind of granularity you are looking for, you'll need to use quotas. Unfortunately you can't set a quota to have a "per second" value on a distributed quota (distributed quota shares the count among message processors rather than having each message processor have its own counter). The best you can do is per minute, which in your case would be 300 calls per minute. Otherwise you can use a non-distributed quota (dividing the quota between the 6 message processors), but the issue you'll have there is that calls that land on some MPs will be rejected while others will be accepted, which can be confusing to your developers.
For distributed quotas you'd set the 300 calls per minute in an API Product (see the docs), and assign that product to your four apps. Then, in your code, if that product is not assigned for the current API call's app, you'd use a quota that is hardcoded to 10 per second (600 per minute) and use a constant identifier rather than the client_id, so that all other traffic uses that quota.
Quotas don't keep you from submitting all your requests nearly simultaneously, and I'm assuming your backend can't handle 1200+ requests all at the same time. You'll need to smooth the traffic using a SpikeArrest policy. You'll want to allow the maximum traffic through the SpikeArrest that your backend can handle. This will help protect against traffic spikes, but you'll probably get some traffic rejected that would normally be allowed by the Quota. The SpikeArrest policy should be checked before the Quota, so that rejected traffic is not counted against the app's quota.
As you can probably see, configuring for situations like yours is more of an art than a science. My suggestion would be to do significant performance/load testing, and tune it until you find the correct values. If you can figure out how to use non-distributed quotas to get acceptable performance and predictability, that will let you work with per second numbers instead of per minute numbers, which will probably make massive spikes less likely.
Good luck!
Unlike Quota limits, the Spike Arrest cannot be synchronized across MP.
But, as you're setting them on a per minute level, you could use Quota Policy instead -- then set it to Distributed and Synchronized and it will coordinate across MP.
Keep in mind there will always be some latency on the synchronization across machines so it will never be a completely precise number.
I've seen the 600 calls / 600 seconds rate limit mentioned by some (e.g. on quora).
What I want to know is whether I am allowed to do 600 batch requests in 600 secs (a batch request consists of up to 50 requests).
You should handle the rate limiting programmatically by checking for the following error message. You should then put in a time-wait loop before your next call if you encounter the error. One of my high traffic applications accounts watches for this error and will slow down.
From: https://developers.facebook.com/docs/bestpractices/
Rate limited (API_EC_TOO_MANY_CALLS) If your application is making too many calls, the API server might rate limit you automatically,
returning an "API_EC_TOO_MANY_CALLS" error. Generally, this should not
happen. If it does, it is because your application has been determined
to be making too many API calls. Iterate on your code so you're making
as few calls as possible in order to maintain the user experience
needed. You should also avoid complicated FQL queries. To understand
if your application is being throttled, go to Insights and click
"Throttling".
edit
As reported by Igy in the comment thread, each request in that batch counts as 1. For your example of 600 being the max limit, that means you can fire off 15 batch requests containing 50 calls each.
According to FB docs, each element in a batch counts as a separate call.
We currently limit the number of requests which can be in a
batch to 50, but each call within the batch is counted separately for
the purposes of calculating API call limits and resource limits. For
example, a batch of 10 API calls will count as 10 calls and each call
within the batch contributes to CPU resource limits in the same
manner.
Quoted from: https://developers.facebook.com/docs/reference/api/batch/
I don't have empirical evidence however.
David
From my experience, they count individual requests regardless the way they were made (in batch or not).
For example, if I'm trying to do 1 batch/sec containing 10 requests each, I soon get 'TOO MANY CALLS'.
If I'm doing 1 batch/10 sec, each batch containg 10 requests, I never see TOO MANY CALLS.
I personally do not see any reason to prefer batches over regular API calls.
I have quite a big and painful experience now with the Facebook API and I can state that :
If a batch request contains 50 requests, then it counts as 50 requests on Facebook
1 request != 1 call. Facebook has its own definition of what a call is. If your request is big, return a lot of data or consume a lot of cpu then it will count as several calls.
The most frequent graph API call I am doing contains a lot of nested fields and I have noticed that I reached the "600 calls / 600 seconds" after doing it only 200 times. So basically this call count for 3 in my case...
You have a lot of other rate limits but none of of them are properly documented...
Batch calls definitely are counted per item in the batch. One batch call with 50 items is the equivalent of 50 api calls using the graph.
I receive Graph API error #613 (message: "Calls to mailbox_fql have exceeded the rate of 300 calls per 600 seconds", type:OAuthException) when testing my app. It's a desktop app, and the only copy is the one running on my machine (so there's only one access_token and one user - me).
I query the inbox endpoint once every 15 seconds or so. Combined, the app makes about 12 API calls (to various endpoints) per minute. It consistently fails on whichever call fetches the 300th thread (there are about 25 threads on the first page of the inbox endpoint, and I'm only fetching the first page). I am not batching any calls to the Graph API.
I'm developing on Mac OS X 10.7 using Objective-C. I use NSURLConnection to call the Graph API asynchronously. As far as I know, each request processed by NSURLConnection should only result in one request to Facebook's API.
Going on the above, I'm having trouble figuring out why I am receiving this error. I suspect that it is because a single call to the inbox endpoint (i.e. a call to the URI https://graph.facebook.com/me/inbox?access_token=...) is counted as more than one call to mailbox_fql. In particular, I think that a single call that returns <n> threads counts as <n> calls against mailbox_fql. If this is the case, is there a way to reduce the number of calls to mailbox_fql per API call (e.g. by fetching only the <n> most recent threads in the inbox, rather than the whole first page)?
The documentation appears to be pretty sparse on this topic, so I've had to get by mostly through trial and error. I'd be thrilled if anyone else knows how to tackle this issue.
Edit: It turns out that you can pass a limit GET parameter that, unsurprisingly, limits the number of results. However, the Developer blog notes some limitations with this approach (namely that fewer results than requested may be returned if some are not visible to your user).
The blog recommends using until and/or since as GET parameters when calling the standard Graph API. These parameters take any strtotime()-compliant string (or Unix epoch time) and limit your results accordingly.
Original answer follows:
After some further research, it looks like my options are to fetch less frequently or use custom FQL queries to limit the number of calls to mailbox_fql. I haven't been able to find any way to limit the response of the standard Graph API call to the inbox endpoint. In the present case, I'm using an FQL query of the following form:
https://graph.facebook.com/fql?q=SELECT <fields> FROM thread WHERE folder_id=1 LIMIT <n>&access_token=...
<fields> is a comma-separated list of fields (described in Facebook's thread FQL docs). thread is the literal name of the table corresponding to the inbox endpoint; the new thread endpoint corresponds to the unified_thread table, but it's not publicly available yet. folder_id=1 indicates that we want to use the inbox (as opposed to outbox or updates folders).
In practice, I'm setting <n> to 5, which results in a reasonable 200 calls to mailbox_fql in a 10-minute span when using 15-second call intervals. In my tests, I haven't been receiving error #613, so I guess it works.
I imagine that most people here were already familiar with the ins and outs of FQL, but it was new to me. I hope that this helps some other newbies dealing with similar issues!