The problem: Running into Google Sheets API read/write quota limits. Specifically, the read/write requests per 100 seconds and read/write requests per 100 seconds per user quotas.
Some background:
For the past few months I've been developing a web app for students and staff in our school district which uses a Google spreadsheet as the database. Each school in our district was assigned a different Google spreadsheet, and a service account was created to make read and write calls to these spreadsheets on behalf of the web app.
We started with one school of approximately 1000 students, but it has now expanded to two other schools with a total user load of around 4000. Due to the nature of a school day schedule, we started hitting our quota limit (per 100 sec & per 100 sec per user) since almost everyone uses the app at the same time.
I found the usage limits guide for the Google sheets API, and as per the instructions I created a billing account, and linked the associated service account project to it. I then went to the quotas section in the developers console and applied for a higher quota. This involved filling out a Google form which asked "How much quota do you need? Express in number of API queries per day." Again, queries per day is not the problem, rather it's the number of queries per 100 seconds and per user (service account). After a couple of weeks our limit was increased to 2500 read/write requests per 100 seconds and 500 read/write requests per 100 seconds per user. The billing account was not charged, and after a little searching, I realized this was a free increase. This bump in our quota limit helped, but it's still going to be an issue because our district wants to add more schools in the future.
Here's what I need to know:
1) [ESSENTIAL QUESTION] Does Google have an upper limit or maximum to the number of read/write requests a single service account/user/IP can make within the 100 second time frame, and if so what is it?
2) If it is possible to go beyond our current quota limit (2500/500), is there another way of requesting/applying for the increase. Once again we have a billing account established for the project and are willing to pay for the service.
I've been pulling (what's left of) my hair out trying to find definitive answers to my questions. This post came close to what I was looking for, and I even did some of the things the OP suggested, but I just need a direct answer to my "essential" question.
Couple more things.
I understand that Google Charts Visualization doesn't have a quota limitation, and I'd consider using it however, for privacy reasons I can't have the spreadsheet keys exposed in plain javascript. Are there other options here?
Also, one might suggest creating multiple service accounts, but I'd rather avoid this if possible.
Thank you for your help. I'm very much a novice and I greatly appreciate your time and expertise.
To answer your questions:
1) [ESSENTIAL QUESTION] Does Google have an upper limit or maximum to the number of read/write requests a single service account/user/IP can make within the 100 second time frame, and if so what is it?
*The provided documentation only stated that Google Sheets API has a limit of 500 requests per 100 seconds per project, and 100 requests per 100 seconds per user. Check this post for additional information.*
2) If it is possible to go beyond our current quota limit (2500/500), is there another way of requesting/applying for the increase. Once again we have a billing account established for the project and are willing to pay for the service.
AFAIK, you can request for a higher quota limit and the Google Engineers may grant the request as long as you are making a reasonable request.
Also, you may check this thread for additional tips:
You can use spreadsheets.get to read the entire spreadsheet in a single call, rather than 1 call per request. Alternately, you
can use spreadsheets.values.batchGet to read multiple different
ranges in a single call, if all you need are the values.
The Drive API offers "push notifications", so you can get notified when changes occur and react to those, instead of polling for
them. The latency of the notifications is a little on the slow side,
but it gets the job done.
Related
According to this link, it says that JSON Custom Search API provides 100 search queries per day for free. If you need more, you may sign up for billing in the API Console. Additional requests cost $5 per 1000 queries, up to 10k queries per day.
I have been using the JSON Customsearch API which is https://www.googleapis.com/customsearch/v1 for quite a while now. Yesterday, I fired around 360 requests. So does it mean that since I crossed 100 search queries, I'll be charged for it?
The reason that I asked this was because my Dashboard doesn't show any charges yet.
Yes, you need to pay it.
According to my experience, the charge information on Dashboard do not refresh all the time.
I want to make a website that will fetch data from Google spreadsheet. So, my question is, how many simultaneous requests for JSON data can Google sheets handle? I will write a script in Google sheets to automatically update data every hour in the sheet.
Thanks :)
If you are pertaining to requests limit, you may check this documentation about Usage Limits.
This version of the Google Sheets API (v4) has a limit of 500 requests per 100 seconds per project, and 100 requests per 100 seconds per user. Limits for reads and writes are tracked separately. There is no daily usage limit.
To view or change usage limits for your project, or to request an increase to your quota, do the following:
If you don't already have a billing account for your project, then create one.
Visit the Enabled APIs page of the API library in the API Console, and select an API from the list.
To view and change quota-related settings, select Quotas. To view usage statistics, select Usage.
Hope this helps!
Would someone be kind enough to tell me whether the Google API usage limits specified here: https://developers.google.com/maps/documentation/geocoding/usage-limits are calculated set at the project level, or account level please?
I'm using one API key for several maps on our website. Total calls per day limit is no problem at all. We're occasionally clocking more than 50 requests per second in peak times though.
If I create a new project, and get a new API key in the same account, will that mean we can hit 50 requests per second on one API key, and 50 requests per second separately on another API key...or are they calculated at the account level?
Many thanks everyone!
The documentation states the following:
Most of the Google Maps APIs have a complimentary per-day quota that can be set in the Google API Console. The daily default and maximum query limits vary by API. You can increase the complimentary daily limits by enabling billing, or purchasing a Google Maps APIs Premium Plan license. Quota limits are enforced on a unique project basis, and you may not take any action to circumvent quota limits. For example, you may not create multiple projects to compound and exceed quota limits.
https://developers.google.com/maps/faq#usage-limits
So, as you can see the usage quota is calculated on the per project basis. If you use two API keys from different projects each one will have its own usage limits. Also you cannot create unlimited number of project for one account. As far as I know you can create approximately 16 projects within one account.
I hope this clarifies your doubt.
The usage limits are calculated at the account level, not the project or key level. They do this to prevent people from just creating unlimited projects to get around the acceptable usage limits that they are providing.
If I may ask, I was wondering how the API request limit handles multiple store calls?
Scenario:
We have one backend service "polling" a store with one core request and 'x' number of requests for images by productId for each item in "LineItems" (I doubt this will be an extraordinary figure) every 5 seconds, but I'm curious to know if we had Five stores and Five background service polling respective stores, would this total to the request limit? I.E how is this tracked, by IP?
I'm hoping that it's on a per store basis thus other stores have there own "Bucket". I have read through "Some" documentation but not sure that info is giving the knowledge I require.
There a lot of older posts and articles, all with conflicting info.
I fully appreciate this is not a programming issue per-say but was hoping this was the place for such a question. As ever, appreciate everyone's time.
Regards,
All shops are provided with a two API calls per second limit, after you've hammered your bucket for 40 other calls. If you are polling a shop every 5 seconds, then you are in no danger of hitting any limits. Any one App can call N stores and this limit is per store, not per App.
Anybody know if there is any API call limits per second, hour or day for Tumblr API? It seems to me the limits do exist when I make a lot of api calls in a short period via OAuth. However, I couldn't find any document on Tumblr API website or on Google. Many thanks.
I have been using Tumblr API for about 2 years now, and I must admit that "Rate Limit Exceeded" issue has no deterministic and, more important, officially confirmed answer.
In Tumblr's API Agreement you may find some reference to limitations under section "Respect for Limitations" which says
In addition, you will comply with any limitations imposed by Tumblr on the frequency of access, calls and use of the Tumblr API and Tumblr Firehose
We ask that you respect these limitations, as well as any rate limits that we may place on actions, which are designed to protect our systems
Notes:
There is a special Tumblr tagged blog "rate-limit-exceeded" dedicated to this. However, it does not say much about number of request per period of time that a reported person used when facing this problem.
For example here you can find avg 1000 requests per minute to be the limit.
As for my application the request rate is approximately 1 request per second. The application runs for about a year already in 24/7 manner. There were several times though this issue occurred to me even with this relatively low rate. However, I consider the failure rate to be insignificant.
From: https://www.tumblr.com/oauth/apps
Newly registered consumers are rate limited to 1,000 requests per hour, and 5,000 requests per day.
If you go to that link it looks like you can get the rate limit removed if you ask nicely! :)