I am creating a Python script that gets information from one API and after some formatting and etc it writes the data in a spreadsheet in Google Sheets. I am using the Gspread library and the script is working tho there is one issue that is actually a 'dealbreaker'. The G Sheets API has limitations of read and write requests per minute or per 100 seconds. The limit is 100 requests per 100 seconds per user. I've been monitoring my request and I have never exceeded this limitation but I get the 429 error - "requests quota exceeded".
So my question would be, could it be the case that 100 request per 100 seconds resonates to 1 request per 1 second so that when my script does 2 or more request in 1 second the 429 error is raised?
What lead me to the thought this could be the case is that I tried using time.sleep with many different values so that I make sure I never exceed even 60 requests per minute but I am still getting the 429 error raised.
It has a spam protection system which limits to 10 requests per second.
I found out after hours of searching, I was stuck at the same.
You can delay your calls and prevent that.
Related
I have a problem with the Google Calendar API.
We are using the API with OAuth2 authentication in Python code.
It was working fine for about three years, but after today, it started to output the following error log.
<HttpError 403 when requesting https://www.googleapis.com/calendar/v3/calendars/**********************/events?alt=json returned "Quota exceeded for quota metric 'Queries' and limit 'Queries per day' of service 'calendar-json.googleapis.com' for consumer ‘project_number:*************'.">
However, the above log is not always output, and the error rate is about 30%.
We have set the allocation limit to 1,000,000, and there are about 3,000 queries for 24 hours.
Even though the number of times per day has not reached the limit, we are being told to use up our Quota for the day.
The API methods we are using are as follows
calendar.v3.Events.Insert
calendar.v3.Events.Get
calendar.v3.Events.Update
Delete calendar.v3.Events.
calendar.v3.CalendarList.
List calendar.v3.Calendars.
List calendar.v3.Acl.
Calendars.Insert calendar.v3.
Calendars.v3.Events.Insert is requested about 2000 times in 24 hours.
Calendars.Insert has been requested 2 times in 24 hours.
Has anyone encountered this before?
Thank you.
We have the exact same problem here.
Everything was working fine and suddenly starting this morning we have these 403 errors (RateLimitExceeded) coming up, with a 50% rate.
Our API usage is exactly the same as before. Our quota sits at 600 requests/minute/user and 1000000 requests per day. All we do is manual, so I don't see how we got past that limit suddenly...
It looks like other people have the same problem, it must be a bug in Google APIs.
The bug has been reported to Google already, see here: https://issuetracker.google.com/issues/182497593
I suggest you star this issue and wait for Google's answer.
I used GetTwitter processor in Apache Nifi to process live tweets using UserId. Initially it works fine but after few hours no tweets is retrieved from Twitter.
May I know why I am getting this issue.
Note:
Since I am using 25000 user id to fetch live tweets from them. As I referred some blogs it shows 5000 UserId is maximum limit for GetTwitter processor. I used 5 GetTwitter processor with same access credentias to divide the 25000 user ids to 5 parts.
It sounds like you are encountering a rate limiting issue. Twitter's documentation on it can be found at https://developer.twitter.com/en/docs/basics/rate-limiting
In general, for our usage of the Google Sheet API, the data appears to be eventually consistent-- that is, updates in the UI (Chrome) take some time to be reflected when the sheet is read via the API. In some instances, this delay is roughly half an hour.
I was just wondering if there's an expected or maximum value for this delay or if it's just "best effort".
I've found this question that mentions eventual consistency with the Sheet API, but that's about it.
I'm using version 4 of the Java API to access the data.
According to the Google Docs documentation:
Functions that pull data from outside the spreadsheet recalculate at the following times:
ImportRange: 30 minutes
ImportHtml, ImportFeed, ImportData, ImportXml: 1 hour
GoogleFinance: may be delayed up to 20 minutes
It turns out that the issue reported by the user dealt with roughly a half-hour delay with the ImportRange function.
I have routines that synchronize Class/Roster information between an SIS and Google Classroom. Everything has been running smoothly until very recently (11/1/2016). Now we're seeing the following message in all of our log files for routines that handle Classroom syncs.
Insufficient tokens for quota group and limit 'DefaultGroupUSER-100s' of service 'classroom.googleapis.com', using the limit by ID...
We perform batch requests whenever possible and these errors are showing up in individual batch "part" responses. The fact that these errors suddenly started showing up for ALL of our Classroom routines makes me think that something changed on the Google end of things.
I've been playing around with the throttling on our end by changing both the number of requests that we send in each batch (docs say that you can send 1000 per batch) as well as the total number of requests/batches that we're sending per 100 seconds (docs say you can send 50/s/client and also 5/s/user). Interestingly, the quotas indicated in the development console display slightly different but I assume they are to be interpreted in conjunction with one another.
I've throttled things down to the point where we're not even getting close to 5 requests per second and I'm still getting these errors back from the server.
Can someone provide some suggestions or solutions. Has anyone experienced this lately?
Let me know if I any additional information is needed.
I'm trying to write an application that creates mail accounts for thousands of users using the Google Directory API. Creating them one by one works, but is extremely slow. I tried to use the batch requests which is suppose to support up to 1000 requests at once. However with that, only around 50 users are created successfully and the rest of the requests throw 403 errors. If I change the batch size to 40 instead, after the first batch, many requests fail with 5xx errors.
If the batch requests are still limited by the same rate limits, the seem to be worthless as I could just send those requests individually at that slow rate. Is there a better way to do this or is there something else I should do instead?
Batching the requests will certainly save network roundtrips (which can be pretty expensive if you have thousand of users to process). However, the server will still have to execute the request one by one even if it is batched. Take a look at the documentation on Admin SDK
https://developers.google.com/admin-sdk/directory/v1/guides/batch
The special note said: "A set of n requests batched together counts toward your usage limit as n requests, not as one request. The batch request is taken apart into a set of requests before processing."