Experienced a weird problem with BigQuery UI this morning - for a specific project, all Job History both Personal and Project has disappeared. Load jobs are still showing up in the last month when i use the BQ LS command.
Has anyone seen this before, any advice? I've raised a call with the service desk but wondered what you guys think.
best wishes
Dave
It seems to be a bug in the BigQuery UI, there is a public issue reporting this scenario [1], as a workaround you can list all the jobs by using the CLI [2]
[1] https://issuetracker.google.com/118569383
[2] https://cloud.google.com/bigquery/docs/managing-jobs#listing_jobs_in_a_project
Turns out there is a shared, 1000 query / job list maximum in the UI. As we had begun to run many hundreds of queries per day, this was 'pushing' the job queries out of the list. I've requested this to be increased as a feature request in future
best wishes
Dave
Related
We use python to programmatically grant authorized view / routine access to a large number of views to various datasets.
However since this week we have been receiving the following error :
Dataset time travel window can only be modified once in 1 hours. The previous change happened 0 hours ago
This is preventing our current deployment process.
And so far we have not been able to find a work around to resolve this error. Note that we do not touch the time travel configurations at all as a part of our process.
This seems to be an issue with the BigQuery API.
Google have said that they will be rolling back the breaking change to restore functionality within the day
I'm in a situation where I have a server running sql 2012 with roughly two hundred scheduled jobs (all are SSIS package executions). I'm facing a directive from management where I need to run some custom software to create a bug report ticket whenever a job fails. Right now I'm relying on half the jobs jobs notifying an operator on failure, while the other half do like a "go to step X- send failure email" for each step on failure, where "step X" is some sql that queries the DB and sends out an email saying which job failed at which step.
So what I'm looking for is some universal solution where I can have every job do the same thing when it fails (in this case, run some program that creates a bug tracking ticket). I am trying to avoid the situation where I manually go into every single job and add a new step at the end, with all previous steps changing to "go to step Y on failure" where step Y is this thing that creates the bug report.
My first thought was to create a new job that queries the execution history tables and looks for unhandled failures and then does the bug report creation itself. However, I already made the mistake of presenting this idea to the manager and was told it's not a viable solution because it's "reactive and not proactive" and also not creating tickets in real-time. I should know better than to brainstorm with non-programming management but it's too late, so that option is off the table and I haven't been able to uncover any other methods.
Any suggestions?
I'm proposing this as an answer, though it's not a technical solution. Present the possible solutions and let the manager decide:
Update all the Agent Jobs - This will take a lot of time and every job will need to be tested, which will also take a lot of time. I'd guess 2-8 weeks depending on how it's done.
Create an error handler job that monitors the logs and creates tickets based on those errors. This has two drawbacks - it is not "real-time" (as desired by the manager) and something will need to be put into place to insure errors are only reported once. This has the upside of being one change to manage. Also it can be made near real time if it were run on the minute.
A third option, which would be more a preliminary step, is to create an error report based off of the logs. This will help to understand the quantity and types of failures. This may help to shape the ultimate solution - do we want all these tickets, can they be broken up into different categories, do we want tickets for errors that are self-healing (i.e. connection errors which have built-in retries)?
I need some help to understand what happened to our cloud to have BigQuery resource running every 12h to our cloud without configuring it. Also, it seems very intense because we got charged, in average, one dollar every day for the past month.
After checking in Logs Explorer, I saw several logs regarding the BigQuery resource
I saw the email from one of our software guy. Since I removed him from our Firebase project, there is no more requests.
Though, that person did not do or configure anything regarding the BigQuery so we are a bit lost here and this is why we are asking some help to investigate and understand what is going on.
Hope you will be able to help. Let me know if you need more information.
Thanks in advance
NB: I did not try to add the software guy's email yet. I wanted to see how it will go for the rest of the month.
The most likely causes I've observed for this in the wild:
A scheduled query was setup by a user.
A data studio dashboard was setup and configured to periodically refresh data in the background.
Someone's setup a workflow the queries BigQuery, such as cloud composer or a cloud function, etc.
It's also possible its just something like a script running in a crontab on someone's machine. The audit log should have relevant details like request origin for cases where it's just something running as part of a bespoke process.
I have a batch process that does a json job load usually once per hour. My process checks to see if there's a previous job still pending or running before it continues. It's not uncommon for jobs to be in PENDING state for some time before completing, I have seen upwards of an hour to a few hours.
In this case I have a job that has been in "RUNNING" state for about 4 hours now, I've never seen it quite this long. I have checked other posts here to see if anyone else had the issue and I did find one chap who's job took about 4 hours to complete.
My job ID for the job in question is: #job_Canl6-TBi4F6sWDOjB6ng7PxoZA. I know about why jobs can be stuck in PENDING states due to queue times, but I was not aware this was the case in the RUNNING state too - can anyone confirm in their experience that this is not abnormal? In my experience (I have been running this process for over a year) it is and can anyone confirm this is not a current back-end issue with BigQuery?
Many thanks in advance.
This issue should be resolved now. See comments #9 and #10 here:
http://code.google.com/p/google-bigquery/issues/detail?id=103
In the future, I'd recommend filing a bug via BigQuery's public issue tracker to notify us of bugs or other issues with the service. Your issue will likely get more prompt attention from the team that way. Also, StackOverflow is more appropriate for questions about BigQuery usage that will be relevant long-term; I suspect the moderators would prefer we not use StackOverflow as a bug tracker.
The public issue tracker is here:
https://code.google.com/p/google-bigquery/issues/list
Thanks, and apologies for the trouble today!
This is really a question for Jordan Tigani and Google's BigQuery support that recommends we use stackoverflow:
I have a BigQuery job that has been executing daily for the past several months but now has started erroring out with an internal 500 error. One example is job id job_4J9LL4vp3xtM30WgqduvQqFFUN4 - would it be possible to know why this job is causing an internal bigquery error and if there's something I can do to fix it?
This is the same issue as bigquery error:Unexpected. Please try again
We've got a fix that we're currently testing, it should go out in this week's release. Unfortunately, there is no workaround, other than to use a table decorator that doesn't include the latest streaming data.