All my datasheets, tables, and ALL items inside BQ are un EU. When I try to do a View->to->Table 15 min scheduled query I get an error regarding my location, which is incorrect, because all, source and destiny are both on EU...
Anyone knows why?
There is a transient known issue matching your situation, GCP support team needs more time for troubleshooting. There may be a potential issue in the UI. I would ask you to try the following steps:
Firstly, try to make the same operation in Chrome's incognito mode.
Another possible workaround is trying to follow this official guide using a different approach than the UI (CLI for instance).
I hope it helps.
Related
I just started a new project on Google Cloud, set up some bigquery datasets and tables. I now want to set up some scheduled queries. I have already enabled BigQuery Data Transfer API. My query is valid (it's just SELECT * FROM table). I can't find anything about this error online.
See screenshot
UPDATE: I've experimented a bit and it seems to be an organization wide issue. All projects, new and old within my organization get this same error when trying to schedule a query. I tried for a project in a different organization and did not have the issue. What could be causing this error for ALL projects in an organization?
UPDATE 2:
By querying a table that is not empty the error change to "Error creating scheduled query: Yn" instead of "Error creating scheduled query: er" (when the scheduled query would have queried an empty table).
I faced the same issue than you, and basically I just needed to run the query first before creating the the scheduled query... And that did the trick.
from the BQ FAQs :
"Scheduled queries use features of BigQuery Data Transfer Service. Verify that you have completed all actions required in Enabling BigQuery Data Transfer Service."
basically, what this means is that you need to enable the data transfer api in your project, AND give the user who creates the scheduled query a BQ admin role in order to have the right permissions to access that transfer service.
If done right, you should get a popup when creating the scheduled query to confirm that the data transfer service has access to your uses account (if you block popups you might not see this message and get stuck)
If this error only occurs in your organisation, I believe it might be caused by a organisation policy on Google Cloud. I would encourage you to double check if there is any org policy causing this error. If that's not the case, open a support ticket with GCP.
What worked for me was signing in through Incognito Mode with just my account and attempting to save the scheduled query. I have multiple Google Accounts signed it at one time and for whatever reason, BigQuery throws this generic error after authorization is successful and BigQuery is granted the access it requested.
You need to make sure that you are creating the query under the project targeted not in any other projects because it won't appear
Also you need to enable the API as one of the above answers
This eventually worked for me when i ran this in an cognito window
I have a dataset located in Europe-west3, and i'm trying to setup scheduled queries on that dataset. However, when setting up the scheduled query, the "processing location" option doesn't contain Europe-west3 as an option. Leaving it as "default" makes the processing location be US, and then the query is unable to run. There are only like 7 procesing locations available, i tried both EU and Europe-west2, but neither work.
I don't really know what to do to get my queries to run on schedule. I can run the queries just fine normally, but trying to schedule them the processing location simply wont let me pick the correct location.
Any ideas?
Currently Schedule Queries does not support region europe-west3. Follow (star) this public issue tracker to stay updated.
Right now if you need to implement scheduled queries you should create a replica of that dataset in another region that is supported and run them there. I would suggest creating a copy of that dataset in another region. However, this feature is also not available for region europe-west3 right now.
I hope you can achieve what you desire without many headaches.
I observed something very strange today when trying to steam records into bigquery table , sometimes after the successful stream, it shows all the records being steamed into, something it only shows part of it? What I did was I deleted the table, and recreated it. Has anyone encountered any scenario like this? I am seriously concerned.
Many thanks.
Regards,
I've experienced a similar issue after deleting and recreating the table in a short time span, which is part of our e2e testing plan. As long as you do not delete/recreate your table streaming API works great. In our case workaround was to customize streaming table suffix for e2e execution only.
I am not sure it this was addressed or not, but I would expect constant improvement.
I've also created a test project reproducing the issue and shared it with BigQuery team.
Today, after many successful loads into a BigQuery table, received this error message:
tableUnavailable
Something went wrong with the table you queried. Contact the table owner for assistance
I do not see this error in the error table: https://cloud.google.com/bigquery/troubleshooting-errors#errortable
What conditions could cause this error? Other load jobs, using the same code and in same dataset, do not display this error
What causes a "tableUnavailable" message?
There are two cases that I can think of:
First, this error can be returned for queries over a (small) set of tables that BigQuery exposes access to, but are not directly managed by the BigQuery team itself. You can consider these equivalent to "internalError" from a troubleshooting perspective.
These data sources are typically accessible to GCP customers that have specific relationships with Google product teams exposing their data in BigQuery.
We expose these under a different error code since you will resolve the issue more quickly by contacting the group that granted you access to their data. Going through BigQuery customer support to get this resolved will work too, it'll just take a little longer.
Second, you encountered this through a load job so this is clearly not the case above! We are testing a new load implementation that is faster than the current implementation, and I suspect some errors are mapped slightly differently now.
In this case, I suspect you encountered a "backendError" and should try the operation again. If you can give us a project_id:job_id of a job that hit this problem, we can verify this and make sure the error mapping is more consistent.
Thank you!
Is there a plugin/package to display status information for a PBS queue? I am currently running an apache webserver on the login-node of my PBS cluster. I would like to display status info and have the ability to perform minimal queries without writing it from scratch (or modifying an age old python script, ala jobmonarch). Note, the accepted/bountied solution must work with Ubuntu.
Update: In addition to ganglia as noted below, I also looked that the Rocks Cluster Toolkit, but I firmly want to stay with Ubuntu. So I've updated the question to reflect that.
Update 2: I've also looked at PBSWeb as well as MyPBS neither one appears to suit my needs. The first is too out-of-date with the current system and the second is more focused on cost estimation and project budgeting. They're both nice, but I'm more interested in resource availability, job completion, and general status updates. So I'm probably just going to write my own from scratch -- starting Aug 15th.
Have you tried Ganglia?
I have no personal experience but few sysadmin I know are using it.
Following pages may help,
http://taos.groups.wuyasea.com/articles/how-to-setup-ganglia-to-monitor-server-stats/3
http://coe04.ucalgary.ca/rocks-documentation/2.3.2/monitoring-pbs.html
my two cents
Have you tried using nagios: http://www.nagios.org/ ?