I am trying to work with the github data which has been uploaded to Google's big data. I ran a few queries (which generated a lot of rows -
eg: a query SELECT actor_attributes_login, repository_watchers , repository_forks FROM [githubarchive:github.timeline]
where repository_watchers > 2 and REGEXP_MATCH(repository_created_at, '2012-')
ORDER BY actor_attributes_login;
The answer had more than 2,20,000 rows. When I attempted to download to CSV , it said
Download Unavailable
This result set contains too many rows for direct download. Please use "Save as Table" and then export the resulting table.
When I tried to do it as Save as Table I got the following error:
Access Denied: Job publicdata:job_c2338ba91e494b21970854e13cdc4b2a: RUN_JOB
Also, I ran queries where I limited the number of rows to 200 or so, even in such cases I got the error as mentioned above. However I was able to download it as CSV.
Any solution to this problem?
#Anerudh You don't have access to modify the publicdata samples dataset. Create a brand new dataset, and try to save your query results to a new table in that dataset.
Related
I want to create a Table for my dataset in BigQuery. I want to upload CSV file. When I upload it and clicked on "create table" it is saying:
unexpected error. Tracking number c986854671035387
What is this error and How can I solve this? (I also upgraded my BigQuery to 90 days free trial).
You need to check the data inside CSV. If it has column names and no faulty records.
you can download any sample CSV file from here and try
http://www.mytrapture.com/sampledata/
Im trying to export data from big query table to zip file in command line by using BQ extract. It generated multiple empty files (with header) , along with one file with correct data. Can someone please let me know , why empty files are generated.
Thanks
This is a BigQuery issue already reported. I suggest starring the issue and asking for an update on it.
I faced the same empty files issue when using EXPORT DATA.
After doing a bit of R&D found the solution. Put LIMIT xxx in your SELECT SQL and it will do the trick.
You can find the count, and put that as LIMIT value.
SELECT ....
FROM ...
WHERE ...
LIMIT xxx
I just ran a query in Google BigQuery. It produced just over 10 million results.
BigQuery tells me that I cannot save query results that big directly as CSV, JSON, etc. I can only save it to a table.
So I try to save it to a table, using the new UI, Google automatically fills in the "Project name" and "Dataset name". I have tried random table names as well as created a new table and tried to save there.
Every time I get an error:
Not found: Dataset ethereum-account-balances:Eth_Account_Balances_Dataset
Where Ethereum-Account-Balances is the project name and Eth_Account_Balances_Dataset is the Dataset name
I have already created this dataset and confirmed it works. Even re-ran the query after creating the new data set.
Anyone has seen this before?
The new dataset must be in the same location as the dataset whose tables are referenced in the query.
Reference - https://cloud.google.com/bigquery/docs/managing-tables#limitations_on_copying_tables
I had the same problem when trying to save the new BigQuery table in a different dataset of the original table.
What I did to get around this was:
1 - Saved a new BigQuery table in the same dataset or the original table
2 - Copied this new table to the other dataset, using copy table functionality
I clicked a table on bigquery dashboard, got this error:
However, I can get data when I do a select on this table. (That means the table does exist)
I already have the highest admin privilege so it shouldn't be a permission issue.
I created this table with python script, which collects data, writes into a csv file, and upload the csv file to bigquery everyday. After I created the table I once changed the schema both in the script and on the dashboard. Not sure if that's the cause, but the table loading error occurred several days after I changed the schema.
If you have Addblock extensions, this might be the root cause of this issue. Thus, try disabling it, then try running your query again.
Hope it helps.
I am querying the GDELT 2.0 database and am tried to export the entire query but it is too big. I did a limit for 10000 and got those and I now want to get the next 10000 records and so on until I collect the entire query. This is the original command
SELECT * FROM [gdelt-bq:gdeltv2.eventmentions] WHERE MentionIdentifier LIKE '%bitcoin%'
I tried doing what was explained in this link: http://www.plus2net.com/sql_tutorial/sql_limit.php
However it does not go through I'm getting an EOF error and putting in a semicolon doesn't fix it. If you know how to do this please help.