I am brand new to google big query so apologize if this is obvious.
I am simply trying to test the product out right now. I am able to upload a 5 MB file without any issues.
When I move to 10 MB+ using google storage, I am having no luck.
I have a bucket named teststir and a file name verify_sift.csv
When I try to create a new data set I select google cloud and put:
gs://teststir/verify_sift.csv as the path.
Unfortunately the job keeps failing:
Not found: URI gs://teststir/verify_sift.csv
(I have triple checked names, tried multiple files but no luck). Am I missing something obvious? Thank you for your help!
I've encountered the same problem.
I've just created new bucket with another location and it helped :)
Related
I am using the the google cloud logging web ui to export google compute engine logs to a big query dataset. According to the docs, you can even create the big query dataset from this web ui (It simply asks to give the dataset a name). It also automatically sets up the correct permissions on the dataset.
It seems to save the export configuration without errors but a couple of hours have passed and I don't see any tables created for the dataset. According to the docs, exporting the logs will stream the logs to big query and will create the table with the following template:
my_bq_dataset.compute_googleapis_com_activity_log_YYYYMMDD
https://cloud.google.com/logging/docs/export/using_exported_logs#log_entries_in_google_bigquery
I can't think of anything else that might be wrong. I am the owner of the project and the dataset is created in the correct project (I only have one project).
I also tried exporting the logs to a google storage bucket and still no luck there. I set the permissions correctly using gsutil according to this:
https://cloud.google.com/logging/docs/export/configure_export#setting_product_name_short_permissions_for_writing_exported_logs
And finally I made sure that the 'source' I am trying to export actually has some log entries.
Thanks for the help!
Have you ingested any log entries since configuring the export? Cloud Logging only exports entries to BigQuery or Cloud Storage that arrive after the export configuration is set up. See https://cloud.google.com/logging/docs/export/using_exported_logs#exported_logs_availability.
You might not have given edit permission for 'cloud-logs#google.com' in the Big Query console. Refer this.
I keep trying to upload a new table onto my companies BQ, but I keep getting the error you see in the title ("does not contain valid backup metadata.").
For reference, I'm uploading a .csv file that has been saved to our Google Cloud data storage. It's being uploaded as a native table.
Can anyone help me troubleshoot this?
It sounds like you are specifying the file type DATASTORE_BACKUP. When you specify that file type, BigQuery will take whatever uri you provide (even if it has a .CSV suffix) and search for the Google Cloud Data Storage Backup files relative to that url.
I'm trying google refine out to address name disambiguation in my data.
Whenever I upload a CSV, however, I keep getting this error.
I've been following the tutorial at this link Tutorial
Error uploading data
.import-temp/1405348781604/raw-data/spreadsheet/ccc (No such file or directory)
I also came across this in my google search, naming a similar problem I'm facing.
https://github.com/OpenRefine/OpenRefine/issues/670
But I don't know how to fix the issue. Am I supposed to go into the source code and edit the lines mentioned? If so, can someone please give me some directions about how to do that?
The reason for this error message is that you don't have write access for the folder that contains the OpenRefine files.
I first had the files in C:\Program Files (x86)\OpenRefine\. Then I moved the folder to D:\Documents\OpenRefine and the error disappeared.
Here are my two issues with CloudFront at the moment:
Some users it seems to be connecting really slow to my CloudFront CDN. It just hangs. But it's fine for others.
Some users don't even load some files (e.g: stylesheets). But like above, others are fine.
I am regularly making updates to my CDN files and use a query string to tell the users browser that the file has been updated. However I have a query string across the whole website so if I update one files, all files get an updated query string. Could that be the issue?
Has anyone else had issues like this before?
Thanks for your help!
What I have decided to do is remove the use of Query Strings and simply rename the files each time they changed.
So for example I'd call a file:
style_23623276323.css
The "3623276323" part is the MD5 signature of the file.
More details can be found in this article.
Hope that helps somebody.
I uploaded a lot of files (about 5,800) to Amazon S3, which seemed to work perfectly well, but a few of them (about 30) had their filenames converted to lowercase.
The first time, I uploaded with Cyberduck. When I saw this problem, I deleted them all and re-uploaded with Transmit. Same result.
I see absolutely no pattern that would link the files that got their names changed, it seems very random.
Has anyone had this happen to them?
Any idea what could be going on?
Thank you!
Daniel
I let you know first that Amazon S3 object URLs are case sensitive. So when you upload file file with upper case and access that file with same URL, it was working. But after renaming objects in lower case and I hope you are trying same older URL so you may get access denied/NoSuchKey error message.
Can you try Bucket Explorer to generate the file URL for Amazon S3 object and then try to access that file?
Disclosure: I work for Bucket Explorer.
When I upload to Amazon servers, I always use Filezilla and STFP. I never had such a problem. I'd guess (and honestly, this is just a guess since I haven't used Cyberduck nor Transmit) that the utilities you're using are doing the filename changing. Try it with Filezilla and see what the result is.