How can I restore accidentally overwritten table in BigQuery? I have tried to do a copy (bq cp) of a snapshot by timestamp previous to the overwritten timestamp but it does not work.
bq cp my_table#1480406400000 new_table
This might be equivalent to Undeleting a Table which is possible within two days (and performed on a best-effort basis and are not guaranteed)
Timestamp in your question looks like 3-4 days back - so that's might be an explanation
As an option try to query that snapshot and see if old data is still there.
Related
Is it possible to query from a recently expired view in Big Query and save a snapshot? (expired 2h ago)
You can try Managing tables
In the documentation there is some examples on how to do that in section Restoring deleted tables.
You can undelete a table within seven days of deletion, including explicit deletions and implicit deletions due to table expiration. After seven days, it is not possible to undelete a table using any method, including opening a support ticket.
You can restore a deleted table by:
Using the # snapshot decorator in the bq command-line tool
Using the client libraries
To restore a table, use a table copy operation with the # snapshot decorator. First, determine a UNIX timestamp of when the table existed (in milliseconds). Then, use the bq copy command with the snapshot decorator.
For example, enter the following command to copy mydataset.mytable at the time 1418864998000 into a new table mydataset.newtable.
bq cp mydataset.mytable#1418864998000 mydataset.newtable
(Optional) Supply the --location flag and set the value to your location.
You can also specify a relative offset. The following example copies the version of a table from one hour ago:
bq cp mydataset.mytable#-3600000 mydataset.newtable
For more information, see Restore a table from a point in time.
I run the following query in Google Big Query:
DELETE FROM mydataset.mytable_wrong
WHERE time = "2019-09-01 13:00:00 UTC"
and then I realised it was the wrong table. Can I recover those rows somehow or undo the query?
The table still exists.
Thanks
Edit to add more info:
Table is partitioned.
You can use an snapshot decorator to query a point-in-time snapshot of your data. You can revert changes with it. Check how to restore a deleted table, I think you can extrapolate to restore your data.
Also, here's a similar Stackoverflow answer with more info.
I am streaming data into BQ, every day I run a scheduled job in Dataprep that takes 24 hours of data and modifies some data and creates a new table in the BQ dataset with 24 hours of data.
The original table though stays unmodified and keeps on gathering data.
What I would like to do is delete all rows in the table after the dataprep makes a copy so that a new 24 hours of data streaming is gathered
How can I make this automated, I can't seem to find anything in dataprep that drops the original table and creating a new table.
You can do this setting up your table as partitioned table due to you are ingesting data constantly.
This option is to do it manually:
bq rm '[YOUR_DATASET].[YOUR_TABLE]$xxxxxxx'
And with the expiration time you can set the time when the data of the table will be deleted:
bq update --time_partitioning_expiration [INTEGER] [YOUR_PROJECT_ID]:[YOUR_DATASET].[YOUR_TABLE]
You could use a Scheduled Query to clear out the table:
https://cloud.google.com/bigquery/docs/scheduling-queries
Scheduled queries support DDL so you could schedule a query that deletes all the rows from this table, or deletes the table entirely, on a daily basis. at a specific time.
Is there a way to find last time updated date from a table without using sys.dm_db_index_usage_stats?? I have been searching for this for an hour now but all answers I found were using this property which seems to be reset on SQL database restart.
Thanks.
You can use this property (which is greatly advised).
Or you can code your own ON UPDATE TRIGGER that will populate this table
(or another homemade) on its own.
Also if you just wish to collect some data about current usage,
you can setup a SQL Profiler that will do the job
(then parse the results somehow, Excel or whatever)
Last option, restore successively the backups you have taken (on a copy).
Hoping you have enough backup retention to find the data you're searching for.
I accidentally deleted a table in BigQuery and I'd very much like to undelete it. I've read the answer in "How can I undelete a BigQuery table?" but in my case I already created a table with the same name, and now when I try to copy a snapshot with the timestamp being the time I created the new table with the same name, I get an error saying:
"error ... invalid snapshot time for table ... cannot read before
"
BTW, the timestamp is from a few hours ago. The whole thing happened in the last 24 hours.
So it seems to me that the problem is that BQ is trying to get a snapshot of the new table, while I need a snapshot of the old table with the same name.
Is there a way for me to access a snapshot of the older incarnation of the table to restore it?
If you're getting an error in the operation described in How can I undelete a BigQuery table?, I'm afraid you're out of luck. Undelete / time travel is a best-effort operation. Sorry about that!