I have a situation where I have to pull data from Oracle database at different point in time during a day but there is a condition with this.
For Example, after running the query in the morning at 11 AM, the query fetched 2 records.
When I re-run the same query at 2 PM, the query should return the records excluding the ones which it had fetched in the morning 11 AM execution.
I am playing around with createdinfo.time and timestamps but I am not able to get the results in a single query.
I don't know even if it is possible to use a single query and get the data filtered as mentioned above or I have to use 2 different queries here for 2 different time intervals.
Any suggestions are appreciated.
Related
I am currently working on a task that queries for a certain type of record. If it is null or less than 1 for the current date an email sends out. That part is pretty straight forward. Now a requirement has been added to get a count for how many days consecutively this has been happening.
If there are no records in the table, there are no dates to check on, so I'm not entirely sure what to do.
If I query on multiple dates then it destroys the code which is built upon emailing on a null result.
By default how long a temp table will be on to get data? I know that we can set up the expiration, but what is the default?
And about the job? What's the default expiration time if it has?
I tried to find these in the documentation but i couldn't find, we return the jobId to the client so he can get the data when the job is complete, but some of them like to store and tries to fetch data with a jobId from 2 weeks ago, 1 month ago.
What's the default time here so i can explain them better?
Query results are stored for 24 hours:
All query results, including both interactive and batch queries, are cached in temporary tables for approximately 24 hours with some exceptions.
https://cloud.google.com/bigquery/docs/cached-results
As was mentioned by Alexey, they query results are stored for 24 hours when using cache.
Regarding the timelife for BigQuery jobs, you can get the job history for the last six months
By the other side, based on your description, creating a new table from your query results with expiration time seems to be the most appropriate strategy. Also, you could check if using materialized views could help you to store some results of recurring queries.
I am looking to pull data between two time periods at only 15 to 30 mins apart. I want to be able to rerun the code multiple times to constantly update the data I had already pulled. I know there is a function for current system time but I am unable to use it effectively in SQL developer.
I have tried using the function CURRENT_TIMESTAMP but could not get it to work effectively.
Currently i am using the following code and just pulling over a broad time frame, but i would like to shrink that down to 15 to 30 minute intervals that could be used to continue to pull updated data.
I expect to be able to pull current data within 15 to 30 minute segments of time.
I have a database which captures everyday every column data as binary 0 or 1 in a SQL database. Now I want to calculate what is the avg no. of days a user takes to complete that column. How can I run the query for that?
Please note timeline for every user is different and I want data to be in 3 days slab (for example 1-3,4-6 so on and so forth).
I've been using the Table Range Decorators feature daily since May in order to only query the data from the last 7 days in some of my tables.
Since 2 weeks, I've noticed that sometimes some data is missing when I use that feature. For example, if do a query to get the results for the last 7 days (by adding "#-604800000--1" to table), some data will be missing as opposed to if I query on the whole table (without a table decorator).
I wonder what could explain this and if there is a fix coming soon to address this?
If this can help the BigQuery team, I've noticed that when using Table Decorators some data was missing for us for October 16th between around 16:00 and 20:00 UTC time.
For the BigQuery team here are 2 jobs ids where some data is missing: job_-xtL4PlIYhNjQ5weMnssvqDmd6U , job_9ASNxqq_swjCd1eMmiQ6SmPpxlQ
and 1 job id where data is correct(without decorators): job_QbcRwYGbQv0BZdHreQEvRlYh-mM
This is a known issue with table decorators containing a time range. Due to a bug in BigQuery, it is possible for certain time ranges to omit data that should be included within the time range.
We're working on a fix and plan to have it released next week. After this fix is deployed time range decorators should again work as expected.