I need to regularly extract data from an Oracle 11 table using sqlplus. For example, I need every day to extract the new rows inserted into that table.
On a table with a primary key such as RECORD_ID (assuming it is inserted incrementally), that query would be:
SELECT * from TABLE WHERE RECORD_ID > &LAST_RECORD_ID_FROM_PREVIOUS_QUERY
On a table with a RECORD_DATE timestamp, this could similarly done like:
SELECT * from TABLE WHERE RECORD_DATE > &LAST_RECORD_DATE_FROM_PREVIOUS_QUERY
My question is: how do you do this when you have no timestamps and no incremental column you could use? Can this be achieved with ROWID?
One way would be to enable flashback and then you could do:
SELECT * FROM table_name
MINUS
SELECT * FROM table_name AS OF TIMESTAMP SYSTIMESTAMP - INTERVAL '1' DAY;
As I suspected there isn't any easy solution. It has to be one of:
Adding an identity or timestamp column
Do a diff using flashback
Add a trigger on insert on the table
Unfortunately none of which is practical in my environment. Case closed!
Related
ERROR: no partition of relation "test_table" found for row DETAIL:
Partition key of the failing row contains (start_time) = (2021-04-25
00:00:00). SQL state: 23514
I am inserting a data where i have a column start time (2021-04-25 00:00:00)
This is my Schema
CREATE TABLE test_table (
start_time timestamp NULL,
)
PARTITION BY RANGE (start_time);
This sounds as if you have no partition-tables defined for this table.
You might need something like this:
CREATE TABLE test_table_2021 PARTITION OF test_table
FOR VALUES FROM ('2021-01-01') TO ('2022-01-01');
After you defined this partition for your partitioned table, you should be able to insert the data (as long as start_time is anywhen in 2021).
See the docs: https://www.postgresql.org/docs/current/ddl-partitioning.html
Can we compare the columns of same table and get a result by selecting all the rows where both columns data does not match. Compare CREATE_DATE and UPDATE_DATE
Table with data
Expected output:
SELECT *
FROM Table
WHERE TIME_CREATED <> UPDATE_DATE
There should be no issue comparing date to timestamp, please update your OP if you have tried this already
If there is an issue, read this post and try something like this:
Oracle comparing timestamp with date (also shows use of truncate to disregard time of day)
SELECT *
FROM Table
WHERE to_timestamp(TIME_CREATED,'D/MM/YYYY HH:MM:SS') <> UPDATE_DATE
OR
SELECT *
FROM Table
WHERE TIME_CREATED <> TO_DATE(UPDATE_DATE,'DD.MM.YYYY:HH24:MI:SS')
You might have to play with the timestamp format a bit, see https://www.akadia.com/services/ora_date_time.html
IS there an sql query in DB2 that will fetch table/records where any record has been updated since a given time stamp ?
The following query I used only fetched if it is table change and not if records are updated.
SELECT TRIM(TABSCHEMA) || '.' || TRIM(TABNAME),
MAX(CREATE_TIME,ALTER_TIME)
FROM SYSCAT.TABLES
ORDER BY 2 DESC
The generic solution would be to add a timestamp tracking column to each table, defaulting to current timestamp then select all MAX() values with a union and filter with a timestamp parameter.
The other solution is to create a trigger that updates a tracking table with the table name and current timestamp.
create table archive as select * from EMP where 1=2;
I want to add extra field current date in archive table
where 1=2 will always be false thus returning 0 records. Anyway, you can get it work like below but DATETIME functions are RDBMS specific. NOW() will work only if it's MySQL
create table archive as
select *,NOW() as Create_date from EMP
I am trying to insert into a Hive table from another table that does not have a column for todays date. The partition I am trying to create is at the date level. What I am trying to do is something like this:
INSERT OVERWRITE TABLE table_2_partition
PARTITION (p_date = from_unixtime(unix_timestamp() - (86400*2) , 'yyyy-MM-dd'))
SELECT * FROM table_1;
But when I run this I get the following error:
"cannot recognize input near 'from_unixtime' '(' 'unix_timestamp' in constant"
If I query a table and make one of the columns that it work just fine. Any idea how to set the partition date to current system date in HiveQL?
Thanks in advance,
Craig
What you want here is Hive dynamic partitioning. This allows the decision for which partition each record is inserted into be determined dynamically as the record is selected. In your case, that decision is based on the date when you run the query.
To use dynamic partitioning your partition clause has the partition field(s) but not the value. The value(s) that maps to the partition field(s) is the value(s) at the end of the SELECT, and in the same order.
When you use dynamic partitions for all partition fields you need to ensure that you are using nonstrict for your dynamic partition mode (hive.exec.dynamic.partition.mode).
In your case, your query would look something like:
SET hive.exec.dynamic.partition.mode=nonstrict;
INSERT OVERWRITE TABLE table_2_partition
PARTITION (p_date)
SELECT
*
, from_unixtime(unix_timestamp() - (86400*2) , 'yyyy-MM-dd')
FROM table_1;
Instead of using unix_timestamp() and from_unixtime() functions, current_date() can used to get current date in 'yyyy-MM-dd' format.
current_date() is added in hive 1.2.0. official documentation
revised query will be :
SET hive.exec.dynamic.partition.mode=nonstrict;
INSERT OVERWRITE TABLE table_2_partition
PARTITION (p_date)
SELECT
*
, current_date()
FROM table_1;
I hope you are running a shell script and then you can store the current date in a variable. Then you create an empty table in Hive using beeline with just partition column. Once done then while inserting the data into partition table you can add that variable as your partition column and data will be inserted.