unix_timestamp is behaving incorrect for Hive - hive

I have below 2 Insert queries in single HQL, where I am inserting few fields from table "Final_table_temp" to Final_table1 & Final_table2. Both target table have exact same structure.
insert into Final_table1
PARTITION(event_date,service_id)
select from_unixtime(unix_timestamp(event_timestamp ,'yyyy-MM-dd HH'), 'yyyy-MM-dd HH:00:00.0'),
from_unixtime(unix_timestamp(event_timestamp ,'yyyy-MM-dd HH'), 'yyyy-MM-dd') as event_date,
service_id
from Final_table_temp;
insert into Final_table2
PARTITION(event_date,service_id)
select from_unixtime(unix_timestamp(event_timestamp ,'yyyy-MM-dd HH'), 'yyyy-MM-dd HH:00:00.0'),
from_unixtime(unix_timestamp(event_timestamp ,'yyyy-MM-dd HH'), 'yyyy-MM-dd') as event_date,
service_id
from Final_table_temp;
event_timestamp value in Final_table_temp:
2017-10-26 22
event_timestamp value in Final_table1:
2017-10-26 22:00:00.000
event_timestamp value in Final_table2:
2017-10-26 21:00:00.000
Please help me understand why it is changing the value for table2. It should be as same as table1 as their is no change in query & source is same too?
hive> desc extended Final_table1;
OK
event_timestamp timestamp
event_date date
service_id int
# Partition Information
# col_name data_type comment
event_date date
service_id int
Detailed Table Information Table(tableName:Final_table1, dbName:rwdb, owner:hdfs, createTime:1496391931, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:event_timestamp, type:timestamp, comment:null), FieldSchema(name:event_date, type:date, comment:null), FieldSchema(name:service_id, type:int, comment:null)], location:hdfs://R333:8020/user/hive/warehouse/rwdb.db/Final_table1, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:event_date, type:date, comment:null), FieldSchema(name:service_id, type:int, comment:null)], parameters:{transient_lastDdlTime=1496391931}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE)
=======================================================
hive> desc extended Final_table2;
OK
event_timestamp timestamp
event_date date
service_id int
# Partition Information
# col_name data_type comment
event_date date
service_id int
Detailed Table Information Table(tableName:Final_table2, dbName:rwdb, owner:hdfs, createTime:1509000492, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:event_timestamp, type:timestamp, comment:null), FieldSchema(name:event_date, type:date, comment:null), FieldSchema(name:service_id, type:int, comment:null)], location:hdfs://R333:8020/user/hive/warehouse/rwdb.db/Final_table2, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:event_date, type:date, comment:null), FieldSchema(name:service_id, type:int, comment:null)], parameters:{transient_lastDdlTime=1509000492}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE)


Replace
'yyyy-MM-dd HH:00:00.0' with 'yyyy-MM-dd HH:mm:ss.S'

Related

How can i insert DateTime in SQL?

how can i insert this DateTime in SQL?
1/21/2020 8:30:53.527000 AM +00:00
I know that for 12/12/2020 we can make TO_DATE('12/12/2020', 'DD/MM/YYYY')

In Oracle, in your INSERT statement you can use a TIMESTAMP literal:
TIMESTAMP '2020-01-21 08:30:53.527000 +00:00'
Or use TO_TIMESTAMP_TZ:
TO_TIMESTAMP_TZ(
'1/21/2020 8:30:53.527000 AM +00:00',
'MM/DD/YYYY HH12:MI:SS.FF6 AM TZH:TZM'
)

Oracle partitioning by range

I need to split the table into partitions, namely into three partitions according to the EVENT_TIME field, where the first partition is an interval of a week from today, the second last week and the third partition is history, where data that does not pass into the first two partitions are placed.
In addition, I need to add a trigger that will clear the history every two weeks.
CREATE TABLE EVENTS_LOG_TEST_PARTITION
(
ID NUMBER,
METHOD NVARCHAR2(100),
INPUT CLOB,
EVENT_TIME TIMESTAMP(6),
STATUS NVARCHAR2(100),
MESSAGE NVARCHAR2(200)
)
PARTITION BY RANGE (EVENT_TIME)
(
PARTITION CURRENT_WEEK VALUES LESS THAN (TO_DATE(TO_CHAR(CURRENT_TIMESTAMP), 'dd-MM-yyyy HH24:mi:ss'))
)
ENABLE ROW MOVEMENT;
I know that this is not a valid request, so I am writing, please help

It sounds like you want to keep a rolling 2-3 weeks' worth of data. In which case you can use interval partitioning, dropping the oldest partition each week.
Interval partitioning creates a new partition whenever you insert a row with a value greater than the current highest partition boundary.
All you need to define is an initial partition and the time interval. You can choose any value in the past as the boundary for the initial partition.
For example:
create table events_log_test_partition (
id number,
method nvarchar2(100),
input clob,
event_time timestamp(6),
status nvarchar2(100),
message nvarchar2(200)
) partition by range (event_time)
interval ( interval '7' day ) (
partition p_init values less than ( date'2021-01-04' )
);
insert into events_log_test_partition
values ( 1, 'test', 'test', systimestamp - 14, 'test', 'test' );
insert into events_log_test_partition
values ( 2, 'test', 'test', systimestamp, 'test', 'test' );
select partition_name, high_value
from user_tab_partitions
where table_name = 'EVENTS_LOG_TEST_PARTITION';
/*
PARTITION_NAME HIGH_VALUE
P_INIT TIMESTAMP' 2021-01-04 00:00:00'
SYS_P6002 TIMESTAMP' 2021-08-23 00:00:00'
SYS_P6005 TIMESTAMP' 2021-09-06 00:00:00'
*/
select * from events_log_test_partition
partition for ( date'2021-08-18' );
/*
ID METHOD INPUT EVENT_TIME STATUS MESSAGE
1 test test 18-AUG-2021 13.09.17.000000000 test test
*/
select * from events_log_test_partition
partition for ( date'2021-09-01' );
/*
ID METHOD INPUT EVENT_TIME STATUS MESSAGE
2 test test 01-SEP-2021 13.09.17.516073000 test test
*/
alter table events_log_test_partition
drop partition for ( date'2021-08-18' );
select partition_name, high_value
from user_tab_partitions
where table_name = 'EVENTS_LOG_TEST_PARTITION';
/*
PARTITION_NAME HIGH_VALUE
P_INIT TIMESTAMP' 2021-01-04 00:00:00'
SYS_P6005 TIMESTAMP' 2021-09-06 00:00:00'
*/

filter timestamp column in SQL Oracle

I have a table with a Timestamp column which I need to filter after '2020-08-26', but every solution won't work
RUN_TIME
2020-07-22 04:22:07
2020-07-22 04:34:07
2020-07-22 04:45:07
2020-07-22 04:50:07
2020-07-22 04:55:08
I tried the below queries:
WHERE CAST (RUN_DATE AS DATE) >= TO_DATE('2020-08-26', 'yyyy-mm-dd')
WHERE to_date(RUN_DATE, 'YYYY-MM-DD hh24:mi:ss') >= to_date('26-08-2020 23:59:59', 'YYYY-MM-DD hh24:mi:ss')
WHERE RUN_DATE >= TO_TIMESTAMP('26-08-2020 23:59:59')

Use a TIMESTAMP literal:
SELECT *
FROM table_name
WHERE run_time >= TIMESTAMP '2020-08-27 00:00:00';
or, use a DATE literal:
SELECT *
FROM table_name
WHERE run_time >= DATE '2020-08-27';
or, use TO_TIMESTAMP with a format model:
SELECT *
FROM table_name
WHERE run_time >= TO_TIMESTAMP( '2020-08-27', 'YYYY-MM-DD' );
or, use TO_DATE with a format model:
SELECT *
FROM table_name
WHERE run_time >= TO_DATE( '2020-08-27', 'YYYY-MM-DD' );
Which for the sample data:
CREATE TABLE TABLE_NAME ( run_time TIMESTAMP(6) );
INSERT INTO table_name ( run_time )
SELECT TIMESTAMP '2020-07-22 04:22:07' FROM DUAL UNION ALL
SELECT TIMESTAMP '2020-07-22 04:34:07' FROM DUAL UNION ALL
SELECT TIMESTAMP '2020-07-22 04:45:07' FROM DUAL UNION ALL
SELECT TIMESTAMP '2020-07-22 04:50:07' FROM DUAL UNION ALL
SELECT TIMESTAMP '2020-07-22 04:55:08' FROM DUAL UNION ALL
SELECT TIMESTAMP '2020-08-26 23:59:59.999999' FROM DUAL UNION ALL
SELECT TIMESTAMP '2020-08-27 00:00:00' FROM DUAL;
All output:
| RUN_TIME |
| :------------------------ |
| 27-AUG-20 00.00.00.000000 |
db<>fiddle here

Just:
where run_date >= date '2020-08-26'
That's direct filtering against a date literal.
run_date is a timestamp already, so you should not attempt to convert it. Also, to_timestamp() takes a second argument, which specifies the format of the string; without it, it defaults to the nls_timestamp_format of your session (or database), which may not be what you expect

Can I insert date and time with AM/PM to DATE datatype?

INSERT INTO OFFICE VALUES( 23,TO_DATE('07-Jan-2018 12:30 AM', 'DD-MON-YYYY HH24:MI TT'),'Meadow Fair');
Error report -
ORA-01821: date format not recognized
Name Null? Type
------------ -------- -------------
OFF_NO NOT NULL NUMBER(8)
OFF_DATETIME NOT NULL DATE
OFF_LOCATION NOT NULL VARCHAR2(100)
I can't change the datatype. How should I do this?

As the message, says, your date format is incorrect. It should be
'DD-MON-YYYY HH12:MI AM'
See the manual for the format elements. For example:
SELECT TO_DATE('07-Jan-2018 12:30 AM', 'DD-MON-YYYY HH12:MI AM')
FROM dual
Output:
2018-01-07T00:30:00Z

Insert with the current date and time

INSERT INTO PUBLIC.PROJECT(ID, LIBELLE, DESCRIPTION, STATUT, DATE_CREATION, DATE_MODIFICATION, USER_CREATEUR_ID, REFERENT_ID, DATE_DEBUT, DATE_FIN, TYPE, STATUT_PROJET) VALUES
(2651, 'Projet 1', NULL, 'START', DATE '2019-11-15', DATE '2019-11-15', NULL, 952, TIMESTAMP '2019-11-14 23:00:00', TIMESTAMP '2019-12-14 23:00:00', NULL, NULL);
I want to modify this request and use current_timestamp to have the current date and the current date with time
Example: 2019-11-19 23:00:00
I try to do that : date (current_timestamp + time '23:00') but i have some errors.
Someone have idea ?

Use current_date instead:
current_date + time '23:00'
And you don't need to cast the result, use it as written.