Ignoring date parsing error in Kettle - pentaho

On running a transformation with kettle I get following error:
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - ERROR (version 7.0.0.0-25, build 1 from 2016-11-05 15.35.36 by buildguy) : Because of an error, this step can't continue:
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - ERROR (version 7.0.0.0-25, build 1 from 2016-11-05 15.35.36 by buildguy) : org.pentaho.di.core.exception.KettleValueException:
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - CREATION_DATE String : couldn't convert string [20170326 01:10] to a date using format [yyyyMMdd HH:mm] on offset location 14
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - 20170326 01:10
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 -
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.core.row.value.ValueMetaBase.convertStringToDate(ValueMetaBase.java:791)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.core.row.value.ValueMetaBase.getDate(ValueMetaBase.java:2047)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.core.row.value.ValueMetaBase.convertData(ValueMetaBase.java:3672)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.core.row.value.ValueMetaBase.convertBinaryStringToNativeType(ValueMetaBase.java:1371)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.core.row.value.ValueMetaBase.getString(ValueMetaBase.java:1555)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.core.row.RowMeta.getString(RowMeta.java:319)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.core.row.RowMeta.getString(RowMeta.java:827)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.writeToTable(TableOutput.java:372)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:125)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at java.lang.Thread.run(Thread.java:745)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - Caused by: java.text.ParseException: 20170326 01:10
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - at org.pentaho.di.core.row.value.ValueMetaBase.convertStringToDate(ValueMetaBase.java:782)
2017/06/01 17:57:46 - Table PAX_TKT_UPDATES.0 - ... 10 more
Questions:
The date 20170326 01:10 looks absolutely valid for format yyyyMMdd. Why the parse error
I've selected ignore insert errors in my table output step and that ignores insert errors (such as column constraint errors). But it doesn't seem to ignore data conversion errors and stops the transformation. How can I ignore data conversion errors?

I'm going to guess you're running that transformation in a computer located in Europe.
March 26 is the last Sunday of March and at 1:00 timezone changed to Daylight savings time. Therefore, in local time, there is no such time. After 00:59:59 comes 02:00:00.
Your field on the database is probably a Datetime, which is in local time and therefore the timezone changed at 1am. You can use one of the following approaches:
change the data type to to UTC or fixed timezone;
convert your data making the timezone explicit, e.g., using the format 'yyyyMMdd HH:mm +0100'

Related

Oracle: how to substract a date from table column from a fixed date

I have a table containing a column in date format named "date_started". I want to substract all the dates from this column from a fixed date, for example 31.03.2022, resulting in a new column "absolut days" showing difference between two dates in days.
I tried with the following statement:
SELECT ('31.03.2022'- date_started) AS absolut days
FROM ....
Unfortunately i am not able to find a workaround for the resulting Error message:
ORA-00932: Inkonsistente Datentypen: CHAR erwartet, DATE erhalten
00932. 00000 - "inconsistent datatypes: expected %s got %s"
I am beginner in SQL, exspecially in Oracle and looking forward for some help, thx!
'31.03.2022' may look like a date but it is not a DATE data type; it is a string literal.
If you want a DATE data type then you can use a date literal:
SELECT DATE '2022-03-31' - date_started AS absolut_days
FROM your_table;
Or convert your string to a date:
SELECT TO_DATE('31.03.2022', 'DD.MM.YYYY') - date_started AS absolut_days
FROM your_table;
Then, for the sample data:
CREATE TABLE your_table (date_started) AS
SELECT DATE '2022-01-01' FROM DUAL;
Both output:
ABSOLUT_DAYS
89
db<>fiddle here

How to get correct epoch value of a timestamp column in Netezza?

My Netezza server is in EDT (UTC -4:00) timezone.
I have a table containing a timestamp column. In my select query, I want to get epoch directly from the value of this timestamp column.
For example,
SELECT EXTRACT(EPOCH FROM NOW()), NOW();
Consider NOW() as the column with timestamp type. Above query returns:
1594810500 2020-07-15 10:55:00
1594810500 is supposed to be the epoch value of 2020-07-15 10:55:00 date in EDT.
But when I cross verified the value epoch value here(use timezone as America/Detroit), it gives following output:
Wed Jul 15 2020 06:55:00 GMT-0400
I was expecting the output would be 2020-07-15 10:55:00, which is current time in EDT.
What am I doing wrong? How to get correct epoch value?

Getting date time stamp difference minutes in Oracle Database 12c Enterprise Edition

I have this query to get the date time stamp difference minutes in Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
SELECT TO_CHAR(GPS_FULL_DATE+2/24, 'MM-DD-YYYY HH24:MI:SS') GPS_DATE,
TO_CHAR(CREATION_DATE, 'MM-DD-YYYY HH24:MI:SS') CREATION_DATE,
extract(minute from ((GPS_FULL_DATE+2/24)-CREATION_DATE)) mins
FROM server_data sd
But I got this error:
ORA-30076: invalid extract field for extract source
30076. 00000 - "invalid extract field for extract source"
*Cause: The extract source does not contain the specified extract field.
*Action:
Error at Line: 2 Column: 42
(GPS_FULL_DATE + 2/24) - CREATION_DATE
returns number of days between two dates. Therefore, you can't extract number of minutes of it - you could, though, multiply the result by 24 * 60 (hours in a day * minutes in an hour) to get the desired value.

casting timestamp as date oracle

I have this Timestamp field which looks like this:
2015-08-24 16:24:28.763915
and I want to get only the date and insert to date field
I tried this:
select
TO_DATE(CAST (CON1.AF_UPDATE_DT AS VARCHAR(10)), DD/MM/YYYY)
FROM AF_EMR_MEM_CONT CON1
but i get this error
00904. 00000 - "%s: invalid identifier"
If I try to do this sql:
select AF_UPDATE_DT
TO_DATE(CAST (CON1.AF_UPDATE_DT AS VARCHAR(10)), YYYY/MM/DD)
FROM AF_EMR_MEM_CONT CON1
I get the error:
00923. 00000 - "FROM keyword not found where expected"
You can just truncate the TIMESTAMP, result of the TRUNC function is DATE:
SELECT TRUNC(LOCALTIMESTAMP) FROM DUAL;
You can use SUBSTR to extract the date portion from the string, and then TO_DATE to modify it to a date object:
select TO_DATE(
SUBSTR('2015-08-24 16:24:28.763915', 1, 10), 'YYYY-MM-DD') -- 2015-08-24
from DUAL
See this demo

Remove milli second from timstamp in oracle sql

I want to remove millisecond from 21-02-14 10:41:08.000000000 PM.
I try
select to_date('21-02-14 10:41:08.000000000 PM','DD/MM/YY HH:MI:SS:SSSSS PM') from dual;
error:
Error starting at line : 1 in command -
select to_date('21-02-14 10:41:08.000000000 PM','DD/MM/YY HH:MI:SS:PM') from dual
Error report -
SQL Error: ORA-01855: AM/A.M. or PM/P.M. required
01855. 00000 - "AM/A.M. or PM/P.M. required"
*Cause:
*Action:
You're specifying fractional seconds, but your format has "seconds past midnight" - which doesn't have 9 digits. You're also specifying a colon in your format string, when your data has a dot. Try:
select cast(to_timestamp('21-02-14 10:41:08.000000000 PM','DD-MM-YY HH:MI:SS.FF9 PM') as date) from dual;