I am preparing an Informatica mapping
Source:- Oracle
Target :- Teradata
There are 2 columns in the mapping
Source column name : INX_DATE(string) and UPX_DATE (string)
Target column name : INX_DATE(date/time) and UPX_DATE (date/time)
Source column contains something like this
UPX_Date:-08-FEB-16 12.00.23.826315000 AM +00:00
INX_Date:- 08-FEB-16 12.00.23.826315000 AM EUROPE/BELGRADE
Now the issue is how will I get this to my target which is date/time
and should be MM/DD/YYYY HH:MM:SS
You can substring the first 31 characters and then convert it to date.
TO_DATE(SUBSTR(INX_DATE,1,31),'DD-MON-YY HH12:MI:SS.NS AM')
As far as I know, PowerCenter Date/Time cannot store the time zone information. If you need that, you may need to adjust the time separately.
Related
We have a source as a flat file which consists of a date column in format 19/08/2013.
We have a target in oracle table which consists of a date column in format 11-AUG-13.
When we are trying to pass the source column value in target using expression TO_DATE AND
TO_CHAR like
**source column is A ---> Source column
v1=TO_CHAR(A)
O1(output column)=TO_DATE(V1,'DD-MON-YY') we are getting the below error.
Invalid date: [19/8/2013]. The row will be skipped.**
Can anyone please help where I'm going wrong.
Thank you
You need to convert the str to date properly and then infa will load them.
Change the mapping like this -
Change data type in source to a string. read data as string.
Then use your expressions but change the to_date function like below.
v1(var column, data type string)=A
O1(output column, data type date)=IIF(IS_DATE(V1,'DD/MM/YYYY'), TO_DATE(V1,DD/MM/YYYY'),null)
IS_DATE - will check if the data is a date or not, if yes then only it will try to convert to a proper date else put null.
3. Then connect O1 column to a date type in oracle target.
This will make sure all data are loaded and none skipped.
I am creating a pipeline from the SQL database to storage in the Azure data factory.
I am struggling to identify how to search data between the two dates with this format: yyyy-MM-ddTHH:mm:ssZ
I created a variable for the date called: date
Now upon searching query with below codes:
#concat('SELECT * FROM dbo.Task_History
WHERE history_timestamp BETWEEN ''2015-01-01T01:00:00Z'' AND CONVERT(datetime2, ''' , variables('date') , ''', 127)')
It gives an error stating:
Conversion failed when converting date and/or time from character string.
Below is the target column where to search the target data
You can use convertFromUtc() function to get current date time in Singapore Standard time zone. And then you can use string interpolation syntax for dynamically building your SQL query.
In below example, I am trying to build a SQL query which will query data from "dbo.Task_History" table between dates "2015-01-01T01:00:00Z" & current date time Singapore Standard time zone.
Set variable activity to set current date time value in Singapore Standard time zone.
Expression used: #convertFromUtc(utcnow(),'Singapore Standard Time')
Generating SQL query using String Interpolation format:
SELECT * FROM dbo.Task_History
WHERE history_timestamp BETWEEN '2015-01-01T01:00:00Z' AND '#{variables('singaPoreDateTime')}'
Between, could you pls help me understand why you are trying to convert again in SQL to some timezone ?
Im working with Data actory this time this why i ask lot of question about that
My new problem is that my SOURCE(CSV file contains a column DeleveryDate full of Date dd/MM/YYYY) and my table SQl where i specify DElevry date as DateTime but when I map btw source and sink in Data preview source
duplicate columns like in the picture below but in data preview sink the columns always NULL the same in my table NULL.
Thanks
You said column DeleveryDate full of Date dd/MM/YYYY), can you tell me why the column DeleveryDate has the values like '3', '1' in your screenshot? String '3' or '1' are not the date string with format dd/MM/YYYY.
If you want to do some data convert in Data Factory, I still suggest your to learn more about Data Flow.
For now, we can not convert date format from dd/MM/YYYY to datetime yyyy-MM-dd HH:mm:ss.SSS directly, we must do some other converts.
Look at bellow, I have a csv file contained a column with date format dd/MM/YYYY string, I still using DerivedColumn this time:
Add DerivedColumn:
Firstly, using this bellow expression to substring and convert dd/MM/YYYY to YYYY-MM-dd:
substring(Column_2, 7, 4)+'-'+substring(Column_2, 4, 2)+'-'+substring(Column_2, 1,2)
Then using toTimestamp() to convert it:
toTimestamp(substring(Column_2, 7, 4)+'-'+substring(Column_2, 4, 2)+'-'+substring(Column_2, 1,2), 'yyyy-MM-dd')
Sink settings and preview：
My Sink table column tt data type is datetime:
Execute the pipeline:
Check the data in sink table:
Hope this helps.
Please try this-
This is a trick which was a blocker for me, but try this-
Go to sink
Mapping
Click on output format
Select the data format or time format you prefer to store the data into the sink.
Here is my 1 line of data (for brevity):
73831 12/26/2014 1:00:00 AM 0.3220
The 2nd column is the time column which is in string format. I'm using this hive query:
select col2, UNIX_TIMESTAMP(col2,'MM/DD/YYYY hh:mm:ss aaa') from Table
Here is what I get: 1388296800
However, when I check with, http://www.epochconverter.com/ and also from_unixtime(1388296800), I get a different date.
Is there something wrong with my format / pattern string I enter into UNIX_TIMESTAMP in Hive?
Your date format symbols need to conform to those in the Java SimpleDateFormat documentation.
For your date it looks like you want MM/dd/yyyy HH:mm:ss aa.
I have a character field that stamps in the order of MMDDYYHHMMSS (note: not a date but character field). I am wanting to kick this out to a date field in my SQL into this format dd.mm.yyyy. hh24:mi.
My problem is that the sql kicks it out to YYYY-MM-DD field without the time. This section of the sql looks like this:
TO_DATE(SUBSTR(MOPACTIVITY.MOPID,3,2)||'.'||SUBSTR(MOPACTIVITY.MOPID,1,2)
||'.'||'20'||SUBSTR(MOPACTIVITY.MOPID,5,2)||'.'||SUBSTR(MOPACTIVITY.MOPID,7,2)
||':'||SUBSTR(MOPACTIVITY.MOPID,9,2)||':'||SUBSTR(MOPACTIVITY.MOPID,11,2)
, 'dd.mm.yyyy. hh24:mi:ss') "XXX",
Any thoughs on how to get the time to convert too?
No need for such a complicated expression:
to_date(MOPID, 'MMDDYYHH24MISS')
will convert the column to a real DATE column assuming the time part is in 24 hour format (00-23, not 00-12). And this will also fail if you don't really have valid dates in the varchar column.
this out to a date field in my SQL into this format
A DATE column does not have "a format"!
The format is only applied when you display it.
In case you mean you want to convert the varchar stored in your column into another varchar that has a different date formatting, the easiest is probably to simply convert the above expression back to a varchar:
to_char(to_date(MOPID, 'MMDDYYHH24MISS'), 'dd.mm.yyyy. hh24:mi')
Before applying something like that, allow me one comment:
Store dates in DATE columns, never ever store them in a VARCHAR column.
If you had done that from the beginning, all you would have to do know is to simply apply a single to_char() to your DATE column to get the display format you want.