I'm new to sql loader, and i'm having a problem with the date format.
Here is the input file record sample:
Y,1525039510,http-192.168.2.2,15-01-2011 00:00:032:728,64
Y,1525131958,http-192.168.2.2,15-01-2011 00:00:033:613,75
I'm having a problem with the fourth column, the date.
My current field entry is this:
start_time DATE "DD-MM-YYYY HH:MI
How do i parse the last part of the input 032:728 (seconds and milliseconds)
I tried SSS:FF3 and SS.FF3, no luck.
You could treat the input as a string and transform it with a function. Something like this should work:
start_time CHAR to_date(substr(:start_time, 1, 20), 'dd-mm-yyyy hh24:mi":0"ss')
I had to drop the millisecond part since they are not part of the DATE datatype (to_date doesn't recognize the FF format).
You can read this into a timestamp field. The data looks a bit weird: 15-01-2011 00:00:033:613,75 but the conversion format should be dd-mm-yyyy hh[24]:mi:0ss:fff,ff
Make sure that your nls settings are as expected.
Ronald.
Related
I am taking an introductory course to databases so I am a complete beginner. In the database I am supposed to create I have two tables, each contain a DATE datatype.
In the first table, I want it to only display a date (DD-MM-YY) and in the second table display a date and time (DD-MM-YY HH24:MM).
How can I format each attribute to have these respective formats? I've looked around and tried the following command:
ALTER SESSION SET nls_date_format = 'YYYY-MM-DD HH24:MI:SS'
Which works nicely for the date and time field but leaves 00:00:00 for the date only field. Which I do not want, so I reverted it back to nls_date_format = 'DD-MM-YY'
As of right now the following:
INSERT INTO ITEM (ITEM_ENDDATEANDTIME)
VALUES ('13-AUG-13 23:56:00');
Gives me the error: date format picture ends before converting entire input string
Any ideas? Again, I'm a beginner a lot of this is new to me! Thanks!
What you want to do is always store your date values as dates. Data manipulation is infinitely easier when you have stored them in this format instead of in a text based format.
Then, you output them into a more human readable format through a query using syntax similar to this:
SELECT DateField,
TO_CHAR(DateField, 'YYYY-MM-DD HH24:MI:SS') AS Date1,
TO_CHAR(Datefield, 'DD-MM-YY') AS Date2
FROM MyTable
This takes the date data and outputs it as a formatted string. I hope this helps.
My table in hive has a filed of date in the format of '2016/06/01'. but i find that it is not in harmory with the format of '2016-06-01'.
They can not compare for instance.
Both of them are string .
So I want to know how to make them in harmory and can compare them. Or on the other hand, how to change the '2016/06/01' to '2016-06-01' so that them can compare.
Many thanks.
To convert date string from one format to another you have to use two date function of hive
unix_timestamp(string date, string pattern) convert time string
with given pattern to unix time stamp (in seconds), return 0 if
fail.
from_unixtime(bigint unixtime[, string format]) converts the
number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a
string representing the timestamp of that moment in the current
system time zone.
Using above two function you can achieve your desired result.
The sample input and output can be seen from below image:
The final query is
select from_unixtime(unix_timestamp('2016/06/01','yyyy/MM/dd'),'yyyy-MM-dd') from table1;
where table1 is the table name present in my hive database.
I hope this help you!!!
Let's say you have a column 'birth_day' in your table which is in your format,
you should use the following query to convert birth_day into the required format.
date_Format(birth_day, 'yyyy-MM-dd')
You can use it in a query in the following way
select * from yourtable
where
date_Format(birth_day, 'yyyy-MM-dd') = '2019-04-16';
Use :
unix_timestamp(DATE_COLUMN, string pattern)
The above command would help convert the date to unix timestamp format which you may format as you want using the Simple Date Function.
Date Function
cast(to_date(from_unixtime(unix_timestamp(yourdate , 'MM-dd-yyyy'))) as date)
here is my solution (for string to real Date type):
select to_date(replace('2000/01/01', '/', '-')) as dt ;
ps:to_date() returns Date type, this feature needs Hive 2.1+; before 2.1, it returns String.
ps2: hive to_date() function or date_format() function , or even cast() function, cannot regonise the 'yyyy/MM/dd' or 'yyyymmdd' format, which I think is so sad, and make me a little crazy.
Here is my 1 line of data (for brevity):
73831 12/26/2014 1:00:00 AM 0.3220
The 2nd column is the time column which is in string format. I'm using this hive query:
select col2, UNIX_TIMESTAMP(col2,'MM/DD/YYYY hh:mm:ss aaa') from Table
Here is what I get: 1388296800
However, when I check with, http://www.epochconverter.com/ and also from_unixtime(1388296800), I get a different date.
Is there something wrong with my format / pattern string I enter into UNIX_TIMESTAMP in Hive?
Your date format symbols need to conform to those in the Java SimpleDateFormat documentation.
For your date it looks like you want MM/dd/yyyy HH:mm:ss aa.
I have a date format 2011-01-06T06:30:10Z in Excel.I want to just load the date part into a table from excel.How do I get the date part from it.
i.e. 2011-01-06
Thanks
Try this:
select cast(TO_TIMESTAMP_TZ(REPLACE('2011-01-06T06:30:10Z', 'T', ''), 'YYYY-MM-DD HH:MI:SS TZH:TZM') as date) from dual
I think, some more explanation is needed.
Loading data into database is one part, and displaying it after fetching is another part.
If you have loaded the data into database, then all you need to do is use TRUNC. It will truncate the time portion and will display only the date portion.
A DATE always has a datetime part together. TIMESTAMP is an extension to the DATE type. And what you see the date looks like is not the way it is stored in database. The format is for we human beings to understand. A date is stored in 7 byte in internal format.
More information Based on OP's question via comments
NEVER store a DATE as VARCHAR2 datatype. A date is not a string literal. Oracle provides lot of FORMAT MODELS to display the datetime the way you want. Sooner or later, you will run into performance issues due to data conversion. Always use explicit conversion to convert a literal to a perfect DATE to compare it with other date value.
I have the following in my SQL where clause. This is running against an Oracle database. The sc_dt field is defined in the db as a date field.
sc_dt = TO_DATE('2011-11-03 00:00:00.0', 'YYYY-MM-DD')
produces the following error "date format picture ends before converting entire input string"
When I try to account for the fractional seconds (.0 in this case) with the following, I get the following error.
sc_dt = TO_DATE('2011-11-03 00:00:00.0', 'YYYY-MM-DD HH24:MI:SS.FF')
produces the following error "date format not recognized"
I'm really just assuming that I need the .FF to account for the .0 in the "from" string. I've also tried .FF1, .FF2, ..., .FF9 with the same results (I'm grasping at straws at this point).
As far as I can see, the sc_dt field always has the month/day/year portion populated (and not the hour/minute/second portion).
I'm debugging a java program which is executing the above SQL as a prepared statement with the 2011-11-03 00:00:00.0 value.
How can I get around this?
You need to use the seconds past midnight option. Something like:
select TO_DATE('2011-11-03 00:00:01.1', 'YYYY-MM-DD HH24:MI:SS.SSSSS') from dual
Or This:
select TO_TIMESTAMP('2011-11-03 00:00:00.1', 'YYYY-MM-DD HH24:MI:SS.FF') from dual
An Oracle DATE column like sc_dt will always have a day and a time component down to the second. Depending on your query tool and how it is configured (generally the session's NLS_DATE_FORMAT), it is possible that the time component isn't being displayed by default. You can, however, see the time component by doing an explicit TO_CHAR
SELECT to_char( sc_dt, 'YYYY-MM-DD HH24:MI:SS' )
FROM table_name
Because a DATE only stores the time to the second, however, you cannot use fractional seconds in your format mask. So you would need to do something like this to extract just the portion of the string up to the fractional seconds. If you're not guaranteed that the string will always be 19 characters before the decimal point, you could use INSTR as well to look for the decimal point and take everything before that.
TO_DATE( substr('2011-11-03 00:00:00.0', 1, 19), 'YYYY-MM-DD HH24:MI:SS')
Since this is coming from a Java application, however, you're much better off using the correct data type. If you bind a Java date (java.sql.Date) using the setDate method on the prepared statement rather than binding a string, then you won't have to deal with the string format in your SQL statement.
I realize this thread is more than a year old but...
Another option just to throw it in might be:
src_dt=select TO_DATE('2011-11-03 00:00:01.1234', 'YYYY-MM-DD HH24:MI:SS.?????') from dual;
Note: there is an extra '?' thrown in to illustrate that you can even stick in a few extra '?'s. There is no complaint from Oracle if the digits represented by the '?'s do NOT have any corresponding character in the source time string. This might be helpful if you aren't sure of the precision of seconds you are receiving.
This option gives some flexibility to the format of "fractional seconds" from your source time.
I do not know that this is actually documented anywhere.
I did this :
ALTER SESSION
SET NLS_DATE_FORMAT = 'YYYY-MM-DD HH24:MI:SS.?';
--Change the decimal
ALTER SESSION
SET NLS_NUMERIC_CHARACTERS = ',.';
And it worked for me
src_dt=select TO_DATE('2011-11-03 00:00:01.1', 'YYYY-MM-DD HH24:MI:SS.SSSSS') from dual
I guess the above one should work if you just need a date output.