Can anyone help me with the timestamp format in Trips Layer for deck.gl?
Pasting a part of the array,
[-74.21055, 40.80768, 1660.904],
[long, lat, time]
Related
I am getting source data with duration between 2 timestamps as
Duration Start date End date Start station
14h 26min. 2sec. 12/31/2010 23:49 1/1/2011 14:15 10th & U St NW (31111)
how can I import this data ( which is in CSV file ) in Teradata database to store duration in correct data type, so that I can match it properly with the difference between start and end data?
Please help in correct approach here.
Thanks in advance
That's quite tricky.
A pure SQL based solution (without features of your ETL-tool) needs to generate data which can be safely casted.
This will modify your duration into a format which can be passed to to_dsinterval by removing unneccessary characters besides HMS (target column should be defined as INTERVAL HOUR(4) TO SECOND(0))
Cast(to_dsinterval('PT'||Upper(OTranslate(duration, ' in.ec', ''))) AS INTERVAL HOUR(4) TO SECOND(0))
Your input timestamps show single digit day/month, which Teradata doesn't support (don't aks why), the RegEx adds those missing zeroes (when the seconds are missing remove the :ss part of the format):
Cast(RegExp_Replace(start_date, '\b([0-9])\b', '0\1') AS TIMESTAMP(0) Format 'mm/dd/yyyyBhh:mi:ss')
Finally pass duration & timestamps as VarChars and apply the Casts during Insert.
I have done some research on DATETIME and TIMESTAMP data type and I understand that they support date time to be represented in milliseconds and microseconds
like the one below,
YYYY-[M]M-[D]D[( |T)[H]H:[M]M:[S]S[.DDDDDD]]
But, is it possible to load/represent values that has nanoseconds precision?
like,
YYYY-[M]M-[D]D[( |T)[H]H:[M]M:[S]S[.DDDDDDDDD]]
Actually, BigQuery supports up to microsecond precision, and not only millisecond.
No, I don't believe it supports nanosecond precision (maybe a Googler will correct me there), and I certainly can't see anything in the docs. However, this is stated:
An error is produced if the string_expression is invalid, has more
than six subsecond digits (i.e. precision greater than microseconds),
or represents a time outside of the supported timestamp range.
Thus, this will work:
SELECT CAST('2017-01-01 00:00:00.000000' AS TIMESTAMP)
But this will not ("Could not cast literal "2017-01-01 00:00:00.000000000" to type TIMESTAMP"):
SELECT CAST('2017-01-01 00:00:00.000000000' AS TIMESTAMP)
For more context on timestamp precision, consider the supported range of BigQuery timestamps, which is 0001-01-01 00:00:00.000000 to 9999-12-31 23:59:59.999999. With microsecond precision, if you anchor timestamps to the Unix epoch, this means that you can represent the start of this range with the integer value -62135596800000000 and the end with 253402300799999999 (these are the values that you get if you apply the UNIX_MICROS function to the timestamps above).
Now suppose that we wanted nanosecond precision, but we still wanted to be able to express the timestamp as an integer relative to the Unix epoch. The minimum and maximum timestamps would be represented as -62135596800000000000 and 253402300799999999. Looking at the range of int64, though, we would need a wider integer type, since the min and max of int64 are -9223372036854775808 and 9223372036854775807. Alternatively, we would need to restrict the range of timestamps to approximately 1677-09-21 00:12:43 to 2262-04-11 23:47:16, assuming I did the math correctly. Given that nanosecond precision generally isn't that useful, having the wider timestamp range while still being able to use a 64-bit representation is the best compromise.
Problem:
My timestamp is being displayed in scientific notation. I would like to display the column without scientific notation, and create a second column formatted as a long date, yyyy/mm/dd hh:mm:ss.
Steps taken:
I've already converted the column from a UNIX Epoch (1960) timestamp to SAS time (1970) timestamp. But scientific notation persists. I tried date20. doesn't do the trick, either.
Timestamp in Scientific Notation
My current insufficient code fails to format the timestamp column as a date.
proc print data=heart._23a;
format timestamp date9.;
run;
Results:
It results in no errors, but it redimensions my matrix to a 1x3. I need to obtain a matrix of the same dimension, just with a reformatted timestamp. I appreciate any help, but please keep it simple, I am in unknown territory!
datetime17. is the standard timestamp format in SAS, though you have many other choices as well. ymddttm. is the closest to what you're looking for, I believe.
One important distinction here: SAS has two concepts, date and datetime. date is number of days since 1/1/1960 and has no time part, while datetime is number of seconds since 1/1/1960 00:00:00 and has both time and date. You can use datepart to convert datetime -> date, or dhms to convert date -> datetime.
Your question also seems to get the two epochs backwards. UNIX epoch is 1970. SAS epoch is 1960.
Finally, if you want to display the raw number of seconds, use w.d format instead of bestw.d format - format timestampvar 14. for example, where w is number of characters (digits) wide total including decimal.
I am running a query between several tables and I am running into an issue between comparing two time columns on separate tables: "rc1_time" is in a string format and "osemplog_time" is in a time format. both are time only with no date
rc1_time's contents look like this '10560684' which corresponds to HH24MISSMS
osemplog_time's contents look like 07:57:02.917455
how do I format the rc1_time into a "time format" with no date?
what are some options for comparing the two times?
I am newbie at this exposition on your answers would be welcome
below is my query
SELECT
"public".payroll_master.prm1_name,
"public".payroll_master.prm1_oe_init,
"public".receipt.rc1_init,
"public".employee_log.osemplog_ipaddress,
"public".employee_log.osemplog_event,
"public".receipt.rc1_date,
"public".employee_log.osemplog_logdate,
"public".receipt.rc1_code,
"public".employee_log.osemplog_logname,
"public".oslogname.lognm_empname,
"public".receipt.rc1_arname,
"public".receipt.rc1_arnum,
"public".receipt.rc1_time,
"public".employee_log.osemplog_logtime
FROM
"public".receipt
INNER JOIN "public".employee_log ON "public".receipt.rc1_date = "public".employee_log.osemplog_logdate
INNER JOIN "public".payroll_master ON "public".payroll_master.prm1_oe_init = "public".receipt.rc1_init
INNER JOIN "public".oslogname ON "public".oslogname.lognm_empname = "public".payroll_master.prm1_name AND "public".oslogname.lognm_name = "public".employee_log.osemplog_logname
WHERE
"public".receipt.rc1_code = 'CA'
AND
"public".employee_log.osemplog_logdate = "public".receipt.rc1_date
ORDER BY
"public".receipt.rc1_init ASC
Question as stated
You can represent a time without a date using the time data type. To convert a string from a given format into one, you can go through the to_timestamp function and then cast to time:
SELECT to_timestamp('10560684', 'HH24MISSUS')::time;
SELECT to_timestamp('07:57:02.917455', 'HH24:MI:SS.US')::time;
The basic idea is that you parse the time string using to_timestamp. The resulting timestamp will have a default date, and casting to time will remove the date, leaving only the parsed out time portion.
Assumptions:
Your hours are in 24-hour clock format (13-23 for 1 PM to 11 PM and 00 for midnight). If they are not 24 hour times, then you are missing the AM/PM designation and will need to sort that out.
The second "SS" you mention in your first pattern is actually a fractional part of seconds. If not, you'll need to adjust the pattern. If you don't care about the fractional seconds, you might consider just leaving the US and the .US off entirely and working only at the seconds level. Note that US interprets 84 to be 0.84 seconds, not actually 84 microseconds (0.000084 seconds).
Ultimately, you will need to either provide much more precise details about the format or figure out the correct format string yourself. Rather than worry about those details, I've tried to exemplify the general mechanism and leave those to you.
Comparison is then trivial. You just use PostgreSQL's operators (<, >, =, etc.):
SELECT to_timestamp('07:57:02.917455', 'HH24:MI:SS.US')::time < to_timestamp('10560684', 'HH24MISSUS')::time;
Other considerations
Be aware of time zone issues if you are working across them. You'll want to look at timetz (short form of time with time zone) or timestamptz (short form of timestamp with time zone) if you need to deal with time zones. Generally, I would recommend including time zone handling up front in case it becomes a problem later.
In this case, why not build a complete timestamp? You already have the dates: "public".receipt.rc1_date and "public".employee_log.osemplog_logdate.
You don't specify the data types, but whatever the forms of those are, it should be possible. For example, if they are actual date objects, then:
SELECT to_timestamp(to_char("public".receipt.rc1_date, 'YYYY-MM-DD')||' '||"public".receipt.rc1_time, 'YYYY-MM-DD HH24MISSMS');
If they are strings of the form 'YYYY-MM-DD', then:
SELECT to_timestamp("public".receipt.rc1_date||' '||"public".receipt.rc1_time, 'YYYY-MM-DD HH24MISSMS');
And so on. Now you have a real timestamp, which makes simple great/less than comparison much, much easier.
In my experience, it's extremely rare that you actually want to test time stamps with fractional second precision for equality. You might want a more tolerant equality check, something like SELECT t1 - t2 < interval '5 seconds', but this is really up to the application.
ProcessTime: 00:00:00.0000012
RegexResolveTime: 00:00:00.0000421
MessageResolveTime: 00:00:00.0001269
FullProcessTime: 00:00:00.0001734
Ok, I've got 4 columns as above with datatype Time(7). I need to get the average of all the entries for those individual columns but Time(7) isn't a valid type for the AVG operator!
How does one go about getting the average of a Time(7) column?
I'll add that these are Timespans and not discrete points in time even though SQL server considers them such!
You can't average TIME because TIME represents a point in time, not a duration. I suggest one of two approaches.
The preferred approach: store duration as an integer in milliseconds, microseconds, nanoseconds, what have you. This will allow you much more precision if needed.
Apply conversions back and forth so you can average the delta from midnight instead of the actual time value.
DECLARE #x TABLE (ProcessTime TIME(7));
INSERT #x VALUES ('00:00:00.0000012'), ('00:00:00.0000016');
SELECT DATEADD(NANOSECOND, AVG(DATEDIFF(NANOSECOND, '00:00:00.0000000', ProcessTime)),
CONVERT(TIME, '00:00:00.0000000'))
FROM #x;
Results:
00:00:00.0000014
But doesn't that seem wrong to you? If you're not storing a point in time and you're only concerned about duration, store the duration. You can always format it as time when displaying.