I my web REST API service I've decided to return a dateTime data as a string with a timezone which looks like this: "someDate":"2012-01-23T22:52:37.039+02:00" I wonder, how idiomatic is it? If I returned it as Unix time stamp, would it be more sensible?
Personally, I prefer using unix timestamps in situations like this because it is completely timezone agnostic and leaves the timezone interpretation entirely up to the client. Additionally, depending on your client, it's generally less code to parse a unix timestamp into a date than a date string, but that's likely only an ancillary consideration.
Personally I prefer using the “Epoch time”. It’s not very readable, but it’s very well defined and much shorter then other formats.
References:
http://en.wikipedia.org/wiki/System_time
http://en.wikipedia.org/wiki/Unix_time
The format is described in RFC 3339: Date and Time on the Internet: Timestamps
-- a profile of ISO 8601.
It is unambiguous and both human and machine readable.
POSIX timestamp is less readable e.g., 1327351957.039 and there is no indication of a local time.
If parsing datetime strings is a performance bottleneck in your application then you could cache the timestamp value.
Related
I am currently exploring Learning locker which is a LRS and store XAPI statements .I see the timestamp in XAPI should follow ISO 8601 format .I see it can be represented as "2015-01-01T01:00Z" but how can I store the timezone information too like "2007-04-05T12:30−02:00".XAPI documentation suggests setting the timeozone but there is no clear way how we can do it .
Any clue on this will be helpful .
Generally the Z indicates a timezone, specifically the UTC timezone. It is preferred that all statement timestamps are stored using UTC and then if they need to be handled relative to a local timezone then that conversion happens at the point of need rather than via the stored data. If you must store it with the specific timezone then the format you showed with the -02:00 should work. NOTE: In the proposed (nearly accepted) version 2.x of the xAPI specification all timestamps will have to be recorded in UTC, see:
A Timestamp shall be formatted to UTC
(https://gitlab.com/IEEE-SA/xapi/9274.1.1/xapi-base-standard-documentation/-/blob/master/9274.1.1%20xAPI%20Base%20Standard%20for%20Content.md#527-additional-requirements-for-data-types)
So it would likely be better to go ahead and start following that pattern.
If the timezone is relevant for other purposes it would be better to track the timezone itself separately from the timestamp (likely as a result or context extension).
We are working on an API that has an authentication header in which a timestamp has to be processed. Since we might need more use of timestamps in the future and want to be consistent in what timezone we use, I'd like to know if there is a conventional timezone for APIs. UTC+0 for example.
Thanks in advance
In some cases, you will find that HTTP headers (such as the standard Date header) is in RFC5322 format (aka RFC2822, RFC822). For example: Tue, 31 Jan 2017 17:45:00 GMT
In the Date header, the time zone abbreviation is always"GMT" (which is equivalent to UTC for this purpose), even though the RFC5322 format allows for other time zones.
The above format is not really preferred, it's just something we're stuck with. You can certainly use any format you wish for your own HTTP headers though.
A much better format is the RFC3339 format. This is similar to the ISO8601 extended format. An example is: 2017-01-31T17:45:00Z. The Z at the end indicates "Zulu" time, which is the same as GMT or UTC. However, you could also specify a time zone offset from UTC, such as the equivalent local time in the US Pacific time zone: 2017-01-31T09:45:00-08:00
If you have Unix time numbers like 1485884700, the time zone is always UTC. However, this is not a good interchange format because it is not human readable, and offers no context for what epoch or precision is being used. One would have to know those things externally. It is not a good fit for HTTP headers, nor XML or JSON.
If you are talking about the body of your XML/JSON requests, then you should only be using ISO8601. There are a few other formats that people use, but they are not recommended.
With regard to what time zone you should use - that's entirely dependent on context. If you are timestamping - where you take the current time and record it, then you can indeed just work with UTC. I would think for authorization purposes, this would be reasonable. You could also work in a local time, as long as you provided the offset from UTC along with it so there is no ambiguity.
However, you said (in comments) that your data contains information about job vacancies. That sounds to me like you are talking about time in the future - and that is an exception to the "always UTC" rule. Any time you are talking about time in the future, you need to express that in terms of local time, in the most direct form you can, and you also need to provide the identifier (not the offset) of the time zone that applies.
For example, if I am talking about a job vacancy, I might say in my data:
{
"job": "dishwasher",
"available: "2017-02-13",
"start": "08:00",
"end": "16:00",
"tz": "America/New_York"
}
These values would follow the ISO8601 format for date-only and time-only (or if I wanted to combine them 2017-02-13T08:00) - and would not have a time zone specified. Instead, the IANA time zone identifier America/New_York (for US Eastern Time) is provided in a separate field.
This matters because perhaps the job continues into future months. On March 12th, the US Eastern time zone will change from UTC-5 to UTC-4. Therefore, one cannot specify a single offset, nor can you use UTC.
Also, even for a single occurrence, keep in mind that some countries continue to change their minds about what their time zone and DST rules are, which is why there are dozens of updates to the tz database every year. If you were to calculate an offset for a date and time in the future, by the time it rolls around you might find the offset had been changed by the government.
I have a string in the following format:
14:41:21 Dec 15, 2015 PST
I want to convert that to my server's local time, but I think I'm creating an extra step that can be avoided:
Dim testdate As Date
DateTime.TryParseExact(dateinput, "HH:mm:ss MMM dd, yyyy PST", CultureInfo.InvariantCulture, DateTimeStyles.None, testdate)
testdate = TimeZoneInfo.ConvertTimeToUtc(testdate, TimeZoneInfo.FindSystemTimeZoneById("Pacific Standard Time"))
testdate = testdate.ToLocalTime()
I've played around with this but always off by a couple hours either way, and the above is what I've found to work but just wanted to know if there was a better way. Also note it could be deployed on multiple servers, so I don't want to specify the timezone to convert it to explicitly, reason for localtime.
A few things:
If you're going to include fixed text in a format string, put it in single-tick quotes so it can't get misinterpreted as a formatting token. ('PST')
In the general case, time zone abbreviations should only be used for display purposes. They should not be parsed as input, as they could be ambiguous. For example, there are 5 different interpretations of CST. It might be US Central Standard Time, but it could also be China Standard Time, or one of the others. See the list on Wikipedia.
If you have a limited number of time zone abbreviations you want to support, then you could extract it from the string and use a dictionary, select/case statements, or conditional logic to map them. Just be certain you know the entire set of abbreviations you want to support and exactly which time zones you want them to map to. Also be sure to account for daylight time abbreviations, such as PDT.
Note that some older standards, such as RFC 2822 §4.3 indeed hardcode a few abbreviations, so you may choose to support those if you are parsing that particular format. (Yours is similar, but not quite a match.)
Your code is mostly ok, but you should probably check the result of TryParseExact. Otherwise you might as well use ParseExact which will throw an exception on failure instead of just returning false.
You could use ConvertTime with TimeZoneInfo.Local as the destination zone if you wanted to do the conversion in a single step. The code would be slightly smaller, though would have no technical differences.
Are you sure you really want to do this? Relying on the system's local time zone should usually should not be done in server-based applications. That's something more appropriate for desktop and mobile. In general, server-side code should not rely on the system time zone to be anything in particular. Avoid "local time" APIs, including DateTime.Now, TimeZoneInfo.Local, ToLocalTime, and ToUniversalTime (when it assumes the input is local time). It is better to supply the applicable time zone in your business logic or application configuration.
My table has a category 'timestamp' where the timestamps are formatted 2015-06-22 18:59:59
However, using DBVisualizer Free 9.2.8 and Vertica, when I try to pull up rows by timestamp with a
SELECT * FROM table WHERE timestamp = '2015-06-22 18:59:59';
(directly copy-pasting the stamp), nothing comes up. Why is this happening and is there a way around it?
FYI, saying "the timestamps are formatted 2015-06-22 18:59:59" is incorrect if you are indeed using a TIMESTAMP type. Such types have their own internal representation of a date-time value, almost always a count since epoch. In your case with Vertica, 8 bytes are used for such storage. The formatting of the date-time value happens when a string representation is generated. Never confuse the string representation with the date-time value. Conflating the two may well be related to your problem/confusion.
A few different thoughts about possible problems…
String Literals
Are you sure Vertica takes strings as timestamp literals? That format you used is common SQL format. But given that Vertica seems to be a specialized database, I would double-check that.
If strings are not allowed, you may need to call some kind of function to transform the string into a date-time values.
Fractional Second
As the comment by Martin Smith points out, the doc for Timestamp-related data types in Vertica 7.1 says those types can have a fractional second to resolution of microseconds. That means up to 6 decimal places of a fraction.
So if you are searching for "2015-06-22 18:59:59" but the stored value is "2015-06-22 18:59:59.012345", no match on the query.
Half-Open
The fractional seconds issue described above is often the cause of problems people have when handling a span of time. If you naïvely try to pinpoint the ending time, you are likely to have problems. Seeing the "59:59" in your example string makes me think this applies to you.
The better approach to spans of time is "Half-Open" (or Half-Closed, whatever) where the beginning is inclusive while the ending is exclusive. Common notation for this is [). In comparison logic this means: value >= start AND value < stop. Notice the lack of EQUALS SIGN in the stop comparison. In English we would say "look for an hour's worth of invoices starting at 2:00 PM and going up to, but not including, 3:00 PM".
Half-Open for a week means Monday-Monday, for a month the first of one month to the first of the next month, and for a year the January 1 of one year to January 1 of the following year.
Half-Open means not using BETWEEN in SQL. SQL's BETWEEN has often be criticized. Instead do something like the following to look for an hour's worth of invoices. Notice the Z on the end of string literal which means "UTC time zone" ("Z" for "Zulu"). (But verify, as my SQL syntax may need fixing.)
SELECT *
FROM some_table_
WHERE invoice_received_ >= '2015-06-22 18:00:00Z'
AND invoice_received_ < '2015-06-22 19:00:00Z'
;
This query will catch any values such as '2015-06-22 18:59:59.654321" which seems to be eluding you.
Reserved Word
I hope you have not really named your table 'table' and your column 'timestamp'. Such use of keywords and reserved words can cause explicit errors or more subtle weird problems.
Tip: The easy way to avoid any of the over a thousand reserved words in various databases is to append a trailing underscore. The SQL standard explicitly promises to never using a trailing underscore in its reserved words. So use "timestamp_" rather than "timestamp". Another example: "invoice_" table and "received_" column. I recommend doing that as a habit on everything your name in SQL: columns, tables, constraints, indexes, and so on.
Time Zone
You are using the TIMESTAMP which is short for TIMESTAMP WITHOUT TIME ZONE. Or so I presume; the Vertica doc is vague but that is the common usage as seen in the Postgres doc, and may even be standard SQL.
Anyways, TIMESTAMP WITHOUT TIME ZONE is usually the wrong type for most business purposes. The WITH time zone is misnamed and often misunderstood as a consequence: It means "with respect for time zone" where data inputs that include an offset or other time zone information from UTC are adjusted to UTC during the INSERT/UPDATE operations. The WITHOUT type simply ignores any such offset or time zone information.
The WITHOUT type should only be used for the concept of a date-time generally without being tied to any one locality. For example, saying "Christmas this year starts at beginning of December 25, 2015". That means in any time zone rather than a specific time zone. Obviously Christmas starts earlier in Paris, for example, than in Montréal.
If you are timestamping legal documents such as invoices, or booking appointments with people across time zones, or scheduling shipments in various localities, you should be using WITH time zone type.
So back to your possible problem: Test how Vertica or your client app or your database driver is handling your input string. It may be adjusting time zones as part of the parsing of the string using your client machine’s current default time zone. When sent to the database, that value will not match the stored value if during storage no adjustment to UTC was made.
Tip: Generally best practice is to do all your storage and business logic in UTC, adjusting to local time zones only where expected by user.
Pretty simple, I need to disable time zone conversion for specific columns. I will handle any TZ conversion manually, but I need Rails 3 to forego conversion in both writing and reading, and any AREL functions. But, I don't want to disable the conversion for non-specified attributes.
Ok, I know how to disable it for reading:
self.skip_time_zone_conversion_for_attributes = [:test_timestamp]
But this only works for reading. When writing the attribute, it still converts to UTC (yes, I tested this in 3.2.8).
As you note, skip_time_zone_conversion_for_attributes only works for reading, which makes the whole feature pretty useless.
There are two possible solutions:
1.- Accept that with times will be written in UTC, and read accordingly:
def starts_at # override reader method
attributes['starts_at'].in_time_zone(whatever_timezone)
end
cons: overriden method is bypased when using MyModel.pluck(:starts_at).
2.- Store time values as strings, taking care of writing the values in the right format, and reading them in the desired timezone.
def starts_at
DateTime.strptime(attributes['starts_at'], whatever_format).in_time_zone(whatever_timezone)
end
cons: one loses the ability to query the database by using date operators (less than, greater than).