In postgres when i do cast(varchar_col as timestamp) how do i know which formats (ie yyyymmdd, yyyy-mm-dd...etc) are supported?
Not great research on your end, because it's right there in the Postgres 8.4 docs in the section of the eponymous name:
https://www.postgresql.org/docs/8.4/datatype-datetime.html
Valid input for the time stamp types consists of the concatenation of a date and a time, followed by an optional time zone, followed by an optional AD or BC. (Alternatively, AD/BC can appear before the time zone, but this is not the preferred ordering.)
So, you need to combine all input format entries from the "Date Input" table and from the "Time Input" table, which are too many to list sensibly in an answer. Makes no sense either – that's what official documentation is for.
For a more comprehensive documentation on how dates and times are parsed from strings, that's officially documented as well:
https://www.postgresql.org/docs/8.4/datetime-input-rules.html
Related
In Mosaic Decisions Flows, I can see there's a system parameter called "$currentTime" which gives the current timestamp. But this is giving the current timestamp in UTC. I want to convert it into CST timezone. Is there a way I can do that?
Yes, you can use Convert_Timezone function available in transformation node. Below is the syntax for it.
CONVERT_TIMEZONE( column_name, ‘Timezone1’ , ‘Timezone2’);
Column_name – input time based column.
Timezone1 – the timezone the column data is in.
Timezone2 – the timezone in which the column data has to be converted in.
CONVERT_TIMEZONE (NOW(), ‘UTC’ , ‘CST6CDT’ ) -> NOW() will give you the currentTime
Similarly you can use – SystemV/CST6CDT, SystemV/CST6, based on the requirement.
Also you can refer Transformation section in user guide for further details on it [link below] :
https://mosaic.ga.lti-mosaic.com/usermanual/Transformer.html
I see a new data typeDATE being introduced in the BigQuery Web UI but not documented in https://cloud.google.com/bigquery/data-types.
Well DATE as such is not new but I have one major question. Unlike TIMESTAMP does DATE ignore timezone information after being stored.
Quoted text from TIMESTAMP documentation:
You can supply a timezone offset in your date and time strings, but
BigQuery doesn't preserve the offset after converting the value to its
internal format. If you need to preserve the original timezone data,
store the timezone offset in a separate column.
Also whats the expected input for this format. I have tried the known timestamp strings but it does not seem to work.
I see a new data typeDATE being introduced in the BigQuery Web UI but
not documented
Date type was introduced with Standard SQL - see Date type for details
Also whats the expected input for this format
Canonical format
'YYYY-[M]M-[D]D'
YYYY: Four-digit year
[M]M: One or two digit month
[D]D: One or two digit day
Note: The DATE type represents a logical calendar date, independent of time zone.
ISO 8601 recommends the following format for a date and time:
2014-12-29T16:11:20+00:00
I'm rather fond of this format since it allows for lexical ordering. But there is a small problem: some file systems don't allow colons in file names (at least not normally). ISO 8601 does allow for omitting the colons, but I would rather have some symbol there than have the numbers run together:
2014-12-29T161120+0000
Does ISO 8601 allow for a symbol other than colons? I couldn't find any indication that it does. If not, is there another well recognized symbol I could use? (Perhaps another standard proposes such a symbol?)
There is none.
ISO 8601 only allows for a colon (:) for separating time components in the extended format:
The basic format is [hh][mm][ss] and the extended format is [hh]:[mm]:[ss].
There is no provision for an alternate extended format.
There's no other provision and you cannot have a complete datetime expresion with the first half in extended and the last one in basic formats; they must have the same formats.
A workaround, and i emphazise in workaround, is to split the string with something else, e.g., a period or underscore, that separates two valid expresions, the date in extended format and the time in basic format: e.g., myfilename_2020-12-13.T0006Z_otherNotes.ext or myfilename_2020-12-13_T0006Z_otherNotes.ext.
You must be clear with other parts involved in the usage of such files - mainly speaking about machine ones - in that it is not a complete datetime expresion, but 2 valid ones each one in different formats - extended, then basic - separated by an agreed and unambiguous character (i mentioned a period/underscore instead of a dash/hyphen specifically for this reason).
The obvious downsides of this approach are: the time part is harder to read and some other people could believe you badly implemented the ISO or you have poor understanding of it, when not.
The good sides... it works in Windows, it organizes the elements in the filesystem in the right order, despite the OS, it follows the ISO standards (in a special way) and it is compatible with machines (from scripts to AIs and everything in the middle).
It is a workaround. I repeat.
As a final note, i really wonder why the people doing ISO 8601-2019 didn't consider the problem that is using ISO in filenames or if they're totally unaware of it.
For combined date and time representations you must use basic format for both date and time or extended format for both date and time representations to comply with ISO 8601.
Examples of ISO 8601 compliant date and time representations:
2020-12-03T15:05:57+00:00 (extended)
2020-12-03T15:05:57Z (extended)
20201203T150557Z (basic)
For maximum compatibility with various operating systems file naming requirements and ISO 8601 compliance I think the last example or something similar should work.
Reference: https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations
I am designing a database for use with a Ruby on Rails application. For a given object, I need to access the date of an event in both the Gregorian format and the Hebrew calendar equivalent. I can easily convert between the two formats, but the issue is that in the Hebrew calendar, the date changes at sunset, not midnight. Therefore, I'll need to either store two separate dates, or store a Gregorian date and a separate boolean field, after_sunset. Then, whenever I need to access the Hebrew date, I'll need to query for both fields, convert the date, and if after_sunset==true, increment the date.
Which of these options is considered "better"?
And, if I store the Hebrew date separately, is it best to store it as a String, an Integer, or can I use a regular Date?
With an after_sunset flag you store a Gregorian date and add all the additional information needed to know the Hebrew date.
With two dates you would store the two dates explicitely. However, to have data consistent you would install a check constraint to ensure that the dates match. This is because the two dates share part of their information (redundancy). This means the data is not normalized.
For this reason, to have data normalized in your database (and thus not having to install a check constraint to keep the data consistent) the first approach is better. Store the date plus an after- sunset flag.
Store the date in UTC and also store in unix format
You can use conversion function based on the type
This will allow your database to support other date time formats easily in the future
Unless you are going back to the dawn of time, I think I would simply have a many-year lookup table of UTC datetimes and Hebrew dates where the UTC column is the first second of the Hebrew day in a specific time zone (Greenwich?).
Conversions are a quick binary search,
SELECT hebrew_date FROM hebrew_gregorian_lookup
WHERE some_input_time >= gregorian_cutoff
ORDER BY gregorian_cutoff DESC LIMIT 1;
If you index and cluster the lookup table on gregorian_cutoff, it should be very quick, even for 100 years. (If your RDBMS has a way to force a table into RAM, even better.) Also depending on your RDBMS, you may be able to wrap this in a function/procedure with no loss of efficiency.
I suggest storing the Hebrew date not as a string but as a record of three shorts, day, month, year. You can have a tiny lookup table for month to string, or perhaps use an enumeration. That will give you some flexibility in formatting, e.g., Hebrew characters vs. Latin in the output.
I want to find some data of a application which allows using Apache Lucene syntax for search queries. I search data by date and want to find data from concrete date - concrete day. How can I do that?
Queries:
date: [2010-10-4 TO 2010-10-4]
or
date: 2010-10-4
does not work.
Short answer: there is no "standard" for date query syntax in Lucene. You need to find out the format(s) your app supports.
Long answer: For the last couple years or so, Lucene keeps the numeric data specially encoded. Most likely, the date in the index is kept in the timestamp format. This means the query parser needs to take in the query, chew it and spit out the timestamp. Querying against a raw timestamp is not very practical - at least for humans - and your query parser likely has some pre-defined format it is able to understand.
For example, Solr has a pre-defined set of supported date/time formats and is able to parse those into timestamps.
Don't forget Lucene is just a library and each application (including Solr and the one you are using) is meant to use it the way they like.
I found the solution.
For searching by one conrete day, query:
date: [2010-10-4T00:00:00 TO 2010-10-4T23:59:59]
is correct
I put the date range without the hyphens and It worked for me.
date: [20210901 TO 20211101 ]
See the doc Range Searches