Is there a date or datetime data type in Google BigQuery? I looked at the online documentation, but the data types mentioned didn't include any date types
2013 update:
BigQuery now has a Timestamp type.
Look at: BigQuery datatypes
2018: TIMESTAMP, TIME, DATE, DATETIME
2016 update:
BigQuery now also has real DATE type (in addition to TIMESTAMP type) in standard SQL:
https://cloud.google.com/bigquery/sql-reference/data-types#date-type
https://cloud.google.com/bigquery/sql-reference/functions-and-operators#date
Edited to add: There is a date time type, see Fh's answer for more information.
No date data type is available, although if you use ISO-formatted date-times (YYYY-MM-DD HH:MM:SS[.uuuuuu]) or timestamps (convertable to microseconds since 1970) there are a number of date functions that you can use. See this page for more information.
Related
A table with terabytes of data in bigquery got multiple columns set as string format but actually they contain datetime strings like
2016-10-24 15:00:00
I tried answer from this link to convert (CAST) the fields into timestamp format as below
SELECT
CAST( MURDER_DATE AS TIMESTAMP) AS CONVERTED_MURDER_DATE, *
FROM `death_list`;
That works but it converts all strings into timestamps with UTC timezone as below
2007-03-23 15:00:00.000 UTC
I need the data in a different timezone. Any clue?
Try using
DATETIME(CAST( MURDER_DATE AS TIMESTAMP), "Australia/Sydney"))
To my view, it seems to be a current limitation of BigQuery:
Timestamp type is always stored in UTC format. And you have no way to add any "timezone" information to it.
Datetime type nor stores any information about the timezone. You could still have a internal convention in your team/company that says that all the Datetime columns are stored in your local timezone, but I personally find it very awkward.
What we've decided so far in our company is to store everything in Timestamp (thus UTC format), and we never use Datetime due to lack of precision regarding the time zone. Then, if a client wants to get the information in another timezone, it has to do the conversion itself when reading the data.
I see a new data typeDATE being introduced in the BigQuery Web UI but not documented in https://cloud.google.com/bigquery/data-types.
Well DATE as such is not new but I have one major question. Unlike TIMESTAMP does DATE ignore timezone information after being stored.
Quoted text from TIMESTAMP documentation:
You can supply a timezone offset in your date and time strings, but
BigQuery doesn't preserve the offset after converting the value to its
internal format. If you need to preserve the original timezone data,
store the timezone offset in a separate column.
Also whats the expected input for this format. I have tried the known timestamp strings but it does not seem to work.
I see a new data typeDATE being introduced in the BigQuery Web UI but
not documented
Date type was introduced with Standard SQL - see Date type for details
Also whats the expected input for this format
Canonical format
'YYYY-[M]M-[D]D'
YYYY: Four-digit year
[M]M: One or two digit month
[D]D: One or two digit day
Note: The DATE type represents a logical calendar date, independent of time zone.
how good is the datetime for questions like: Compare sales from Monday and Thusday?
Mysql keeps datetime as a unix-timestamp internally? So finding mondays will be quite expensive.
Has anybody got experiance how much better mysql performs when an extra attribute "day" is introduced and given an index? Will indies liked this be used at all? It will only have 7 different states...
how good is the datetime for questions like: Compare sales from Monday and Thusday?
DATETIME is your best choice, as it's mySQL's native format and any date operations are highly optimized for it.
Mysql keeps datetime as a unix-timestamp internally?
Nope. I don't know what mySQL uses to store DATETIMEs internally, but it's not as Unix timestamps:
The DATETIME type is used when you need values that contain both date and time information. MySQL retrieves and displays DATETIME values in 'YYYY-MM-DD HH:MM:SS' format. The supported range is '1000-01-01 00:00:00' to '9999-12-31 23:59:59'.
How do I use an SQL statement on an sqllite database to insert the current date in UTC. I found the NOW function but what format is that in? This will be on mobile devices so everyone will have a different locale, however, I need a standard time format because the device will compare the dates with my server.
Also, is there a way to automatically update a 'modified' field when the data in the row is changed like you can in MySQL?
SELECT DATETIME('now') returns the current UTC datetime. See Date And Time Functions. You can use DATETIME DEFAULT CURRENT_TIMESTAMP with column declaration.
Format 11, the string 'now', is
converted into the current date and
time as obtained from the xCurrentTime
method of the sqlite3_vfs object in
use. Universal Coordinated Time (UTC)
is used
For the 'modified' field you can use a trigger.
You don't specify what you use to develop your application on. I prefer using QDate::toJulianDay and QDate::fromJulianDay in Qt to store dates in an SQLite database as an integer if I only need to store the date.
How would you store a time or time range in SQL?
It won't be a datetime because it will just be let's say 4:30PM (not, January 3rd, 4:30pm).
Those would be weekly, or daily meetings.
The type of queries that I need are of course be for display, but also later will include complex queries such as avoiding conflicts in schedule.
I'd rather pick the best datatype for that now.
I'm using MS SQL Server Express 2005.
Thanks!
Nathan
Personally I would find this a reason to upgrade to 2008 which has a separate time datatype.
I would recommend still using a DateTime data type and ignoring the date values--ideally using the static MinDate for SQL (Google it). This will give you the benefits of working with a strongly typed field and the only cost will be a few extra bytes.
As for ranges, store them in two separate columns. Then you can subtract one from the other to determine the difference.
Edit: did some Googling.
SQL Server 2008 adds a Time data type, so you might want to consider that.
You can use SQL 2005's DateTime type and combine it with the CONVERT function to extract just the HH:MM:SS.MMM
SELECT CONVERT(VARCHAR(12), GETDATE(), 114) AS [HH:MI:SS(24H)] (Found on this handy-dandy page)
Different SQL versions support different minimum dates. You could use a static date that will be supported by all such as 1/1/2000, or you could use SQL 2005's minimum value of 1/1/1753 and append the time values to that startic day
So if you stick with 2005, pick your static date, like 1/1/2000, and store your times on it. So 1m:30s would be 2000-1-1 00:01:30.000, and 1h:15m would be 2000-1-1 01:15:00.000
You can then do Date2 - Date1 and get your result of (1h:15:m - 1m:30s) 2000-01-01 01:13:45.000. CONVERT it and you'll have 1:13:45.
You could store it as an int as 24 hour time and format as needed.
Or store it as a datetime with some fixed date and remove it as needed for display:
Jan 1 2000 4:30PM
I would go with datetime field as it gives you the power of all the datetime related functionality.
You might want to consider storing it as an int column representing the number of minutes since midnight. In your entity you could expose this as a TimeSpan (or int) representing the same thing. You'd only need to convert between your display values (time format) and the database value (minutes) in order to perform your queries and this could easily be done in your entity (TimeSpan.TotalMinutes, for example).
to me it sounds like you're developing a type of meeting scheduler or something to display the meetings.
i think that i would set it p with 2 columns MeetingStart and MeetingEnd, both as datetime fields. This way, you can determine the length of the meeting, and since you already have the date you can easily use it to display it on a calendar or something.