I've been working on parsing strings and I have a test case that has been causing problems for me. When parsing a date/time string with strptime, Daylight Savings Time is NOT accounted for. This is a bug as far as I can tell. I can't find any docs on this bug. Here is a test case in the Rails console. This is ruby 1.9.3-p215 and Rails 3.2.2.
1.9.3-p125 :049 > dt = DateTime.strptime("2012-04-15 10:00 Central Time (US & Canada)", "%Y-%m-%d %H:%M %Z")
=> Sun, 15 Apr 2012 10:00:00 -0600
1.9.3-p125 :050 > dt = DateTime.strptime("2012-04-15 10:00 Central Time (US & Canada)", "%Y-%m-%d %H:%M %Z").utc
=> Sun, 15 Apr 2012 16:00:00 +0000
1.9.3-p125 :051 > dt = DateTime.strptime("2012-04-15 10:00 Central Time (US & Canada)", "%Y-%m-%d %H:%M %Z").utc.in_time_zone("Central Time (US & Canada)")
=> Sun, 15 Apr 2012 11:00:00 CDT -05:00
As you can see, I have to convert to utc and then back to the timezone to get DST to be properly interpreted, but then the time is shifted one hour as well, so it's not what I parsed out of the string. Does someone have a workaround to this bug or a more robust way of parsing a date + time + timezone reliably into a DateTime object where daylight savings time is properly represented? Thank you.
Edit:
Ok, I found a workaround, although I'm not sure how robust it is.
Here is an example:
ActiveSupport::TimeZone["Central Time (US & Canada)"].parse "2012-04-15 10:00"
This parses the date/time string into the correct timezone. I'm not sure how robust the parse method is for handling this so I'd like to see if there is a better workaround, but this is my method so far.
This is a frustrating problem. The Rails method you're looking for is Time.zone.parse. First use DateTime.strptime to parse the string, then run it through Time.zone.parse to set the zone. Check out the following console output:
> Time.zone
=> (GMT-06:00) Central Time (US & Canada)
> input_string = "10/12/12 00:00"
> input_format = "%m/%d/%y %H:%M"
> date_with_wrong_zone = DateTime.strptime(input_string, input_format)
=> Fri, 12 Oct 2012 00:00:00 +0000
> correct_date = Time.zone.parse(date_with_wrong_zone.strftime('%Y-%m-%d %H:%M:%S'))
=> Fri, 12 Oct 2012 00:00:00 CDT -05:00
Notice that even though Time.zone's offset is -6 (CST), the end result's offset is -5 (CDT).
Ok, here is the best way I've found to handle this so far. I created a utility method in a lib file.
# Returns a DateTime object in the specified timezone
def self.parse_to_date(date_string, num_hours, timezone)
if timezone.is_a? String
timezone = ActiveSupport::TimeZone[timezone]
end
result = nil
#Chronic.time_class = timezone # Trying out chronic time zone support - so far it doesn't work
the_date = Chronic.parse date_string
if the_date
# Format the date into something that TimeZone can definitely parse
date_string = the_date.strftime("%Y-%m-%d")
result = timezone.parse(date_string) + num_hours.to_f.hours
end
result
end
Note that I add hours onto the time manually because Chronic.parse wasn't as robust as I liked in parsing times - it failed when no trailing zeros were added to a time, for example, as in 8:0 instead of 8:00.
I hope this is useful to someone. Parsing date/time/timzone strings into a valid date seems to be a very common thing, but I was unable to find any parsing code that incorporated all three together.
Related
I am having dataframe containing time column to be like this:
time
2001-11-28 13:42:46 -0500
2001-10-10 22:14:00 -0400
I know how to convert them into time period but I fail to understand what does -0500 and -4000 even means.
This data I am using is an open source data for bugs related thing. If any one can help me with this it will be very helpful to me.
-0500 means five hours (05) and zero minutes (00) behind (-) UTC. For example New York City sometimes observes this time offset (specifically in winter, when DST is not in effect).
Read more about how UTC offsets work here: https://en.wikipedia.org/wiki/UTC_offset
And more on how Pandas handles times and time zones here (search for "timezone"): https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html
There is timezone offset, you can processing different ways:
#convert to datetimes with different timezones
df['time1'] = pd.to_datetime(df['time'])
#convert to datetimes with utc tiemzone
df['time2'] = pd.to_datetime(df['time'], utc=True)
#remove timezone information
df['time3'] = pd.to_datetime(df['time'].str.rsplit(' -', n=1).str[0])
print (df)
time time1 \
0 2001-11-28 13:42:46 -0500 2001-11-28 13:42:46-05:00
1 2001-10-10 22:14:00 -0400 2001-10-10 22:14:00-04:00
time2 time3
0 2001-11-28 18:42:46+00:00 2001-11-28 13:42:46
1 2001-10-11 02:14:00+00:00 2001-10-10 22:14:00
I want to format the dtime2 field in my query: SELECT FORMAT(MAX(dTime),'yyyy-MM-dd hh:mm:ss') FROM triangulations
This gives the output { result: [ { '': '03:34:30' } ], rowcount: 1 }
The hours should be 15. This is also displayed when leaving the format out of the query. Query: SELECT MAX(dTime) FROM triangulations gives output:
{ result: [ { '': Mon Jul 17 2017 15:34:30 GMT+0000 (Coordinated Universal Time) } ],
rowcount: 1 }
I execute the query in NodeJs with the library node-mssql-connector.
Why is SQL giving my the wrong hours?
In your format string, yyyy-MM-dd hh:mm:ss, hh means you want the hours in the 12-hour-cycle format, so 3 and 15 are always 3 (AM and PM). Use HH to get them in the 24-hour-cycle format:
yyyy-MM-dd HH:mm:ss
Relevant docs, scroll down to the list of format specifiers.
You should use HH instead of hh:
SELECT FORMAT(MAX(dTime),'yyyy-MM-dd HH:mm:ss') FROM triangulations
Usually when you get bad hours while the minutes and dates are fine, in means that you're using the wrong time zone. This could mean that either the time in a wrong time zone was written to the database, or you're getting the date from the database in some other time zone that you're expecting.
You should always be explicit about time zones when working with dates (which you don't seem to be doing here when while putting the dates into the database), and it makes things much easier if you're always saving dates in UTC (which seems to be the case for the dates that you're reading here).
In Node you can convert the dates using the Moment module - Moment Timezone in particular. See:
https://momentjs.com/timezone/
See this answer for some examples - it's about Mongo instead of SQL server but you can use exactly the same conversion here:
Mongoose saving and retrieving dates with UTC time, change to server timezone
Short version:
I have 2 users in DB, each created in a different timezone:
User.find(1).created_at
=> Thu, 04 Aug 2016 11:15:29 IDT +03:00
User.find(33).created_at
=> Sun, 01 Jan 2017 17:50:20 IST +02:00
So my table shows 11:15, and 17:50. so for example I would like to search for 17:50, and later 11:15 as text:
search_param = '17:50'
No problem I'll just convert the date to text, then search it, But user won't be found, since the it's saved as UTC:
User.where("to_char(created_at,'DD-Mon-YYYY HH24:MI:SS') ilike ?", "%#{search_param}%").first
=> nil
To find it I'll just apply the offset to my query (adding time zone UTC+2), and indeed user was found:
User.where("to_char(created_at AT TIME ZONE 'UTC+2','DD-Mon-YYYY HH24:MI:SS') ilike ?", "%#{search_param}%").first
=> User #33 #2017-01-01 17:50:20 +0200
BUT some users are saved as UTC+3 and some as UTC+2.. I can't apply both offsets... So if I change search_param to be 11:15 I won't find user_id_1 because I will also need to change UTC+2 to UTC+3
When I use UTC+2 I will only find users which were created as +2 (like User.find(33)).
When I use UTC+3 I will only find users which were created as +3 (like User.find(1)).
My question: How to do a where query- a text search for both users' created_at hour, as they were both saved in a different timezone offset?
Or in this example a query that for a search_param 17:50 will find user_id_33 and for search_param 11:15 will find user_id_1?
More Details:
I notice in DB they are saved as UTC (I think):
User.select("created_at as created_at_db").find(33).created_at_db
=> "2017-01-01 15:50:20.903289"
User.select("created_at as created_at_db").find(1).created_at_db
=> "2016-08-04 08:15:29.171776"
Time setting:
#application.rb
config.time_zone = 'Jerusalem'
config.active_record.default_timezone = :utc
created_at column info:
User.columns.select{|table_col| table_col.name == 'created_at'}
=> [#<ActiveRecord::ConnectionAdapters::PostgreSQLColumn:0x81f9c48
#coder=nil,
#default=nil,
#limit=nil,
#name="created_at",
#null=false,
#precision=nil,
#primary=false,
#scale=nil,
#sql_type="timestamp without time zone",
#type=:datetime>]
You should be able to query your user entities with:
date_trunc('minute', created_at AT TIME ZONE 'UTC' AT TIME ZONE 'Asia/Jerusalem')::time = '17:50'
Or, with to_char() (but less index-friendly):
to_char(created_at AT TIME ZONE 'UTC' AT TIME ZONE 'Asia/Jerusalem', 'HH24:MI') = '17:50'
The point is that created_at is timestamp (or timestamp without time zone, which is just an alias). First, you can tell PostgreSQL, that interpret its values as they were in UTC, with the <timestamp> AT TIME ZONE <time zone> operator:
created_at AT TIME ZONE 'UTC'
Then, tell PostgreSQL to offset this value as if it were a local time in Asia/Jerusalem with the <timestamp with time zone> AT TIME ZONE <time zone> operator (which is completely different from the operator above with the same name):
created_at AT TIME ZONE 'UTC' AT TIME ZONE 'Asia/Jerusalem'
Now, you just need to truncate this value to extract only the hour & minute part.
Maybe worth mentioning that using these:
created_at AT TIME ZONE 'UTC+2'
created_at AT TIME ZONE 'UTC+3'
worked for you accidentally. These create timestamp with time zone values within the -2 and -3 UTC offsets, respectively. Then, because your config.active_record.default_timezone is set to :utc, it is sent to your client (ruby) within the UTC time zone, which means it added 2 and 3 hours, respectively.
Another issue to keep in mind is that in POSIX time zone names, positive offsets are used for locations west of Greenwich. Everywhere else, PostgreSQL follows the ISO-8601 convention that positive timezone offsets are east of Greenwich.
I have a restaurant that has hours. Each Hour object as an open_at and a close_at time. How do I compare just the time part of the DateTime in the db with the time of Time.now?
hours.today.where("open_at <= ? AND close_at >= ?", Time.now, Time.now).any?
edit: I tried rephrasing the question:
I have an Hour object h with an attribute close_at of type Time. How can I compare just the time part in the db with Time.current?
2.1.3 :014 > h.close_at.class
=> Time
2.1.3 :015 > h.close_at
=> 2000-01-01 22:44:00 UTC
2.1.3 :016 > Time.current
=> Wed, 15 Oct 2014 17:32:27 UTC +00:00
2.1.3 :017 > h.close_at >= Time.current
=> false
I want the last line to return true. (22:44 > 17:32)
I believe that you are trying check if there exists any hour that has been opened today and closed any time later from now:
Assuming this, you can do the required by following:
Hour.where(:open_at => (Time.current.midnight)..(Time.current) , :closed_at => (Time.current)..(1000.years.from_now) ).any?
Basically the key point is that , you can pass ranges in where clause.
And if you dont have any upper bound or lower bound , you can always use this to define the range:
1000.years.from_now
1000.years.ago
for example, any time later from now could be represented as:
Time.current..(1000.years.from_now)
Obviously you would not be making a app that needs to store time for 1000 years back and forth from now! I believe this approach should workout out in your case.
Edit:
I am sorry for misunderstanding your question. This is the workaround to the problem:
closing_time = h.close_at
# closing_time => 2000-01-01 22:44:00 UTC
closing_time = Time.current.midnight + closing_time.hour.hours + closing_time.min.minutes + closing_time.sec
#this shifts the closing date_time to the current date, without affecting hours, minutes and seconds, So,
# closing_time => 2014-10-15 22:44:00 UTC
current_time = Time.current
# current_time => Wed, 15 Oct 2014 17:32:27 UTC +00:00
closing_time >= current_time
#=> returns true (because 22:44 > 17:32 , with other things remaining the same.)
Hope it helps :)
If you want to compare time IN SQL and if you are using MySQL you can use its TIME() function, from docs:
Extracts the time part of the time or datetime expression expr and returns it as a string.
So you can do:
hours.today.where("TIME(open_at) <= ? AND TIME(close_at) >= ?", Time.now, Time.now).any?
I collect user time zone and a time they want something delivered in that time zone.
My application is set to use Bangkok Time.
application.rb
config.time_zone = 'Bangkok'
I want to store the time the user has entered relative to my timezone.
So say for example the user selects the following:
time = "6:00"
timezone = "(GMT-10:00) Hawaii"
I want to convert this time to the time of my application and save it in the database.
How can I convert from 6AM Hawaii time zone to what time this is in Bangkok?
Let us use your example to demonstrate:
config.time_zone = 'Bangkok'
time = "6:00"
timezone = "Hawaii"
The first thing you need to do is set the timezone of your app to the timezone that the user picked:
Time.zone = "Hawaii"
You then use the this new time zone to parse time time:
hawaii_time = Time.zone.parse("6:00")
If we now print out hawaii_time we get something similar to (we have HST -10:00 which means it is correct):
Mon, 15 Aug 2011 06:00:00 HST -10:00
You now use the rails in_time_zone method to convert to your app's timezone:
local_time = hawaii_time.in_time_zone(Rails.configuration.time_zone)
If we print out local_time we'll get something similar to (we have ICT +07:00 which is correct for Bangkok):
Mon, 15 Aug 2011 23:00:00 ICT +07:00
We can now reset the timezone of our app back to its original value:
Time.zone = Rails.configuration.time_zone