I am having a bit of a problem when saving times/dates to MariaDB.
Because of the UTC format that dates and times are saved it can save times 1 hour behind and dates can be saved as the previous day.
Most of the work arounds I have seen, involve pulling the data back from the DB and converting it, however I was keen to save it to the DB with the correct dates/times already entered.
I done this using {{moment.utc("Date_Entry").tz("Europe/London").format()}}
The problem I have at the minute, though, is...
If I want to leave the date/time empty, the function above saves the current date/time.
Ideally I would like to handler the UTC conversion if required, but if the entry is empty, save a NULL value to the DB
Does anyone know of a better solution than what I have currently?
Related
I'm having some trouble with BigQuery and going around in circles with support but it boils down to this.
I have manually entered, handwritten form data I want to get into BigQuery. Later I will be creating some electronic means to replace the paper forms but for the moment I have thousands of rows of data that are largely in excel. I have converted them to CSV, massaged any excel created formatting errors but am stuck when it comes to timezones. BigQuery has 2 formats - timestamp and datetime. The first being a UTC time and the second (after some digging) intended to be just 'what someone would see on a watch' so call it local time.
I can create a table with a field with the datatype of timestamp and do an insert with "2019-12-01 11:34:00 Australia/Adelaide" and it will convert the time to UTC and insert. If I take the exact same value and knock it up in a quick CSV field (manually not through excel) BigQuery will produce an error of 'Unrecognised timezone: Australia/Adelaide; Could not parse "2019-12-01 11:34:00 Australia/Adelaide' as timestamp for field tsfield (position 0) starting at location 0'
To get my historical information in Im probably going to have to give up, convert everything to UTC before I upload just to get it in but Im now concerned about what happens when I try to do this programmatically later too.
edit: For wayward future traveler searching for the same answer... I decided to just convert the values myself for BigQuery and lean into it. I converted all times before uploading into UTC and then let it just apply the timezone stored as a 'timestamp' data type THEN for 'local time' I used the 'datetime' type which does not include a timezone suffix and left it in its original time. I don't have anything telling you where 'local time' is but hopefully can derive from the UTC for most people.
I have a vb.net program, updating the time value in an access database. The database is connected using OleDB.
Basically this is what is happening:
Dim commandBuilder As New OleDb.OleDbCommandBuilder(dataEventAdapter)
eventDataset.Tables("EventList").Rows(selectedEvent)("EventTime") = Format(dateTimePick.Value, "hh:mm tt")
dataEventAdapter.Update(eventDataset, "EventList")
The time is taken from a datetime picker, and it should store only the time value.
The problem is, that the database already has values in it, which only has the time, like: 9:00 AM, but when I'm updating with this, it gets the date as well. And honestly I don't know where it gets the date from. If I
MsgBox(Format(dateTimePick.Value, "hh:mm tt"))
I get only the time, and nothing else.
How can I store the time only?
If you look at the datatypes available in MS-Access you will find that there isn't a type just for Time values but there is a type for Date/Time values. This means that Access will store always the date AND the time for the values that you supply. The display that you observe looking at the MS-Access grid is controlled by the Format setting in the structure page of your table and here you could change it to show just the Time part of your data.
Said that, there is the problem that you don't supply a DateTime value, but a string. Access is gracious(?) enough to not trigger an exception for this, but compensates adding a date by itself thus you should see the current day for every value that you supply.
So you shouldn't be concerned about how your value has been displayed, but more on how you pass that value to the database. If only the time part is meaningful for your program then leaving the database engine convert back your string to a datetime value is not an option. (Without talking about the localization issues that this automation will involve)
I suggest to pass a constant value for the Date part (like DateTime.MinValue or 1/1/1) and add your time to this value. In this way you could easily ignore the date part if you eventually need to use some queries on this data.
Dim dt As DateTime = new DateTime(1,1,1, dateTimePick.Value.Hour, _
dateTimePick.Value.Minute,
dateTimePick.Value.Second)
eventDataset.Tables("EventList").Rows(selectedEvent)("EventTime") = dt
You can make a simple experiment in Access. Open the Immediate window with Ctrl-G and enter
?Format(#00:00:00#,"yyyy/mm/dd hh:nn:ss")Enter
1899/12/30 00:00:00
?Format(#08:31:57#,"yyyy/mm/dd hh:nn:ss")Enter
1899/12/30 08:31:57
The result shows you the origin Access uses for its time axis.
Another experiment shows this:
?#1899/12/30 08:31:57#Enter
08:31:57
Access automatically displays only the time part for the date 1899/12/30.
Therefore I suggest to use this date as a base for time-only data.
Access uses Double values to store dates internally, where the integer part represents the number of days elapsed since 1899/12/30 and the decimal fraction represents the time as fraction of 24h (i.e. 0.25 is 06:00 am and 0.75 is 18:00).
?CDbl(#1899/12/30 08:00:00#)Enter
0.333333333333333
?CDbl(#1899/12/30#)Enter
0
?CDate(0)Enter
00:00:00
?CDate(0.25)Enter
06:00:00
In .NET you can use the System.DateTime.FromOADate(d As Double) As Date method for the conversion of Access Dates given as Double to .NET Dates (VB Date = System.DateTime).
You are confusing data types with formatting. In your database the column you are inserting into has a datetime datatype (Access has no data type for just time). This means that it stores everything that goes in there as a date + a time.
If in Access you are seeing values with only a time, it's likely that Access decided the date is useless (possibly because it was stored with a date of 1/1/1900).
Thing to remember is that the date still being stored. When you re-display the data just format it to only display the time. Judging from your code example you already know how to do that.
I have an application that stores date and time in a string field in an SQL Server 2008 table.
The application stores the date and time according to the regional settings of the PC that is running and we can’t change this behavior.
The problem is that some PCs have to be in UK date format with 12h time (eg. 22/10/2011 1:22:35 pm) some with UK date format with 24h time (eg. 22/10/2011 13:22:25) and some have to be US date format (eg. 10/22/2011 1:22:35 pm) and (eg. 10/22/2011 13:22:25).
Is there any automatic way to change the string every time it changing/added to the table to UK 24h format so it will be always the same format in the database?
Can it be done using some trigger on update or insert? Is there any built-in function that already does that?
Even a script to run it from time to time may be do the job...
I’m thinking to break apart the string to day, month , year, hour, minute, second , AM/PM and then put the day and month part in dd/mm order and somehow change the hour part to 24h if PM, get rid of the “am” and “pm” and then put the modified date/time back to the table.
For example the table has
id datestring value Location
1 15/10/2011 11:55:01 pm BLAHBLAH UK
2 15/10/2011 13:12:20 BLAKBLAK GR
3 10/15/2011 6:00:01 pm SOMESTUFF US
4 10/15/2011 20:16:43 SOMEOTHERSTUFF US
and we want it to be
id datestring value Location
1 15/10/2011 23:55:01 BLAHBLAH UK
2 15/10/2011 13:12:20 BLAKBLAK GR
3 15/10/2011 18:00:01 SOMESTUFF US
4 15/10/2011 20:16:43 SOMEOTHERSTUFF US
We can display the date parts (day,month,year) correctly using the datepart function but with the time part we have problems because it changes too many ways.
Edited to explain some more
mr. p.campbell thanks for the edit .. i didn't know how to beautify it :)
and mr. Matthew, thank you for your quick reply..
We can tell if it is UK date or US date because we have another field i didn't mention with the text "US", "UK", "GR", "IT" according to where the PLC machine is located.
I'm sorry i didn't explain it to well. My english are not so good.
There are two different and independent applications. And they don't have direct relation with the sql server.
The application that only writes data to the database ..lets call it "the writer" for short.. and a different application that reads the data .. lets call it "the reader".
"The writer" is an internal application of a PLC machine that stores values every 1 min to the database that's why we can't change its behavior. It uses the string data type to store the date and the time at the same field according to the regional settings of the pc that a daemon application runs and does the communication between the pc and the PLC machine.
Now "the reader" expects the date and time to be in the format "dd/mm/yyyy 23:23:01" or "yyyy/mm/dd 23:23:01" and the only thing it does for now is doing some calculations with the data in the value field between given dates. eg. from 10/09/2011 10:00:00 to 15/09/2011 14:00:00.
we just need to do something like this ...
select * from table1 where datestring between "10/09/2011 10:00:00" and "15/09/2011 14:00:00"
I could post some of the code but it will be very long post.
At first, I agreed with Matthew, but then I realized that, given the information presented, this actually was possible (well, sorta).
However, some caveats;
You are doing nobody any favors by storing and maintaining the database this way. Your best bet is to change the application to have it give an actual Datetime value, not this mangled string.
This data CANNOT be meaningfully sorted by date or time (not without performing expensive string manipulation).
You appear to be storing all times as local times, but do not appear to be storing a TimeZone or related information. Without this information, you will NOT be able to (completely) correctly translate times 'globally'. For instance, which is later - 4PM in London, or 11AM in New York (for, say, an international conference call)? The answer is that you don't know: it depends on the time of year.
You are storing local times, period. This only works so long as local time is correct. What happens when somebody sets their clock to 1900? You should be storing time based off of the SERVER'S clock.
Your stored timestamp is based on a formatted string. If the user changes how their time is displayed, your data correctness (potentially) goes out the window. For instance, what if somebody removes the am/pm symbols, thinking "I'll look out the window - if the sun is out, it's 'am'"?
Please keep all of that in mind.
As to how to do this....
I'm not going to actually write out the SQL statement for this. Mostly because storing the information this way is pretty terrible. But also because it's going to take a lot of work I'd rather not do. I really recommend stressing to whomever has the keys at your place to get that application changed.
So instead, I'm going to give you a really big clue - and this will only work for so long as your timestamp format remains the same; You should be able to tell what format the date and time are in based on the presence and absence of 'am' and 'pm' in the string (if you don't have both, you're flat-out toast). As Matthew has pointed out, the formatting is also likely different for the date, as well as the time - you will need to translate both. However, this will immediately give you problems due to comparative timestamps (please see point 4, above); any attempt to run scheduling or auditing with this data id pretty much doomed to failure ("When did that happen?" "Well, it's in the UK date format, so..." "But that makes it 1AM here, and he was dead then!").
Most beneficial answer: Change how the information is stored in the database
EDIT:
And then it hits me (especially in light of the new edits) - there are potentially other possibilities that could actually make this work....
First, change your database to actually store some sort of 'globalized' timestamp, based off of the server's clock.
This will of course break your existing application code - it would get a data-type mismatch error. To fix that, rename the table, then create a view, named the same as the original table, that will return the string formatted as indicated in the 'source' column. You'll need to create instead-of triggers for the view, to translate the formatted string to an actual datetime value. The best part is, the application code should never notice the difference. You seem to have indicated that you have sufficient control over the database to allow this to happen; this should allow you to 'fix' the data transparently.
This of course works best if the incoming datetime values are absolute (not local). Hopefully, the values are actually supposed to be 'insert time' - these could likely be safely ignored, in favor of using a special register (like NOW or CURRENT DATE or whatever).
Can't believe this didn't hit me earlier...
You stated that you cannot change the application behavior, thus this is not possible.
Your problem is that your database doesn't know the culture / timezone settings of the client and your client doesn't report it.
You will need to report this data or think of clever ways to infer this information before you can act on it.
EDIT: For example, without knowledge of the client's details how could you tell the difference between the strings:
10/1/2011 12:00:00 (October First, noon, US)
10/1/2011 12:00:00 (January Tenth, noon, UK)
?
I posted a question a couple of days ago (SQL Reporting Services Daylight saving time query) which was I received an answer for (thanks very much) but did not elaborate on the whole problem I am experiencing. Not only did I require the returned date time format to account for day light saving but I also need the search parameter #StartDate to allow for DST.
Currently if I key in a scheduled start time of 31/03/2010 11:00 and because the SQL DB has already taken the hours difference into consideration I get no results back. If I key in 31/03/2010 10:00 then the correct details are returned. Is there away using T-SQL or the like to get the search parameter to pass the adjusted time to the DB?
SQL Server 2008 supports the data type "TIMESTAMP WITH TIMEZONE". Might be worth a try.
You can download a 180-day trial version.
In any case, the timezone for an appointment application is what I'd call critical business information. I'd store it in the database, by one of these methods:
using the right data type
building a user-defined data type
adding a column for time zone info
We have two databases, in two separate locations. One of the databases resides in a separate time zone than our users.
The problem is that when the database that is located in a separate time zone is updated, with a Date value, the database automatically subtracts 1:00 hour from the Date it was passed.
The issue is that, when passing a NULL date (12:00:00), the DAY value is changed to a previous day.
The updates are done via stored procedures, and the front end is a VB.NET smartclient.
How would you handle this the proper way? I basically don't even want to store the TIME at all, but I can't seem to figure out how to do that.
Not clear on what datetime you want in the database, or what the application is passing.
Assume the user's PC is telling him it is Tuesday, 12:30am, and the clock on the Db server is saying Monday, 11:30pm.
If you insert a value for the 'current date' (eg TRUNC(SYSDATE)) then, as far as the database is concerned, it is still Monday.
If you insert a value for the 'current time (eg SYSDATE), it is also still Monday.
if you insert a value for the session's current time (eg CURRENT_TIMESTAMP) and timezone and ask the database to store it in the database, it will store 11:30pm.
If you ask the database to store the datetime '2009-12-31 14:00:00', then that is what it will store. If you ask it to store the datetime/timezone '2009-12-31 14:00:00 +08:00', then you are in the advanced manual. You can ask the database to store timestamps with timezone data. Also consider daylight saving
I would investigate using the TRUNC function in your stored proc method that updates the table. If the data type in the method (that updates the table) is not a DATE type then use the to_date function in conjunction with the TRUNC function.
This is outside of the scope of the question you are asking, but I would recommend in ALL cases where users are accessing a database from different time zones, the server and database clocks time zone should be set to UTC. It is probably too late for that, but setting the datbase server to UTC eliminates the problems caused by daylight savings time and different time zones.
In my opionion, Date/Time data can and should always be stored in UTC. This data can be converted to local time at the point where it is presented to the user. Oracle actually makes this easy with the TIMESTAMP with TIME ZONE data type. It allows you to access the data either as UTC (SYS_EXTRACT_UTC) or local time (Local to the database server.)
It is never the same day all places in the world, so dates cannot be considered without time.
Of course another of my opinions is that Daylight Savings time should be eliminated. But that is another topic.