I'm using an ADO\ACE.OLEDB query to access data in an excel sheet. One of the columns in this sheet is a time stamp in the 00:00:00 format.
My select max query for this column works OK but it returns the wrong "max" time. For example, if I have the following time stamps:
08:15:00
09:45:00
10:00:00
The SELECT MAX query will return 09:45:00, and ignore any subsequent timestamps. If I go to the queried sheet, and convert all the cells in that column to "Time" using text-to-columns, the query will then return the correct value of 10:00:00.
The timestamp that is recorded in the excel sheet is updated very often by many different people throughout the day using their own ADO Update\Insert query so it's not feasible for me to open the sheet and convert the cells to the correct type manually. The update\insert queries have the cell types set to General.
So my question is this.
Is there a way to either
A) Specify the data type in the update\insert into ADO query so the cell has the correct data type? Or
B) Force the SELECT MAX query to interpret the column as a timevalue rather than text?
If not any suggestions on how to procede?
EDIT: I looked closer at the excel sheet i'm querying. I noticed that each cell has a singlequote preceding it. I think if I can get that removed when entries are recorded excel might properly format the cell as time.
Here is a basic example of wha is being used to write entries to the sheet.
Update [Sheet1$] Set StopTime='09:45:00' WHERE Date= '3/12/2014' AND StartTime='9:30:00'
Can this be rewritten so a single quote ins't included? Note there is no end quote.
The single quote at the beginning of a cell in Excel forces Excel to interpret the cell value as text. The single quote is probably being inserted because the update statement uses single quotes, which indicate strings. You may be able to use ## delimiters instead, which specifically indicate date/time values (I say "may" because I haven't tested this). For example:
UPDATE [Sheet1$]
SET StopTime = #09:45:00#
WHERE [...]
On the other side, you can force a column to be interpreted as a date/time by using the CDate function:
SELECT MAX(CDate(StopTime)) AS MaxTime
FROM [Sheet1$]
You just have to be careful in this case about values that cannot be converted to a valid time, which will cause the query to error out.
I didn't even realise this but the examples I gave were flawed. What is logged does NOT include a leading zero. Since I'm using a 24hr time I was able to correct this issue by writing a leading zero into the time that gets recorded.
Basically it's an, if LEN(time)=7 then time = 0 & time.
This appears to be working so far.
Related
A value coming back from a DB2 query passes a zero date as 0001-01-01 00:00:00.000000 and I am having a problem testing for this value in a Conditional Split condition. I tried a few different things but either they stay red or cause an error when run. I even tried testing for the date being less than the SQL start date. Neither of these work.
(DT_Date)DRIVER_TERMDATE < (DT_DATE)"1753-1-1"
DRIVER_TERMDATE != "0001-01-01 00:00:00.000000"
The minimum date for the DT_DATE data type is 1899-12-30, which is after that date and trying to convert it to this data type will cause an error. If you're importing the DRIVER_TERMDATE as text, trying adding a condition for this string, for example
DRIVER_TERMDATE == "0001-01-01 00:00:00.000000"
However it looks like you already tried checking to see if DRIVER_TERMDATE is not equal to this value, so see if trailing zeros are being removed during from the text during execution by right-clicking the output from the DB2 source, selecting Enable Data Viewer, and examining the data when the package runs. Once you've confirmed the exact string that's being sent in, you can add a condition for this.
I have written a program that logs events into a db file as they happen using SQL statement. Whenever I open the table to view, I specifically request the dataset in descending order based on ALARM's date and time. It seems to work only for part of the table. I am using SQLite3 and my program is written in delphi or Pascal.
Here is the SQL statement:
SELECT *
FROM Alarms
ORDER BY datetime(ALARMTIME) DESC
Here is a snapshot of the table. Pay attention to the red arrows. That's where Alarm's date and time doesn't follow descending order. I don't know why this is happening.
I'm not sure how you created your date/time string in your database since that information wasn't given in the question. However, according to the documentation for the datetime() function, the format you have of MM/DD/YYYY HH:MM:SS xx is not one of the accepted formats. In fact, if you do SELECT datetime('1/23/2018 01:40:00 PM') at an SQLite prompt, you get NULL. Whereas, if you use an acceptable format, you do not: SELECT datetime('2018-01-23') gives '2018-01-23 00:00:00'.
So I think the solution is to write the ALARMTIME field using datetime('now'), which does yield a format accepted by datetime(). If you need to sort based upon the format you have now, you can't do it with datetime(). You'd need to reformat the field using string functions to get it in a form you could do a string compare with (e.g., as is done here).
The OP has indicated in a comment that the ALARMTIME is set using the following Pascal code:
FieldByName('AlarmTime').AsDateTime := now;
There's no guarantee that Pascal is going to use a date/time string format in this context that is compatible with SQLite's datetime() function. So Pascal date/time formatting functions can be used to create a format more specifically acceptable by SQLite's datetime(). Then you'd use something like:
FieldByName('AlarmTime').AsString := FormatDateTime('YYYY-MM-DD hh:nn:ss',now);
Now this will change the default view of the date to YYYY-MM-DD .... If you still want your view of the table to show MM/DD/YYYY... then you'll either need to go back to my prior comment about processing the string on the fly in the comparison for sort, or write a little view formatting code so that it displays in a format that's different than what is stored internally, which is a common view/model separation technique.
If you can write your original ALARMTIME format as MM/DD/YYYY and make sure you pre-pad with zeroes (e.g., 01/09/2018 not 1/9/2018) then you can use SQLite's substr function:
SELECT * FROM Alarms
ORDER BY (substr(ALARMTIME,7,4)||substr(ALARMTIME,1,2)||substr(ALARMTIME,4,2)||substr(ALARMTIME,11)) DESC
And you would create your ALARMTIME using this:
FieldByName('AlarmTime').AsString := FormatDateTime('dd-mm-yyyy hh:nn:ss', now);
The above solutions are fairly generic. Depending upon the client library you are using (which you have not specified), there may be another more suitable approach to solving the problem.
You cannot do, for example, SELECT * FROM Alarms ORDER BY ALARMTIME DESC and get an accurate sort since, for example, the date 12/1/2018 would come after 2/1/2018 in that sort ordering even though 2/1/2018 is later in time. This is because 2 follows 1 in the ASCII collating sequence.
If you need to keep your current ALARMTIME string format and not change how you're saving it, which is somewhat free form m/d/yyyy in which the day or month can have one or two digits, you're going to have a bit of work to do in order to sort it if your client library doesn't support some helpers in this regard. Perhaps your only other option would be to use a custom SQLite function. These are written in C and compiled and linked with SQLite. You'd have to find one already written, or write your own.
I am parsing and modifying large files (about a gig per month) which contain a record of every interaction. These files are sent to me by our client, so I am stuck with what they contain. I am using pandas to clean them up a bit, add some information, etc.
I keep running into issues where, out of 1 million+ rows, 1 to 10 values in the datetime column are not a date. A value meant for another column is in the date column due to some issue with comma separation (this is from the client's query, not mine) so it might say the word 'Closed' or something.
How do I drop these rows? I can see the ones with errors when I use df.sort('Datetime'). I just want a way to drop these quickly.
Here are my ideas:
There is a column called 'TransID' which ALWAYS begins with the
letter 'H' (and it always is 9 digits) UNLESS there is an error when another column value has shifted into this column
The date column should always have a value (notnull)
Can someone help think of a way to solve this problem? (I think this date thing is the key issue because I have formulas which subtract StartDate from EndDate.. if one of those contains a word then it messes up the entire process. Maybe I can create some error exception or drop error rows?)
Use the H column to filter out the error rows using a boolean index and the vectorized string methods.
good_rows_mask = df.TransID.str[0] == 'H'
df = df[good_rows_mask]
Is there any way to QUICKLY convert numbers/dates stored as text (without knowing exactly which cells are affected) to their correct type using VBA.
I get data in an ugly text-deliminated format, and I wrote a macro that basically does text-to-columns on it, but is more robust (regular text-to-columns will not work on my data, and I also don't want to waste time going through the wizard every time...). But, since I have to use arrays to process the data efficiently, everything gets stored as a String (and is thus transferred to the worksheet as text).
I don't want to have to cycle through every cell, as this takes a LONG time (these are huge data files - I need to use arrays to process them). Is there a simple command I can apply to the entire range to do this?
Thanks!
This has to do with the data type of the columns modify the column from general to the correct data type and the placement of text data should get automatically converted... here's an example where I pasted the text 012345 into different columns having different data types. Note how the displayed value is different for the different types but the value is retained (except on number and general which truncate a leading 0.
However if you don't know what field is of what type... you're really out of luck.
There is a way is there. Just multiply 1 with the data in the column have text to converted as number, whether it is text or not it will convert to numbers only.
Read the following the link for more.
http://chandoo.org/wp/2014/09/02/convert-numbers-stored-as-text-tip/
I am from non database background. I have created a table with one of the field data type TEXT.
dataF TEXT
INSERTION:
Have inserted three records with the values :
'842-2-4'
'842-2-5'
'842-2-6'
SELECTION:
Tring to get the records based on date now.
... where dateF between '842-2-4' and '842-2-10'
It fails.
Whereas,
... where dateF between '842-2-4' and '842-2-8'
retrieves all the 3 records.
What am i doing wrong ? Why the first statment fails ?
Kindly suggest.
Because you are comparing strings not dates. The computer has no idea these are dates. You have to either store as date and do a date comparison or implement your own logic to analyze strings.
Simply put, it is looking at the 1 in 10 as being less than your values rather than 10 being more. It's string comparison, not date.
Although sqlite doesn't support date types, it does have functions for dealing with them. See here:
http://www.sqlite.org/lang_datefunc.html
When comparing strings, the values are compared left to right...
As one string is shorter that the other, you are kind of comparing this...
'842-2-4'
'842-2-1'
Well, nothing is >= '842-2-4' AND <= '842-2-1'.
Because '842-2-1' comes before '842-2-4'.
And, so, '842-2-10' comes before '842-2-4' too.
Just as 'Squiggled' comes before 'Squiggly'
And as 'xxx-y-az' comes before 'xxx-y-z'
To compare as you desire, make sure all your dates are padded with 0's.
BETWEEN '0842-02-04' AND '0842-02-10'
But that will only work after you have padded out the values in your table too.
EDIT:
Also, note that this assumes that your format is YYYY-MM-DD. As a string is compared left to right, you must have the highest magnitude values to the left.
(This means that you can't use YYYY-DD-MM and still have native string comparisons behave as you would want them.)