Just trying to clean up some functions someone else has done in a postgres.
Would anyone know what the following does? It was working, but the date format changed when it started coming in as '1999-09-07 16:30:00.000'
I don't know what the previous format was.
select
case
when dbDate = '' then null
when dbDate != '^\d.*' then dbDate::timestamp
else '1900-01-01'::date + dbDate::int
end as dbDate
Whenever I call the function with the date it gives me
invalid input syntax for integer: "1999-09-07 16:30:00.000"
This nice function was taking multiple kinds of date inputs and normalizing them.
I assume it expected one of:
blank which it let be null
a date starting with 'MMM' in date format which would not pass '^\d.*' (i.e. something that doesn’t start with a number) which it would cast as a date
a number
The reason that the date was being casted as an INT is because after failing the first two tests the person writing this was expecting an INT. They wanted to add the integer to the beginning of Time (i.e. 1900-01-01) like Excel does.
1999-09-07 16:30:00.000 fails the second test even though it could be cast as time.
This passes through to the else, which fails to cast it as INT, and throws the error.
In this case, you need to change your second test. Make it something that will allow a datetime that you have coming in, but that would reject a number that should be added to 1900-01-01.
If you don’t think you will have numbers coming in that should be added to 1900-01-01, then just get rid of the third test and use
select
case
when dbDate = '' then null
else dbDate::timestamp
end as dbDate
Related
I have a query that was working fine before a server migration and now is not working. I'm trying to convert all dates to a specific month and year, but I keep getting this error:
Conversion failed when converting date and/or time from character string.
Looking into the data, there are no null values in InputDate, which is a date data type column. When I run the Concat() function everything is formatted as 'YYYYMMdd', yet both CAST and CONVERT fail with the same error.
Is there an issue with my query that I'm not seeing?
SELECT RandoSTUFF,
DATEADD(day,2,CAST(CONCAT('2023','02',FORMAT(InputDate,'dd')) AS date)) AS MovedDate
FROM a_table_
I expect the issue is you have date values near the end of their months, and you're trying to compose the equivalent values for February, which is shorter.
So if you have an InputDate value of, say, 2022-12-31 and run the code in the question, it will extract the 31 and concat it with the other values, and you'll end up trying to do this:
CAST('20230231' as Date)
Of course, there is no such date.
As it is, it's not clear whether you want such an input to map to February 28 or March 3. To fix this, you'll need to rethink the problem so you only try to map to valid dates, and ensure the final result is more clearly defined. This is one of the many reasons it's almost always better to use Date/time functions instead of composing dates from strings.
I am trying to convert a date that's stored as a string to a date, e.g.
YYYYMMDD (string) to YYYY-MM-DD (date)
As far as I know there is no conversion function that checks input format and output format, I tried manual logic, e.g.
CASE
WHEN CHAR_LENGTH(TRIM(some_string_date)) = 8
THEN
CAST(
SUBSTRING(TRIM(some_string_date) FROM 1 FOR 4)
|| '-'
|| SUBSTRING(TRIM(some_string_date) FROM 5 FOR 2)
||'-'
|| SUBSTRING(TRIM(some_string_date) FROM 7 FOR 2)
as DATE)
ELSE
NULL
END
However this is not accepted by Apache SQL Validator, does anyone see problem here?
Not directly answering the question, but maybe related, date literals are declared with DATE keyword, e.g. you can see examples in the tests in Beam tests: one, two and in Calcite docs.
Update:
What seems to happen is Calcite adds some indirection when doing CASE. Casting the strings to dates works as expected in general. For example, if input rows have schema (INT f_int, VARCHAR f_string) and dates are in 'YYYYMMDD' (e.g. (1, '2018'), then this works:
SELECT f_int,
CAST(
SUBSTRING(TRIM(f_string) FROM 1 FOR 4)
||'-'
||SUBSTRING(TRIM(f_string) FROM 5 FOR 2)
||'-'
||SUBSTRING(TRIM(f_string) FROM 7 FOR 2) as DATE)
FROM PCOLLECTION
Even directly casting the 'YYYYMMDD' works:
SELECT f_int,
CAST(f_string AS DATE)
FROM PCOLLECTION
You can see all supported date formats here.
But as soon as you wrap it in 'CASE ... ELSE NULL', then Beam/Calcite seem to infer that the expression type is now a 'String'. This means that 'THEN CAST(... AS DATE)' succeeds and returns a 'Date', but then it's converted to 'String' when wrapped in 'CASE'. Then, when returning the result in my test it seems to try to cast it back to 'Date', but the string format now is not 'YYYYMMDD' but some other default format. Unfortunately that format is not in the list of supported, so it fails.
Workaround:
As soon as you change 'ELSE NULL' to something that's known to be a 'Date', e.g. 'ELSE DATE "2001-01-01"' then it works again as Beam/Calcite don't seem to go though 'String'->'Date'->'String'->'Date' path and this works:
SELECT f_int,
CASE WHEN CHAR_LENGTH(TRIM(f_string)) = 8
THEN CAST (
SUBSTRING(TRIM(f_string) FROM 1 FOR 4)
||'-'
||SUBSTRING(TRIM(f_string) FROM 5 FOR 2)
||'-'
||SUBSTRING(TRIM(f_string) FROM 7 FOR 2) AS DATE)
ELSE DATE '2001-01-01'
END
FROM PCOLLECTION
I filed BEAM-5789 to track a better solution.
Update 2:
So, while Calcite generates the plan telling Beam what to do, it's Beam that actually casts/parses the dates in this case. There's an effort to use Calcite's built-in implementations of basic operations instead of re-implementing everything in Beam:
https://github.com/apache/beam/pull/6417 . After this pull request is merged, this CASE ... ELSE NULL path should work automatically, if I'm reading it right (I assume this class will be used for handling date/time values). It will still go through strings, probably unnecessarily, but it should just work.
If is MYSQL.
You may try
SELECT DATE_FORMAT(STR_TO_DATE('20080908', '%Y%m%d'), "%Y-%m-%d");
For validate, you may check whether the string could be converted to date successfully. sometimes. NULL means failed.
My users enter parameters into an sql query from a front end system. I have successfully passed strings before, but I would like to pass a date variable now. My SQL is as follows:
SELECT sum(CASE WHEN P.SNAPSHOT_DATE = '{?startdate}' THEN P.MKT_VAL ELSE 0 END)
FROM P
The front end has some sql that defines what startdate is.
As I say, with strings, this works a dream. Unfortunately my compiler is expecting a date and so I get the following error:
Conversion failed when converting date and/or time from character string.
I have tried surrounding my variable with cast and convert to no luck.
I think in your query, when you get the user input convert that input to a date type ie using SQL
Set #userinput= select CAST(#userinput as date)
On the front end try to limit the amount of manual inputing of data by the user, by giving them the ability to select a date from a calendar rather than manually typing in the date, that should reduce errors I think. Let me know if the converting thing works! :)
I have set up an SQL Stored Procedure where the parameters are optional as in the front end any combination of these filters can have values or not have values. The only problem is when I come pass a null date in. SQL produces an error if I leave this null, however I have managed to do some reading around the problem and found that using the following should resolve the error:
SET #SessionStarted = ISNULL(#SessionStarted, GETDATE())
Which it does however, now it's not returning records it should be returning because it's only looking for records with the current date and time.
The parameter is defined as such:
#SessionStarted datetime = NULL,
And the where clause for this particular parameter is:
(#SessionStarted IS NULL OR VisitDate = #SessionStarted)
Is there anyway round this issue when a value for the can't be specified?
You could just use a different date in your isnull and detect it. Something like '1900-01-01' feels like a good "known invalid" day to use in place of your NULL.
When you assign current date time to a variable (like below)
SET #SessionStarted = ISNULL(#SessionStarted, GETDATE())
and compare it (like below) against a datetime dattyped column, which will never equal and will not return records.
(#SessionStarted IS NULL OR VisitDate = #SessionStarted)
You should make sure that what value your datetime column holds and what is your expected default value to be assigned to the variable in case NULL value is passed in.
I have a table and need to verify that a certain column contains only dates. I'm trying to count the number of records that are not follow a date format. If I check a field that I did not define as type "date" then the query works. However, when I check a field that I defined as a date it does not.
Query:
SELECT
count(case when ISDATE(Date_Field) = 0 then 1 end) as 'Date_Error'
FROM [table]
Column definition:
Date_Field(date, null)
Sample data: '2010-06-27'
Error Message:
Argument data type date is invalid for argument 1 of isdate function.
Any insight as to why this query is not working for fields I defined as dates?
Thanks!
If you defined the column with the Date type, it IS a Date. Period. This check is completely unnecessary.
What you may want to do is look for NULL values in the column:
SELECT SUM(case when Date_Field IS NULL THEN 1 ELSE 0 end) as 'Date_Error' FROM [table]
I also sense an additional misunderstanding about how Date fields, including DateTime and DateTime2, work in Sql Server. The values in these fields are not stored as a string in any format at all. They are stored in a binary/numeric format, and only shown as a string as a convenience in your query tool. And that's a good thing. If you want the date in a particular format, use the CONVERT() function in your query, or even better, let your client application handle the formatting.
ISDATE() only evaluates against a STRING-like parameter (varchar, nvarachar, char,...)
To be sure, ISDATE()'s parameter should come wrapped in a cast() function.
i.e.
Select isdate(cast(parameter as nvarchar))
should return either 1 or 0, even if it's a MULL value.
Hope this helps.
IsDate takes a character string or exression that yeilds a character string as it's argument
The problem is this method ISDATE() only admits arguments of type datetime and smalldatetime within the "time" types, so it won´t work if you are using date type.
Also if you use date as type for that field, you won´t have to check the information there because it won´t admit other type of field.
You shoul only check for null values in your column, that´s all.