A value coming back from a DB2 query passes a zero date as 0001-01-01 00:00:00.000000 and I am having a problem testing for this value in a Conditional Split condition. I tried a few different things but either they stay red or cause an error when run. I even tried testing for the date being less than the SQL start date. Neither of these work.
(DT_Date)DRIVER_TERMDATE < (DT_DATE)"1753-1-1"
DRIVER_TERMDATE != "0001-01-01 00:00:00.000000"
The minimum date for the DT_DATE data type is 1899-12-30, which is after that date and trying to convert it to this data type will cause an error. If you're importing the DRIVER_TERMDATE as text, trying adding a condition for this string, for example
DRIVER_TERMDATE == "0001-01-01 00:00:00.000000"
However it looks like you already tried checking to see if DRIVER_TERMDATE is not equal to this value, so see if trailing zeros are being removed during from the text during execution by right-clicking the output from the DB2 source, selecting Enable Data Viewer, and examining the data when the package runs. Once you've confirmed the exact string that's being sent in, you can add a condition for this.
Related
I have the below Query I am trying to genialize the dates to calculate current quarter in Sienese.
.Am I doing something wrong and what will be the correct way?
I have tried concat(year(current_date()),'Q',quarter(current_date())).It works in dbvisualizer but not in sisense.
The message is indicating a value you are trying to convert to date is null, it is not a matter of how you convert it, it is that null can never be converted to anything. You can review the data and change the conditions of what gets returned to avoid that, or use ISNULL() for error control possibly by specifying a date to use by default..
https://www.w3schools.com/sql/func_sqlserver_isnull.asp
I am trying to insert data from a staging table into the master table. The table has nearly 300 columns, and is a mix of data-typed Varchars, Integers, Decimals, Dates, etc.
Snowflake gives the unhelpful error message of "Numeric value '' is not recognized"
I have gone through and cut out various parts of the query to try and isolate where it is coming from. After several hours and cutting every column, it is still happening.
Does anyone know of a Snowflake diagnostic query (like Redshift has) which can tell me a specific column where the issue is occurring?
Unfortunately not at the point you're at. If you went back to the COPY INTO that loaded the data, you'd be able to use VALIDATE() function to get better information to the record and byte-offset level.
I would query your staging table for just the numeric fields and look for blanks, or you can wrap all of your fields destined for numeric fields with try_to_number() functions. A bit tedious, but might not be too bad if you don't have a lot of numbers.
https://docs.snowflake.com/en/sql-reference/functions/try_to_decimal.html
As a note, when you stage, you should try and use the NULL_IF options to get rid of bad characters and/or try to load them into stage using the actual datatypes in your stage table, so you can leverage the VALIDATE() function to make sure the data types are correct before loading into Snowflake.
Query your staging using try_to_number() and/or try_to_decimal() for number and decimal fields of the table and the use the minus to get the difference
Select $1,$2,...$300 from #stage
minus
Select $1,try_to_number($2)...$300 from#stage
If any number field has a string that cannot be converted then it will be null and then minus should return those rows which have a problem..Once you get the rows then try to analyze the columns in the result set for errors.
I have below values coming from a flat file which may contain single digit month & day field:
9/14/2020 07:20:18.630000
7/7/2020 16:24:57.700000
10/24/2019 03:40:52.380000
11/9/2020 20:21:32.420000
Now I need to load this to a column having TIMESTAMP(6) as the data type.
Can someone please help on this? I am using TD SQL Assistant version 16.
SQL Assistant is not a load utility, e.g. TPT fully supports dealing with intput like this.
Your other post shows that you already use a RegEx to add the missing zeroes and you apply the correct format. This is indicating bad data in your input. You might try to spot the error in the input file (check how many rows have been loaded and check the following lines).
Or you apply TRYCAST which doesn't fail, but returns a NULL for bad dates. But yikes, it doesn't support FORMAT, thus you must rearrange the MDY to YMD first:
trycast(RegExp_Replace(RegExp_Replace(x,'\b([0-9])\b', '0\1'), '(..).(..).(....)(.*)','\3-\1-\2\4') as timestamp(6))
I have a database field, Field260, that stores date representations as text. For instance a value for 3/29/2018 would be stored as a string "03/29/2018"
Dim Db = New DataClasses1DataContext
Return From cf In Db.FileDatas
Where cf.Field260 <= System.DateTime.Today
Returns error
System.Data.SqlClient.SqlException: 'Explicit conversion from data type text to datetime is not allowed.'
When I tried to Parse the date
Where DateTime.Parse(cf.Field260) <= System.DateTime.Today
I receive
System.NotSupportedException: 'Method 'System.DateTime Parse(System.String)' has no supported translation to SQL.'
I'm stumped.
I encountered a similar issue - have an app that I used to have a DatePicker for a field, and my database field defined as DateTime, but the users absolutely wanted to copy/paste whatever garbage they wanted into the field. Problem is, I'm allowing them to search for records by that "Date" field.
As a workaround, I do some basic validations on the garbage they enter to attempt to only allow dates (IsDate function captures most errors, but still allows them to enter years like 208 instead of 2018, so I also check that Year(DateValue(txtDate.Text))>2000, which works for me because it's an application-received date, so should never be older than this year). Then I store it as varchar in the database.
When searching for the records, you have two options - either grab all records, then perform a date conversion on the list returned to pare down the results, or hope and pray that your faulty LINQ WHERE clause will get everything you want.
Figured it out cuz I'm a genius.
Return From cf In Db.FileDatas
Where cf.Field260.ToString() <= System.DateTime.Today.ToString("MM/dd/yyyy")
Somehow comparing string to string works, who knew!
I'm using an ADO\ACE.OLEDB query to access data in an excel sheet. One of the columns in this sheet is a time stamp in the 00:00:00 format.
My select max query for this column works OK but it returns the wrong "max" time. For example, if I have the following time stamps:
08:15:00
09:45:00
10:00:00
The SELECT MAX query will return 09:45:00, and ignore any subsequent timestamps. If I go to the queried sheet, and convert all the cells in that column to "Time" using text-to-columns, the query will then return the correct value of 10:00:00.
The timestamp that is recorded in the excel sheet is updated very often by many different people throughout the day using their own ADO Update\Insert query so it's not feasible for me to open the sheet and convert the cells to the correct type manually. The update\insert queries have the cell types set to General.
So my question is this.
Is there a way to either
A) Specify the data type in the update\insert into ADO query so the cell has the correct data type? Or
B) Force the SELECT MAX query to interpret the column as a timevalue rather than text?
If not any suggestions on how to procede?
EDIT: I looked closer at the excel sheet i'm querying. I noticed that each cell has a singlequote preceding it. I think if I can get that removed when entries are recorded excel might properly format the cell as time.
Here is a basic example of wha is being used to write entries to the sheet.
Update [Sheet1$] Set StopTime='09:45:00' WHERE Date= '3/12/2014' AND StartTime='9:30:00'
Can this be rewritten so a single quote ins't included? Note there is no end quote.
The single quote at the beginning of a cell in Excel forces Excel to interpret the cell value as text. The single quote is probably being inserted because the update statement uses single quotes, which indicate strings. You may be able to use ## delimiters instead, which specifically indicate date/time values (I say "may" because I haven't tested this). For example:
UPDATE [Sheet1$]
SET StopTime = #09:45:00#
WHERE [...]
On the other side, you can force a column to be interpreted as a date/time by using the CDate function:
SELECT MAX(CDate(StopTime)) AS MaxTime
FROM [Sheet1$]
You just have to be careful in this case about values that cannot be converted to a valid time, which will cause the query to error out.
I didn't even realise this but the examples I gave were flawed. What is logged does NOT include a leading zero. Since I'm using a 24hr time I was able to correct this issue by writing a leading zero into the time that gets recorded.
Basically it's an, if LEN(time)=7 then time = 0 & time.
This appears to be working so far.