I need to work on an IBM DB2 database.
The LOCATION field is a CHARACTER(8) field of numbers.
To sort the table, the column is cast to an INTEGER:
SELECT LOCATION, PARTNO, INSTOCK
FROM INVENTORY
ORDER BY CAST(LOCATION AS INTEGER)
Currently, this fails with:
ERROR [22018] [IBM][DB2/AIX64] SQL0420N Invalid character found in a character string argument of the function "INTEGER".
Is there a quick way to determine which row is failing?
IBM's solution is to "Insure that the results set for the query item that the cast it being applied to does not contain non numeric SQL constants when casting to a numeric type."
That wasn't really helpful.
Thinking someone inserted a letter O or lower case L, I tried this:
SELECT DISTINCT LOCATION
FROM LOCATIONS
WHERE LOCATION LIKE '%l%' OR LOCATION LIKE '%O%'
ORDER BY LOCATION
Zero records returned.
That wasn't really helpful.
That's IBM error messages and documentation in a nutshell.
One place to start is the TRANSLATE() function.
SELECT LOCATION, PARTNO, INSTOCK
FROM INVENTORY
WHERE TRANSLATE(LOCATION, '', ' 0123456789') <> ''
You can add other characters, like -, ., etc. depending on what you find.
Related
I want to format amounts to salary format, e.g. 10000 becomes 10,000, so I use to_char(amount, '99,999,99')
SELECT SUM(DECODE(e.element_name,'Basic Salary',to_char(v.screen_entry_value,'99,999,99'),0)) Salary,
SUM(DECODE(e.element_name,'Transportation Allowance',to_char(v.screen_entry_value,'99,999,99'),0)) Transportation,
SUM(DECODE(e.element_name,'GOSI Processing',to_char(v.screen_entry_value,'99,999,99'),0)) GOSI,
SUM(DECODE(e.element_name,'Housing Allowance',to_char(v.screen_entry_value,'99,999,99'),0)) Housing
FROM values v,
values_types vt,
elements e
WHERE vt.value_type = 'Amount'
this gives error invalid number because not all values are numbers until value_type is equal to Amount but I guess decode check all values anyway although what I know is that the execution begins with from then where then select, what's going wrong here?
You said you added decode(...), but it looks like you might have actually added sum(decode(...)).
You are converting your values to strings with to_char(v.screen_entry_value,'99,999,99'), so your decode() generates a string - the default 0 will be converted to '0' - giving you a value like '1,234,56'. Then you are aggregating those, so sum() has to implicitly convert those strings to numbers - and it is throwing the error when it tries to do that:
select to_number('1,234,56') from dual
will also get "ORA-01722: invalid number", unless you supply a similar format mask so it knows how to interpret it. You could do that, e.g.:
SUM(to_number(DECODE(e.element_name,'Basic Salary',to_char(v.screen_entry_value,'99,999,99'),0),'99,999,99'))
... but it's maybe more obvious that something is strange, and even if you did, you would end up with a number, not a formatted string.
So instead of doing:
SUM(DECODE(e.element_name,'Basic Salary',to_char(v.screen_entry_value,'99,999,99'),0))
you should format the result after aggregating:
to_char(SUM(DECODE(e.element_name,'Basic Salary',v.screen_entry_value,0)),'99,999,99')
fiddle with dummy tables, data and joins.
I'm trying to load some data from stage to relational environment and something is happening I can't figure out.
I'm trying to run the following query:
SELECT
CAST(SPLIT_PART(some_field,'_',2) AS BIGINT) cmt_par
FROM
public.some_table;
The some_field is a column that has data with two numbers joined by an underscore like this:
some_field -> 38972691802309_48937927428392
And I'm trying to get the second part.
That said, here is the error I'm getting:
[Amazon](500310) Invalid operation: Invalid digit, Value '1', Pos 0,
Type: Long
Details:
-----------------------------------------------
error: Invalid digit, Value '1', Pos 0, Type: Long
code: 1207
context:
query: 1097254
location: :0
process: query0_99 [pid=0]
-----------------------------------------------;
Execution time: 2.61s
Statement 1 of 1 finished
1 statement failed.
It's literally saying some numbers are not valid digits. I've already tried to get the exactly data which is throwing the error and it appears to be a normal field like I was expecting. It happens even if I throw out NULL fields.
I thought it would be an encoding error, but I've not found any references to solve that.
Anyone has any idea?
Thanks everybody.
I just ran into this problem and did some digging. Seems like the error Value '1' is the misleading part, and the problem is actually that these fields are just not valid as numeric.
In my case they were empty strings. I found the solution to my problem in this blogpost, which is essentially to find any fields that aren't numeric, and fill them with null before casting.
select cast(colname as integer) from
(select
case when colname ~ '^[0-9]+$' then colname
else null
end as colname
from tablename);
Bottom line: this Redshift error is completely confusing and really needs to be fixed.
When you are using a Glue job to upsert data from any data source to Redshift:
Glue will rearrange the data then copy which can cause this issue. This happened to me even after using apply-mapping.
In my case, the datatype was not an issue at all. In the source they were typecast to exactly match the fields in Redshift.
Glue was rearranging the columns by the alphabetical order of column names then copying the data into Redshift table (which will
obviously throw an error because my first column is an ID Key, not
like the other string column).
To fix the issue, I used a SQL query within Glue to run a select command with the correct order of the columns in the table..
It's weird why Glue did that even after using apply-mapping, but the work-around I used helped.
For example: source table has fields ID|EMAIL|NAME with values 1|abcd#gmail.com|abcd and target table has fields ID|EMAIL|NAME But when Glue is upserting the data, it is rearranging the data by their column names before writing. Glue is trying to write abcd#gmail.com|1|abcd in ID|EMAIL|NAME. This is throwing an error because ID is expecting a int value, EMAIL is expecting a string. I did a SQL query transform using the query "SELECT ID, EMAIL, NAME FROM data" to rearrange the columns before writing the data.
Hmmm. I would start by investigating the problem. Are there any non-digit characters?
SELECT some_field
FROM public.some_table
WHERE SPLIT_PART(some_field, '_', 2) ~ '[^0-9]';
Is the value too long for a bigint?
SELECT some_field
FROM public.some_table
WHERE LEN(SPLIT_PART(some_field, '_', 2)) > 27
If you need more than 27 digits of precision, consider a decimal rather than bigint.
If you get error message like “Invalid digit, Value ‘O’, Pos 0, Type: Integer” try executing your copy command by eliminating the header row. Use IGNOREHEADER parameter in your copy command to ignore the first line of the data file.
So the COPY command will look like below:
COPY orders FROM 's3://sourcedatainorig/order.txt' credentials 'aws_access_key_id=<your access key id>;aws_secret_access_key=<your secret key>' delimiter '\t' IGNOREHEADER 1;
For my Redshift SQL, I had to wrap my columns with Cast(col As Datatype) to make this error go away.
For example, setting my columns datatype to Char with a specific length worked:
Cast(COLUMN1 As Char(xx)) = Cast(COLUMN2 As Char(xxx))
I have a table of people. Each person can have several regnums (mostly integers but some like M/2344 and W345). To make things a bit more complicated, there are NULLs, empties, and strings like 'NA'. Due to their unpredictable composition, the regnums are stored in a text array field (e.g. {12345,M/2344} and {3459,NA}).
Because most people have regnums that can be treated as integers, I would like to be able to do things with this field like find people with a regnum between, say, 491555 and 491685.
I've tried:
SELECT id,forename,surname,regnum FROM (SELECT *, unnest(regnum) reg FROM people) as TBL WHERE reg BETWEEN '491555' AND '491685';
but results include out-of-range regnums, e.g. 49162. I assume this is because the unnested regnum field is still a text field(?)
I've also tried casting the regnum as an integer field - unnest(regnum::integer[]) - but I get errors:
Error in query: ERROR: invalid input syntax for integer: "NA"
I think I'm on the right track, but I don't get how to ignore non-int-like regnums. Any ideas?
You can test if a text value consists only of digits by checking it with regular expression, like this:
SELECT '1234' ~ '^[0-9]+$' -- true
SELECT 'NA' ~ '^[0-9]+$' -- false
So, in your case you need to cast value to integer only if it is numerical:
WHERE (CASE WHEN reg ~ '^[0-9]+$' THEN reg::integer ELSE null END) BETWEEN 491555 AND 491685
SELECT logicalTime, traceValue, unitType, entName
FROM vwSimProjAgentTrace
WHERE valueType = 10
AND agentName ='AtisMesafesi'
AND ( entName = 'Hawk-1')
AND simName IN ('TipSenaryo1_0')
AND logicalTime IN (
SELECT logicalTime
FROM vwSimProjAgentTrace
WHERE valueType = 10 AND agentName ='AtisIrtifasi'
AND ( entName = 'Hawk-1')
AND simName IN ('TipSenaryo1_0')
AND CONVERT(FLOAT , traceValue) > 123
) ORDER BY simName, logicalTime
This is my sql command and table is a view table...
each time i put "convert(float...) part " i get
Msg 8114, Level 16, State 5, Line 1
Error converting data type nvarchar to float.
this error...
One (or more) of the rows has data in the traceValue field that cannot be converted to a float.
Make sure you've used the right combination of dots and commas to signal floating point values, as well as making sure you don't have pure invalid data (text for instance) in that field.
You can try this SQL to find the invalid rows, but there might be cases it won't handle:
SELECT * FROM vwSimProjAgentTrace WHERE NOT ISNUMERIC(traceValue)
You can find the documentation of ISNUMERIC here.
If you look in BoL (books online) at the convert command, you see that a nvarchar conversion to float is an implicit conversion. This means that only "float"-able values can be converted into a float. So, every numeric value (that is within the float range) can be converted. A non-numeric value can not be converted, which is quite logical.
Probably you have some non numeric values in your column. You might see them when you run your query without the convert. Look for something like comma vs dot. In a test scenario a comma instead of a dot gave me some problems.
For an example of isnumeric, look at this sqlfiddle
We have legacy table where one of the columns part of composite key was manually filled with values:
code
------
'001'
'002'
'099'
etc.
Now, we have feature request in which we must know MAX(code) in order to give user next possible value, in example case form above next value is '100'.
We tried to experiment with this but we still can't find any reasonable explanation how DB2 engine calculates that
MAX('001', '099', '576') is '576'
MAX('099', '99', 'www') is '99' and so on.
Any help or suggestion would be much appreciated!
You already have the answer to getting the maximum numeric value, but to answer the other part with regard to 'www','099','99'.
The AS/400 uses EBCDIC to store values, this is different to ASCII in several ways, the most important for your purposes is that Alpha characters come before numbers, which is the opposite of Ascii.
So on your Max() your 3 strings will be sorted and the highest EBCDIC value used so
'www'
'099'
'99 '
As you can see your '99' string is really '99 ' so it is higher that the one with the leading zero.
Cast it to int before applying max()
For the numeric maximum -- filter out the non-numeric values and cast to a numeric for aggregation:
SELECT MAX(INT(FLD1))
WHERE FLD1 <> ' '
AND TRANSLATE(FLD1, '0123456789', '0123456789') = FLD1
SQL Reference: TRANSLATE
And the reasonable explanation:
SQL Reference: MAX
This max working well in your type definition, when you want do max on integer values then convert values to integer before calling MAX, but i see you mixing max with string 'www' how you imagine this works?
Filter integer only values, cast it to int and call max. This is not good designed solution but looking at your problem i think is enough.
Sharing the solution for postgresql
which worked for me.
Suppose here temporary_id is of type character in database. Then above query will directly convert char type to int type when it gives response.
SELECT MAX(CAST (temporary_id AS Integer)) FROM temporary
WHERE temporary_id IS NOT NULL
As per my requirement I've applied MAX() aggregate function. One can remove that also and it will work the same way.