Selecting and comparing columns which contain constants - sql

Follow up to this question.
Say that on postgres, I have a table TABLE1 containing the columns id (integer), name (string):
create table table1 (id int primary key,name varchar(100));
insert into table1(id,name) values(5,'a');
insert into table1(id,name) values(6,'b');
insert into table1(id,name) values(7,'c');
insert into table1(id,name) values(55,'a');
And attempt to run the following queries:
with base (x) as (select 5 as x from table1)
select table1.name from base, table1 where table1.id = base.x;
with base (x) as (select 'a' as x from table1)
select table1.name from base, table1 where table1.name = base.x;
On sqlfiddle, the former yields a result, whilst the latter fails with the message:
ERROR: failed to find conversion function from unknown to text
On postgres 13.3 which I have installed locally, however, neither errs. (Nor similar queries on oracle and sqlite.)
My first question is, does this error stem from an issue within sqlfiddle, or has it persisted within earlier versions of postgres?
And second, does this count as a bug? Generally, are constant columns (or values) in SQL assumed to be typeless, and any operations on them are undefined unless there happens to be an implicit / explicit cast in place?
Per my understanding, using constant columns for joining is generally inadvisable as it thwarts indexing, but it seems a little odd in any programming language to have difficulties telling one constant format apart from another.

The cause is that a string literal in PostgreSQL is of type unknown, not text, because all data types have a string representation. Normally, the actual type is determined by context.
A number literal, however, has data type integer, bigint or numeric, based on its size and the presence of a fractional part:
SELECT pg_typeof('a');
pg_typeof
═══════════
unknown
(1 row)
SELECT pg_typeof(5);
pg_typeof
═══════════
integer
(1 row)
Now the subquery select 'a' as x from table1 has no context to determine a better type than unknown for the result column, which makes the comparison in the join condition fail.
Since this is strange and undesirable behavior, PostgreSQL v10 has demoted unknown from being a regular data type (you could create columns of that type!) to a “pseudo-type”. It also coerced unknown to text in SELECT lists, which is why you cannot reproduce that on v10 or later.

Related

How to do explicit conversion of FLOAT to a VARCHAR field in SQL

I'm getting a query from a column float (with no precision) and inserting it in another table and comumn float (with no precision as well) but I'm getting this error:
Error (265) Insufficient result space for explicit conversion of FLOAT value '3.8833137793643' to a VARCHAR field.
The query:
INSERT INTO TableA
SELECT DISTINCT max(price_1) AS PriceValue
FROM TableB
This does may or may not answer your question. But the query should be written as:
INSERT INTO TableA (<column name>)
SELECT MAX(price_1) AS PriceValue
FROM TableB;
Notes:
An aggregation query with no GROUP BY returns exactly one row. SELECT DISTINCT is not necessary.
You should include the columns being inserted.
Your problem is clearly that the column is not wide enough. If you have defined the table as:
create table tableA (
col varchar
);
Then you have not specified a length and are depending on the default in the database. Do you even know what the default is? In this case, it is 1. And that is not long enough for your value. You just need to provide a long-enough length:
create table tableA (
col varchar(255)
);
All that said. I strongly discourage you from storing number values as strings. That is likely to create problems now and in the future. Use the appropriate types for your data.

SQL0802 - invalid numeric data

I'm on a db2 database over as400 system.
I have a select query that is throwing the error in the title: SQL0802 code 6 which is "invalid numeric data" (translated).
I have tried separating the query in different parts and testing each part one by one to see if it works, I am 99% convinced that the problem comes because of a "CAST" clause I am using in a subquery(to cast CHAR to INT), I just don't understand why the subquery works by itself but it doesn't work as a part of the main query.
So if I run the subquery with the "CAST" clause it works fine, but when I run the main query that uses the subquery it doesn't work and the error arises.
Main query can be divided in 2 queries, see the code below.
query1 looks something like this:
select SUM(Price) from TABLE1
where X = 1
group by Country
having SUM(Price) = (query2);
query2 looks something like this:
SELECT SUM(UnitPrice * AmountStocked)
FROM TABLE2
WHERE J = X and ItemNumber in (
SELECT CAST(ItmNumbr AS INT) from TABLE3
where Id in (select Id from TABLE4 where Z=Y)
)
Notes:
*query2 will return a single number.
*Running query2 by itself works fine.
*Running query1 without the "having" clause works fine too.
*If I substitute the "SELECT CAST..." subquery in query2 with something like "(2002, 9912, 1234)" and then run the main query it works fine, so this pretty much confirms that the problem is the "CAST" clause.
*I have to CAST ItmNumbr to INT because ItemNumber is of Numeric type and
ItmNumbr is of Char type.
You said:
*I have to CAST ItmNumbr to INT because ItemNumber is of Numeric type and ItmNumbr is of Char type.
But this is not true. You could cast the other way around:
SELECT SUM(UnitPrice * AmountStocked)
FROM TABLE2
WHERE J = X and CHAR(ItemNumber) in (
SELECT TRIM(ItmNumbr) from TABLE3
where Id in (select Id from TABLE4 where Z=Y)
)
The advantage here is that non-numeric characters in ItmNumber will not blow you up, and CHAR(ItemNumber) should also not fail.
One thing to know about DB2 for i is that there are two ways to create database tables, and the two differ slightly in the characteristics of the resulting table. If the table is created using DDL (CREATE TABLE ...), then that table cannot contain bad data. The data types are verified on write, no matter how you write the data, it is validated before being written to the table. If the table is created by DDS (CRTPF ...), the table can indeed contain bad data because the data is not validated until it is read and loaded into a variable. Old style programs that write data to DDS tables by writing a record from a program described data structure are able to put whatever they want into a DDS defined table, including numeric data in character fields or worse, character data in numeric fields. This usually is only found in very old databases that have been migrated from the System/36 (circa 1980's) which used flat files rather than database files (it had no notion of a database). I only posit this because it is possible. Check the data in your file using hex() to see if there is anything funky in the ItmNumbr or ItemNumber fields.
I am not sure but I am thinking the issue has to do with your join of "WHERE J = X" since we don't know what "J" is and it may not join to "X" (not the correct data type).
Based on your analysis:
"*If I substitute the "SELECT CAST..." subquery in query2 with something like "(2002, 9912, 1234)" and then run the main query it works fine, so this pretty much confirms that the problem is the "CAST" clause."
Check the content of TABLE3.ItmNumbr. If it is defined as NUMERIC (unpacked decimal) it may contain non-numeric values (typically spaces). That may be causing the error you are observing.

SQL Server subquery behaviour

I have a case where I want to check to see if an integer value is found in a column of a table that is varchar, but is a mix of values that can be integers some are just strings. My first thought was to use a subquery to just select the rows with numeric-esque values. The setup looks like:
CREATE TABLE #tmp (
EmployeeID varchar(50) NOT NULL
)
INSERT INTO #tmp VALUES ('aa1234')
INSERT INTO #tmp VALUES ('1234')
INSERT INTO #tmp VALUES ('5678')
DECLARE #eid int
SET #eid = 5678
SELECT *
FROM (
SELECT EmployeeID
FROM #tmp
WHERE IsNumeric(EmployeeID) = 1) AS UED
WHERE UED.EmployeeID = #eid
DROP TABLE #tmp
However, this fails, with: "Conversion failed when converting the varchar value 'aa1234' to data type int.".
I don't understand why it is still trying to compare #eid to 'aa1234' when I've selected only the rows '1234' and '5678' in the subquery.
(I realize I can just cast #eid to varchar but I'm curious about SQL Server's behaviour in this case)
You can't easily control the order things will happen when SQL Server looks at the query you wrote and then determines the optimal execution plan. It won't always produce a plan that follows the same logic you typed, in the same order.
In this case, in order to find the rows you're looking for, SQL Server has to perform two filters:
identify only the rows that match your variable
identify only the rows that are numeric
It can do this in either order, so this is also valid:
identify only the rows that are numeric
identify only the rows that match your variable
If you look at the properties of this execution plan, you see that the predicate for the match to your variable is listed first (which still doesn't guarantee order of operation), but in any case, due to data type precedence, it has to try to convert the column data to the type of the variable:
Subqueries, CTEs, or writing the query a different way - especially in simple cases like this - are unlikely to change the order SQL Server uses to perform those operations.
You can force evaluation order in most scenarios by using a CASE expression (you also don't need the subquery):
SELECT EmployeeID
FROM #tmp
WHERE EmployeeID = CASE IsNumeric(EmployeeID) WHEN 1 THEN #eid END;
In modern versions of SQL Server (you forgot to tell us which version you use), you can also use TRY_CONVERT() instead:
SELECT EmployeeID
FROM #tmp
WHERE TRY_CONVERT(int, EmployeeID) = #eid;
This is essentially shorthand for the CASE expression, but with the added bonus that it allows you to specify an explicit type, which is one of the downsides of ISNUMERIC(). All ISNUMERIC() tells you is if the value can be converted to any numeric type. The string '1e2' passes the ISNUMERIC() check, because it can be converted to float, but try converting that to an int...
For completeness, the best solution - if there is an index on EmployeeID - is to just use a variable that matches the column data type, as you suggested.
But even better would be to use a data type that prevents junk data like 'aa1234' from getting into the table in the first place.

UNION causes "Conversion failed when converting the varchar value to int"

I tried to search for previous articles related to this, but I can't find one specific to my situation. And because I'm brand new to StackOverflow, I can't post pictures so I'll try to describe it.
I have two datasets. One is 34 rows, 1 column of all NULLs. The other 13 rows, 1 column of varchars.
When I try to UNION ALL these two together, i get the following error:
Conversion failed when converting the varchar value to data type int.
I don't understand why I'm getting this error. I've UNIONed many NULL columns and varchar columns before, among many other types and I don't get this conversion error.
Can anyone offer suggestions why this error occurs?
The error occurs because you have corresponding columns in the two of the subqueries where the type of one is an integer and the type of the other is a character. Then, the character value has -- in at least one row -- a value that cannot be automatically converted to an integer.
This is easy to replicate:
select t.*
from (select 'A' as col union all
select 1
) t;
Here is the corresponding SQL Fiddle.
SQL Server uses pretty sophisticated type precedence rules for determining the destination type in a union. In practice, though, it is best to avoid using implicit type conversions. Instead, explicitly cast the columns to the type you intend.
EDIT:
The situation with NULL values is complicated. By itself, the NULL value has no type. So, the following works fine:
select NULL as col
union all
select 'A';
If you type the NULL, then the query will fail:
select cast(NULL as int) as col
union all
select 'A';
Also, if you put SQL Server in a position where it has to assign a type, then SQL Server will make the NULL an integer. Every column in a table or result set needs a type, so this will also fail:
select (select NULL) as col
union all
select 'A';
Perhaps your queries are doing something like this.
I have also encountered this error when I accidentally had the fields out of sequence in the 2 SELECT queries that I was unioning. Adjusting the fields' sequence fixed the problem.

SQL error "ORA-01722: invalid number"

A very easy one for someone,
The following insert is giving me the
ORA-01722: invalid number
why?
INSERT INTO CUSTOMER VALUES (1,'MALADY','Claire','27 Smith St Caulfield','0419 853 694');
INSERT INTO CUSTOMER VALUES (2,'GIBSON','Jake','27 Smith St Caulfield','0415 713 598');
INSERT INTO CUSTOMER VALUES (3,'LUU','Barry','5 Jones St Malvern','0413 591 341');
INSERT INTO CUSTOMER VALUES (4,'JONES','Michael','7 Smith St Caulfield','0419 853 694');
INSERT INTO CUSTOMER VALUES (5,'MALADY','Betty','27 Smith St Knox','0418 418 347');
An ORA-01722 error occurs when an attempt is made to convert a character string into a number, and the string cannot be converted into a number.
Without seeing your table definition, it looks like you're trying to convert the numeric sequence at the end of your values list to a number, and the spaces that delimit it are throwing this error. But based on the information you've given us, it could be happening on any field (other than the first one).
Suppose tel_number is defined as NUMBER - then the blank spaces in this provided value cannot be converted into a number:
create table telephone_number (tel_number number);
insert into telephone_number values ('0419 853 694');
The above gives you a
ORA-01722: invalid number
Here's one way to solve it. Remove non-numeric characters then cast it as a number.
cast(regexp_replace('0419 853 694', '[^0-9]+', '') as number)
Well it also can be :
SELECT t.col1, t.col2, ('test' + t.col3) as test_col3
FROM table t;
where for concatenation in oracle is used the operator || not +.
In this case you get : ORA-01722: invalid number ...
This is because:
You executed an SQL statement that tried to convert a string to a
number, but it was unsuccessful.
As explained in:
Oracle/PLSQL: ORA-01722 Error.
To resolve this error:
Only numeric fields or character fields that contain numeric values
can be used in arithmetic operations. Make sure that all expressions
evaluate to numbers.
As this error comes when you are trying to insert non-numeric value into a numeric column in db it seems that your last field might be numeric and you are trying to send it as a string in database. check your last value.
Oracle does automatic String2number conversion, for String column values! However, for the textual comparisons in SQL, the input must be delimited as a String explicitly: The opposite conversion number2String is not performed automatically, not on the SQL-query level.
I had this query:
select max(acc_num) from ACCOUNTS where acc_num between 1001000 and 1001999;
That one presented a problem: Error: ORA-01722: invalid number
I have just surrounded the "numerical" values, to make them 'Strings', just making them explicitly delimited:
select max(acc_num) from ACCOUNTS where acc_num between '1001000' and '1001999';
...and voilà: It returns the expected result.
edit:
And indeed: the col acc_num in my table is defined as String. Although not numerical, the invalid number was reported. And the explicit delimiting of the string-numbers resolved the problem.
On the other hand, Oracle can treat Strings as numbers. So the numerical operations/functions can be applied on the Strings, and these queries work:
select max(string_column) from TABLE;
select string_column from TABLE where string_column between '2' and 'z';
select string_column from TABLE where string_column > '1';
select string_column from TABLE where string_column <= 'b';
In my case the conversion error was in functional based index, that I had created for the table.
The data being inserted was OK. It took me a while to figure out that the actual error came from the buggy index.
Would be nice, if Oracle could have gave more precise error message in this case.
If you do an insert into...select * from...statement, it's easy to get the 'Invalid Number' error as well.
Let's say you have a table called FUND_ACCOUNT that has two columns:
AID_YEAR char(4)
OFFICE_ID char(5)
And let's say that you want to modify the OFFICE_ID to be numeric, but that there are existing rows in the table, and even worse, some of those rows have an OFFICE_ID value of ' ' (blank). In Oracle, you can't modify the datatype of a column if the table has data, and it requires a little trickery to convert a ' ' to a 0. So here's how to do it:
Create a duplicate table: CREATE TABLE FUND_ACCOUNT2 AS SELECT * FROM FUND_ACCOUNT;
Delete all the rows from the original table: DELETE FROM FUND_ACCOUNT;
Once there's no data in the original table, alter the data type of its OFFICE_ID column: ALTER TABLE FUND_ACCOUNT MODIFY (OFFICE_ID number);
But then here's the tricky part. Because some rows contain blank OFFICE_ID values, if you do a simple INSERT INTO FUND_ACCOUNT SELECT * FROM FUND_ACCOUNT2, you'll get the "ORA-01722 Invalid Number" error. In order to convert the ' ' (blank) OFFICE_IDs into 0's, your insert statement will have to look like this:
INSERT INTO FUND_ACCOUNT (AID_YEAR, OFFICE_ID) SELECT AID_YEAR, decode(OFFICE_ID,' ',0,OFFICE_ID) FROM FUND_ACCOUNT2;
I have found that the order of your SQL statement parameters is also important and the order they are instantiated in your code, this worked in my case when using "Oracle Data Provider for .NET, Managed Driver".
var sql = "INSERT INTO table (param1, param2) VALUES (:param1, :param2)";
...
cmd.Parameters.Add(new OracleParameter("param2", Convert.ToInt32("100")));
cmd.Parameters.Add(new OracleParameter("param1", "alpha")); // This should be instantiated above param1.
Param1 was alpha and param2 was numeric, hence the "ORA-01722: invalid number" error message. Although the names clearly shows which parameter it is in the instantiation, the order is important. Make sure you instantiate in the order the SQL is defined.
For me this error was a bit complicated issue.
I was passing a collection of numbers (type t_numbers is table of number index by pls_integer;) to a stored procedure. In the stored proc there was a bug where numbers in this collection were compared to a varchar column
select ... where ... (exists (select null from table (i_coll) ic where ic.column_value = varchar_column))
Oracle should see that ic.column_value is integer so shouldn't be compared directly to varchar but it didn't (or there is trust for conversion routines).
Further complication is that the stored proc has debugging output, but this error came up before sp was executed (no debug output at all).
Furthermore, collections [<empty>] and [0] didn't give the error, but for example [1] errored out.
The ORA-01722 error is pretty straightforward. According to Tom Kyte:
We've attempted to either explicity or implicity convert a character string to a number and it is failing.
However, where the problem is is often not apparent at first. This page helped me to troubleshoot, find, and fix my problem. Hint: look for places where you are explicitly or implicitly converting a string to a number. (I had NVL(number_field, 'string') in my code.)
This happened to me too, but the problem was actually different: file encoding.
The file was correct, but the file encoding was wrong. It was generated by the export utility of SQL Server and I saved it as Unicode.
The file itself looked good in the text editor, but when I opened the *.bad file that the SQL*loader generated with the rejected lines, I saw it had bad characters between every original character. Then I though about the encoding.
I opened the original file with Notepad++ and converted it to ANSI, and everything loaded properly.
In my case it was an end of line problem, I fixed it with dos2unix command.
In my case I was trying to Execute below query, which caused the above error ( Note : cus_id is a NUMBER type column)
select *
from customer a
where a.cus_id IN ('115,116')
As a solution to the caused error, below code fragment(regex) can be used which is added in side IN clause (This is not memory consuming as well)
select *
from customer a
where a.cus_id IN (select regexp_substr (
com_value,
'[^,]+',
1,
level
) value
from (SELECT '115,116' com_value
FROM dual)rws
connect by level <=
length ( com_value ) - length ( replace ( com_value, ',' ) ) + 1)
try this as well, when you have a invalid number error
In this
a.emplid is number and b.emplid is an varchar2 so if you got to convert one of the sides
where to_char(a.emplid)=b.emplid
You can always use TO_NUMBER() function in order to remove this error.This can be included as INSERT INTO employees phone_number values(TO_NUMBER('0419 853 694');