Dynamically cast type of array elements to match some expression type in PostgreSQL query - sql

I want to use array_position function in PostgreSQL (which takes array of some type and expression or value of the same type) for constructing query that returns rows in some arbitrary order (additional context: I want to enhance Ruby on Rails in_order_of feature which is currently implemented via unreadable CASE statement):
SELECT id, title, type
FROM posts
ORDER BY
array_position(ARRAY['SuperSpecial','Special','Ordinary']::varchar[], type),
published_at DESC;
The problem here is that requirement to do explicit type casting from type inferred by PostgreSQL from array literal (ARRAY['doh'] is text[]) to type of expression (type is varchar here). While varchar and text are coercible to each other, PostgreSQL requires explicit type cast, otherwise if omit it (like in array_position(ARRAY['doh'], type)) PostgreSQL will throw error (see this answer for details):
ERROR: function array_position(text[], character varying) does not exist
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
While it is not a problem to specify explicit type cast in some static queries, it is problem in autogenerated queries when type of expression is unknown beforehand: array_position(ARRAY[1,2,3], id * 2) (what type has id * 2?)
I thought that pg_typeof() could help me, but it seems that it can't be used neither in :: operator nor in CAST operator (I've seen information that both forms aren't function forms, but syntax constructs, see this question for details):
SELECT id, title, type
FROM posts
ORDER BY array_position(CAST(ARRAY['SpecialPost','Post','Whatever'] AS pg_typeof(type)), type), id;
ERROR: type "pg_typeof" does not exist
LINE 1: ...on(CAST(ARRAY['SpecialPost','Post','Whatever'] AS pg_typeof(...
Question:
How to do dynamic typecast to expression type (say, to type of "posts"."id" * 2) in the same SQL query?
I would prefer to avoid extra roundtrip to database server (like executing SELECT pg_typeof("id" * 2) FROM "posts" LIMIT 1 and then using its result in generating of a new query) or writing some custom functions. Is it possible?

Better query
I want to enhance Ruby on Rails in_order_of feature which is currently implemented via unreadable CASE statement:
For starters, neither the awkward CASE construct nor array_position() are ideal solutions.
SELECT id, title, type
FROM posts
ORDER BY
array_position(ARRAY['SuperSpecial','Special','Ordinary']::varchar[], type),
published_at DESC;
There is a superior solution in Postgres:
SELECT id, title, type
FROM posts
LEFT JOIN unnest(ARRAY['SuperSpecial','Special','Ordinary']::varchar[]) WITH ORDINALITY o(type, ord) USING (type)
ORDER BY o.ord, published_at DESC;
This avoids calling the function array_position() for every row and is cheaper.
Equivalent short syntax with array literal and implicit column name:
SELECT id, title, type
FROM posts
LEFT JOIN unnest('{SuperSpecial,Special,Ordinary}'::varchar[]) WITH ORDINALITY type USING (type)
ORDER BY ordinality, published_at DESC;
db<>fiddle here
Added "benefit": it works with type-mismatch in Postgres 13 - as long as array type and column type are compatible.
The only possible caveat I can think of: If the passed array has duplicate elements, joined rows are duplicated accordingly. That wouldn't happen with array_position(). But duplicates would be nonsense for the expressed purpose in any case. Make sure to pass unique array elements.
See:
ORDER BY the IN value list
PostgreSQL unnest() with element number
Improved functionality in Postgres 14
The error you report is going away with Postgres 14:
ERROR: function array_position(text[], character varying) does not exist
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
Quoting the release notes:
Allow some array functions to operate on a mix of compatible data
types (Tom Lane)
The functions array_append(), array_prepend(), array_cat(),
array_position(), array_positions(), array_remove(),
array_replace(), and width_bucket() now take anycompatiblearray
instead of anyarray arguments. This makes them less fussy about
exact matches of argument types.
And the manual on anycompatiblearray:
Indicates that a function accepts any array data type, with automatic promotion of multiple arguments to a common data type
So, while this raises the above error msg in Postgres 13:
SELECT array_position(ARRAY['a','b','c']::text[], 'd'::varchar);
.. the same just works in Postgres 14.
(Your query and error msg show flipped positions for text and varchar, but all the same.)
To be clear, calls with compatible types now just work, incompatible types still raise an exception:
SELECT array_position('{1,2,3}'::text[], 3);
(The numeric literal 3 defaults to type integer, which is incompatible with text.)
Answer to actual question
.. which may be irrelevant by now. But as proof of concept:
CREATE OR REPLACE FUNCTION posts_order_by(_order anyarray)
RETURNS SETOF posts
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE format (
$$
SELECT p.*
FROM posts p
LEFT JOIN unnest($1) WITH ORDINALITY o(type, ord) ON (o.type::%s = p.type)
ORDER BY o.ord, published_at DESC
$$
, (SELECT atttypid::regtype
FROM pg_attribute
WHERE attrelid = 'posts'::regclass
AND attname = 'type')
)
USING _order;
END
$func$;
db<>fiddle here
Doesn't make a whole lot of sense, as the type of posts.id should be well-known at the time of writing the function, but there may be special cases ...
Now both of these calls work:
SELECT * FROM posts_order_by('{SuperSpecial,Special,Ordinary}'::varchar[]);
SELECT * FROM posts_order_by('{1,2,3}'::int[]);
Though the second typically doesn't make sense.
Related, with links to more:
Executing queries dynamically in PL/pgSQL

Related

Selecting and comparing columns which contain constants

Follow up to this question.
Say that on postgres, I have a table TABLE1 containing the columns id (integer), name (string):
create table table1 (id int primary key,name varchar(100));
insert into table1(id,name) values(5,'a');
insert into table1(id,name) values(6,'b');
insert into table1(id,name) values(7,'c');
insert into table1(id,name) values(55,'a');
And attempt to run the following queries:
with base (x) as (select 5 as x from table1)
select table1.name from base, table1 where table1.id = base.x;
with base (x) as (select 'a' as x from table1)
select table1.name from base, table1 where table1.name = base.x;
On sqlfiddle, the former yields a result, whilst the latter fails with the message:
ERROR: failed to find conversion function from unknown to text
On postgres 13.3 which I have installed locally, however, neither errs. (Nor similar queries on oracle and sqlite.)
My first question is, does this error stem from an issue within sqlfiddle, or has it persisted within earlier versions of postgres?
And second, does this count as a bug? Generally, are constant columns (or values) in SQL assumed to be typeless, and any operations on them are undefined unless there happens to be an implicit / explicit cast in place?
Per my understanding, using constant columns for joining is generally inadvisable as it thwarts indexing, but it seems a little odd in any programming language to have difficulties telling one constant format apart from another.
The cause is that a string literal in PostgreSQL is of type unknown, not text, because all data types have a string representation. Normally, the actual type is determined by context.
A number literal, however, has data type integer, bigint or numeric, based on its size and the presence of a fractional part:
SELECT pg_typeof('a');
pg_typeof
═══════════
unknown
(1 row)
SELECT pg_typeof(5);
pg_typeof
═══════════
integer
(1 row)
Now the subquery select 'a' as x from table1 has no context to determine a better type than unknown for the result column, which makes the comparison in the join condition fail.
Since this is strange and undesirable behavior, PostgreSQL v10 has demoted unknown from being a regular data type (you could create columns of that type!) to a “pseudo-type”. It also coerced unknown to text in SELECT lists, which is why you cannot reproduce that on v10 or later.

How to check which function uses a type?

I have a type which I'd like to change but I don't know who else is using it.
How can I check for all functions that return this type?
You can find all dependencies in the system catalog pg_depend.
This returns all functions depending on the type. I.e. not only those with the type in the RETURNS clause, but also those with the type as function parameter:
SELECT objid::regproc AS function_name
, pg_get_functiondef(objid) AS function_definition
, pg_get_function_identity_arguments(objid) AS function_args
, pg_get_function_result(objid) AS function_returns
FROM pg_depend
WHERE refclassid = 'pg_type'::regclass
AND refobjid = 'my_type'::regtype -- insert your type name here
AND classid = 'pg_proc'::regclass; -- only find functions
This also works for table functions:
...
RETURNS TABLE (foo my_type, bar int)
Using system catalog information functions.
There may be other dependencies (not to functions). Remove the last WHERE condition from my query to test (and adapt the SELECT list, obviously).
And there is still the possibility of the type being used explicitly (in a cast for instance) in queries in the function body or in dynamic SQL. You can only identify such use cases by parsing the text of the function body. There are no explicit dependencies registered in the system.
Related:
How to get function parameter lists (so I can drop a function)
DROP FUNCTION without knowing the number/type of parameters?
As mentioned by Erwin Brandstetter, this only works for functions directly returning the data type.
SELECT * FROM information_schema.routines r
WHERE r.type_udt_name = 'YOUR_DATA_TYPE' ORDER BY r.routine_name;

Fetching records which have either of two values in an array column in postgres

I have an array_agg column in postgresql which has values like these:
"{Metabolic/Endocrinology}"
"{Cardiovascular}"
"{Oncology}"
"{Autoimmune/Inflammation}"
Basically a string variable being array_agg by an id.
Now I want to fetch all records from this table where either of Oncology or Autoimmune/Inflammation is present.
I am doing something like this but I am not sure why it is throwing an error.
select * from Table where id = ANY('{Oncology,Autoimmune/Inflammation}')
It throws the following error.
ERROR: operator does not exist: text[] = text
SQL state: 42883
Hint: No operator matches the given name and argument type(s). You may need to add explicit type casts.
Character: 67
Please note I have also used ::TEXT [] and it still gives an error.
You want to use the array-overlaps operator &&.
See array operators.
e.g.
select * from (
VALUES
(ARRAY['Oncology','Pediatrics']),
(ARRAY['Autoimmune/Inflammation','Oncology']),
(ARRAY['Autoimmune/Inflammation']),
(ARRAY['Pediatrics']),
(ARRAY[]::text[])
) "Table"(id)
where id && ARRAY['Oncology','Autoimmune/Inflammation'];
By the way, I suggest using the SQL-standard ARRAY[...] constructor where possible.
Also, it's almost certainly a terrible idea to have an id column (presumably a primary key, if not, the name is confusing) defined as an array type.

Syntax error passing SQL result to PostgreSQL function accepting array

I tried to pass the result of a SQL query to a function, but I got a syntax error.
contacts=> SELECT count(*) FROM update_name(contact_ids := select array(select id from contact where name is NULL));
ERROR: syntax error at or near "select"
LINE 1: SELECT count(*) FROM update_name(contact_ids := select array...
The subselect returns BIGINTs, and the function accepts an array of BIGINTs. I verified that running the subselect and turning the result into an array of BIGINTs works.
Switching to positional notation did not make a difference. Using an Array constructor did not change anything, either.
Following an intuition, I wrapped the argument in parens:
SELECT count(*) FROM update_name(contact_ids := (select array(select id from contact where name is NULL)));
And that worked. I don't get why. The docs on expressions state that arguments in a function call are expressions. Function calls and Array constructors are expressions, so at least using the Array constructor should have worked.
Why do I need the parens? Where does the necessity come from, i.e. how could I have known?
The expression form you are using is called a Scalar Subquery. The manual says:
A scalar subquery is an ordinary SELECT query in parentheses that
returns exactly one row with one column ... The SELECT query is executed and
the single returned value is used in the surrounding value expression.
Your subquery returns a single value (which happens to be an array, prepared from the result of another subquery).
As a basic rule of thumb, subqueries are always in parenthesis.

ERROR: failed to find conversion function from unknown to text

There is an error on PostgreSQL that it gives on one of my select statements. I searched the web for an answer and came out empty handed. The answer given in another question did not suit my problem.
ERROR: failed to find conversion function from unknown to text
********** Error **********
ERROR: failed to find conversion function from unknown to text
SQL state: XX000
My query looks something like this:
Select *
from (select 'string' as Rowname, Data
From table)
Union all
(select 'string2' as Rowname, Data
From table)
The point of doing this is to specify what the row is at one point. The string being the name of the row. Here is my desired output:
Rowname Data
string 53
string2 87
Any possible way to fix this error?
Update: Type resolution in later versions of Postgres became smarter and this rule for UNION, CASE, and Related Constructs resolves it to text without explicit cast:
If all inputs are of type unknown, resolve as type text (the preferred type of the string category). [...]
SELECT 'string' AS rowname, data FROM tbl1
UNION ALL
SELECT 'string2', data FROM tbl2;
In older versions before Postgres 9.4 (?), or for non-default types you may still need to add an explicit cast like below.
Your statement has a couple of problems. But the error message implies that you need an explicit cast to declare the (yet unknown) data type of the string literal 'string':
SELECT text 'string' AS rowname, data FROM tbl1
UNION ALL
SELECT 'string2', data FROM tbl2;
It's enough to cast in one SELECT of a UNION query. Typically the first one, where column names are also decided. Subsequent SELECT lists with unknown types will fall in line.
In other contexts (like the VALUES clause attached to an INSERT) Postgres derives data types from target columns and tries to coerce to the right type automatically.
Select * from (select CAST('string' AS text) as Rowname, Data
From table) Union all
(select CAST('string2' AS text) as Rowname, Data
From table)
Reference