I am trying to create BTREE index as follows:
CREATE BTREE INDEX index_r_connected FOR ()-[r:connected_to]->() on (r.source,r.target)
But I am getting following error :
> Invalid input 'I': expected whitespace, comment, '=', node labels,
> MapLiteral, a parameter, a parameter (old syntax), a relationship
> pattern, ',', FROM GRAPH, USE GRAPH, CONSTRUCT, LOAD CSV, START,
> MATCH, UNWIND, MERGE, CREATE UNIQUE, CREATE, SET, DELETE, REMOVE,
> FOREACH, WITH, CALL, RETURN, UNION, ';' or end of input (line 1,
> column 14 (offset: 13)) "CREATE BTREE INDEX index_r_connected FOR
> ()-[r:connected_to]->() on (r.source,r.target)"
Related
When using primitive types such as Integer, I can without any problems do a query like this:
with connection.cursor() as cursor:
cursor.execute(sql='''SELECT count(*) FROM account
WHERE %(pk)s ISNULL OR id %(pk)s''', params={'pk': 1})
Which would either return row with id = 1 or it would return all rows if pk parameter was equal to None.
However, when trying to use similar approach to pass a list/tuple of IDs, I always produce a SQL syntax error when passing empty/None tuple, e.g. trying:
with connection.cursor() as cursor:
cursor.execute(sql='''SELECT count(*) FROM account
WHERE %(ids)s ISNULL OR id IN %(ids)s''', params={'ids': (1,2,3)})
works, but passing () produces SQL syntax error:
psycopg2.ProgrammingError: syntax error at or near ")"
LINE 1: SELECT count(*) FROM account WHERE () ISNULL OR id IN ()
Or if I pass None I get:
django.db.utils.ProgrammingError: syntax error at or near "NULL"
LINE 1: ...LECT count(*) FROM account WHERE NULL ISNULL OR id IN NULL
I tried putting the argument in SQL in () - (%(ids)s) - but that always breaks one or the other condition. I also tried playing around with pg_typeof or casting the argument, but with no results.
Notes:
the actual SQL is much more complex, this one here is a simplification for illustrative purposes
as a last resort - I could alter the SQL in Python based on the argument, but I really wanted to avoid that.)
At first I had an idea of using just 1 argument, but replacing it with a dummy value [-1] and then using it like
cursor.execute(sql='''SELECT ... WHERE -1 = any(%(ids)s) OR id = ANY(%(ids)s)''', params={'ids': ids if ids else [-1]})
but this did a Full table scan for non empty lists, which was unfortunate, so a no go.
Then I thought I could do a little preprocessing in python and send 2 arguments instead of just the single list- the actual list and an empty list boolean indicator. That is
cursor.execute(sql='''SELECT ... WHERE %(empty_ids)s = TRUE OR id = ANY(%(ids)s)''', params={'empty_ids': not ids, 'ids': ids})
Not the most elegant solution, but it performs quite well (Index scan for non empty list, Full table scan for empty list - but that returns the whole table anyway, so it's ok)
And finally I came up with the simplest solution and quite elegant:
cursor.execute(sql='''SELECT ... WHERE '{}' = %(ids)s OR id = ANY(%(ids)s)''', params={'ids': ids})
This one also performs Index scan for non empty lists, so it's quite fast.
From the psycopg2 docs:
Note You can use a Python list as the argument of the IN operator using the PostgreSQL ANY operator.
ids = [10, 20, 30]
cur.execute("SELECT * FROM data WHERE id = ANY(%s);", (ids,))
Furthermore ANY can also work with empty lists, whereas IN () is a SQL syntax error.
I have a table where one column is array:
CREATE TABLE inherited_tags (
id serial,
tags text[]
);
Sample values:
INSERT INTO inherited_tags (tags) VALUES
(ARRAY['A','B','C']), -- id: 1
(ARRAY['D','E']), -- id: 2
(ARRAY['A','B']), -- id: 3
(ARRAY['C','D']), -- id: 4
(ARRAY['D','F']), -- id: 5
(ARRAY['A']); -- id: 6
I want to find rows which tags column contains some subset of words inside array. For example for input:
ARRAY[ARRAY['A','C'], ARRAY['F'], ARRAY['E']]::text[][]
I want to find all rows that contain ('A' and 'C') OR ('F') OR ('E'). So for example above I should get rows with ids: 1, 2, 5.
I was hoping that I could use syntax like this:
SELECT * FROM inherited_tags WHERE
tags #> ANY(ARRAY[ARRAY['A','C'], ARRAY['F'], ARRAY['E']]::text[][])
but I get error:
ERROR: operator does not exist: text[] #> text
LINE 1: SELECT * FROM inherited_tags where tags <# ANY(ARRAY[ARRAY['...
Postgres 9.6
plpgsql solution is acceptable but SQL is preferred.
DB-FIDDLE: https://www.db-fiddle.com/f/cKCr7Sfab6u8rqaCHhJvPk/0
The problem comes from the fact that the text[] and text[][] data types are internally the same data type. An array has a base type and dimensions, and the ANY operator will always extract the base type to compare, which will always be text and not text[]. It doesn't help that multidimensional arrays require that each subelement has the same length as every other. You can have ARRAY[ARRAY['A','C'],ARRAY['B','N']], but not ARRAY[ARRAY[2,3],ARRAY[1]].
In short, there is no direct way to make that particular query work. I tried to create a function and an operator for this as well, and that doesn't work, either, for different reasons. See how that went:
CREATE OR REPLACE FUNCTION check_tag_matches(
IN leftside text[],
IN rightside text)
RETURNS BOOLEAN AS
$BODY$
DECLARE rightarr text[];
BEGIN
SELECT CAST(rightside as text[]) INTO rightarr;
RETURN SELECT leftside #> rightarr;
END;
$BODY$
LANGUAGE plpgsql STABLE;
CREATE OPERATOR public.>>(
PROCEDURE = check_tag_matches,
LEFTARG = text[],
RIGHTARG = text,
COMMUTATOR = >>);
Then when testing it:
test=# SELECT * FROM inherited_tags WHERE
tags >> ANY(ARRAY[ARRAY['A','M'], ARRAY['F','E'], ARRAY['E','R']]::text[][]);
ERROR: malformed array literal: "A"
DETAIL: Array value must start with "{" or dimension information.
CONTEXT: SQL statement "SELECT CAST(rightside as text[])"
PL/pgSQL function check_tag_matches(text[],text) line 4 at SQL statement
It seems that when you try using a multidimensional array like ARRAY[ARRAY['A','M'], ARRAY['F','E'], ARRAY['E','R']]::text[][] in ANY(), it iterates not over ARRAY['A','M'], then ARRAY['F','E'], then ARRAY['E','R'], but over 'A','M','F','E','E','R'. The same thing happens when with unnest.
test=# SELECT unnest(ARRAY[ARRAY['A','M'], ARRAY['F','E'], ARRAY['E','R']]::text[][]);
unnest
--------
A
M
F
E
E
R
(6 rows)
Your remaining optiona are to define a function that will read array_length(rightside,1) and array_length(rightside,2) and use nested loops to check it all, or you can send multiple queries to get the inherited tags for each tag, or restructure your data somehow. And you can't even access the ARRAY['A','M'] element using rightside[1] to iterate over it, you're forced to go to the deepest level.
I don't think you can do that with a single condition because of the "contains A and C" requirement.
SELECT *
FROM inherited_tags
WHERE tags #> ARRAY['A','C']
OR tags && array['F', 'E'];
tags #> ARRAY['A','C'] selects those where tags contains all elements from ARRAY['A','C'] and tags && array['F', 'E'] selects those rows that contain at least one of the tags from array['F', 'E']
Updated DB Fiddle: https://www.db-fiddle.com/f/rXsjqEN3ry67uxJtEs3GM9/0
u can try
SELECT * FROM table WHERE
tags #> ARRAY['A','C']::varchar[]
OR
tags #> ARRAY['E']::varchar[]
OR
tags #> ARRAY['F']::varchar[]
I have a nullable array column in my table: Array(Nullable(UInt16)). I want to be able to query this column using arrayExists (or arrayAll) to check if it contains a value above a certain threshold but I'm getting an exception when the array contains a null value:
Exception: Expression for function arrayExists must return UInt8, found Nullable(UInt8)
My query is below where distance is the array column:
SELECT * from TracabEvents_ArrayTest
where arrayExists(x -> x > 9, distance);
I've tried updating the comparison in the lambda to "(isNotNull(x) and x > 9)" but I'm still getting the error. Is there any way of handling nulls in these expressions or are they not supported yet?
Add a condition to filter rows with empty list using notEmpty and assumeNotNull for x in arrayExists.
SELECT * FROM TracabEvents_ArrayTest WHERE notEmpty(distance) AND arrayExists(x -> assumeNotNull(x) > 9, distance)
I have a 1 to many Organization: Users relationship.
I want to fetch the usernames of all User model of an Organization, capture a part of that username and append/substitute it with new value.
Here is how I am doing:
Form the raw SQL to Get the matching usernames and replace them with new value.
raw = "SELECT REGEXP_REPLACE($1::string[], '(^[a-z0-9]+)((\s[a-z0-9]+))*\#([a-z0-9]+)$', m.name[1] || '#' || $2) FROM (SELECT REGEXP_MATCHES($1::string[], '(^[a-z0-9]+)((\s[a-z0-9]+))*\#([a-z0-9]+)$') AS name) m"
Get the matching usernames and replace them with new value.
usernames: list of usernames retrieved from queryable
Repo.query(raw, [usernames, a_string])
Error I am getting
SELECT REGEXP_REPLACE($1::string[], '(^[a-z0-9]+)(( [a-z0-9]+))#([a-z0-9]+)$', m.name[1] || '#' || $2) FROM (SELECT REGEXP_MATCHES($1::string[], '(^[a-z0-9]+)(( [a-z0-9]+))#([a-z0-9]+)$') AS name) m [["tradeboox#trdbx18"], "trdbx17"]
{:error,
%Postgrex.Error{connection_id: 7222, message: nil,
postgres: %{code: :undefined_object, file: "parse_type.c", line: "257",
message: "type \"string[]\" does not exist", pg_code: "42704",
position: "137", routine: "typenameType", severity: "ERROR",
unknown: "ERROR"}}}
FYI: The username field of User model is of type citext
Once I get the replaced values, I want to update the User with something like
update([u], set: [username: new_values])
Any ideas on how to proceed with this?
`
There is no string type in PostgreSQL.
Function regexp_matches accepts as first parameter only text and it can't be array. So what you need to do is first change that type to text, then unnest($1::text[]) your array. Iterate over resulting set of rows with those regexp.
raw = "SELECT REGEXP_REPLACE(m.item, '(^[a-z0-9]+)((\s[a-z0-9]+))*\#([a-z0-9]+)$', m.name[1] || '#' || $2)
FROM (
SELECT item, REGEXP_MATCHES(item, '(^[a-z0-9]+)((\s[a-z0-9]+))*\#([a-z0-9]+)$') AS name
FROM unnest($1::text[]) AS items(item)
) m"
If I understand it correctly, you are trying to replace everything after # with some different string - if that is the case, then your regexp will put anything after spacebar into second element of matches array. You would need this instead: ((^[a-z0-9]+)(\s[a-z0-9]+)*).
If above is true, then you can do all that much easier with this:
SELECT REGEXP_REPLACE(item, '((^[a-z0-9]+)(\s[a-z0-9]+)*)\#([a-z0-9]+)$', '\1' || $2) AS name
FROM unnest($1::text[]) AS items(item)
Best practice however is to simply do replace in UPDATE statement:
UPDATE "User" SET
name = concat(split_part(name, '#', 1), '#', $2)
WHERE organization_id = $3
AND name ~* '^[a-z0-9]+(\s[a-z0-9]+)*\#[a-z0-9]+$'
It will split name by #, take first part, then append # and whatever is assigned to $2 (domain name I guess). It will update only rows that have organization_id matching to some id and have names matching your regexp (you can omit regexp if you want to change all names from organization). Make sure that table in actually named User, case sensitive, or remove double quotes to have case insensitive version.
I sadly do not know how to do this in your ORM.
I am trying to query a table with a column with the postgresql array data type in Rails 4.
Here is the table schema:
create_table "db_of_exercises", force: true do |t|
t.text "preparation"
t.text "execution"
t.string "category"
t.datetime "created_at"
t.datetime "updated_at"
t.string "name"
t.string "body_part", default: [], array: true
t.hstore "muscle_groups"
t.string "equipment_type", default: [], array: true
end
The following query works:
SELECT * FROM db_of_exercises WHERE ('Arms') = ANY (body_part);
However, this query does not:
SELECT * FROM db_of_exercises WHERE ('Arms', 'Chest') = ANY (body_part);
It throws this error:
ERROR: operator does not exist: record = character varying
This does not work for me either:
SELECT * FROM "db_of_exercises" WHERE "body_part" IN ('Arms', 'Chest');
That throws this error:
ERROR: array value must start with "{" or dimension information
So, how do I query a column with an array data type in ActiveRecord??
What I have right now is:
#exercises = DbOfExercise.where(body_part: params[:body_parts])
I want to be able to query records that have more than one body_part associated with them, which was the whole point of using an array data type, so if someone could enlighten me on how to do this that would be awesome. I don't see it anywhere in the docs.
Final solution for posterity:
Using the overlap operator (&&):
SELECT * FROM db_of_exercises WHERE ARRAY['Arms', 'Chest'] && body_part;
I was getting this error:
ERROR: operator does not exist: text[] && character varying[]
so I typecasted ARRAY['Arms', 'Chest'] to varchar:
SELECT * FROM db_of_exercises WHERE ARRAY['Arms', 'Chest']::varchar[] && body_part;
and that worked.
I don't think that it has related to rails.
What if you do the follow?
SELECT * FROM db_of_exercises WHERE 'Arms' = ANY (body_part) OR 'Chest' = ANY (body_part)
I know that rails 4 supports Postgresql ARRAY datatype, but I'm not sure if ActiveRecord creates new methods for query the datatype. Maybe you can use Array Overlap I mean the && operator and then doind something like:
WHERE ARRAY['Arms', 'Chest'] && body_part
or maybe give a look to this gem: https://github.com/dockyard/postgres_ext/blob/master/docs/querying.md
And then do a query like:
DBOfExercise.where.overlap(:body_part => params[:body_parts])
#Aguardientico is absolutely right that what you want is the array overlaps operator &&. I'm following up with some more explanation, but would prefer you to accept that answer, not this one.
Anonymous rows (records)
The construct ('item1', 'item2', ...) is a row-constructor unless it appears in an IN (...) list. It creates an anonymous row, which PostgreSQL calls a "record". The error:
ERROR: operator does not exist: record = character varying
is because ('Arms', 'Chest') is being interpreted as if it were ROW('Arms', 'Chest'), which produces a single record value:
craig=> SELECT ('Arms', 'Chest'), ROW('Arms', 'Chest'), pg_typeof(('Arms', 'Chest'));
row | row | pg_typeof
--------------+--------------+-----------
(Arms,Chest) | (Arms,Chest) | record
(1 row)
and PostgreSQL has no idea how it's supposed to compare that to a string.
I don't really like this behaviour; I'd prefer it if PostgreSQL required you to explicitly use a ROW() constructor when you want an anonymous row. I expect that the behaviour shown here exists to support SET (col1,col2,col3) = (val1,val2,val3) and other similar operations where a ROW(...) constructor wouldn't make as much sense.
But the same thing with a single item works?
The reason the single ('Arms') case works is because unless there's a comma it's just a single parenthesised value where the parentheses are redundant and may be ignored:
craig=> SELECT ('Arms'), ROW('Arms'), pg_typeof(('Arms')), pg_typeof(ROW('Arms'));
?column? | row | pg_typeof | pg_typeof
----------+--------+-----------+-----------
Arms | (Arms) | unknown | record
(1 row)
Don't be alarmed by the type unknown. It just means that it's a string literal that hasn't yet had a type applied:
craig=> SELECT pg_typeof('blah');
pg_typeof
-----------
unknown
(1 row)
Compare array to scalar
This:
SELECT * FROM "db_of_exercises" WHERE "body_part" IN ('Arms', 'Chest');
fails with:
ERROR: array value must start with "{" or dimension information
because of implicit casting. The type of the body_part column is text[] (or varchar[]; same thing in PostgreSQL). You're comparing it for equality with the values in the IN clause, which are unknown-typed literals. The only valid equality operator for an array is = another array of the same type, so PostgreSQL figures that the values in the IN clause must also be arrays of text[] and tries to parse them as arrays.
Since they aren't written as array literals, like {"FirstValue","SecondValue"}, this parsing fails. Observe:
craig=> SELECT 'Arms'::text[];
ERROR: array value must start with "{" or dimension information
LINE 1: SELECT 'Arms'::text[];
^
See?
It's easier to understand this once you see that IN is actually just a shorthand for = ANY. It's an equality comparison to each element in the IN list. That isn't what you want if you really want to find out if two arrays overlap.
So that's why you want to use the array overlaps operator &&.