No unique or exclusion constraint matching the ON CONFLICT - sql

I'm getting the following error when doing the following type of insert:
Query:
INSERT INTO accounts (type, person_id) VALUES ('PersonAccount', 1) ON
CONFLICT (type, person_id) WHERE type = 'PersonAccount' DO UPDATE SET
updated_at = EXCLUDED.updated_at RETURNING *
Error:
SQL execution failed (Reason: ERROR: there is no unique or exclusion
constraint matching the ON CONFLICT specification)
I also have an unique INDEX:
CREATE UNIQUE INDEX uniq_person_accounts ON accounts USING btree (type,
person_id) WHERE ((type)::text = 'PersonAccount'::text);
The thing is that sometimes it works, but not every time. I randomly get
that exception, which is really strange. It seems that it can't access that
INDEX or it doesn't know it exists.
Any suggestion?
I'm using PostgreSQL 9.5.5.
Example while executing the code that tries to find or create an account:
INSERT INTO accounts (type, person_id, created_at, updated_at) VALUES ('PersonAccount', 69559, '2017-02-03 12:09:27.259', '2017-02-03 12:09:27.259') ON CONFLICT (type, person_id) WHERE type = 'PersonAccount' DO UPDATE SET updated_at = EXCLUDED.updated_at RETURNING *
SQL execution failed (Reason: ERROR: there is no unique or exclusion constraint matching the ON CONFLICT specification)
In this case, I'm sure that the account does not exist. Furthermore, it never outputs the error when the person has already an account. The problem is that, in some cases, it also works if there is no account yet. The query is exactly the same.

Per the docs,
All table_name unique indexes that, without regard to order, contain exactly the
conflict_target-specified columns/expressions are inferred (chosen) as arbiter
indexes. If an index_predicate is specified, it must, as a further requirement
for inference, satisfy arbiter indexes.
The docs go on to say,
[index_predicate are u]sed to allow inference of partial unique indexes
In an understated way, the docs are saying that when using a partial index and
upserting with ON CONFLICT, the index_predicate must be specified. It is not
inferred for you. I learned this
here, and the following example demonstrates this.
CREATE TABLE test.accounts (
id int PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY,
type text,
person_id int);
CREATE UNIQUE INDEX accounts_note_idx on accounts (type, person_id) WHERE ((type)::text = 'PersonAccount'::text);
INSERT INTO test.accounts (type, person_id) VALUES ('PersonAccount', 10);
so that we have:
unutbu=# select * from test.accounts;
+----+---------------+-----------+
| id | type | person_id |
+----+---------------+-----------+
| 1 | PersonAccount | 10 |
+----+---------------+-----------+
(1 row)
Without index_predicate we get an error:
INSERT INTO test.accounts (type, person_id) VALUES ('PersonAccount', 10) ON CONFLICT (type, person_id) DO NOTHING;
-- ERROR: there is no unique or exclusion constraint matching the ON CONFLICT specification
But if instead you include the index_predicate, WHERE ((type)::text = 'PersonAccount'::text):
INSERT INTO test.accounts (type, person_id) VALUES ('PersonAccount', 10)
ON CONFLICT (type, person_id)
WHERE ((type)::text = 'PersonAccount'::text) DO NOTHING;
then there is no error and DO NOTHING is honored.

A simple solution of this error
First of all let's see the cause of error with a simple example. Here is the table mapping products to categories.
create table if not exists product_categories (
product_id uuid references products(product_id) not null,
category_id uuid references categories(category_id) not null,
whitelist boolean default false
);
If we use this query:
INSERT INTO product_categories (product_id, category_id, whitelist)
VALUES ('123...', '456...', TRUE)
ON CONFLICT (product_id, category_id)
DO UPDATE SET whitelist=EXCLUDED.whitelist;
This will give you error No unique or exclusion constraint matching the ON CONFLICT because there is no unique constraint on product_id and category_id. There could be multiple rows having the same combination of product and category id (so there can never be a conflict on them).
Solution:
Use unique constraint on both product_id and category_id like this:
create table if not exists product_categories (
product_id uuid references products(product_id) not null,
category_id uuid references categories(category_id) not null,
whitelist boolean default false,
primary key(product_id, category_id) -- This will solve the problem
-- unique(product_id, category_id) -- OR this if you already have a primary key
);
Now you can use ON CONFLICT (product_id, category_id) for both columns without any error.
In short: Whatever column(s) you use with on conflict, they should have unique constraint.

The easy way to fix it is by setting the conflicting column as UNIQUE

I did not have a chance to play with UPSERT, but I think you have a case from
docs:
Note that this means a non-partial unique index (a unique index
without a predicate) will be inferred (and thus used by ON CONFLICT)
if such an index satisfying every other criteria is available. If an
attempt at inference is unsuccessful, an error is raised.

I solved the same issue by creating one UNIQUE INDEX for ALL columns you want to include in the ON CONFLICT clause, not one UNIQUE INDEX for each of the columns.
CREATE TABLE table_name (
element_id UUID NOT NULL DEFAULT gen_random_uuid(),
timestamp TIMESTAMP NOT NULL DEFAULT now():::TIMESTAMP,
col1 UUID NOT NULL,
col2 STRING NOT NULL ,
col3 STRING NOT NULL ,
CONSTRAINT "primary" PRIMARY KEY (element_id ASC),
UNIQUE (col1 asc, col2 asc, col3 asc)
);
Which will allow to query like
INSERT INTO table_name (timestamp, col1, col2, col3) VALUES ('timestamp', 'uuid', 'string', 'string')
ON CONFLICT (col1, col2, col3)
DO UPDATE timestamp = EXCLUDED.timestamp, col1 = EXCLUDED.col1, col2 = excluded.col2, col3 = col3.excluded;

Related

Ignore specific rows and merge operation using on conflict in Postgresql

Table Structure:
create table example_test (a_id integer, b_id integer, c_id integer, flag integer);
Unique Constraint:
Alter table example_test
add constraint u_key unique(a_id, b_id, c_id);
My code:
with a_ins_upd as (
Insert into example (a_id, b_id, c_id, flag)
select x.a_id, x.b_id, x.c_id, x.flag
from <input_tableType> x
on conflict on constraint u_key
do update
set
a_id = excluded.a_id,
b_id = excluded.b_id,
c_id = excluded.c_id,
flag = excluded.flag
where flag = 0
)
Operations on Data:
I want to ignore the records with flag=1, and do the Upsert on the other records.
Basically I think you want a filtered unique key.
Instead of:
alter table example_test
add constraint u_key unique(a_id, b_id, c_id);
You could do:
create unique index example_idx on example_test(a_id, b_id, c_id) where flag = 0;
You can then use a regular insert ... on conflict clause (without the where clause in on conflict).
ok, Exclusion Constraint is not supported by on conflict clause. Makes sense, it can update multiple records. Only way is to handle programmatically.

Check for uniqueness of column in postgres table

I need to ensure that the values in a column from a table are unique as part of a larger process.
I'm aware of the UNIQUE constraint, but I'm wondering if there is a better way to do the check.
I'm running the queries using psycopg2 so adding that tag on the off chance there's something in there that can help with this.
If the column is unique I can add a constraint. If the column is not unique adding the constraint will return an error.
If there is already a constraint of the same name a useful error is returned. in this case would prefer to just check for the existing constraint.
If the column is the primary key, the unique constraint can be added without error but in this case it would be preferable to just recognize that the column must be unique based on the primary key.
Code examples of this below.
DROP TABLE IF EXISTS unique_test;
CREATE TABLE unique_test (
pkey INT PRIMARY KEY,
unique_yes CHAR(1),
unique_no CHAR(1)
);
INSERT INTO unique_test (pkey, unique_yes, unique_no)
VALUES(1, 'a', 'a'),
(2, 'b', 'a');
CREATE UNIQUE INDEX CONCURRENTLY u_test_1 ON unique_test (unique_yes);
ALTER TABLE unique_test
ADD CONSTRAINT unique_target_1
UNIQUE USING INDEX u_test_1;
-- the above runs no problem
-- check what happens when column is not unique
CREATE UNIQUE INDEX CONCURRENTLY u_test_2 ON unique_test (unique_no);
ALTER TABLE unique_test
ADD CONSTRAINT unique_target_2
UNIQUE USING INDEX u_test_2;
-- returns:
-- SQL Error [23505]: ERROR: could not create unique index "u_test_2"
-- Detail: Key (unique_no)=(a) is duplicated.
CREATE UNIQUE INDEX CONCURRENTLY u_test_1 ON unique_test (unique_yes);
ALTER TABLE unique_test
ADD CONSTRAINT unique_target_1
UNIQUE USING INDEX u_test_1;
-- returns
-- SQL Error [42P07]: ERROR: relation "unique_target_1" already exists
-- test what happens if adding constrint to primary key column
CREATE UNIQUE INDEX CONCURRENTLY u_test_pkey ON unique_test (pkey);
ALTER TABLE unique_test
ADD CONSTRAINT unique_target_pkey
UNIQUE USING INDEX u_test_pkey;
-- this runs no problem but is inefficient.
If all you want to do is verify that values are unique, then use a query:
select unique_no, count(*)
from unique_test
group by unique_no
having count(*) > 1;
If it needs to be boolean output:
select not exists (
select unique_no, count(*)
from unique_test
group by unique_no
having count(*) > 1
);
If you just want a flag, you can use:
select count(*) <> count(distinct uniq_no) as duplicate_flag
from unique_test;
DELETE FROM
zoo x
USING zoo y
WHERE
x.animal_id < y.animal_id
AND x.animal = y.animal;
I think this is simpler, https://kb.objectrocket.com/postgresql/delete-duplicate-rows-in-postgresql-762 for reference

Postgres 9.3 index involving JSON column

I have this table structure:
CREATE TABLE user_items
(
user_id bigint references users(id) NOT NULL,
item_id bigint references items(id) NOT NULL,
col1 json DEFAULT '[{"text":""}]',
col2 json DEFAULT '[{"date":"","text":""}]',
col3 json DEFAULT '{"text":""}',
PRIMARY KEY (user_id, item_id)
)
I will be running queries such as this:
SELECT * FROM user_items WHERE item_id = '?' AND col1 IS NOT NULL
Do I need an index (item_id, col1) in this case ?
And if so, what's the right way to do it, because when trying it Postgres is throwing an error since col1 is a JSON type.
I suggest to use a partial index on item_id:
CREATE INDEX foo_idx ON user_items (item_id)
WHERE col1 IS NOT NULL
The data type of col1 is irrelevant here. Be sure to include the verbatim WHERE clause in queries to allow Postgres to use this index.
You would only need it if each item_id occurs many times and col1 is usually NULL. If either of those is not true, just make the index on (item_id). The database will have to visit the row and filter out the ones where col1 is NULL, but if NULL is rare that will be no big deal.
If NULL is common, then try a "functional" or "expression" index on (item_id, (col1 is not null))

how to create a Foreign-Key constraint to a subset of the rows of a table?

I have a reference table, say OrderType that collects different types of orders:
CREATE TABLE IF NOT EXISTS OrderType (name VARCHAR);
ALTER TABLE OrderType ADD PRIMARY KEY (name);
INSERT INTO OrderType(name) VALUES('sale-order-type-1');
INSERT INTO OrderType(name) VALUES('sale-order-type-2');
INSERT INTO OrderType(name) VALUES('buy-order-type-1');
INSERT INTO OrderType(name) VALUES('buy-order-type-2');
I wish to create a FK constraint from another table, say SaleInformation, pointing to that table (OrderType). However, I am trying to express that not all rows of OrderType are eligible for the purposes of that FK (it should only be sale-related order types).
I thought about creating a view of table OrderType with just the right kind of rows (view SaleOrderType) and adding a FK constraint to that view, but PostgreSQL balks at that with:
ERROR: referenced relation "SaleOrderType" is not a table
So it seems I am unable to create a FK constraint to a view (why?). Am I only left with the option of creating a redundant table to hold the sale-related order types? The alternative would be to simply allow the FK to point to the original table, but then I am not really expressing the constraint as strictly as I would like to.
I think your schema should be something like this
create table order_nature (
nature_id int primary key,
description text
);
insert into order_nature (nature_id, description)
values (1, 'sale'), (2, 'buy')
;
create table order_type (
type_id int primary key,
description text
);
insert into order_type (type_id, description)
values (1, 'type 1'), (2, 'type 2')
;
create table order_nature_type (
nature_id int references order_nature (nature_id),
type_id int references order_type (type_id),
primary key (nature_id, type_id)
);
insert into order_nature_type (nature_id, type_id)
values (1, 1), (1, 2), (2, 1), (2, 2)
;
create table sale_information (
nature_id int default 1 check (nature_id = 1),
type_id int,
foreign key (nature_id, type_id) references order_nature_type (nature_id, type_id)
);
If the foreign key clause would also accept an expression the sale information could omit the nature_id column
create table sale_information (
type_id int,
foreign key (1, type_id) references order_nature_type (nature_id, type_id)
);
Notice the 1 in the foreign key
You could use an FK to OrderType to ensure referential integrity and a separate CHECK constraint to limit the order types.
If your OrderType values really are that structured then a simple CHECK like this would suffice:
check (c ~ '^sale-order-type-')
where c is order type column in SaleInformation
If the types aren't structured that way in reality, then you could add some sort of type flag to OrderType (say a boolean is_sales column), write a function which uses that flag to determine if an order type is a sales order:
create or replace function is_sales_order_type(text ot) returns boolean as $$
select exists (select 1 from OrderType where name = ot and is_sales);
$$ language sql
and then use that in your CHECK:
check(is_sales_order_type(c))
You don't of course have to use a boolean is_sales flag, you could have more structure than that, is_sales is just for illustrative purposes.

What options are available for applying a set level constraint in PostgreSQL?

I have a situation where I need to ensure that there is only one active record with the same object_id and user_id at any time. Here is a representative table:
CREATE TABLE actions (
id SERIAL PRIMARY KEY,
object_id integer,
user_id integer,
active boolean default true,
created_at timestamptz default now()
);
By only one active record at a time, I mean you could have a sequence of inserts like the following:
insert into actions (object_id, user_id, active) values (1, 1, true);
insert into actions (object_id, user_id, active) values (1, 1, false);
but doing a subsequent
insert into actions (object_id, user_id, active) values (1, 1, true);
should fail because at this point in time, there already exists 1 active tuple with object_id = 1 and user_id = 1.
I'm using PostgreSQL 8.4.
I saw this post which looks interesting, but its Oracle specific.
I also saw this post but it requires more care regarding the transaction isolation level. I don't think it would work as-is in read committed mode.
My question is what other options are available to unsure this kind of constraint?
Edit: Removed the third insert in the first set. I think it was confusing the example. I also added the created_at time stamp to help with the context. To reiterate, there can be multiple (object_id, user_id, false) tuples, but only one (object_id, user_id, true) tuple.
Update: I accepted Craig's answer, but for others who may stumble upon something similar, here is another possible (though suboptimal) solution.
CREATE TABLE action_consistency (
object_id integer,
user_id integer,
count integer default 0,
primary key (object_id, user_id),
check (count >= 0 AND count <= 1)
);
CREATE OR REPLACE FUNCTION keep_action_consistency()
RETURNS TRIGGER AS
$BODY$
BEGIN
IF NEW.active THEN
UPDATE action_consistency
SET count = count + 1
WHERE object_id = NEW.object_id AND
user_id = NEW.user_id;
INSERT INTO action_consistency (object_id, user_id, count)
SELECT NEW.object_id, NEW.user_id, 1
WHERE NOT EXISTS (SELECT 1
FROM action_consistency
WHERE object_id = NEW.object_id AND
user_id = NEW.user_id);
ELSE
-- assuming insert will be active for simplicity
UPDATE action_consistency
SET count = count - 1
WHERE object_id = NEW.object_id AND
user_id = NEW.user_id;
END IF;
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql;
CREATE TRIGGER ensure_action_consistency AFTER INSERT OR UPDATE ON actions
FOR EACH ROW EXECUTE PROCEDURE keep_action_consistency();
It requires the use of a tracking table. For what I hope are obvious reasons, this is not at all desirable. It means that you have an additional row each distinct (object_id, user_id) in actions.
Another reason why I accepted #Craig Ringer's answer is that there are foreign key references to actions.id in other tables that are also rendered inactive when a given action tuple changes state. This why the history table is less ideal in this scenario. Thank you for the comments and answers.
Given your specification that you want to limit only one entry to being active at a time, try:
CREATE TABLE actions (
id SERIAL PRIMARY KEY,
object_id integer,
user_id integer,
active boolean default true,
created_at timestamptz default now()
);
CREATE UNIQUE INDEX actions_unique_active_y ON actions(object_id,user_id) WHERE (active = 't');
This is a partial unique index, a PostgreSQL specific feature - see partial indexes. It constrains the set such that only one (object_id,user_id) tuple may exist where active is true.
While that strictly answers your question as you explained further in comments, I think wildplasser's answer describes the more correct choice and best approach.
You can use UNIQUE constraint to ensure that the column contains the unique value...
Here, set of object_id and user_id have been made unique....
CREATE TABLE actions (
id SERIAL PRIMARY KEY,
object_id integer,
user_id integer,
active boolean default true,
UNIQUE (object_id , user_id )
);
Check Out SQLFIDDLE
Similary, if you want to make set of object_id,user_id and active as UNIQUE, you can simply add the column name in the list of UNIQUE.
CREATE TABLE actions (
id SERIAL PRIMARY KEY,
object_id integer,
user_id integer,
active boolean default true,
UNIQUE (object_id , user_id,active )
);
Check Out SQLFIDDLE
Original:
CREATE TABLE actions (
id SERIAL PRIMARY KEY,
object_id integer,
user_id integer,
active boolean default true
);
my version:
CREATE TABLE actions (
object_id integer NOT NULL REFERENCES objects (id),
user_id integer NOT NULL REFERENCES users(id),
PRIMARY KEY (user_id, object_id)
);
What are the differences:
omitted the surrogate key. It is useless, it enforces no constraint, and nobody will ever reference it
added a (composite) primary key, which happens to be the logical key
changed the two fields to NOT NULL, and made them into foreign keys (what would be the meaning of a row that would not exist in the users or objects table?
removed the boolean flag. What is the semantic difference between a {user_id,object_id} tuple that does not exist versus one that does exist but has it's "active" flag set to false? Why create three states when you only need two?