PostgreSQL constraint using prefixes - sql

Question

Let's say I have the following PostgreSQL table:
id | key
---+--------
1 | 'a.b.c'
I need to prevent inserting records with a key that is a prefix of another key. For example, I should be able to insert:
'a.b.b'
But the following keys should not be accepted:
'a.b'
'a.b.c'
'a.b.c.d'
Is there a way to achieve this - either by a constraint or by a locking mechanism (check the existance before inserting)?

Answer 1

This solution is based on PostgreSQL user-defined operators and exclusion constraints (base syntax, more details).
NOTE: more testing shows this solution does not work (yet). See bottom.
Create a function has_common_prefix(text,text) which will calculate logically what you need. Mark the function as IMMUTABLE.
CREATE OR REPLACE FUNCTION
has_common_prefix(text,text)
RETURNS boolean
IMMUTABLE STRICT
LANGUAGE SQL AS $$
SELECT position ($1 in $2) = 1 OR position ($2 in $1) = 1
$$;
Create an operator for the index
CREATE OPERATOR <~> (
PROCEDURE = has_common_prefix,
LEFTARG = text,
RIGHTARG = text,
COMMUTATOR = <~>
);
Create exclusion constraint
CREATE TABLE keys ( key text );
ALTER TABLE keys
ADD CONSTRAINT keys_cannot_have_common_prefix
EXCLUDE ( key WITH <~> );
However, the last point produces this error:
ERROR: operator <~>(text,text) is not a member of operator family "text_ops"
DETAIL: The exclusion operator must be related to the index operator class for the constraint.
This is because to create an index PostgreSQL needs logical operators to be bound with physical indexing methods, via entities calles "operator classes". So we need to provide that logic:
CREATE OR REPLACE FUNCTION keycmp(text,text)
RETURNS integer IMMUTABLE STRICT
LANGUAGE SQL AS $$
SELECT CASE
WHEN $1 = $2 OR position ($1 in $2) = 1 OR position ($2 in $1) = 1 THEN 0
WHEN $1 < $2 THEN -1
ELSE 1
END
$$;
CREATE OPERATOR CLASS key_ops FOR TYPE text USING btree AS
OPERATOR 3 <~> (text, text),
FUNCTION 1 keycmp (text, text)
;
ALTER TABLE keys
ADD CONSTRAINT keys_cannot_have_common_prefix
EXCLUDE ( key key_ops WITH <~> );
Now, it works:
INSERT INTO keys SELECT 'ara';
INSERT 0 1
INSERT INTO keys SELECT 'arka';
INSERT 0 1
INSERT INTO keys SELECT 'barka';
INSERT 0 1
INSERT INTO keys SELECT 'arak';
psql:test.sql:44: ERROR: conflicting key value violates exclusion constraint "keys_cannot_have_common_prefix"
DETAIL: Key (key)=(arak) conflicts with existing key (key)=(ara).
INSERT INTO keys SELECT 'bark';
psql:test.sql:45: ERROR: conflicting key value violates exclusion constraint "keys_cannot_have_common_prefix"
DETAIL: Key (key)=(bark) conflicts with existing key (key)=(barka).
NOTE: more testing shows this solution does not work yet: The last INSERT should fail.
INSERT INTO keys SELECT 'a';
INSERT 0 1
INSERT INTO keys SELECT 'ac';
ERROR: conflicting key value violates exclusion constraint "keys_cannot_have_common_prefix"
DETAIL: Key (key)=(ac) conflicts with existing key (key)=(a).
INSERT INTO keys SELECT 'ab';
INSERT 0 1

Answer 2

You can use ltree module to achieve this, it will let you to create hierarchical tree-like structures. Also will help you to prevent from reinventing the wheel, creating complicated regular expressions and so on. You just need to have postgresql-contrib package installed. Take a look:
--Enabling extension
CREATE EXTENSION ltree;
--Creating our test table with a pre-loaded data
CREATE TABLE test_keys AS
SELECT
1 AS id,
'a.b.c'::ltree AS key_path;
--Now we'll do the trick with a before trigger
CREATE FUNCTION validate_key_path() RETURNS trigger AS $$
BEGIN
--This query will do our validation.
--It'll search if a key already exists in 'both' directions
--LIMIT 1 because one match is enough for our validation :)
PERFORM * FROM test_keys WHERE key_path #> NEW.key_path OR key_path <# NEW.key_path LIMIT 1;
--If found a match then raise a error
IF FOUND THEN
RAISE 'Duplicate key detected: %', NEW.key_path USING ERRCODE = 'unique_violation';
END IF;
--Great! Our new row is able to be inserted
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER test_keys_validator BEFORE INSERT OR UPDATE ON test_keys
FOR EACH ROW EXECUTE PROCEDURE validate_key_path();
--Creating a index to speed up our validation...
CREATE INDEX idx_test_keys_key_path ON test_keys USING GIST (key_path);
--The command below will work
INSERT INTO test_keys VALUES (2, 'a.b.b');
--And the commands below will fail
INSERT INTO test_keys VALUES (3, 'a.b');
INSERT INTO test_keys VALUES (4, 'a.b.c');
INSERT INTO test_keys VALUES (5, 'a.b.c.d');
Of course I did not bother creating primary key and other constraints for this test. But do not forget to do so. Also, there is much more on ltree module than I'm showing, if you need something different take a look on its docs, perhaps you'll find the answer there.

Answer 3

You can try below trigger. Please note that key is sql reserve word. So I would suggest you avoid using that as column name in your table.
I have added my create table syntax also for testing purpose:
CREATE TABLE my_table
(myid INTEGER, mykey VARCHAR(50));
CREATE FUNCTION check_key_prefix() RETURNS TRIGGER AS $check_key_prefix$
DECLARE
v_match_keys INTEGER;
BEGIN
v_match_keys = 0;
SELECT COUNT(t.mykey) INTO v_match_keys
FROM my_table t
WHERE t.mykey LIKE CONCAT(NEW.mykey, '%')
OR NEW.mykey LIKE CONCAT(t.mykey, '%');
IF v_match_keys > 0 THEN
RAISE EXCEPTION 'Prefix Key Error occured.';
END IF;
RETURN NEW;
END;
$check_key_prefix$ LANGUAGE plpgsql;
CREATE TRIGGER check_key_prefix
BEFORE INSERT OR UPDATE ON my_table
FOR EACH ROW
EXECUTE PROCEDURE check_key_prefix();

Answer 4

Here is a CHECK - based solution - it may satisfy your needs.
CREATE TABLE keys ( id serial primary key, key text );
CREATE OR REPLACE FUNCTION key_check(text)
RETURNS boolean
STABLE STRICT
LANGUAGE SQL AS $$
SELECT NOT EXISTS (
SELECT 1 FROM keys
WHERE key ~ ( '^' || $1 )
OR $1 ~ ( '^' || key )
);
$$;
ALTER TABLE keys
ADD CONSTRAINT keys_cannot_have_common_prefix
CHECK ( key_check(key) );
PS. Unfortunately, it fails in one point (multi - row inserts).

Answer 5

SQL is a very powerful language. Usually you can do most of the things by plain select statements. I.e. if you do not like triggers, you can use a this method for your inserts.
The only assumption is there exists at least 1 row in the table. (*)
The table:
create table my_table
(
id integer primary key,
key varchar(100)
);
Because of the assumption, we'll have at least 1 row.(*)
insert into my_table (id, key) values (1, 'a.b.c');
Now the magic sql. The trick is replace the p_key value by your key value to insert. I have, intentionally, not put that statement into a stored procedure. Because I want it to be straight forward if you want to carry it to your application side. But usually putting sql into stored procedure is better.
insert into my_table (id, key)
select (select max(id) + 1 from my_table), p_key
from my_table
where not exists (select 'p' from my_table where key like p_key || '%' or p_key like key || '%')
limit 1;
Now the tests:
-- 'a.b.b' => Inserts
insert into my_table (id, key)
select (select max(id) + 1 from my_table), 'a.b.b'
from my_table
where not exists (select 'p' from my_table where key like 'a.b.b' || '%' or 'a.b.b' like key || '%')
limit 1;
-- 'a.b' => does not insert
insert into my_table (id, key)
select (select max(id) + 1 from my_table), 'a.b'
from my_table
where not exists (select 'p' from my_table where key like 'a.b' || '%' or 'a.b' like key || '%')
limit 1;
-- 'a.b.c' => does not insert
insert into my_table (id, key)
select (select max(id) + 1 from my_table), 'a.b.c'
from my_table
where not exists (select 'p' from my_table where key like 'a.b.c' || '%' or 'a.b.c' like key || '%')
limit 1;
-- 'a.b.c.d' does not insert
insert into my_table (id, key)
select (select max(id) + 1 from my_table), 'a.b.c.d'
from my_table
where not exists (select 'p' from my_table where key like 'a.b.c.d' || '%' or 'a.b.c.d' like key || '%')
limit 1;
(*) If you wish you can get rid of this existence of the single row by introducing an Oracle like dual table. If you wish modifying the insert statement is straight forward. Let me know if you wish to do so.

Answer 6

One possible solution is to create a secondary table that holds the prefixes of your keys, and then use a combination of unique and exclusion constraints with an insert trigger to enforce the uniqueness semantics you want.
At a high level, this approach breaks each key down into a list of prefixes and applies something similar to readers-writer lock semantics: any number of keys may share a prefix as long as none of the keys equals the prefix. To accomplish that, the list of prefixes includes the key itself with a flag that marks it as a terminal prefix.
The secondary table looks like this. We use a CHAR rather than a BOOLEAN for the flag because later on we’ll be adding a constraint that doesn’t work on boolean columns.
CREATE TABLE prefixes (
id INTEGER NOT NULL,
prefix TEXT NOT NULL,
is_terminal CHAR NOT NULL,
CONSTRAINT prefixes_id_fk
FOREIGN KEY (id)
REFERENCES your_table (id)
ON DELETE CASCADE,
CONSTRAINT prefixes_is_terminal
CHECK (is_terminal IN ('t', 'f'))
);
Now we’ll need to define a trigger on insert into your_table to also insert rows into prefixes, such that
INSERT INTO your_table (id, key) VALUES (1, ‘abc');
causes
INSERT INTO prefixes (id, prefix, is_terminal) VALUES (1, 'a', ‘f’);
INSERT INTO prefixes (id, prefix, is_terminal) VALUES (1, 'ab', ‘f’);
INSERT INTO prefixes (id, prefix, is_terminal) VALUES (1, 'abc', ’t’);
The trigger function might look like this. I’m only covering the INSERT case here, but the function could be made to handle UPDATE as well by deleting the old prefixes and then inserting the new ones. The DELETE case is covered by the cascading foreign-key constraint on prefixes.
CREATE OR REPLACE FUNCTION insert_prefixes() RETURNS TRIGGER AS $$
DECLARE
is_terminal CHAR := 't';
remaining_text TEXT := NEW.key;
BEGIN
LOOP
IF LENGTH(remaining_text) <= 0 THEN
EXIT;
END IF;
INSERT INTO prefixes (id, prefix, is_terminal)
VALUES (NEW.id, remaining_text, is_terminal);
is_terminal := 'f';
remaining_text := LEFT(remaining_text, -1);
END LOOP;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
We add this function to the table as a trigger in the usual way.
CREATE TRIGGER insert_prefixes
AFTER INSERT ON your_table
FOR EACH ROW
EXECUTE PROCEDURE insert_prefixes();
An exclusion constraint and a partial unique index will enforce that a row where is_terminal = ’t’ can't collide with another row of the same prefix regardless of its is_terminal value, and that there's only one row with is_terminal = ’t’:
ALTER TABLE prefixes ADD CONSTRAINT prefixes_forbid_conflicts
EXCLUDE USING gist (prefix WITH =, is_terminal WITH <>);
CREATE UNIQUE INDEX ON prefixes (prefix) WHERE is_terminal = 't';
This allows new rows that don’t conflict but prevents ones that do conflict, including in multi-row INSERTs.
db=# INSERT INTO your_table (id, key) VALUES (1, 'a.b.c');
INSERT 0 1
db=# INSERT INTO your_table (id, key) VALUES (2, 'a.b.b');
INSERT 0 1
db=# INSERT INTO your_table (id, key) VALUES (3, 'a.b');
ERROR: conflicting key value violates exclusion constraint "prefixes_forbid_conflicts"
db=# INSERT INTO your_table (id, key) VALUES (4, 'a.b.c');
ERROR: duplicate key value violates unique constraint "prefixes_prefix_idx"
db=# INSERT INTO your_table (id, key) VALUES (5, 'a.b.c.d');
ERROR: conflicting key value violates exclusion constraint "prefixes_forbid_conflicts"
db=# INSERT INTO your_table (id, key) VALUES (6, 'a.b.d'), (7, 'a');
ERROR: conflicting key value violates exclusion constraint "prefixes_forbid_conflicts"