Is it safe to drop and then create the foreign key constraints inside a transaction? - sql

Question

I have a table A that references a table B. Table B needs to be populated with updated data from an external source and for efficiency I use TRUNCATE followed by a COPY. This is done even when the application is live.
To further improve efficiency, as suggested in the documentation, I want to drop and then recreate the foreign keys.
However I have some doubts.
If I drop the FKs, COPY and then recreate FKs inside the same transaction, can I be sure that the constraint is preserved even on data inserted in table A during the transaction? I ask this because in theory a transaction is atomic, but in the docs, about the temporary removal of FKs say:
there is a trade-off between data load speed and loss of error checking while the constraint is missing.
If there's a chance that a wrong reference is inserted in the meantime, what happens when you try to recreate the FK constraints?

Answer 1

TRUNCATE is not allowed on any table referenced by a foreign key, unless you use TRUNCATE CASCADE, which will also truncate the referencing tables. The DEFERRABLE status of the constraint does not affect this. I don't think there is any way around this; you will need to drop the constraint.
However, there is no risk of an integrity violation in doing so. ALTER TABLE ... ADD CONSTRAINT locks the table in question (as does TRUNCATE), so your import process is guaranteed to have exclusive access to the table for the duration of its transaction. Any attempts at concurrent inserts will simply hang until the import has committed, and by the time they are allowed to proceeed, the constraint will be back in place.

Answer 2

You can make the foreign key constraint deferrable (initially deferred). That way it will be checked just once at the end of the transaction.
ALTER TABLE
xxx
ADD CONSTRAINT
xxx_yyy_id_fk FOREIGN KEY (yyy_id)
REFERENCES
yyy
DEFERRABLE INITIALLY DEFERRED;
In all the cases, transactions are fully atomic in PostgreSQL (not only in theory), including DDL statements (such as CREATE/DROP constraint), so even if you drop a foreign key, then insert data, then create the foreign key and do everything in one transaction, then you are safe - if the recreation of the foreign key constraint fails, then the inserted data will also be dismissed.
Still, it is better to switch to deferred foreign keys, rather than dropping and then creating them.

Answer 3

Analytic answer: measure the number of new/same/updated/deleted records.
There are four cases:
The key in the B table is not present in the b_import: delete
The key in the b_import is not present on the old B: insert
The key is present in both old B and new B, but the contents are the same: ignore
The keys are the same, but the attribete values differ: Update
-- some test data for `A`, `B` and `B_import`:
CREATE TABLE b
( id INTEGER NOT NULL PRIMARY KEY
, payload varchar
);
INSERT INTO b(id,payload) SELECT gs, 'bb_' || gs::varchar
FROM generate_series(1,20) gs;
CREATE TABLE b_import
( id INTEGER NOT NULL PRIMARY KEY
, payload varchar
);
INSERT INTO b_import(id,payload) SELECT gs, 'bb_' || gs::varchar
FROM generate_series(10,15) gs;
-- In real life this table will be filled by a `COPY b_import FROM ...`
INSERT INTO b_import(id,payload) SELECT gs, 'b2_' || gs::varchar
FROM generate_series(16,25) gs;
CREATE TABLE a
( id SERIAL NOT NULL PRIMARY KEY
, b_id INTEGER references b(id) ON DELETE SET NULL
, aaaaa varchar
);
INSERT INTO a(b_id,aaaaa)
SELECT gs,'aaaaa_' || gs::text FROM generate_series(1,20) gs;
CREATE INDEX ON a(b_id); -- index supporting the FK
-- show it
SELECT a.id, a.aaaaa
,b.id, b.payload AS oldpayload
FROM a
FULL JOIN b ON a.b_id=b.id
ORDER BY a.id;
-- Do the actual I/U/D and report the numbers of affected rows
-- EXPLAIN
WITH ins AS ( -- INSERTS
INSERT INTO b(id, payload)
SELECT b_import.id, b_import.payload
FROM b_import
WHERE NOT EXISTS (
SELECT 1 FROM b
WHERE b.id = b_import.id
)
RETURNING b.id
)
, del AS ( -- DELETES
DELETE FROM b
WHERE NOT EXISTS (
SELECT 2 FROM b_import
WHERE b_import.id = b.id
)
RETURNING b.id
)
, upd AS ( -- UPDATES
UPDATE b
SET payload=b_import.payload
FROM b_import
WHERE b_import.id = b.id
AND b_import.payload IS DISTINCT FROM b.payload -- exclude idempotent updates
-- AND NOT EXISTS ( -- exclude deleted records
-- SELECT 3 FROM del
-- WHERE del.id = b_import.id
-- )
-- AND NOT EXISTS ( -- avoid touching freshly inserted rows
-- SELECT 4 FROM ins
-- WHERE ins.id = b_import.id
-- )
RETURNING b.id
)
SELECT COUNT(*) AS orgb
, (SELECT COUNT(*) FROM b_import) AS newb
, (SELECT COUNT(*) FROM ins) AS ninserted
, (SELECT COUNT(*) FROM del) AS ndeleted
, (SELECT COUNT(*) FROM upd) AS nupdated
FROM b
;
Dropping a constraint and rebuilding it after the import is expensive: all the records in both A and B are involved.
temporally ignoring the constraint is dangerous: the new B table could miss some rows that are still referenced by A's FK.
ergo: You could end up with a crippled model, and you'd have to rebuild As references (which is basically impossible, without additional information (which would be redundant, BTW))