I get this error with Monetdb when I try to load .tbl data in tables where there are primary key and foreign key, what's wrong?
This is the command:
COPY INTO monet.CUSTOMER FROM '/home/nicola/Scrivania/ssb-dbgen-master/1gb/customer.tbl' USING DELIMITERS '|', '|\n' LOCKED;
It is always good to bulk-load into tables with (foreign-) keys disabled. You can add them after the load with the ALTER statement.
see https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/LoadingBulkData
Another part of MonetDB's documentation says: "WARNING It is advised to add integrity constraints to the table after the file has been loaded. The ALTER statements perform bulk integrity checking and perform these checks often more efficiently." https://www.monetdb.org/Documentation/Manuals/SQLreference/CopyInto
Generally, for bulk loading into an existing table, is is advised to drop the indexes/foreign keys/other constraints, load the table, and then recreate the indexes/foreign keys/other constraints.
Related
So I need to do multiple bulk inserts into a table with row level triggers. I thought it would be a good idea to gather the generated ids first, combine them with my data and then do a direct=true sql load. Normally this would work fine but the table is partitioned by reference so it cannot disable the foreign key constraint that would allow me to do the direct load.
Does anyone know of anyway around this? My first solution of bulk collecting into a varray and inserting every 100,000 went moderately fast but if I was able to do a direct load, that would be much faster.
ERROR: SQL*Loader-965: Error -1 disabling constraint client_fk on
table my_table
The manual implies there is no way to have SQL*Loader use a direct path load but not disable the foreign keys.
But direct-path inserts can work on reference partitioned tables, even with the foreign keys enabled, as I demonstrated in this question and answer.
Convert the process from SQL*Loader to an external table INSERT statement. SQL*Loader and external tables use similar mechanisms so the conversion shouldn't be too difficult. External tables require a little more work - you have to write the INSERT with an append hint, and manually disable and re-enable triggers and perhaps other objects. But that extra control allows loading data quickly with direct-path inserts.
I have two tables in two different databases. In table1 (in database1) there is a column called column1 and it is a primary key. Now in table2 (in database2) there is a column called column2 and I want to add it as a foreign key.
I tried to add it and it gave me the following error:
Msg 1763, Level 16, State 0, Line 1
Cross-database foreign key references are not supported. Foreign key Database2.table2.
Msg 1750, Level 16, State 0, Line 1
Could not create constraint. See previous errors.
How do I do that since the tables are in different databases.
You would need to manage the referential constraint across databases using a Trigger.
Basically you create an insert, update trigger to verify the existence of the Key in the Primary key table. If the key does not exist then revert the insert or update and then handle the exception.
Example:
Create Trigger dbo.MyTableTrigger ON dbo.MyTable, After Insert, Update
As
Begin
If NOT Exists(select PK from OtherDB.dbo.TableName where PK in (Select FK from inserted) BEGIN
-- Handle the Referential Error Here
END
END
Edited: Just to clarify. This is not the best approach with enforcing referential integrity. Ideally you would want both tables in the same db but if that is not possible. Then the above is a potential work around for you.
If you need rock solid integrity, have both tables in one database, and use an FK constraint. If your parent table is in another database, nothing prevents anyone from restoring that parent database from an old backup, and then you have orphans.
This is why FK between databases is not supported.
You could use check constraint with a user defined function to make the check. It is more reliable than a trigger. It can be disabled and reenabled when necessary same as foreign keys and rechecked after a database2 restore.
CREATE FUNCTION dbo.fn_db2_schema2_tb_A
(#column1 INT)
RETURNS BIT
AS
BEGIN
DECLARE #exists bit = 0
IF EXISTS (
SELECT TOP 1 1 FROM DB2.SCHEMA2.tb_A
WHERE COLUMN_KEY_1 = #COLUMN1
) BEGIN
SET #exists = 1
END;
RETURN #exists
END
GO
ALTER TABLE db1.schema1.tb_S
ADD CONSTRAINT CHK_S_key_col1_in_db2_schema2_tb_A
CHECK(dbo.fn_db2_schema2_tb_A(key_col1) = 1)
In my experience, the best way to handle this when the primary authoritative source of information for two tables which are related has to be in two separate databases is to sync a copy of the table from the primary location to the secondary location (using T-SQL or SSIS with appropriate error checking - you cannot truncate and repopulate a table while it has a foreign key reference, so there are a few ways to skin the cat on the table updating).
Then add a traditional FK relationship in the second location to the table which is effectively a read-only copy.
You can use a trigger or scheduled job in the primary location to keep the copy updated.
The short answer is that SQL Server (as of SQL 2008) does not support cross database foreign keys--as the error message states.
While you cannot have declarative referential integrity (the FK), you can reach the same goal using triggers. It's a bit less reliable, because the logic you write may have bugs, but it will get you there just the same.
See the SQL docs # http://msdn.microsoft.com/en-us/library/aa258254%28v=sql.80%29.aspx Which state:
Triggers are often used for enforcing
business rules and data integrity. SQL
Server provides declarative
referential integrity (DRI) through
the table creation statements (ALTER
TABLE and CREATE TABLE); however, DRI
does not provide cross-database
referential integrity. To enforce
referential integrity (rules about the
relationships between the primary and
foreign keys of tables), use primary
and foreign key constraints (the
PRIMARY KEY and FOREIGN KEY keywords
of ALTER TABLE and CREATE TABLE). If
constraints exist on the trigger
table, they are checked after the
INSTEAD OF trigger execution and prior
to the AFTER trigger execution. If the
constraints are violated, the INSTEAD
OF trigger actions are rolled back and
the AFTER trigger is not executed
(fired).
There is also an OK discussion over at SQLTeam - http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=31135
Achieving referential integrity accross databases is not an easy task.
Here is a list of frequently employed mechanisms:
Clone & Sync: The referenced data is regularly cloned/merged into the referencing database. This may be suitable if the referenced data is rarely changing. You end up with two physical copies of the same data, and need a reliable process to keep them in sync (e.g. with an ETL pipeline).
Triggers: Changes to the referencing data and the referenced data are caught by SQL triggers, which ensure referential integrity. However, triggers can be slow, and may not fire at a database restore. It cannot hurt to run scheduled consistency checks as part of the operations monitoring. Write access to the referenced database is required for installing and maintaining the trigger.
Check constraints: SQL-Server offers user-defined contraints, which ensure that every row satisfies a given condition. One can exploit this functionality by writing a user defined function that checks the existence of a row in the referenced data, and then use this function as a CHECK's predicate in the referencing table. This does not catch changes in the referenced data. It is an RDBMS-specific solution, but works accross server boundaries (e.g. using linked servers). It is a good choice for referencing globally unique IDs, such as article codes in a company's ERP system, which never get deleted or re-assigned.
Re-think database architecture: When all the above mechanisms are unsatisfactory, multiple databases may be merged in a single database. The originating database names can become schema names, allowing effective grouping of database objects.
As the error message says, this is not supported on sql server.
The only way to ensure refrerential integrity is to work with triggers.
I want to truncate all tables under a specific schema in DB2 which is worked on a Linux Server. But I have no right to ALTER TABLE to disable the foreign-key constraints.
Is there anyway to do this?
I'm considering performing a topology sort based on the constraints between tables, but it is a little bit complex.
Any good idea on this problem?
You don't say what platform you're on. This answer is specific to DB2 on Linux, UNIX and Windows.
If you have LOAD, INSERT and DELETE privileges on the table(s) you can use the LOAD command with an empty file to truncate the tables, regardless of whether there are foreign key constraints:
LOAD from /dev/null of del replace into yourschema.yourtable nonrecoverable
This will place any dependent tables in check pending state… Once you have truncated all of your tables you would use the SET INTEGRITY statement to take all of the tables out of check pending.
I have two tables in two different databases. In table1 (in database1) there is a column called column1 and it is a primary key. Now in table2 (in database2) there is a column called column2 and I want to add it as a foreign key.
I tried to add it and it gave me the following error:
Msg 1763, Level 16, State 0, Line 1
Cross-database foreign key references are not supported. Foreign key Database2.table2.
Msg 1750, Level 16, State 0, Line 1
Could not create constraint. See previous errors.
How do I do that since the tables are in different databases.
You would need to manage the referential constraint across databases using a Trigger.
Basically you create an insert, update trigger to verify the existence of the Key in the Primary key table. If the key does not exist then revert the insert or update and then handle the exception.
Example:
Create Trigger dbo.MyTableTrigger ON dbo.MyTable, After Insert, Update
As
Begin
If NOT Exists(select PK from OtherDB.dbo.TableName where PK in (Select FK from inserted) BEGIN
-- Handle the Referential Error Here
END
END
Edited: Just to clarify. This is not the best approach with enforcing referential integrity. Ideally you would want both tables in the same db but if that is not possible. Then the above is a potential work around for you.
If you need rock solid integrity, have both tables in one database, and use an FK constraint. If your parent table is in another database, nothing prevents anyone from restoring that parent database from an old backup, and then you have orphans.
This is why FK between databases is not supported.
You could use check constraint with a user defined function to make the check. It is more reliable than a trigger. It can be disabled and reenabled when necessary same as foreign keys and rechecked after a database2 restore.
CREATE FUNCTION dbo.fn_db2_schema2_tb_A
(#column1 INT)
RETURNS BIT
AS
BEGIN
DECLARE #exists bit = 0
IF EXISTS (
SELECT TOP 1 1 FROM DB2.SCHEMA2.tb_A
WHERE COLUMN_KEY_1 = #COLUMN1
) BEGIN
SET #exists = 1
END;
RETURN #exists
END
GO
ALTER TABLE db1.schema1.tb_S
ADD CONSTRAINT CHK_S_key_col1_in_db2_schema2_tb_A
CHECK(dbo.fn_db2_schema2_tb_A(key_col1) = 1)
In my experience, the best way to handle this when the primary authoritative source of information for two tables which are related has to be in two separate databases is to sync a copy of the table from the primary location to the secondary location (using T-SQL or SSIS with appropriate error checking - you cannot truncate and repopulate a table while it has a foreign key reference, so there are a few ways to skin the cat on the table updating).
Then add a traditional FK relationship in the second location to the table which is effectively a read-only copy.
You can use a trigger or scheduled job in the primary location to keep the copy updated.
The short answer is that SQL Server (as of SQL 2008) does not support cross database foreign keys--as the error message states.
While you cannot have declarative referential integrity (the FK), you can reach the same goal using triggers. It's a bit less reliable, because the logic you write may have bugs, but it will get you there just the same.
See the SQL docs # http://msdn.microsoft.com/en-us/library/aa258254%28v=sql.80%29.aspx Which state:
Triggers are often used for enforcing
business rules and data integrity. SQL
Server provides declarative
referential integrity (DRI) through
the table creation statements (ALTER
TABLE and CREATE TABLE); however, DRI
does not provide cross-database
referential integrity. To enforce
referential integrity (rules about the
relationships between the primary and
foreign keys of tables), use primary
and foreign key constraints (the
PRIMARY KEY and FOREIGN KEY keywords
of ALTER TABLE and CREATE TABLE). If
constraints exist on the trigger
table, they are checked after the
INSTEAD OF trigger execution and prior
to the AFTER trigger execution. If the
constraints are violated, the INSTEAD
OF trigger actions are rolled back and
the AFTER trigger is not executed
(fired).
There is also an OK discussion over at SQLTeam - http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=31135
Achieving referential integrity accross databases is not an easy task.
Here is a list of frequently employed mechanisms:
Clone & Sync: The referenced data is regularly cloned/merged into the referencing database. This may be suitable if the referenced data is rarely changing. You end up with two physical copies of the same data, and need a reliable process to keep them in sync (e.g. with an ETL pipeline).
Triggers: Changes to the referencing data and the referenced data are caught by SQL triggers, which ensure referential integrity. However, triggers can be slow, and may not fire at a database restore. It cannot hurt to run scheduled consistency checks as part of the operations monitoring. Write access to the referenced database is required for installing and maintaining the trigger.
Check constraints: SQL-Server offers user-defined contraints, which ensure that every row satisfies a given condition. One can exploit this functionality by writing a user defined function that checks the existence of a row in the referenced data, and then use this function as a CHECK's predicate in the referencing table. This does not catch changes in the referenced data. It is an RDBMS-specific solution, but works accross server boundaries (e.g. using linked servers). It is a good choice for referencing globally unique IDs, such as article codes in a company's ERP system, which never get deleted or re-assigned.
Re-think database architecture: When all the above mechanisms are unsatisfactory, multiple databases may be merged in a single database. The originating database names can become schema names, allowing effective grouping of database objects.
As the error message says, this is not supported on sql server.
The only way to ensure refrerential integrity is to work with triggers.
Is there a way to truncate tables with constraints ?
I tried to DEACTIvATE with this:
DECLARE #SQLtxt varchar(max)
SET #SQLtxt = '-- DESACTIVER LES CONTRAINTES' + CHAR(10)
SELECT #SQLtxt = #SQLtxt + 'ALTER TABLE [' + name + '] NOCHECK CONSTRAINT ALL;' FROM sys.tables
PRINT #SQLtxt
Of course, it didn't worked. I have to drop the constraint then recreate them ! The only way I could make it work is by extracting the script to drop and recreate the contraint.
Is there another way ? BTW I don't want to delete because it would use the Transaction Log.
Here is a script that may help you get going scripting out FK's. Script out your foreign keys.
I use a modified version to dump the constraint definitions into a temp table, then do the TRUNCATE magic and then recreate the constraints from the temp table. However, this is only for my own convenience when restoring the production database onto a non-production environment to get rid of most of the data. Not sure, I would use it in a production scenario though. I would rather prefer deleting in small batches knowing that everything is fully logged.
Btw, womb's reference to the SQL Server 2000 Books Online is a bit misleading. TRUNCATE TABLE has always been a minimally logged operation.
TRUNCATE TABLE removes the data by deallocating the data pages used to store the table's data, and only the page deallocations are recorded in the transaction log.
This is has specified more precisely in later versions of Books Online.
The TRUNCATE command will not work on tables with FK references. Straight from the documentation:
You cannot use TRUNCATE TABLE on a
table referenced by a FOREIGN KEY
constraint; instead, use DELETE
statement without a WHERE clause.
Because TRUNCATE TABLE is not logged,
it cannot activate a trigger.
You sort of answered the question yourself - if there's a foreign key referencing your table, SQL Server needs that information in the transaction log. Since TRUNCATE TABLE effectively bypasses the log, it's not allowed on tables referenced by foreign keys.
You'll either have to DROP the foreign key constraint, or use a DELETE statement.
It's only referencing (eg the REFERENCES bit) foreign keys you need to drop.
This should make it easier...