I am new to SQL Server. I have a Batch process that loads data into my stage tables. I have some foreign Keys on that table. I want to dump all the foreign key errors encountered while loading into a error table. How do I do that?
Thanks
New Novice
Use SSIS to load the data. Records which fail validation can be sent off to a an exception table.
One approach would be to load the data into a temporary table which has no FK constraints, remove the bad records (that violate the FK constraints), then move the data from the temp table into the stage table. If you have lots of FKs on your table this will probably be a bit tedious, so you would probably want to automate the process.
Here's some pseudo-code to show what I mean...
-- First put the raw data into MyTempTable
-- Find the records that are "bad" -- you can SELECT INTO a "bad records" table
-- for later inspection if you want...
SELECT *
INTO #BadRecords
FROM MyTempTable
WHERE ForeignKeyIDColumn NOT IN
(
SELECT ID FROM ForeignKeyTable
)
-- Remove the bad records now
DELETE
FROM MyTempTable
WHERE ForeignKeyIDColumn NOT IN
(
SELECT ID FROM ForeignKeyTable
)
-- Now the data is "clean" (won't violate the FK) so you can insert it
-- from MyTempTable into the stage table
Related
I want to populate 5000 records in the below format to a particular table.
Insert into #Table
(c1,c2,c3,c4,c5)
Values
(1,2,3,4,5),
(2,2,3,4,5),
(3,2,3,4,5),
(4,2,3,4,5),
(5,2,3,4,5)
....
....
Up to 1000 rows
When I try to execute it. I got a foreign Key violation. I know the reason since one of the value did not exist in its corresponding parent table.
There are few records causing this violation. It's very hard to find those violated rows among the 1000 rows so I want to insert at least the valid records to my target table leaving the violated rows as it is for now.
I am not sure how to perform this. Please suggest me any ideas to do this.
If this is a one time thing, then you can do the following:
Drop the FK constraint
ALTER TABLE MyTAble
DROP CONSTRAINT FK_Contstraint
GO
Execute INSERT
Find the records with no matching parent id.
SELECT * FROM MyTable MT WHERE NOT EXISTS (SELECT 1 FROM ParentTable PT WHERE MT.ParentId = PT.ID)
DELETE those records or do something else with them.
Recreate the FK constraint.
Disable the foreign key or fix your data.
Finding the bad data is simple - you can always temporarily insert it into a buffer table and run queries to find which data is missing in the related table.
I am planning for an incremental load into warehouse (especially for updates of source tables in RDBMS).
Capturing the updated rows in staging tables from RDBMS based the updates datetime. But how do I determine which column of a particular row needs to be updated in the target warehouse tables?
Or do I just delete a particular row in the warehouse table (based on the primary key of the row in staging table) and insert the new updated row?
Which is the best way to implement the incremental load between the RDBMS and Warehouse using PL/SQL and SQL coding?
In my opinion, the easiest way to accomplish this is as follows:
Create a stage table identical to your host table. When you do your incremental/net-change load, load all changed records into this table (based on whatever your "last updated" field is)
Delete the records from your actual table based on the primary key. For example, if your primary key is customer, part, the query might look like this:
delete from main_table m
where exists (
select null
from stage_table s
where
m.customer = s.customer and
m.part = s.part
);
Insert the records from the stage to the main table.
You could also do an update existing records / insert new records, but either way that's two steps. The advantage of the method I listed is that it will work even if your tables have partitions and the newly updated data violates one of the original partition rules, whereas an update would not accomplish that. Also, the syntax is much simpler as your update would have to list every single field, whereas the delete from / insert into allows you list only the primary key fields.
Oracle also has a merge clause that will update if it exists or insert if it does not. I honestly don't know how that would be impacted if you had partitions.
One major caveat. If your updates include deletes -- records that need to be deleted from the main table, none of these will resolve that and you will need some other way to handle that. It may not be necessary, depending on your circumstances, but it's something to consider.
i am trying to copy table information from a backup dummy database to our live sql database(as an accident happened in our program, Visma Business, where someone managed to overwrite 1300 customer names) but i am having a hard time figuring out the perfect code for this, i've looked around and yes there are several similar problems, but i just can't get this to work even though i've tried different solutions.
Here is the simple code i used last time, in theory all i need is the equivilant of mysqls On Duplicate, which would be MERGE on SQL server? I just didn't quite know what to write to get that merge to work.
INSERT [F0001].[dbo].[Actor]
SELECT * FROM [FDummy].[dbo].[Actor]
The error message i get with this is:
Violation of PRIMARY KEY constraint 'PK__Actor'. Cannot insert duplicate key in object 'dbo.Actor'.
What error message says is simply "You cant add same value if an attribute has PK constraint". If you already have all the information in your backup table what you should do is TRUNCATE TABLE which removes all rows from a table, but the table structure and its columns, constraints, indexes, and so on remain.
After that step you should follow this answer . Or alternatively i recommend a tool called Kettle which is open source and easy to use for these kinds of data movements. That will save you a lot of work.
Here are thing which can be the reason :
You have multiple row in [FDummy].[dbo].[Actor] with same data in a column which is going to be inserted in primary key column of [F0001].[dbo].[Actor].
You have existing rows in [FDummy].[dbo].[Actor] with some value x in primary key column and there is/are row(s) in [F0001].[dbo].[Actor] with same value x in the column which is going to be inserted in primary key column.
List item
-- to check first point. if it returns row then you have some problem
SELECT ColumnGoingToBeMappedWithPK,
Count(*)
FROM [FDummy].[dbo].[Actor]
GROUP BY ColumnGoingToBeMappedWithPK
HAVING Count(*) > 1
-- to check second point. if count is greater than 0 then you have some problem
SELECT Count(*)
FROM [FDummy].[dbo].[Actor] a
JOIN [F0001].[dbo].[Actor] b
ON a.ColumnGoingToBeMappedWithPK = b.PrimaryKeyColumn
The MERGE statement will be possibly the best for you here, unless the primary key of the Actor table is reused after a previous record is deleted, so not autoincremented and say record with id 13 on F0001.dbo.Actor is not the same "actor" information as on FDummy.dbo.Actor
To use the statement with your code, it will look something like this:
begin transaction
merge [F0001].[dbo].[Actor] as t -- the destination
using [FDummy].[dbo].[Actor] as s -- the source
on (t.[PRIMARYKEY] = s.[PRIMARYKEY]) -- update with your primary keys
when matched then
update set t.columnname1 = s.columnname1,
t.columnname2 = s.columnname2,
t.columnname3 = s.columnname3
-- repeat for all your columns that you want to update
output $action,
Inserted.*,
Deleted.*;
rollback transaction -- change to commit after testing
Further reading can be done at the sources below:
MERGE (Transact-SQL)
Inserting, Updating, and Deleting Data by Using MERGE
Using MERGE in SQL Server to insert, update and delete at the same time
Query:
Delete from test_table; (i am using this delete command in Procedure to perform daily)
This table contains 2 column name like scenario_id, item_id, these 2 column are composite primary key. So Each scenario_id will have 2 millions item_id, How to delete this table quicky.
The fastest way to delete the table might be DROP TABLE test_table command, and recreate the table using CREATE TABLE... command. DROP TABLE... will drop the table immediately. Well, actually it will move the table into recyclebin. You should PURGE RECYCLEBIN if want to completely remove the table.
Other way to delete the data in the table is to use TRUNCATE TABLE.... This is a little slower than DROP TABLE..., however, much faster than DELETE FROM.... Since there's no way to rollback the data after you truncate the table, you should be careful when you use TRUNCATE TABLE.
Check if there are any Foreign keys pointing onto this table. If they are, there MUST be indexes on these referring columns. Otherwise maintenance of the referential integrity will take ages
(Transactional) Delete on Oracle is slow by nature. Simple because of deleted data must be copied into UNDO tablespace, REDO logs and also archived REDO logs. This is the way how Oracle protects your data.
If our are deleting more than 50% of data, it is much faster when you simply create new table as select * from old_table where ... and then drop the new old one and rename new to old.
You can achieve similar goal by exchanging partitions in the partitioned table.
Or you can simply drop table's partitions if your table is partitioned wisely
I want to delete some records from a table based on criteria in another table. How do you delete from one of those tables without removing the records in both table?
I am looking to delete a table which are joined with other tables and the query looks something like this.
DELETE DeletingFromTable
FROM DeletingFromTable
INNER JOIN CriteriaTable ON DeletingFromTable.field_id = CriteriaTable.id
WHERE CriteriaTable.criteria = "value" ;
This should work:
DELETE DeleteFromTable FROM DeleteFromTable AS DT
JOIN CriteriaFromTable AS CT ON DT.SomeId = CT.SomeId
WHERE CT.SomeId=[value]
Your question is not 100% clear on what your issue is, but this query will drop tables 1,2 and 3 at the same time:
DROP TABLE table1,table2,table3
You can only delete data from one table at a time.
To delete from multiple table
Write multiple queries separated by semicolon and execute it at onces like
delete from table1;
delete from table2;
delete from table3;
Or you can write the procedure to do this task.
Please check this thread as well
Drop multiple tables in one shot in mysql
You can use:
DELETE FROM TableName
Which will remove all the data, but if you have any seeded columns, these will not be reset. If you want to DELETE data and reset the seeding of PK's, then use TRUNCATE...
TRUNCATE TABLE TableName
But, you need to consider whether you have other tables that have referential integrity, if this is the case, see this post here SQL Server: How to ignore referential integrity until COMMIT?
EDIT:
Your comment above...
delete query like this DELETE FROM table_name WHERE
some_column=some_value;
...suggests you are looking to delete specific rows?
You can just write a query to DROP the tables like so:
DROP TABLE [TABLE_1]
DROP TABLE [TABLE_2]
DROP TABLE [TABLE_3]
Depending on the tables and any constraints you may have between them, you will need to DROP the tables in the correct order.
If you right click any table (depending on SQL version), you should be able to 'View Dependencies'. If the 3 tables you are planning to DROP are only dependant on each other, you need to DROP the tables with no child dependencies first to avoid it failing.
For example, if you try to delete a parent table where it's primary key is referenced in a child table as a foreign key, the DROP will fail because of this. So deleting the child table with the foreign key first will allow you to subsequently DROP the parent table.
If however, the tables have other dependencies outside the tables you are deleting, you will need to remove the dependencies before this will work.