Scenario:
I need to write an SQL script which will remove all records from some or all tables in a database with around 100 tables.
Some tables are 'data' tables, some are 'lookup' tables. There is nothing in their names to indicate which they are.
Sometimes I will want the script to only remove records from the 'data' tables, on other occasions I will want to use it to remove data from all tables.
The records have to be removed from the tables in a very specific order to prevent foreign key constraint violations.
My original idea was to create a variable at the start of the script - something like #EmptyLookupTables - which I could set to true or false, and then I could wrap the DELETE statements in an IF... statement so that they were only executed if the value of the variable was true.
However, due to the foreign key constraints I need to include the GO command after just about every DELETE statement, and variables are not persisted across these batches.
How can I write a script which deletes records from my tables in the correct order but skips over certain tables based on the value of a single variable? The database is in Microsoft SQL Server 2016.
The only way I know of doing this without writing a parser for DDL in TSQL is to turn it on its head.
Create a new database with the same schema; populate the lookup tables, but without the records you don't want. Then populate the data tables, but again leave out the records you don't want. Finally, rename or delete the old database, and rename the new database to the original name.
It's still hard, though.
Create a #temp table and store your variable in it, it will persist across GO separated batches. Then just check the temp table inside every batch.
SELECT #EmptyLookupTables AS EmptyLookupTables INTO #tmp
GO
DECLARE #EmptyLookupTables BIT
SELECT #EmptyLookupTables = EmptyLookupTables FROM #tmp
DELETE FROM YourLookupTable WHERE #EmptyLookupTables = 1
GO
or you can even join directly on #temp table in delete command
DELETE l FROM YourLookupTable l
INNER JOIN #tmp t ON t.EmptyLookupTables = 1
Related
I have a table people with less than 100,000 records and I have taken a backup of this table using the following:
create table people_backup as select * from people
I add some new records to my people table over time, but eventually I want to merge the records from my backup table into people. Unfortunately I cannot simply DROP my table as my new records will be lost!
So I want to update the records in my people table using the records from people_backup, based on their primary key id and I have found 2 ways to do this:
MERGE the tables together
use some sort of fancy correlated update
Great! However, both of these methods use SET and make me specify what columns I want to update. Unfortunately I am lazy and the structure of people may change over time and while my CTAS statement doesn't need to be updated, my update/merge script will need changes, which feels like unnecessary work for me.
Is there a way merge entire rows without having to specify columns? I see here that not specifying columns during an INSERT will direct SQL to insert values by order, can the same methodology be applied here, is this safe?
NB: The structure of the table will not change between backups
Given that your table is small, you could simply
DELETE FROM table t
WHERE EXISTS( SELECT 1
FROM backup b
WHERE t.key = b.key );
INSERT INTO table
SELECT *
FROM backup;
That is slow and not particularly elegant (particularly if most of the data from the backup hasn't changed) but assuming the columns in the two tables match, it does allow you to not list out the columns. Personally, I'd much prefer writing out the column names (presumably those don't change all that often) so that I could do an update.
i am trying to copy table information from a backup dummy database to our live sql database(as an accident happened in our program, Visma Business, where someone managed to overwrite 1300 customer names) but i am having a hard time figuring out the perfect code for this, i've looked around and yes there are several similar problems, but i just can't get this to work even though i've tried different solutions.
Here is the simple code i used last time, in theory all i need is the equivilant of mysqls On Duplicate, which would be MERGE on SQL server? I just didn't quite know what to write to get that merge to work.
INSERT [F0001].[dbo].[Actor]
SELECT * FROM [FDummy].[dbo].[Actor]
The error message i get with this is:
Violation of PRIMARY KEY constraint 'PK__Actor'. Cannot insert duplicate key in object 'dbo.Actor'.
What error message says is simply "You cant add same value if an attribute has PK constraint". If you already have all the information in your backup table what you should do is TRUNCATE TABLE which removes all rows from a table, but the table structure and its columns, constraints, indexes, and so on remain.
After that step you should follow this answer . Or alternatively i recommend a tool called Kettle which is open source and easy to use for these kinds of data movements. That will save you a lot of work.
Here are thing which can be the reason :
You have multiple row in [FDummy].[dbo].[Actor] with same data in a column which is going to be inserted in primary key column of [F0001].[dbo].[Actor].
You have existing rows in [FDummy].[dbo].[Actor] with some value x in primary key column and there is/are row(s) in [F0001].[dbo].[Actor] with same value x in the column which is going to be inserted in primary key column.
List item
-- to check first point. if it returns row then you have some problem
SELECT ColumnGoingToBeMappedWithPK,
Count(*)
FROM [FDummy].[dbo].[Actor]
GROUP BY ColumnGoingToBeMappedWithPK
HAVING Count(*) > 1
-- to check second point. if count is greater than 0 then you have some problem
SELECT Count(*)
FROM [FDummy].[dbo].[Actor] a
JOIN [F0001].[dbo].[Actor] b
ON a.ColumnGoingToBeMappedWithPK = b.PrimaryKeyColumn
The MERGE statement will be possibly the best for you here, unless the primary key of the Actor table is reused after a previous record is deleted, so not autoincremented and say record with id 13 on F0001.dbo.Actor is not the same "actor" information as on FDummy.dbo.Actor
To use the statement with your code, it will look something like this:
begin transaction
merge [F0001].[dbo].[Actor] as t -- the destination
using [FDummy].[dbo].[Actor] as s -- the source
on (t.[PRIMARYKEY] = s.[PRIMARYKEY]) -- update with your primary keys
when matched then
update set t.columnname1 = s.columnname1,
t.columnname2 = s.columnname2,
t.columnname3 = s.columnname3
-- repeat for all your columns that you want to update
output $action,
Inserted.*,
Deleted.*;
rollback transaction -- change to commit after testing
Further reading can be done at the sources below:
MERGE (Transact-SQL)
Inserting, Updating, and Deleting Data by Using MERGE
Using MERGE in SQL Server to insert, update and delete at the same time
So there's this table of just about 40,000 rows I am looking to update. Colleague said it's best to incrementally update the table instead of complete delete and load.
So I've tried hashing out the design and logic of a script to do this, but my inexperience is getting to me. I just don't know what's efficient and unneeded to incrementally update a table.
Currently, the warehouse looks like this: data comes from source into a table (let's call this T1) in Teradata. Then it's sent into another table (let's call this T2) in Teradata with some added fields such as timestamp. Lastly, a view is built on that last table for security reasons.
So with that laid out, I was thinking of creating a temp/volatile table with data from T1. This would have all the data up to the time the script is run with new records. Then, go through the entire table seeing if the ID (primary index) already exists in T2, and if not, add it to another temp table. Then somehow combine the second temp table with T2 and override T2 and build a view on top of that.
Does this make any sense?
There's also the possibility of records being updated. So they would already exist in T2, but have updated data in a new version of T1. I think comparing the values of all the columns from T1 to T2 would be highly inefficient, but can't think of another way to do this
A 40,000 row delete and insert should be pretty painless for any modern database. Ditto for updates.
The real reason for doing and incremental delete/update/insert is so you can log the changes and timestamp rows in the permanent table with the date/time of nsertion and/or last update. The usual technique goes something like this:
remove rows from the permanent table that don't exist in the temp table
update rows that exist in both tables
insert rows that exist in the temp table, but don't exist in the permanent table.
Looking at the Teradata docs, that would be something like this (no warranties about this being syntactically correct, since I don't have a Teradata instance to play with):
delete permanent p
where not exists ( select *
from temp t
where t.id = p.id
)
update p
from permanent p ,
temp t
set ...
where t.id = p.id
insert permanent
select ...
from temp t
where not exists ( select *
from permanent p
where p.id = t.id
)
One might note that the deletes might get a little hairy if there are dependent foreign key constraints involved.
One might also note that on the update, the where clause might get a tad...complicated if you want to check for actual changes to column values: not much point in updating a row if nothing has changed.
There's a Teradata MERGE command that you might find useful, check this post:
https://forums.teradata.com/forum/database/merge-syntax-simple-version
merge into merge_tmp as t using (select 1 as a,'stf' as b,'uuj' as c) as s
on t.a = s.a
when matched then update set c = s.c
when not matched then insert values (s.a,s.b,s.c);
If you need to match on more columns simple put an and in the on statement.
Edit: If you want to use MERGE you might also need to use a delete statement like the one in nicholas' post.
I want to delete some records from a table based on criteria in another table. How do you delete from one of those tables without removing the records in both table?
I am looking to delete a table which are joined with other tables and the query looks something like this.
DELETE DeletingFromTable
FROM DeletingFromTable
INNER JOIN CriteriaTable ON DeletingFromTable.field_id = CriteriaTable.id
WHERE CriteriaTable.criteria = "value" ;
This should work:
DELETE DeleteFromTable FROM DeleteFromTable AS DT
JOIN CriteriaFromTable AS CT ON DT.SomeId = CT.SomeId
WHERE CT.SomeId=[value]
Your question is not 100% clear on what your issue is, but this query will drop tables 1,2 and 3 at the same time:
DROP TABLE table1,table2,table3
You can only delete data from one table at a time.
To delete from multiple table
Write multiple queries separated by semicolon and execute it at onces like
delete from table1;
delete from table2;
delete from table3;
Or you can write the procedure to do this task.
Please check this thread as well
Drop multiple tables in one shot in mysql
You can use:
DELETE FROM TableName
Which will remove all the data, but if you have any seeded columns, these will not be reset. If you want to DELETE data and reset the seeding of PK's, then use TRUNCATE...
TRUNCATE TABLE TableName
But, you need to consider whether you have other tables that have referential integrity, if this is the case, see this post here SQL Server: How to ignore referential integrity until COMMIT?
EDIT:
Your comment above...
delete query like this DELETE FROM table_name WHERE
some_column=some_value;
...suggests you are looking to delete specific rows?
You can just write a query to DROP the tables like so:
DROP TABLE [TABLE_1]
DROP TABLE [TABLE_2]
DROP TABLE [TABLE_3]
Depending on the tables and any constraints you may have between them, you will need to DROP the tables in the correct order.
If you right click any table (depending on SQL version), you should be able to 'View Dependencies'. If the 3 tables you are planning to DROP are only dependant on each other, you need to DROP the tables with no child dependencies first to avoid it failing.
For example, if you try to delete a parent table where it's primary key is referenced in a child table as a foreign key, the DROP will fail because of this. So deleting the child table with the foreign key first will allow you to subsequently DROP the parent table.
If however, the tables have other dependencies outside the tables you are deleting, you will need to remove the dependencies before this will work.
I have two tables in my database, one is Transactions and the other is TransactionHistories. The latter is essentially an auditing table, whereby a trigger executes on insert, update and delete on Transactions to capture a screenshot of the data.
I am successfully retrieving all of the data stored in the Transactions table where the columns match, but the difficulty comes where I am trying to retrieve data from another table using a foreign key. For instance:
The transaction table has a field "TransactionType_TransactionTypeId", but in the audit table we wish to store its 'name' equivalent as "TransactionTypeName". This needs to be populated from the "TransactionTypes" table, which has the fields "TransactionTypeId" and "Name".
I am struggling to write a query to retrieve this as we wish. I am trying something similar to the following but having little success:
SELECT #TransactionTypeName=Name
FROM TransactionTypes
WHERE inserted.TransactionType_TransactionTypeId=TransactionTypes.TransactionTypeId;
I'm assuming that is a syntactic nightmare. If someone could point me in the right direction I would be extremely grateful!
well to get a name you should do the following
select #TransactionTypeName = TT.Name
from inserted as i
left outer join TransactionTypes as TT on TT.TransactionTypeId = i.TransactionType_TransactionTypeId
but you have to know that inserted table can have more than one row, and you are getting value for only one row.