In this example, I have two tables in my DB. Table A receives data via SSIS from an external DB once a day. This information then needs to update records on Table B. The tables are joined by a two part primary key.
Table A
Mref
Rref
HD_Status
SL_Status
JB_Status
Table B
MrefIn
RrefIn
HD Status
SL Status
JB Status
The keys link by Mref = mrefIn & Rref = RrefIn.
The update script I've written is
UPDATE dbo.Table A
set HD_Status = dbo.Table B.HD Status,
SL_Status = dbo.Table B.SL Status,
JB_Status = dbo.Table B.JB Status
from dbo.Table B
where dbo.Table A.Mref = dbo.Table B.MrefIn AND
dbo.Table A.Rref = dbo.Table B.RrefIn
My question is: what is the best way to run this script, once a morning at 6am? I'm thinking SSIS but not exactly sure if this is the best practice. There should be no more than 1000 records to update.
Any replies are greatly appreciated!
Steve
A scheduled Stored Procedure was the way to go.
Thanks.
Related
I have a SQL Server table with student, cohort, program, grad date, and status. Some students have records that are specifically tied together – if the cohort for one row changes, the cohort for another row with status 2 needs to change as well. In most cases I catch this programmatically but there are a LOT of ways to change a student’s cohort so we’re missing some.
Essentially if someone runs the first line update statement, the second also needs to run
UPDATE StudentPrograms
SET Cohort = 201610
WHERE Student = 'A1234' AND Program = 'MBA' AND Cohort = 201510
UPDATE StudentPrograms
SET Cohort = 201610
WHERE Student = 'A1234' AND Cohort = 201510 AND Status = 2
I would like to put an update trigger on the table to change both cohorts, but the problem is I can’t figure out how to capture both the starting cohort that identifies the record that needs to change and the cohort that it would need to change to. Once the update has run and the trigger fires, all I have is the updated cohort, so I don’t know which record to change.
CREATE TRIGGER [dbo].[tr_SP_Cohort]
ON [dbo].StudentPrograms
AFTER UPDATE
AS
DECLARE #student VARCHAR(10);
DECLARE #cohort INT;
SELECT #student = i.student FROM inserted i
SELECT #cohort = i.cohort FROM inserted i
SET NOCOUNT ON;
IF UPDATE(Cohort)
UPDATE dbo.StudentPrograms
SET Cohort = #cohort
WHERE Student = #student AND Status = 2 AND Cohort = ???
Unfortunately the records do have to be in this table and not a sub table for a lot of reasons. The records are occasionally also deleted / reinserted, so I run into the same problems if I tie them together by adding another field with a key - if that record gets deleted / updated / reinserted, I still need to capture the information, record it, then record the change.
Thanks for the immediate help! Here's what I ended up with, changed a bit so it works with batch updates.
ALTER TRIGGER [dbo].[tr_SP_Cohort]
ON [dbo].StudentPrograms
AFTER UPDATE
AS
SET NOCOUNT ON;
IF UPDATE(Cohort)
UPDATE a
SET Cohort = i.Cohort
FROM dbo.StudentPrograms AS a
INNER JOIN inserted AS i ON a.Student = i.Student
INNER JOIN deleted AS d ON a.Cohort = d.Cohort
AND a.Student = d.Student
WHERE a.Status = 2
When creating a trigger, you can access the old values using the deleted table
WHERE Student = #student AND Status = 2 AND Cohort = (SELECT TOP 1 Cohort from deleted)
Ideally, you should handle this in a stored procedure but as a temporary fix you can use the following code.
They way you are handling updates in your code, it will never work if more than one row was updated.
Triggers are fired once in response to a triggering action (update,delete,insert) for all the rows affected by that action, NOT for each row affected by the triggering action.
Keeping all this in mind your temporary solution should look something like...
CREATE TRIGGER [dbo].[tr_SP_Cohort]
ON [dbo].StudentPrograms
AFTER UPDATE
AS
BEGIN
SET NOCOUNT ON;
UPDATE s
SET s.Cohort = i.Cohort
FROM deleted d
INNER JOIN dbo.StudentPrograms s ON d.Student = s.Student
AND d.Cohort = s.Cohort
INNER JOIN inserted i ON i.Student = s.Student
WHERE s.[Status] = 2;
END
I have a table with 10 millions rows that I need to join with another table and update all data. This is taking more than 1 one hour and it is increasing my transaction log in 10+ GBs. Is there another way to enhance this performance?
I believe that after each update, the indexes and constraints are checked and all information are logged. Is there a way to tell SQL Server to check constraints only after the update is finished and to minimally log the update action?
My query follows below. I've modified some names so it becomes easier to read.
UPDATE o
SET o.Info1 = u.Info1, o.Info2 = u.Info2, o.Info3 = u.Info3
FROM Orders o
INNER JOIN Users u
ON u.ID = o.User_ID
EDIT: as asked in comments, the table definition would be something like the following (simplifying again to create a generic question).
Table Orders
ID int PK
OrderNumber nvarchar(20)
User_ID int FK to table Users
Info1 int FK to table T1
Info2 int FK to table T2
Info2 int FK to table T3
Table Users
ID int PK
UserName nvarchar(20)
Info1 int FK to table T1
Info2 int FK to table T2
Info2 int FK to table T3
First of all there is no such thing as BULK UPDATE, a few things that you can do are as follow:
If possible put your database in simple recovery mode before doing this operation.
Drop indexes before doing update and create them again once update is completed.
do updates in smaller batches , something like
WHILE (1=1)
BEGIN
-- update 10,000 rows at a time
UPDATE TOP (10000) O
FROM Table O inner join ... bla bla
IF (##ROWCOUNT = 0)
BREAK;
END
Note
if you go with the simple mode option, dont forget to take a full-backup after you switch the recovery mode back to full. Since simply switching it back to full recovery mode will not strat logging until you take a full backup.
For my case, load data as need(Dotnet winform), and try create a new table, apply bulk and update the basic table with join by bulk table!, for 1M rows , it take me about 5 second
struggling with this one, quite a lengthy description so ill explain best I can:
I have a table with 12 columns in, 1 being a primary key with identity_insert, 1 a foreign key , the other 10 being cost values, ive created a statement to group them into 5 categories, shown below:
select
(ProductID)ProjectID,
sum(Cost1)Catagory1,
sum(Cost2)Catagory2,
sum(Cost3 + Cost4 + Cost5 + Cost6 + Cost7) Catagory3,
sum(Cost 8 + Cost 9)Catagory4,
sum(Cost10)Catagory5
from ProductTable group by ProductID
ive changed the names of the data to make it more generic, they aren't actually called Cost1 etc by the way ;)
the foreign key can appear multiple times (ProductID) so in the above query the related fields are calculated together based upon this... Now what ive been trying to do is put this query into a table, which i have done successfully, and then update the data via a procedure. the problem im having is that all the data in the table is overwritten by row 1 and when theres is thousands of rows this is a problem.
I have also tried putting the above query into a view and the same result... any suggestions would be great :)
update NewTable set
ProductID = (ProductView.ProductID ),
Catagory1 = (ProductView.Catagory1 ),
Catagory2 = (ProductView.Catagory2 ),
Catagory3 = (ProductView.Catagory3 ),
Catagory4 = (ProductView.Catagory4 ),
Catagory5 = (ProductView.Catagory5 )
from ProductView
I need something along the lines like above.... but one that doesn't overwrite everything with row 1 haha ;)
ANSWERED BY: Noman_1
create procedure NewProducts
insert into NewTable
select ProductID.ProductTable,
Catagory1.ProductView,
Catagory2.ProductView,
Catagory3.ProductView,
Catagory4.ProductView,
Catagory5.ProductView
from ProductView
inner join ProductTable on ProductView.ProductID = ProductTable.ProductID
where not exists(select 1 from NewTable where ProductView.ProductID = NewTable.ProductID)
above procedure locates the new Product that has been created within a view, the procedure query detects that there is a Product that is not located in the NewTable and inserts it via the procedure
As far as i know, and since you want to update all the products in the table, and each product uses all the sums of the product itself from origin, you actually need to update each row 1 by 1, and as consecuence when you do an update like the next, its your only main way
update newtable
set category1 = (select sum(cost1) from productTable where productTable.productId = newtable.ProductID),
category2 = (select sum(cost2) from productTable where productTable.productId = newtable.ProductID),
etc..
Keep in mind that if you have new products, they wont get inserted with the update, you would need like this in order to add them:
Insert into newtable
Select VALUES from productTable a where productId not exists(select 1 from newTable b where a.ProductId = b.ProductId);
A second way, and since you want allways to update all the data, is to simply truncate and do a insert select right after.
Maybe on an Oracle, you would be albe to use a MERGE but im unaware if it would really improve anything.
I asume that simply having a view would not work due the amount of data you state you have.
EDIT, I never knew that the MERGE STATMENT is actually avaiable on SQL Server 2008 and above, with this single statment you could do an UPDATE/INSERT on all but it's efficiency is unknown to me, you may want to test it with your high amount of data:
MERGE newtable AS TARGET
USING select ProductId, sum(cost1) cat1, sum(cost2) cat2 ...
FROM productTable Group by ProductId AS SOURCE
ON TARGET.ProductId = SOURCE.ProductID
WHEN MATCHED
THEN UPDATE SET TARGET.category1 = cat1, TARGET.category2 = cat2...
WHEN NOT MATCHED
THEN INSERT (ProductId, category1, category2,...)
VALUES (SOURCe.ProductId, SOURCE.cat1, SOURCE.cat2...);
More info about merge here:
http://msdn.microsoft.com/library/bb510625.aspx
The example at the end may give you a good overview of the sintax
You haven't given any join condition. SQL Server cannot know that you meant to update rows matched by productid.
update NewTable set
ProductID = (ProductView.ProductID ),
Catagory1 = (ProductView.Catagory1 ),
Catagory2 = (ProductView.Catagory2 ),
Catagory3 = (ProductView.Catagory3 ),
Catagory4 = (ProductView.Catagory4 ),
Catagory5 = (ProductView.Catagory5 )
from NewTable
join ProductView pv on NewTable.productid = pv.productid
You don't need a view. Just past the view query to the place where I said ProductView. Of course, you can use a view.
I'm trying to create a stored procedure that will update a record on one table depending on certain values in a related table:
MainTable contains a status field and is related to SubTable which also has a status field. I need to update the status of each record in MainTable where ALL of the related records in SubTable has the same status x. I've tried several queries but think I am going about it the wrong way. Any assistance would greatly be appreciated. Thanks.
for example having following tables :
Producer ( Code,Name,Status)
Goods ( GCode,PCode( Producer Code),Name,Status)
and Query :
Update Producer set Status=F
where not exists (select * from Goods where Status <> X and Goods.PCode= Producer.Code)
Try this :
Update schema.yourtable s set s.fieldtoupdate = (
Select e.Relationalfield from schema.RealtionalTable e where e.STATUSFIELD = s.STATUSFIELD);
Hope it helps
I have imported some data into a new sql table and have managed to update the imported data into the correct table but its unable to update for records without a customer number, i have tried updating the customer number but it says it cant update a primary key record.
This is the code i used to update the fields so far...
update SalBudgetCust
Set
SalBudgetCust.Sales1 = SalBudgetCust_temp.Sales1,
SalBudgetCust.Sales2 = SalBudgetCust_temp.Sales2,
SalBudgetCust.Sales3 = SalBudgetCust_temp.Sales3,
SalBudgetCust.Sales4 = SalBudgetCust_temp.Sales4,
SalBudgetCust.Sales5 = SalBudgetCust_temp.Sales5,
SalBudgetCust.Sales6 = SalBudgetCust_temp.Sales6,
SalBudgetCust.Sales7 = SalBudgetCust_temp.Sales7,
SalBudgetCust.Sales8 = SalBudgetCust_temp.Sales8,
SalBudgetCust.Sales9 = SalBudgetCust_temp.Sales9,
SalBudgetCust.Sales10 = SalBudgetCust_temp.Sales10,
SalBudgetCust.Sales11 = SalBudgetCust_temp.Sales11,
SalBudgetCust.Sales12 = SalBudgetCust_temp.Sales12
From
SalBudgetCust_temp where SalBudgetCust.Customer = SalBudgetCust_temp.Customer
Would anyone be able to offer some light on how if the customer record doesnt exist already it populates from the other table and then includes the corresponding sales figures?
Try SELECT INTO statement as below for inserting records where customer doesnt exist:
SELECT * INTO SalBudgetCust FROM SalBudgetCust_temp WHERE NOT EXISTS (SELECT Customer FROM SalBudgetCust WHERE Customer = SalBudgetCust_temp.Customer)
This SO Answer might also help you to the solution:
How to avoid duplicates in INSERT INTO SELECT query in SQL Server?