I want a basic update procedure that updates a temporary table and orders it by PrimID and myDates, and then updates a permanent table. The data structure looks like this:
PrimID MyDates Price
1 1/1/2014 1
1 1/2/2014 2
2 1/1/2014 11
2 1/2/2014 12
3 1/1/2014 21
3 1/2/2014 22
The csv file looks exactly the same, just without the header column names. Here is my code thus far:
CREATE Table #TempT
(
PrimID Int,
myDate Date,
myPrice Float
);
BULK
INSERT #TempT
FROM 'D:\MyWerk\SQL\TEST_dPrice_Data.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
Select * From #TempT
Order by PrimID,myDate
Drop Table #TempT
What is missing, and what I am trying to get to, is the UPDATE of the permanent table with the ordered #TempT, ordered by PrimID and then myDates(oldest to lastest). If there are PrimID & myDates data in the csv that are already in the permanent table, I want to overwrite the data in the permanent file as well. Also, is there a better way to get the data in chronological order, other than using order by?
I use SQL Server 2012.
Much appreciated.
Don't try to store your data in SQL tables in some kind of row order -- this is inefficient. You can sort when you query the data.
As for the insert/update behavior, a SQL merge does this quite well. After your Bulk Insert, you can execute something like this:
MERGE PermanentT AS [TARGET]
USING #TempT AS [SOURCE]
ON [TARGET].PrimId = [SOURCE].PrimId
AND [TARGET].MyDates = [SOURCE].MyDates
WHEN MATCHED AND [TARGET].PRICE <> [SOURCE].PRICE
THEN UPDATE SET [TARGET].PRICE = [SOURCE].PRICE
WHEN NOT MATCHED
THEN INSERT (PrimID, myDate, myPrice)
VALUES (SOURCE.PrimID, SOURCE.myDate, SOURCE.myPrice);
Related
[UPDATE: 2/20/19]
I figured out a pretty trivial solution to solve this problem.
CREATE TRIGGER TriggerClaims_History on Claims
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON
INSERT INTO Claims_History
SELECT name, status, claim_date
FROM Claims
EXCEPT SELECT name, status, claim_date FROM Claims_History
END
GO
I am standing up a SQL Server database for a project I am working on. Important info: I have 3 tables - enrollment, cancel, and claims. There are files located on a server that populate these tables every day. These files are NOT deltas (i.e. each new file placed on server every day contains data from all previous files) and because of this, I am able to simply drop all tables, create tables, and then populate tables from files each day. My question is regarding my claims table - since tables will be dropped and created each night, I need a way to keep track of all the different status changes.
I'm struggling to figure out the best way to go about this.
I was thinking of creating a claims_history table that is NOT dropped each night. Essentially I'd want my claims_history table to be populated each time an initial new record is added to the claims table. Then I'd want to scan the claims table and add a row to the claims_history table if and only if there was a change in the status column (i.e. claims.status != claims_history.status).
Day 1:
select * from claims
id | name | status
1 | jane doe | received
select * from claims_history
id | name | status | timestamp
1 | jane doe | received | datetime
Day 2:
select * from claims
id | name | status
1 | jane doe | processed
select * from claims_history
id | name | status | timestamp
1 | jane doe | received | datetime
1 | jane doe | processed | datetime
Is there a SQL script that can do this? I'd also like to automatically have the timestamp field populate in claims_history table each time a new row is added (status change). I know I could write a python script to handle something like this, but i'd like to keep it in SQL if at all possible. Thank you.
Acording to your questions you need to create a trigger after update of the column claims.status and it very simple to do that use this link to know and see how to do a simple trigger click here create asimple sql server trigger
then as if there is many problem to manipulate dateTime in a query a would suggest you to use UNIX time instead of using datetime you can use Long or bigInt UNix time store the date as a number to know the currente time simple use the query SELECT UNIX_TIMESTAMP()
A very common approach is to use a staging table and a production (or final) table. All your ETLs will truncate and load the staging table (volatile) and then you execute an Stored Procedure that adds only the new records to your final table. This requires that all the data you handle this way have some form of key that identifies unequivocally a row.
What happens if your files suddenly change format or are badly formatted? You will drop your table and won't be able to load it back until you fix your ETL. This approach will save you from that, since the process will fail while loading the staging table and won't impact the final table. You can also keep deleted records for historic reasons instead of having them deleted.
I prefer to separate the staging tables into their proper schema, for example:
CREATE SCHEMA Staging
GO
CREATE TABLE Staging.Claims (
ID INT,
Name VARCHAR(100),
Status VARCHAR(100))
Now you do all your loads from your files into these staging tables, truncating them first:
TRUNCATE TABLE Staging.Claims
BULK INSERT Staging.Claims
FROM '\\SomeFile.csv'
WITH
--...
Once this table is loaded you execute a specific SP that adds your delta between the staging content and your final table. You can add whichever logic you want here, like doing only inserts for new records, or inserting already existing values that were updated on another table. For example:
CREATE TABLE dbo.Claims (
ClaimAutoID INT IDENTITY PRIMARY KEY,
ClaimID INT,
Name VARCHAR(100),
Status VARCHAR(100),
WasDeleted BIT DEFAULT 0,
ModifiedDate DATETIME,
CreatedDate DATETIME DEFAULT GETDATE())
GO
CREATE PROCEDURE Staging.UpdateClaims
AS
BEGIN
BEGIN TRY
BEGIN TRANSACTION
-- Update changed values
UPDATE C SET
Name = S.Name,
Status = S.Status,
ModifiedDate = GETDATE()
FROM
Staging.Claims AS S
INNER JOIN dbo.Claims AS C ON S.ID = C.ClaimID -- This has to be by the key columns
WHERE
ISNULL(C.Name, '') <> ISNULL(S.Name, '') AND
ISNULL(C.Status, '') <> ISNULL(S.Status, '')
-- Insert new records
INSERT INTO dbo.Claims (
ClaimID,
Name,
Status)
SELECT
ClaimID = S.ID,
Name = S.Name,
Status = S.Status
FROM
Staging.Claims AS S
WHERE
NOT EXISTS (SELECT 'not yet loaded' FROM dbo.Claims AS C WHERE S.ID = C.ClaimID) -- This has to be by the key columns
-- Mark deleted records as deleted
UPDATE C SET
WasDeleted = 1,
ModifiedDate = GETDATE()
FROM
dbo.Claims AS C
WHERE
NOT EXISTS (SELECT 'not anymore on files' FROM Staging.Claims AS S WHERE S.ClaimID = C.ClaimID) -- This has to be by the key columns
COMMIT
END TRY
BEGIN CATCH
DECLARE #v_ErrorMessage VARCHAR(MAX) = ERROR_MESSAGE()
IF ##TRANCOUNT > 0
ROLLBACK
RAISERROR (#v_ErrorMessage, 16, 1)
END CATCH
END
This way you always work with dbo.Claims and the records are never lost (just updated or inserted).
If you need to check the last status of a particular claim you can create a view:
CREATE VIEW dbo.vClaimLastStatus
AS
WITH ClaimsOrdered AS
(
SELECT
C.ClaimAutoID,
C.ClaimID,
C.Name,
C.Status,
C.ModifiedDate,
C.CreatedDate,
DateRanking = ROW_NUMBER() OVER (PARTITION BY C.ClaimID ORDER BY C.CreatedDate DESC)
FROM
dbo.Claims AS C
)
SELECT
C.ClaimAutoID,
C.ClaimID,
C.Name,
C.Status,
C.ModifiedDate,
C.CreatedDate,
FROM
ClaimsOrdered AS C
WHERE
DateRanking = 1
I have a table set up in the following manner.
CustomerNumber June15_Order June15_Billing July15_Order July15_Billing August15_Order August15_Billing
12345 5 55 3 45
5431 6 66 5 67
I would prefer it to be:
CustomerNumber Date Order Billing
12345 01/06/2015 5 55
12345 01/07/2015 3 45
5431 01/06/2015 6 66
5431 01/07/2015 5 67
Any thoughts as to how I would accurately transpose this table?
If you're just trying to get the old data into the new, you'll basically need to use brute force:
INSERT INTO NewTable
(CustomerNumber, [Date], [Order], Billing)
(
SELECT CustomerNumber, '06/15/2015', June15_Order, June15_Billing
FROM OldTable
UNION
SELECT CustomerNumber, '07/15/2015', July15_Order, July15_Billing
FROM OldTable
UNION
SELECT CustomerNumber, '08/15/2015', August15_Order, August15_Billing
FROM OldTable
)
Presuming there are columns for any month and any years, this gets ugly really fast. If the columns are set and hard-coded, use #John Pasquet's solution (+1). If you need the ability to work with any set of columns of the form MMMMDD_Type, here's an outline.
First pass:
Write a SELECT... UNPIVOT... query to transform the table
Map the resulting "label" column to a Date datatype and a "Type" (Order, Billing)
However, mapping result set column names of "July15" to "Jul 1, 2015" (or 01/07/2015) is hard, if not crazy hard. This leads to a second pass:
Build a "lookup" list of columns from sys.tables and sys.colmns
Pick out those that are to be unpivoted
Figure out the dates and types for each of them
Build the SELECT... UNPIVOT... in dynamic SQL, dumping the results to a temp table
Join this temp table to the lookup list by original column name, which (via the join) gets you the prepared date and type values
Seriously, this could get ridiculously complex. The smart money is on rebuild the tables with columns for date and type.
First create the a new table with the desired structure, after that you will need to create a stored procedure for that task, which will iterate over all rows.
On the columns you know old_col to new_col just take the value and save in a variable, for the others you will need to create condition for each month like a "contains june" and save in two variables date and value, after that each time you found a new month with value > 0 perform a insert on the new table with all the variables.
I am new to sql. Can someone help me with this requirement.
I have table with 10000 records like this
CompanyID Name
300001 A
300004 B
300005 C
300007 D
|
|
|
310000 XXX
And I have a another list of companyIDs that I am going to update the above table(It is just an excel sheet not a table)
OldID NewID
300001 500001
300002 500002
300003 500003
300004 500004
300005 500005
|
|
310000 510000
My requirement is, If I found the companyID in the first table I need to update it with the NewID and If I didn't find the companyId in the first table I have to create a new row in the table with the NewID regardless of oldID.
Is there any possibility to do both update and insert in a single query?
You're describing an "upsert" or MERGE statement, typically:
merge into table_a
using (<some_statement>)
on (<some_condition>)
when matched then
update
set ...
when not matched then
insert (<column_list>)
values (<column_list>);
However, a MERGE can't update a value that's referenced in the ON clause, which is what will be required in order to do what you're asking. You will, therefore, require two statements:
update table_to_be_updated t
set companyid = (select newid from new_table where oldid = t.companyid )
insert into table_to_be_updated
select newid
from newtable t
where not exists ( select 1
from table_to_be_updated
where t.newid = companyid )
If it's possible for a newid and an oldid to be the same then you're going to run into problems. This also assumes that your new table is unique on oldid and newid - it has to be unique in order to do what you want so I don't think this is an unreasonable assumption.
I have created two tables customersrc and customertemp with the columns:
customertemp
ID name age addr cityid isactive
34 Gi 24 Chennai 1 1
customersrc
CustomerId CustomerName CustomerAge CustomerAddress
1 Gi 24 madurai
2 Pa 23 Tirupur
3 MI 27 Tirupur
Now I need to insert pa and mi data value to the temp table bcz it is not matched with the rows of customertemp. And the row gi data will be updated which was matched.
I used the following MERGE statement
DECLARE #cityid INT SET #cityid=1
MERGE Temp.dbo.customersrc as src_customer
USING ( SELECT CustomerName,CustomerAge,CustomerAddress FROM customertemp) as temp_customer
ON src_customer.name=temp_customer.CustomerName
AND
src_customer.cityid=#cityid
WHEN MATCHED THEN
UPDATE SET
src_customer.age=temp_customer.CustomerAge,
src_customer.addr=temp_customer.CustomerAddress,
src_customer.isactive=1
WHEN NOT MATCHED BY SOURCE THEN
UPDATE SET src_customer.isactive=0 ; -- here i need the insert statement to insert in another table
Questions:
is it possible to write insert statement inside the when not matched by source query?
if it is not possible then how to achieve this using merge?
in a simple set theory I need to put the customersrc(table_B)-customertemp (table_A). B-A value into the another or temp table.
One of the main usages of the MERGE statement is to perform so called "UPSERTS" (Update matching records, insert new records), so it is definitely possible to do what you want. Just add the following to the last part of your MERGE statement:
WHEN NOT MATCHED BY TARGET THEN
INSERT (name, age, addr, cityid, isactive)
VALUES (CustomerName, CustomerAge, CustomerAddress, #cityid, 1)
If you also need to insert data into a 3rd table, depending on whether rows are updated or inserted, you can use the OUTPUT clause of the merge statement. Check out the documentation: http://technet.microsoft.com/en-us/library/ms177564.aspx
Me: Why do you want to insert to another table?
You: To show the user who are not in the customertemp table.
So your requirement is not to insert into another table. Your requirement is to get the missing users.
You could do that with a dummy UPDATE (SET SomeCol = SomeCol) and OUTPUT. But that is a hack that I would try to avoid.
It is probably easier to do this in two statements. Here's how you'd get the missing rows:
SELECT temp_customer.*
FROM (SELECT CustomerName,CustomerAge,CustomerAddress FROM customertemp) as temp_customer
LEFT JOIN customersrc ON src_customer.name=temp_customer.CustomerName AND src_customer.cityid=#cityid
WHERE customersrc.cityid IS NULL
Got sql to pad 0's from: Formatting Numbers by padding with leading zeros in SQL Server
What I want to do is get a users order history from our Navigator database tables. We have a process that puts the orders from the website tables into the the navigator tables (because the orders need to be formatted for the nav table).
I want to query the website tables to get the orders from a logged in user using their user id:
SELECT OrderID FROM db1.zm.dbo.zOrder WHERE AccountID = 631180 (this returns multiple order id's)
db1 is the server, zm is database.
The OrderID's returned are formatted like 4565, 5675, ect. I want to insert them into a temp table like: Z0004565 with a Z and enough leading 0's to hit 7 digits for the number.
How do I modifiy the select statement to do so? Or can I.
SQL for padded 0's: SELECT REPLICATE('0', 7-LEN(4665)) + 4665
SQL Following Comments:
DECLARE #OrderNumTable table (orderNum varchar(20))
INSERT INTO #OrderNumTable EXEC( SELECT (''Z'' + REPLICATE(''0'', 7-len(OrderID)) + OrderID)FROM db1.zm.dbo.zOrder WHERE AccountID = 631180
SELECT OrderID,
'Z'+RIGHT('0000000'+CAST(OrderID AS VARCHAR(7)),7)
FROM db1.zm.dbo.zOrder
WHERE AccountID = 631180
Updated following the question edit
DECLARE #OrderNumTable table (orderNum varchar(8))
INSERT INTO #OrderNumTable(orderNum)
SELECT 'Z'+RIGHT('0000000'+CAST(OrderID AS VARCHAR(7)),7)
FROM db1.zm.dbo.zOrder
WHERE AccountID = 631180
Can you not just add a Z to the front?
SELECT 'Z' + REPLICATE('0', 7-LEN(OrderID)) + LTRIM(STR(OrderID))