SQL Server 2014 Replace Distinct? DeIdentify Data - sql

I am going to explain again what I am trying to do in hopes that you can help.
Table 1 has 4061 rows with columns that include
[Name],[Address1],[Address2],[Address3],[City],[State],[Zip],[Country],[Phone]
and 20 other columns. Table 1 is data that needs to be deidentified. Table 1 has 1534 distinct [Name] rows out of 4061 rows total.
Table 2 has auto generated data which includes the same columns. I would like to replace the above mentioned columns in table 1 with data from table 2. I want to select distinct based on [Name] from table one and then [Name],[Address1],[Address2],[Address3],[City],[State],[Zip],[Country],[Phone] with a new set of distinct data from table 2.
I do not want to just update each row with a new address as that will screw up the data consistency. By replacing only distinct this will allow me to preserve the data consistency while changing the row data in table 1. When I am done I would like to have 1534 distinct new de-identified [Name] [Address1],[Address2],[Address3],[City],[State],[Zip],[Country],[Phone] in table 1 from table 2.

You would use join in the update. You can generate a join key for 1500 rows using row_number():
update toupdate
set t.address = f.address
from (select t.*, row_number() over (order by newid()) as seqnum
from table t
) toupdate join
(select f.*, row_number() over (order by newid()) as seqnum
fake f
) f
on toupdate.seqnum = f.seqnum and t.seqnum <= 1500;

Here is how I ended up doing it.
First I ran a statement to select distinct and inserted it into a table.
Select Distinct [Name],[Address1],[City],[State],[Zip],[Country],[Phone]
INTO APMAST2
FROM APMAST
I then added name2 column in APMAST2 and used a statement to create a sequential id field into APMAST2.
DECLARE #id INT
SET #id = 0
UPDATE APMAST2
SET #id = id = #id + 1
GO
Now I have my distinct info plus a blank name field and a sequential ID field in APMAST2. Now I can join this date with my fakenames table which I generated from. HERE using their bulk tool.
Using a Join Statement I joined my fake data with APMAST2
Update dbo.APMAST2
SET dbo.APMAST2.Name = dbo.fakenames.company,
dbo.APMAST2.Address1 = dbo.fakenames.streetaddress,
dbo.APMAST2.City = dbo.fakenames.City,
dbo.APMAST2.State = dbo.fakenames.State,
dbo.APMAST2.Zip = dbo.fakenames.zipcode,
dbo.APMAST2.Country = dbo.fakenames.countryfull,
dbo.APMAST2.Phone = dbo.fakenames.telephonenumber
FROM
dbo.APMAST2
INNER JOIN
dbo.fakenames
ON dbo.fakenames.number = dbo.APMAST2.id
Now I have my fake data loaded but I kept my original Name field so I could reload this data into my full table ARMAST so now I can do a join between ARMAST2 and ARMAST.
Update dbo.APMAST
SET dbo.APMAST.Name = dbo.APMAST2.Name,
dbo.APMAST.Address1 = dbo.APMAST2.Address1,
dbo.APMAST.City = dbo.APMAST2.City,
dbo.APMAST.State = dbo.APMAST2.State,
dbo.APMAST.Zip = dbo.APMAST2.Zip,
dbo.APMAST.Country = dbo.APMAST2.Country,
dbo.APMAST.Phone = dbo.APMAST2.Phone
FROM
dbo.APMAST
INNER JOIN
dbo.apmast2
ON dbo.apmast.name = dbo.APMAST2.name2
Now my original table has all fake data in it but it keeps the integrity it had , well most of it, so the data looks good when reported on but is de-identified. You can now remove APMAST2 or keep it if you need to match this with other data later on. I know this is long and I am sure there is a better way to do it but this is how I did it, suggestions welcome.

Related

Needing to access table from subquery in main update query

I'm in need of a way to access a column from the inner table from a subquery, which I have included below with database/table names changed. The inner query returns all of the Item_Ids of Items that have a specific term config and belong to a specific customer. Obviously, the SET currently doesn't work as I can't access any of the rows of let from the main query.
I'm fairly new to SQL and am struggling to wrap my head around how I might convert this to using joins instead of a subquery. The problem is that I need to be able to set ALL of the Grand Totals for any of the Item_Ids that come from the subquery.
UPDATE [Database].[dbo].[Items]
SET GrandTotal = GrandTotal / let.CurrentValue
WHERE Id IN (
SELECT let.Item_Id
FROM [Database].[dbo].[ItemTerms] let
WHERE TermConfig_Id = 'TERM_CONFIG_ID'
AND Item_Id IN (
SELECT le2.Id
FROM [Database].[dbo].[LaneExhibits] le2
WHERE Customer_Id = 'CUST_ID'
)
)
EDIT: Add Sample Row from Subquery
Columns are Id, DataValueStatus, CurrentValue, PreviousValue, Item_Id, TermConfig_Id. The subquery only returns the Item_Id but that is the rest of the data.
424C8BF4-0FCB-E711-80C9-005056BA0972 1 460 NULL 2D4C8BF4-0FCB-E711-80C9-005056BA0972 B8FCE730-27BE-E711-80C9-005056BA0972
The inner join version:
UPDATE i
SET GrandTotal = GrandTotal / let.CurrentValue
FROM [Database].[dbo].[Items] i
INNER JOIN [Database].[dbo].[ItemTerms] let ON i.Id=let.Item_Id
WHERE TermConfig_Id = 'TERM_CONFIG_ID'
AND Item_Id IN (
SELECT le2.Id
FROM [Database].[dbo].[LaneExhibits] le2
WHERE Customer_Id = 'CUST_ID'
)
Merge is your friend https://learn.microsoft.com/en-us/sql/t-sql/statements/merge-transact-sql?view=sql-server-2017
simplified sample from your example
MERGE [Database].[dbo].[Items] AS target
USING (SELECT let.Item_Id, let.CurrentValue FROM [Database].[dbo].[ItemTerms] let
/* other joins and filters here */
) AS source (Item_Id, CurrentValue)
ON (target.Item_Id = source.id /* or what you want */)
WHEN MATCHED THEN
UPDATE SET GrandTotal = source.GrandTotal / source.CurrentValue

SQL Update Skipping duplicates

Table 1 looks like the following.
ID SIZE TYPE SERIAL
1 4 W-meter1 123456
2 5 W-meter2 123456
3 4 W-meter 585858
4 4 W-Meter 398574
As you can see. Items 1 and 2 both have the same Serial Number. I have an innerjoin update statement that will update the UniqueID on these devices based on linking their serial number to the list.
What I would like to do. Is modify by hand the items with duplicate serial numbers and scripted update the ones that are unique. Im presuming I have to reference the distinct command here somewhere buy not sure.
This is my update statement as is. Pretty simple and straight forward.
update UM00400
Set um00400.umEquipmentID = tb2.MIUNo
from UM00400 tb1
inner join AA_Meters tb2 on
tb1.umSerialNumber = tb2.Old_Serial_Num
where tb1.umSerialNumber <> tb2.New_Serial_Num
;WITH CTE
AS
(
SELECT * , rn = ROW_NUMBER() OVER (PARTITION BY SERIAL ORDER BY SERIAL)
FROM UM00400
)
UPDATE CTE
SET CTE.umEquipmentID = tb2.MIUNo
inner join AA_Meters tb2
on CTE.umSerialNumber = tb2.Old_Serial_Num
where tb1.umSerialNumber <> tb2.New_Serial_Num
AND CTE.rn = 1
This will update the 1st record of multiple records with the same SERIAL.
If i understand your question correctly below query will help you out :
;WITH CTE AS
(
// getting those serial numbers which are not duplicated
SELECT umSerialNumber,COUNT(umSerialNumber) as CountOfSerialNumber
FROM UM00400
GROUP BY umSerialNumber
HAVING COUNT(umSerialNumber) = 1
)
UPDATE A SET A.umEquipmentID = C.MIUNo
FROM UM00400 A
INNER JOIN CTE B ON A.umSerialNumber = B.umSerialNumber
INNER JOIN AA_Meters C ON A.umSerialNumber = C.Old_Serial_Num

How to increment a column based on two tables that are joined

I am trying to increment a column on a sql server table based on the join between the initial table and the joined table. The idea is to update tblForm10Objectives, set the ObjectiveNumber column to an increment number starting with 1 based on the number of rows returned from the join of tblForm10GoalsObjectives and tblForm10Objectives where ID_Form10Goal equals a number. Example query so far:
Update tblForm10Objectives
Set ObjectiveNumber = rn
From (
Select ROW_NUMBER() over (PARTITION by OG.ID_Form10Goal) as rn
, *
From (
Select *
From tblForm10GoalsObjectives OG
Join tblForm10Objectives O On OG.ID_Form10Objective = O.ID_Form10Objective
Where OG.ID_Form10Goal = 4
Order by O.ID_Form10Objective
) as tblForm10Objectives;
If the select portion of the query is performed the columns are displayed so you can see the ObjectiveNumber is currently 0 where ID_Form10Goal = 4
Once the update runs I need for the ObjectiveNumber to show 1 , 2; since there are two rows for ID_Form10Goal = 4.
I had to introduce a new table to the logic of this update statement, the table name is tblForm10Goals. The objectives need to be pulled by ID_Agency instead of ID_Form10Goal I am getting an error message stating a "a multipart identifier 'dbo.tblForm10Objectives.ID_Form10Objective = rns.ID_Form10Objective' could not be bound. I am using the following SQL Update statement:
UPDATE dbo.tblForm10Objectives
SET ObjectiveNumber = rn
FROM tblForm10Goals As g
Left Join tblForm10GoalsObjectives gobs ON g.ID_Form10Goal = gobs.ID_Form10Goal
Right Join
(
SELECT
ROW_NUMBER() OVER (PARTITION BY g.ID_Agency
ORDER BY OB.ID_Form10Objective) AS rn,
OB.ID_Form10Objective
FROM tblForm10Goals g
LEFT JOIN dbo.tblForm10GoalsObjectives gobs ON g.ID_Form10Goal = gobs.ID_Form10Goal
RIGHT JOIN dbo.tblForm10Objectives OB ON gobs.ID_Form10Objective = OB.ID_Form10Objective
Where g.ID_Agency = 2
) rns ON dbo.tblForm10Objectives.ID_Form10Object = rns.ID_Form10Objective
Your example seems to be missing a closing parenthesis somewhere, and without the table structures to look at, I can't be certain of my answer. It seems you have two tables:
tblForm10Objectives
-------------------
ID_Form10Objective
ObjectiveNumber
...
and
tblForm10GoalsObjectives
------------------------
ID_Form10Goal
ID_Form10Objective
...
If this is the case, the following query should give you the results you desire:
UPDATE dbo.tblForm10Objectives
SET ObjectiveNumber = rn
FROM dbo.tblForm10Objectives INNER JOIN
(
SELECT
ROW_NUMBER() OVER (PARTITION BY OG.ID_Form10Goal
ORDER BY O.ID_Form10Objective) AS rn,
O.ID_Form10Objective
FROM dbo.tblForm10Objectives O INNER JOIN
dbo.tblForm10GoalsObjectives OG ON OG.ID_Form10Objective = O.ID_Form10Objective
Where OG.ID_Form10Goal = 4
) rns ON dbo.tblForm10Objectives.ID_Form10Objective = rns.ID_Form10Objective
If you run the inner SELECT statement, you will see the desired ObjectiveNumber values and the corresponding ID_Form10Objective that will get updated with those values.
If you post your table structures, I or someone else may be able to be of more help.

Selective update in SQL Server

I've created a junction table like this one:
http://imageshack.us/scaled/landing/822/kantotype.png
I was trying to figure out a query that could able to select some rows - based on the PokémonID - and then updating only the first or second row after the major "filtering".
For example:
Let's suppose that I would like to change the value of the TypeID from the second row containing PokémonID = 2. I cannot simply use UPDATE KantoType SET TypeID = x WHERE PokémonID = 2, because it will change both rows!
I've already tried to use subqueries containing IN,EXISTS and LIMIT, but with no success.
Its unclear what are your trying to do. However, you can UPDATE with JOIN like so:
UPDATE
SET k1.TypeID = 'somethng' -- or some value from k2
FROM KantoType k1
INNER JOIN
(
Some filtering and selecting
) k2 ON k1.PokémonID = k2.PokémonID
WHERE k1.PokémonID = 2;
Or: if you want to UPDATE only the two rows that have PokémonID = 2 you can do this:
WITH CTE
AS
(
SELECT *,
ROW_NUMBER() OVER(ORDER BY TypeID) rownum
FROM KantoType
WHERE PokemonID = 2
)
UPDATE c
SET c.TypeID = 5
FROM CTE c
WHERE c.rownum = 1;
SQL Fiddle Demo
I can suggest something like this if you just need to update a single line in your table:
UPDATE kantotype
SET
type = 2
WHERE pokemon = 2
AND NOT EXISTS (SELECT * FROM kantotype k2
WHERE kantotype.type > k2.type
AND kantotype.pokemon = k2.pokemon)
It would be easier to get the first or last item of the table if you had unique identifier field in your table.
Not sure even if you are trying to update the row with PokemenID =2 by doing a major filtering on TypeID... So just out of assumptiong (big one), you can give a try on Case
UPDATE yourtable a
LEFT JOIN youtable b on a.pokeid = b.pokeid
SET a.typeid = (CASE
WHEN a.typeid < b.typeid THEN yourupdatevalue
WHEN a.typeid > b.typeid THEN someothervalue
ELSE a.typeid END);
If you know the pokemon ID and the type id then just add both to the where clause of your query.
UPDATE KantoType
SET TypeID = x
WHERE PokémonID = 2
AND TypeID=1
If you don't know the type ID, then you need to provide more information about what you're trying to accomplish. It's not clear why you don't have this information.
Perhaps think about what is the unique identifier in your data set.

SQL Server Incrementing a counter in Update

I'm updating some columns and incrementing a counter whenever I change a row.
The update statement is the result of a join (simplified code below):
update #to
set
t.num += 1
from #to t
join #source s
on t.id = s.id
When I update one row more than once, the columns hold the last value (as they should), but the counter is only incremented once. So if the join returns (id = 1, id = 1), my table holds (id = 1, num = 1) rather than (id = 1, num = 2).
There are ways to get around this (another join on a select count, for example), but I wonder if there's a way to keep it simple.
There's not really a way to get the count without getting the count. Here is one way to do that (and still only referencing #source once):
;WITH s AS
(
SELECT id, c = COUNT(*)
FROM #source
GROUP BY id
)
UPDATE t SET t.num += s.c
FROM #to AS t
INNER JOIN s
ON t.id = s.id;
Hopefully the rows that end up in #source are already filtered down to only those that will also be found in #to. If not, you can add more conditions to the initial CTE.