I have looked for the codes but I couldn't make the codes work in my SQL server 2017.
I need to create a stored procedure to avoid data duplication from a table and delete all the data duplicated.
I have created this code:
CREATE PROCEDURE deldupl_LSBU_Staff AS
SELECT Phone_number, COUNT(*) as CNT
FROM LSBU_Staff
GROUP BY Phone_number
DELETE FROM LSBU_Staff
WHERE Phone_number > 1;
BUT when I execute my code, it deletes all the records from the table and I do not want this. I just want to delete all the duplicated data.
I have also created another code to delete the duplicated data from the table LSBU_Staff:
SELECT ROW_NUMBER() OVER(PARTITION BY Phone_number ORDER BY Phone_number)
AS del_dupl_record
FROM LSBU_Staff
WHERE Phone_number > 1
DELETE FROM LSBU_Staff
WHERE Phone_number > 1;
And it still deletes all the data.
LSBU_Staff columns are: Staff_id, LastName, FirstName, Speciality_type and Phone_number. I chose Phone_number as its identification.
Try this. Not super elegant and can be cleaned up but should do the trick. This will keep the first of the group. If you prefer to keep the last change to "l2.Staff_id > l1.Staff_id"
--
DROP TABLE IF EXISTS LSBU_Staff
CREATE TABLE LSBU_Staff
( Staff_id INT IDENTITY(1,1)
, LastName VARCHAR(32)
, FirstName VARCHAR(32)
, Speciality_type VARCHAR(32)
, Phone_number VARCHAR(32)
)
INSERT INTO LSBU_Staff (LastName, FirstName, Speciality_type, Phone_number)
VALUES
('Stilskin', 'Rumple', 'dancer' , '305-305-3050')
, ('Lamb', 'Mary', 'shepherd' , '305-123-4567')
, ('Lamb', 'Aurthur', 'shepherd' , '305-123-4567')
, ('Fenokee', 'Okee', 'swimmer' , '305-305-3051')
SELECT * FROM LSBU_Staff
DELETE LSBU_Staff
WHERE Staff_id IN
(
SELECT Staff_id
FROM LSBU_Staff l1
WHERE EXISTS (SELECT 1 FROM LSBU_Staff l2 WHERE l2.Phone_number = l1.Phone_number
AND l2.Staff_id < l1.Staff_id)
)
SELECT * FROM LSBU_Staff
DROP TABLE IF EXISTS LSBU_Staff
Related
When i am inserting fields from table T_EMP_MASTER into temp table , it's appearing in #temp_empdet as per the order of values of emp_id,emp_id2,emp_id3 .
I want emp_id,emp_id2,emp_id3 fields to to be inserted as per the order of insertion, like
1. emp_id
2. emp_id2
3. emp_id3
How can I do the same here , using same insertion with union or using anything else ?
CREATE TABLE #TEMP_EMPDET
(SN INT,EMP_ID INT)
INSERT INTO #TEMP_EMPDET
SELECT EMP_ID FROM T_EMP_MASTER WHERE CODE = #CODE
UNION
SELECT EMP_ID2 FROM T_EMP_MASTER WHERE CODE = #CODE
UNION
SELECT EMP_ID3 FROM T_EMP_MASTER WHERE CODE = #CODE
If you want to guarantee the order of insertion, use three separate statements:
INSERT INTO #TEMP_EMPDET (EMP_ID)
SELECT EMP_ID FROM T_EMP_MASTER WHERE CODE = #CODE;
INSERT INTO #TEMP_EMPDET (EMP_ID)
SELECT EMP_ID2 FROM T_EMP_MASTER WHERE CODE = #CODE;
INSERT INTO #TEMP_EMPDET (EMP_ID)
SELECT EMP_ID3 FROM T_EMP_MASTER WHERE CODE = #CODE;
Because SQL tables represent unordered sets, you will only see the "ordering" in an identity column or if EMP_ID is a clustered key.
CREATE TABLE #TEMP_EMPDET (
EMPDET_ID INT IDENTITY PRIMARY KEY,
SN INT,
EMP_ID INT
);
Your code is not inserting values in order specifically because of the UNION. That eliminates duplicates and re-arranges the data. If you want that functionality, things get a bit more complicated. I think I would recommend:
INSERT INTO #TEMP_EMPDET
SELECT v.EMP_ID
FROM T_EMP_MASTER EM CROSS APPLY
(VALUES (1, EMP_ID), (2, EMP_ID2), (3, EMP_ID3)
) V(WHICH, EMP_ID)
WHERE CODE = #CODE
GROUP BY v.EMP_ID
ORDER BY MIN(WHICH);
I have a temp table that looks like this:
FirstName
LastName
DOB
Sex
Age
ExternalID
In my stored procedure I'm inserting these values into a regular table that has the following structure:
ID identity(1,1)
FirstName
LastName
So, I do this:
Insert into myTable
select FirstName, LastName from TempTable
During the insert I need to insert primary key from main table back into temp table "ExternalID" column. How can this be achieved?
I tried using OUTPUT statement but it only allows to insert to a separate table and then I have no way to map back to temp table
I need to insert generated IDs to column ExternalID in temp table right after the insert. FirstName and LastName are not unique.
One possible solution would be to use loop and insert one row at a time. This way, I can update temp table row with scope_identity(). But I want to avoid using loops.
Try using MERGE instead of INSERT.
MERGE allows you to output a column you didn't insert, such as an identifier on your temp table. Using this method, you can build another temporary table that maps your temp table to the inserted rows (named #TempIdTable in the sample below).
First, give #TempTable its own primary key. I'll call it TempId. I'll also assume you have a column on #TempTable to store the returned primary key from MyTable, ID.
--Make a place to store the associated ID's
DECLARE #TempIdTable TABLE
([TempId] INT NOT NULL
,[ID] INT NOT NULL)
--Will only insert, as 1 never equals 0.
MERGE INTO myTable
USING #TempTable AS tt
ON 1 = 0
WHEN NOT MATCHED
THEN
INSERT ([FirstName]
,[LastName])
VALUE (t.[FirstName]
,t.[LastName])
OUTPUT tt.[TempId], inserted.[ID] --Here's the magic
INTO #TempIdTable
--Associate the new primary keys with the temp table
UPDATE #TempTable
SET [ID] = t.[ID]
FROM #TempIdTable t
WHERE #TempTable.[TempId] = t.[TempId]
I was working on a similar issue and found this trick over here: Is it possible to for SQL Output clause to return a column not being inserted?
Here's the full code I used in my own testing.
CREATE TABLE [MQ]
([MESSAGEID] INT IDENTITY PRIMARY KEY
,[SUBJECT] NVARCHAR(255) NULL);
CREATE TABLE [MR]
([MESSAGESEQUENCE] INT IDENTITY PRIMARY KEY
,[TO] NVARCHAR(255) NOT NULL
,[CC] NVARCHAR(255) NOT NULL
,[BCC] NVARCHAR(255) NOT NULL);
CREATE TABLE #Messages (
[subject] nvarchar(255) NOT NULL
,[to] nvarchar(255) NOT NULL
,[cc] nvarchar(255) NULL
,[bcc] nvarchar(255) NULL
,[MESSAGEID] INT NULL
,[sortKey] INT IDENTITY PRIMARY KEY
);
INSERT INTO #Messages
VALUES ('Subject1','to1','cc1','bcc1', NULL)
,('Subject2','to2', NULL, NULL, NULL);
SELECT * FROM #Messages;
DECLARE #outputSort TABLE (
[sortKey] INT NOT NULL
,[MESSAGEID] INT NOT NULL
,[subject] NVARCHAR(255)
);
MERGE INTO [MQ]
USING #Messages M
ON 1 = 0
WHEN NOT MATCHED
THEN
INSERT ([SUBJECT])
VALUES (M.[subject])
OUTPUT M.[SORTKEY]
,inserted.[MESSAGEID]
,inserted.[SUBJECT]
INTO #outputSort;
SELECT * FROM #outputSort;
SELECT * FROM [MQ];
UPDATE #Messages
SET MESSAGEID = O.[MESSAGEID]
FROM #outputSort O
WHERE #Messages.[sortKey] = O.[sortKey];
SELECT * FROM #Messages;
DROP TABLE #Messages;
As you said, FirstName and LastName are not unique. This means you cannot use a trigger because there can be the same FirstName + LastName so you cannot join on them.
But you can do the inverse thing: first update your temp table ExternalID (I suggest you to use sequence object and just do update #t set ExternalID = next value for dbo.seq1;) and then just insert your rows including ExternalID into myTable. To be able to insert into identity field you can use set identity_insert myTable on or you can re-design your destination table to contain no identity at all as now you use sequence for the same purpose.
We need a unique column for able to make the comparison at the update operation after the insert. That's why we are using ExternalID column temporarily. ExternalID updated by row_nubmber.
;WITH CTE AS
(
SELECT *, RN = ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) FROM #TempTable
)
UPDATE CTE SET ExternalID = RN
We are keeping the output of the insert operation in a temp table. The trick is order by with ExternalID, it will help us for making the unique row number for same first and last name
DECLARE #output TABLE (
ID INT,
FirstName VARCHAR(10),
LastName VARCHAR(10))
Insert into #myTable
OUTPUT inserted.ID, inserted.FirstName, inserted.LastName INTO #output(ID, FirstName, LastName)
select FirstName, LastName from #TempTable T
order by ExternalID
For replacing the ExternalID column with inserted id value, we are making comparing with first name, last name and row number.
;WITH TMP_T AS(
SELECT *, RN = ROW_NUMBER() OVER(PARTITION BY FirstName, LastName ORDER BY ExternalID) FROM #TempTable )
,OUT_T AS(
SELECT *, RN = ROW_NUMBER() OVER(PARTITION BY FirstName, LastName ORDER BY ID) FROM #output )
UPDATE TMP_T SET ExternalID = OUT_T.ID
FROM
TMP_T INNER JOIN OUT_T ON
TMP_T.FirstName = OUT_T.FirstName
AND TMP_T.LastName = OUT_T.LastName
AND TMP_T.RN = OUT_T.RN
Sample Data:
DECLARE #TempTable TABLE (
FirstName VARCHAR(10),
LastName VARCHAR(10),
DOB VARCHAR(10),
Sex VARCHAR (10),
Age VARCHAR(10),
ExternalID INT)
INSERT INTO #TempTable VALUES
('Serkan1', 'Arslan1', 'A','M','1',NULL),
('Serkan2', 'Arslan2', 'B','M','1',NULL),
('Serkan3', 'Arslan', 'C','M','1',NULL),
('Serkan3', 'Arslan', 'D','M','1',NULL)
DECLARE #myTable TABLE (
ID INT identity(100,1), -- started from 100 for see the difference
FirstName VARCHAR(10),
LastName VARCHAR(10))
Result:
MyTable
ID FirstName LastName
----------- ---------- ----------
100 Serkan1 Arslan1
101 Serkan2 Arslan2
102 Serkan3 Arslan
103 Serkan3 Arslan
TempTable
FirstName LastName DOB Sex Age ExternalID
---------- ---------- ---------- ---------- ---------- -----------
Serkan1 Arslan1 A M 1 100
Serkan2 Arslan2 B M 1 101
Serkan3 Arslan C M 1 102
Serkan3 Arslan D M 1 103
One way to do this is by duplicating the data into a second temp table like so:
SELECT *
INTO #TEMPTABLE
FROM (VALUES (1, 'Adam'), (2, 'Kate'), (3, 'Jess')) AS X (Id, Name)
SELECT TOP 0 CAST(NULL AS INT) AS IdentityValue, *
INTO #NEWTEMPTABLE
FROM #TEMPTABLE
CREATE TABLE #TABLEFORINSERT (
IdentityColumn INT IDENTITY(1,1),
Id INT,
Name VARCHAR(255)
)
INSERT INTO #TABLEFORINSERT (Id, Name)
OUTPUT INSERTED.IdentityColumn, INSERTED.Id, Inserted.Name INTO #NEWTEMPTABLE
SELECT Id, Name FROM #TEMPTABLE
--New temp table with identity values
SELECT * FROM #NEWTEMPTABLE
I have a table tblApply which have primary key(email_id, Job_id) which are foreign key of tblEmail(email) and tblJob(Job_id). I want to insert new values in tblApply but avoid those which are already there in a combination of email_id, Job_id. Below is my code. How should i do it. I am doing this inside a stored procedure.
insert into tblApply(email_Id, Job_Id, Apply_Date)
select #emailId, tblJobsJob_Id, GetDate() from tblJobs
where Job_Active = 1
Well from what I understand you wish to design a stored procedure for inserting rows that will avoid inserting duplicate unique key values.
If that's the case you could do:
insert into tblApply(email_Id, Job_Id, Apply_Date)
select #emailId, tblJobsJob_Id, GetDate() from tblJobs
where Job_Active = 1 AND
NOT EXISTS(SELECT 1 FROM tblApply
WHERE email_Id = #emailId AND Job_Id = tblJobsJob_Id)
To insert the non-existing records, you can use the condition check by LEFT JOIN with NULL check.
INSERT INTO tblApply (email_Id, Job_Id, Apply_Date)
SELECT #emailId, tblJobsJob_Id, GetDate()
FROM tblJobs TJ
LEFT JOIN tblApply TA ON TA.email_id = #emailId AND TA.Job_id = TJ.tblJobsJob_Id
WHERE TY.Job_Active = 1 AND TA.Job_id IS NULL
Objective: using SqlServer 2005, Select multiple columns, but ensure that 1 specific column is not a duplicate
Issue: The following code does not remove the duplicates. The field that has duplicates is email.
SELECT DISTINCT
email,
name,
phone
FROM
database.dbo.table
WHERE
status = 'active'
GROUP BY
email,
name,
phone
Thank you in advance for any comments, suggestions or recommendations.
It removes email duplicates but you have to decide which name, phone you need. The result is based on name, phone sort order.
WITH cl
as
(
SELECT email, name, phone, ROW_NUMBER() OVER(PARTITION BY email ORDER BY name, phone) rn
FROM
database.dbo.table
WHERE
status = 'active')
select *
from cl
where rn =1
This is a way of doing it
DECLARE #Table AS TABLE
(email NVARCHAR(100), name NVARCHAR(100), phone NVARCHAR(100))
INSERT INTO #Table
( email , name , phone)
VALUES ( N'fred', -- email - nvarchar(100)
N'bob', -- name - nvarchar(100)
N'steve' -- phone- nvarchar(100)
)
INSERT INTO #Table
( email , name , phone)
VALUES ( N'fred', -- email - nvarchar(100)
N'bob2', -- name - nvarchar(100)
N'ste1ve' -- phone- nvarchar(100)
)
INSERT INTO #Table
( email , name , phone)
VALUES ( N'fred1', -- email - nvarchar(100)
N'bob3', -- name - nvarchar(100)
N'steve3' -- phone- nvarchar(100)
)
SELECT email , MAX(name ) c2, MAX(col3) c3 FROM #Table GROUP BY email
I am moving old project that used single table inheritance in to a new database, which is more structured. How would I write a SQL script to port this?
Old structure
I've simplified the SQL for legibility.
CREATE TABLE customers (
id int(11),
...
firstName varchar(50),
surname varchar(50),
address1 varchar(50),
address2 varchar(50),
town varchar(50),
county varchar(50),
postcode varchar(50),
country varchar(50),
delAddress1 varchar(50),
delAddress2 varchar(50),
delTown varchar(50),
delCounty varchar(50),
delPostcode varchar(50),
delCountry varchar(50),
tel varchar(50),
mobile varchar(50),
workTel varchar(50),
);
New structure
CREATE TABLE users (
id int(11),
firstName varchar(50),
surname varchar(50),
...
);
CREATE TABLE addresses (
id int(11),
ForeignKey(user),
street1 varchar(50),
street2 varchar(50),
town varchar(50),
county varchar(50),
postcode varchar(50),
country varchar(50),
type ...,
);
CREATE TABLE phone_numbers (
id int(11),
ForeignKey(user),
number varchar(50),
type ...,
);
With appropriate cross-database notations for table references if appropriate:
INSERT INTO Users(id, firstname, surname, ...)
SELECT id, firstname, surname, ...
FROM Customers;
INSERT INTO Addresses(id, street1, street2, ...)
SELECT id, street1, street2, ...
FROM Customers;
INSERT INTO Phone_Numbers(id, number, type, ...)
SELECT id, phone, type, ...
FROM Customers;
If you want both the new and the old address (del* version), then repeat the address operation on the two sets of source columns with appropriate tagging. Similarly, for the three phone numbers, repeat the phone number operation. Or use a UNION in each case.
First make sure to backup your existing data!
The process is differnt if you are going to use the original id field or generate a new one.
Assuming you are going to use the orginal, make sure that you have the ability to insert id fields into the table before you start (the SQL Server equivalent if you are autogenrating the number is Set identity Insert on, not sure what mysql would use). Wirte an insert from the old table to the parent table:
insert newparenttable (idfield, field1, field2)
select idfield, field1, field2 from old parent table
then write similar inserts for all the child tables depending on what fields you need. Where you have multiple phone numbers in differnt fields, for instance, you would use a union all stament as your insert select.
Insert newphone (phonenumber, userid, phonetype)
select home_phone, id, 100 from oldparenttable
union all
select work_phone, id, 101 from oldparenttable
Union all
select cell_phone, id, 102 from oldparenttable
If you are going to have a new id generated, then create the table with a field for the old id. You can drop this at the end (although I'd keep it for about six months). Then you can join from the new parent table to the old parent table on the oldid and grab the new id from the new parent table when you do you inserts to child tables. Something like:
Insert newphone (phonenumber, userid, phonetype)
select home_phone, n.id, 100 from oldparenttable o
join newparenttable n on n.oldid = o.id
union all
select work_phone, n.id, 101 fromoldparenttable o
join newparenttable n on n.oldid = o.id
Union all
select cell_phone, n.id, 102 from oldparenttable o
join newparenttable n on n.oldid = o.id