Dynamic delete based on update table - sql

I want to write a dynamic script that removes duplicates. I want to try and avoid a CURSOR so I've been looking into writing strings instead that will have table in one column and corresponding table attributes in another. I have also tried dynamic SQL using WITH. But this is what I have so far. This I intend to use as parameters in dynamic SQL later on
STUFF example. However this results in repeating the same column names for every row:
select name as table_name,
stuff(( select ', ' +char(10)+ ac.[name] FROM DW.sys.columns ac
inner join DW.sys.tables t on ac.object_id=t.object_id
where ac.name not in ('ModifiedOn','ValidFrom','ValidTo')
FOR XML PATH('')
), 1, 1, '')
from sys.tables
What I want is this output:
TableName || ColumnName
table1 || aa,ab,ac
table2 || ba,bb,bc
table3 || ca,cb,cc
My idea is to use this to this effect or similair:
'WITH DELETEDUPLICATE AS (
SELECT '+#ColumnName+',
ROW_NUMBER() OVER(PARTITION BY '+#ColumnName+' ORDER BY '+#ColumnName+') AS Duplicate_Row_Count
FROM '+#TableName+'
)
DELETE
FROM DELETEDUPLICATE
WHERE Duplicate_Row_Count > 1
Any ideas appreciated!
UPDATE:
With satishcse's suggestion i get the table I wanted. I had problem with getting multiple rows in the dynamic WITH step so I just removed that part as a varaible (removed away 'SET #WITH =' ). But how to execute every row? what i get now is:
WITH DELETEDUPLICATE AS(....
For every table per row

In OpenQuery you have to run the query using execute() function. The answer can solve your problem, but I do not suggest you use OpenQuery.
declare #query as nvarchar(max)
set
#query =
'WITH DELETEDUPLICATE AS (
SELECT '+#ColumnName+',
ROW_NUMBER() OVER(PARTITION BY '+#ColumnName+' ORDER BY '+#ColumnName+') AS Duplicate_Row_Count
FROM '+#TableName+'
)
DELETE
FROM DELETEDUPLICATE
WHERE Duplicate_Row_Count > 1'
execute(#query)

try the following for the first part:
select name as table_name,
stuff(( select ', ' +char(10)+ ac.[name] FROM DW.sys.columns ac
inner join DW.sys.tables t on ac.object_id=t.object_id
where ac.name not in ('ModifiedOn','ValidFrom','ValidTo')
and st.name = t.name
order by 1
FOR XML PATH('')
), 1, 1, '')
from sys.tables st

Related

SQL Search for Data in Multiple Columns

Dears,
I have a table as shown below as a sample, and I want to run one query by which i can find all the yellow highlighted ones by using %AAA%.
Instead of running the Where command on each column one by one, I can do one general find option and it will list all the rows.
Thank you in advance!!
You can include all the conditions in one where clause using or:
where col1 like '%aaa%' or
col2 like '%aaa%' or
. . . -- and so on for all the columns
Unpivot the columns and do a WHERE based on that:
select *
from Table
where exists (select 1
from (values (col1), (col2), (col3) ) AS v (allCols) -- etc
where v.allCols like '%aaa%'
);
If you can't be bothered to type them out, try this little query:
select STRING_AGG('(' + c.name + ')', ', ')
from sys.columns c
where c.object_id = OBJECT_ID('Name_Of_Table_Here');
If you are using sql server then you can write dynamic query to do so. Please try below query:
declare #sql as varchar(max);
select #sql = 'select * from [TableName] where '
+ stuff((
select ' or [' + [column_name] + '] like ''%AAA%'''
from information_schema.columns
where table_name = 'TableName'
for xml path('')
)
, 1, 5, ''
);
exec(#sql);
This query will return every row in which at least one column contains AAA.
If you are using PostgreSQL, you can use its JSON functionality:
select t.*
from the_table t
where exists (select *
from jsonb_each(to_jsonb(t)) as x(col,val)
where val like '%AAA%');
If you are using Postgres 12 or later you can use a SQL/JSON path expression:
select t.*
from the_table t
where to_jsonb(t) ## '$.* like_regex "AAA" flag "i"'

sql server column value to be converted in comma seperated

Before this question is marked as duplicate, i know how it can be done but without doing a declare statement i want to do it within a query itself
like i have this query
select distinct costcenterid,costcentername,costcenterdesc,contactid,expirationdate,portal_id,
active,customername,branchid,id from costcenter cc
inner join branchesinportals bp on bp.portalid = cc.portal_id
the branchid and the id fields have different values but all other rows have same values so if i remove those and do a distinct it works good, i get one record
i want that it should always return me one record and combine the columns branchid and id as a comma separated values
i tried looking a this link which seems to be working but how can i integrate that link code with query
http://www.codeproject.com/Tips/635166/SQL-Column-Values-as-Comma-Separated-String
You can use FOR XML to solve this problem. Here is a list of column names (you can run it in any SQL Server Database):
Select Stuff((
Select ', ' + cast(COLUMN_NAME as varchar(max))
From INFORMATION_SCHEMA.COLUMNS
For XML PATH('')
), 1, 2, '');
Here is how to have a one-to-many value set show up:
Select Distinct C1.TABLE_NAME,
Stuff((
Select ', ' + Cast(COLUMN_NAME as VarChar (Max))
From INFORMATION_SCHEMA.COLUMNS C2
Where C1.TABLE_NAME = C2.TABLE_NAME
For Xml Path ('')
), 1, 2, '') Columns
From INFORMATION_SCHEMA.COLUMNS C1
Here is the output from my master database tables and columns:

Dynamic Pivot in MS SQL Server

I am trying to do a dynamic pivot on the last two columns that i take from one table and am joining onto the contents of another table. I need the Name values to pivot to the header fields and the Value values to fill in correspondingly underneath. This is my current query:
USE Innovate
DECLARE #DynamicPivotQuery AS NVARCHAR(MAX)
DECLARE #ColumnName AS NVARCHAR(MAX)
--Get distinct values of the PIVOT Column
SELECT #ColumnName= ISNULL(#ColumnName + ',','')
+ QUOTENAME(NAME)
FROM (SELECT DISTINCT NAME FROM Innovate.dbo.Table1 WHERE Name IS NOT NULL) AS ATTRIBUTE_NAME
WHERE Name LIKE 'Suture_-_2nd_Needle_Code'
OR Name LIKE 'Suture_-_Absorbable'
OR Name LIKE 'Suture_-_Antibacterial'
OR Name LIKE 'Suture_-_Armed'
OR Name LIKE 'Suture_-_Barbed'
OR Name LIKE 'Suture_-_Brand_Name'
OR Name LIKE 'Suture_-_C/R_2nd_Needle_Code'
OR Name LIKE 'Suture_-_C/R_Brand_Name'
OR Name LIKE 'Suture_-_C/R_length'
OR Name LIKE 'Suture_-_C/R_Needle_Code'
OR Name LIKE 'Suture_-_Coating'
OR Name LIKE 'Suture_-_Dyed'
OR Name LIKE 'Suture_-_Filament'
OR Name LIKE 'Suture_-_length_inches'
OR Name LIKE 'Suture_-_Looped'
OR Name LIKE 'Suture_-_Material'
OR Name LIKE 'Suture_-_Needle_Code'
OR Name LIKE 'Suture_-_Needle_Shape'
OR Name LIKE 'Suture_-_Needle_Style'
OR Name LIKE 'Suture_-_Noun'
OR Name LIKE 'Suture_-_pleget'
OR Name LIKE 'Suture_-_Popoff'
OR Name LIKE 'Suture_-_Suture_count'
OR Name LIKE 'Suture_-_Suture_size'
--Prepare the PIVOT query using the dynamic
SET #DynamicPivotQuery =
N'SELECT Table1.Primary_Key, Company_Name, Part_Number, Product_Desc, Innovate_Description, ' + #ColumnName + '
FROM Table1 AS P
LEFT JOIN Table2 AS A ON P.Primary_Key = A.Primary_Key
PIVOT(MAX(A.VALUE)
FOR A.NAME IN (' + #ColumnName + ')) AS PVTTable'
--Execute the Dynamic Pivot Query
EXECUTE sp_executesql #DynamicPivotQuery
;
And this is the Result for the Query that I keep getting: Msg 8156,
Level 16, State 1, Line 5 The column 'Primary_Key' was specified
multiple times for 'PVTTable'. Msg 4104, Level 16, State 1, Line 1 The
multi-part identifier "Table1.Primary_Key" could not be bound.
Can anyone help me pivot these columns without the error message? I only specified the Primary_Key in the code once so I do not know how I specified it multiple times and how it is unbound.
Try with the below script..
SET #DynamicPivotQuery =
N'SELECT P.Primary_Key, Company_Name, Part_Number, Product_Desc, Innovate_Description, ' + #ColumnName + '
FROM Table1 AS P
LEFT JOIN Table2 AS A ON P.Primary_Key = A.Primary_Key
PIVOT(MAX(A.VALUE)
FOR A.NAME IN (' + #ColumnName + ')) AS PVTTable'
--Execute the Dynamic Pivot Query
EXECUTE sp_executesql #DynamicPivotQuery
you have a couple of things going on with your PIVOT statement that are problematic. First you are attempting to reference table aliases for Table1 and Table2 but those aliases are not available in the final select of a PIVOT the Pivot command is kind of like an outer select and the only table alias that is then available is the pivot alias.
Next pivot's documenation states "You can use the PIVOT and UNPIVOT relational operators to change a table-valued expression into another table" (https://technet.microsoft.com/en-us/library/ms177410(v=sql.105).aspx). Basically that means that you need a single table expression of only the uniquely named columns you want to be involved pivot being passed to the PIVOT command. The later part is likely the issue as Table1 and Table2 probably both have a column Primary_Key so pivot doesn't understand the reference.
To fix you can either move your Table1 & 2. join to an inner select and alias the table or build a cte and use the cte in your command. Here is the former way:
SET #DynamicPivotQuery =
N'SELECT * FROM
(
SELECT Table1.Primary_Key, Company_Name, Part_Number, Product_Desc, Innovate_Description, ' + #ColumnName + '
FROM Table1 AS P
LEFT JOIN Table2 AS A ON P.Primary_Key = A.Primary_Key
) t
PIVOT(MAX(A.VALUE)
FOR NAME IN (' + #ColumnName + ')) AS PVTTable'
Those Name values get hardcoded to calculate that #ColumnName variable.
If the values are hardcoded anyway, then you might as well run the pivot with a join without building a SQL statement to execute.
SELECT A.Company_Name, A.Part_Number, A.Product_Desc, A.Innovate_Description, P.*
FROM (select Primary_Key, Name, Value from Innovate.dbo.Table1) T1
PIVOT(MAX(VALUE) FOR NAME IN (
[Suture_-_2nd_Needle_Code],
[Suture_-_Absorbable],
[Suture_-_Antibacterial],
[Suture_-_Armed],
[Suture_-_Barbed],
[Suture_-_Brand_Name],
[Suture_-_C/R_2nd_Needle_Code],
[Suture_-_C/R_Brand_Name],
[Suture_-_C/R_length],
[Suture_-_C/R_Needle_Code],
[Suture_-_Coating],
[Suture_-_Dyed],
[Suture_-_Filament],
[Suture_-_length_inches],
[Suture_-_Looped],
[Suture_-_Material],
[Suture_-_Needle_Code],
[Suture_-_Needle_Shape],
[Suture_-_Needle_Style],
[Suture_-_Noun],
[Suture_-_pleget],
[Suture_-_Popoff],
[Suture_-_Suture_count],
[Suture_-_Suture_size]
)
) P
LEFT JOIN Innovate.dbo.Table2 AS A ON (A.Primary_Key = P.Primary_Key);
Fair enough, this has a disadvantage that if that [Primary_Key] needs to be the first column, that the P.* should be replaced by those literal column values. Or use the EXEC approach after all.
Anyway, to build name list for that #ColumnName variable, it can be done without all the OR's :
DECLARE #ColumnName NVARCHAR(MAX);
--Get distinct values of the PIVOT Column
SELECT #ColumnName = ISNULL(#ColumnName + ',','') + QUOTENAME(NAME)
FROM Innovate.dbo.Table1
WHERE Name Like 'Suture_-_%'
AND SUBSTRING(Name,10,30) IN (
'2nd_Needle_Code',
'Absorbable',
'Antibacterial',
'Armed',
'Barbed',
'Brand_Name',
'C/R_2nd_Needle_Code',
'C/R_Brand_Name',
'C/R_length',
'C/R_Needle_Code',
'Coating',
'Dyed',
'Filament',
'length_inches',
'Looped',
'Material',
'Needle_Code',
'Needle_Shape',
'Needle_Style',
'Noun',
'pleget',
'Popoff',
'Suture_count',
'Suture_size')
GROUP BY Name;

Convert a row as column and merge two column as its value

I have stuck in a select statement, converting rows into columns. I have tried with PIVOT, i was able to convert the single column. But my requirement is little different. I have explained the requirement below.
I have a table structure as below,
I want to select the data as below,
The values in the table are dynamic, which is not a problem for me to deal with that. But i need a way to get the below result.
Could someone please give me a hint on doing it, may be a way to modify the PIVOT below.
select *
from
(
select TSID,AID,Count,BID
from tbl TS
WHERE TS.TPID = 1
) src
pivot
(
sum(Count)
for AID in (AID1,AID2,AID3)
) piv
Thank you..
You may check this fiddle
EDIT
This will work for not previously known column names
DECLARE #Columns AS VARCHAR(MAX)
DECLARE #SQL AS VARCHAR(MAX)
SELECT #Columns = STUFF(( SELECT DISTINCT ',' + AID
FROM Table1
FOR
XML PATH('')
), 1, 1, '')
SET #SQL = '
;WITH MyCTE AS
(
SELECT TSID,
AID,
STUFF(( SELECT '','' + CONVERT(VARCHAR,[Count] )
FROM Table1 I Where I.TSID = O.TSID
FOR
XML PATH('''')
), 1, 1, '''') AS CountList
FROM Table1 O
GROUP BY TSID,
AID
)
SELECT *
FROM MyCTE
PIVOT
(
MAX(CountList)
FOR AID IN
(
' + #Columns + '
)
) AS PivotTable'
EXEC(#SQL)

SQL query to find duplicate rows, in any table

I'm looking for a schema-independent query. That is, if I have a users table or a purchases table, the query should be equally capable of catching duplicate rows in either table without any modification (other than the from clause, of course).
I'm using T-SQL, but I'm guessing there should be a general solution.
I believe that this should work for you. Keep in mind that CHECKSUM() isn't 100% perfect - it's theoretically possible to get a false positive here (I think), but otherwise you can just change the table name and this should work:
;WITH cte AS (
SELECT
*,
CHECKSUM(*) AS chksum,
ROW_NUMBER() OVER(ORDER BY GETDATE()) AS row_num
FROM
My_Table
)
SELECT
*
FROM
CTE T1
INNER JOIN CTE T2 ON
T2.chksum = T1.chksum AND
T2.row_num <> T1.row_num
The ROW_NUMBER() is needed so that you have some way of distinguishing rows. It requires an ORDER BY and that can't be a constant, so GETDATE() was my workaround for that.
Simply change the table name in the CTE and it should work without spelling out the columns.
I'm still confused about what "detecting them might be" but I'll give it a shot.
Excluding them is easy
e.g.
SELECT DISTINCT * FROM USERS
However if you wanted to only include them and a duplicate is all the fields than you have to do
SELECT
[Each and every field]
FROM
USERS
GROUP BY
[Each and every field]
HAVING COUNT(*) > 1
You can't get away with just using (*) because you can't GROUP BY *
so this requirement from your comments is difficult
a schema-independent means I don't want to specify all of the columns
in the query
Unless that is you want to use dynamic SQL and read the columns from sys.columns or information_schema.columns
For example
DECLARE #colunns nvarchar(max)
SET #colunns = ''
SELECT #colunns = #colunns + '[' + COLUMN_NAME +'], '
FROM INFORMATION_SCHEMA.columns
WHERE table_name = 'USERS'
SET #colunns = left(#colunns,len(#colunns ) - 1)
DECLARE #SQL nvarchar(max)
SET #SQL = 'SELECT ' + #colunns
+ 'FROM USERS' + 'GROUP BY '
+ #colunns
+ ' Having Count(*) > 1'
exec sp_executesql #SQL
Please note you should read this The Curse and Blessings of Dynamic SQL if you haven't already
I have done this using CTEs in SQL Server.
Here is a sample on how to delete dupes but you should be able to adapt it easily to find dupes:
WITH CTE (COl1, Col2, DuplicateCount)
AS
(
SELECT COl1,Col2,
ROW_NUMBER() OVER(PARTITION BY COl1,Col2 ORDER BY Col1) AS DuplicateCount
FROM DuplicateRcordTable
)
DELETE
FROM CTE
WHERE DuplicateCount > 1
GO
Here is a link to an article where I got the SQL:
http://blog.sqlauthority.com/2009/06/23/sql-server-2005-2008-delete-duplicate-rows/
I recently was looking into the same issue and noticed this question.
I managed to solve it using a stored procedure with some dynamic SQL. This way you only need to specify the table name. And it will get all the other relevant data from sys tables.
/*
This SP returns all duplicate rows (1 line for each duplicate) for any given table.
to use the SP:
exec [database].[dbo].[sp_duplicates]
#table = '[database].[schema].[table]'
*/
create proc dbo.sp_duplicates #table nvarchar(50) as
declare #query nvarchar(max)
declare #groupby nvarchar(max)
set #groupby = stuff((select ',' + [name]
FROM sys.columns
WHERE object_id = OBJECT_ID(#table)
FOR xml path('')), 1, 1, '')
set #query = 'select *, count(*)
from '+#table+'
group by '+#groupby+'
having count(*) > 1'
exec (#query)