I'm trying to translate the query from my question in SQL multiple rows as columns (optimizing). It is in MySQL but I need it to also run on a MS SQL Server.
One problem is that there is no GROUP_CONCAT in MS SQL, but there seems to be ways to simulate this however (Simulating group_concat MySQL function in Microsoft SQL Server 2005?).
Also, I can't find a way to to store the first SELECT statement into the #sql variable the same way which troubles me as I don't know how to then reference colkey as I currently do.
The MySQL statement:
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT('MAX(CASE
WHEN ckm.colkey = ', colkey, ' THEN
(ccdr.value)
END) AS ', CONCAT('`ExtraColumn_', colkey, '`'))
) INTO #sql
FROM test_customkeymapping;
SET #sql = CONCAT('SELECT c.Name, ', #sql, '
FROM customers c
LEFT JOIN customercustomdatarels ccdr
ON c.Id = ccdr.customer
LEFT JOIN customdatas cd
ON cd.Id = ccdr.customdata
LEFT JOIN test_customkeymapping ckm
ON cd.key = ckm.customkey
GROUP BY c.Id');
PREPARE stmt FROM #sql;
EXECUTE stmt;
In SQL Server you need to make the following changes
Explicitly declare your variable with a type
Use + to concatenate strings instead of CONCAT (Unless you are using SQL Server 2012 or later)
Use brackets ([]) for object names/aliases instead of backticks (``) - QUOTENAME will do this for you
Use XML extensions to concatenate rows
Include c.Name in the group by as it is contained in the select
Use SP_EXECUTESQL to actually execute your query
So your query becomes something like:
DECLARE #SQL NVARCHAR(MAX);
SET #SQL = 'SELECT c.Name' + ( SELECT DISTINCT
', MAX(CASE WHEN ckm.colkey = '
+ QUOTENAME(colKey AS VARCHAR(10))
+ ' THEN (ccdr.value) END) AS '
+ QUOTENAME('ExtraColumn_' + CAST(colKey AS VARCHAR(10))
FROM test_customkeymapping
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)') +
'FROM customers c
LEFT JOIN customercustomdatarels ccdr
ON c.Id = ccdr.customer
LEFT JOIN customdatas cd
ON cd.Id = ccdr.customdata
LEFT JOIN test_customkeymapping ckm
ON cd.[key] = ckm.customkey
GROUP BY c.ID, c.Name';
EXECUTE SP_EXECUTESQL #SQL;
Related
This question already has answers here:
How to get Max Date Value of Date column in Multiple tables
(4 answers)
Closed 12 months ago.
I want to find the max value of a column which is present in multiple tables. Eg.
SELECT MAX(UpdatedDatetime) FROM schema_1.table_1
UNION ALL
SELECT MAX(UpdatedDatetime) FROM schema_1.table_2
UNION ALL
SELECT MAX(UpdatedDatetime) FROM schema_2.table_1
UNION ALL
SELECT MAX(UpdatedDatetime) FROM schema_2.table_2
Clearly the above method is quite cumbersome when there are say more than 100 tables.
Is there any way to dynamically generate the table name as above and form the query string?
I mean using INFORMATION_SCHEMA.COLUMNS?
Below should do the work. You can always use String_AGG function. But in my sql server this function is not supported so using XML path.
Declare #SQL varchar(max)
SELECT #SQL=STUFF((SELECT ' ' + CAST(Name AS VARCHAR(max)) [text()]
FROM (
SELECT 'select Max(UpdatedDatetime) from ' + QUOTENAME(ss.name) + '.' + QUOTENAME(st.name) + ' union all' as Name
FROM sys.tables st
INNER JOIN sys.schemas ss on st.[schema_id] = ss.[schema_id]
WHERE st.is_ms_shipped = 0
AND EXISTS (
SELECT 1
FROM sys.columns sc
WHERE sc.[object_id] = st.[object_id]
AND sc.name = 'UpdatedDatetime'
)
) ap
FOR XML PATH(''), TYPE)
.value('.','NVARCHAR(MAX)'),1,2,' ')
set #SQL = left(#SQL, len(#SQL) - 10) -- remove last union all
select #SQL
EXEC (#SQL)
Based on questions like SQL to find the number of distinct values in a column and https://gis.stackexchange.com/questions/330932/get-line-length-using-sql-in-qgis
I see we can get a count and list of unique values using SQL but I can't see anything where we can do this without knowing the name of the field.
Is it possible in SQL for QGIS which only allows these commands? I found this option for another flavor -https://dataedo.com/kb/query/sql-server/list-table-columns-in-database
In Mapbasic I have used the following but would like to do this in SQL...
'Get Column Name list
dim x as integer
dim sColName as string
dim aColName as Alias
For x=1 to TableInfo(temptable, TAB_INFO_NCOLS)
sColName = ColumnInfo(temptable, "col"+str$(x), COL_INFO_NAME)
if (sColName not in ("GID","GID_New")) then
aColName = sColName
Select aColName, count(*) from temptable group by aColName into "g_"+sColName
Browse * from "g_"+sColName
Export "g_"+sColName Into WFolder+RSelection.col2+"_"+sColName+".csv" Type "ASCII" Delimiter "," CharSet "WindowsLatin1" Titles
End If
Next
I guess in SQL we would use http://www.sqlservertutorial.net/sql-server-basics/sql-server-select-distinct/ but how can I tell it to just use every column in the table without knowing/specifying the name?
UPDATE
If I run
SELECT DISTINCT * FROM Drainage_Lines_Clip;
I get
But I need something like the following without having to specify the column name. Ref
It should look like this extract from running Unique on a google sheet of the data (except with counts)
So this answer is based upon dynamic SQL. You'll get people saying 'don't use it it's dangerous', but they're the kind of people that think the best access to a system for users is none.. Anyway. Be aware of the security risks with SQL injection when using dynamic SQL. I'll leave that part up to you..
The below goes off to the sys.columns table and grabs all of the column names in the table, then a SQL statement is constructed to count all of the values in each column in your target table.
DECLARE #ReturnVar NVARCHAR(MAX);
SELECT #ReturnVar = COALESCE(#ReturnVar + ' UNION ALL ', '') + 'SELECT ''' + c.[name] + ''' [ColumnName], CAST(' + c.[name] + ' AS VARCHAR(MAX)) [ColumnValue], CAST(COUNT(1) AS VARCHAR(MAX)) [Count] FROM dbo.Admissions GROUP BY CAST(' + c.[name] + ' AS VARCHAR(MAX))'
FROM sys.columns c
INNER JOIN sys.objects o ON o.object_id = c.object_id
INNER JOIN sys.schemas s ON s.schema_id = o.schema_id
WHERE o.[name] = 'Drainage_Lines_Clip'
AND s.[name] = 'dbo'
AND c.[name] != 'GID_New';
EXEC sp_executesql #ReturnVar;
I ended up having to use a combination of PyQGIS and SQL to get what's needed.
layer = qgis.utils.iface.activeLayer()
fields=[] # List of fields
Lquery=[] # List of queries to join together with Union All statement
Cquery=[] # Combined Query to use
for field in layer.fields():
if field.name() not in ('GID_New'):
fields.append(field.name())
query = "Select '{0}' as 'Column', {0} as 'Value', count(*) as 'Unique' from {1} group by {0}".format(field.name(), layer.name())
Lquery.append(query)
else:
print (field.name())
# query = "Select {0}, count(*) from {1} group by {0} order by 2 Desc".format(field.name(), layer.name())
for L in Lquery:
Cquery.append(L+' Union All ')
query=''.join(map(str, Fquery))
query=query[:-11]+' Order by Column'
vlayer = QgsVectorLayer( "?query={}".format(query), 'counts_'+layer.name(), "virtual" )
QgsProject.instance().addMapLayer(vlayer)
How to do concatenation in SQL Server?
I wrote below query and its return always null value. I want to execute if condition for data exists or not
SET #Query = 'SELECT con.ID, BusinessName, FirstName, LastName, con.MobileNo, TelephoneNo,con.Email AS Email,
Address1, ISNULL(con.IsActive,0) AS IsActive, ISNULL(Status,0)AS Status, st.Name AS StatusName, ISNULL(st.ColorID,0)AS ColorID,
(SELECT TOP 1 cfu.Remark FROM CRM_ContactFollowUp cfu WHERE cfu.ContactID = con.ID ORDER BY ID DESC) AS Remark, con.CreatedBy,
emp.Name AS CreatedName,con.CreatedOn, con.DueDate
FROM CRM_Contacts con
LEFT JOIN CRM_Status st ON st.ID = con.Status
LEFT JOIN PMS_Employee emp ON emp.ID=con.CreatedBy
WHERE con.OrganizationID='+ Convert(Varchar,#OrganizationID)+' AND('+#WhereClause +') ORDER BY ID DESC OFFSET ('+ Convert(Varchar,#PageIndex) +')
ROWS FETCH NEXT '+ Convert(Varchar,#PageSize) +' ROWS ONLY';
SET #NextQuery ='IF(EXISTS (SELECT con.ID FROM CRM_Contacts con WHERE con.OrganizationID= '+ Convert(Varchar,#OrganizationID)+' AND
ORDER BY ID DESC OFFSET ('+ Convert(Varchar,#NextPage) +') ROWS FETCH NEXT '+ Convert(Varchar,#PageSize) +' ROWS ONLY) )
SET '+ Convert(Varchar,#IsMoreRecords)+'= 1
ELSE
SET '+ Convert(Varchar,#IsMoreRecords)+'= 0';
EXEC(#Query);
EXEC(#NextQuery);
Here is my complete procedure https://jsfiddle.net/npathak56/aLsngae5/
#Query execute properly but #NextQuery not Executed Why?
I did not run your procedure, as it is too complex to replicate locally, but I don't like this statement:
SET '+ Convert(Varchar,#IsMoreRecords)+'= 1
In my opinion it should be
SET #IsMoreRecords = 1
You want to assign an output variable, but you are using its value.And, of course, so for the second similar statement.
SET #IsMoreRecords = 0
Your most basic problem is that you can't concatenate strings that have single quotes inside them using that method. SQL will interpret those single quotes as part of the concatenation function instead of part of the string. So if you do:
SET #Query = 'SELECT [field] from [table]
where [field] like '%%whatevs%%'
and [field2] like ' + #var
You can see just with the Stack Overflow syntax highlighting that %%whatevs%% ends up outside the string because like uses single quotes. So your query will never work when written like that.
I have 3 tables and I want to return result as one dynamic table, and use it with an ASP.NET GridView. However when I run my query I get a syntax error:
Declare #AghlamTitle_Topic nvarchar(max)
Declare #query nvarchar(max)
Select
#AghlamTitle_Topic =
stuff((select distinct ','+QuoteName([TopicTitle])
from Tbl_Topic
where Topic_PID = 29
for xml path('')), 1, 1, '')
Set #Query = ' Select *
From (Select
t2.Aghlam_Marasemat_PID,
View_Marasem_ALL.MarasemID ,
t2.[AghlamDateReg],
t1.[TopicTitle],
t2.AghlamCount
from
Tbl_Aghlam_Num t2 View_Marasem_ALL
inner join
Tbl_Topic t1
pivot (max([AghlamCount]) for [TopicTitle] in ( ' +#AghlamTitle_Topic + ' ) ) p
inner join t2 on View_Marasem_ALL.MarasemID = t2.Aghlam_Marasemat_PID'
exec sp_executeSql #query
Does someone have a solution?
Tbl_Aghlam_Num t2 View_Marasem_ALL
This is a syntax error. You have a Table name followed by the alias name t2 followed by View_Marasem_ALL, whatever that is.
If you need to also join to View_Marasem_ALL, then you need to add an additional JOIN clause.
I'm looking for a schema-independent query. That is, if I have a users table or a purchases table, the query should be equally capable of catching duplicate rows in either table without any modification (other than the from clause, of course).
I'm using T-SQL, but I'm guessing there should be a general solution.
I believe that this should work for you. Keep in mind that CHECKSUM() isn't 100% perfect - it's theoretically possible to get a false positive here (I think), but otherwise you can just change the table name and this should work:
;WITH cte AS (
SELECT
*,
CHECKSUM(*) AS chksum,
ROW_NUMBER() OVER(ORDER BY GETDATE()) AS row_num
FROM
My_Table
)
SELECT
*
FROM
CTE T1
INNER JOIN CTE T2 ON
T2.chksum = T1.chksum AND
T2.row_num <> T1.row_num
The ROW_NUMBER() is needed so that you have some way of distinguishing rows. It requires an ORDER BY and that can't be a constant, so GETDATE() was my workaround for that.
Simply change the table name in the CTE and it should work without spelling out the columns.
I'm still confused about what "detecting them might be" but I'll give it a shot.
Excluding them is easy
e.g.
SELECT DISTINCT * FROM USERS
However if you wanted to only include them and a duplicate is all the fields than you have to do
SELECT
[Each and every field]
FROM
USERS
GROUP BY
[Each and every field]
HAVING COUNT(*) > 1
You can't get away with just using (*) because you can't GROUP BY *
so this requirement from your comments is difficult
a schema-independent means I don't want to specify all of the columns
in the query
Unless that is you want to use dynamic SQL and read the columns from sys.columns or information_schema.columns
For example
DECLARE #colunns nvarchar(max)
SET #colunns = ''
SELECT #colunns = #colunns + '[' + COLUMN_NAME +'], '
FROM INFORMATION_SCHEMA.columns
WHERE table_name = 'USERS'
SET #colunns = left(#colunns,len(#colunns ) - 1)
DECLARE #SQL nvarchar(max)
SET #SQL = 'SELECT ' + #colunns
+ 'FROM USERS' + 'GROUP BY '
+ #colunns
+ ' Having Count(*) > 1'
exec sp_executesql #SQL
Please note you should read this The Curse and Blessings of Dynamic SQL if you haven't already
I have done this using CTEs in SQL Server.
Here is a sample on how to delete dupes but you should be able to adapt it easily to find dupes:
WITH CTE (COl1, Col2, DuplicateCount)
AS
(
SELECT COl1,Col2,
ROW_NUMBER() OVER(PARTITION BY COl1,Col2 ORDER BY Col1) AS DuplicateCount
FROM DuplicateRcordTable
)
DELETE
FROM CTE
WHERE DuplicateCount > 1
GO
Here is a link to an article where I got the SQL:
http://blog.sqlauthority.com/2009/06/23/sql-server-2005-2008-delete-duplicate-rows/
I recently was looking into the same issue and noticed this question.
I managed to solve it using a stored procedure with some dynamic SQL. This way you only need to specify the table name. And it will get all the other relevant data from sys tables.
/*
This SP returns all duplicate rows (1 line for each duplicate) for any given table.
to use the SP:
exec [database].[dbo].[sp_duplicates]
#table = '[database].[schema].[table]'
*/
create proc dbo.sp_duplicates #table nvarchar(50) as
declare #query nvarchar(max)
declare #groupby nvarchar(max)
set #groupby = stuff((select ',' + [name]
FROM sys.columns
WHERE object_id = OBJECT_ID(#table)
FOR xml path('')), 1, 1, '')
set #query = 'select *, count(*)
from '+#table+'
group by '+#groupby+'
having count(*) > 1'
exec (#query)