I have a SQL query to load from a view into a table, it is running for more than 45 minutes now. I have checked if there is any blocking issue but this is the only query running and I am not using any index in a destination table. It only takes 5 minutes to create the underlying view. It has 5 million records.
INSERT INTO [dbo].[Table1]
SELECT
Name,
Service = STUFF ((SELECT ',' + Service
FROM Table2 T1
WHERE T1.Resource = T2.Resource
ORDER BY Service
FOR XML PATH('')), 1, 1, ''),
Tag = STUFF ((SELECT ',' + Tag
FROM Table2 T1
WHERE T1.Resource = T2.Resource
ORDER BY Tag
FOR XML PATH('')), 1, 1, ''),
Resource,
MAX(SubjectIdentifier) AS SubjectIdentifier,
DataType
FROM
Table2 T2 WITH (NOLOCK)
GROUP BY
Name, Resource, DataType
How can I optimize this query and load it fast? Thanks
Let me assume that you cannot use string_agg() because you are using an old version of SQL Server.
I would suggest rewriting your query a little bit and then adding indexes:
SELECT Name,
Service = STUFF((SELECT ',' + TT2.Service
FROM Table2 TT2
WHERE TT2.Resource = T2.Resource
ORDER BY TT2.Service
FOR XML PATH('')
), 1, 1, ''
),
Tag = STUFF((SELECT ',' + TT2.Tag
FROM Table2 TT2
WHERE TT2.Resource = T2.Resource
ORDER BY TT2.Tag
FOR XML PATH('')
), 1, 1, ''
),
Resource,
MAX(SubjectIdentifier) AS SubjectIdentifier,
DataType
FROM (SELECT DISTINCT Name, Resource, DataType
FROM Table2 T2
) T2;
Then, you want indexes on:
Table2(Name, Resource, DataType)
Table2(Resource, Service)
Table2(Resource, Tax)
In addition, it is suspicious that you are aggregating by three columns but only using one of them in the subqueries. I would expect all three in the subqueries.
Also notice that I qualified all column references.
Related
This question already has answers here:
String_agg for SQL Server before 2017
(2 answers)
Closed 2 years ago.
I want to rewrite my SQL query with stuff or other option available instead of string_agg function as my SQL server doesn't support it.
Can anyone please help me with this?
select String_agg(air.code,',') AS Code,String_agg(air.Id,',') AS AId,res.ResId
from Table1 air
inner join
Table2 rmap on air.Id=airmap.Id
inner join Table3 res on rmap.ResId=res.ResId
group by res.ResId
with output like below
Presumably, you want something like this:
select res.ResId,
stuff( (select ',' + air.code
from table1 air inner join
Table2 rmap
on air.Id = airmap.Id
where rmap.ResId = res.ResId
for xml path ('')
), 1, 1, '') as codes,
stuff( (select ',' + air.aid
from table1 air inner join
Table2 rmap
on air.Id = airmap.Id
where rmap.ResId = res.ResId
for xml path ('')
), 1, 1, '') as aids
from Table3 res ;
I have 4 columns in Table A viz., Inv_Num1, Inv_Date1, Inv_Amt1, Inv_DocNum1
I have 4 columns in Table B viz., Inv_Num2, Inv_Date2, Inv_Amt2, Inv_Status2
I would like to match the rows between Table A and Table B by using an inner join where condition on is
Invoice_Num1=Invoice_Num2 AND Invoice_Date1=Invoice_Date2 AND
Invoice_Amt1=Invoice_Amt2
When I do this matching I may get more than 1 row as a result in Table
A (Invoice_DocNum1 column)
I tried XML Path code but I dont know how to implement in Update statement
update cis2
set cis2.Inv_Status2 =
(SELECT
TypeName = STUFF((
SELECT '; ' + imd1.Inv_DocNum1
FROM [VRS].[Table_B] cis1
INNER JOIN [Table_A] imd1
ON cis1.Inv_Num1 = imd1.Inv_Num2
WHERE cis1.Inv_Num1 = imd1.Inv_Num2
AND cis1.Inv_Date1 = imd1.Inv_Date2
AND cis1.Inv_Amt1 = imd1.Inv_Amt2
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '')
) FROM Table_B cis2
Doing this to your database is against good practices since it violates 1NF. But you could still this if you are deadset on doing it. Something along these lines should work.
with myCte as
(
SELECT Inv_Num1
, TypeName = STUFF((
SELECT '; ' + imd1.Inv_DocNum1
FROM [VRS].[Table_B] cis1
INNER JOIN [Table_A] imd1
ON cis1.Inv_Num1 = imd1.Inv_Num2
WHERE cis1.Inv_Num1 = imd1.Inv_Num2
AND cis1.Inv_Date1 = imd1.Inv_Date2
AND cis1.Inv_Amt1 = imd1.Inv_Amt2
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '')
from Table_A
group by Inv_Num1
)
update tb
set Inv_Status2 = c.TypeName
from Table_B tb
join myCte c on c.Inv_Num1 = tb.Inv_Num2
The answer has two parts. First, you need to produce your comma-separated list per row. The best way to do it is STRING_AGG (https://learn.microsoft.com/en-us/sql/t-sql/functions/string-agg-transact-sql?view=sql-server-2017)
You will need to use it with the group by, like select ..., STRING_AGG(Inv_DocNum1, ',') group by ... where ... stands for your three fields forming unique key.
Second, you need to use update ... from syntax, see https://learn.microsoft.com/en-us/sql/t-sql/queries/update-transact-sql?view=sql-server-2017#l-specifying-a-table-alias-as-the-target-object. In your case, it will be from your target table joining the resultset you computed at step one.
What i have tried
Results wanted like thisFrom One table , for each ID there can be multiple email id's based on some condition Ex
ID EmailID's
1 Mike.Foster#Mail.com
1 lilly.Foster#Mail.com
2 Michel.Josh#Mail.com
2 Nash.Ted#Mail.com
I have to get email Name from these Ids from another table, something like this
Output i Need
Email_Name
Foster.Mike,Foster.Lilly
Josh.Michel,Ted.Nash
This is what i tried.
SELECT User_Email =
STUFF((SELECT DISTINCT ', ' + User_Email
FROM table1 b
WHERE b.Component_ID= a.Component_ID
and [Role] ='Team Lead' and Functional_Group ='Product'
FOR XML PATH('')), 1, 2, '')
FROM [WFS].table1 a
GROUP BY table1
Now another table i want Email Names
Select EmailNamefrom Table2 where EmailIDs IN ( 'code for Email')
Table1 schema
ID Component_ID EmailIDs Role Functional_Group
1 1 Mike.Foster#Mail.com Team Lead Product
2 1 lilly.Foster#Mail.com Team Lead Product
3 2 Michel.Josh#Mail.com Team Lead Product
4 2 Nash.Ted#Mail.com Team Lead Product
Table 2 schema
ID EmailIDs EmailName
1 Mike.Foster#Mail.com Foster.Mike
2 lilly.Foster#Mail.com Foster.Lilly
Any suggestions are welcome.Thanks in advance
Disregard this answer as I found out that GROUP_CONCAT() is a MySQL specific function, which means it won't work in SQL Server, however, I'll let it stay for future references.
SELECT
GROUP_CONCAT(EmailName SEPARATOR ', ') as name
FROM
table1
INNER JOIN table2 ON table1.EmailIDs=table2.EmailIDs
WHERE
table1.EmailIDs=table2.EmailIDs
GROUP BY
table1.Component_ID
Output:
Foster.Mike, Foster.Lilly
Ted.Nash, Josh.Michel
Working SQL fiddle
Docs: https://dev.mysql.com/doc/refman/8.0/en/group-by-functions.html#function_group-concat
You need CTE with STUFF() :
WITH t AS (
SELECT t1.*, t2.emailname
FROM table1 t1 INNER JOIN
table2 t2
ON t2.emailids = t1.emailids
)
SELECT id, STUFF ( (SELECT DISTINCT ','+t1.emailname
FROM t t1
WHERE t1.id = t.id
FOR XML PATH ('')
), 1, 1, ''
) AS Email_Name
FROM t
GROUP BY id;
You were actually close but your SQL doesn't match your schema. This one works as you want:
SELECT Email_Name =
STUFF((SELECT DISTINCT ', ' + EmailIDs
FROM table1 b
WHERE b.Component_ID= a.Component_ID
and [Role] ='Team Lead' and Functional_Group ='Product'
FOR XML PATH('')), 1, 2, '')
FROM table1 a
GROUP BY a.Component_ID;
EDIT: I didn't understand what you are asking exactly. Might this be what you meant?
SELECT STUFF((SELECT ', ' + EmailName
FROM Table2 where EmailIDs IN ( SELECT EmailIDs
FROM table1
WHERE [Role] ='Team Lead' and Functional_Group ='Product')
FOR XML PATH('')), 1, 2, '')
Or you meant this:
SELECT DISTINCT Component_ID, emailNames
FROM table1
CROSS APPLY (SELECT STUFF((SELECT ', '+t2.EmailName
FROM table2 t2
INNER JOIN TABLE1 t1 ON t1.EmailIDs = t2.EmailIDs
WHERE t1.Component_ID = Table1.Component_ID
FOR XML PATH('')), 1, 2, '')
) t(EmailNames)
WHERE [Role] ='Team Lead' and Functional_Group ='Product'
Here is SQLFiddle Demo
I am trying to only display the rows in which there is date for Researchers.
I cannot manage to omit the rows with Null Values. I even tried this solution How to remove null rows from sql query result?..
This is my Query:
SELECT Submission.Title AS [Submission_Title], CA.Surname AS [Researchers], Submission.Status AS [Status]
FROM Submission
CROSS APPLY (SELECT STUFF((SELECT DISTINCT ', ' + r.Surname
FROM ResearcherSubmission rs INNER JOIN Researcher r
ON r.ResearcherID = rs.ResearcherID
WHERE CONCAT (DATENAME(MONTH,[Submission].[SubmissionDate]), ' ',DATEPART (YEAR,[Submission].[SubmissionDate])) = 'October 2015'
AND Submission.SubmissionID = rs.SubmissionID
FOR XML PATH (''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, ' ')) AS CA (Surname)
GROUP BY convert(varchar(10),datename(month,Submission.SubmissionDate)), Submission.Title, CA.Surname, Submission.Status;
This is my Current output:
any suggestion. Thank you
Quickfix, without reading query:
WITH cte AS
(
SELECT Submission.Title AS [Submission_Title], CA.Surname AS [Researchers], Submission.Status AS [Status]
FROM Submission
CROSS APPLY (SELECT STUFF((SELECT DISTINCT ', ' + r.Surname
FROM ResearcherSubmission rs INNER JOIN Researcher r
ON r.ResearcherID = rs.ResearcherID
WHERE CONCAT (DATENAME(MONTH,[Submission].[SubmissionDate]), ' ',DATEPART (YEAR,[Submission].[SubmissionDate])) = 'October 2015'
AND Submission.SubmissionID = rs.SubmissionID
FOR XML PATH (''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, ' ')) AS CA (Surname)
GROUP BY convert(varchar(10),datename(month,Submission.SubmissionDate)), Submission.Title, CA.Surname, Submission.Status
)
SELECT *
FROM cte
WHERE Researchers IS NOT NULL;
There is probably more elegant solution, but you need to share sample data and structures.
This part may cause problems:
SELECT DISTINCT ', ' + r.Surname
try with CONCAT instead or :
SELECT DISTINCT ', ' + ISNULL(r.Surname, '')
You should filter out the researchers before the group by rather than afterwards. When possible, it is better (performance-wise) to put conditions before aggregation.
SELECT s.Title AS Submission_Title, CA.Surname AS Researchers, s.Status
FROM Submission s CROSS APPLY
(SELECT STUFF((SELECT DISTINCT ', ' + r.Surname
FROM ResearcherSubmission rs INNER JOIN
Researcher r
ON r.ResearcherID = rs.ResearcherID
WHERE s.SubmissionID = rs.SubmissionID
FOR XML PATH (''), TYPE).value('.', 'NVARCHAR(MAX)'
), 1, 2, ' '))
) AS CA(Surname)
WHERE s.SubmissionDate >= '2015-10-01' AND s.SubmissionDate < '2015-11-01' AND
ca.Surname IS NULL
GROUP BY YEAR(s.SubmissionDate), MONTH(s.SubmissionDate), s.Title, CA.Surname, s.Status;
Note the changes made:
Table aliases make the query easier to write and to read.
I changed the date comparison to have no functions on the date itself. This would allow SQL Server to use an index, if appropriate.
I also moved the date comparison from the CROSS APPLY subquery to the outer query. This could be a big gain in efficiency. Why do the extra work for rows that will be filtered out anyway?
I added the NOT NULL condition to the WHERE clause.
The date key in the outer GROUP BY is redundant because the query is only using one month of data. I simplified the logic but left it.
In sql (MS sql server specifically) is it possible to combine multiple rows into a single string as an expression which is itself part of an update that is being applied to multiple rows. I have come across the approaches of using COALESCE or FOR XML PATH (e.g. How to get multiple rows into one line as a string? ) but can't get them to work in my more complex case with the extra dimension of 'listiness'.
My problem boils down to, in words:
A Project has some Launches. A Launch has a LaunchType and a date. I have a big output table of projects ProjectOutput and I want to update a column in it with a CSV string of all the launch type names for that project that happen in the same month as the first (chronologically) launch of that project.
In sql:
UPDATE ProjectOutput
SET LaunchNamesColumn = <INSERT MAGICAL SQL STRING CONCATTING ACROSS ROWS FUNCTION HERE> of Launch.name
FROM ProjectOuput
INNER JOIN Launch ON Launch.projectId = ProjectOutput.projectId
INNER JOIN LaunchType AS lt ON LaunchType.launchTypeId = Launch.launchTypeId
OUTER APPLY (
SELECT TOP 1 Launch.month, Launch.year
FROM Launch
INNER JOIN Project ON Project.projectId = Launch.projectId
WHERE Project.projectId = ProjectOutput.projectId
--In reality there's loads more JOINS and WHERE conditions here
ORDER BY Launch.date
) firstLaunch
WHERE Launch.month = firstLaunch.month AND Launch.year = firstLaunch.year
If there were only 1 Launch per Project then the stuff would not be needed and just
SET LaunchNameColumn = Launch.name
However as there can be several Launches per Project some operation is needed to join them. I tried:
SET LaunchNamesColumn = STUFF((SELECT ', ' + lt.name FROM lt FOR XML PATH('')), 1, 2, '')
However that doesn't work (error, invalid name) because it doesn't know what the alias lt is inside that SELECT. If you just say LaunchType or dbo.LaunchType then the query runs but then you are just looping over all the possible launch types rather than only those returned by the big query below. What I really want is for that FROM in the SELECT FOR XML PATH is to be the result set of the giant query below (whereas in all the examples I've found so far it's just a simple table), but copying and pasting that in seems so wrong. Maybe there is some mental block or sql feature I'm unaware of that would make this work, or is it not possible?
The problem you have is that in the SET stage of your query you only have access to one of the matching Launches as there is no grouping applied.
You can achieve want you want by moving your Launch lookup into a sub-query over the ProjectOutput rows. A simplified example:
UPDATE ProjectOutput
SET LaunchNamesColumn = STUFF((
SELECT ', ' + Launch.name
FROM Launch
-- OUTER APPLY is not required within the sub-query.
INNER JOIN (
SELECT TOP 1 Launch.month, Launch.year
FROM Launch
-- Filter results to specific project.
WHERE Launch.projectId = ProjectOutput.projectId
ORDER BY Launch.date
) firstLaunch ON Launch.month = firstLaunch.month AND Launch.year = firstLaunch.year
-- Filter results to specific project.
WHERE Launch.projectId = ProjectOutput.projectId
FOR XML PATH('')
), 1, 2, '')
FROM ProjectOutput
Logically the sub query is run once per ProjectOutput record, allowing you to filter and group by each ProjectId.
Also nice bit of syntax that may simplify your query is SELECT TOP WITH TIES,
UPDATE ProjectOutput
SET LaunchNamesColumn = STUFF((
SELECT TOP (1) WITH TIES ', ' + Launch.name
FROM Launch
WHERE Launch.projectId = ProjectOutput.projectId
ORDER BY Launch.Year, Launch.Month
FOR XML PATH('')
), 1, 2, '')
FROM ProjectOutput
This will return all the matching Launches that have the lowest Year then Month value.
It's a little bit difficult to understand your SQL without description of the tables, but what you should do is have the query with the XML path so that it returns only those items that you want to be concatenated for that single row, so my guess is that you want actually something like this:
UPDATE O
SET LaunchNamesColumn = STUFF((SELECT ', ' + lt.Name
From Launch L
INNER JOIN Launch L ON L.projectId = O.projectId
INNER JOIN LaunchType AS lt ON lt.launchTypeId = L.launchTypeId
WHERE L.month = FL.month AND L.year = FL.year
FOR XML PATH('')), 1, 2, '')
FROM ProjectOutput O
CROSS APPLY (
SELECT TOP 1 L2.month, L2.year
FROM Launch L2
WHERE L2.projectId = O.projectId
-- Removed the other tables from here. Are they really needed?
ORDER BY L2.date
) FL
Couldn't really test this, but hopefully this helps.
Can you add the Launch and LaunchType tables into your STUFF and filter it based on the Project table or Launch table in the main query?
STUFF((SELECT ', ' + lt.name
FROM Launch l
JOIN LaunchType lt2 ON lt2.launchTypeId = l.launchTypeId
WHERE
l.projectId = Launch.projectId
FOR XML PATH('')), 1, 2, '')
Or you could maybe create a CTE and select all of the launches then use your Stuff statement on the CTE
WITH cteLaunch AS (
SELECT l.projectId,
lt.NAME
FROM Launch ON Launch.projectId = ProjectOutput.projectId
INNER JOIN LaunchType AS lt ON LaunchType.launchTypeId = Launch.launchTypeId
OUTER APPLY (SELECT TOP 1
Launch.month,
Launch.year
FROM
Launch
INNER JOIN Project ON Project.projectId = Launch.projectId
WHERE
Project.projectId = ProjectOutput.projectId
ORDER BY Launch.date
) firstLaunch
WHERE Launch.month = firstLaunch.month
AND Launch.year = firstLaunch.year
)
UPDATE
ProjectOutput
SET
LaunchNamesColumn = STUFF((SELECT ', ' + lt.name
FROM cteLaunch cte
WHERE cte.projectId = ProjectOuput.projectId
FOR XML PATH('')), 1, 2, '')
FROM
ProjectOuput
INNER JOIN cteLaunch ON cteLaunch.projectId = ProjectOutput.projectId
I think you are really close; it's the alias getting in the way:
SET LaunchNamesColumn = STUFF((SELECT ', ' + lt.name
FROM LaunchType AS lt
WHERE lt.launchTypeId = Launch.launchTypeId FOR XML PATH('')), 1, 2, '')