how to omit null values using SQL query - sql

I am trying to only display the rows in which there is date for Researchers.
I cannot manage to omit the rows with Null Values. I even tried this solution How to remove null rows from sql query result?..
This is my Query:
SELECT Submission.Title AS [Submission_Title], CA.Surname AS [Researchers], Submission.Status AS [Status]
FROM Submission
CROSS APPLY (SELECT STUFF((SELECT DISTINCT ', ' + r.Surname
FROM ResearcherSubmission rs INNER JOIN Researcher r
ON r.ResearcherID = rs.ResearcherID
WHERE CONCAT (DATENAME(MONTH,[Submission].[SubmissionDate]), ' ',DATEPART (YEAR,[Submission].[SubmissionDate])) = 'October 2015'
AND Submission.SubmissionID = rs.SubmissionID
FOR XML PATH (''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, ' ')) AS CA (Surname)
GROUP BY convert(varchar(10),datename(month,Submission.SubmissionDate)), Submission.Title, CA.Surname, Submission.Status;
This is my Current output:
any suggestion. Thank you

Quickfix, without reading query:
WITH cte AS
(
SELECT Submission.Title AS [Submission_Title], CA.Surname AS [Researchers], Submission.Status AS [Status]
FROM Submission
CROSS APPLY (SELECT STUFF((SELECT DISTINCT ', ' + r.Surname
FROM ResearcherSubmission rs INNER JOIN Researcher r
ON r.ResearcherID = rs.ResearcherID
WHERE CONCAT (DATENAME(MONTH,[Submission].[SubmissionDate]), ' ',DATEPART (YEAR,[Submission].[SubmissionDate])) = 'October 2015'
AND Submission.SubmissionID = rs.SubmissionID
FOR XML PATH (''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, ' ')) AS CA (Surname)
GROUP BY convert(varchar(10),datename(month,Submission.SubmissionDate)), Submission.Title, CA.Surname, Submission.Status
)
SELECT *
FROM cte
WHERE Researchers IS NOT NULL;
There is probably more elegant solution, but you need to share sample data and structures.
This part may cause problems:
SELECT DISTINCT ', ' + r.Surname
try with CONCAT instead or :
SELECT DISTINCT ', ' + ISNULL(r.Surname, '')

You should filter out the researchers before the group by rather than afterwards. When possible, it is better (performance-wise) to put conditions before aggregation.
SELECT s.Title AS Submission_Title, CA.Surname AS Researchers, s.Status
FROM Submission s CROSS APPLY
(SELECT STUFF((SELECT DISTINCT ', ' + r.Surname
FROM ResearcherSubmission rs INNER JOIN
Researcher r
ON r.ResearcherID = rs.ResearcherID
WHERE s.SubmissionID = rs.SubmissionID
FOR XML PATH (''), TYPE).value('.', 'NVARCHAR(MAX)'
), 1, 2, ' '))
) AS CA(Surname)
WHERE s.SubmissionDate >= '2015-10-01' AND s.SubmissionDate < '2015-11-01' AND
ca.Surname IS NULL
GROUP BY YEAR(s.SubmissionDate), MONTH(s.SubmissionDate), s.Title, CA.Surname, s.Status;
Note the changes made:
Table aliases make the query easier to write and to read.
I changed the date comparison to have no functions on the date itself. This would allow SQL Server to use an index, if appropriate.
I also moved the date comparison from the CROSS APPLY subquery to the outer query. This could be a big gain in efficiency. Why do the extra work for rows that will be filtered out anyway?
I added the NOT NULL condition to the WHERE clause.
The date key in the outer GROUP BY is redundant because the query is only using one month of data. I simplified the logic but left it.

Related

SELECT STUFF rewrite to Redshift

I have a SQL script, that I need to convert to redshift.
Here is the part, where I have a problem:
LEFT JOIN
(
SELECT STUFF((
SELECT ','+ clo.name
FROM public.label_entities cl
JOIN public.label_history clo
ON clo.id = cl.labelid
WHERE clo.parentid = 993
AND cl.entityid = clv.contactid
FOR XML PATH('')
) ,1,1,'') AS Services
) AS labelServices
I have read that I can use SELECT LISTAGG and try to use it like this:
LEFT JOIN
(
SELECT LISTAGG((
SELECT ','+ clo.name
FROM public.label_entities cl
JOIN public.label_history clo
ON clo.id = cl.labelid
WHERE clo.parentid = 993
AND cl.entityid = clv.contactid
FOR XML PATH('')
) ,1,1,'') AS Services
) AS labelServices
But it does not work.
So how I can rewrite it to be correct?
You don't need all the XML stuff. In fact, it is XML that is doing the aggregation in SQL Server, not STUFF(). STUFF() is just used for beautifying the string after it is created.
So, something like this:
LEFT JOIN
(SELECT cl.entityid, LISTAGG(clo.name, ', ') WITHIN GROUP (ORDER BY clo.name) as names
FROM public.label_entities cl JOIN
public.label_history clo
ON clo.id = cl.labelid
WHERE clo.parentid = 993
GROUP BY cl.entityid
) AS labelServices
ON labelServices.entityid = clv.contactid
use replace() instead of stuff()
select STUFF(', hai, hello, fine', 1, 1, '')
select replace(','+', hai, hello, fine', ',,', '') --- ', hai, hello, fine' would be
--- returned by inner select
EDIT 1
select REPLACE(','+
(
SELECT ','+ clo.name
FROM public.label_entities cl
JOIN public.label_history clo
ON clo.id = cl.labelid
WHERE clo.parentid = 993
AND cl.entityid = clv.contactid
FOR XML PATH('')
)
,',,'
,''
)

The multi-part identifier could not be bound - stuff cmd

I'm attempting a Stuff Cmd to combine multiple rows to a single entry. I keep getting "The multi-part identifier "SPCLT.CD_VAL_DESC" could not be bound." (under the first SELECT statement)
STUFF(
(SELECT
**',' + SPCLT.CD_VAL_DESC**
FROM
(
SELECT DISTINCT
SPCLT.CD_VAL_DESC SPECIALTY
FROM PIN_STATUS PS
INNER JOIN PROV_TYPE_SPCLT SPC
ON PS.PROV_ID = SPC.PROV_ID
AND SPC.VLDT_IND = 'Y'
INNER JOIN CODE_REF SPCLT
ON SPC.SPCLT_CD = SPCLT.CD_VAL
AND SPCLT.CD_REF_NM = 'SPECIALTY'
AND SPCLT.VLDT_IND = 'Y'
WHERE SPC.VLDT_IND = 'Y'
) SPCLTY
for xml
path('')
)
,1,1,'') SPECIALTIES
You need to pay attention to the format of your SQL, and then the answer would probably jump out and bite you on the nose... happens to everyone.
Your query:
STUFF(
(
SELECT
',' + SPCLT.CD_VAL_DESC
FROM
(
SELECT DISTINCT
SPCLT.CD_VAL_DESC SPECIALTY
FROM PIN_STATUS PS
INNER JOIN PROV_TYPE_SPCLT SPC
ON PS.PROV_ID = SPC.PROV_ID
AND SPC.VLDT_IND = 'Y'
INNER JOIN CODE_REF SPCLT
ON SPC.SPCLT_CD = SPCLT.CD_VAL
AND SPCLT.CD_REF_NM = 'SPECIALTY'
AND SPCLT.VLDT_IND = 'Y'
WHERE SPC.VLDT_IND = 'Y'
) SPCLTY
for xml path('')
)
,1,1,'') SPECIALTIES
...is divided into sub-queries. The STUFF() function is acting on the first SELECT beneath it.
That first SELECT is taking data FROM a sub-query, which has been aliased as SPCLTY. So, naturally, within that SELECT, you need to be referencing SPCLTY and not SPCLT.
Adding a bit of whitespace makes it a little clearer, I think.

Complex SQL Pivot query

A quick background so that my problem makes sense: The system collects data from the user in the form of questionnaires. Users belong to Organisations, Organisations belong to Sectors, and Questions/Calculations (as found on the questionnaires) differ across the Sectors. (Questions are answered by users ; Calculations are calculated by the system).
The following tables exist:
Sectors (SectorID, Name)
Organisations (OrganisationID, Name, SectorID)
Years (YearID, Name)
Questions (QuestionID, DisplayText, CommonName, SectorID)
Answers (AnswerID, Answer, OrganisationID, YearID, QuestionID)
Calculations (CalculationID, DisplayText, CommonName, SectorID)
CalculationResults (CalculationResultID, Result, OrganisationID, YearID, CalculationID)
I need to display data in the following way:
The thing that makes this particularly complex (for me) is that questions are displayed (to the user) in different ways across the different sectors that they belong to, but some of them can still be common questions. E.g. "Manufacturing sales" would be the same thing as "Sales (manufacturing)". I need to be using the CommonName field to determine commonality.
I've managed to use SQL Pivot to get close to what I want - SQL Fiddle (if you run the SQL you'll notice the nulls and the "commonality" issue). However some things are missing from my attempt:
Commonality and column names - I need the column names to be the CommonName field, not the QuestionID field.
I've only selected from the Answers table - I need to also select from the CalculationResults table which is identically structured.
Edit: Desired result with the SQL Fiddle data is:
(The two blocks with the orange corners need to shift all the way to the left, so that there are a total of 3 columns for the Questions - the 3 unique CommonName values. The next 3 columns are for the 3 unique CommonName values for Calculations. I hope I've made sense, if not let me know.)
Edit2: Another edit just for fun. I've definitely thought about redesigning the db but it's not an option at this stage - too risky on this legacy system. In case anyone saw the design and thought that. I need a solution in the form of Pivot hopefully.
Sometimes instead of PIVOT you can use [Aggregate](CASE EXPRESSION) to get the same data. And sometimes it's faster.
For your problem you can use OUTER APPLY with dynamic MAX(CASE)
DECLARE #Questions NVARCHAR(MAX),
#Calculations NVARCHAR(MAX),
#Sql NVARCHAR(MAX)
SELECT #Questions = COALESCE(#Questions + ', ', '')
+ 'MAX(CASE WHEN q.CommonName = ''' + CommonName + ''' THEN a.Answer END) AS ' + QUOTENAME(CommonName)
FROM Questions
GROUP BY CommonName
SELECT #Calculations = COALESCE(#Calculations + ', ', '')
+ 'MAX(CASE WHEN c.CommonName = ''' + CommonName + ''' THEN cr.Result END) AS ' + QUOTENAME(CommonName)
FROM Calculations
GROUP BY CommonName
SET #Sql = N'
SELECT
o.Name As [Organisation],
y.Name As [Year],
q.*,
c.*
FROM
Organisations o
CROSS JOIN Years y
OUTER APPLY (
SELECT ' + #Questions + '
FROM Answers a
JOIN Questions q ON a.QuestionID = q.QuestionID
WHERE a.OrganisationID = o.OrganisationID
AND a.YearID = y.YearID
) q
OUTER APPLY (
SELECT ' + #Calculations + '
FROM CalculationResults cr
JOIN Calculations c ON cr.CalculationID = c.CalculationID
WHERE cr.OrganisationID = o.OrganisationID
AND cr.YearID = y.YearID
) c
'
SQL FIDDLE DEMO
Basically we want to get the order of the QuestionID Grouped By SectorID, and Name.
You can do this using PARTITION BY with something like this:
ROW_NUMBER() OVER(PARTITION BY q.SectorID, y.Name ORDER BY a.QuestionID)
this should do it:
DECLARE #cols AS NVARCHAR(MAX)
, #query AS NVARCHAR(MAX);
SELECT #cols = STUFF(
(SELECT DISTINCT
','+QUOTENAME(CAST(ROW_NUMBER() OVER(PARTITION BY q.SectorID
, y.Name ORDER BY a.QuestionID) AS VARCHAR(10)))
FROM Answers a
LEFT JOIN Years y ON a.YearID = y.YearID
LEFT JOIN Organisations o ON a.OrganisationID = o.OrganisationID
LEFT JOIN Questions q ON a.QuestionID = q.QuestionID
FOR XML PATH(''), TYPE).value
('.', 'NVARCHAR(MAX)'), 1, 1, '');
SET #query = '
SELECT Organisation, Year, '+#cols+' from
(
SELECT QuestionID = ROW_NUMBER() OVER(PARTITION BY q.SectorID
, y.Name ORDER BY a.QuestionID)
, o.Name AS Organisation
, y.Name AS Year
, a.Answer
FROM Answers a
LEFT JOIN Years y ON a.YearID = y.YearID
LEFT JOIN Organisations o ON a.OrganisationID = o.OrganisationID
LEFT JOIN Questions q ON a.QuestionID = q.QuestionID
) src
pivot
(
max(Answer)
for QuestionID in ('+#cols+')
) piv
order by Organisation, Year
';
PRINT(#query);
EXECUTE (#query);
RESULT:

display more than one value using a SQL query

I am trying to display multiple authors per title in a single column. At the moment there a repeating rows, due to the fact that some Titles have more than 1 FirstName. Is there a form of concatenation that can be used to resolve this and display all the authors in a single filed and perhaps separated by a comma.
This is my current query:
SELECT
Submission.Title, Researcher.FirstName, Submission.Type
FROM
Submission
INNER JOIN
((Faculty
INNER JOIN
School ON Faculty.FacultyID = School.[FacultyID])
INNER JOIN
(Researcher
INNER JOIN
ResearcherSubmission ON Researcher.ResearcherID = ResearcherSubmission.ResearcherID)
ON School.SchoolID = Researcher.SchoolID)
ON Submission.SubmissionID = ResearcherSubmission.SubmissionID
GROUP BY
Submission.Title, Researcher.FirstName, Submission.Type;
This the output it generates:
[
this is the output I am trying to generate:
Title FirstName Type
---------------------------------------------------------------------------
21st Century Business Matthew, Teshar Book Chapter
A Family Tree... Keshant, Lawrence Book Chapter
Benefits of BPM... Jafta Journal Article
Business Innovation Matthew, Morna, Teshar Book Chapter
You may inclde the concantenation logic within a CROSS APPLY
SELECT
Submission.Title
, CA.FirstNames
, Submission.Type
FROM Submission
CROSS APPLY (
SELECT
STUFF((
SELECT /* DISTINCT ??? */
', ' + r.FirstName
FROM ResearcherSubmission rs
INNER JOIN Researcher r ON r.ResearcherID = rs.ResearcherID
WHERE Submission.SubmissionID = rs.SubmissionID
FOR XML PATH (''), TYPE
).value('.', 'NVARCHAR(MAX)'), 1, 2, ' ')
) AS CA (FirstNames)
GROUP BY
Submission.Title
, CA.FirstNames
, Submission.Type
;
NB: I'm not sure if you need to include DISTINCT into the subquery when concatenating the names, e.g. if these was 'Jane' (Smith) and 'Jane' (Jones) do you want the final list as: 'Jane' or 'Jane, Jane'?
You can do this in your application logic as well.
But if you want to do this with a query. You should be able do something like this:
SELECT DISTINCT
sm.Title,
STUFF(
(SELECT ', ' + r.FirstName
FROM ResearcherSubmission rs
INNER JOIN Researcher r ON r.ResearcherID = rs.ResearcherID
WHERE sm.SubmissionID = rs.SubmissionID
FOR XML PATH('')), 1, 2, '') AS FirstNames,
sm.Type
FROM Submission sm
You can use the below query to generate the o/p as you want from the o/p that you have got.
CREATE TABLE #temptable(Title VARCHAR(200), FirstName VARCHAR(200), Type VARCHAR(200))
INSERT INTO #temptable
SELECT 'Book1','Matt','Chapter' UNION
SELECT 'Book1','Tesh','Chapter' UNION
SELECT 'BPM','Jafta','Article' UNION
SELECT 'Ethics','William','Journal' UNION
SELECT 'Ethics','Lawrence','Journal' UNION
SELECT 'Ethics','Vincent','Journal' UNION
SELECT 'Cellular','Jane','Conference'
SELECT Title
,STUFF((SELECT ', ' + CAST(FirstName AS VARCHAR(10)) [text()]
FROM #temptable
WHERE Title = t.Title
FOR XML PATH(''), TYPE)
.value('.','NVARCHAR(MAX)'),1,2,' ') List_Output
,Type
FROM #temptable t
GROUP BY Title,Type

SQL comma-separated row with Group By clause

I have the following query:
SELECT
Account,
Unit,
SUM(state_fee),
Code
FROM tblMta
WHERE MTA.Id = '123'
GROUP BY Account,Unit
This of course throws an exception because the Code is not in the group by clause. Each state_fee has a code. How do I get this code to display in 1 record (1 code per state_fee which is multiple state_fee per unit) as a comma-separated list? I looked into different solutions on here but I couldn't find any that worked with a group by.
You want to use FOR XML PATH construct:
SELECT ACCOUNT,
unit,
SUM(state_fee),
Stuff((SELECT ', ' + code
FROM tblmta t2
WHERE t2.ACCOUNT = t1.ACCOUNT
AND t2.unit = t1.unit
AND t2.id = '123'
FOR XML PATH('')), 1, 2, '') [Codes]
FROM tblmta t1
WHERE t1.id = '123'
GROUP BY ACCOUNT,
unit
See other examples here:
SQL same unit between two tables needs order numbers in 1 cell
SQL Query to get aggregated result in comma seperators along with group by column in SQL Server
There is no built-in aggregate function to concatenate, but this article discusses several alternative solutions, including a user-defined concatenate aggregate function:
https://www.simple-talk.com/sql/t-sql-programming/concatenating-row-values-in-transact-sql/
This will show you the table, index name, index type, indexed columns, and included columns:
with [indexes] (table_name, index_name, column_name, index_id, key_ordinal, object_id, type_desc)
as(
SELECT distinct
T.[name] AS [table_name], I.[name] AS [index_name],
AC.[name] AS [column_name],
I.[index_id], IC.[key_ordinal], T.[object_id], i.type_desc
FROM sys.[tables] AS T
INNER JOIN sys.[indexes] I ON T.[object_id] = I.[object_id]
INNER JOIN sys.[index_columns] IC ON I.[object_id] = IC.[object_id] and IC.index_id=I.index_id
LEFT OUTER JOIN sys.[all_columns] AC ON T.[object_id] = AC.[object_id] AND IC.[column_id] = AC.[column_id]
WHERE T.[is_ms_shipped] = 0 AND I.[type_desc] <> 'HEAP'
)
select
distinct
db_name() as dbname,
type_desc,
table_name,
index_name,
column_name,
STUFF((
select ', ' + column_name
from [indexes] t2
where t1.table_name=t2.table_name and t1.[index_name]=t2.[index_name] and t2.[key_ordinal] = 0
for xml path('')), 1, 2, '') inc_cols
from [indexes] t1
where t1.[key_ordinal] = 1
GROUP BY table_name, index_name, type_desc, column_name
In SQL Server 2017 (14.x) and later you can use the STRING_AGG function:
https://learn.microsoft.com/en-us/sql/t-sql/functions/string-agg-transact-sql?view=sql-server-ver16
SELECT
Account,
Unit,
SUM(state_fee),
STRING_AGG(Code, ',')
FROM tblMta
WHERE MTA.Id = '123'
GROUP BY Account,Unit
Depending on the data type of Code you might need to convert it:
SELECT
Account,
Unit,
SUM(state_fee),
STRING_AGG(CONVERT(NVARCHAR(max), Code), ',')
FROM tblMta
WHERE MTA.Id = '123'
GROUP BY Account,Unit