I have the following query which takes around 4 minutes to execute.
DECLARE #tdate DATETIME = '2019-09-01 00:00:00.000'
SELECT c.id AS clid,
h.id AS hlid,
h.holdinNo,
c.cliendID,
c.clientName,
h.floor,
h.connect_radius
FROM [db_land].[dbo].tbl_client AS c
INNER JOIN [db_land].[dbo].tx_holding AS h
ON c.id = h.clid
WHERE h.status = 1
AND h.connect_radius IS NOT NULL
AND c.status = 1
AND h.type = 'Residential'
AND h.holdinNo NOT IN (SELECT holdingNo
FROM [db_land].[dbo].tbl_bill
WHERE year(date_month) = YEAR(#tdate)
AND MONTH(date_month) = MONTH(#tdate)
AND ( update_by IS NOT NULL
OR ispay = 1 ))
I found the inner join takes only few seconds.
SELECT c.id AS clid,
h.id AS hlid,
h.holdinNo,
c.cliendID,
c.clientName,
h.floor,
h.connect_radius
FROM [db_land].[dbo].tbl_client AS c
INNER JOIN [db_land].[dbo].tx_holding AS h
ON c.id = h.clid
WHERE h.status = 1
AND h.connect_radius IS NOT NULL
AND c.status = 1
AND h.type = 'Residential'
It's the NOT IN checking which takes a lot of time. How I can optimize this query? For me it's needed to execute the query at least with in minute.
Make sure the WHERE and JOIN clause predicates are sargable. Applying a function to a column (e.g. YEAR(date_month)) prevents indexes on the column from being used efficiently.
Try this expression instead to avoid the functions. There are other methods depending on the SQL Server version.
WHERE
date_month >= DATEADD(day, 1, DATEADD(month, -1, EOMONTH(#tdate)))
AND date_month < DATEADD(day, 1, DATEADD(month, 1, EOMONTH(#tdate)))
Try by replacing NOT IN with a LEFT JOIN of the table [db_land].[dbo].tbl_bill on all the conditions and adding in the WHERE clause holdingNo is null so the returned rows are the non matching rows:
select c.id as clid, h.id as hlid,h.holdinNo, c.cliendID, c.clientName, h.floor, h.connect_radius
from [db_land].[dbo].tbl_client as c
inner join [db_land].[dbo].tx_holding as h
on c.id= h.clid
left join [db_land].[dbo].tbl_bill as b
on b.holdingNo = h.holdinNo and year(b.date_month) = YEAR(#tdate) and MONTH(b.date_month) = MONTH(#tdate)
and (b.update_by is not null or b.ispay = 1)
where h.status = 1 and h.connect_radius is not null and c.status=1 and h.type='Residential' and b.holdingNo is null
I would recommend changing the NOT IN to NOT EXISTS and adding an index:
WHERE . . . AND
NOT EXISTS (SELECT 1
FROM [db_land].[dbo].tbl_bill b
WHERE b.holdingNo = h.holdingNo AND
b.date_month >= DATEFROMPARTS(YEAR(#tdate), MONTH(#tdate), 1) AND
b.date_month < DATEADD(month, 1, DATEFROMPARTS(YEAR(#tdate), MONTH(#tdate), 1)) AND
(b.update_by IS NOT NULL OR b.ispay = 1
)
Then the index that you want is on tbl_bill(holdingNo, date_month, update_by, ispay).
Put your sub query into temp table :
DECLARE #tdate DATETIME = '2019-09-01 00:00:00.000'
SELECT holdingNo
into #TmpholdingNo
FROM [db_land].[dbo].tbl_bill
WHERE year(date_month) = YEAR(#tdate)
AND MONTH(date_month) = MONTH(#tdate)
AND ( update_by IS NOT NULL
OR ispay = 1 )
SELECT c.id AS clid,
h.id AS hlid,
h.holdinNo,
c.cliendID,
c.clientName,
h.floor,
h.connect_radius
FROM [db_land].[dbo].tbl_client AS c
INNER JOIN [db_land].[dbo].tx_holding AS h
ON c.id = h.clid
WHERE h.status = 1
AND h.connect_radius IS NOT NULL
AND c.status = 1
AND h.type = 'Residential'
AND h.holdinNo NOT IN (SELECT holdingNo from #TmpholdingNo)
drop table #TmpholdingNo
Rather than using functions in your WHERE clause try calculating the start and end filter dates, using OPTION (RECOMPILE) can help SQL to use the actual values of your variables in your query plan. I would also change NOT IN to NOT EXISTS:
DECLARE #tdate DATETIME = '2019-09-01 00:00:00.000'
DECLARE #startDate DATE = DATEFROMPARTS(YEAR(#tdate), MONTH(#tdate), 1)
DECLARE #endDate DATE = DATEADD(day,1,EOMONTH(#tdate))
SELECT c.id AS clid,
h.id AS hlid,
h.holdinNo,
c.cliendID,
c.clientName,
h.floor,
h.connect_radius
FROM [db_land].[dbo].tbl_client AS c
INNER JOIN [db_land].[dbo].tx_holding AS h
ON c.id = h.clid
WHERE h.status = 1
AND h.connect_radius IS NOT NULL
AND c.status = 1
AND h.type = 'Residential'
AND NOT EXISTS (SELECT holdingNo
FROM [db_land].[dbo].tbl_bill
WHERE holdingNo = h.holdinNo AND
date_month >= #startDate AND
date_month < #endDate AND
AND ( update_by IS NOT NULL
OR ispay = 1 ))
OPTION (RECOMPILE)
give a try try this:
select main.* from
(SELECT c.id AS clid,
h.id AS hlid,
h.holdinNo,
c.cliendID,
c.clientName,
h.floor,
h.connect_radius
FROM [db_land].[dbo].tbl_client AS c
INNER JOIN [db_land].[dbo].tx_holding AS h
ON c.id = h.clid
WHERE h.status = 1
AND h.connect_radius IS NOT NULL
AND c.status = 1
AND h.type = 'Residential')main
left join
(select holdingNo from
(SELECT holdingNo, update_by, ispay
FROM [db_land].[dbo].tbl_bill
WHERE year(date_month) = YEAR(#tdate)
AND MONTH(date_month) = MONTH(#tdate))bill1
where update_by IS NOT NULL OR ispay = 1)bill2
on main.holdinNo = bill2.holdinNo
where bill2.holdinNo is null
put the filter list at variable,then them apply the filter
DECLARE #filter TABLE INSERT INTO #filter SELECT FROM [db_land].[dbo].tbl_bill
them apply the filter
DECLARE #tdate DATETIME = '2019-09-01 00:00:00.000'
SELECT c.id AS clid,
h.id AS hlid,
h.holdinNo,
c.cliendID,
c.clientName,
h.floor,
h.connect_radius
FROM [db_land].[dbo].tbl_client AS c
INNER JOIN [db_land].[dbo].tx_holding AS h ON c.id= h.clid
WHERE h.status=1
AND h.connect_radius IS NOT NULL
AND c.status=1
AND h.type='Residential'
AND h.holdinNo NOT IN (filter)
SELECT FutureDatedEmployeeRecordsKey INTO #FutureDatedExclude
FROM dbo.vwRptDimEmployee_FutureDated FD1
WHERE EXISTS (SELECT 1 FROM dbo.vwRptDimEmployeeAll EE1 WITH (NOLOCK)
WHERE FD1.EmployeeID = EE1.EmployeeID AND FD1.EmployeeRecord = EE1.EmployeeRecord
AND FD1.JobEffectiveDate = EE1.JobEffectiveDT AND FD1.JobEffectiveDateSequence = EE1.JobEffectiveDateSequence
AND FD1.ActionCode = EE1.ActionCode AND FD1.ActionReasonCode = EE1.ActionReasonCode)
declare #JobStartDate date = '07/01/2014', #JobEndDate date = '06/30/2015'
SELECT DISTINCT
E.LastName,
E.SecondLastName,
E.FirstName,
E.MiddleName,
E.PreferredName,
E.PreferredFirstName,
E.NameAC,
E.LastNameAC,
E.FirstNameAC,
E.MiddleNameAC,
E.GUI,
E.EmployeeID,
E.LPN,
E.GPN,
E.EmployeeRecord,
E.JobEffectiveDT JobEffectiveDate,
E.JobEffectiveDateSequence,
E.ActionCode,
E.Action,
E.ActionDate,
E.ActionReasonCode,
AR.Description ActionReason,
E.EmployeeStatusCode,
E.EmployeeStatusDesc,
CASE WHEN YEAR(E.LeaveEffectiveDT) > 2100 THEN NULL ELSE E.LeaveEffectiveDT END LeaveEffectiveDate,
CASE WHEN YEAR(E.ExpectedReturnDate) > 2100 THEN NULL ELSE E.ExpectedReturnDate END ExpectedReturnDate,
E.FullPartTime,
E.ShiftCode FWACode,
E.Shift FWAName,
E.TeleWork,
E.StandardHoursFrequency,
E.StandardHours,
E.FTE,
E.PaidFTE,
E.OvertimeEligibility,
E.EmployeeClassCode,
E.EmployeeClass,
E.RegularVersusTemporary RegularTemporary,
E.EmployeeType,
E.PersonnelStatusDesc,
E.PersonOrganizationRelationshipCode,
P.PersonOfInterest,
P.PersonOfInterestDesc,
E.PaygroupCode,
E.EmployeeCategoryCode,
E.EmployeeSubcategoryCode,
P.EmploymentCategory,
E.NonEmployeeNonWorkTypeCD NonEmployeeNonWorkTypeCode,
P.NonEmployeeNonWorkTypeDesc,
A.GlobalAssignmentProgramCD GlobalAssignmentProgramCode,
A.GlobalAssignmentProgramDesc,
CASE WHEN YEAR(E.GlobalAssignmentStartDT) > 2100 THEN NULL ELSE E.GlobalAssignmentStartDT END GlobalAssignmentStartDate,
CASE WHEN YEAR(E.GlobalAssignmentEndDT) > 2100 THEN NULL ELSE E.GlobalAssignmentEndDT END GlobalAssignmentEndDate,
E.InPatExPatStatus,
E.HomeCountry,
E.HomeHostCountry HostCountry,
CASE WHEN YEAR(E.EYStartDate) > 2100 THEN NULL ELSE E.EYStartDate END EYStartDate,
CASE WHEN YEAR(E.LastRehireDate) > 2100 THEN NULL ELSE E.LastRehireDate END LastRehireDate,
CASE WHEN YEAR(E.SeniorityDate) > 2100 THEN NULL ELSE E.SeniorityDate END SeniorityDate,
CASE WHEN YEAR(E.EmployeeEffectiveDate) > 2100 THEN NULL ELSE E.EmployeeEffectiveDate END CurrentEmploymentDate,
CASE WHEN YEAR(E.PartnerAdmissionDate) > 2100 THEN NULL ELSE E.PartnerAdmissionDate END PartnerAdmissionDate,
R.RankCDName RankCodeName,
E.Rank,
R.RankDesc,
E.BusinessTitle,--NEW
R.RankGroup1,--NEW
E.GFISRank,
E.ExperienceLevel,
E.GlobalGrade,
E.JobCode,--NEW
E.JobCDDesc JobCodeDesc,--NEW
E.DepartmentCode,
E.DepartmentName,
E.CompanyCode,
C.Description Company,
C.DescrAc CompanyAC,
E.ManagerialCountryCD ManagerialCountry,
O.CodeBlock,
O.BUCD BU,
O.OUCD OU,
O.MUCD MU,
O.SMUCD SMU,
O.BUName,
O.OUName,
O.MUName,
O.SMUName,
O.UserDefSLHierarchy1 ServiceLine,
O.UserDefSLHierarchy2 SubSL1,
O.UserDefSLHierarchy3 SubSL2,
O.AlternateServiceLine,
O.UserDefAreaHierarchy1 BULevel1,
O.UserDefAreaHierarchy2 BULevel2,
O.UserDefAreaHierarchy3 BULevel3,
L.Location LocationCode,
L.City LocationCity,
L.State LocationStateProv,
L.Country LocationCountry,
L.UserDefinedHRGeo1 GeoLevel1,
L.UserDefinedHRGeo2 GeoLevel2,
L.UserDefinedHRGeo3 GeoLevel3,
L.UserDefinedHRGeo4 GeoLevel4,
L.UserDefinedHRGeo5 GeoLevel5,
E.CounselorGUI,--NEW
E.CounselorName,--NEW
E.BillRate,
E.Source,
--**** confidential fields ****
E.GenderCode,
E.TermCD TermCode,
E.TerminationReasonCode,
E.CompensationCurrency,
E.CompensationRate,
E.CompensationFrequency,
E.MonthlyCompensationRate,
E.AnnualCompensationRate,
CASE WHEN YEAR(P.SalaryEffectiveDT) > 2100 THEN NULL ELSE P.SalaryEffectiveDT END SalaryEffectiveDate,
E.SalaryAdminPlanCode,
E.SalaryAdminPlan,
E.SalaryGrade,
CASE WHEN YEAR(E.SalaryGradeEntryDate) > 2100 THEN NULL ELSE E.SalaryGradeEntryDate END SalaryGradeEntryDate,
NULL JobKEY,
NULL RowOrder
FROM dbo.vwRptFactEmployee F WITH (NOLOCK)
INNER JOIN dbo.vwRptDimEmployee E WITH (NOLOCK) ON (F.DimEmployeeKey = E.DimEmployeeKey)
INNER JOIN dbo.vwRptDimRank R WITH (NOLOCK) ON (F.DimRankKey = R.DimRankKey)
INNER JOIN dbo.vwRptDimOrganization O WITH (NOLOCK) ON (F.DimOrganizationKey = O.DimOrganizationKey)
INNER JOIN dbo.vwRptDimLocation L WITH (NOLOCK) ON (F.DimLocationKey = L.DimLocationKey)
INNER JOIN dbo.vwRptDimAssignment A WITH (NOLOCK) ON (F.DimAssignmentKey = A.DimAssignmentKey)
--INNER JOIN dbo.vwRptDimDate D WITH (NOLOCK) ON (F.TransEffectiveDateKey = D.DimDateKey)
INNER JOIN dbo.vwRptDimEmployeeV2 P WITH (NOLOCK) ON (F.DimEmployeeKey = P.DimEmployeeKey)
LEFT OUTER JOIN (SELECT ActionCode, ActionReasonCode, Description, row_number() over (partition by ActionCode, ActionReasonCode order by EffectiveDate DESC) as RowOrder
FROM PISupport.vwRptSetfActionReason WITH (NOLOCK)) AR
ON (AR.ActionCode = E.ActionCode AND AR.ActionReasonCode = E.ActionReasonCode AND AR.RowOrder = 1)
LEFT OUTER JOIN (SELECT DISTINCT C1.*, ROW_NUMBER() OVER (PARTITION BY CompanyCode ORDER BY EffectiveDate DESC) as RowOrder
FROM PISupport.vwRptSetfCompany C1 WITH (NOLOCK)) C
ON (C.CompanyCode = E.CompanyCode AND C.RowOrder = 1)
WHERE (E.JobEffectiveDT BETWEEN #JobStartDate AND #JobEndDate)
-- AND (E.ActionCode in ('ADD','DTA','HIR','POI','REH','PER','TER'))
--AND (O.BUCD+O.OUCD+O.MUCD+O.SMUCD LIKE '%'+#CodeBlock+'%' OR #CodeBlock IS NULL)
--AND (E.GPN = #GPN OR #GPN IS NULL)
--AND (E.GUI = #GUI OR #GUI IS NULL)
--AND (L.UserDefinedHRGeo1 in (#GeoArea) )
AND (L.UserDefinedHRGeo2 in ('UK and Ireland'))
--AND (L.UserDefinedHRGeo3 in (#Country) )
--AND (O.UserDefAreaHierarchy1 in (#Area) )
--AND (O.UserDefAreaHierarchy2 in (#Region) )
--AND (O.UserDefSLHierarchy1 in (#ServiceLine) )
--AND (O.UserDefSLHierarchy2 in (#SubServiceLine) )
--AND (R.RankCD in (#RankCode) )
UNION
SELECT DISTINCT * FROM (
SELECT DISTINCT
ISNULL(E.LastName,N.LastName) LastName,
ISNULL(E.SecondLastName,N.SecondLastName) SecondLastName,
ISNULL(E.FirstName,N.FirstName) FirstName,
ISNULL(E.MiddleName,N.MiddleName) MiddleName,
E.PreferredName,
ISNULL(E.PreferredFirstName,N.PreferredFirstName) PreferredFirstName,
ISNULL(E.NameAC,N.NameAlternateCharacter) NameAC,
E.LastNameAC,
E.FirstNameAC,
E.MiddleNameAC,
E.GUI,
FD.EmployeeID,
FD.LPN,
FD.GPN,
FD.EmployeeRecord,
FD.JobEffectiveDate,
FD.JobEffectiveDateSequence,
FD.ActionCode,
FD.Action,
FD.ActionDate,
FD.ActionReasonCode,
FD.ActionReason,
FD.EmployeeStatusCode,
FD.EmployeeStatus,
NULL LeaveEffectiveDate,
CASE WHEN YEAR(FD.ExpectedReturnDate) > 2100 THEN NULL ELSE FD.ExpectedReturnDate END ExpectedReturnDate,
FD.FullPartTime,
FD.ShiftCode FWACode,
FD.Shift FWAName,
NULL Telework,
FD.StandardHoursFrequency,
NULL StandardHours,
FD.FTE,
FD.PaidFTE,
NULL OvertimeEligibility,
FD.EmployeeClassCode,
FD.EmployeeClass,
FD.RegularVersusTemporary RegularTemporary,
FD.EmployeeType,
NULL PersonnelStatusDesc,
FD.PersonOrganizationRelationshipCode,
NULL PersonOfInterest,
NULL PersonOfInterestDesc,
FD.PaygroupCode,
FD.EmployeeCategoryCode,
FD.EmployeeSubcategoryCode,
NULL EmploymentCategory,
NULL NonEmployeeNonWorkTypeCode,
NULL NonEmployeeNonWorkTypeDesc,
NULL GlobalAssignmentProgramCode,
NULL GlobalAssignmentProgramDesc,
NULL GlobalAssignmentStartDate,
NULL GlobalAssignmentEndDate,
NULL InPatExPatStatus,
NULL HomeCountry,
NULL HostCountry,
NULL EYStartDate,
NULL LastRehireDate,
NULL SeniorityDate,
NULL CurrentEmploymentDate,
NULL PartnerAdmissionDate,
R.RankCDName RankCodeName,
FD.Rank,
R.RankDesc,
FD.BusinessTitle,--NEW
R.RankGroup1,--NEW
FD.GFISRank,
NULL ExperienceLevel,
NULL GlobalGrade,
FD.JobCode,--NEW
NULL JobCodeDesc,--NEW
FD.DepartmentCode,
NULL DepartmentName,
FD.CompanyCode,
C.Description Company,
C.DescrAc CompanyAC,
NULL ManagerialCountry,
O.CodeBlock,
O.BUCD BU,
O.OUCD OU,
O.MUCD MU,
O.SMUCD SMU,
O.BUName,
O.OUName,
O.MUName,
O.SMUName,
O.UserDefSLHierarchy1 ServiceLine,
O.UserDefSLHierarchy2 SubSL1,
O.UserDefSLHierarchy3 SubSL2,
O.AlternateServiceLine,
O.UserDefAreaHierarchy1 BULevel1,
O.UserDefAreaHierarchy2 BULevel2,
O.UserDefAreaHierarchy3 BULevel3,
L.Location LocationCode,
L.City LocationCity,
L.State LocationStateProv,
L.Country LocationCountry,
L.UserDefinedHRGeo1 GeoLevel1,
L.UserDefinedHRGeo2 GeoLevel2,
L.UserDefinedHRGeo3 GeoLevel3,
L.UserDefinedHRGeo4 GeoLevel4,
L.UserDefinedHRGeo5 GeoLevel5,
NULL CounselorGUI,--NEW
NULL CounselorName,--NEW
NULL BillRate,
FD.Source,
--**** confidential fields ****
NULL GenderCode,
NULL TermCode,
FD.TerminationReasonCode,
FD.CompensationCurrency,
FD.CompensationRate,
FD.CompensationFrequency,
FD.MonthlyCompensationRate,
FD.AnnualCompensationRate,
NULL SalaryEffectiveDate,
FD.SalaryAdminPlanCode,
FD.SalaryAdminPlan,
FD.SalaryGrade,
CASE WHEN YEAR(FD.SalaryGradeEntryDate) > 2100 THEN NULL ELSE FD.SalaryGradeEntryDate END SalaryGradeEntryDate,
FD.Job_KEY,
row_number() over (partition by FD.EmployeeID, FD.EmployeeRecord, FD.JobEffectiveDate,
FD.JobEffectiveDateSequence, FD.ActionCode, FD.ActionReasonCode order by FD.Job_KEY DESC) as RowOrder
FROM dbo.vwRptDimEmployee_FutureDated FD
INNER JOIN dbo.vwRptDimOrganization O WITH (NOLOCK) ON (FD.DimOrganizationKey = O.DimOrganizationKey)
INNER JOIN dbo.vwRptDimLocation L WITH (NOLOCK) ON (FD.DimLocationKey = L.DimLocationKey)
LEFT OUTER JOIN dbo.vwRptDimRank R WITH (NOLOCK) ON (FD.Rank = R.RankCD)
LEFT OUTER JOIN dbo.vwRptDimEmployeeAll E WITH (NOLOCK) ON (FD.GPN = E.GPN AND FD.GPN <> '' AND E.RowIsCurrent = 'Y')
LEFT OUTER JOIN (SELECT *, ROW_NUMBER() OVER (PARTITION BY EmployeeID, NameType ORDER BY EffectiveDate DESC) AS RowOrder
FROM PISupport.vwRptPersonACNames WITH (NOLOCK)
) N ON (N.EmployeeID = FD.EmployeeID AND N.NameType = 'PRI' AND N.CountryNameFormat = FD.SetIDLaborAgreement AND N.RowOrder = 1)
LEFT OUTER JOIN (SELECT DISTINCT C1.*, ROW_NUMBER() OVER (PARTITION BY CompanyCode ORDER BY EffectiveDate DESC) as RowOrder
FROM PISupport.vwRptSetfCompany C1 WITH (NOLOCK)) C
ON (C.CompanyCode = FD.CompanyCode AND C.RowOrder = 1)
WHERE
FD.JobEffectiveDate BETWEEN #JobStartDate AND #JobEndDate
AND FD.EDWIsCurrentRecord = 1
AND FD.EmployeeID IS NOT NULL
--AND (E.ActionCode in ('ADD','DTA','HIR','POI','REH','PER','TER'))
--AND (O.BUCD+O.OUCD+O.MUCD+O.SMUCD LIKE '%'+#CodeBlock+'%' OR #CodeBlock IS NULL)
--AND (FD.GPN = #GPN OR #GPN IS NULL)
--AND (L.UserDefinedHRGeo1 in (#GeoArea) )
AND (L.UserDefinedHRGeo2 in ('UK and Ireland'))
--AND (L.UserDefinedHRGeo3 in (#Country) )
--AND (O.UserDefAreaHierarchy1 in (#Area) )
--AND (O.UserDefAreaHierarchy2 in (#Region) )
--AND (O.UserDefSLHierarchy1 in (#ServiceLine) )
--AND (O.UserDefSLHierarchy2 in (#SubServiceLine) )
--AND (FD.Rank in (#RankCode) )
AND FD.FutureDatedEmployeeRecordsKey NOT IN (SELECT FutureDatedEmployeeRecordsKey FROM #FutureDatedExclude)
) X
WHERE RowOrder = 1
DROP TABLE #FutureDatedExclude
You most likely ran this code before you added the DROP TABLE at the bottom of your code. Therefore, the table is created and yet to be dropped. Instead of dropping it at the end, or in addition to dropping it if you want, place this at the very top of your script:
IF OBJECT_ID('tempdb..#FutureDatedExclude') IS NOT NULL DROP TABLE #FutureDatedExclude
I have this query and I am trying to group by surveyname but im getting this error:
Msg 8120, Level 16, State 1, Line 1
Column 'pvt.Follow Up' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.
This is the query:
SELECT
surveyname, [Follow Up] AS Follow_Up, [Ambiance] AS Ambiance,
[Consultation] AS Consultation, [Procedure/Service] AS Procedure_Service
FROM
(SELECT
s.name surveyname, q.question, subq.answer subquestion,aw.answerweight,
aw.score, rc.categoryname, sc.cweight
FROM survey.dbo.results r
JOIN survey.dbo.questions q ON r.questionidfk = q.id
LEFT JOIN survey.dbo.answers subq ON r.itemidfk = subq.id
LEFT JOIN survey.dbo.answers a ON r.answeridfk = a.id
JOIN survey.dbo.surveys s ON q.surveyidfk = s.id
join sigweb.dbo.survey_types_main stm on s.id = stm.surveyidfk
join survey.dbo.survey_results sr on r.owneridfk = sr.ownerid
join sigweb.dbo.BosleySurvey bs on bs.contactid = sr.contactid and stm.clientsurveytypeid = bs.surveytype
join sigweb.dbo.contact c on sr.contactid = c.contactid
join sigweb.dbo.patient p on p.contactid = c.contactid
join sigweb.dbo.doctor d on p.doctorid = d.doctorid
join sigweb.dbo.survey_tracking st on st.contactid = c.contactID and st.surveytypeid = stm.surveytypeid
left join survey.dbo.answerweighting aw on isnull(r.itemidfk, r.questionidfk) = aw.questionitemidfk and r.answeridfk = aw.answeridfk
left join survey.dbo.rating_categories rc on aw.categoryidfk = rc.id
left join survey.dbo.survey_categories sc on aw.categoryidfk = sc.categoryidfk and s.id = sc.surveyidfk
where
aw.answerWeight is not null) ps
PIVOT
(
AVG(score)
FOR categoryname IN
( [Follow Up], [Ambiance], [Consultation], [Procedure/Service])
) AS pvt
group by surveyname
This is an example of the results im getting
SURVEYNAME FOLLOW_UP Ambiance Consultation Procedure_Service
Review NULL NULL NULL 9.81
Review 9.54 NULL NULL NULL
Consultation 5 NULL NULL NULL
Consultation NULL 5 NULL NULL
Consultation NULL 5 NULL NULL
Consultation NULL 5 NULL NULL
Consultation NULL 5 NULL NULL
Consultation NULL 5 NULL NULL
Consultation NULL NULL 5 NULL
Consultation 5 NULL NULL NULL
Consultation NULL NULL 5 NULL
This is an example of the data before the pivot:
Review 6 Follow Up
Review 9 Procedure/Service
Consultation 5 Ambiance
Consultation 5 Ambiance
Consultation 5 Ambiance
Consultation 5 Ambiance
Consultation 5 Ambiance
Consultation 5 Ambiance
Consultation 5 Consultation
Consultation 5 Consultation
The idea is to group by the surveyname and have only two results in the end.
It appears that you are including too many columns in the inner SELECT, try to remove the columns:
q.question, subq.answer subquestion, aw.answerweight, sc.cweight
They are most likely making the rows DISTINCT so the GROUP BY does not work properly. So your query will be:
SELECT surveyname,
[Follow Up] AS Follow_Up,
[Ambiance] AS Ambiance,
[Consultation] AS Consultation,
[Procedure/Service] AS Procedure_Service
FROM
(
SELECT s.name surveyname,
aw.score,
rc.categoryname,
FROM survey.dbo.results r
JOIN survey.dbo.questions q
ON r.questionidfk = q.id
LEFT JOIN survey.dbo.answers subq
ON r.itemidfk = subq.id
LEFT JOIN survey.dbo.answers a
ON r.answeridfk = a.id
JOIN survey.dbo.surveys s
ON q.surveyidfk = s.id
join sigweb.dbo.survey_types_main stm
on s.id = stm.surveyidfk
join survey.dbo.survey_results sr
on r.owneridfk = sr.ownerid
join sigweb.dbo.BosleySurvey bs
on bs.contactid = sr.contactid
and stm.clientsurveytypeid = bs.surveytype
join sigweb.dbo.contact c
on sr.contactid = c.contactid
join sigweb.dbo.patient p
on p.contactid = c.contactid
join sigweb.dbo.doctor d on p.doctorid = d.doctorid
join sigweb.dbo.survey_tracking st
on st.contactid = c.contactID
and st.surveytypeid = stm.surveytypeid
left join survey.dbo.answerweighting aw
on isnull(r.itemidfk, r.questionidfk) = aw.questionitemidfk
and r.answeridfk = aw.answeridfk
left join survey.dbo.rating_categories rc
on aw.categoryidfk = rc.id
left join survey.dbo.survey_categories sc
on aw.categoryidfk = sc.categoryidfk and s.id = sc.surveyidfk
where aw.answerWeight is not null
) ps
PIVOT
(
AVG(score)
FOR categoryname IN
( [Follow Up], [Ambiance], [Consultation], [Procedure/Service])
) AS pvt
I am not sure where the error is coming from in what you have posted (I assume it is when you try and add GROUP BY SurveyName to the end of the query you have posted), but you need to remove the redundant columns from your subquery, so you only select the 3 columns you need, surveyname, score, and categoryname:
SELECT
surveyname, [Follow Up] AS Follow_Up, [Ambiance] AS Ambiance,
[Consultation] AS Consultation, [Procedure/Service] AS Procedure_Service
FROM
(SELECT
s.name surveyname, aw.score, rc.categoryname
FROM survey.dbo.results r
JOIN survey.dbo.questions q ON r.questionidfk = q.id
LEFT JOIN survey.dbo.answers subq ON r.itemidfk = subq.id
LEFT JOIN survey.dbo.answers a ON r.answeridfk = a.id
JOIN survey.dbo.surveys s ON q.surveyidfk = s.id
join sigweb.dbo.survey_types_main stm on s.id = stm.surveyidfk
join survey.dbo.survey_results sr on r.owneridfk = sr.ownerid
join sigweb.dbo.BosleySurvey bs on bs.contactid = sr.contactid and stm.clientsurveytypeid = bs.surveytype
join sigweb.dbo.contact c on sr.contactid = c.contactid
join sigweb.dbo.patient p on p.contactid = c.contactid
join sigweb.dbo.doctor d on p.doctorid = d.doctorid
join sigweb.dbo.survey_tracking st on st.contactid = c.contactID and st.surveytypeid = stm.surveytypeid
left join survey.dbo.answerweighting aw on isnull(r.itemidfk, r.questionidfk) = aw.questionitemidfk and r.answeridfk = aw.answeridfk
left join survey.dbo.rating_categories rc on aw.categoryidfk = rc.id
left join survey.dbo.survey_categories sc on aw.categoryidfk = sc.categoryidfk and s.id = sc.surveyidfk
where
aw.answerWeight is not null) ps
PIVOT
(
AVG(score)
FOR categoryname IN
( [Follow Up], [Ambiance], [Consultation], [Procedure/Service])
) AS pvt
In the background you are also grouping your end results by q.question, subq.answer subquestion,aw.answerweight, sc.cweight because they are included in the subquery, but because the are not in the select list you are not seeing immediately the effect this is having.