SQL Sort / Order By pivoted fields while COALESCE function - sql

I have some rates for resources for all countries
The rows will be Resource IDs
Columns should be Country Codes
Challenge here, I cannot sort the Country Codes in ASC
It would be so grateful if you could help me on this.
When I query, I get the list of country codes, but not sorted. i.e., USA,BRA,ARG etc. But the expected result should be ARG,BRA,USA in columns of the pivot.
Here is my code:
DECLARE #idList nvarchar(MAX)
SELECT
#idList = COALESCE(#idList + ',', '') + CountryCodeISO3
FROM
(
SELECT
DISTINCT CountryCodeISO3
FROM
Published.RateCardsValues
WHERE
CardID = 55
) AS SRC
DECLARE #sqlToRun nvarchar(MAX)
SET
#sqlToRun = '
SELECT *
FROM (
SELECT
[ResourceCode]
,[TITLES]
,[MostRepresentativeTitle]
,[ABBR_RES_DESC]
,[TypicalJobGrade]
,[BidGridResourceCode]
,[OpUnit]
,[PSResType]
,[JobGradeORResCat]
,[CountryCodeISO3]
--,[CurrencyCode]
,[RateValue]
FROM
[Published].[RateCardsValues] rc
WHERE
CardID = 55) As src
PIVOT (
MAX(RateValue) FOR [CountryCodeISO3] IN (' + #idList + ')
) AS pvt'
EXEC (#sqlToRun)

As you have discovered, PIVOT in T-SQL requires you to know at development time what the values will be that you will be pivoting on.
This is limiting, because if you want something like "retrieve data for all the countries where Condition X is true, then pivot on their IDs!", you have to resort to dynamic SQL to do it.
If Condition X is constant -- I'm guessing that belonging to CardID = 55 doesn't change often -- you can look up the values, and hardcode them in your code.
If the CardID you're looking up is always 55 and you have relatively few countries in that category, I'd actually advise doing that.
But if your conditions for picking countries can change, or the number of columns you want can vary -- something like "all the countries where there were sales of product Y, for month Z!" -- then you can't predict them, which means that the T-SQL PIVOT can't be set up (without dynamic SQL.)
In that case, I'd strongly suggest that you have whatever app you plan to use the data in do the pivoting, not T-SQL. (SSRS and Excel can both do it themselves, and code can be written to do it in .NET langauges.) T-SQL, as you have seen, does not lend itself to dynamic pivoting.
What you have will "work" in the sense that it will execute without errors, but there's another downside, in the next stage of your app: not only will the number of columns potentially change over time, the names of the columns will change, as countries move in and out of Card ID 55. That may cause problems for whatever app or destination you have in mind for this data.
So, my two suggestions would be: either hard-code your country codes, or have the next stage in your app (whatever executes the query) do the actual pivoting.

You need to sort the columns while creating the dynamic SQL
Also:
Do not use variable coalescing, use STRING_AGG or FOR XML instead
Use QUOTENAME to escape the column names
sp_executesql allows you to pass parameters to the dynamic query
DECLARE #idList nvarchar(MAX)
SELECT
#idList = STRING_AGG(QUOTENAME(CountryCodeISO3), ',') WITHIN GROUP (ORDER BY CountryCodeISO3)
FROM
(
SELECT
DISTINCT CountryCodeISO3
FROM
Published.RateCardsValues
WHERE
CardID = 55
) AS SRC;
DECLARE #sqlToRun nvarchar(MAX);
SET
#sqlToRun = '
SELECT *
FROM (
SELECT
[ResourceCode]
,[TITLES]
,[MostRepresentativeTitle]
,[ABBR_RES_DESC]
,[TypicalJobGrade]
,[BidGridResourceCode]
,[OpUnit]
,[PSResType]
,[JobGradeORResCat]
,[CountryCodeISO3]
--,[CurrencyCode]
,[RateValue]
FROM
[Published].[RateCardsValues] rc
WHERE
CardID = 55) As src
PIVOT (
MAX(RateValue) FOR [CountryCodeISO3] IN (' + #idList + ')
) AS pvt'
EXEC sp_executesql #sqlToRun;
On earlier versions of SQL Server, you cannot use STRING_AGG. You need to hack it with FOR XML. You need to also use STUFF to strip off the first separator.
DECLARE #idList nvarchar(MAX)
DECLARE #separator nvarchar(20) = ',';
SET #idList =
STUFF(
(
SELECT
#sep + QUOTENAME(CountryCodeISO3)
FROM
Published.RateCardsValues
WHERE
CardID = 55
GROUP BY
CountryCodeISO3
ORDER BY
CountryCodeISO3
FOR XML PATH(''), TYPE
).value('text()[1]','nvarchar(max)'),
1, LEN(#separator), '')
;
DECLARE #sqlToRun nvarchar(MAX);
SET
#sqlToRun = '
SELECT *
FROM (
SELECT
[ResourceCode]
,[TITLES]
,[MostRepresentativeTitle]
,[ABBR_RES_DESC]
,[TypicalJobGrade]
,[BidGridResourceCode]
,[OpUnit]
,[PSResType]
,[JobGradeORResCat]
,[CountryCodeISO3]
--,[CurrencyCode]
,[RateValue]
FROM
[Published].[RateCardsValues] rc
WHERE
CardID = 55) As src
PIVOT (
MAX(RateValue) FOR [CountryCodeISO3] IN (' + #idList + ')
) AS pvt'
EXEC sp_executesql #sqlToRun;

Related

Convert Rows to Columns SQL

I am trying to convert rows to columns in SQL server. I am trying to convert the value's a product gets while being tested during quality. I have tried the pivot function but having trouble doing so as the same values do get repeated and it can not be easily sorted into rows. The table I am trying to pivot holds ~30K data row's so hoping to find a dynamic solution for this.
The maximum number of new columns is 30 but sometimes a product doesn't get tested as much so it can be less. The new column would be based off my inspection_unit_number field. Is this possible to achieve in SQL
Current data
What I hope to achieve
Current Attempt
SELECT BATCH , characteristic, [1],[2],[3],[4],[5],[6],[7],[8],[9],[10],[11],[12],[13],[14],[15],[16],[17],[18],[19],[20],[21],[22],[23],[24],[25],[26],[27],[28],[29],[30]
from
(
select inspection_lot ,node_number ,characteristic ,inspector ,inspection_unit_number ,start_date ,measured_value ,original_value ,material_no ,batch
from stg.IQC_Tensile_TF
) d
pivot
(
max(measured_value)
for
INSPECTION_UNIT_NUMBER in ([1],[2],[3],[4],[5],[6],[7],[8],[9],[10],[11],[12],[13],[14],[15],[16],[17],[18],[19],[20],[21],[22],[23],[24],[25],[26],[27],[28],[29],[30])
) piv;
You will have to go for a dynamic query, check if this will suit your needs.
I created a common table expression to be able to use distinct and then order by in the stuff function:
DECLARE #QUERY NVARCHAR(MAX)
DECLARE #Columns NVARCHAR(MAX)
WITH cte_unique_inspection_unit_number AS
(
SELECT DISTINCT QUOTENAME('TestResults' + CAST(inspection_unit_number AS VARCHAR)) TestResultsN,
inspection_unit_number
FROM IQC_Tensile_TF
)
SELECT #Columns = STUFF((SELECT ', ' + TestResultsN
FROM cte_unique_inspection_unit_number
ORDER BY inspection_unit_number
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1,2,''),
#query = 'SELECT batch, node_number, characteristic, ' + #Columns + ' from
(
select batch,
node_number,
characteristic,
measured_value,
''TestResults'' + CAST(inspection_unit_number AS VARCHAR) TestResultsN
from IQC_Tensile_TF
) x
pivot
(
max(measured_value)
for TestResultsN in (' + #Columns + ')
) p '
EXEC(#query)
To view the execution in fiddle:
https://dbfiddle.uk/?rdbms=sqlserver_2014&fiddle=7898422e4422faacb25d7f3c2285f14a
If you find my answer useful, i would appreciate if you vote up and mark as accepted =D

Assistance with a SQL query parsing JSON

I have a database table called QueueItems that contains the following fields:
SpecificData contains JSON data so we have used the well known parseJSON SQL function that is about on the internet.
So an example of the JSON is below:
{"DynamicProperties":{"IdentificationIndex":"CK","PaymentMethod":"C","Variants1":"010 ZERO BAL,716 ZERO BAL,717 ZERO BAL","Variants2":"CHECK_010,CHECK_716,CHECK_717","CustomerCode":"NO","FreeSelectionField":"NO","FreeSelectionValue":"NO","Variants1Line":"RFFOAVIS","Variants2Line":"ZRFFOUS_C","VendorFrom":"1","VendorTo":"999999999","DueDateCheck":"Yes","PaymentMethodSel":"Yes","PaymentMethodSelOnFail":"No","LineItemsOfPayDocs":"Yes","StartImmediately":"Yes","CreatePaymentMedium":"No","ExportFormat":"HTML Format","ExcludeValues":"Yes"}}
When I run the SQL function parseJSON it returns me the data in this format:
Now, the query I am trying to write is:
SELECT QueueItemID, QueueItemStatus, StartProcessing, EndProcessing, [THEN append each column from parseJSON but this needs to be transposed first]
So far I have managed to transpose the JSON into columns with a single row using:
DECLARE
#Cols AS VARCHAR(MAX) = ''
,#Query AS NVARCHAR(MAX) = ''
,#ParamDef AS NVARCHAR(MAX)
,#Json AS VARCHAR(MAX) = '{"DynamicProperties":{"IdentificationIndex":"CK","PaymentMethod":"C","Variants1":"010 ZERO BAL,716 ZERO BAL,717 ZERO BAL","Variants2":"CHECK_010,CHECK_716,CHECK_717","CustomerCode":"NO","FreeSelectionField":"NO","FreeSelectionValue":"NO","Variants1Line":"RFFOAVIS","Variants2Line":"ZRFFOUS_C","VendorFrom":"1","VendorTo":"999999999","DueDateCheck":"Yes","PaymentMethodSel":"Yes","PaymentMethodSelOnFail":"No","LineItemsOfPayDocs":"Yes","StartImmediately":"Yes","CreatePaymentMedium":"No","ExportFormat":"HTML Format","ExcludeValues":"Yes"}}'
SELECT
#Cols += ',' + Name
FROM
parseJson(#Json)
WHERE
Name NOT IN ('DynamicProperties', '-')
SET #Cols = SUBSTRING(#Cols,2,LEN(#Cols))
SET #Query = N'SELECT *
FROM
(
SELECT [StringValue], [Name]
FROM parseJson(''' + #Json + ''')
) [d]
PIVOT
(
MAX([StringValue])
FOR [Name] IN (' + #Cols + ')
) [piv]'
EXECUTE (#Query)
As you can see the JSON may contain ANY data, so I am dynamically inserting the column names into the PIVOT statement FOR IN.
Now to Join this data as additional columns onto my main query SELECT QueueItemId.. FROM QueueItems I was originally going to put this code in a UDF and call it as part of my main query / stored procedure but I have two issues:
1) The sp_executesql or EXEC statement - not allowed in a UDF
2) To return a table from a UDF I need to define the fields... The fields are dynamic... I could get this working by passing back XML to my stored procedure but I still have issue number 1.
So the question is:
1) Is there any better way or writing this then using a UDF?
2) Is there any other way or transposing the output from parseJSON?
3) Is there any other way of using the PIVOT but not having to specify the columns?
Any help would be much appreciated.
EDIT IN RESPONSE TO M ALI
I know the syntax is wrong and it doesn't work but the query would look something like this:
DECLARE #Cols AS VARCHAR(MAX) = ''
,#Query AS NVARCHAR(MAX) = ''
SELECT
QI.QueueItemID,
QI.QueueItemStatus,
QI.StartProcessing,
QI.EndProcessing,
SD.*
FROM QueueItems AS QI
LEFT JOIN
(
SELECT
#Cols += ',' + Name
FROM
parseJson(QI.SpecificData)
WHERE
Name NOT IN ('DynamicProperties', '-')
SET #Cols = SUBSTRING(#Cols,2,LEN(#Cols))
SET #Query = N'SELECT *
FROM
(
SELECT [StringValue], [Name]
FROM parseJson(''' + QI.SpecificData + ''')
) [d]
PIVOT
(
MAX([StringValue])
FOR [Name] IN (' + #Cols + ')
) [piv]'
EXECUTE (#Query)
) AS SD
And the results would be the columns from QueueItems, joined on the left by columns from the parsed JSON string.
QueueItemID, StartProcessing, EndProcessing, IndentificationIndex, PaymentMethod etc...

Dynamic Pivot - SQL Server

I have a test SQL database the following query:
USE DataBase1
Select Data.MonthDate,
Data.AccountID,
Data.MonthID,
Data.Sales,
Data.AccountName
From Test1 as Data with(nolock)
That I need to pivot based off of the sales column. The problem is the months when I run this query will always change (though there will always be 4 of them) and they need to be ordered left-to-right/oldest-newest in the pivoted result based off of the MonthDate column. The initial return when the query is run looks like this:
And the final result needs to look like this:
I'm using Excel here to demonstrate and I highlighted the 0's because those are technically NULL values but I need them to come back as 0.
I'm using SQL Server Management Studio and the actual database I'll be running this against is over 200,000 rows.
Any thoughts?
Thanks,
Joshua
Use Dynamic Query.
DECLARE #col_list VARCHAR(max)='',
#sel_list VARCHAR(max)='',
#sql NVARCHAR(max)
SELECT DISTINCT #col_list += '[' + Isnull(MonthID, '') + '],'
FROM Test1
ORDER BY MonthID
SELECT #col_list = LEFT(#col_list, Len(#col_list) - 1)
SELECT DISTINCT #sel_list += 'Isnull([' + Isnull(MonthID, '') + '],0) ' + '['+ MonthID + '],'
FROM Test1
ORDER BY MonthID
SELECT #sel_list = LEFT(#sel_list, Len(#sel_list) - 1)
SET #sql ='select Data.AccountID,Data.AccountName,'+ #sel_list+ ' from (
Select
Data.AccountID,
Data.MonthID,
Data.Sales,
Data.AccountName
From Test1 as Data ) A
pivot (sum(Sales) for monthid in('+ #col_list + ')) piv'
--PRINT #sql
EXEC Sp_executesql #sql
Basically you need to dynamically build the PIVOT query and use sp_exec to run it.
SQL Server, out of the box, has no support for dynamic ever-changing columns as the columns need to be defined in the PIVOT query.
Here's an example of how to accomplish this: http://sqlhints.com/tag/dynamic-pivot-column-names/

TSQL: How to add automatically select columns

I have a problem and can't solve it. Furthermore I can't find an answer anywhere in the internet.
Simplified I have a big table with coloumns, where values to products with an ID are stored by year:
year
id
value
In my stored procedure the attributes for getting information are:
#year
#id
If you want to get information about more than one product, you can use a comma-seperated list of product-ids like ('654654,543543,987987').
My TSQL should be like this:
select year,
sum(case when id = #id[1] then value),
sum(case when id = #id[2] then value),
[...]
from table myTable
where year = #year
group by year
order by year
What I want to do is iterate throught the comma-seperated ids and for each id, I want to add a new select attribut like this (sum(case when id = #id[x] then value).
Can you help me with this problems? Any suggestions to solve it?!
Thanks for your help!
PIVOT operation could simplify the query.
But, anyway, it seems that the only way to construct such a query is to use dynamic SQL.
DECLARE
#Ids NVARCHAR(MAX),
#stmt NVARCHAR(MAX)
SET #Ids = '1,2'
-- Transform Ids into the format PIVOT understands - with square brackets.
-- Primitive way, to not overcomplicate sample.
SET #Ids = '[' + REPLACE(#Ids, ',', '], [') + ']'
PRINT #Ids -- [1], [2]
SET #Stmt = '
SELECT *
FROM Products as p
PIVOT
(
SUM(p.Value)
FOR p.Id IN (' + #Ids + ')
) AS t
ORDER BY Year'
EXEC sp_executesql #Stmt
If you need more accurate way of splitting a comma separated list into an array (table), please see this article for details.
This example is available on SQL Fiddle
As you are using stored procedure, you can use sp_executesql to executed dynimically build SQL statement.
So, you have to iterate over the CSV like this:
DECLARE #List NVARCHAR(MAX) = N'1001,dada,1002,1003'
DECLARE #ProductsID TABLE ( [ID] BIGINT )
DECLARE #XML xml = N'<r><![CDATA[' + REPLACE(#List, ',', ']]></r><r><![CDATA[') + ']]></r>'
INSERT INTO #ProductsID ([ID])
SELECT DISTINCT CAST(Tbl.Col.value('.', 'float') AS bigint)
FROM #xml.nodes('//r') Tbl(Col)
WHERE ISNUMERIC(Tbl.Col.value('.', 'varchar(max)')) = 1
SELECT [ID] FROM #ProductsID
Then, having a table with the ID to dynamically build you SQL statement and execute it.

Dynamic SQL: Grouping by one variable, counting another for column names

I am trying to do a dynamic sql query, similar to some that have appeared on this forum, but for the life of me, I cannot get it to work.
I am using SQL Server 2008. I have a table with a series of order_ref numbers. Each of these numbers has a varying number of advice_refs associated with it. advice_ref numbers are unique (they are a key from another table). There is at least one advice_ref for each order_ref. There are a bunch of columns that describe information for each advice_ref.
What I want to do is create a table with a row for each unique order_ref, with columns for each advice_ref, in ascending order. The columns would be Advice01, Advice02, ....Advice10, Advice11, etc. Not all the Advice# columns would be filled in for every order_ref and the number of advice# columns would depend on the order_ref with the greatest number of advice_refs.
The table would look like:
Order Advice01 Advice02 Advice03 Advice04.....
1 1 2 3
2 5 8 9 20
3 25
The code I've tried to use is:
DECLARE #SQL NVARCHAR(MAX)
DECLARE #PVT NVARCHAR(MAX)
SELECT #SQL = #SQL + ', COALESCE(' + QUOTENAME('Advice' + RowNum) + ', '''') AS ' + QUOTENAME('Advice' + RowNum),
#PVT = #PVT + ', ' + QUOTENAME('Advice' + RowNum)
FROM (SELECT case when RowNum2 < 10 then '0'+RowNum2 when RowNum2 >=10 then RowNum2 end [RowNum] From
( SELECT DISTINCT CONVERT(VARCHAR, ROW_NUMBER() OVER(PARTITION BY order_ref ORDER BY advice_ref)) [RowNum2]
FROM [ED_dups].[dbo].[NewEDDupsLongForm]
) rn2 ) rn
SET #SQL = 'SELECT order_ref' + #SQL + '
FROM ( SELECT order_ref,
advice_ref,
case when CONVERT(VARCHAR, ROW_NUMBER() OVER(PARTITION BY order_ref ORDER BY advice_ref)) < 10
then ''Advice0'' + CONVERT(VARCHAR, ROW_NUMBER() OVER(PARTITION BY order_ref ORDER BY advice_ref))
else ''Advice'' + CONVERT(VARCHAR, ROW_NUMBER() OVER(PARTITION BY order_ref ORDER BY advice_ref))
end [AdviceID]
FROM [ED_dups].[dbo].[NewEDDupsLongForm]
) data
PIVOT
( MAX(advice_ref)
FOR AdviceID IN (' + STUFF(#PVT, 1, 2, '') + ')
) pvt'
EXECUTE SP_EXECUTESQL #SQL
SQL server tells me that the query executed successfully, but there is no output. When I run snippets of the code, it seems that the problem either lies in the pivot statement, near
+ STUFF(#PVT, 1, 2, '') + ')
and/or in the select statement, near
''Advice0'' +
Thanks in advance for any help--I've been at this for days!
I think you have to initialize variables like
DECLARE #SQL NVARCHAR(MAX) = ''
DECLARE #PVT NVARCHAR(MAX) = ''
or
DECLARE #SQL NVARCHAR(MAX)
DECLARE #PVT NVARCHAR(MAX)
SELECT #SQL = '', #PVT = ''
Otherwise your #SQL would be null
fist thing that comes to my mind is - do you really need SQL to fetch you dataset with dynamic number of columns? If you are writting an application, then your user interface, being it a web page or desktop app form, would be much nicer place to transform your data into a desired structure.
If you really need to do so, you will make your life much easier when you will not try to do everything in one big and rather complicated query, but rather split it into smaller tasks done step by step. What I would do is to use temporary tables to store working results, then use cursors to process order by order and advice by advice while inserting my data into temporary table or tables, in the end return a content of this table. Wrap everything in a stored procedure.
This method will also allow you to debug it easier - you can check every single step if it has done what it was expected to do.
And final advice - share a definition of your NewEDDupsLongForm table - someone might write some code to help you out then.
cheers