Query Transposing certain rows into column names - sql

I have a couple of tables which look like this
Table 1
user_id | name
-------------------------
x111 | Smith, James
x112 | Smith, Jane
etc..
Table 2
id | code | date | incident_code | user_id
-----------------------------------------------------------------
1 | 102008 | 10/20/2008 | 1 | x111
2 | 113008 | 11/30/2008 | 3 | x111
3 | 102008 | 10/20/2008 | 2 | x112
4 | 113008 | 11/30/2008 | 5 | x112
What i'd like to display is something like this
user_id | user_name | INCIDENT IN OCT 2008 | INCIDENT IN NOV 2008
------------------------------------------------------------------------------
x111 | Smith, John | 1 | 3
x112 | Smith, Jane | 2 | 5
etc..
The incident_code would be replaced by the actual description of the incident which is located in another table, but i thought i'd see how this would work first.
Some of the column headers would be static while others would be created based on the date.
Does anyone one know how i can do this using sql server 2005? Some examples would be very helpful.
Thanks in advance

Here's a solution which generates and runs the dynamic SQL with a PIVOT:
DECLARE #pivot_list AS VARCHAR(MAX)
--
;
WITH cols
AS ( SELECT DISTINCT
'INCIDENT IN ' + LEFT(UPPER(CONVERT(VARCHAR, [date], 107)),
3) + ' '
+ SUBSTRING(UPPER(CONVERT(VARCHAR, [date], 107)), 9, 4) AS col
FROM so926209_2
)
SELECT #pivot_list = COALESCE(#pivot_list + ', ', '') + '[' + col + ']'
FROM cols
--
DECLARE #template AS VARCHAR(MAX)
SET #template = 'WITH incidents AS (
SELECT [user_id],
incident_code,
''INCIDENT IN '' + LEFT(UPPER(CONVERT(VARCHAR, [date], 107)), 3)
+ '' '' + SUBSTRING(UPPER(CONVERT(VARCHAR, [date], 107)), 9, 4) AS col
FROM so926209_2
)
,results AS (
SELECT * FROM incidents PIVOT (MAX(incident_code) FOR col IN ({#pivot_list})) AS pvt
)
SELECT results.[user_id]
,so926209_1.[name]
,{#select_list}
FROM results INNER JOIN so926209_1 ON so926209_1.[user_id] = results.[user_id]
'
DECLARE #sql AS VARCHAR(MAX)
SET #sql = REPLACE(REPLACE(#template, '{#pivot_list}', #pivot_list), '{#select_list}', #pivot_list)
--PRINT #sql
EXEC (#sql)
Where so926209_1, so926209_2 are your table 1 and table 2
Note that if you have multiple incidents in a month for the same person, your example doesn't show how you want that handled. This example only takes the last incident in the month.

You want to Pivot
http://msdn.microsoft.com/en-us/library/ms177410.aspx

This sounds like a reporting task. Reporting, often referred to from a database perspective as OLAP, Online Aanalytical Processing, tends to differ quite frequently from "traditional" database access, OLTP (Online Transaction Processing) in that it is quite often made up of large aggregations of data spanning greater periods of time. Quite frequently, the kind of aggregation your looking for.
Use of a Pivot as Tetraneutron suggested will be sufficient for smaller data sets. However, as the volume of data you need to report on grows, you may need something more advanced. OLAP is provided for by SQL Server Analysis Services (SSAS), available in 2005 and 2008. Using SSAS you can create multidimensional data repositories that pre-aggregate data from either an OLTP database directly, or from an intermediary data warehouse database. Multidimensional data (usually referred to as cubes), provide a much faster way to access the kind of data you can get from a Pivot, without interfering with the performance of your standard transaction processing in your OLTP database.
If you have more than a small amount of data you need to report on, I recommend you check out SQL Server Analysis Services 2005, OLAP, Cubes, and MDX (Multidimensional Extensions for T-SQL.) There is a larger learnig curve to set up an OLAP Cube, but once it is set up, the benefits of having one can be huge if you have significant reporting needs.

A query like this would work:
select
u.User_id,
u.Name,
Okt2008Sum = sum(case when i.date between
'2008-10-01' and '2008-11-01' then 1 else 0 end),
Nov2008Sum = sum(case when i.date between
'2008-11-01' and '2008-12-01'then 1 else 0 end)
from #incidents i
inner join #users u on i.user_id = u.user_id
group by u.user_id, u.name
Depending on your client and how often you have to run it, you can generate this query. In SQL this would look like:
create table #months (
MonthName varchar(25),
StartDate datetime
)
insert into #months values ('Okt2008','2008-10-01')
insert into #months values ('Nov2008','2008-11-01')
declare #query varchar(8000)
select #query = 'select u.User_id, u.Name '
select #query = #query + ', ' + MonthName +
' = sum(case when i.date between ''' + cast(StartDate as varchar) +
''' and ''' + cast(dateadd(m,1,StartDate) as varchar) +
''' then 1 else 0 end) '
from #Months
select #query = #query + '
from #incidents i
inner join #users u on i.user_id = u.user_id
group by u.user_id, u.name'
exec (#query)

Related

How to convert data in one row into one column in SQL Server [duplicate]

How do I simply switch columns with rows in SQL?
Is there any simple command to transpose?
ie turn this result:
Paul | John | Tim | Eric
Red 1 5 1 3
Green 8 4 3 5
Blue 2 2 9 1
into this:
Red | Green | Blue
Paul 1 8 2
John 5 4 2
Tim 1 3 9
Eric 3 5 1
PIVOT seems too complex for this scenario.
There are several ways that you can transform this data. In your original post, you stated that PIVOT seems too complex for this scenario, but it can be applied very easily using both the UNPIVOT and PIVOT functions in SQL Server.
However, if you do not have access to those functions this can be replicated using UNION ALL to UNPIVOT and then an aggregate function with a CASE statement to PIVOT:
Create Table:
CREATE TABLE yourTable([color] varchar(5), [Paul] int, [John] int, [Tim] int, [Eric] int);
INSERT INTO yourTable
([color], [Paul], [John], [Tim], [Eric])
VALUES
('Red', 1, 5, 1, 3),
('Green', 8, 4, 3, 5),
('Blue', 2, 2, 9, 1);
Union All, Aggregate and CASE Version:
select name,
sum(case when color = 'Red' then value else 0 end) Red,
sum(case when color = 'Green' then value else 0 end) Green,
sum(case when color = 'Blue' then value else 0 end) Blue
from
(
select color, Paul value, 'Paul' name
from yourTable
union all
select color, John value, 'John' name
from yourTable
union all
select color, Tim value, 'Tim' name
from yourTable
union all
select color, Eric value, 'Eric' name
from yourTable
) src
group by name
See SQL Fiddle with Demo
The UNION ALL performs the UNPIVOT of the data by transforming the columns Paul, John, Tim, Eric into separate rows. Then you apply the aggregate function sum() with the case statement to get the new columns for each color.
Unpivot and Pivot Static Version:
Both the UNPIVOT and PIVOT functions in SQL server make this transformation much easier. If you know all of the values that you want to transform, you can hard-code them into a static version to get the result:
select name, [Red], [Green], [Blue]
from
(
select color, name, value
from yourtable
unpivot
(
value for name in (Paul, John, Tim, Eric)
) unpiv
) src
pivot
(
sum(value)
for color in ([Red], [Green], [Blue])
) piv
See SQL Fiddle with Demo
The inner query with the UNPIVOT performs the same function as the UNION ALL. It takes the list of columns and turns it into rows, the PIVOT then performs the final transformation into columns.
Dynamic Pivot Version:
If you have an unknown number of columns (Paul, John, Tim, Eric in your example) and then an unknown number of colors to transform you can use dynamic sql to generate the list to UNPIVOT and then PIVOT:
DECLARE #colsUnpivot AS NVARCHAR(MAX),
#query AS NVARCHAR(MAX),
#colsPivot as NVARCHAR(MAX)
select #colsUnpivot = stuff((select ','+quotename(C.name)
from sys.columns as C
where C.object_id = object_id('yourtable') and
C.name <> 'color'
for xml path('')), 1, 1, '')
select #colsPivot = STUFF((SELECT ','
+ quotename(color)
from yourtable t
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1,1,'')
set #query
= 'select name, '+#colsPivot+'
from
(
select color, name, value
from yourtable
unpivot
(
value for name in ('+#colsUnpivot+')
) unpiv
) src
pivot
(
sum(value)
for color in ('+#colsPivot+')
) piv'
exec(#query)
See SQL Fiddle with Demo
The dynamic version queries both yourtable and then the sys.columns table to generate the list of items to UNPIVOT and PIVOT. This is then added to a query string to be executed. The plus of the dynamic version is if you have a changing list of colors and/or names this will generate the list at run-time.
All three queries will produce the same result:
| NAME | RED | GREEN | BLUE |
-----------------------------
| Eric | 3 | 5 | 1 |
| John | 5 | 4 | 2 |
| Paul | 1 | 8 | 2 |
| Tim | 1 | 3 | 9 |
This normally requires you to know ALL the column AND row labels beforehand. As you can see in the query below, the labels are all listed in their entirely in both the UNPIVOT and the (re)PIVOT operations.
MS SQL Server 2012 Schema Setup:
create table tbl (
color varchar(10), Paul int, John int, Tim int, Eric int);
insert tbl select
'Red' ,1 ,5 ,1 ,3 union all select
'Green' ,8 ,4 ,3 ,5 union all select
'Blue' ,2 ,2 ,9 ,1;
Query 1:
select *
from tbl
unpivot (value for name in ([Paul],[John],[Tim],[Eric])) up
pivot (max(value) for color in ([Red],[Green],[Blue])) p
Results:
| NAME | RED | GREEN | BLUE |
-----------------------------
| Eric | 3 | 5 | 1 |
| John | 5 | 4 | 2 |
| Paul | 1 | 8 | 2 |
| Tim | 1 | 3 | 9 |
Additional Notes:
Given a table name, you can determine all the column names from sys.columns or FOR XML trickery using local-name().
You can also build up the list of distinct colors (or values for one column) using FOR XML.
The above can be combined into a dynamic sql batch to handle any table.
I'd like to point out few more solutions to transposing columns and rows in SQL.
The first one is - using CURSOR. Although the general consensus in the professional community is to stay away from SQL Server Cursors, there are still instances whereby the use of cursors is recommended. Anyway, Cursors present us with another option to transpose rows into columns.
Vertical expansion
Similar to the PIVOT, the cursor has the dynamic capability to append more rows as your dataset expands to include more policy numbers.
Horizontal expansion
Unlike the PIVOT, the cursor excels in this area as it is able to expand to include newly added document, without altering the script.
Performance breakdown
The major limitation of transposing rows into columns using CURSOR is a disadvantage that is linked to using cursors in general – they come at significant performance cost. This is because the Cursor generates a separate query for each FETCH NEXT operation.
Another solution of transposing rows into columns is by using XML.
The XML solution to transposing rows into columns is basically an optimal version of the PIVOT in that it addresses the dynamic column limitation.
The XML version of the script addresses this limitation by using a combination of XML Path, dynamic T-SQL and some built-in functions (i.e. STUFF, QUOTENAME).
Vertical expansion
Similar to the PIVOT and the Cursor, newly added policies are able to be retrieved in the XML version of the script without altering the original script.
Horizontal expansion
Unlike the PIVOT, newly added documents can be displayed without altering the script.
Performance breakdown
In terms of IO, the statistics of the XML version of the script is almost similar to the PIVOT – the only difference is that the XML has a second scan of dtTranspose table but this time from a logical read – data cache.
You can find some more about these solutions (including some actual T-SQL exmaples) in this article:
https://www.sqlshack.com/multiple-options-to-transposing-rows-into-columns/
Based on this solution from bluefeet here is a stored procedure that uses dynamic sql to generate the transposed table. It requires that all the fields are numeric except for the transposed column (the column that will be the header in the resulting table):
/****** Object: StoredProcedure [dbo].[SQLTranspose] Script Date: 11/10/2015 7:08:02 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: Paco Zarate
-- Create date: 2015-11-10
-- Description: SQLTranspose dynamically changes a table to show rows as headers. It needs that all the values are numeric except for the field using for transposing.
-- Parameters: #TableName - Table to transpose
-- #FieldNameTranspose - Column that will be the new headers
-- Usage: exec SQLTranspose <table>, <FieldToTranspose>
-- =============================================
ALTER PROCEDURE [dbo].[SQLTranspose]
-- Add the parameters for the stored procedure here
#TableName NVarchar(MAX) = '',
#FieldNameTranspose NVarchar(MAX) = ''
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE #colsUnpivot AS NVARCHAR(MAX),
#query AS NVARCHAR(MAX),
#queryPivot AS NVARCHAR(MAX),
#colsPivot as NVARCHAR(MAX),
#columnToPivot as NVARCHAR(MAX),
#tableToPivot as NVARCHAR(MAX),
#colsResult as xml
select #tableToPivot = #TableName;
select #columnToPivot = #FieldNameTranspose
select #colsUnpivot = stuff((select ','+quotename(C.name)
from sys.columns as C
where C.object_id = object_id(#tableToPivot) and
C.name <> #columnToPivot
for xml path('')), 1, 1, '')
set #queryPivot = 'SELECT #colsResult = (SELECT '',''
+ quotename('+#columnToPivot+')
from '+#tableToPivot+' t
where '+#columnToPivot+' <> ''''
FOR XML PATH(''''), TYPE)'
exec sp_executesql #queryPivot, N'#colsResult xml out', #colsResult out
select #colsPivot = STUFF(#colsResult.value('.', 'NVARCHAR(MAX)'),1,1,'')
set #query
= 'select name, rowid, '+#colsPivot+'
from
(
select '+#columnToPivot+' , name, value, ROW_NUMBER() over (partition by '+#columnToPivot+' order by '+#columnToPivot+') as rowid
from '+#tableToPivot+'
unpivot
(
value for name in ('+#colsUnpivot+')
) unpiv
) src
pivot
(
sum(value)
for '+#columnToPivot+' in ('+#colsPivot+')
) piv
order by rowid'
exec(#query)
END
You can test it with the table provided with this command:
exec SQLTranspose 'yourTable', 'color'
I'm doing UnPivot first and storing the results in CTE and using the CTE in Pivot operation.
Demo
with cte as
(
select 'Paul' as Name, color, Paul as Value
from yourTable
union all
select 'John' as Name, color, John as Value
from yourTable
union all
select 'Tim' as Name, color, Tim as Value
from yourTable
union all
select 'Eric' as Name, color, Eric as Value
from yourTable
)
select Name, [Red], [Green], [Blue]
from
(
select *
from cte
) as src
pivot
(
max(Value)
for color IN ([Red], [Green], [Blue])
) as Dtpivot;
Adding to #Paco Zarate's terrific answer above, if you want to transpose a table which has multiple types of columns, then add this to the end of line 39, so it only transposes int columns:
and C.system_type_id = 56 --56 = type int
Here is the full query that is being changed:
select #colsUnpivot = stuff((select ','+quotename(C.name)
from sys.columns as C
where C.object_id = object_id(#tableToPivot) and
C.name <> #columnToPivot and C.system_type_id = 56 --56 = type int
for xml path('')), 1, 1, '')
To find other system_type_id's, run this:
select name, system_type_id from sys.types order by name
This way Convert all Data From Filelds(Columns) In Table To Record (Row).
Declare #TableName [nvarchar](128)
Declare #ExecStr nvarchar(max)
Declare #Where nvarchar(max)
Set #TableName = 'myTableName'
--Enter Filtering If Exists
Set #Where = ''
--Set #ExecStr = N'Select * From '+quotename(#TableName)+#Where
--Exec(#ExecStr)
Drop Table If Exists #tmp_Col2Row
Create Table #tmp_Col2Row
(Field_Name nvarchar(128) Not Null
,Field_Value nvarchar(max) Null
)
Set #ExecStr = N' Insert Into #tmp_Col2Row (Field_Name , Field_Value) '
Select #ExecStr += (Select N'Select '''+C.name+''' ,Convert(nvarchar(max),'+quotename(C.name) + ') From ' + quotename(#TableName)+#Where+Char(10)+' Union All '
from sys.columns as C
where (C.object_id = object_id(#TableName))
for xml path(''))
Select #ExecStr = Left(#ExecStr,Len(#ExecStr)-Len(' Union All '))
--Print #ExecStr
Exec (#ExecStr)
Select * From #tmp_Col2Row
Go
I like to share the code i'm using to transpose a splited text based on +bluefeet answer. In this aproach i'm implemented as a procedure in MS SQL 2005
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: ELD.
-- Create date: May, 5 2016.
-- Description: Transpose from rows to columns the user split function.
-- =============================================
CREATE PROCEDURE TransposeSplit #InputToSplit VARCHAR(8000)
,#Delimeter VARCHAR(8000) = ','
AS
BEGIN
SET NOCOUNT ON;
DECLARE #colsUnpivot AS NVARCHAR(MAX)
,#query AS NVARCHAR(MAX)
,#queryPivot AS NVARCHAR(MAX)
,#colsPivot AS NVARCHAR(MAX)
,#columnToPivot AS NVARCHAR(MAX)
,#tableToPivot AS NVARCHAR(MAX)
,#colsResult AS XML
SELECT #tableToPivot = '#tempSplitedTable'
SELECT #columnToPivot = 'col_number'
CREATE TABLE #tempSplitedTable (
col_number INT
,col_value VARCHAR(8000)
)
INSERT INTO #tempSplitedTable (
col_number
,col_value
)
SELECT ROW_NUMBER() OVER (
ORDER BY (
SELECT 100
)
) AS RowNumber
,item
FROM [DB].[ESCHEME].[fnSplit](#InputToSplit, #Delimeter)
SELECT #colsUnpivot = STUFF((
SELECT ',' + quotename(C.NAME)
FROM [tempdb].sys.columns AS C
WHERE C.object_id = object_id('tempdb..' + #tableToPivot)
AND C.NAME <> #columnToPivot
FOR XML path('')
), 1, 1, '')
SET #queryPivot = 'SELECT #colsResult = (SELECT '',''
+ quotename(' + #columnToPivot + ')
from ' + #tableToPivot + ' t
where ' + #columnToPivot + ' <> ''''
FOR XML PATH(''''), TYPE)'
EXEC sp_executesql #queryPivot
,N'#colsResult xml out'
,#colsResult OUT
SELECT #colsPivot = STUFF(#colsResult.value('.', 'NVARCHAR(MAX)'), 1, 1, '')
SET #query = 'select name, rowid, ' + #colsPivot + '
from
(
select ' + #columnToPivot + ' , name, value, ROW_NUMBER() over (partition by ' + #columnToPivot + ' order by ' + #columnToPivot + ') as rowid
from ' + #tableToPivot + '
unpivot
(
value for name in (' + #colsUnpivot + ')
) unpiv
) src
pivot
(
MAX(value)
for ' + #columnToPivot + ' in (' + #colsPivot + ')
) piv
order by rowid'
EXEC (#query)
DROP TABLE #tempSplitedTable
END
GO
I'm mixing this solution with the information about howto order rows without order by (SQLAuthority.com) and the split function on MSDN (social.msdn.microsoft.com)
When you execute the prodecure
DECLARE #RC int
DECLARE #InputToSplit varchar(MAX)
DECLARE #Delimeter varchar(1)
set #InputToSplit = 'hello|beautiful|world'
set #Delimeter = '|'
EXECUTE #RC = [TransposeSplit]
#InputToSplit
,#Delimeter
GO
you obtaint the next result
name rowid 1 2 3
col_value 1 hello beautiful world
I was able to use Paco Zarate's solution and it works beautifully. I did have to add one line ("SET ANSI_WARNINGS ON"), but that may be something unique to the way I used it or called it. There is a problem with my usage and I hope someone can help me with it:
The solution works only with an actual SQL table. I tried it with a temporary table and also an in-memory (declared) table but it doesn't work with those. So in my calling code I create a table on my SQL database and then call SQLTranspose. Again, it works great. It's just what I want. Here's my problem:
In order for the overall solution to be truly dynamic I need to create that table where I temporarily store the prepared information that I'm sending to SQLTranspose "on the fly", and then delete that table once SQLTranspose is called. The table deletion is presenting a problem with my ultimate implementation plan. The code needs to run from an end-user application (a button on a Microsoft Access form/menu). When I use this SQL process (create a SQL table, call SQLTranspose, delete SQL table) the end user application hits an error because the SQL account used does not have the rights to drop a table.
So I figure there are a few possible solutions:
Find a way to make SQLTranspose work with a temporary table or a declared table variable.
Figure out another method for the transposition of rows and columns that doesn't require an actual SQL table.
Figure out an appropriate method of allowing the SQL account used by my end users to drop a table. It's a single shared SQL account coded into my Access application. It appears that permission is a dbo-type privilege that cannot be granted.
I recognize that some of this may warrant another, separate thread and question. However, since there is a possibility that one solution may be simply a different way to do the transposing of rows and columns I'll make my first post here in this thread.
EDIT: I also did replace sum(value) with max(value) in the 6th line from the end, as Paco suggested.
EDIT:
I figured out something that works for me. I don't know if it's the best answer or not.
I have a read-only user account that is used to execute strored procedures and therefore generate reporting output from a database. Since the SQLTranspose function I created will only work with a "legitimate" table (not a declared table and not a temporary table) I had to figure out a way for a read-only user account to create (and then later delete) a table.
I reasoned that for my purposes it's okay for the user account to be allowed to create a table. The user still could not delete the table though. My solution was to create a schema where the user account is authorized. Then whenever I create, use, or delete that table refer it with the schema specified.
I first issued this command from a 'sa' or 'sysadmin' account:
CREATE SCHEMA ro AUTHORIZATION
When any time I refer to my "tmpoutput" table I specify it like this example:
drop table ro.tmpoutput

Making columns from one column's data in SQL Server

I have a SQL Server table like this
ID amount type
1 10 material
1 5 spare parts
1 5 material
I need to make a query and get outputs like this
ID material spare parts
1 15 5
But I have too many IDS and types, so I need to add items dynamically whatever their count.
You are looking for a dynamic pivot. Basically this works selecting the list of types from the table, then building a query from that information. You can then execute the query with sp_executesql.
For your table structure:
declare #sql nvarchar(max);
select #sql = string_agg(
'sum(case when type = ''' + type + ''' then amount else 0 end) [' + type + ']',
', ')
from (select distinct type from mytable) t;
set #sql = N'select id, ' + #sql + ' from mytable group by id';
select #sql; -- debug
-- exec sp_executesql #sql; -- execute for real
For your sample data, this generates the following query (I added line breaks added for readability):
select
id,
sum(case when type = 'material' then amount else 0 end) [material],
sum(case when type = 'spare parts' then amount else 0 end) [spare parts]
from mytable
group by id
After execution, you get result:
id | material | spare parts
-: | -------: | ----------:
1 | 15 | 5
Demo on DB Fiddle

Transforming an SQL table

I am trying to write a nice SQL query, but I am failing. I am trying to create a select statement that returns the following:
Date | Cash | Checking | Savings
--------------------------------------
May 2017 | 30 | 300 | 3000
Apr 2017 | 40 | 400 | 4000
My tables are like this:
Balances
* id
* date
* amount
* item_id
BalanceItems
* id
* name
So the column names (cash, checking and savings for example) are stored in BalanceItems. Basically I don't know how I can get the name from BalanceItems in the column header.
I am building this in Django, so ideas of how to elegantly do this there are also appreciated. These are the models:
class Balances(models.Model):
item = models.ForeignKey('BalanceItems', related_name='balances_balance_items')
date = models.DateField()
amount = models.DecimalField(max_digits=9, decimal_places=2)
class BalanceItems(models.Model):
name = models.CharField(max_length=100, unique=True)
If we know the expected values, we can use a pivot:
select [date], [Cash], [Checking], [Savings]
from
(
select [date]
, amount
, name
from Balances b
inner join BalanceItems bi
on bi.id = b.item_id
) source
pivot
(
sum(amount)
for name in ([Cash], [Checking], [Savings])
) pvt
order by [date]
Otherwise we need dynamic sql; though the solution's otherwise the same as the one above:
declare #sql nvarchar(max)
select #sql = coalesce(#sql + ', ', '') + quotename(name)
from #BalanceItems
set #sql = '
select [date], ' + #sql + '
from
(
select [date]
, amount
, name
from Balances b
inner join BalanceItems bi
on bi.id = b.item_id
) source
pivot
(
sum(amount)
for name in (' + #sql + ')
) pvt
order by [date]
'
exec (#sql)
It seems I might be able to use pivot to get my desired result but that command doesn't exist in sqlite. I don't want to use case statements because the amount of columns can change.
So I have ended up accepting that this one view is going to do a whole bunch of database calls and use Python to keep everything organized.

T-Sql Query with dynamic (unknown) number of columns

We have a project where we should provide the possible to the user to add own custom columns to various tables.
Edit: these are 2 tables, not one.
**Products**
ProductId
Name
Price
Date
UserId
**ProductsCustomColumns**
ProductId
ColumnName
ColumnValue
EDIT: Please note that the dynamic columns are recorded as values and we don't know the count of these...it can be 0 or 200 or any.
Here is an example:
Now when we query the products tables we want to show all the predefined columns and after them all custom columns.
Obviously each user can have own number of columns with values and names.
SELECT *, (and the custom columns) FROM Products WHERE UserId = 3 AND ProductId = 1
Here are 2 questions:
Would that be good solution from performance point of view or there is better approach for solving the dynamic columns requirement?
How can I create a query that could read all records from ProductsCustomColumns for given userId and productId and append the records as columns to the query?
Thanks.
You need to write dynamic Query
DECLARE #COLUMNS VARCHAR(MAX)='', #QRY VARCHAR(MAX);
SELECT #COLUMNS = #COLUMNS +COLUMN_NAME +',' FROM
INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='Products'
SELECT #COLUMNS =SUBSTRING (#COLUMNS,1 ,LEN(#COLUMNS)-1)
SELECT #QRY ='SELECT '+#COLUMNS + ' FROM Products WHERE UserId = 3 AND ProductId = 1'
EXEC (#QRY)
EDIT: From your Comments & Edited Question
Schema I assumed from your Question
CREATE TABLE Products (
ProductId INT,
Name VARCHAR(250),
Price DECIMAL(18,2),
DateS DATETIME,
UserId INT)
INSERT INTO Products
SELECT 1,'Oil Product', 2000, GETDATE(), 3
UNION ALL
SELECT 2,'Amway', 600, GETDATE(), 2
UNION ALL
SELECT 3,'Thermal', 5000, GETDATE(), 1
UNION ALL
SELECT 4,'Oil Product', 500, GETDATE(), 4
CREATE TABLE ProductsCustomColumns
(
ProductId INT ,
ColumnName VARCHAR(200),
ColumnValue VARCHAR(15))
INSERT INTO ProductsCustomColumns
SELECT 1, 'Licence_No', '1545'
UNION ALL
SELECT 1, 'Location ', 'Atlanta'
UNION ALL
SELECT 2, 'Qty ', '5'
UNION ALL
SELECT 3, 'Gross', '80000'
Now your Dynamic Code goes here
DECLARE #COLUMN_PCC VARCHAR(MAX)='', #PRODUCT_ID INT=1,#USER_ID INT=3, #QRY VARCHAR(MAX) ;
--preparing Custom Column Name List with comma ','
SELECT #COLUMN_PCC = #COLUMN_PCC+ [COLUMNNAME] +',' FROM ProductsCustomColumns
WHERE ProductId= #PRODUCT_ID
SELECT #COLUMN_PCC =SUBSTRING(#COLUMN_PCC,1,LEN(#COLUMN_PCC)-1)
--Preparing Dynamic Query
SELECT #QRY =' SELECT P.*, AV.* FROM Products P
INNER JOIN
(
SELECT * FROM (
SELECT * FROM ProductsCustomColumns WHERE ProductId= '+CAST(#PRODUCT_ID AS VARCHAR(50))+'
)
AS A
PIVOT
(
MAX (COLUMNVALUE)
FOR [COLUMNNAME] IN ('+#COLUMN_PCC +')
)AS PVT
)AS AV ON P.ProductId= AV.ProductId
AND P.UserId='++CAST(#USER_ID AS VARCHAR(50))+'
'
EXEC ( #QRY)
And the Result will be
+-----------+-------------+---------+-------------------------+--------+-----------+------------+----------+
| ProductId | Name | Price | DateS | UserId | ProductId | Licence_No | Location |
+-----------+-------------+---------+-------------------------+--------+-----------+------------+----------+
| 1 | Oil Product | 2000.00 | 2016-12-09 18:06:24.090 | 3 | 1 | 1545 | Atlanta |
+-----------+-------------+---------+-------------------------+--------+-----------+------------+----------+
You need dynamic sql no other way to do this
DECLARE #sql VARCHAR(max),
#cust_col VARCHAR(max)
SET #cust_col = (SELECT Quotename(CustomColumns) + ','
FROM ProductsCustomColumns
FOR xml path(''))
SELECT #cust_col = LEFT(#cust_col, Len(#cust_col) - 1)
SET #sql = 'SELECT *, ' + #cust_col + ' FROM Products WHERE UserId = 3 AND ProductId = 1'
EXEC (#sql)
In general it is a very bad idea to add custom data in additional columns of your main table. Just imagine 100 customers using this. All of them have differing table schemas and you wnat to write an update script for all of them?
It is a pain in the neck, if you have to deal with result sets where you don't know the structure in advance!
You have several choices:
Add one column of type XML. The advantage: The resultset is fix. You just need a customer specific rule, how to interpret the XML. You can solve this with an inline table valued function. Pass in the XML and get a derived table back. Call this with CROSS APPLY and you are out...
Add a new table with the customerID and Key-Value-Pairs
If the additional data is not completely different, add some of the columns to your main table as SPARSE columns

How do i transform rows into columns in sql server 2005

There is a question here in stackoverflow with the same title but that is not what I am looking for.
I have a table like the one below
Name | Count
----------------
Chery | 257
Drew | 1500
Morgon | 13
Kath | 500
Kirk | 200
Matt | 76
I need to trasform this result set into something like this
Chery | Drew | Morgon | Kath | Kirk | Matt
-------------------------------------------
257 1500 13 500 200 76
How do i acheive this using sql server 2005?
There are similar questions here,here answered in stackoverflow.
You need to use the operator PIVOT in your query to acheive this.Here is the example and explanation on how you can do that.The example is referenced from this source.
---I assumed your tablename as TESTTABLE---
DECLARE #cols NVARCHAR(2000)
DECLARE #query NVARCHAR(4000)
SELECT #cols = STUFF(( SELECT DISTINCT TOP 100 PERCENT
'],[' + t.Name
FROM TESTTABLE AS t
ORDER BY '],[' + t.Name
FOR XML PATH('')
), 1, 2, '') + ']'
SET #query = N'SELECT '+ #cols +' FROM
(SELECT t1.Name , t1.Count FROM TESTTABLE AS t1) p
PIVOT (MAX([Count]) FOR Name IN ( '+ #cols +' ))
AS pvt;'
EXECUTE(#query)
Explanation
1.The first part of the query
SELECT #cols = STUFF(( SELECT DISTINCT TOP 100 PERCENT
'],[' + t.Name
FROM TESTTABLE AS t
ORDER BY '],[' + t.Name
FOR XML PATH('')
), 1, 2, '') + ']'
gives you a nice flattened result of your Name column values in a single row as follow
[Cheryl],[Drew],[Karen],[Kath],[Kirk],[Matt]
You can learn more about the STUFF and XML PATH here and here.
2.SELECT + #cols + FROM will select all the rows as coloumn names for the final result set (pvt - step 3)
i.e
Select [Chery],[Drew],[Morgan],[Kath],[Kirk],[Matt]
3.This query pulls all the rows of data that we need to create the cross-tab results. The (p) after the query is creating a temporary table of the results that can then be used to satisfy the query for step 1.
(SELECT t1.Name, t1.Count FROM TESTTABLE AS t1) p
4.The PIVOT expression
PIVOT (MAX (Count) FOR Name IN ( #cols) AS pvt
does the actual summarization and puts the results into a temporary table called pvt as
Chery | Drew | Morgon | Kath | Kirk | Matt
-------------------------------------------
257 1500 13 500 200 76
See Using PIVOT and UNPIVOT.
You can use the PIVOT and UNPIVOT
relational operators to change a
table-valued expression into another
table. PIVOT rotates a table-valued
expression by turning the unique
values from one column in the
expression into multiple columns in
the output, and performs aggregations
where they are required on any
remaining column values that are
wanted in the final output. UNPIVOT
performs the opposite operation to
PIVOT by rotating columns of a
table-valued expression into column
values.
The quick answer is
SELECT Chery, Drew, Morgon, Kath, Kirk, Matt
FROM
(SELECT [Name], [Count] From Foo)
PIVOT
(
MIN([Count])
FOR [Name] IN (Chery, Drew, Morgon, Kath, Kirk, Matt)
) AS PivotTable
If you want to avoid anything complicated like a pivot or the accepted answer, you can do this! (most of the code is just setting up the test data just in case anyone wants to try it)
/* set up your test table */
declare #TestData table (Name Varchar(80),[Count] int)
insert into #TestData (Name, [count])
Select 'Chery' as name, 257 as [count]
union all select 'Drew', 1500
union all select 'Morgon',13
union all select 'Kath', 500
union all select 'Kirk', 200
union all select 'Matt', 76
/* the query */
Declare #Query Varchar(max)
Select #Query=Coalesce(#query+', ','SELECT ') +Convert(VarChar(5),[count]) +' as ['+name+']'
from #TestData
Execute (#Query)
/* result
Chery Drew Morgon Kath Kirk Matt
----------- ----------- ----------- ----------- ----------- -----------
257 1500 13 500 200 76
*/