I have many rows for the same data but each row has a different columns populated. I want to join all these rows into a single row. I have tried group by, but the dataset has 1000 columns. Any suggestions?
SELECT TOP 100 *
FROM [Hilltopsamplerfix].[dbo].[tempHilltopWaterQualityExtractPivot]
WHERE [SiteName] = 'site' AND [RunDate] = 'xxx'
Example of the data
I can't paste an image because my reputation is under 10
+------------------------------+
| Site column1 column2 column3 |
+------------------------------+
| SITE1 NULL NULL 76 |
| SITE1 NULL 23 NULL |
| SITE1 NULL NULL NULL |
+------------------------------+
Desired output:
+------------------------------+
| Site column1 column2 column3 |
+------------------------------+
| SITE1 NULL 23 76 |
+------------------------------+
You need to use group by:
select site, max(column1) as column1, max(column2) as column2, . . .
from [Hilltopsamplerfix].[dbo].[tempHilltopWaterQualityExtractPivot]
group by site;
You can get the columns in the table from information_schema.columns and construct the logic in the SQL or Excel. For example
select ' max(' + column_name + ') as ' + column_name + ', '
from information_schema.columns
where table_name = 'tempHilltopWaterQualityExtractPivot' and
column_name <> 'site';
Then copy the results into the query window.
Try this below SQL, Based on Gordon
Declare #Cols Varchar(Max)
select #Cols= coalesce(#Cols+',', '')+ ' max(' + column_name + ') as ' + column_name
from information_schema.columns
where table_name = 'tempHilltopWaterQualityExtractPivot' and Column_Name <> 'site'
Declare #Query Varchar(max)
SET #Query='Select Site, '+ #Cols+' From tempHilltopWaterQualityExtractPivot Group by Site'
EXEC(#Query)
Related
I wish to add multiple rows for a particular pivot table.
Table name : MasterTable
FieldName | FieldValue
------------------------
Field1 | F1value1
Field2 | F2value1
Field3 | F3value1
Field1 | F1value2
Field2 | F2value2
Field3 | F3value2
Expected result:
Field1 | Field2 | Field3
---------------------------------
F1value1 | F2value1 | F3value1
F1value2 | F2value2 | F3value2
I tried this code but result me with only one row.
DECLARE #cols AS NVARCHAR(MAX),
#query AS NVARCHAR(MAX)
select #cols = STUFF((SELECT distinct ',' + QUOTENAME(FieldName) from MasterTable FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)') ,1,1,'')
set #query = 'SELECT ' + #cols + ' from (select FieldName, FieldValue from MasterTable ) x
pivot
(
max(FieldValue)
for FieldName in (' + #cols + ')
) p '
execute(#query)
Output:
Field1 | Field2 | Field3
---------------------------------
F1value1 | F2value1 | F3value1
Could someone please help me to have multiple rows using pivote table.
For the subquery, use:
(select FieldName, FieldValue,
row_number() over (partition by FieldName order by FieldName) as seqnum
from MasterTable
)
pivot will take this into account in the pivoting.
Note: This will guarantee the number of rows, but the ordering of each column is arbitrary. You may want to replace the order by with a reasonable column for ordering.
I have a Table which contain around 1000 columns. When I use
Select *
from Table
Its Return entire record of the table. But I just want only limited column of the record.
col1 | col2 | col3 | col4 | col 5 | ......................... | col1000 |
| | | | | ------------------------- | |
| | | | | ------------------------- | |
| | | | | ------------------------- | |
| | | | | ------------------------- | |
------------------------------------------------------------------------------------------------------
I just need col5 to col1000 record data only.
you have to write all the columns that you need in select
select col5, col6, ......... ,col1000 from table
there is no shot-cut way with out it and select * means all the columns of your table
If you really want to do without typing each column name, one way is using dynamic query.
For example in SQL Server you can write the dynamic query like following.
DECLARE #selstmt AS NVARCHAR(max);
SET #selstmt = 'select ' + Stuff((SELECT ', ' + Quotename(NAME) FROM
( SELECT c.NAME FROM
sys.columns c JOIN sys.tables t ON c.object_id = t.object_id
WHERE t.NAME = 'yourtablename'
AND c.NAME NOT IN('col1', 'col2', 'col3', 'col4')
)t FOR
xml path(''), type).value('.',
'NVARCHAR(MAX)'), 1, 1, '');
SET #selstmt = #selstmt + ' from yourtable'
EXEC sp_executesql #selstmt
Specify column names that you want to select instead of using * operator.
SELECT * will return all columns for table
SELECT col5, col6, col7,...., col1000 (col5 upto col1000) will return only specified columns of the table
There actually is one easy way in SSMS.
SELECT * FROM TableA
Select this text, press CNTRL SHIFT Q
Then you have all the columns and can easily remove a few.
You have to write all columns.
SELECT col1, col2, col3 from table;
But...
Tested on MySQL
Since you have too many columns, you can run a query to select the desired columns from the table:
SELECT GROUP_CONCAT(COLUMN_NAME SEPARATOR ', ')
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'schemaName' AND TABLE_NAME = 'table'
AND COLUMN_NAME IN ('col1', 'col2', 'col3');
# or to filter the columns
# AND COLUMN_NAME NOT IN ('col1', 'col2', 'col3');
And build the query with the resultant columns.
GROUP_CONCAT outputs the values in a single row separated with a ,, so you can use them to query the table directly
SELECT
col1,col2,col3
FROM
table_name
The following a table structure:
'----ID-----'----NAME----'----FIELD1----'----FIELD2----'
' 1 ' val ' 123 ' 321 '
' 2 ' val2 ' 234 ' 212 '
Need to get the following result:
'----ID-----'----NAME----'----FIELDS----'----VALUES----'
' 1 ' val ' FIELD1 ' 123 '
' 1 ' val ' FIELD2 ' 321 '
' 2 ' val2 ' FIELD1 ' 234 '
' 2 ' val2 ' FIELD2 ' 212 '
How write this query? I can get column names from INFORMATION_SCHEMA.COLUMNS. But how to join table with INFORMATION_SCHEMA.COLUMNS? Also how can rotating a part of table?
As living example. Following is table:
On screenshot only several fields but in table there are a lot of fields. I wrote the following query:
Select p.GUID, p.myvalues, p.Fields
from myTable gz
unpivot( [myvalues] for Fields in ([area], [davlplastmax])) p
But this query doesn't return null values.
Also I want get columns from INFORMATION_SCHEMA.COLUMNS and past them in ([area], [davlplastmax]).
For example:
unpivot( [values] for Fields in (
SELECT [MyDb].INFORMATION_SCHEMA.COLUMNS.COLUMN_NAME
FROM [MyDb].INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = N'MyTable'
)
Unpivot?
select u.id, u.name, u.fields, u.values
from MyTable t
unpivot
(
values
for fields in (Field1, Field2)
) u;
You can use unpivot as below:
Select * from #data
unpivot( [values] for Fields in ([Field1],[Field2])) p
Output as below:
+----+------+--------+--------+
| Id | Name | values | Fields |
+----+------+--------+--------+
| 1 | val | 123 | Field1 |
| 1 | val | 321 | Field2 |
| 2 | val2 | 234 | Field1 |
| 2 | val2 | 212 | Field2 |
+----+------+--------+--------+
You can use dynamic query as below for getting columns from Information_Schemas
Declare #cols1 varchar(max)
Declare #query nvarchar(max)
Select #cols1 = stuff((select ','+QuoteName(Column_Name) from INFORMATION_SCHEMA.COLUMNS where TABLE_NAME = 'TestData'
and COLUMN_NAME not in ('Id','Name') for xml path('')),1,1,'')
Select #query = ' Select * from
(Select * from #data )a
unpivot( [values] for Fields in (' + #cols1+ ')) p '
Exec sp_executeSql #query
I need to analyze application data, and it stores data in separate table from definition (column names). Something like this:
Data table1:
id | 1234 | 1235 | 1236
---|------|------|-----
1 | val1 | val2 | val3
Model:
tableID | colID | Name
--------|-------|-----
table1 | 1234 | Name
table1 | 1235 | DOB
table1 | 1236 | Sex
table2 | 1237 | Manager
etc...
I want to get this data like this:
Data:
id | Name | DOB | Sex
---|------|------|-----
1 | val1 | val2 | val3
Is it possible? Additional problem is that in each data table there might be different number of columns.
UPD.
Problem solved by #Prdp
Though, i've changed query a little bit:
DECLARE #col_list VARCHAR(max)
SET #col_list = Stuff((SELECT ',' + Quotename(af.ColID) + ' as ' + Quotename(af.Name)
FROM [Model].dbo.Fields af
WHERE af.[App.ID] = 123
FOR xml path ('')), 1, 1, '')
EXEC('select '+#col_list+' from [Data].dbo.[123]')
As mentioned in comments better to handle this in application layer. If in case you want a sql solution then you need dynamic query
DECLARE #col_list VARCHAR(8000)
SET #col_list = Stuff((SELECT ',' + Quotename(colID) + ' as ' + Quotename(NAME)
FROM Model
WHERE EXISTS (SELECT 1
FROM information_schema.columns
WHERE table_name = 'Data'
AND Cast(colID AS VARCHAR(50)) = column_name)
FOR xml path ('')), 1, 1, '')
EXEC('select '+#col_list+' from data')
Here's a sample of my table
[myTable]
id | random1 | random2 | random3 | random4
1 | 123 | 5357 | 10 | 642
2 | 423 | 34 | 20 | 531
3 | 9487 | 234 | 30 | 975
4 | 34 | 123 | 40 | 864
Here's my current query, but it isn't working like I'd expect it to:
SELECT
cols.*,
(SELECT SUM(cols.column_name) FROM myTable t)
FROM
(SELECT
table_name::text, column_name::text
FROM
information_schema.columns
where
table_name = 'myTable') as cols
I'm getting the error: function sum(text) does not exist - which makes sense. I'm pretty sure that mysql is can be messy enough to allow a reference like that, but I don't know how to do this in postgres.
What I'd really like to have is an end result somewhere along the lines of...
table_name | column_name | sum
myTable | id | 10
myTable | random1 | 10067
myTable | random2 | 5748
myTable | random3 | 100
myTable | random4 | 3012
I want to take this query a lot further, but I'm getting really hung up on being able to reference the column name.
SQL queries are static. They select before-known columns from before-known tables. You cannot make them look up table names and columns from the database dictionary and then magically glue these names into themselves.
What you can do: Write a program (Java, C#, PHP, whatever you like) doing the following:
Send a query to the DBMS to find the column names for the table you are interested in.
Build a SQL query string with the column names got.
Send this query to the DBMS.
declare #tableName varchar(255) = 'myTable' --Change this to the table name you want
/*create table and column name dataSet and insert values*/
if object_id('tempdb.dbo.#objectSet') is not null
drop table #objectSet
create table #objectSet
(table_name varchar(256),
columnID int,
column_name varchar(256),
[sum] int)
insert into #objectSet
(table_name,
columnID,
column_name)
select O.name table_name,
C.column_id columnID,
C.name column_name
from sys.all_objects O
join sys.all_columns C
on O.object_id = C.object_id
join sys.types T
on C.user_type_id = T.user_type_id
where O.object_id = object_id(#tableName)
and T.name in ('int', 'tinyint', 'smallint', 'bigint') --Columns with Aggregatable datatypes only, all other columns will be excluded from the set.
/*Create loop variables for each column*/
declare #SQL as varchar(4000),
#counter int = 1,
#maxCount int
select #maxCount = SQ.maxCount
from ( select count(*) maxCount
from #objectSet OS) SQ
/*Run loop, updating each column as it goes*/
while #counter <= #maxCount
begin
select #SQL = 'update OS set OS.[sum] = SQ.[sum] from #objectSet OS join (select sum(DS.' + OS.column_name + ') [sum] from ' + #tableName + ' DS) SQ on OS.column_name = ''' + OS.column_name + ''''
from #objectSet OS
where OS.columnID = #counter
exec (#SQL)
select #counter += 1
end
/*Display Results*/
select OS.table_name,
OS.column_name,
OS.[sum]
from #objectSet OS
Using system object tables, some dynamic T-SQL, and a loop should do it.