Splitting one row into more rows -- table transformation - sql

I have one source table that should be converted to one destination table. The source table contains four columns with sensor values. The destination table should contain four rows with the single sensor value and with one column for number of the sensor -- for each row from the source table. In other words, the destination table will have four times more rows. (I believe this is called normalization. At least, I think it will be more practical in future when more or less or different sensors are to be used.)
More background information to explain. I have already successfully tried an insert trigger that does that for a single line:
CREATE TRIGGER dbo.temperatures_to_sensors
ON dbo.Data
AFTER INSERT
AS
BEGIN
DECLARE #line_no TINYINT;
SET #line_no = 2; -- hardwired for the production line
DECLARE #UTC DATETIME;
DECLARE #value1 FLOAT;
DECLARE #value2 FLOAT;
DECLARE #value3 FLOAT;
DECLARE #value4 FLOAT;
SELECT
#UTC = CAST((CAST(LEFT(inserted.UTC, 16) AS FLOAT) - 2415020.5) AS DATETIME),
#value1 = inserted.temperature_1,
#value2 = inserted.temperature_2,
#value3 = inserted.temperature_3,
#value4 = inserted.temperature_4
FROM inserted;
INSERT INTO dbo.line_sensor_values
(UTC, line_no, sensor_no, sensor_value)
VALUES (#UTC, #line_no, 1, #value1),
(#UTC, #line_no, 2, #value2),
(#UTC, #line_no, 3, #value3),
(#UTC, #line_no, 4, #value4);
END;
GO
Now, I would like to initialize the destination table from the old table once. After that, the trigger will continue to fill the values.
I am not good in SQL. I tried:
CREATE PROCEDURE dbo.init_line_sensor_values
AS
BEGIN
DECLARE #line_no TINYINT;
SET #line_no = 2; -- hardwired for the production line
DECLARE #UTC DATETIME;
DECLARE #value1 FLOAT;
DECLARE #value2 FLOAT;
DECLARE #value3 FLOAT;
DECLARE #value4 FLOAT;
INSERT INTO dbo.line_sensor_values
(UTC, line_no, sensor_no, sensor_value)
VALUES (#UTC, #line_no, 1, #value1),
(#UTC, #line_no, 2, #value2),
(#UTC, #line_no, 3, #value3),
(#UTC, #line_no, 4, #value4)
SELECT
#UTC = CAST((CAST(LEFT(t.UTC, 16) AS FLOAT) - 2415020.5) AS DATETIME),
#value1 = t.temperature_1,
#value2 = t.temperature_2,
#value3 = t.temperature_3,
#value4 = t.temperature_4
FROM dbo.Data AS t;
END;
GO
EXECUTE dbo.init_line_sensor_values
GO
... but it fails with
Cannot insert the value NULL into column 'UTC', table '1000574.dbo.line_sensor_values'; column does not allow nulls. INSERT fails.
It is apparent that the SELECT should be used differently to feed the INSERT. Or do I have to use the loop? (Cursor created and FETCH NEXT... and WHILE...)
UPDATED
The source table can be created this way (simplified):
CREATE TABLE dbo.Data(
UTC varchar(32) NOT NULL,
temperature_1 float NULL,
temperature_2 float NULL,
temperature_3 float NULL,
temperature_4 float NULL
PRIMARY KEY CLUSTERED
(
UTC ASC
)
GO
The destination table was created this way:
CREATE TABLE dbo.line_sensor_values (
UTC DATETIME NOT NULL,
line_no TINYINT NOT NULL, -- line number: 1, 2, 3, etc.
sensor_no TINYINT NOT NULL, -- sensor number: 1, 2, 3, etc.
sensor_value float NULL, -- the measured value
PRIMARY KEY CLUSTERED (
UTC ASC,
line_no ASC,
sensor_no ASC
)
)
GO
Thanks for your help, Petr

If all you need to do is to convert a table with four columns into a single table where each row represents a row number from a source table and a column from a source table, then here is an example:
Here is SQLFiddle
create table fourColumns
(
column1 varchar(50),
column2 varchar(50),
column3 varchar(50),
column4 varchar(50)
)
insert into fourColumns select 'A','B','C','D'
insert into fourColumns select 'E','F','G','H'
;with MyCTE (lineNumber, columnNumber, Result)
as
(
select ROW_NUMBER() OVER(ORDER BY column1 ASC) AS Row, 1, column1
from fourColumns
union all
select ROW_NUMBER() OVER(ORDER BY column2 ASC) AS Row, 2, column2
from fourColumns
union all
select ROW_NUMBER() OVER(ORDER BY column3 ASC) AS Row, 3, column3
from fourColumns
union all
select ROW_NUMBER() OVER(ORDER BY column4 ASC) AS Row, 4, column4
from fourColumns
)
-- add insert here
select lineNumber,
columnNumber,
Result
from MyCTE
order by lineNumber

INSERT INTO dbo.line_sensor_value
(UTC, line_no, sensor_no, sensor_value)
select UTC, line_no, sensor_no, temperature_1 as sensor_value from dbo.Data
union
select UTC, line_no, sensor_no, temperature_2 as sensor_value from dbo.Data
union
select UTC, line_no, sensor_no, temperature_3 as sensor_value from dbo.Data
union
select UTC, line_no, sensor_no, temperature_4 as sensor_value from dbo.Data

Related

SQL return bitwise flag value for non null columns indicating data significance

I have a case where I need to create a unique weight value based on how much data is contained with a distinct row in a dataset. Each column is assigned a bit value indicating its significance. For instance Col1 = 1, Col2 = 2 would signify that Col2 carries more weight than Col1. Data in both Col1 and Col2 (Col1 | Col2) = 3 would be more significant than a row with data in either column.
Columns at a later time can be reclassified by significance. Therefore I am looking for a solution which would be more versatile than hard coding the bitwise operation into the SQL query (see Option 1).
I have devised Option 2 as a way to overcome the reclassification requirement but I am not sure if this is the best way to write the query. My production dataset will range from 4M-25M records.
Can any provide another example, or improve this example, of how to write this query (Option 2) to improve performance and the ability for a user to change the column's significance?
Test Code Below:
Environment: SQL Server 2014
--EXAMPLE SETUP
IF OBJECT_ID('tempdb..#DataImage') IS NOT NULL
BEGIN
DROP TABLE #DataImage
DROP TABLE #Source
END
CREATE TABLE #DataImage (ID INT PRIMARY KEY CLUSTERED IDENTITY(1,1), ColumnOne BIT, ColumnTwo BIT, Flag INT)
CREATE TABLE #Source (ID INT PRIMARY KEY CLUSTERED IDENTITY(1,1), DataOne NVARCHAR(50), DataTwo INT)
--END SETUP
--CREATE ROW COMBINATION FLAGS (Real Flag Size = 10 combinations)
INSERT INTO #DataImage (ColumnOne, ColumnTwo, Flag) SELECT 1, 0, 1
INSERT INTO #DataImage (ColumnOne, ColumnTwo, Flag) SELECT 0, 1, 2
INSERT INTO #DataImage (ColumnOne, ColumnTwo, Flag) SELECT 1, 1, 3
--CREATE TEST DATA (Real Data Size = 10M records)
INSERT INTO #Source (DataOne, DataTwo) SELECT NULL, 2
INSERT INTO #Source (DataOne, DataTwo) SELECT N'Foo', NULL
INSERT INTO #Source (DataOne, DataTwo) SELECT N'Bar', 100
--QUERY SETUPS--
--OPTION 1: CALCULATE THE FLAG FOR EVERY ROW
SELECT SourceId = s.ID, Flag = IIF(s.DataOne IS NULL, 0, 1) + IIF(s.DataTwo IS NULL, 0, 2)
FROM #Source s
--OPTION 2: RETURN FLAG FOR SOURCE DATA
;WITH t0 (SourceId, Flag, C1, C2) AS (
SELECT s.ID, i.Flag, IIF(s.DataOne IS NULL, 0, 1), IIF(s.DataTwo IS NULL, 0, 1)
FROM #Source s CROSS JOIN #DataImage i
INTERSECT
SELECT s.ID, i.Flag, ColumnOne, ColumnTwo
FROM #Source s CROSS JOIN #DataImage i
)
SELECT t0.SourceId, t0.Flag
FROM t0

Assign multiple values to Table variable in SQL

DECLARE #ID INT
SET #ID = (select top 1 USER_REQ_JOB_ID
from T8504_USER_REQ_JOB
where JOB_GRP_ID = 160
order by LST_UPDT_TS desc)
SELECT INPUT_PARM_VAL_TX
from TBL_RPT_JOB_INPUT_PARAM
where USER_REQ_JOB_ID = #ID
This returns these results:
USA
USCC
6
7
2
These five records what I get I want to assign to five different variables to use in stored procedure.
I was trying with table variable like this :
declare #CID table (
Region Char(3)
,Segment Char(3)
,MasterContractId int
,ctcid int
,templateid int)
insert into #CID (Region,Segment,MasterContractId,ctcid,templateid)
But how to insert that 5 rows here?
INSERT INTO #CID
select * from
(
select
'Temp' + convert(char(1), row_number() over (order by (select 0))) as columnName,
INPUT_PARM_VAL_TX as Value
from TBL_RPT_JOB_INPUT_PARAM where USER_REQ_JOB_ID = #ID
) d
pivot
(
max(value)
for columnname in (Temp1, Temp2, Temp3, Temp4, Temp5)
) piv;
See if this helps.
Take a look at this fiddle for an example.
Courtesy:
Add row number to this T-SQL query
Efficiently convert rows to columns in sql server
EDIT: The sql adds an extra column to generate row numbers to use it as an extra column, which is pivoted as column heading.
it's really gross, but one way you could probably do it is this (though you'll need to apply it to your case):
http://sqlfiddle.com/#!6/d41d8/21507
declare #table TABLE (value varchar(50))
INSERT INTO #table
VALUES ('first')
INSERT INTO #table
VALUES ('second')
INSERT INTO #table
VALUES (3)
INSERT INTO #table
VALUES (4)
DECLARE #temp TABLE (id int identity(1,1), value varchar(50))
INSERT INTO #temp
SELECT [value]
FROM #table t
SELECT *
FROM #temp
DECLARE #CID TABLE (Region varchar(50), cont varchar(50), another int, andAnother int)
INSERT INTO #CID
(
Region,
cont,
another,
andAnother
)
VALUES
(
(SELECT value FROM #temp WHERE id = 1), -- Region - varchar
(SELECT value FROM #temp WHERE id = 2), -- cont - varchar
(SELECT value FROM #temp WHERE id = 3), -- another - int
(SELECT value FROM #temp WHERE id = 4) -- andAnother - int
)
SELECT * FROM #cid
note that i assumed you're using mssql, you did not specify

setting rowID as identity (should be ascending number) in SQL

I have this following sql script
DECLARE #tmpTable TABLE (rowID int IDENTITY,
woID varchar(100), srID varchar(100),
woWorkOrderNumber varchar(100),
woSequenceCount varchar(100),
WorkOrderNumber varchar(100)
)
INSERT INTO #tmpTable (woID, srID, woWorkOrderNumber, woSequenceCount, WorkOrderNumber)
SELECT
woID, srID, woWorkOrderNumber, woSequenceCount,
SUBSTRING(woWorkOrderNumber, 11, 20 ) AS WorkOrderNumber
FROM
WorkOrder
WHERE
codeSICurrentStatusCode NOT IN (3, 4)
AND SUBSTRING(woWorkOrderNumber, 11, 20) = ''
SELECT * FROM #tmpTable
But I'm getting these results on my rowID column:
As you can see, the rowID seems to be the row number from the table I selected on. What I'm trying to achieve on this temp table is that the rowID starts from 1 then 2 then 3 and so on and so forth.. What's wrong with my code?
You need to specify ORDER BY on the INSERT statement SELECT clause in order to control the order of identity value assignment. You can alternatively use ROW_NUMBER() instead of IDENTITY to ensure there are no gaps and provide complete control over the values.

Need multiple copies of one resultset in sql without using loop

Following is the sample data. I need to make 3 copies of this data in t sql without using loop and return as one resultset. This is sample data not real.
42 South Yorkshire
43 Lancashire
44 Norfolk
Edit: I need multiple copies and I have no idea in advance that how many copies I need I have to decide this on the basis of dates. Date might be 1st jan to 3rd Jan OR 1st jan to 8th Jan.
Thanks.
Don't know about better but this is definatley more creative! you can use a CROSS JOIN.
EDIT: put some code in to generate a date range, you can change the date range, the rows in the #date are your multiplier.
declare #startdate datetime
, #enddate datetime
create table #data1 ([id] int , [name] nvarchar(100))
create table #dates ([date] datetime)
INSERT #data1 SELECT 42, 'South Yorkshire'
INSERT #data1 SELECT 43, 'Lancashire'
INSERT #data1 SELECT 44, 'Norfolk'
set #startdate = '1Jan2010'
set #enddate = '3Jan2010'
WHILE (#startdate <= #enddate)
BEGIN
INSERT #dates SELECT #startdate
set #startdate=#startdate+1
END
SELECT [id] , [name] from #data1 cross join #dates
drop table #data1
drop table #dates
You could always use a CTE to do the dirty work
Replace the WHERE Counter < 4 with the amount of duplicates you need.
CREATE TABLE City (ID INTEGER PRIMARY KEY, Name VARCHAR(32))
INSERT INTO City VALUES (42, 'South Yorkshire')
INSERT INTO City VALUES (43, 'Lancashire')
INSERT INTO City VALUES (44, 'Norfolk')
/*
The CTE duplicates every row from CTE for the amount
specified by Counter
*/
;WITH CityCTE (ID, Name, Counter) AS
(
SELECT c.ID, c.Name, 0 AS Counter
FROM City c
UNION ALL
SELECT c.ID, c.Name, Counter + 1
FROM City c
INNER JOIN CityCTE cte ON cte.ID = c.ID
WHERE Counter < 4
)
SELECT ID, Name
FROM CityCTE
ORDER BY 1, 2
DROP TABLE City
This may not be the most efficient way of doing it, but it should work.
(select ....)
union all
(select ....)
union all
(select ....)
Assume the table is named CountyPopulation:
SELECT * FROM CountyPopulation
UNION ALL
SELECT * FROM CountyPopulation
UNION ALL
SELECT * FROM CountyPopulation
Share and enjoy.
There is no need to use a cursor. The set-based approach would be to use a Calendar table. So first we make our calendar table which need only be done once and be somewhat permanent:
Create Table dbo.Calendar ( Date datetime not null Primary Key Clustered )
GO
; With Numbers As
(
Select ROW_NUMBER() OVER( ORDER BY S1.object_id ) As [Counter]
From sys.columns As s1
Cross Join sys.columns As s2
)
Insert dbo.Calendar([Date])
Select DateAdd(d, [Counter], '19000101')
From Numbers
Where [Counter] <= 100000
GO
I populated it with a 100K dates which goes into 2300. Obviously you can always expand it. Next we generate our test data:
Create Table dbo.Data(Id int not null, [Name] nvarchar(20) not null)
GO
Insert dbo.Data(Id, [Name]) Values(42,'South Yorkshire')
Insert dbo.Data(Id, [Name]) Values(43, 'Lancashire')
Insert dbo.Data(Id, [Name]) Values(44, 'Norfolk')
GO
Now the problem becomes trivial:
Declare #Start datetime
Declare #End datetime
Set #Start = '2010-01-01'
Set #End = '2010-01-03'
Select Dates.[Date], Id, [Name]
From dbo.Data
Cross Join (
Select [Date]
From dbo.Calendar
Where [Date] >= #Start
And [Date] <= #End
) As Dates
By far the best solution is CROSS JOIN. Most natural.
See my answer here: How to retrieve rows multiple times in SQL Server?
If you have a Numbers table lying around, it's even easier. You can DATEDIFF the dates to give you the filter on the Numbers table

SQL single query update

I'm having to insert values into a new column in our database but I can't get my head around doing this in a consistent manner. There is a lot of data so doing anything manually is pretty much out of the question. Let me set the stage:
We have a table called Occurrence and a table called OccurenceBuckets where each occurrence is reference the bucket to which it has been assigned. Previously this was a one-way reference but for various reasons we have decided to add a reference back from the OccurrenceBucket to the first Occurrence (first in time, that is). The tables now look like this:
CREATE TABLE Occurrence
OccurrenceID uniqueidentifier,
OccurrenceBucketID uniqueidentifier,
OccurrenceTime datetime,
OccurrenceMessage nvarchar
...other meta data...
CREATE TABLE OccurrenceBucket
OccurrenceBucketID uniqueidentifier,
...other meta data...
FirstOccurrenceID uniqueidentifier,
FirstOccurrenceTime datetime,
FirstOccurrenceMessage nvarchar
I'm looking for a way to determine the first occurrence belonging to a bucket and assigning the FirstOccurrenceID, FirstOccurrenceTime and FirstOccurrenceMessage with values from this first occurrence for all my occurrencebuckets.
Do any of you sql-fu experts out there have the time to help me out, all my attempts seen to produce incorrect or incomplete selection of occurrences.
You can try this
DECLARE #Occurrence TABLE(
OccurrenceID INT,
OccurrenceBucketID INT,
OccurrenceTime DATETIME,
OccurrenceMessage VARCHAR(MAX)
)
INSERT INTO #Occurrence (OccurrenceID,OccurrenceBucketID,OccurrenceTime,OccurrenceMessage)
SELECT 1, 1, '01 Jan 2009', 'A'
INSERT INTO #Occurrence (OccurrenceID,OccurrenceBucketID,OccurrenceTime,OccurrenceMessage)
SELECT 2, 1, '02 Jan 2009', 'B'
INSERT INTO #Occurrence (OccurrenceID,OccurrenceBucketID,OccurrenceTime,OccurrenceMessage)
SELECT 3, 1, '03 Jan 2009', 'C'
INSERT INTO #Occurrence (OccurrenceID,OccurrenceBucketID,OccurrenceTime,OccurrenceMessage)
SELECT 4, 2, '04 Jan 2009', 'D'
INSERT INTO #Occurrence (OccurrenceID,OccurrenceBucketID,OccurrenceTime,OccurrenceMessage)
SELECT 5, 2, '05 Jan 2009', 'E'
INSERT INTO #Occurrence (OccurrenceID,OccurrenceBucketID,OccurrenceTime,OccurrenceMessage)
SELECT 6, 2, '06 Jan 2009', 'F'
SELECT * FROM #Occurrence
DECLARE #OccurrenceBucket TABLE(
OccurrenceBucketID INT,
FirstOccurrenceID INT,
FirstOccurrenceTime DATETIME,
FirstOccurrenceMessage VARCHAR(MAX)
)
INSERT INTO #OccurrenceBucket (OccurrenceBucketID) SELECT 1
INSERT INTO #OccurrenceBucket (OccurrenceBucketID) SELECT 2
SELECT * FROM #OccurrenceBucket
UPDATE #OccurrenceBucket
SET FirstOccurrenceID = OccurrenceID,
FirstOccurrenceTime = OccurrenceTime,
FirstOccurrenceMessage = OccurrenceMessage
FROM #OccurrenceBucket oc INNER JOIN
(
SELECT o.*
FROM #Occurrence o INNER JOIN
(
SELECT OccurrenceBucketID,
MIN(OccurrenceID) FirstOccurrenceID
FROM #Occurrence
GROUP BY OccurrenceBucketID
) Mins ON o.OccurrenceID = mins.FirstOccurrenceID
) Vals ON oc.OccurrenceBucketID = Vals.OccurrenceBucketID
SELECT * FROM #OccurrenceBucket
EDIT:
UPDATE #OccurrenceBucket
SET FirstOccurrenceID = OccurrenceID,
FirstOccurrenceTime = OccurrenceTime,
FirstOccurrenceMessage = OccurrenceMessage
FROM #OccurrenceBucket oc INNER JOIN
(
SELECT o.*
FROM #Occurrence o INNER JOIN
(
SELECT OccurrenceBucketID,
MIN(OccurrenceTime) FirstOccurrenceTime
FROM #Occurrence
GROUP BY OccurrenceBucketID
) Mins ON o.OccurrenceTime = mins.FirstOccurrenceTime
) Vals ON oc.OccurrenceBucketID = Vals.OccurrenceBucketID
Well, to find the first occurrence in a bucket shouldn't the following work?
SELECT TOP 1
OccurranceID, OccurranceTime, OccurranceMessage
FROM Occurance
WHERE
OccurranceBucketID = #OccurranceBucketID
ORDER BY
OccurranceTime ASC
You can assign the returned fields to variables and then update your OccurranceBucket accordingly.
NB: "occurrence" has no "a" in it.
This answer relies on the OccurrenceTime being unique for each Occurrence: -
update OccBuck set
OccBuck.FirstOccurrenceID = Occ.OccurrenceID,
OccBuck.FirstOccurrenceTime = Occ.OccurrenceTime,
OccBuck.FirstOccurrenceMessage = Occ.OccurrenceMessage
from
dbo.OccurrenceBucket as OccBuck
inner join dbo.Occurrence as Occ on OccBuck.OccurrenceBucketID = Occ.OccurrenceBucketID
inner join (select OccurrenceBucketID,
MIN(OccurrenceTime) as 'MinOccurrenceTime'
from dbo.Occurrence
group by OccurrenceBucketID) as minOcc on Occ.OccurrenceBucketID= minOcc.OccurrenceBucketID and
Occ.OccurrenceTime = minOcc.MinOccurrenceTime;