In Azure Stream Analytic (IoT Hub), no matter what time window I specify, the TumblingWindow function fails to compare time and won't return the window I need. I am trying to use the below SQL code block to return me a 2-second window, but the output includes all events. As there is no pivot function in Stream Analytic, I am using the method suggested by #Joe-Zhang
In this case, there is an IoT read event every 2 seconds, and I am expecting only one event to be returned -
with tempone as (
select
cast(dataArr.ArrayValue.SourceTimestamp as datetime) as SourceTimestamp,
cast(valuesArr.ArrayValue.Address as bigint) as Address2,
max(cast(valuesArr.ArrayValue.Value as float)) as Value2
from iotinput i
cross apply GetArrayElements(i.Content) as contentArr
cross apply GetArrayElements(contentArr.ArrayValue.Data) as dataArr
cross apply GetArrayElements(dataArr.ArrayValue.[Values]) as valuesArr
WHERE cast(valuesArr.ArrayValue.Address as bigint) = 30002
GROUP BY cast(dataArr.ArrayValue.SourceTimestamp as datetime),
cast(valuesArr.ArrayValue.Address as bigint),
TumblingWindow(second, 2)
),
temptwo AS (
select
cast(dataArr.ArrayValue.SourceTimestamp as datetime) as SourceTimestamp,
cast(valuesArr.ArrayValue.Address as bigint) as Address3,
max(cast(valuesArr.ArrayValue.Value as float)) as Value3
from iotinput i
cross apply GetArrayElements(i.Content) as contentArr
cross apply GetArrayElements(contentArr.ArrayValue.Data) as dataArr
cross apply GetArrayElements(dataArr.ArrayValue.[Values]) as valuesArr
WHERE cast(valuesArr.ArrayValue.Address as bigint) = 30003
GROUP BY cast(dataArr.ArrayValue.SourceTimestamp as datetime),
cast(valuesArr.ArrayValue.Address as bigint),
TumblingWindow(second, 2)
)
select tempone.SourceTimestamp, tempone.Value2 as Temperature, temptwo.Value3 as Humidity from tempone
join temptwo on tempone.SourceTimestamp = temptwo.SourceTimestamp
and DATEDIFF(second,tempone, temptwo) BETWEEN 0 AND 2
Returned values -
If you don't use TIMESTAMP BY, the time logic will be based on the ingestion time.
Here it looks like you expect time processing to be done on SourceTimestamp, but you don't TIMESTAMP BY on it.
Related
Below is the code to a view created to pull delimited values from a table into new rows in a view.
This works well, but I can't figure out how to set the TASDateTimeStart column value to the previous row's TASDateTimeEnd column value where the id in both rows are the same
SELECT
MallaghanApp.dbo.EmployeeClockIn.Id,
Employee,
JobType,
EmployeeName,
EmployeeNumber,
value AS SerialNumber,
CAST([TASDateTimeEnd] AS smalldatetime) [TASDateTimeEnd],
CAST([TASDateTimeStart] AS smalldatetime) [TASDateTimeStart],
ISNULL(((SELECT DATEDIFF(MINUTE, TASDateTimeStart, TASDateTimeEnd)) /
((SELECT LEN([SerialNo]) - LEN(REPLACE([SerialNo], ',', ''))) + 1)), (DATEDIFF(MINUTE, TASDateTimeStart, CURRENT_TIMESTAMP)) / ((SELECT LEN([SerialNo]) - LEN(REPLACE([SerialNo], ',', ''))) + 1)) AS MinutesClocked,
Department,
DepartmentNumber
FROM
EmployeeClockIn
CROSS APPLY
STRING_SPLIT([SerialNo], ',')
LEFT JOIN
EmployeeInfo ON Employee = EmployeeCombinedInfo
I keep getting an error
Subquery returned more than 1 value.This is not permitted when the subquery follows =,!=,<,<=,>,>= or when the subquery is used as an expression
Does anyone know the code needed here?
You can use the LAG function to get this value:
SELECT eci.Id,
eci.TASDateTimeEnd,
LAG(eci.TASDateTimeEnd) OVER (PARTITION BY eci.Id ORDER BY eci.TASDateTimeEnd) as PreviousTASDateTimeEnd
FROM EmployeeClockIn eci;
I am running the below unpivot scode but it errors with
"The type of column "TransDate" conflicts with the type of other columns specified in the UNPIVOT list " Can someone advise what I need to convert etc? I seems to not like the datetime column transdate. everything else is nvarchar in the table.
select DataLoadSysId, DataLoadBatchSysId, Rowid, ColumnName, ColumnValue As ColumnValue
from (
select ExtractSource, RecordTypeNo, RecordLevel1Code, RecordLevel2Code, TransDate,
MainAccount, Amount, PeriodCode, DataAreaId, SourceFile, DataLoadBatchSysId, LoadDate, ValidationErrors, DataLoadSysId, RowId
from [Staging].[FactFinancialsCoded_Abbas_InitialValidationTest]
) x
UNPIVOT
(
ColumnValue
FOR ColumnName
IN ([ExtractSource], [RecordTypeNo], [RecordLevel1Code], [RecordLevel2Code], [TransDate], [MainAccount], [Amount], [PeriodCode], [DataAreaId])
)
As UnpivotExample
I'm not a fan of the unpivot keyword. I find it easier to just use apply:
select ivt.DataLoadSysId, ivt.DataLoadBatchSysId, ivt.Rowid,
v.ColumnName, v.ColumnValue
from [Staging].[FactFinancialsCoded_Abbas_InitialValidationTest] ivt CROSS APPLY
(VALUES ('ExtractSource', ExtractSource),
('RecordTypeNo', RecordTypeNo),
('RecordLevel1Code', RecordLevel1Code),
('RecordLevel2Code', RecordLevel2Code),
('TransDate', TransDate),
('MainAccount', MainAccount),
('Amount', Amount),
('PeriodCode', PeriodCode),
('DataAreaId', DataAreaId)
) v(columname, columnvalue);
This doesn't fix the problem. I prefer this because apply is very powerful and unpivoting is one convenient application to learn about the syntax (technically implementing "lateral joins").
Your problem is competing types. You need to convert everything to a string. I can only guess what some of the non-string values are, but something like:
select ivt.DataLoadSysId, ivt.DataLoadBatchSysId, ivt.Rowid,
v.ColumnName, v.ColumnValue
from [Staging].[FactFinancialsCoded_Abbas_InitialValidationTest] ivt CROSS APPLY
(VALUES ('ExtractSource', ExtractSource),
('RecordTypeNo', RecordTypeNo),
('RecordLevel1Code', RecordLevel1Code),
('RecordLevel2Code', RecordLevel2Code),
('TransDate', convert(varchar(255), TransDate)),
('MainAccount', MainAccount),
('Amount', convert(varchar(255), Amount)),
('PeriodCode', PeriodCode),
('DataAreaId', DataAreaId)
) v(columname, columnvalue);
Table has a column with values
ColA
------
a.b.c.d.e (car.make.model, car.la, kg)
ab.cd.ef (car.make.model)
a1.b2.c3.d4.e5(car.make.model, car.la, kg, av.vc.de)
I want to write a sql query to split the ColA by delimiter "." and pick last but one.
Expected output
Result
------
d
cd
d4
I have tried ParseName but dont see option to pick last but one.
Thank you
Using Jeff Moden's DelimitedSplit8K:
USE Sandbox;
GO
CREATE TABLE #Sample (ColA varchar(500));
GO
INSERT INTO #Sample
VALUES ('a.b.c.d.e'),
('ab.cd.ef'),
('a1.b2.c3.d4.e5');
GO
SELECT *
FROM #Sample;
WITH Split AS(
SELECT S.ColA,
DS.*,
MAX(DS.ItemNumber) OVER (PARTITION BY S.ColA) AS Items
FROM #Sample S
CROSS APPLY DelimitedSplit8K(S.ColA,'.') DS)
SELECT Item
FROM Split
WHERE ItemNumber = Items - 1;
GO
DROP TABLE #Sample
Ideally, though, don't store your data in a delimited format. :)
Just to play around using STRING_SPLIT:
SELECT ColA, t.value
FROM table1
CROSS APPLY(SELECT value,
COUNT(*) OVER () as cnt,
ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS rn
FROM STRING_SPLIT(ColA, '.')) AS t
WHERE t.rn = t.cnt - 1
Note: The function is available from SQL Server 2016.
Note 2: The query works provided that the function returns each value in the same order as the one it appears inside the string.
Why not simply using substring?
DECLARE #ColA NVARCHAR(100) = 'a1.b2.c3.d4.e5(car.make.model, car.la, kg, av.vc.de)';
SELECT REVERSE(LEFT(RIGHT(REVERSE(LEFT(#ColA, CHARINDEX('(', #ColA)-1)), LEN(LEFT(#ColA, CHARINDEX('(', #ColA)-1))-CHARINDEX('.',REVERSE(LEFT(#ColA, CHARINDEX('(', #ColA)-1)))), CHARINDEX('.',RIGHT(REVERSE(LEFT(#ColA, CHARINDEX('(', #ColA)-1)), LEN(LEFT(#ColA, CHARINDEX('(', #ColA)-1))-CHARINDEX('.',REVERSE(LEFT(#ColA, CHARINDEX('(', #ColA)-1)))))-1))
However, this last edit does NOT handle the case when there is no . or no ( in the string - feel free t o extend the query accordingly
Try This
;WITH CTE(ColA)
AS
(
SELECT 'a.b.c.d.e' UNION ALL
SELECT 'ab.cd.ef' UNION ALL
SELECT 'a1.b2.c3.d4.e5'
)
SELECT ColA,REVERSE(SUBSTRING(ReqColA,0,CHARINDEX('.',(ColA)))) AS ReqColA
FROM
(
SELECT ColA ,SUBSTRING(REVERSE(ColA),CHARINDEX('.',REVERSE(ColA))+1,LEN(REVERSE(ColA))) AS ReqColA FROM CTE
)dt
Result
ColA ReqColA
-----------------------
a.b.c.d.e d
ab.cd.ef cd
a1.b2.c3.d4.e5 d4
Have a column ID that I would like to ORDER in a specific format. Column has a varchar data type and always has an alphabetic value, typically P in front followed by three to four numeric values. Possibly even followed by an underscore or another alphabetic value. I have tried few options and none are returning what I desire.
SELECT [ID] FROM MYTABLE
ORDER BY
(1) LEN(ID), ID ASC
/ (2) LEFT(ID,2)
OPTIONS TRIED (3) SUBSTRING(ID,2,4) ASC
\ (4) ROW_NUMBER() OVER (ORDER BY SUBSTRING(ID,2,4))
(5) SUBSTRING(ID,PATINDEX('%[0-9]%',ID),LEN(ID))
(6) LEFT(ID, PATINDEX('%[0-9]%', ID)-1)
Option 1 seems to be closest to what I am looking for except when an _ or Alphabetic values follow the numeric value. See results from Option 1 below
P100
P208
P218
P301
P305
P306
P4200
P4510
P4511
P4512
P5011
P1400A
P4125H
P4202A
P4507L
P4706A
P1001_2
P2103_B
P4368_RL
Would like to see..
P100
P208
P218
P301
P305
P306
P1001_2
P1400A
P2103_B
P4125H
P4200
P4202A
P4368_RL
P4507L
P4510
P4511
P4512
P4706A
P5011
ORDER BY
CAST(SUBSTRING(id, 2, 4) AS INT),
SUBSTRING(id, 6, 3)
http://sqlfiddle.com/#!6/9eecb7db59d16c80417c72d1e1f4fbf1/9464
And one that's still less complex than a getOnlyNumbers() UDF, but copes with varying length of numeric part.
CROSS APPLY
(
SELECT
tail_start = PATINDEX('%[0-9][^0-9]%', id + '_')
)
stats
CROSS APPLY
(
SELECT
numeric = CAST(SUBSTRING(id, 2, stats.tail_start-1) AS INT),
alpha = RIGHT(id, LEN(id) - stats.tail_start)
)
id_tuple
ORDER BY
id_tuple.numeric,
id_tuple.alpha
http://sqlfiddle.com/#!6/9eecb7db59d16c80417c72d1e1f4fbf1/9499
Finally, one that can cope with there being no number at all (but still assumes the first character exists and should be ignored).
CROSS APPLY
(
SELECT
tail_start = NULLIF(PATINDEX('%[0-9][^0-9]%', id + '_'), 0)
)
stats
CROSS APPLY
(
SELECT
numeric = CAST(SUBSTRING(id, 2, stats.tail_start-1) AS INT),
alpha = RIGHT(id, LEN(id) - ISNULL(stats.tail_start, 1))
)
id_tuple
ORDER BY
id_tuple.numeric,
id_tuple.alpha
http://sqlfiddle.com/#!6/9eecb7db59d16c80417c72d1e1f4fbf1/9507
This is a rather strange way to sort but now that I understand it I figured out a solution. I am using a table valued function here to strip out only the numbers from a string. Since the function returns all numeric characters I also need to check for the _ and only pass in the part of the string before that.
Here is the function.
create function GetOnlyNumbers
(
#SearchVal varchar(8000)
) returns table as return
with MyValues as
(
select substring(#SearchVal, N, 1) as number
, t.N
from cteTally t
where N <= len(#SearchVal)
and substring(#SearchVal, N, 1) like '[0-9]'
)
select distinct NumValue = STUFF((select number + ''
from MyValues mv2
order by mv2.N
for xml path('')), 1, 0, '')
from MyValues mv
This function is using a tally table. If you have one you can tweak that code slightly to fit. Here is my tally table. I keep it as a view.
create View [dbo].[cteTally] as
WITH
E1(N) AS (select 1 from (values (1),(1),(1),(1),(1),(1),(1),(1),(1),(1))dt(n)),
E2(N) AS (SELECT 1 FROM E1 a, E1 b), --10E+2 or 100 rows
E4(N) AS (SELECT 1 FROM E2 a, E2 b), --10E+4 or 10,000 rows max
cteTally(N) AS
(
SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) FROM E4
)
select N from cteTally
GO
Next of course we need to have some data to work. In this case I just created a table variable to represent your actual table.
declare #Something table
(
SomeVal varchar(10)
)
insert #Something values
('P100')
, ('P208')
, ('P218')
, ('P301')
, ('P305')
, ('P306')
, ('P4200')
, ('P4510')
, ('P4511')
, ('P4512')
, ('P5011')
, ('P1400A')
, ('P4125H')
, ('P4202A')
, ('P4507L')
, ('P4706A')
, ('P1001_2')
, ('P2103_B')
, ('P4368_RL')
With all the legwork and setup behind us we can get to the actual query needed to accomplish this.
select s.SomeVal
from #Something s
cross apply dbo.GetOnlyNumbers(case when charindex('_', s.SomeVal) = 0 then s.SomeVal else left(s.SomeVal, charindex('_', s.SomeVal) - 1) end) x
order by convert(int, x.NumValue)
This returns the rows in the order you listed them in your question.
You can break down ID in steps to extract the number. Then, order by the number and ID. I like to break down long string manipulation into steps using CROSS APPLY. You can do it inline (it'd be long) or bundle it into an inline TVF.
SELECT t.*
FROM MYTABLE t
CROSS APPLY (SELECT NoP = STUFF(ID, 1, 1, '')) nop
CROSS APPLY (SELECT FindNonNumeric = LEFT(NoP, ISNULL(NULLIF(PATINDEX('%[^0-9]%', NoP)-1, -1), LEN(NoP)))) fnn
CROSS APPLY (SELECT Number = CONVERT(INT, FindNonNumeric)) num
ORDER BY Number
, ID;
I think your best bet is to create a function that strips the numbers out of the string, like this one, and then sort by that. Even better, as #SeanLange suggested, would be to use that function to store the number value in a new column and sort by that.
In a query I create lots of fields using the CASE expression.
I need to reference these fields later on in the query but it seems I can't access the field using its alias - I have to repeat the CASE expression every time I want to reference its value.
Is there a simple way to access these fields?
You can use CTEs (assuming SQL Server 2005+), like this very basic example:
DECLARE #Val INT
SET #Val = 1
;WITH CTEExample AS
(
SELECT CASE #Val WHEN 1 THEN 'A' ELSE 'B' END AS MyCaseField1
)
SELECT * FROM CTEExample WHERE MyCaseField1 = 'A'
why not simply make a subquery
eg
SELECT foo.column
FROM (
SELECT
CASE WHEN yourcase THEN 'a'
ELSE 'b'
END AS 'column'
FROM yourtable) AS foo
but this can be done with CTEs either (look at this answer)
You can also use CROSS APPLY for this as in this tip by Itzik Ben Gan.
SELECT SalesOrderID, OrderDate, Week_Day
FROM Sales.SalesOrderHeader
CROSS APPLY
(SELECT DATEPART(weekday, DATEADD(day, ##DATEFIRST - 7, OrderDate)) AS Week_Day) AS A
WHERE Week_Day NOT IN (1, 7);
You should be aware that reusing aliases in a where clause will render the predicate unsargable however so should be used with caution (this applies regardless of whether you use APPLY or a CTE)