I have a table containing weights over time which I want to evaluate as flow:
Scan TimeStamp Position Weight
01 14/11/01 12:00 0 0
01 14/11/01 12:10 10 1.6
02 14/11/01 13:00 0 2.6
02 14/11/01 13:10 10 4.2
...
Now I want to calculate the flow during a scan (begin to end).
My query looks like that:
Select MeanTime, TheFlow From
(Select AVG(TheTimeStamp) as MeanTime From flow Where ScanNumber=73),
(Select Weightdiff / TimeSpan as TheFlow From
(Select (MaxWeight - MinWeight) as WeightDiff From
(Select Weight as MAXWEIGHT from Flow Where ScanNumber=73 HAVING "POSITION"=MAX("POSITION")),
(Select Weight as MINWEIGHT from FLOW Where ScanNumber=73 HAVING "POSITION"=MIN("POSITION")),
(Select (MaxTime - MinTime) * 24 as TimeSpan From
(Select MAX("THETIMESTAMP") as MaxTime From FLOW Where ScanNumber=73),
(Select MIN("THETIMESTAMP") as MinTime From Flow Where ScanNumber=73))));
I get an error:
SQL error code = -104.
Invalid expression in the select list (not contained in either an aggregate function or the GROUP BY clause).
What's wrong?
To clarify my question, I need to extract the following information out of the data:
the mean time between the start (eg. 12:00) and the end eg. 12:10) of a scan (MeanTime)
e.g. Scannumber 01), i.e. 12:05
I need the weight difference between end and start
I have to calculate the "Flow" from the weight diff and the time between start and end
All in all I need two data Meantime and flow, which I want to plot (flow over time)
This should do the job for an individual Scan, which appears to be the requirement.
DECLARE #Scan INT = 1
SELECT
MeanTime = DATEADD(SECOND, DATEDIFF(SECOND, FirstScan.TimeStamp, LastScan.TimeStamp), FirstScan.TimeStamp)
, WeightDifference = LastScan.Weight - FirstScan.Weight
FROM
(SELECT Position = MIN(Position) FROM Flow WHERE Scan = #Scan) MinScan
CROSS JOIN (SELECT Position = MAX(Position) FROM Flow WHERE Scan = #Scan) MaxScan
INNER JOIN Flow FirstScan ON MinScan.Position = FirstScan.Position
AND FirstScan.Scan = #Scan
INNER JOIN Flow LastScan ON MaxScan.Position = LastScan.Position
AND LastScan.Scan = #Scan
Related
I have a log file tblktime that contains all activity for 'Ourfile'. A second summary table tblktimefst (one record per ourfile) has been created to capture the first occurrence of particular codes in the log file and summarize the 'duration' column between the first record for that ourfile and when it appears in the tblktime log. (The first occurrence of 'LSF' is stored as fstLSFBtime.)
The Select statement that I have written accurately returns the unique id field 'Ourfile' and the correct sum as 'TotDur' that I would like to update back to tblktimefst.fstLSFDurSum
SELECT DISTINCT
ktf.ourfile,
SUM(CASE WHEN kt.btime <= ktf.fstlsfbtime THEN kt.duration ELSE 0 END) AS TotDur
FROM
tblktime kt
INNER JOIN
tblktimefst ktf ON ktf.ourfile = kt.ourfile
AND kt.btime <= ktf.fstLSFBtime
GROUP BY
ktf.ourfile
SQL Server returns:
Ourfile
TotDur
47661
48
50265
48
36453
13
38270
54
39730
27
46156
190
37392
46
51905
25
and so on
The column that I'd like to update in tblktimefst is fstLSFDurSum for its unique 'Ourfile' record. I cant get the syntax correct to do so and hope I have provided enough info that someone may be able to assist.
If I can get the syntax correct for this one then I should be able to complete duration summaries between additional codes as well.
Thank you in advance.
What about (untested):
UPDATE tblktimefst
SET fstLSFDurSum = TotDur
FROM tblktime t2
INNER JOIN
(
select distinct
ktf.ourfile,
SUM( CAse when kt.btime <= ktf.fstlsfbtime then kt.duration else 0 end) as TotDur
from tblktime kt
Inner JOIN tblktimefst ktf on ktf.ourfile = kt.ourfile and kt.btime <= ktf.fstLSFBtime
group by ktf.ourfile) x ON x.ourfile = t2.ourfile
! - I'm not looking for paid software which will do this job (as too expensive)
We have an issue with cash management to match the values.
I have two SQL Tables, let's call it SHOP_CASH and BANK_CASH
1) The matching should be happens based on ShopName-CashAmount-Date.
2) Here I faced two issues
The cash should be round up to nearest £50, ideally, 12 400 and 12 499 should round up to 12 450, OR this just IDEAL is a match based on cash difference which less than 50, dry to match different value if the difference is less than 50, match them, but here is the question how to match the value up.. this is just the stupid ideas))??? Hmmm...stuck.
Dates, the shop can cash up a few days later, so need to join based on cash-up date (for example 2018-10-26) with bank date RANGE 2018-10-26 to (+7 days) 2018-11-02
Currently, I do not understand the possible way (logical) of matching in this circumstance. Any logical path of calculation/joining will be extremely appreciated
TRY:
Let's say I can join two tables by SHOPNAME - Cool
Then I will try to join by date, which potentially will be:
SELECT * FROM SHOP_CASH AS SC
LEFT JOIN BANK_CASH AS BC
ON SC.SHOP_NAME_SC = BC.SHOP_NAME_BC
AND SC.DATE_SC = (ANY DATE FROM SC.DATE_SC TO SC.DATE_SC (+7 DAYS) = TO DATE_BC - not sure how)
AND FLOOR(SC.CASH_SC / 50) * 50 = FLOOR(BC_CASH_BC / 50) * 50
P.S. For this project will be using the Google Big Query.
This is my (temporary solution)
WITH MAIN AS(SELECT
CMS.Store_name AS STORE_NAME,
CMS.Date AS SHOP_DATE,
CMB.ENTRY_DATE AS BANK_DATE,
SUM(CMS.Cash) AS STORE_CASH,
SUM(CMB.AMOUNT) AS BANK_CASH
FROM `store_data` CMS
LEFT JOIN `bank_data` AS CMB
ON CMS.store_name = CMB.STRAIGHT_LOOKUP
AND FLOOR(CMS.Cash / 50) * 50 = FLOOR(CMB.AMOUNT / 50) * 50
AND CAST(FORMAT_DATE("%F",CMB.ENTRY_DATE) AS STRING) > CAST(FORMAT_DATE("%F",CMS.Date) AS STRING)
AND CAST(FORMAT_DATE("%F",CMB.ENTRY_DATE) AS STRING) <= CAST(FORMAT_DATE("%F",DATE_ADD(CMS.Date, INTERVAL 4 day)) AS STRING)
GROUP BY STORE_NAME,SHOP_DATE,BANK_DATE)
SELECT
MAIN2.*
FROM (
SELECT
ARRAY_AGG(MAIN ORDER BY MAIN.SHOP_DATE ASC LIMIT 1)[OFFSET(0)] AS MAIN2
FROM
MAIN AS MAIN
GROUP BY MAIN.SHOP_DATE, MAIN.STORE_CASH)
this is quite interesting case.
You haven't provided any sample data so I'm not able to test it, but this may work. Some modification may be required since not sure about date format. Let me know if there is an issue.
SELECT * FROM SHOP_CASH AS SC
LEFT JOIN BANK_CASH AS BC
ON SC.SHOP_NAME_SC = BC.SHOP_NAME_BC
AND SC.DATE_SC BETWEEN BC.DATE_BC AND DATE_ADD(BC.DATE_BC, DAY 7)
AND trunc(SC.CASH_SC, -2) + 50 = trunc(BC.CASH_BC,2) + 50
I was doing a test, when I generated data that was 30 days old.
When sent to SA job all that input was dropped, but per settings in event ordering blade I was expecting that all will be passed thru.
Part of job query contains:
---------------all incoming events storage query
SELECT stream.*
INTO [iot-predict-SA2-ColdStorage]
FROM [iot-predict-SA2-input] stream TIMESTAMP BY stream.UtcTime
so my expectation is to have everything that was pushed to SA job in blob storage.
When I sent events that were only 5 hours old - then the input was marked as late (expected) and processed.
Per SS first marked area is showing outdated events input, but no output (red), the second part shows late processed events.
full query
WITH AlertsBasedOnMin
AS (
SELECT stream.SensorGuid
,stream.Value
,stream.SensorName
,ref.AggregationTypeFlag
,ref.MinThreshold AS threshold
,ref.Count
,CASE
WHEN (ref.MinThreshold > stream.Value)
THEN 1
ELSE 0
END AS isAlert
FROM [iot-predict-SA2-input] stream TIMESTAMP BY stream.UtcTime
JOIN [iot-predict-SA2-referenceBlob] ref ON ref.SensorGuid = stream.SensorGuid
WHERE ref.AggregationTypeFlag = 8
)
,AlertsBasedOnMax
AS (
SELECT stream.SensorGuid
,stream.Value
,stream.SensorName
,ref.AggregationTypeFlag
,ref.MaxThreshold AS threshold
,ref.Count
,CASE
WHEN (ref.MaxThreshold < stream.Value)
THEN 1
ELSE 0
END AS isAlert
FROM [iot-predict-SA2-input] stream TIMESTAMP BY stream.UtcTime
JOIN [iot-predict-SA2-referenceBlob] ref ON ref.SensorGuid = stream.SensorGuid
WHERE ref.AggregationTypeFlag = 16
)
,alertMinMaxUnion
AS (
SELECT *
FROM AlertsBasedOnMin
UNION ALL
SELECT *
FROM AlertsBasedOnMax
)
,alertMimMaxComputed
AS (
SELECT SUM(alertMinMaxUnion.isAlert) AS EventCount
,alertMinMaxUnion.SensorGuid AS SensorGuid
,alertMinMaxUnion.SensorName
FROM alertMinMaxUnion
GROUP BY HoppingWindow(Duration(minute, 1), Hop(second, 30))
,alertMinMaxUnion.SensorGuid
,alertMinMaxUnion.Count
,alertMinMaxUnion.AggregationTypeFlag
,alertMinMaxUnion.SensorName
HAVING SUM(alertMinMaxUnion.isAlert) > alertMinMaxUnion.Count
)
,alertsMimMaxComputedMergedWithReference
AS (
SELECT System.TIMESTAMP [TimeStampUtc]
,computed.EventCount
,0 AS SumValue
,0 AS AvgValue
,0 AS StdDevValue
,computed.SensorGuid
,computed.SensorName
,ref.MinThreshold
,ref.MaxThreshold
,ref.TimeFrameInSeconds
,ref.Count
,ref.GatewayGuid
,ref.SensorType
,ref.AggregationType
,ref.AggregationTypeFlag
,ref.EmailList
,ref.PhoneNumberList
FROM alertMimMaxComputed computed
JOIN [iot-predict-SA2-referenceBlob] ref ON ref.SensorGuid = computed.SensorGuid
)
,alertsAggregatedByFunction
AS (
SELECT Count(1) AS eventCount
,stream.SensorGuid AS SensorGuid
,stream.SensorName
,ref.[Count] AS TriggerThreshold
,SUM(stream.Value) AS SumValue
,AVG(stream.Value) AS AvgValue
,STDEV(stream.Value) AS StdDevValue
,ref.AggregationTypeFlag AS flag
FROM [iot-predict-SA2-input] stream TIMESTAMP BY stream.UtcTime
JOIN [iot-predict-SA2-referenceBlob] ref ON ref.SensorGuid = stream.SensorGuid
GROUP BY HoppingWindow(Duration(minute, 1), Hop(second, 30))
,ref.AggregationTypeFlag
,ref.[Count]
,ref.MaxThreshold
,ref.MinThreshold
,stream.SensorGuid
,stream.SensorName
HAVING
--as this is alert then this factor will be relevant to all of the aggregated queries
Count(1) >= ref.[Count]
AND (
--average
(
ref.AggregationTypeFlag = 1
AND (
AVG(stream.Value) >= ref.MaxThreshold
OR AVG(stream.Value) <= ref.MinThreshold
)
)
--sum
OR (
ref.AggregationTypeFlag = 2
AND (
SUM(stream.Value) >= ref.MaxThreshold
OR Sum(stream.Value) <= ref.MinThreshold
)
)
--stdev
OR (
ref.AggregationTypeFlag = 4
AND (
STDEV(stream.Value) >= ref.MaxThreshold
OR STDEV(stream.Value) <= ref.MinThreshold
)
)
)
)
,alertsAggregatedByFunctionMergedWithReference
AS (
SELECT System.TIMESTAMP [TimeStampUtc]
,0 AS EventCount
,computed.SumValue
,computed.AvgValue
,computed.StdDevValue
,computed.SensorGuid
,computed.SensorName
,ref.MinThreshold
,ref.MaxThreshold
,ref.TimeFrameInSeconds
,ref.Count
,ref.GatewayGuid
,ref.SensorType
,ref.AggregationType
,ref.AggregationTypeFlag
,ref.EmailList
,ref.PhoneNumberList
FROM alertsAggregatedByFunction computed
JOIN [iot-predict-SA2-referenceBlob] ref ON ref.SensorGuid = computed.SensorGuid
)
,allAlertsUnioned
AS (
SELECT *
FROM alertsAggregatedByFunctionMergedWithReference
UNION ALL
SELECT *
FROM alertsMimMaxComputedMergedWithReference
)
---------------alerts storage query
SELECT *
INTO [iot-predict-SA2-Alerts-ColdStorage]
FROM allAlertsUnioned
---------------alerts to alert events query
SELECT *
INTO [iot-predict-SA2-Alerts-EventStream]
FROM allAlertsUnioned
---------------alerts to stream query
SELECT *
INTO [iot-predict-SA2-TSI-EventStream]
FROM allAlertsUnioned
---------------all incoming events storage query
SELECT stream.*
INTO [iot-predict-SA2-ColdStorage]
FROM [iot-predict-SA2-input] stream TIMESTAMP BY stream.UtcTime
---------------all incoming events to time insights query
SELECT stream.*
INTO [iot-predict-SA2-TSI-AlertStream]
FROM [iot-predict-SA2-input] stream TIMESTAMP BY stream.UtcTime
Since you are using "TIMESTAMP BY", Stream Analytics job event ordering settings are taking effects. Please check your job's "event ordering" settings, specifically below two:
Events that arrive late -- the late arrival limit between 0 second and 21 days.
Handling other events -- error handling policy, drop or adjust the application time to system clock time.
I guess that, most likely, your late arrival limit was more than 5 hours, so that those 5-hours old events could be processed.
You may already figure out from above that Stream Analytics job can only process "old" events up to 21 days late. To work around this limitation, you can consider one of below options:
Remove TIMESTAMP BY, then all your windowing aggregate will be using enqueue time. This might generate incorrect result according to your query logic.
Select "adjust" as the error handling policy. Again, this might generate incorrect result according to your query logic.
Shifting the application time (stream.UtcTime) to a more resent time by using DATEADD() function, for example TIMESTAMP BY DATEADD(day, 10, UtcTime). This works well when this is a onetime task, and you know the time range of your events.
Use batch job(outside Stream Analytics) to process data that 30 days old.
After a chat with guys from MS, it emerged that my test have to had an extra step to perform.
To have late events processed, regardless late event settings, we need to start this job in a way, that late event is considered as a sent when job was started, so in this particular case, we have to start SA job using custom start date and set it 30 days ago.
I've been working in a time attendance application and need some enlightment.
This is my current SQL query:
SELECT M.IdMarcacao, M.IdFuncionario, M.Data, M.Hora, Extra,
(M.Hora-(convert(varchar(11),dateadd(ms,cast(Extra*3600000 as bigint),'12/30/1899'),108))) as teste
FROM TimeReport.dbo.Marcacoes M
INNER JOIN TimeReport.dbo.Resultados R ON M.IdFuncionario = R.IdFuncionario
AND M.Data = R.Data
WHERE (R.Extra <> 0 AND M.[Tipo Marcacao] = 'SAI')
AND M.Hora=(SELECT max(hora)
FROM timereport.dbo.marcacoes
WHERE data = M.Data)
This returns the lines where:
1 - The person has made overtime hours.
2 - The last logout time
3 - The total of overtime hours.
4 - The difference between logout time and overtime = schedule time
5 - Make sure it's a logout time (Tipo = SAI)
Without entering into much detail about the application itself, what I really need is to turn this into an UPDATE statement.
I used to do this:
UPDATE [TimeReport].[dbo].[Marcacoes]
SET [Hora] = [Hora] - convert(datetime,'01:00:00',108)
WHERE [Hora] > '1899-12-30 19:00:00.000'
For every single hour until 0pm :( It's not a good solution I know.
This update, will change the original logout time to minus 1 hour if the worker left at 19h, when the company schedule is 18h.
What I am trying to do, is simplify and automate the process.
Which brings my previous question... How can I do a update based on that select statement?
Or... in other words, something like:
UPDATE TimeReport.dbo.Marcacoes
SET Hora = (The value from the statement above, field "teste")
WHERE IdMarcacao = IdMarcacao(from the statement above)
Note: This "IdMarcacao" is a unique identifier for the row.
Thank you!
update TimeReport.dbo.Marcacoes set
Hora = (M.Hora-(convert(varchar(11),dateadd(ms,cast(Extra*3600000 as bigint),'12/30/1899'),108)))
--select M.IdMarcacao, M.IdFuncionario, M.Data, M.Hora, Extra, (M.Hora-(convert(varchar(11),dateadd(ms,cast(Extra*3600000 as bigint),'12/30/1899'),108))) teste
from TimeReport.dbo.Marcacoes M
INNER JOIN TimeReport.dbo.Resultados R ON M.IdFuncionario = R.IdFuncionario AND M.Data = R.Data
WHERE (R.Extra <> 0 AND M.[Tipo Marcacao] = 'SAI')
AND M.Hora=(select max(hora) from timereport.dbo.marcacoes where data = M.Data)
I have to determine stopped time of an vehicle that sends back to server its status data every 30 second and this data is stored in a table of a database.
The fields of a status record consist of (vehicleID, ReceiveDate, ReceiveTime, Speed, Location).
Now what I want to do is, determine each suspension time at the point that vehicle speed came to zero to the status the vehicle move again and so on for next suspension time.
For example on a given day, a given vehicle may have 10 stopped status and I must determine duration of each by a query.
The result can be like this:
id Recvdate Rtime Duration
1 2010-05-01 8:30 45min
1 2110-05-01 12:21 3hour
This is an application of windows functions (called analytic functions in Oracle).
Your goal is to assign a "block number" to each sequence of stops. That is, all stops in a sequence (for a vehicle) will have the same block number, and this will be different from all other sequences of stops.
Here is a way to assign the block number:
Create a speed flag that says 1 when speed > 0 and 0 when speed = 0.
Enumerate all the records where the speed flag = 1. These are "blocks".
Do a self join to put each flag = 0 in a block (this requires grouping and taking the max blocknum).
Summarize by duration or however you want.
The following code is a sketch of what I mean. It won't solve your problem, because you are not clear about how to handle day breaks, what information you want to summarize, and it has an off-by-1 error (in each sequence of stops it includes the previous non-stop, if any).
with vd as
(
select vd.*,
(case when SpeedFlag = 1
then ROW_NUMBER() over (partition by id, SpeedFlag) end) as blocknum
from
(
select vd.*, (case when speed = 0 then 0 else 1 end) as SpeedFlag
from vehicaldata vd
) vd
)
select id, blocknum, COUNT(*) as numrecs, SUM(duration) as duration
from
(
select vd.id, vd.rtime, vd.duration, MAX(vdprev.blocknum) as blocknum
from vd
left outer join vd vdprev
on vd.id = vdprev.id
and vd.rtime > vdprev.rtime
group by vd.id, vd.rtime, vd.duration
) vd
group by id, blocknum