i need to detect rising and falling edges from a loading state in my logs and need to list all loading cycles.
Lets say i have a table LOG
UTS | VALUE | STATE
1438392102 | 1000 | 0
1438392104 | 1001 | 1
1438392106 | 1002 | 1
1438392107 | 1003 | 0
1438392201 | 1007 | 1
1438392220 | 1045 | 1
1438392289 | 1073 | 0
1438392305 | 1085 | 1
1438392310 | 1090 | 1
1438392315 | 1095 | 1
And need all cycles where STATE = 1
I need to know when they started how long they lasted
and how much VALUE changed in each cycle.
I also might have a situation where the last cycle isn't
finished yet.
Do you have an idea how i can do this in SQL in a
good performing way ? Cuz i might run into situations
where my logs return several hundred of thausends of rows.
Thanks for help
Related
I'm currently trying to query up a list of the top 15 occurring faults on a PLC in the warehouse. I've gotten that part down:
Select top 15 fault_number, fault_message, count(*) FaultCount
from Faults_Stator
where T_stamp> dateadd(hour, -18, getdate())
Group by Fault_number, Fault_Message
Order by Faultcount desc
HOOOWEVER I now need to find out the accumulated downtime of said faults in the top 15 list, information in another column "Fault_duration". How would I go about doing this? Thanks in advance, you've all helped me so much already.
+--------------+---------------------------------------------+------------+
| Fault Number | Fault Message | FaultCount |
+--------------+---------------------------------------------+------------+
| 122 | ST10: Part A&B Failed | 23 |
| 4 | ST16: Part on Table B | 18 |
| 5 | ST7: No Spring Present on Part A | 15 |
| 6 | ST7: No Spring Present on Part B | 12 |
| 8 | ST3: No Pin Present B | 8 |
| 1 | ST5: No A Housing | 5 |
| 71 | ST4: Shuttle Right Not Loaded | 4 |
| 144 | ST15: Vertical Cylinder did not Retract | 3 |
| 98 | ST8: Plate Loader Can not Retract | 3 |
| 72 | ST4: Shuttle Left Not Loaded | 2 |
| 94 | ST8: Spring Gripper Cylinder did not Extend | 2 |
| 60 | ST8: Plate Loader Can not Retract | 1 |
| 83 | ST6: No A Spring Present | 1 |
| 2 | ST5: No B Housing | 1 |
| 51 | ST4: Vertical Cylinder did not Extend | 1 |
+--------------+---------------------------------------------+------------+
I know I wouldn't be using the same query, but I'm at a loss at how to do this next step.
Fault duration is a column which dictates how long the fault lasted in ms. I'm trying to have those accumulated next to the corresponding fault. So the first offender would have those 23 individual fault occurrences summed next to it, in another column.
You should be able to use the SUM accumulator:
Select top 15 fault_number, fault_message, count(*) FaultCount, SUM (Fault_duration) as FaultDuration
from Faults_Stator
where T_stamp> dateadd(hour, -18, getdate())
Group by Fault_number, Fault_Message
Order by Faultcount desc
I have a web application that sends data to my sqlite database into different tables depending on the information. I would like to make a view that merges multiple tables together based on cownumber and TS[timestamp] (There are no updates to my table, so a change to the same cownumber send the full record as a new entry with a new timestamp). The ajax calls are made table by table so the TS do not exactly sync up generally they can be 5-20 seconds off depending on the connection
Here is a sample of the three tables
+----master_animal-----+
+----------------------------------------------------+
| cownumber | height | weight | ts |
+-----------+----------+--------+--------------------+
| 1 | 150 | ... | 2017-12-01 12:28:00|
| 2 | 170 | ... | 2017-12-03 17:16:00|
| 3 | 60 | ... | 2017-12-03 08:09:00|
| 4 | 109 | ... | 2017-12-04 23:23:00|
+----animal_inventory-----+
+-------------------------------------------------------------+
| cownumber | brandlocation| dateacquired| ts |
+-----------+--------------+-------------+--------------------+
| 1 | ... | ... | 2017-12-01 12:28:50|
| 2 | ... | ... | 2017-12-03 17:16:30|
| 3 | ... | ... | 2017-12-03 08:09:12|
| 4 | ... | ... | 2017-12-04 23:23:23|
+----experiment-----+
+-------------------------------------------------------------+
| cownumber | ageatwean | birthweight | ts |
+-----------+--------------+-------------+--------------------+
| 1 | ... | ... | 2017-12-01 12:28:20|
| 2 | ... | ... | 2017-12-03 17:16:41|
| 3 | ... | ... | 2017-12-03 08:09:24|
| 4 | ... | ... | 2017-12-04 23:23:11|
The View I wrote
CREATE VIEW testing
AS SELECT a.height,a.weight,a.cownumber,
b.brandlocation,b.dateacquired,
c.ageatwean,c.birthweight
FROM master_animal a, animal_inventory b, experiment c
WHERE a.cownumber=b.cownumber
AND ROUND(a.ts/10000) = ROUND(b.ts/10000)
AND a.cownumber=c.cownumber
AND ROUND(a.ts/10000) = ROUND(c.ts/10000);
The query I wrote
Select * from testing where cownumber = 1;
What I was hoping to get back was
+----testing-----+
+----------------------------------------------------+
| cownumber | height | weight | brandlocation| dateacquired | ageatwean |birthweight |
+-----------+--------+--------+--------------+--------------+-----------+------------+
| 941 | 0 | ... | ... | ... | ... | .. |
Where there will be one row for cownumber 941 as long as all the correlated records were within a few seconds of each other. I am not exactly sure if I need to divide by 10000 or smaller. The same record should be no more than 50 seconds apart from each other. Anything more than 50 seconds apart should be considered a new record.
When I test this where there is only one record for that cownumber it works fine. But lets say I change some information from each table. I provide a new height, a new brandlocation.
Instead of getting two rows. The first row being the initial data entry and the second row showing the same cownumber with the changed values, I get back 8 rows with partial changes.
height|weight|cownumber|brandlocation|dateacquired|ageatwean|birthweight|
0.0|0.0|941|0|0|0.0|0
0.0|0.0|941|0|0|0.0|0
0.0|0.0|941|Left Hip|0|0.0|0
0.0|0.0|941|Left Hip|0|0.0|0
50.0|0.0|941|0|0|0.0|0
50.0|0.0|941|0|0|0.0|0
50.0|0.0|941|Left Hip|0|0.0|0
50.0|0.0|941|Left Hip|0|0.0|0
I assume the issue is in my where clause but I am not sure exactly how to fix it
The timestamps are stored as strings. When you try to divide it, the database tries to convert it to a number, which results in 2017. So all timestamps end up being the same.
Dividing cannot determine the distance; the values 9999 and 10000 will end up different although they are right near each other. (And an integer division results in an integer result, so the ROUND() has no effect.)
To compute the distance, convert the timestamp into a number of seconds first, and then use abs():
SELECT ...
FROM master_animal m
JOIN animal_inventory i ON m.cownumber = i.cownumber
AND abs(strftime('%s', m.ts) - strftime('%s', i.ts)) <= 50
JOIN experiment e ON m.cownumber = e.cownumber
AND abs(strftime('%s', m.ts) - strftime('%s', e.ts)) <= 50;
I have an influxdb table lets call it my_table
my_table is structured like this (simplified):
+-----+-----+-----
| Time| m1 | m2 |
+=====+=====+=====
| 1 | 8 | 4 |
+-----+-----+-----
| 2 | 1 | 12 |
+-----+-----+-----
| 3 | 6 | 18 |
+-----+-----+-----
| 4 | 4 | 1 |
+-----+-----+-----
However I was wondering if it is possible to find out how many of the metrics are larger than a certain (dynamic) threshold for each time.
So lets say I want to know how many of the metrics (columns) are higher than 5,
I would want to do something like this:
select fieldcount(/m*/) from my_table where /m*/ > 5
Returning:
1
1
2
0
I am relatively restricted in structuring the database as I'm using diamond collector (python) which takes care of all datacollection for me and flushes it to my influxdb without me telling what the tables should look like.
EDIT
I am aware of a possible solution if I hardcode the threshold and add a third metric named mGreaterThan5:
+-----+-----+------------------+
| Time| m1 | m2 |mGreaterThan5|
+=====+=====+====+=============+
| 1 | 8 | 4 | 1 |
+-----+-----+----+-------------+
| 2 | 1 | 12 | 1 |
+-----+-----+----+-------------+
| 3 | 6 | 18 | 2 |
+-----+-----+----+-------------+
| 4 | 4 | 1 | 0 |
+-----+-----+----+-------------+
However this means that I cant easily change this threshold to 6 or any other number so thats why I would prefer a better solution if there is one.
EDIT2
Another similar problem occurs with trying to retrieve the highest x amount of metrics. Eg:
On Jan 1st what were the highest 3 values of m? Given table:
+-----+-----+----+-----+----+-----+----+
| Time| m1 | m2 | m3 | m4 | m5 | m6 |
+=====+=====+====+=====+====+=====+====+
| 1/1 | 8 | 4 | 1 | 7 | 2 | 0 |
+-----+-----+----+-----+----+-----+----+
Am I screwed if I keep the table structured this way?
I have a joining table that logs the changes in a File's state (Active/Deleted/Archived etc).
e.g.
+----+---------+------------+-------+
| PK | File_FK | Date | State |
+----+---------+------------+-------+
| 1 | 100 | 11-9-2015 | 1 |
| 2 | 200 | 14-09-2015 | 2 |
| 3 | 300 | 14-07-2015 | 0 |
| 4 | 300 | 12-9-2015 | 2 |
| 5 | 200 | 14-09-2015 | 2 |
| 6 | 300 | 14-09-2015 | 0 |
| 7 | 300 | 13-09-2015 | 1 |
+----+---------+------------+-------+
There are a number of records which are were not inserted accordingly, on a certain date e.g. July 15.
Is there a way how to automate the updating of certain records?
I want to change the state to the last option it was (latest date), if the state is 0 and if there are no previous 0 as states (for the same file foreign key) without amending previous states which had a zero.
To try and simplify (taking File_FK = 300 as example):
I want:
The state of the file to update to be 1.
State '0' is invalid since it was made on 14-09-2015 (the last one)
Prevent updating the '0' that was made on 14-07-2015.
Basically I want to change the state of a file to the state it was before it was '0' without changing the 0s that where done earlier.
I need to provide users with exact time of data integrity in case of forced service with possible data loss.
I guess that I can find last LSN using:
SELECT [mirroring_failover_lsn]
FROM [master].[sys].[database_mirroring]
But that won't give me exact time.
Read How to read and interpret the SQL Server log. You'll see that LOP_BEGIN_XACT contains a timestamp. Given an LSN, you can analize the log and find all pending transactions (that is all xact_ids that do not have a commit or rollback logged before the given LSN). All the pending transations will be rolled back in case of failover. This will be the data lost if a forced failover occurs. There will be a number of transactions pending that will be undone, and these various transactions have started at various times. If you want to attach a 'exact time of data integrity' then you can say that no data loss will occur for anything earlier than the earliest pending lop_begin_xact. Eg. given the following log stream:
+-----+-----------+---------+------------+
| LSN | Operation | xact_id | timestampt |
+-----+-----------+---------+------------+
| 1 |INSERT | 1 | |
| 2 |BEGIN_XACT | 2 | 12:00 |
| 3 |INSERT | 1 | |
| 4 |BEGIN_XACT | 3 | 12:02 |
| 5 |COMMIT_XACT| 1 | |
| 6 |INSERT | 2 | |
| 7 |INSERT | 3 | |
| 8 |COMMIT_XACT| 3 | |
| 9 |COMMIT_XACT| 2 | |
Lets say that the mirroring failover LSN is 8. In this case you can say that not data loss will occur earlier than 12:00, because xact_id 2 is not committed at LSN 8 and therefore it will be rolled back. Note that xact_id 3 is commited by LSN 8 so it won't be lost, even though it has a later time stamp. So your timestamp is no t absolute, this is why I say 'no data loss will occur earlier than...' rather than 'data after ... will be lost'.