In dask or even pandas how would you go about grouping an dask data frame that has a 3 columns of time / level / spread into a set of fixed ranges by time.
Time is only used to move one direction. Like a loop counting up. So the end result would be start time and end time with high of level, low of level, first value of level and last value of level over the fixed range? Example
12:00:00, 10, 1
12:00:01, 11, 1
12:00:02, 12, 1
12:00:03, 11, 1
12:00:04, 9, 1
12:00:05, 6, 1
12:00:06, 10, 1
12:00:07, 14, 1
12:00:08, 11, 1
12:00:09, 7, 1
12:00:10, 13, 1
12:00:11, 8, 1
For a fixed level range of (7). So level from start to end can not be more than 7 total distance from start to end for each bin of level. Just because first bin is only 8 difference in time and second is only 2 different in time, this dose not madder one the high to low madders that the total distance from high to low dose not go passed 7 the fixed bin size. The first bin could have been 5 not 8 for first bin and 200 for next bin not 2 in the example below. So the First few rows in dask would look something like this.
First Time, Last Time, High Level, Low Level, First Level, Last Level, Spread
12:00:00, 12:00:07, 13, 6, 10, 13, 1
12:00:07, 12:00:09, 14, 7, 13, 7, 1
12:00:09, X, 13, 7, X, X, X
How could this be aggregated in dask with a fix window of level moving forward in time binning each time level moves above X or equal too high/low with in X or below X?
Related
I am writing a model to calculate the maximum production capacity for a machine in a year based on 15-min data. As the maximum capacity is not the sum of the required capacity for all 15-min over the year, I want to write a piece of code that determines the maximum value in the list and then adds this maximum value and the three next consecutive values after this maximum value to a new variable. An simplified example would be:
fifteen_min_capacity = [10, 12, 3, 4, 8, 12, 10, 9, 2, 10, 4, 3, 15, 8, 9, 3, 4, 10]
The piece of code I want to write would be able to determine the maximum capacity in this list (15) and then add this capacity plus the three consecutive ones (8,9,3) to a new variables:
hourly_capacity = 35
Does anyone now the code that would give this output?
I have tried using the max(), the sum() and a combination of both. However, I do not get a working code. Any help would be much appreciated!
I am a beginner level in this program. I try to improve this loop according to this condition. The details are as follows:
When CUTI(k) = CUTI(k)-4 then,
1)If the result shows this CUTI(k) value greater than 0, then print this CUTI(k) value.
2)If the result shows CUTI(k) value less than 0, then print this CUTI(k) value is added 12 with showing a word "*" after the number in display, e.g. 10*, 9*
I am not sure this loop is correct and enough to add this condition. Look forward to seeing your recoomendation. :)
set k /1*20/;
parameter
CUTI(k)/1 6, 2 2, 3 8, 4 5, 5 1, 6 3, 7 7, 8 8, 9 6, 10 8,11 1, 12 2, 13 4, 14 7,
15 5, 16 2, 17 8, 18 9, 19 2, 20 10/;
loop(k,
if(CUTI(k)-4 > 0,
CUTI(k) = CUTI(k)-4;
else
CUTI(k) = (CUTI(k)-4)+12 ;
)
);
display CUTI;
Your logic looks correct. However, instead of the loop/if/else you could simplify this to one assignment:
CUTI(k) = CUTI(k)-4+12$(CUTI(k)<=4);
However, modifying the display statement by adding a * to some elements is not possible. If you need to distinguish the cases in such a statement, you might assign the values to two different parameters and display them individually.
The following SQL query is supposed to return the max consecutive numbers in a set.
WITH RECURSIVE Mystery(X,Y) AS (SELECT A AS X, A AS Y FROM R)
UNION (SELECT m1.X, m2.Y
FROM Mystery m1, Mystery m2
WHERE m2.X = m1.Y + 1)
SELECT MAX(Y-X) + 1 FROM Mystery;
This query on the set {7, 9, 10, 14, 15, 16, 18} returns 3, because {14 15 16} is the longest chain of consecutive numbers and there are three numbers in that chain. But when I try to work through this manually I don't see how it arrives at that result.
For example, given the number set above I could create two columns:
m1.x
m2.y
7
7
9
9
10
10
14
14
15
15
16
16
18
18
If we are working on rows and columns, not the actual data, as I understand it WHERE m2.X = m1.Y + 1 takes the value from the next row in Y and puts it in the current row of X, like so
m1.X
m2.Y
9
7
10
9
14
10
15
14
16
15
18
16
18
Null?
The main part on which I am uncertain is where in the SQL recursion actually happens. According to Denis Lukichev recursion is the R part - or in this case the RECURSIVE Mystery(X,Y) - and stops when the table is empty. But if the above is true, how would the table ever empty?
Since I don't know how to proceed with the above, let me try a different direction. If WHERE m2.X = m1.Y + 1 is actually a comparison, the result should be:
m1.X
m2.Y
14
14
15
15
16
16
But at this point, it seems that it should continue recursively on this until only two rows are left (nothing else to compare). If it stops here to get the correct count of 3 rows (2 + 1), what is actually stopping the recursion?
I understand that for the above example the MAX(Y-X) + 1 effectively returns the actual number of recursion steps and adds 1.
But if I have 7 consecutive numbers and the recursion flows down to 2 rows, should this not end up with an incorrect 3 as the result? I understand recursion in C++ and other languages, but this is confusing to me.
Full disclosure, yes it appears this is a common university question, but I am retired, discovered this while researching recursion for my use, and need to understand how it works to use similar recursion in my projects.
Based on this db<>fiddle shared previously, you may find it instructive to alter the CTE to include an iteration number as follows, and then to show the content of the CTE rather than the output of final SELECT. Here's an amended CTE and its content after the recursion is complete:
Amended CTE
WITH RECURSIVE Mystery(X,Y) AS ((SELECT A AS X, A AS Y, 1 as Z FROM R)
UNION (SELECT m1.X, m2.A, Z+1
FROM Mystery m1
JOIN R m2 ON m2.A = m1.Y + 1))
CTE Content
x
y
z
7
7
1
9
9
1
10
10
1
14
14
1
15
15
1
16
16
1
18
18
1
9
10
2
14
15
2
15
16
2
14
16
3
The Z field holds the iteration count. Where Z = 1 we've simply got the rows from the table R. The, values X and Y are both from the field A. In terms of what we are attempting to achieve these represent sequences consecutive numbers, which start at X and continue to (at least) Y.
Where Z = 2, the second iteration, we find all the rows first iteration where there is a value in R which is one higher than our Y value, or one higher than the last member of our sequence of consecutive numbers. That becomes the new highest number, and we add one to the number of iterations. As only three numbers in our original data set have successors within the set, there are only three rows output in the second iteration.
Where Z = 3, the third iteration, we find all the rows of the second iteration (note we are not considering all the rows of the first iteration again), where there is, again, a value in R which is one higher than our Y value, or one higher than the last member of our sequence of consecutive numbers. That, again, becomes the new highest number, and we add one to the number of iterations.
The process will attempt a fourth iteration, but as there are no rows in R where the value is one more than the Y values from our third iteration, no extra data gets added to the CTE and recursion ends.
Going back to the original db<>fiddle, the process then searches our CTE content to output MAX(Y-X) + 1, which is the maximum difference between the first and last values in any consecutive sequence, plus one. This finds it's value from the record produced in the third iteration, using ((16-14) + 1) which has a value of 3.
For this specific piece of code, the output is always equivalent to the value in the Z field as every addition of a row through the recursion adds one to Z and adds one to Y.
I have made a selection form a huge amount of ID's, using the following query:
select ID from [tabelname] where id > 0 and id < 31
This gives me 30 ID's to work with.
What I would like to do now, is to use 3 threads, with the first one using ID 1, 4, 7, 10 etc, the second ID 2, 5, 8, 11 etc and the third one ID 3, 6, 9, 12 etc.
Up until now, I have only been able to have all threads use ID 1 through 30 parallel to each other. Would it be at all possible to do this?
Thanks in advance!
JMeter has a build-in operation that you can use in combination with a pre-processor to find the current thread number:
https://jmeter.apache.org/api/org/apache/jmeter/threads/JMeterContext.html#getThreadNum()
If you now use ctx.getThreadGroup().getNumThreads() to find the number of threads you're using, you can basically divide your testset into subsets and let each thread compute on its own subset (e.g. thread1 computes on 0..10, thread2 on 11..20, thread3 on 21..30, etc..)
Lets say we are dealing with the keys 1-15. To get the worst case performance of a regular BST, you would insert the keys in ascending or descending order as follows:
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
Then the BST would essentially become a linked list.
For best case of a BST you would insert the keys in the following order, they are arranged in such a way that the next key inserted is half of the total range to be inserted, so the first is 15/2 = 8, then 8/2 = 4 etc...
8, 4, 12, 2, 6, 10, 14, 1, 3, 5, 7, 9, 11, 13, 15
Then the BST would be a well balanced tree with optimal height 3.
The best case for a red black tree can also be constructed with the best case from a BST. But how do we construct the worst case for a red black tree? Is it the same as the worst case for a BST? Is there a specific pattern that will yield the worst case?
You are looking for a skinny tree, right? This can be produced by inserting [1 ... , 2^(n+1)-2] in reverse order.
You won't be able to. A Red-Black Tree keeps itself "bushy", so it would rotate to fix the imbalance. The length of your above worst case for a Red-Black Tree is limited to two elements, but that's still not a "bad" case; it's what's expected, as lg(2) = 1, and you have 1 layer past the root with two elements. As soon as you add the third element, you get this:
B B
\ / \
R => R R
\
R