I am trying to understand the meaning of the display output when using SCIP to solve an MILP using Branch-and-cut code. I use the TSP example as a reference.
time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl.
p 1.9s| 1 | 0 | 0 | - | vbounds| 0 |1861 | 130 | 126 | 0 | 0 | 3 | 0 | 0.000000e+00 | 5.523000e+03 | Inf | unknown
2.2s| 1 | 0 | 1320 | - | 15M | 0 |1861 | 130 | 126 | 0 | 0 | 3 | 0 | 9.795000e+02 | 5.523000e+03 | 463.86%| unknown
2.2s| 1 | 0 | 1321 | - | 15M | 0 |1861 | 130 | 126 | 0 | 0 | 3 | 0 | 9.800000e+02 | 5.523000e+03 | 463.57%| unknown
4.6s| 1 | 0 | 1341 | - | 15M | 0 |1861 | 130 | 128 | 2 | 1 | 3 | 0 | 9.800000e+02 | 5.523000e+03 | 463.57%| unknown
4.6s| 1 | 0 | 1393 | - | 15M | 0 |1861 | 130 | 129 | 3 | 2 | 3 | 0 | 9.800000e+02 | 5.523000e+03 | 463.57%| unknown
4.7s| 1 | 0 | 1422 | - | 15M | 0 |1861 | 130 | 136 | 10 | 3 | 3 | 0 | 9.800000e+02 | 5.523000e+03 | 463.57%| unknown
4.8s| 1 | 0 | 1472 | - | 16M | 0 |1861 | 130 | 139 | 13 | 4 | 3 | 0 | 9.860000e+02 | 5.523000e+03 | 460.14%| unknown
4.8s| 1 | 0 | 1472 | - | 16M | 0 |1861 | 130 | 139 | 13 | 4 | 3 | 0 | 9.860000e+02 | 5.523000e+03 | 460.14%| unknown
4.9s| 1 | 0 | 1479 | - | 17M | 0 |1861 | 130 | 144 | 18 | 5 | 3 | 0 | 9.925000e+02 | 5.523000e+03 | 456.47%| unknown
4.9s| 1 | 0 | 1480 | - | 17M | 0 |1861 | 130 | 144 | 18 | 5 | 3 | 0 | 9.930000e+02 | 5.523000e+03 | 456.19%| unknown
5.0s| 1 | 0 | 1489 | - | 17M | 0 |1861 | 130 | 148 | 22 | 6 | 3 | 0 | 9.930000e+02 | 5.523000e+03 | 456.19%| unknown
5.0s| 1 | 0 | 1530 | - | 17M | 0 |1861 | 130 | 151 | 25 | 7 | 3 | 0 | 9.930000e+02 | 5.523000e+03 | 456.19%| unknown
5.1s| 1 | 0 | 1558 | - | 17M | 0 |1861 | 130 | 153 | 27 | 8 | 3 | 0 | 9.957500e+02 | 5.523000e+03 | 454.66%| unknown
5.1s| 1 | 0 | 1559 | - | 17M | 0 |1861 | 130 | 153 | 27 | 8 | 3 | 0 | 9.960000e+02 | 5.523000e+03 | 454.52%| unknown
5.2s| 1 | 0 | 1680 | - | 17M | 0 |1861 | 130 | 160 | 34 | 9 | 3 | 0 | 1.019750e+03 | 5.523000e+03 | 441.60%| unknown
time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl.
5.2s| 1 | 0 | 1681 | - | 17M | 0 |1861 | 130 | 160 | 34 | 9 | 3 | 0 | 1.020000e+03 | 5.523000e+03 | 441.47%| unknown
5.4s| 1 | 0 | 1795 | - | 17M | 0 |1861 | 130 | 165 | 39 | 10 | 3 | 0 | 1.040500e+03 | 5.523000e+03 | 430.80%| unknown
5.4s| 1 | 0 | 1796 | - | 17M | 0 |1861 | 130 | 165 | 39 | 10 | 3 | 0 | 1.041000e+03 | 5.523000e+03 | 430.55%| unknown
5.5s| 1 | 0 | 1822 | - | 17M | 0 |1861 | 130 | 170 | 44 | 11 | 3 | 0 | 1.041000e+03 | 5.523000e+03 | 430.55%| unknown
5.6s| 1 | 0 | 1859 | - | 18M | 0 |1861 | 130 | 162 | 48 | 12 | 3 | 0 | 1.041000e+03 | 5.523000e+03 | 430.55%| unknown
5.7s| 1 | 0 | 1880 | - | 18M | 0 |1861 | 130 | 172 | 58 | 13 | 3 | 0 | 1.041000e+03 | 5.523000e+03 | 430.55%| unknown
5.8s| 1 | 0 | 1917 | - | 18M | 0 |1861 | 130 | 177 | 63 | 14 | 3 | 0 | 1.044500e+03 | 5.523000e+03 | 428.77%| unknown
5.8s| 1 | 0 | 1918 | - | 18M | 0 |1861 | 130 | 177 | 63 | 14 | 3 | 0 | 1.045000e+03 | 5.523000e+03 | 428.52%| unknown
5.9s| 1 | 0 | 1993 | - | 18M | 0 |1861 | 130 | 182 | 68 | 15 | 3 | 0 | 1.047643e+03 | 5.523000e+03 | 427.18%| unknown
5.9s| 1 | 0 | 1994 | - | 18M | 0 |1861 | 130 | 182 | 68 | 15 | 3 | 0 | 1.048000e+03 | 5.523000e+03 | 427.00%| unknown
6.0s| 1 | 0 | 2022 | - | 18M | 0 |1861 | 130 | 171 | 70 | 16 | 3 | 0 | 1.048750e+03 | 5.523000e+03 | 426.63%| unknown
6.0s| 1 | 0 | 2023 | - | 18M | 0 |1861 | 130 | 171 | 70 | 16 | 3 | 0 | 1.049000e+03 | 5.523000e+03 | 426.50%| unknown
6.1s| 1 | 0 | 2106 | - | 18M | 0 |1861 | 130 | 176 | 75 | 17 | 3 | 0 | 1.052250e+03 | 5.523000e+03 | 424.88%| unknown
6.1s| 1 | 0 | 2107 | - | 18M | 0 |1861 | 130 | 176 | 75 | 17 | 3 | 0 | 1.053000e+03 | 5.523000e+03 | 424.50%| unknown
6.3s| 1 | 0 | 2148 | - | 19M | 0 |1861 | 130 | 178 | 77 | 18 | 3 | 0 | 1.053375e+03 | 5.523000e+03 | 424.31%| unknown
time | node | left |LP iter|LP it/n|mem/heur|mdpt |vars |cons |rows |cuts |sepa|confs|strbr| dualbound | primalbound | gap | compl.
6.3s| 1 | 0 | 2149 | - | 19M | 0 |1861 | 130 | 178 | 77 | 18 | 3 | 0 | 1.054000e+03 | 5.523000e+03 | 424.00%| unknown
6.4s| 1 | 0 | 2210 | - | 20M | 0 |1861 | 130 | 162 | 81 | 19 | 3 | 0 | 1.054167e+03 | 5.523000e+03 | 423.92%| unknown
6.5s| 1 | 0 | 2211 | - | 20M | 0 |1861 | 130 | 162 | 81 | 19 | 3 | 0 | 1.055000e+03 | 5.523000e+03 | 423.51%| unknown
6.6s| 1 | 0 | 2269 | - | 20M | 0 |1861 | 130 | 165 | 84 | 20 | 3 | 0 | 1.058111e+03 | 5.523000e+03 | 421.97%| unknown
6.6s| 1 | 0 | 2270 | - | 20M | 0 |1861 | 130 | 165 | 84 | 20 | 3 | 0 | 1.059000e+03 | 5.523000e+03 | 421.53%| unknown
6.7s| 1 | 0 | 2276 | - | 20M | 0 |1861 | 130 | 166 | 85 | 21 | 3 | 0 | 1.059000e+03 | 5.523000e+03 | 421.53%| unknown
8.3s| 1 | 2 | 2700 | - | 21M | 0 |1861 | 136 | 166 | 85 | 23 | 9 | 22 | 1.066655e+03 | 5.523000e+03 | 417.79%| unknown
*14.4s| 30 | 25 | 4452 | 75.0 | LP | 7 |1861 | 136 | 138 | 125 | 0 | 9 | 167 | 1.067000e+03 | 5.459000e+03 | 411.62%| 0.86%
29.9s| 100 | 91 | 6915 | 46.9 | 23M | 17 |1861 | 147 | 145 | 288 | 2 | 20 | 654 | 1.067000e+03 | 5.459000e+03 | 411.62%| 1.46%
*33.4s| 131 | 100 | 7858 | 42.9 |strongbr| 18 |1861 | 153 | 144 | 349 | 1 | 26 | 788 | 1.067000e+03 | 1.530000e+03 | 43.39%| 1.53%
42.5s| 200 | 155 | 9970 | 38.7 | 24M | 18 |1861 | 170 | 142 | 455 | 2 | 43 |1097 | 1.067000e+03 | 1.530000e+03 | 43.39%| 2.17%
51.6s| 300 | 237 | 13159 | 36.4 | 26M | 18 |1861 | 224 | 139 | 640 | 2 | 97 |1277 | 1.067000e+03 | 1.530000e+03 | 43.39%| 2.33%
57.4s| 400 | 309 | 15846 | 34.0 | 27M | 19 |1861 | 238 | 152 | 820 | 2 | 113 |1426 | 1.067000e+03 | 1.530000e+03 | 43.39%| 3.10%
63.4s| 500 | 391 | 19168 | 33.9 | 29M | 24 |1861 | 319 | 145 | 926 | 2 | 195 |1560 | 1.067000e+03 | 1.530000e+03 | 43.39%| 3.22%
L68.4s| 584 | 111 | 22609 | 34.9 | rins| 26 |1861 | 330 | 149 |1016 | 1 | 256 |1619 | 1.067000e+03 | 1.127000e+03 | 5.62%| 10.59%
I also used display display to get some idea about what the column headers in the display output mean.
cons is just the globally valid constraints in the problem
rows is number of LP rows in current node
cuts is total number of cuts applied to the LPs
sepa is number of separation rounds performed at the current node
So I have a a few questions about this. I assumed rows should be 0 when the problem starts but I am not sure why there at 126 rows at the beginning of the problem. I add LP rows using the following function:
SCIP_ROW *row;
SCIP_CALL(SCIPcreateEmptyRowConshdlr(scip, &row, conshdlr, "subtour_elimination", -SCIPinfinity(scip), tour.size(), FALSE, FALSE, TRUE));
Are cuts added by SCIP automatically?
How does SCIP add globally valid constraints to the constraints pool? Does it take from the rows or from the cuts or from both? Is there a way to add globally valid constraints directly from the constraint handler?
The TSP example is one of the most popular examples that the SCIP team uses to highlight SCIP's constraint capabilities. You can find the mathematical model of this example in the slides of this introduction to SCIP, Section Constraint Integer Programming.
The LP relaxation initially consists of the so-called node-degree constraints, which require that each node is adjacent to exactly two edges in a solution. The subtour elimination constraint is present as a single constraint without row in the LP, but only adds actual linear rows to the LP relaxation if necessary.
Other general-purpose separators such as Gomory or CMIR cuts are active throughout the search. You can use display statistics and browse the "Separators" section to learn which cutting plane methods were enabled.
In order to inspect the initial LP relaxation, you may use the various writing possibilities that SCIP has to dump the current model into an LP file.
You can, for example, stop SCIP after the initial root LP relaxation (just press CTRL + C timely) and use
write lp tsp.lp for the LP relaxation
write mip tsp.lp for the MIP relaxation
write transproblem tsp.cip for the current transformed problem
The first two methods are restricted to the LP format, which is nicely readable. Only the last format, SCIP's own CIP format, will print the subtour-elimination constraints, as well. Rows/Cuts that were added to the current LP relaxation may not be part of the transformed problem.
tldr; choose wisely in which format you print the relaxation :)
I am working on an sql query to calculate MTBF
I have following set of data
+-----+-------------------------+------+
| ID | DateTime | Sec |
+-----+-------------------------+------+
| 101 | 2019-07-22 09:10:10.000 | 900 |
| 100 | 2019-07-22 08:45:00.000 | 900 |
| 99 | 2019-07-22 08:30:00.000 | 800 |
| 98 | 2019-07-22 08:15:00.000 | 800 |
| 97 | 2019-07-22 07:10:10.000 | 600 |
| 96 | 2019-07-22 06:50:00.000 | 600 |
| 95 | 2019-07-22 06:40:00.000 | 400 |
| 94 | 2019-07-21 15:40:00.000 | 720 |
| 93 | 2019-07-21 13:25:00.000 | 400 |
| 92 | 2019-07-21 10:43:10.000 | 900 |
| 91 | 2019-07-20 07:30:00.000 | 800 |
| 90 | 2019-07-19 20:40:10.000 | 900 |
| 89 | 2019-07-19 18:30:30.000 | 700 |
| 88 | 2019-07-19 17:50:00.000 | 400 |
| 87 | 2019-07-19 16:40:00.000 | 400 |
| 86 | 2019-07-19 15:20:25.000 | 1000 |
| 85 | 2019-07-19 14:50:20.000 | 900 |
| 84 | 2019-07-19 14:30:00.000 | 8000 |
| 83 | 2019-07-19 14:10:10.000 | 600 |
| 82 | 2019-07-19 13:59:00.000 | 200 |
| 81 | 2019-07-19 13:50:40.000 | 300 |
| 80 | 2019-07-19 13:40:00.000 | 400 |
+-----+-------------------------+------+
I want to calculate the difference between the ID 101 and 100, and than between 100 and 99 and so on.
But here is difficult part
I don't want to calculate the difference between for weekend dates in this case for a date 20-07-2019 and 21-07-2019.
I always want to calculate the difference only for week days.
so for given Sample data the output has to be following.
+-----+-------------------------+------+---------+
| ID | DateTime | Sec | Diff |
+-----+-------------------------+------+---------+
| 101 | 2019-07-22 09:10:10.000 | 900 | Null |
| 100 | 2019-07-22 08:45:00.000 | 900 | 0:25:10 |
| 99 | 2019-07-22 08:30:00.000 | 800 | 0:15:00 |
| 98 | 2019-07-22 08:15:00.000 | 800 | 0:15:00 |
| 97 | 2019-07-22 07:10:10.000 | 600 | 1:04:50 |
| 96 | 2019-07-22 06:50:00.000 | 600 | 0:20:10 |
| 95 | 2019-07-22 06:40:00.000 | 400 | 0:10:00 |
| 94 | 2019-07-21 15:40:00.000 | 720 | Null |
| 93 | 2019-07-21 13:25:00.000 | 400 | Null |
| 92 | 2019-07-21 10:43:10.000 | 900 | Null |
| 91 | 2019-07-20 07:30:00.000 | 800 | Null |
| 90 | 2019-07-19 20:40:10.000 | 900 | Null |
| 89 | 2019-07-19 18:30:30.000 | 700 | 2:09:40 |
| 88 | 2019-07-19 17:50:00.000 | 400 | 0:40:30 |
| 87 | 2019-07-19 16:40:00.000 | 400 | 1:10:00 |
| 86 | 2019-07-19 15:20:25.000 | 1000 | 1:19:35 |
| 85 | 2019-07-19 14:50:20.000 | 900 | 0:30:05 |
| 84 | 2019-07-19 14:30:00.000 | 8000 | 0:20:20 |
| 83 | 2019-07-19 14:10:10.000 | 600 | 0:19:50 |
| 82 | 2019-07-19 13:59:00.000 | 200 | 0:11:10 |
| 81 | 2019-07-19 13:50:40.000 | 300 | 0:08:20 |
| 80 | 2019-07-19 13:40:00.000 | 400 | 0:10:40 |
+-----+-------------------------+------+---------+
after that i wan to sum all the difference and divide by number (count) of id in week days.
Below is the query i have tried until now
SELECT *, DATEDIFF( SECOND, DateTime, LEAD(DateTime) OVER (ORDER BY [ID])) AS [diff] FROM [Stoerungen] where [DateTime] between '20190719 00:00:00.000' and '20190722 23:59:00.000' and ((DATEPART(dw, DateTime) + ##DATEFIRST) % 7) NOT IN (0, 1) order by [ID] OFFSET 0 ROWS
I am able to exclude the weekend data but this query makes a difference from last Friday to next Monday so I have wrong data.
As you don't want exclude non-week days but only set Diff to null, move this condition to CASE expression
SELECT *
, (case When (((DATEPART(dw, DateTime) + ##DATEFIRST) % 7) NOT IN (0, 1))
then DATEDIFF( SECOND, DateTime, LEAD(DateTime) OVER (ORDER BY [ID]))
else null
end) AS [diff]
FROM [Stoerungen]
WHERE [DateTime] between '20190719 00:00:00.000' and '20190722 23:59:00.000'
ORDER BY [ID]
OFFSET 0 ROWS
I am new to SQL-Server and want to generate query from two tables - Bill(master table) and BillDetail(detail table). These two table data are given below :
Bill Table Data :
+------------------+--------------+---------------+------------+----------+-------------------------+
| ContractorBillId | ContractorId | BillNo | BillDate |IsDeleted | UpdatedDate |
+------------------+--------------+---------------+------------+----------+-------------------------+
| 1050 | 2 | 2-20190329-W | 2019-03-29 | 0 | 2019-03-29 13:53:51.447 |
| 1051 | 2 | 2-20190329-W1 | 2019-03-29 | 0 | 2019-03-29 18:48:48.077 |
| 1052 | 2 | 2-20190402-W | 2019-04-02 | 0 | 2019-04-02 15:54:16.267 |
| 1053 | 1 | 1-20190402-E | 2019-04-02 | 0 | 2019-04-02 18:58:50.753 |
| 1078 | 1 | 1-20190403-A | 2019-04-03 | 0 | 2019-04-03 10:59:18.083 |
| 1079 | 1 | 1-20190403-A1 | 2019-04-03 | 0 | 2019-04-03 11:00:37.197 |
| 1080 | 1 | 1-20190403-E | 2019-04-03 | 0 | 2019-04-03 11:33:41.333 |
+------------------+--------------+---------------+------------+----------+-------------------------+
BillDetail Table Data :
+----------------------+------------------+---------------------+------------+--------+-------------+
| ContractorBillItemId | ContractorBillId | ContractorPayTypeId | WeekEnding | Hours | GrossAmount |
+----------------------+------------------+---------------------+------------+--------+-------------+
| 1064 | 1050 | 2 | 2019-03-29 | 145.00 | 725000.0000 |
| 1065 | 1051 | 1 | 2019-03-29 | 50.00 | 75000.0000 |
| 1066 | 1052 | 1 | 2019-04-05 | 10.00 | 15000.0000 |
| 1067 | 1052 | 2 | 2019-04-05 | 12.00 | 60000.0000 |
| 1068 | 1053 | 4 | 2019-04-02 | 1.00 | 100.0000 |
| 1069 | 1053 | 3 | 2019-04-03 | 1.00 | 100.0000 |
| 1070 | 1053 | 2 | 2019-04-04 | 1.00 | 100.0000 |
| 1071 | 1053 | 4 | 2019-04-05 | 1.00 | 100.0000 |
| 1072 | 1053 | 4 | 2019-04-06 | 1.00 | 100.0000 |
| 1089 | 1078 | 1006 | 2019-04-05 | 1.00 | 100.0000 |
| 1090 | 1079 | 1006 | 2019-04-05 | 12.00 | 1200.0000 |
| 1091 | 1080 | 4 | 2019-04-02 | 1.00 | 100.0000 |
| 1092 | 1080 | 4 | 2019-04-03 | 1.00 | 100.0000 |
| 1093 | 1080 | 4 | 2019-04-04 | 1.00 | 100.0000 |
| 1094 | 1080 | 4 | 2019-04-05 | 1.00 | 100.0000 |
| 1095 | 1080 | 4 | 2019-04-06 | 1.00 | 100.0000 |
+----------------------+------------------+---------------------+------------+--------+-------------+
From these two table, I want result like below
+------------------+--------------+---------------+------------+----------+-------------------------+--------------------+------------+--------+
| ContractorBillId | ContractorId | BillNo | BillDate |IsDeleted | UpdatedDate | ContractorPayTypeId| WeekEnding | Hours |
+------------------+--------------+---------------+------------+----------+-------------------------+--------------------+------------+--------+
| 1050 | 2 | 2-20190329-W | 2019-03-29 | 0 | 2019-03-29 13:53:51.447 | 2 | 2019-03-29 | 145.00 |
| 1051 | 2 | 2-20190329-W1 | 2019-03-29 | 0 | 2019-03-29 18:48:48.077 | 1 | 2019-03-29 | 50.00 |
| 1052 | 2 | 2-20190402-W | 2019-04-02 | 0 | 2019-04-02 15:54:16.267 | 1 | 2019-04-05 | 22.00 |
| 1053 | 1 | 1-20190402-E | 2019-04-02 | 0 | 2019-04-02 18:58:50.753 | 4 | 2019-04-02 | 5.00 |
| 1078 | 1 | 1-20190403-A | 2019-04-03 | 0 | 2019-04-03 10:59:18.083 | 1006 | 2019-04-05 | 1.00 |
| 1079 | 1 | 1-20190403-A1 | 2019-04-03 | 0 | 2019-04-03 11:00:37.197 | 1006 | 2019-04-05 | 12.00 |
| 1080 | 1 | 1-20190403-E | 2019-04-03 | 0 | 2019-04-03 11:33:41.333 | 4 | 2019-04-02 | 5.00 |
+------------------+--------------+---------------+------------+----------+-------------------------+--------------------+------------+--------+
Please help me with generating query for it. In result, From detail table Hours and GrossAmount should be sum and ContractorPayTypeId and WeekEnding should be first of all item.
You can use advanced SQL queries such as SUM() OVER PARTITION BY, FIRST_VALUE() ETC.
SELECT BillTable.* ,
CS.SumGrossAmount ,
CS.SumHours ,
CS.FirstWeekEnding ,
CS.FirstContractorPayTypeId
FROM dbo.BillTable
CROSS APPLY
( SELECT TOP 1 S.SumGrossAmount ,
S.SumHours ,
S.FirstWeekEnding ,
S.FirstContractorPayTypeId
FROM ( SELECT * ,
SUM(GrossAmount) OVER ( PARTITION BY ContractorBillId
ORDER BY ContractorBillItemId
) AS SumGrossAmount ,
SUM(Hours) OVER ( PARTITION BY ContractorBillId
ORDER BY ContractorBillItemId
) AS [SumHours] ,
FIRST_VALUE(WeekEnding) OVER ( PARTITION BY ContractorBillId
ORDER BY ContractorBillItemId
) AS [FirstWeekEnding] ,
FIRST_VALUE(ContractorPayTypeId) OVER ( PARTITION BY ContractorBillId
ORDER BY ContractorBillItemId
) AS [FirstContractorPayTypeId]
FROM dbo.BillDetail
) S
WHERE S.ContractorBillId = dbo.BillTable.ContractorBillId
ORDER BY S.ContractorBillItemId DESC
) CS;
I am working with data where I need to use a calculation that is supposed to reference a previously calculated value in the previous row.
For example, take this dataset:
SELECT
generate_series('2015-01-01', '2019-12-01', '1 month'::interval)::date AS dates,
generate_series(1,60) AS nums;
There's are NULL values starting at 2019-03-01.
I'd like to write a calculation on another column that fills it in based off the previous row, that is derived from that same calculation. So I tried to use some lag() functions. But after a while it turns to NULL, probably because the previous calculation is also null.
with
mynumbers AS (
SELECT
generate_series('2015-01-01', '2025-12-01', '1 month'::interval)::date AS dates,
generate_series(1,50) AS nums),
mynumbers_lag AS (
SELECT *, lag(nums) OVER (ORDER BY dates ASC) AS previous1
FROM mynumbers)
SELECT dates, nums, previous1, (coalesce(nums,previous1)+lag(coalesce(nums,previous1), 5) OVER (ORDER BY dates ASC))*4 AS moving_calculation FROM mynumbers_lag;
The result starts to deviate from what I'd like it to be at 2019-03-01. I'd like my calculation to continue all the way through the table. Anyone know how I can accomplish this?
Edit: borrowing unutbu's table.. I want to yield this:
| dates | nums | previous1 | moving_calculation |
|------------+------+-----------+--------------------|
| 2015-01-01 | 1 | | |
| 2015-02-01 | 2 | 1 | |
| 2015-03-01 | 3 | 2 | |
| 2015-04-01 | 4 | 3 | |
| 2015-05-01 | 5 | 4 | |
| 2015-06-01 | 6 | 5 | 28 |
| 2015-07-01 | 7 | 6 | 36 |
| 2015-08-01 | 8 | 7 | 44 |
| 2015-09-01 | 9 | 8 | 52 |
| 2015-10-01 | 10 | 9 | 60 |
...
| 2018-12-01 | 50 | 49 | 364 |
| 2019-01-01 | 50 | 49 | 372 |
| 2019-02-01 | 50 | 49 | 380 |
| 2019-03-01 | 50 | 49 | 388 |
| 2019-04-01 | 50 | 49 | 1744 |
| 2019-05-01 | 50 | 49 | 7172 |
| 2019-06-01 | | | 28888 |
| 2019-07-01 | | | 117104 |
| 2019-08-01 | | | 475392 |
| 2019-09-01 | | | 1930256 |
On 2019-04-01, the 1744 is calculated from (388+48)*4. The 388 is one
cell from the previously calculated value because nums is NULL. Eventually,
starting on 2018-07-01, both nums are NULL, so it will calculating using
only from moving_calculations (values 380 and 7172)
The values in the moving_calculation column (denoted m0 below) depend on
prior values in the same column. They are defined by a recurrence
relation. There might even be a closed-form formula for m0. You might want to
ask a question on Mathematics stackexchange if you wish to find a closed-form
formula. If we knew the closed-form formula, clearly computing values in
Postgresql would be a breeze.
However, if we regard this problem as a programming problem, then I believe the calculation -- if it is to be done in Postgresql -- is most easily expressed using WITH RECURSIVE.
The calculation sort of feels like the calculation of Fibonacci numbers.
WITH RECURSIVE r(a, b) AS (
SELECT 0::int, 1::int
UNION ALL
SELECT b, a + b FROM r WHERE b < 50
)
SELECT a, b FROM r;
yields
| a | b |
|----+----|
| 0 | 1 |
| 1 | 1 |
| 1 | 2 |
| 2 | 3 |
| 3 | 5 |
| 5 | 8 |
| 8 | 13 |
| 13 | 21 |
| 21 | 34 |
| 34 | 55 |
If you understand the use of WITH RECURSIVE in that Fibonacci example, then I believe you'll see the solution below
is merely an extension of the same idea.
WITH RECURSIVE r(dates, nums, prev, m0, m1, m2, m3) AS (
SELECT * FROM (VALUES ('2019-02-01'::date, 50::numeric, 47::numeric, 388::numeric, NULL::numeric, NULL::numeric, NULL::numeric)) AS t1
UNION ALL
SELECT (dates + '1 month'::interval)::date
, m0
, coalesce(m3, prev+1)
, (m0+coalesce(m3, prev+1))*4
, m0
, m1
, m2
FROM r
WHERE dates <= '2020-01-01'
)
SELECT * FROM r
yields
| dates | nums | prev | m0 | m1 | m2 | m3 |
|------------+------------+----------+------------+------------+-----------+-----------|
| 2019-02-01 | 50 | 47 | 388 | | | |
| 2019-03-01 | 388 | 48 | 1744 | 388 | | |
| 2019-04-01 | 1744 | 49 | 7172 | 1744 | 388 | |
| 2019-05-01 | 7172 | 50 | 28888 | 7172 | 1744 | 388 |
| 2019-06-01 | 28888 | 388 | 117104 | 28888 | 7172 | 1744 |
| 2019-07-01 | 117104 | 1744 | 475392 | 117104 | 28888 | 7172 |
| 2019-08-01 | 475392 | 7172 | 1930256 | 475392 | 117104 | 28888 |
| 2019-09-01 | 1930256 | 28888 | 7836576 | 1930256 | 475392 | 117104 |
| 2019-10-01 | 7836576 | 117104 | 31814720 | 7836576 | 1930256 | 475392 |
| 2019-11-01 | 31814720 | 475392 | 129160448 | 31814720 | 7836576 | 1930256 |
| 2019-12-01 | 129160448 | 1930256 | 524362816 | 129160448 | 31814720 | 7836576 |
| 2020-01-01 | 524362816 | 7836576 | 2128797568 | 524362816 | 129160448 | 31814720 |
| 2020-02-01 | 2128797568 | 31814720 | 8642449152 | 2128797568 | 524362816 | 129160448 |
To combine this table with the original table, use UNION:
WITH mytable AS (
SELECT *, (nums+prev)*4 AS m0, NULL::numeric AS m1, NULL::numeric AS m2, NULL::numeric AS m3
FROM (
SELECT *
, lag(nums, 3) OVER (ORDER BY dates ASC) AS prev
FROM (
SELECT
generate_series('2015-01-01', '2025-12-01', '1 month'::interval)::date AS dates,
generate_series(1,50)::numeric AS nums) t
) t2
WHERE nums IS NOT NULL
), last_row AS (
SELECT * FROM mytable
WHERE nums IS NOT NULL
ORDER BY dates DESC
LIMIT 1
)
SELECT * FROM mytable
UNION (
WITH RECURSIVE r(dates, nums, prev, m0, m1, m2, m3) AS (
SELECT * FROM last_row
UNION ALL
SELECT (dates + '1 month'::interval)::date
, m0
, coalesce(m3, prev+1)
, (m0+coalesce(m3, prev+1))*4
, m0
, m1
, m2
FROM r
WHERE dates <= '2020-01-01')
SELECT * FROM r)
ORDER BY dates
which yields
| dates | nums | prev | m0 | m1 | m2 | m3 |
|------------+------------+----------+------------+------------+-----------+-----------|
| 2015-01-01 | 1 | | | | | |
| 2015-02-01 | 2 | | | | | |
| 2015-03-01 | 3 | | | | | |
| 2015-04-01 | 4 | 1 | 20 | | | |
| 2015-05-01 | 5 | 2 | 28 | | | |
| 2015-06-01 | 6 | 3 | 36 | | | |
| 2015-07-01 | 7 | 4 | 44 | | | |
| 2015-08-01 | 8 | 5 | 52 | | | |
| 2015-09-01 | 9 | 6 | 60 | | | |
| 2015-10-01 | 10 | 7 | 68 | | | |
| 2015-11-01 | 11 | 8 | 76 | | | |
| 2015-12-01 | 12 | 9 | 84 | | | |
| 2016-01-01 | 13 | 10 | 92 | | | |
| 2016-02-01 | 14 | 11 | 100 | | | |
| 2016-03-01 | 15 | 12 | 108 | | | |
| 2016-04-01 | 16 | 13 | 116 | | | |
| 2016-05-01 | 17 | 14 | 124 | | | |
| 2016-06-01 | 18 | 15 | 132 | | | |
| 2016-07-01 | 19 | 16 | 140 | | | |
| 2016-08-01 | 20 | 17 | 148 | | | |
| 2016-09-01 | 21 | 18 | 156 | | | |
| 2016-10-01 | 22 | 19 | 164 | | | |
| 2016-11-01 | 23 | 20 | 172 | | | |
| 2016-12-01 | 24 | 21 | 180 | | | |
| 2017-01-01 | 25 | 22 | 188 | | | |
| 2017-02-01 | 26 | 23 | 196 | | | |
| 2017-03-01 | 27 | 24 | 204 | | | |
| 2017-04-01 | 28 | 25 | 212 | | | |
| 2017-05-01 | 29 | 26 | 220 | | | |
| 2017-06-01 | 30 | 27 | 228 | | | |
| 2017-07-01 | 31 | 28 | 236 | | | |
| 2017-08-01 | 32 | 29 | 244 | | | |
| 2017-09-01 | 33 | 30 | 252 | | | |
| 2017-10-01 | 34 | 31 | 260 | | | |
| 2017-11-01 | 35 | 32 | 268 | | | |
| 2017-12-01 | 36 | 33 | 276 | | | |
| 2018-01-01 | 37 | 34 | 284 | | | |
| 2018-02-01 | 38 | 35 | 292 | | | |
| 2018-03-01 | 39 | 36 | 300 | | | |
| 2018-04-01 | 40 | 37 | 308 | | | |
| 2018-05-01 | 41 | 38 | 316 | | | |
| 2018-06-01 | 42 | 39 | 324 | | | |
| 2018-07-01 | 43 | 40 | 332 | | | |
| 2018-08-01 | 44 | 41 | 340 | | | |
| 2018-09-01 | 45 | 42 | 348 | | | |
| 2018-10-01 | 46 | 43 | 356 | | | |
| 2018-11-01 | 47 | 44 | 364 | | | |
| 2018-12-01 | 48 | 45 | 372 | | | |
| 2019-01-01 | 49 | 46 | 380 | | | |
| 2019-02-01 | 50 | 47 | 388 | | | |
| 2019-03-01 | 388 | 48 | 1744 | 388 | | |
| 2019-04-01 | 1744 | 49 | 7172 | 1744 | 388 | |
| 2019-05-01 | 7172 | 50 | 28888 | 7172 | 1744 | 388 |
| 2019-06-01 | 28888 | 388 | 117104 | 28888 | 7172 | 1744 |
| 2019-07-01 | 117104 | 1744 | 475392 | 117104 | 28888 | 7172 |
| 2019-08-01 | 475392 | 7172 | 1930256 | 475392 | 117104 | 28888 |
| 2019-09-01 | 1930256 | 28888 | 7836576 | 1930256 | 475392 | 117104 |
| 2019-10-01 | 7836576 | 117104 | 31814720 | 7836576 | 1930256 | 475392 |
| 2019-11-01 | 31814720 | 475392 | 129160448 | 31814720 | 7836576 | 1930256 |
| 2019-12-01 | 129160448 | 1930256 | 524362816 | 129160448 | 31814720 | 7836576 |
| 2020-01-01 | 524362816 | 7836576 | 2128797568 | 524362816 | 129160448 | 31814720 |
| 2020-02-01 | 2128797568 | 31814720 | 8642449152 | 2128797568 | 524362816 | 129160448 |
input:
| a.user_id | a_stream_length | b_stream_length | subtract_inactive |
-----------------------------------------------------------------------------
| a | 11 | 1686 | 22 |
| a | 1686 | 328 | 12 |
| a | 328 | 732 | 22 |
| a | 732 | 11 | 1699 |
| a | 11 | 2123 | 18 |
| a | 2123 | 160 | 2 |
| a | 160 | 1358 | 0 |
| a | 1358 | 129 | 1 |
| a | 129 | 4042 | 109334 |
output:
| a | (1686+11+328+732) (if subtract_inactive < 1000) |
| a | 732(a_stream_length) if subtract_inactive > 1000) |