Qunatify total time saved by prioritizing tasks based on the failure rate probability of each task - optimization

Question

I am trying to solve a problem where I am trying to prioritize the tasks in a job based on the failure rates of each task. For ex:
Task p(Failure) TimeTaken(sec)
A 0.7 10
B 0.1 15
C 0.5 3
D 0.3 5
This is a sequence of tasks and if even one task fails, the entire job fails. So I want to prioritize my tasks to save the max amount of time. To do that I am trying to run the tasks in the order of failure probability. So my current order of performing the tasks is A,C,D and then B. I feel the problem with my approach is I am not considering the time factor. Is there a better way to prioritize my tasks based on the time taken also into consideration?

Answer 1

Well, your intuition is correct. The things you'd like to do first are things that have either a real high failure probability (so you don't waste a bunch of time and fail later) and things that are short duration so if you do fail, you haven't wasted time on a long task.
Here is a brute-force solution that looks at all possible sequences. It scales OK. There are probably more elegant solutions and maybe even some math model that doesn't come to mind too quickly.
Anyhow. Assumptions: All failures are independent of each other, failure occurs (or is recognized) at the end of the task, and that a failure occurs. We know from probability theory that if the tasks are independent, then the P{success} does not depend on the sequence of the tasks, so all sequences have the same likelihood of failure overall as well. Just that the lost time is different depending on the sequence and when in the sequence it occurs.
The code below calculates the expected value of x given that a failure occurs where x is the wasted time, and it is just the sum-product of the possible intermediate sequences times and probability of occurring.
# scheduling with failures
from itertools import permutations as perm
from math import prod
from tabulate import tabulate
tasks = { 'A': (0.7, 10), # (P{fail}, time)
'B': (0.1, 15),
'C': (0.5, 3),
'D': (0.3, 5)}
# let's start with finding P{success}
p_s = prod( (1-tasks[k][0]) for k in tasks)
success_time = sum(tasks[k][1] for k in tasks)
print(f'the probability the sequence completes successfully is: {p_s:0.3f}')
print(f'when successful, the time is: {success_time}')
min_E_x = success_time # upper bound on min_E_x: The minimum expected value of x
best = None
sequences, vals = [], []
for seq in perm(tasks,len(tasks)) : # all permutations
E_x = 0 # the expected value of x for this sequence, where x = time wasted
for i in range(len(seq)):
p = tasks[seq[i]][0] # p{fail on last}
earlier = prod((1-tasks[seq[j]][0]) for j in range(i)) # p{all earlier events pass}
if earlier:
p *= earlier
# get elapsed time for this sequence
time = sum(tasks[seq[j]][1] for j in range(i+1))
# normalize the probability (we know a failure has occurred)
p = p/(1-p_s)
E_x += p*time # E[x] = sum of all p*x
# print(seq[0:i+1], time, p, E_x)
sequences.append(seq)
vals.append(E_x)
if E_x < min_E_x:
best = seq
min_E_x = E_x
print(f'\nThe best selection with minimal wasted time given a failure is: {best} with E[wasted time]: {min_E_x:0.3f}\n')
print(tabulate(zip(sequences, vals), floatfmt=".2f", headers=['Sequence', 'E[x]|failure']))
Yields
the probability the sequence completes successfully is: 0.095
when successful, the time is: 33
The best selection with minimal wasted time given a failure is: ('C', 'A', 'D', 'B') with E[wasted time]: 7.959
Sequence E[x]|failure
-------------------- --------------
('A', 'B', 'C', 'D') 14.21
('A', 'B', 'D', 'C') 14.69
('A', 'C', 'B', 'D') 11.82
('A', 'C', 'D', 'B') 11.16
('A', 'D', 'B', 'C') 13.36
('A', 'D', 'C', 'B') 11.69
('B', 'A', 'C', 'D') 24.70
('B', 'A', 'D', 'C') 25.18
('B', 'C', 'A', 'D') 21.82
('B', 'C', 'D', 'A') 22.07
('B', 'D', 'A', 'C') 25.67
('B', 'D', 'C', 'A') 23.66
('C', 'A', 'B', 'D') 8.62
('C', 'A', 'D', 'B') 7.96
('C', 'B', 'A', 'D') 13.87
('C', 'B', 'D', 'A') 14.12
('C', 'D', 'A', 'B') 8.23
('C', 'D', 'B', 'A') 11.91
('D', 'A', 'B', 'C') 13.91
('D', 'A', 'C', 'B') 12.24
('D', 'B', 'A', 'C') 21.26
('D', 'B', 'C', 'A') 19.24
('D', 'C', 'A', 'B') 10.00
('D', 'C', 'B', 'A') 13.67
[Finished in 0.1s]