For the following problem I came up with the following algorithm. I just wondering whether I have calculated the complexity of the algorithm correctly or not.
Problem:
Given a list of integers as input, determine whether or not two integers (not necessarily distinct) in the list have a product k. For example, for k = 12 and list [2,10,5,3,7,4,8], there is a pair, 3 and 4, such that 3×4 = 12.
My solution:
// Imagine A is the list containing integer numbers
for(int i=0; i<A.size(); i++) O(n)
{
for(int j=i+1; j<A.size()-1; j++) O(n-1)*O(n-(i+1))
{
if(A.get(i) * A.get(j) == K) O(n-2)*O(n-(i+1))
return "Success"; O(1)
}
}
return "FAILURE"; O(1)
O(n) + O(n-1)*O(n-i-1) + O(n-2)*O(n-i-1)) + 2*O(1) =
O(n) + O(n^2-ni-n) + O(-n+i+1) + O(n^2-ni-n) + O(-2n+2i+2) + 2O(1) =
O(n) + O(n^2) + O(n) + O(n^2) + O(2n) + 2O(2) =
O(n^2)
Apart from my semi-algorithm, is there any more efficient algorithm?
Let's break down what your proposed algorithm is essentially doing.
For every index i (s.t 0 ≤ i ≤ n) you compare i to all unique indices j (i ≠ j) to determine whether: i * j == k.
An invariant for this algorithm would be that at every iteration, the pair {i,j} being compared hasn't been compared before.
This implementation (assuming it compiles and runs without the runtime exceptions mentioned in the comments) makes a total of nC2 comparisons (where nC2 is the binomial coefficient of n and 2, for choosing all possible unique pairs) and each such comparison would compute at a constant time (O(1)). Note it can be proven that nCk is not greater than n^k.
So O(nC2) makes for a more accurate upper bound for this algorithm - though by common big O notation this would still be O(n^2) since nC2 = n*(n-1)/2 = (n^2-n)/2 which is still order of n^2.
Per your question from the comments:
Is it correct to use "i" in the complexity, as I have used O(n-(i+1))?
i is a running index, whereas the complexity of your algorithm is only affected by the size of your sample, n.
IOW, the total complexity is calculated for all iterations in the algorithm, while i refers to a specific iteration. Therefore it is incorrect to use 'i' in your complexity calculations.
Apart from my semi-algorithm, is there any more efficient algorithm?
Your "semi-algorithm" seems to me the most efficient way to go about this. Any comparison-based algorithm would require querying all pairs in the array, which translates to the runtime complexity detailed above.
Though I have not calculated a lower bound and would be curious to hear if someone knows of a more efficient implementation.
edit: The other answer here shows a good solution to this problem which is (generally speaking) more efficient than this one.
Your algorithm looks like O(n^2) worst case and O(n*log(n)) average case, because the longer the list is, the more likely the loops will exit before evaluating all n^2 pairs.
An algorithm with O(n) worst case and O(log(n)) average case is possible. In real life it would be less efficient than your algorithm for lists where the factors of K are right at the start or the list is short, and more efficient otherwise. (pseudocode not written in any particular language)
var h = new HashSet();
for(int i=0; i<A.size(); i++)
{
var x = A.get(i);
if(x%K == 0) // If x is a factor of K
{
h.add(x); // Store x in h
if(h.contains(K/x))
{
return "Success";
}
}
}
return "FAILURE";
HashSet.add and HashSet.contains are O(1) on average (but slower than List.get even though it is also O(1)). For the purpose of this exercise I am assuming they always run in O(1) (which is not strictly true but close enough for government work). I have not accounted for edge cases, such as the list containing a 0.
If I have a algorithm where a part of it has complexity big-O(nlogn) and part of it has complexity big-O(n). What would the final complexity of the algorithm be? As far as I am aware, it would be big-O(nlogn).
You are right, the worst case possible is what counts,
in your case o(nlog(n)).
It depends on what you mean by "part of it" ...
Let's assume you have a for loop that has a complexity of O(n) and a binary search for O(logn)
If you program look like this:
for(int i=0; i < n; i++) { // O(n)
/// some stuff here
}
binarySearch(); // O(logn)
Time Complexity would be O(n)
However if have this situation:
for(int i=0; i < n; i++){ // O(n)
binarySearch(); // O(n * logn)
}
Time Complexity would be O(nlogn)
Edit:
If the algorithm is composed of different blocks with different time complexities, then algorithm time complexity = max(O(block1), O(block2), ...)
How does one calculate the time complexity with conditional statements that may or may not lead to higher oder results?
For example:
for(int i = 0; i < n; i++){
//an elementary operation
for(int j = 0; j < n; j++){
//another elementary operation
if (i == j){
for(int k = 0; k < n; k++){
//yet another elementary operation
}
} else {
//elementary operation
}
}
}
And what if the contents in the if-else condition were reversed?
Your code takes O(n^2). First two loops take O(n^2) operations. The "k" loop takes O(n) operations and gets called n times. It gives O(n^2). The total complexity of your code will be O(n^2) + O(n^2) = O(n^2).
Another try:
- First 'i' loop runs n times.
- Second 'j' loop runs n times. For each of is and js (there are n^2 combinations):
- if i == j make n combinations. There are n possibilities that i==j,
so this part of code runs O(n^2).
- if it's not, it makes elementary operation. There are n^2 - n combinations like that
so it will take O(n^2) time.
- The above proves, that this code will take O(n) operations.
That depends on the kind of analysis you are performing. If you are analysing worst-case complexity, then take the worst complexity of both branches. If you're analysing average-case complexity, you need to calculate the probability of entering one branch or another and multiply each complexity by the probability of taking that path.
If you change the branches, just switch the probability coefficients.
Let's say I have a function f defined on interval [0,1], which is smooth and increases up to some point a after which it starts decreasing. I have a grid x[i] on this interval, e.g. with a constant step size of dx = 0.01, and I would like to find which of those points has the highest value, by doing the smallest number of evaluations of f in the worst-case scenario. I think I can do much better than exhaustive search by applying something inspired with gradient-like methods. Any ideas? I was thinking of something like a binary search perhaps, or parabolic methods.
This is a bisection-like method I coded:
def optimize(f, a, b, fa, fb, dx):
if b - a <= dx:
return a if fa > fb else b
else:
m1 = 0.5*(a + b)
m1 = _round(m1, a, dx)
fm1 = fa if m1 == a else f(m1)
m2 = m1 + dx
fm2 = fb if m2 == b else f(m2)
if fm2 >= fm1:
return optimize(f, m2, b, fm2, fb, dx)
else:
return optimize(f, a, m1, fa, fm1, dx)
def _round(x, a, dx, right = False):
return a + dx*(floor((x - a)/dx) + right)
The idea is: find the middle of the interval and compute m1 and m2- the points to the right and to the left of it. If the direction there is increasing, go for the right interval and do the same, otherwise go for the left. Whenever the interval is too small, just compare the numbers on the ends. However, this algorithm still does not use the strength of the derivatives at points I computed.
Such a function is called unimodal.
Without computing the derivatives, you can work by
finding where the deltas x[i+1]-x[i] change sign, by dichotomy (the deltas are positive then negative after the maximum); this takes Log2(n) comparisons; this approach is very close to what you describe;
adapting the Golden section method to the discrete case; it takes Logφ(n) comparisons (φ~1.618).
Apparently, the Golden section is more costly, as φ<2, but actually the dichotomic search takes two function evaluations at a time, hence 2Log2(n)=Log√2(n) .
One can show that this is optimal, i.e. you can't go faster than O(Log(n)) for an arbitrary unimodal function.
If your function is very regular, the deltas will vary smoothly. You can think of the interpolation search, which tries to better predict the searched position by a linear interpolation rather than simple halving. In favorable conditions, it can reach O(Log(Log(n)) performance. I don't know of an adaptation of this principle to the Golden search.
Actually, linear interpolation on the deltas is very close to parabolic interpolation on the function values. The latter approach might be the best for you, but you need to be careful about the corner cases.
If derivatives are allowed, you can use any root solving method on the first derivative, knowing that there is an isolated zero in the given interval.
If only the first derivative is available, use regula falsi. If the second derivative is possible as well, you may consider Newton, but prefer a safe bracketing method.
I guess that the benefits of these approaches (superlinear and quadratic convergence) are made a little useless by the fact that you are working on a grid.
DISCLAIMER: Haven't test the code. Take this as an "inspiration".
Let's say you have the following 11 points
x,f(x) = (0,3),(1,7),(2,9),(3,11),(4,13),(5,14),(6,16),(7,5),(8,3)(9,1)(1,-1)
you can do something like inspired to the bisection method
a = 0 ,f(a) = 3 | b=10,f(b)=-1 | c=(0+10/2) f(5)=14
from here you can see that the increasing interval is [a,c[ and there is no need to that for the maximum because we know that in that interval the function is increasing. Maximum has to be in interval [c,b]. So at the next iteration you change the value of a s.t. a=c
a = 5 ,f(a) = 14 | b=10,f(b)=-1 | c=(5+10/2) f(6)=16
Again [a,c] is increasing so a is moved on the right
you can iterate the process until a=b=c.
Here the code that implements this idea. More info here:
int main(){
#define STEP (0.01)
#define SIZE (1/STEP)
double vals[(int)SIZE];
for (int i = 0; i < SIZE; ++i) {
double x = i*STEP;
vals[i] = -(x*x*x*x - (0.6)*(x*x));
}
for (int i = 0; i < SIZE; ++i) {
printf("%f ",vals[i]);
}
printf("\n");
int a=0,b=SIZE-1,c;
double fa=vals[a],fb=vals[b] ,fc;
c=(a+b)/2;
fc = vals[c];
while( a!=b && b!=c && a!=c){
printf("%i %i %i - %f %f %f\n",a,c,b, vals[a], vals[c],vals[b]);
if(fc - vals[c-1] > 0){ //is the function increasing in [a,c]
a = c;
}else{
b=c;
}
c=(a+b)/2;
fa=vals[a];
fb=vals[b];
fc = vals[c];
}
printf("The maximum is %i=%f with %f\n", c,(c*STEP),vals[a]);
}
Find points where derivative(of f(x))=(df/dx)=0
for derivative you could use five-point-stencil or similar algorithms.
should be O(n)
Then fit those multiple points (where d=0) on a polynomial regression / least squares regression .
should be also O(N). Assuming all numbers are neighbours.
Then find top of that curve
shouldn't be more than O(M) where M is resolution of trials for fit-function.
While taking derivative, you could leap by k-length steps until derivate changes sign.
When derivative changes sign, take square root of k and continue reverse direction.
When again, derivative changes sign, take square root of new k again, change direction.
Example: leap by 100 elements, find sign change, leap=10 and reverse direction, next change ==> leap=3 ... then it could be fixed to 1 element per step to find exact location.
I am assuming that the function evaluation is very costly.
In the special case, that your function could be approximately fitted with a polynomial, you can easily calculate the extrema in least number of function evaluations. And since you know that there is only one maximum, a polynomial of degree 2 (quadratic) might be ideal.
For example: If f(x) can be represented by a polynomial of some known degree, say 2, then, you can evaluate your function at any 3 points and calculate the polynomial coefficients using Newton's difference or Lagrange interpolation method.
Then its simple to solve for the maximum for this polynomial. For a degree 2 you can easily get a closed form expression for the maximum.
To get the final answer you can then search in the vicinity of the solution.
I am implementing a simulation of the wave equation using an array to discretely model a spatial region in which waves can propagate. Currently, waves reflect off the boundaries of the spatial region. However, I want to eliminate this reflection so that waves appear to propagate off forever.
I am aware there are many academic papers discussing nonreflecting / absorbing boundary conditions (e.g. perfectly matched layers?), but most seem to focus on analytic solutions. I cannot figure out how to implement nonreflecting boundaries numerically in my simulation. This is the code I am writing:
for (var i = 1; i < width - 1; ++i) {
for (var j = 1; j < height - 1; ++j) {
var d2f_dx2 = f[i + 1][j] - f[i][j] * 2 + f[i - 1][j];
var d2f_dy2 = f[i][j + 1] - f[i][j] * 2 + f[i][j - 1];
var d2f_dt2 = c2[i][j] * (d2f_dx2 + d2f_dy2);
df_dt[i][j] += d2f_dt2;
}
}
for (var i = 1; i < width - 1; ++i) {
for (var j = 1; j < height - 1; ++j) {
f[i][j] += df_dt[i][j];
}
}
where f is the field, df_dt is the partial derivative of the field with respect to time, d2f_dt2 is the second partial derivative of the field with respect to time, d2f_dx2 is the second partial derivative of the field in the x direction, and d2f_dy2 is the second partial derivative of the field in the y direction.
Does anyone know how I can adjust this code to have nonreflecting boundaries?
After clearing a few 25 year old cobwebs, the solution to your problem will depend on your setting the equations to satisfy the following initial conditions and condition at infinity. It has been far to long for me to translate initial and infinite boundary conditions into the partial differential and then into code for you, but knowing the correct boundary conditions to apply will provide the numerical model your are trying to create. Hopefully this will help.
For the undamped non-reflecting condition, the boundary value problem you are looking to model is described in the wikipedia article you site under the last paragraph of The_Sturm-Liouville_formulation. The The_Sturm-Liouville formulation itself may not provide the proper model, but the boundary conditions discussed in the last paragraph under the heading are those you must satisfy. The derivation is explained in a single dimension, but as noted in the article, the numerical solution for the one-dimension problem can be extended to any number of dimensions.
Boundary Conditions for Undamped Infinite Propagation
boundary value at t=0 == value at t=infinity after X whole periods, where
y = Asin(Bx - C) + D or y = Acos(Bx - C) + D.
The solution for f(x)t and f(y)t will be periodic trigonomic functions with the waves propagating on to infinity. If you think about it, the conditions are clear. At any point in time, the wave you wish to describe is simply an undamped periodic harmonic that will be modeled on a sine, cosine, etc.. The only difference in description of the wave at any point in time will be amplitude and phase as it cycles though a normal period. The identity of which trigonomic function will satisfy your initial condition will depend on the phase angle and shift at time t=0. The boundary condition as time approaches infinity will be that same function after a whole number of periods are complete.