I'm new with getting time complexities and I can't seem to understand the logic behind getting this at the end:
100 (n(n+1) / 2)
For this function:
function a() {
int i,j,k,n;
for(i=1; i<=n; i++) {
for(j=1; j<=i; j++) {
for(k=1; k<=100; k++) {
print("hello");
}
}
}
}
Here's how I understand its algorithm:
i = 1, 2, 3, 4...n
j = 1, 2, 3, 4...(dependent to i, which can be 'n')
k = 1(100), 2(100), 3(100), 4(100)...n(100)
= 100 [1, 2, 3, 4.....]
If I'll use this algorithm above to simulate the end equation, I'll get this result:
End Equation:
100 (n(n+1) / 2)
Simulation
i = 1, 2, 3, 4... n
j = 1, 2, 3, 4... n
k = 100, 300, 600, 10000
I usually study these in youtube and get the idea of Big O, Omega & Theta but when it comes to this one, I can't figure out how they end with the equation such as what I have given. Please help and if you have some best practices, please share.
EDIT:
As for my own assumption of answer, it think it should be this one:
100 ((n+n)/2) or 100 (2n / 2)
Source:
https://www.youtube.com/watch?v=FEnwM-iDb2g
At around: 15:21
I think you've got i and j correct, except that it's not clear why you say k = 100, 200, 300... In every loop, k runs from 1 to 100.
So let's think through the inner loop first:
k from 1 to 100:
// Do something
The inner loop is O(100) = O(1) because its runtime does not depend on n. Now we analyze the outer loops:
i from 1 to n:
j from 1 to i:
// Do inner stuff
Now lets count how many times Do inner stuff executes:
i = 1 1 time
i = 2 2 times
i = 3 3 times
... ...
i = n n times
This is our classic triangular sum 1 + 2 + 3 + ... n = n(n+1) / 2. Therefore, the time complexity of the outer two loops is O(n(n+1)/2) which reduces to O(n^2).
The time complexity of the entire thing is O(1 * n^2) = O(n^2) because nesting loops multiplies the complexities (assuming the runtime of the inner loop is independent of the variables in the outer loops). Note here that if we had not reduced at various phases, we would be left with O(100(n)(n+1)/2), which is equivalent to O(n^2) because of the properties of big-O notation.
SOME TIPS:
You asked for some best practices. Here are some "rules" that I made use of in analyzing the example you posted.
In time complexity analysis, we can ignore multiplication by a constant. This is why the inner loop is still O(1) even though it executes 100 times. Understanding this is the basis of time complexity. We are analyzing runtime on a large scale, not counting the number of clock cycles.
With nested loops where the runtime is independent of each other, just multiply the complexity. Nesting the O(1) loop inside the outer O(N^2) loops resulted in O(N^2) code.
Some more reduction rules: http://courses.washington.edu/css162/rnash/quarters/current/labs/bigOLab/lab9.htm
If you can break code up into smaller pieces (in the same way we analyzed the k loop separately from the outer loops) then you can take advantage of the nesting rule to find the combined complexity.
Note on Omega/Theta:
Theta is the "exact bound" for time complexity whereas Big-O and Omega are upper and lower bounds respectively. Because there is no random data (like there is in a sorting algorithm), we can get an exact bound on the time complexity and the upper bound is equal to the lower bound. Therefore, it does not make any difference if we use O, Omega or Theta in this case.
Having two complex arrays of dimension 2 I want to calculate a point wise multiplication (Hadamard product):
complex(8) :: A(N,N), B(N,N), C(N,N)
...
do j = 1, N
do i = 1, N
C(i,j) = A(i,j)*B(i,j)
enddo
enddo
Is there any BLAS routine to optimize that or is this actually already the most efficient way to write the Hadamard product? Or does the compiler do the job for me in such a simple case?
I code in Fortran so the first index is the fast index.
The outer loop executes n times while the inner loop executes ? So the total time is n*something.
Do i need to learn summation,if yes then any book to refer?
for(int i=1;i<=n;i++)
for(int j=1;j<=n;j+=i)
printf("*");
This question can be approached by inspection:
n = 16
i | j values | # terms
1 | 1, 2, ..., 16 | n
2 | 1, 3, 5, ..., 16 | n / 2
.. | .. | n / 3
16 | 16 | n / n
In the above table, i is the outer loop value, and j values show the iterations of the inner loop. By inspection, we can see that the loops will take n * (1 + 1/2 + 1/3 + ... + 1/n) steps. This is a bounded harmonic series. As this Math Stack Exchange article shows, there is no closed form for the above expression in terms of n. However, as this SO article shows, there is an upper bound of O(n*ln(n)).
So, the running time for your two loops is O(n*ln(n)).
I believe the time complexity of that is O(n*log(n)). Here is why:
Let us pick some arbitrary natural number i and see how many steps the inner loop takes for this given i. Well for this i, you are going from j=1 to j<=n with a jump of i in between. So basically you are doing this summation many steps:
summation = 1 + (1+i) + (1+2i) + ... (1+ki)
where k is the largest integer such that 1+ki <= n. That is, k is the number of steps and this is what we want to solve for. Well we can solve for k in the equality resulting in k <= (n-1)/i and thus k = ā(n-1)/iā. That is, k is the floor function/integer division of (n-1)/i. Since we are dealing with time complexities, this floor function doesn't matter so we will just say k = n/i for simplicity. This is the number of steps that the inner loop will take for a given i. So we basically need to add all these for i = 1 to i <= n.
So numsteps will be this addition:
numsteps = n/1 + n/2 + n/3 + ... n/n
= n(1 + 1/2 + 1/3 + ... 1+n)
So we need to find the sum of 1 + 1/2 + ... 1/n to finish this. There is actually no good closed form for this sum but it is on the order of ln(n). You can read more about this here. You can also guess this since the integral from 1 to n of 1/x is ln(n). Again, since we are dealing with time complexity, we can just use ln(n) to represent its complexity. Thus we have:
numsteps = n(ln(n))
And so the time complexity is O(n*log(n)).
Edit: My bad, i was calculating the sum :P
I'm trying to get a good grasp with this problem but I'm struggling.
Let's say that I have a S={1,2,3,4,5}, an L={(1,3,4),(2,3),(4,5),(1,3),(2),(5)} and an other tuple with the costs of L like C={10,20,12,15,4,10}
I want to make a constraint program in Prolog so as to take the solution that solves the problem with the minimum cost.(in this occasion it is the total sum of the costs of the subsets i will get)
My problem is that I cannot understand the way I'll make my modelisation. What I know is that I should choose a modelisation of binary variables {0,1} but I hardly understand how i will manage to express it via Prolog.
There is an easy way to do it: You can use Boolean indicators to denote which elements comprise a subset. For example, in your case:
subsets(Sets) :-
Sets = [[1,0,1,1,0]-10, % {1,3,4}
[0,1,1,0,0]-20, % {2,3}
[0,0,0,1,1]-12, % {4,5}
[1,0,1,0,0]-15, % {1,3}
[0,1,0,0,0]-4, % {2}
[0,0,0,0,1]-10]. % {5}
I now use SICStus Prolog and its Boolean constraint solver to express set covers:
:- use_module(library(lists)).
:- use_module(library(clpb)).
setcover(Cover, Cost) :-
subsets(Sets),
keys_and_values(Sets, Rows, Costs0),
transpose(Rows, Cols),
same_length(Rows, Coeffs),
maplist(cover(Coeffs), Cols),
labeling(Coeffs),
phrase(coeff_is_1(Coeffs, Rows), Cover),
phrase(coeff_is_1(Coeffs, Costs0), Costs),
sumlist(Costs, Cost).
cover(Coeffs, Col) :-
phrase(coeff_is_1(Col,Coeffs), Cs),
sat(card([1],Cs)).
coeff_is_1([], []) --> [].
coeff_is_1([1|Cs], [L|Ls]) --> [L], coeff_is_1(Cs, Ls).
coeff_is_1([0|Cs], [_|Ls]) --> coeff_is_1(Cs, Ls).
For each subset, a Boolean variable is used to denote whether that subset is part of the cover. Cardinality constraints make sure that each element is covered exactly once.
Example query and its result:
| ?- setcover(Cover, Cost).
Cover = [[0,0,0,1,1],[1,0,1,0,0],[0,1,0,0,0]],
Cost = 31 ? ;
Cover = [[1,0,1,1,0],[0,1,0,0,0],[0,0,0,0,1]],
Cost = 24 ? ;
no
I leave picking a cover with minimum cost as an easy exercise.
Maybe an explicit model for your problem instance makes things a bit clearer:
cover(SetsUsed, Cost) :-
SetsUsed = [A,B,C,D,E,F], % a Boolean for each set
SetsUsed #:: 0..1,
A + D #= 1, % use one set with element 1
B + E #= 1, % use one set with element 2
A + B + D #= 1, % use one set with element 3
A + C #= 1, % use one set with element 4
C + F #= 1, % use one set with element 5
Cost #= 10*A + 20*B + 12*C + 15*D + 4*E + 10*F.
You can solve this e.g. in ECLiPSe:
?- cover(SetsUsed,Cost), branch_and_bound:minimize(labeling(SetsUsed), Cost).
SetsUsed = [1, 0, 0, 0, 1, 1]
Cost = 24
Yes (0.00s cpu)
I just wanted to make sure I'm going in the right direction. I want to find a max value of an array by recursively splitting it and find the max of each separate array. Because I am splitting it, it would be 2*T(n/2). And because I have to make a comparison at the end for the 2 arrays, I have T(1).
So would my recurrence relation be like this:
T = { 2*T(n/2) + 1, when n>=2 ;T(1), when n = 1;
and and therefore my complexity would be Theta(nlgn)?
The formula you composed seems about right, but your analysis isn't perfect.
T = 2*T(n/2) + 1 = 2*(2*T(n/4) + 1) + 1 = ...
For the i-th iteration you'll get:
Ti(n) = 2^i*T(n/2^i) + i
now what you want to know for which i does n/2^i equals 1 (or just about any constant, if you like) so you reach the end-condition of n=1.
That would be the solution to n/2^I = 1 -> I = Log2(n). Plant it in the equation for Ti and you get:
TI(n) = 2^log2(n)*T(n/2^log2(n)) + log2(n) = n*1+log2(n) = n + log2(n)
and you get T(n) = O(n + log2(n) (just like #bdares said) = O(n) (just like #bdares said)
No, no... you are taking O(1) time for each recursion.
How many are there?
There are N leaves, so you know it's at least O(N).
How many do you need to compare to find the absolute maximum? That's O(log(N)).
Add them together, don't multiply. O(N+log(N)) is your time complexity.