A fast coding tip somehow ended up making the code slower in Julia - optimization

Question

I heard that being conscious of type-stability contributes a lot to the high performance in Julia programming, so I tried to measure how much time I can save when rewriting the type-unstable function into type-stable version.
As many people say, I assumed that type-stable coding of course has higher performance than type-unstable one. However, the result was otherwise:
# type-unstable vs type-stable
#　type-unstable
function positive(x)
if x < 0
return 0.0
else
return x
end
end
# type-stable
function positive_safe(x)
if x < 0
return zero(x)
else
return x
end
end
#time for n in 1:100_000_000
a = 2^( positive(-n) + 1 )
end
#time for n in 1:100_000_000
b = 2^( positive_safe(-n) + 1 )
end
result:
0.040080 seconds
0.150596 seconds
I cannot believe this. Are there some mistakes in my code? Or this is the fact?
Any information would be appreciated.
Context
Operating System and version: Windows 10
Browser and version: Google Chrome 90.0.4430.212（Official Build） （64 bit)
JupyterLab version: 3.0.14
#btime result
Just replacing #time with #btime for my code above
#btime for n in 1:100_000_000
a = 2^( positive(-n) + 1 )
end
# -> 1.500 ns
#btime for n in 1:100_000_000
b = 2^( positive_safe(-n) + 1 )
end
# -> 503.146 ms
Still weird.
the exact same code DNF showed me
using BenchmarkTools
#btime 2^(positive(-n) + 1) setup=(n=rand(1:10^8))
# -> 32.435 ns (0 allocations: 0 bytes)
#btime 2^(positive_safe(-n) + 1) setup=(n=rand(1:10^8))
#-> 3.103 ns (0 allocations: 0 bytes)
Works as expected.
I still don't understand what is happening.
I feel like I have to know better about the usage of #btime and benchmarking process.
By the way, as I said above, I'm trying this benchmarking on Jupyterlab.

Answer 1

The problem with your benchmark, you testing different logic code:
2 ^ (integer value)
and
2 ^ (float value)
But the most crucial part, if a and b is not defined before the loop, Julia compiler may remove the block. Your performance very much depends was the a and b defined before and were defined in the global scope or not.
And power is the time-consuming central part of your code (not the type unstable part).
positive function returns Float in your case, positive_safe returns Int)
The code similar to your case (by logic) could look like that:
# type-unstable
function positive(x)
if x < 0
return 0.0
else
return x
end
end
# type-stable
function positive_safe(x)
if x < 0
return 0.0
else
return Float64(x)
end
end
function test1()
a = 0.0
for n in 1:100_000_000
a += 2^( positive(-n) + 1 )
end
a
end
function test2()
b = 0.0
for n in 1:100_000_000
b += 2^( positive_safe(-n) + 1 )
end
b
end
#btime test1()
#btime test2()
98.045 ms (0 allocations: 0 bytes)
2.0e8
97.948 ms (0 allocations: 0 bytes)
2.0e8
The results are almost the same since your type unstable is not a bottleneck for the case.
If to test the function (which is similar to your case when a/b was not defined):
function test3()
b = 0.0
for n in 1:100_000_000
b += 2^( positive_safe(-n) + 1 )
end
nothing
end
#btime test3()
Benchmark will show results:
1.611 ns
This is not because my laptop did 100_000_000 iterations per 1.611 ns, but because Julia compiler smart enough to understand that the test3 function may be replaced with nothing.

Answer 2

This is benchmarking problem. The #time macro is not suitable for microbenchmarks. Use the BenchmarkTools.jl package, and read the user manual. It is easy to make mistakes when benchmarking.
Here's how to do it:
jl> using BenchmarkTools
jl> #btime 2^(positive(-n) + 1) setup=(n=rand(1:10^8))
6.507 ns (0 allocations: 0 bytes)
2.0
jl> #btime 2^(positive_safe(-n) + 1) setup=(n=rand(1:10^8))
2.100 ns (0 allocations: 0 bytes)
2
As you see, the type stable function is faster.

Answer 3

The problem, as Vitaliy said, is that powers in floating point done with logs can be faster than the integer ones that can be done as loop multiplies:
using BenchmarkTools
# type-unstable vs type-unstable
#　type-unstable
function positive_float_unstable(x)
if x < 0
return 0.0
else
return x
end
end
#　type-unstable
function positive_int_unstable(x)
if x < 0
return 0
else
return x
end
end
#　type-stable
function positive_float_stable(x)
if x < 0
return 0.0
else
return Float64(x)
end
end
#　type-stable
function positive_int_stable(x)
if x < 0
return 0
else
return Int(x)
end
end
println("unstable float")
#btime for n in 1:100_000_000
a = 2^( positive_float_unstable(-n) + 1 )
end
println("unstable int")
#btime for n in 1:100_000_000
b = 2^( positive_int_unstable(-n) + 1 )
end
println("stable float")
#btime for n in 1:100_000_000
a = 2^( positive_float_stable(-n) + 1 )
end
println("stable int")
#btime for n in 1:100_000_000
b = 2^( positive_int_stable(-n) + 1 )
end
Results:
unstable float
1.300 ns (0 allocations: 0 bytes)
unstable int
179.232 ms (0 allocations: 0 bytes)
stable float
1.300 ns (0 allocations: 0 bytes)
stable int
178.990 ms (0 allocations: 0 bytes)