Greater than in ActiveRecord query fails with floats? - sql

Question

I need to calculate ranking for values in my Rails app.
I was following example in this question
def rank
User.where("points > ?", points).count + 1
end
Initially I verfied it with integers and it was working. But I also have need to rank floats.
For example, I have following values
0.6238564767774734
0.03700210614260772
0.022441047654982744
0.00935025180031852
0.0016195952859973067
0.0010382902478650936
0.0009367068270665785
0.0004916500182958447
0.00016560735047205894
If I call query
User.where("points > ?", 0.6238564767774734).count + 1
It returns 2. Why is that, shouldn't it return 0 as there are no values that are bigger than it? Also, queries with fourth and fifth values both return value of 5.
SQL queries from console as follows:
SELECT `users`.* FROM `users` WHERE (points > 0.623856)
SELECT `users`.* FROM `users` WHERE (points > 0.00935025)
SELECT `users`.* FROM `users` WHERE (points > 0.0016196)
Just in case I also tried length and size instead of count.
What is wrong and how I could I fix it? All help appreciated.

Answer 1

It looks like a problem with difference between that how mysql rounds floats and how ruby rounds floats. Using decimal instead of float might be a better idea.
Also take a look at
http://dev.mysql.com/doc/refman/5.0/en/floating-point-types.html

Answer 2

0.6238564767774734 goes beyond the precision of a float.
What you'd get in Postgres (I'm unaware of a pg_typeof() equivalent in MySQL):
denis=# select pg_typeof(0.6238564767774734);
pg_typeof
-----------
numeric
(1 row)
denis=# select 0.6238564767774734::decimal, 0.6238564767774734::float;
numeric | float8
--------------------+-------------------
0.6238564767774734 | 0.623856476777473
(1 row)
On its end, Ruby is using a BigDecimal. The MySQL type that would match it (more or less in MySQL, since you need to specify the precision) would be the decimal type:
http://dev.mysql.com/doc/refman/5.7/en/fixed-point-types.html
Be wary that MySQL requires a precision in this case:
mysql> select cast(0.6238564767774734 as decimal);
+-------------------------------------+
| cast(0.6238564767774734 as decimal) |
+-------------------------------------+
| 1 |
+-------------------------------------+
1 row in set (0.00 sec)
mysql> select cast(0.6238564767774734 as decimal(20,20));
+--------------------------------------------+
| cast(0.6238564767774734 as decimal(20,20)) |
+--------------------------------------------+
| 0.62385647677747340000 |
+--------------------------------------------+
1 row in set (0.00 sec)
Lastly, note that you'll still get errors due to rounding problems related to how floating point types are represented, if you stick to floats and adjust your criteria:
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
(I'm guessing you're using decimals internally in there somewhere, but the above set of problems related to floats are good to have in mind when doing comparisons.)