I used String.format() to round to the third decimal place. However this didn't work and I solved it using DecimalFormat.
Is there anything I implemented wrong?
val value = 23.695f
Timber.e("format: ${"%.2f".format(value)}")
Expect: 23.70
Result: 23.69
In your string formatting example you're not rounding the number you're just taking the first two decimal places. So your code drops the .005 and keeps the .69
If .70 is the desired result then DecimalFormat sounds correct to me
Related
I can find a workaround, but it is really annoying and I may certainly be missing something. Redshift's ROUND function doesn't round to the number of decimals specified.
For example,
select round(cast(176 as float)/cast(492 as float),4) as result;
Above select statement will return 0.35769999999999996.
However, this statement:
select round(cast(229 as float)/cast(491 as float),4) as result;
... will return 0.4664.
Why? I can work around this, but seems like it should work and return only four decimal places.
If your issues is all those 9999s, then the issue is floating point representation. Convert to a decimal to get fixed-point precision:
select round(cast(176 as float)/cast(492 as float), 4)::decimal(10, 4) as result;
Elaborating more on Gordon's answer -
So you’ve written some absurdly simple code, say for example:
0.1 + 0.2
and got a really unexpected result:
0.30000000000000004
Because internally, computers use a format (binary floating-point) that cannot accurately represent a number like 0.1, 0.2 or 0.3 at all.
When the code is compiled or interpreted, your “0.1” is already rounded to the nearest number in that format, which results in a small rounding error even before the calculation happens.
What can I do to avoid this problem?
That depends on what kind of calculations you’re doing.
If you really need your results to add up exactly, especially when you work with money: use a decimal datatype.
If you just don’t want to see all those extra decimal places: simply format your result rounded to a fixed number of decimal places when displaying it.
Shamelessly stolen from : Floating Point
try multiplying by 10 to the power of your desired places after the decimal point, rounding, and then dividing it out again:
-- exclude decimal point inside ROUND(), include outside ROUND()
SELECT ROUND(10000 * 176 / 492) / 10000.0
which will return the expected 0.3577.
I need to convert a price into a string of 8 digits and right now, I'm using this:
=Format(Replace(Round([total],2),",",""),"00000000")
If the price is (eg: 105.55) it converts like this: 00010455 and this is ok!
The problem:
When the price ends with a zero (like this: 147.60). In this case, it returns 00001476and it's missing the last zero which I need to correctly solve the rest.
Even if I remove the Round part, I get the same problem.
=Format(Replace([total],",",""),"00000000")
I can't figure why this is happening and how to do it right...
Try this simpler approach:
=Format([total] * 100, "00000000")
I'm using google bigquery, and a column has values I want to round. If I do, and the rounded value ends in a zero, the zero is not displayed.
I've tried the function FORMAT, which apparently has some .number function, but I have no idea how to use it. Whenever I include any 2 things separated by a comma inside its brackets, it complains that it only takes 1 value.
You would use FORMAT() with the precision specifier. For four decimal places always -- including zeros:
select format('%.4f', 1.23)
If the BQ documentation does not answer your questions, I find that that the function seems to be inspired by the classic C printf()/sprintf() functions.
Unaware if in BigQuery (haven't used it ever) there is a better way I guess this will fix your problem since I just tried it in their console.
Cast your float to a string and then check if your last digit is a 0. In case it's not add it:
SELECT case when RIGHT(cast(0.9 as string), 1) <> '0' then cast(0.9 as string)+'0' else cast(0.9 as string) end as FormattedNumber
I'm trying to extract some numbers with decimals but need to remove the decimal part. There's no fixed length on any of both sides.
I have already tried:
INT()
INTR()
ROUND()
Usually INT() should solve this but sometimes it doesn't return the correct number (for example, INT(3) returns 2).
Did you try the function TRUNC(number, [, decimal_places]) ?
EDIT
For the round precision you can see this link
I ended up solving it on my own! ROUND didn't work and FLOOR was also undefined...
I ended up needing to use two INTs with a FORMAT in one of them.
Thank you anyway.
Can any one please help me how to get float value as it is from text box
for Ex: I have entered 40.7
rateField=[[rateField text] floatValue];
I am getting rateField value as 40.7000008 but I want 40.7 only.
please help me.
thanks in advance
Thanks Every body,
I tried all the possibilities but I am not able to get what I want. I am not looking to print the value to convert into string.I want to use that value for computation. If i use Number Formatter again when i am converting from number to float it is giving same problem.So i want float value only but it should be whatever i have given in the text box it should not be padded with any values.This is my requirement.Please help me.
thanks®ards Balu
Thanks Every body,
I tried all the possibilities but I am not able to get what I want. I am not looking to print the value to convert into string.I want to use that value for computation. If i use Number Formatter again when i am converting from number to float it is giving same problem.So i want float value only but it should be whatever i have given in the text box it should not be padded with any values.This is my requirement.Please help me.
thanks®ards
Balu
This is ok. There is not guaranteed that you will get 40.7 if you will use even double.
If you want to output 40.7 you can use %.1f or NSNumberFormatter
Try using a double instead. Usually solves that issue. Has to do with the storage precision.
double dbl = [rateField.text doubleValue];
When using floating point numbers, these things can happen because of the way the numbers are stored in binary format in the computers memory.
It's similar to the way 1/3 = 0.33333333333333... in decimal numbers.
The best way to deal with this is to use number formatters in the textbox that displays the value.
You are already resolved float value.
Floating point numbers have limited precision. Although it depends on
the system, float relative error due to rounding will be around 1.1e-8
Non elementary arithmetic operations may give larger errors, and, of
course, error progragation must be considered when several operations
are compounded.
Additionally, rational numbers that are exactly representable as
floating point numbers in base 10, like 0.1 or 0.7, do not have an
exact representation as floating point numbers in base 2, which is
used internally, no matter the size of the mantissa. Hence, they
cannot be converted into their internal binary counterparts without a
small loss of precision. This can lead to confusing results: for
example, floor((0.1+0.7)*10) will usually return 7 instead of the
expected 8, since the internal representation will be something like
7.9999999999999991118....
So if you're using those numbers for output, you should use some rounding mechanism, even for double values.