I made simple variable
and i want to count how many times it occur
so I made code like this
a = close>open(it is just examplary, not my real code)
gross = ~~~~~~
num = cum(a ? 1 : 0)
total = sum(a ? gross : na, num)
I put the num variable into length position of sum function
but it turns error message like this
cannot call 'sum' with 'length'=series[float]. the argument should be of type: series[integer];
so to check the value of num is float, I made plot(num,~~) and it returns 128.0000
I think condition a occured 128times.
so the value of num isn't float am i right?
I don't know why error message happens
I already used round() and `int()
please help me solve this problem
Related
In the code below, lv_sum_openamount should be 3.45 but the program rounds the number as 3.
I want lv_sum_openamount as 3.
How can I do that ?
DATA(lv_sum_openamount) = REDUCE dmbtr_cs( INIT sum = 0 FOR wa_amnt IN <fs_comp> NEXT sum += wa_amnt-open_amount.
LOOP AT <fs_comp> ASSIGNING <fs_comp_alv>.
TRY.
<fs_comp_alv>-pull_amount = ( <fs_pack>-reamount / lv_sum_openamount ) * <fs_comp_alv>-open_amount.
CATCH cx_sy_zerodivide.
<fs_comp_alv>-pull_amount = 0.
ENDTRY.
ENDLOOP.
The culprit is the part INIT sum = 0.
0 is an integer, so the type for sum gets automatically derived as an integer. That means that the REDUCE-loop then uses integer arithmetic, so its output is rounded down.
Try INIT sum = CONV dmbtr_cs( 0 ) instead. This will convert the literal of 0 to the type you need and in turn force sum to also get that type.
I use remainder inside my code with kotlin in android project but with this value I don't get the correct answer.
variable is :
val vv = 1529.71
val ratio = 0.01
val remainder = vv.rem(ratio)
it's must be zero but remainder value is :
4.5363018896793506E-15
I don't understand why this happened.
The answer is because vv isn't actually 1529.71 but the closest possible Double, the exact value is 1529.7100000000000363797880709171295166015625 (the easiest way to see it is println(java.math.BigDecimal(vv))). If you want to represent decimal numbers exactly, use BigDecimal and pass the fraction as a string:
val vv = BigDecimal("1529.71")
val ratio = BigDecimal("0.01")
val remainder = vv.rem(ratio)
Read more about floating point here: https://floating-point-gui.de/
In my case, i had to get only the exact digits of two numbers after the decimal point.
I achieved it by doing this:
val input = 30.47f
val remainder = (input * 100).toInt() - (input.toInt() * 100)
// remainder = 47 exactly, and not 469999999...
Hope this would be helpful for someone.
I'm trying to divide one number by another and then immediately ceil() the result. These would normally be variables, but for simplicity let's stick with constants.
If I try any of the following, I get 3 when I want to get 4.
double num = ceil(25/8); // 3
float num = ceil(25/8); // 3
int num = ceil(25/8); // 3
I've read through a few threads on here (tried the nextafter() suggestion from this thread) as well as other sites and I don't understand what's going on. I've checked and my variables are the numbers I expect them to be and I've in fact tried the above, using constants, and am still getting unexpected results.
Thanks in advance for the help. I'm sure it's something simple that I'm missing but I'm at a loss at this point.
This is because you are doing integer arithmetic. The value is 3 before you are calling ceil, because 25 and 8 are both integers. 25/8 is calculated first using integer arithmetic, evaluating to 3.
Try:
double value = ceil(25.0/8);
This will ensure the compiler treats the constant 25.0 as a floating point number.
You can also use an explicit cast to achieve the same result:
double value = ceil(((double)25)/8);
This is because the expressions are evaluated before being passed as an argument to the ceil function. You need to cast one of them to a double first so the result will be a decimal that will be passed to ceil.
double num = ceil((double)25/8);
I am working on an app that needs to utilize a ratio of a given number and multiply that ratio times another number. Problem is that I can't get numbers less that 1 to give me the proper decimal ratio, instead it gives me zero (when it should be .5).
Example:
float number = 1/2; // This gives me zero
double number = 1/2; // This also gives me zero
If you don't specify decimal places you're using integers which means the calculation is performed with integer precision before the result is cast to the type on the LHS. You want to do the the following when using hard coded numbers in your code:
float number = 1.0f / 2.0f;
double number = 1.0 / 2.0;
If you're aiming to use integer variables for an operation, you'll want to cast them to the type that you want for your result.
Try this
float number = 1.0/2.0;
Remember that 1 is an int, so you are essentially taking
(int)1 / (int)2
which returns
(int)0
To cast variables that are ints, do
float number = (float)numerator / (float)denominator;
I am doing a calculation of values using Microsoft Excel vba.
I have a portion of codes like the below:
INT(151.2 * 100)
I notice that the result from this calculation is 15119 but the correct result should be 15220.
It is ok if i remove the INT()
151.2 * 100
The result returned will be 15220.
Can anyone advise why is there such a difference and whether it is correct to simply remove the INT() to achieve the correct result?
Floating-point arithmetic.
a = (151.2 * 100)
Print Int(a)
15119
Print a
15120
Print a = 15120
False
There is no double-precision number to represent the result of 151.2 * 100 exactly as 15120. The closest one is apparently just under 15120. When you Int it, it gets truncated, i.e. rounded down to 15119.
Instead of truncating, you could round:
Print CInt(a)
15120
Print Round(a)
15120
Note that if you have a variable i of type Integer and you just say i = 151.2 * 100 as you suggest, then you are implicitly coercing the result to an integer i.e. implicitly saying i = CInt(151.2 * 100). I think it's better practice to be explicit.