I am stuck with an exercise at JetBrain Academy, with BigInteger(Kotlin). I should write a code snippet to convert a (BigInteger) number of exbibyte into bits: if the input is 1 as BigInteger, it displays 9223372036854775808. In the hints it says I should use pow(63).(Exbibyte = 2^63 = 9 223 372 036 854 775 808 Bit)
My problem is if I use pow(), I can use only double as the biggest range. But with double I cannot display a BigInteger number.
In your case, I hope it should work:
val exbibytes = BigInteger.valueOf(1)
val oneExbibyteInBits = BigInteger.valueOf(2).pow(63)
val exbibytesInBits = exbibytes.multiply(oneExbibyteInBits)
Related
I have scales application. I want to display i.e. 123.456g
I have uint8 that represents 123 (scales range is 0 - 150g)
I also have uint16 that represents 456 (scales range .0 - .999g)
My code:
val strValue = characteristic.getIntValue(FORMAT_UINT16,0).toFloat() + ((characteristic.getIntValue(FORMAT_UINT16,1) or characteristic.getIntValue(FORMAT_UINT16,2).shl(8)) / 1000f)
Produces 51323.457 instead of 123.456
What is wrong?
You're adding floats, not generating a string.
Convert your digits to String, then concatenate them.
I'm trying to display the total price from my TipCalculator as $147.00, but it is only returning a single 0 at the end $147.0. Is there a way to fix this?
You might be looking for DecimalFormat.
It can be used to format a number with fixed decimals, for example
val num = 147
val df = DecimalFormat("#.00")
println(df.format(num)) //prints 147.00
Does anyone know why, using SQLServer 2005
SELECT CONVERT(DECIMAL(30,15),146804871.212533)/CONVERT(DECIMAL (38,9),12499999.9999)
gives me 11.74438969709659,
but when I increase the decimal places on the denominator to 15, I get a less accurate answer:
SELECT CONVERT(DECIMAL(30,15),146804871.212533)/CONVERT(DECIMAL (38,15),12499999.9999)
give me 11.74438969
For multiplication we simply add the number of decimal places in each argument together (using pen and paper) to work out output dec places.
But division just blows your head apart. I'm off to lie down now.
In SQL terms though, it's exactly as expected.
--Precision = p1 - s1 + s2 + max(6, s1 + p2 + 1)
--Scale = max(6, s1 + p2 + 1)
--Scale = 15 + 38 + 1 = 54
--Precision = 30 - 15 + 9 + 54 = 72
--Max P = 38, P & S are linked, so (72,54) -> (38,20)
--So, we have 38,20 output (but we don use 20 d.p. for this sum) = 11.74438969709659
SELECT CONVERT(DECIMAL(30,15),146804871.212533)/CONVERT(DECIMAL (38,9),12499999.9999)
--Scale = 15 + 38 + 1 = 54
--Precision = 30 - 15 + 15 + 54 = 84
--Max P = 38, P & S are linked, so (84,54) -> (38,8)
--So, we have 38,8 output = 11.74438969
SELECT CONVERT(DECIMAL(30,15),146804871.212533)/CONVERT(DECIMAL (38,15),12499999.9999)
You can do the same math if follow this rule too, if you treat each number pair as
146804871.212533000000000 and 12499999.999900000
146804871.212533000000000 and 12499999.999900000000000
To put it shortly, use DECIMAL(25,13) and you'll be fine with all calculations - you'll get precision right as declared: 12 digits before decimal dot, and 13 decimal digits after.
Rule is: p+s must equal 38 and you will be on safe side!
Why is this?
Because of very bad implementation of arithmetic in SQL Server!
Until they fix it, follow that rule.
I've noticed that if you cast the dividing value to float, it gives you the correct answer, i.e.:
select 49/30 (result = 1)
would become:
select 49/cast(30 as float) (result = 1.63333333333333)
We were puzzling over the magic transition,
P & S are linked, so:
(72,54) -> (38,29)
(84,54) -> (38,8)
Assuming (38,29) is a typo and should be (38,20), the following is the math:
i. 72 - 38 = 34,
ii. 54 - 34 = 20
i. 84 - 38 = 46,
ii. 54 - 46 = 8
And this is the reasoning:
i. Output precision less max precision is the digits we're going to throw away.
ii. Then output scale less what we're going to throw away gives us... remaining digits in the output scale.
Hope this helps anyone else trying to make sense of this.
Convert the expression not the arguments.
select CONVERT(DECIMAL(38,36),146804871.212533 / 12499999.9999)
Using the following may help:
SELECT COL1 * 1.0 / COL2
if we write 12wkd3, how to choose/filter 123 as integer in octave?
example in octave:
A = input("A?\n")
A?
12wkd3
A = 123
while 12wkd3 is user keyboard input and A = 123 is the expected answer.
assuming that the general form you're looking for is taking an arbitrary string from the user input, removing anything non-numeric, and storing the result it as an integer:
A = input("A? /n",'s');
A = int32(str2num(A(isdigit(A))));
example output:
A?
324bhtk.p89u34
A = 3248934
to clarify what's written above:
in the input statement, the 's' argument causes the answer to get stored as a string, otherwise it's evaluated by Octave first. most inputs would produce errors, others may be interpreted as functions or variables.
isdigit(A) produces a logical array of values for A with a 1 for any character that is a 0-9 number, and 0 otherwise.
isdigit('a1 3 b.') = [0 1 0 1 0 0 0]
A(isdigit(A)) will produce a substring from A using only those values corresponding to a 1 in the logical array above.
A(isdigit(A)) = 13
that still returns a string, so you need to convert it into a number using str2num(). that, however, outputs a double precision number. so finally to get it to be an integer you can use int32()
I'm writing a task in pascal.
Everything is ok, just my result is not right.
I'm summing some numbers
Example: 2.3 + 3.4+ 3.3 = 9
But output shows: 9.000000 + EEE or something like that.
So- how to convert, to be only 9, not this REAL variable.
To actually convert:
var
i: integer;
...
i := round(floatVar);
To output only the integer part:
writeln(floatVar:9:0);
Let's consider this quite simpler equation:
3.5 + 2.5
What do you expect? 6, right? Let's try this code
write(3.5 + 2.5);
Unfortunately, it's a floating-point number, so it would produce a number represented in a scientific way:
6.00000000000E+00
or 6.0000000000 x 100, or 6 x 10o. Whatever, you only care about 6, who need this weird useless long number? So the idea is to cut off the decimal part and output to the console only the integer part, which can be done with this line of code:
write(3.5 + 2.5 : 0 : 0);
Ok, now it outputs a beautiful number as expected
6
Seems like the problem is solved, but you say that:
I'm summing some numbers
Example: 2.3 + 3.4+ 3.3 = 9
Ohh so that the evenly, beautiful integer is just randomly appeared? Here the problem comes, how do you expect this equation would output?
3.6 + 2.5
It should be 6.1, right? Let's try it with the worked line of code:
write(3.6 + 2.5 : 0 : 0);
And the output is...
6
Unexpected, right? So how about rounding to some decimal places, like 1?
write(3.5 + 2.5 : 0 : 1);
write(3.6 + 2.5 : 0 : 1);
Then, 3.5 + 2.5 = 6.0 and 3.6 + 2.5 = 6.1. But 6.0 may look quite long, so how to make it output 6 for 6.0 and 6.1 for 6.1?
Actually, you can't make the program auto-detect if a real variable contains an integer value because the way a real var is stored is completely different from an integer var (how different they are, please contact Google; but you can do it manually by making a function to do the job).
So my solution is, to be easy, making the output rounded to some decimal places, and that's it.
For purpose of showing pretty output on the screen you can use something like this:
Writeln(result:0:2);
Result on screen would be this:
9.00
What this means someone would ask? Well first number 0 means how wide filed is. if you say it's 0 then Pascal writes it at the very left side of screen. If you said writeln(result:5:2) result would be:
9.00
In other words i would print form the right side and leave 5 chars to do so.
Second number 2, in this example means you want that result printed with 2 decimal places. You can place it only if you want to print on screen value that is real, single, double, extended and so on.You can round to any number of decimals, and if you do writeln(result:0:0) you would get ouput:
9
If you are printing integer and want to have some length of field, lets sat 5 you would do: writeln(int:5). If you added :2 to the end you would get compile time error.
This all also works for something like this: writeln(5/3.5+sqrt(3):0:3),
You should know that this does not round variable itself but just formats output. This is also legal:
program test;
var
a:real;
n,m:integer;
begin
readln(a,m,n);
writeln(a:m:n);
end.
What i did here is i asked user if on how many decimals and with what length of field he wants to write entered number a. This can be useful so i'm pointing it out. Thank you for reading. I hope i helped
You can convert to string, get the int part, e convert to int number!
Or Float to Str than Str to Int:
nPage := StrToInt(FloatToStr(Int(nReg / nTPages))) + 1;