So I am trying to rewrite the following formula to Obj-C (C):
double level = 10;
double xp = ((1/8 * level) * (level - 1)) + (75 * ( ((pow(2,(level - 1)/7)- 1) / (1 - pow(2, -1/7)))));
When I print it out I get inf meaning there is something wrong with how I've written the formula. I've re-written the thing many times for the past two hours - any help?
I'd worry about the sections "1/8" and "-1/7" being processed as ints and resulting with "0" in both cases. Just putting a .0 after each literal tells the compiler (and fellow coders) to treat them as doubles
double xp = ((1.0/8.0 * level) * (level - 1.0)) + (75.0 * ( ((pow(2.0,(level - 1.0)/7.0)- 1.0) / (1.0 - pow(2.0, -1.0/7.0)))));
Related
In Teradata when computing a percentage (see example below), I get 0 as an answer because Terada rounds results based on the number of decimal places the number have. to my knowledge I have found 2 possible solution to avoid the zero value return.
1- define the decimal number in table with a larger scale value. e.g decimal(18,4) instead of decimal(18,2)
2- cast the first arithmetic operation that occurs to a decimal number with a larger scale value. e.g. select (cast(2.0 as decimal(10,6)) / 10.0) * 100;
has anyone encounter this problem before and what is the solution you went with to solve the problem? thank you in advance.
these are the actual values and the correct answer should be 0.25
select ((28.97 + 28.97) / (11586.87 + 11586.87)) * 100
,(cast((28.97 + 28.97) as decimal(20,4)) / (11586.87 + 11586.87)) * 100
,((cast(28.97 as decimal(18,4)) + 28.97) / (11586.87 + 11586.87))
,(100 * (28.97 + 28.97) / (11586.87 + 11586.87))
,(28.97 + 28.97) , (11586.87 + 11586.87);
Trying to make a calculation for student grades in an MCQ. 30 q's, 4 marks for correct and -1 for incorrect.
The code below works when I do not put in the /120 part. For a record with 16 correct answers and 7 wrong answers, it returns a value of 57, which is what I want it to. But when I try to divide it by 120 to get a percentage value, it returns 63.94, can anybody explain how or why this happens?
I can upload more code if needs be.
Without "/ 120"
(Math.Round(((CDbl(objAssessment1.Item("Correct")) * 4) - (CDbl(objAssessment1.Item("Wrong")))), 2))
With "/ 120"
(Math.Round(((CDbl(objAssessment1.Item("Correct")) * 4) - (CDbl(objAssessment1.Item("Wrong"))) / 120), 2))
multiplication/division before addition/substraction
What you got, if we put in the actual values is: 16 * 4 - 7 / 120
What you actually want is: (16 * 4 - 7) / 120
(Math.Round(((CDbl(objAssessment1.Item("Correct")) * 4) - (CDbl(objAssessment1.Item("Wrong"))) / 120), 2))
(Math.Round(((CDbl(16) * 4) - (CDbl(7)) / 120), 2))
(Math.Round(((16 * 4) - (7) / 120), 2))
(Math.Round(((16 * 4) - (7) / 120), 2))
(Math.Round(((64) - (7) / 120), 2))
The 64 and the 7 in parenthesis just evaluate to themselves.
(Math.Round((64 - 7 / 120), 2))
Now remember order of precedence from algebra - multiplication and division first.
(Math.Round((64 - 0.0583333333333333), 2))
Now the subtraction.
(Math.Round(63.94166666666667, 2))
Now the Math.Round
(63.94)
Removing the extra set of parenthesis 63.94 evaluates to itself
63.94
The whole problem was too many parenthesis and you missed on set to perform the subtraction before the division.
jmcilhinney's suggestion in comments would be the best way to straighten this out. This really wasn't a programming problem but an arithmetic problem. :-)
I need to get the nearest airport in my database table from the current users position. I found this formula: https://de.scribd.com/presentation/2569355/Geo-Distance-Search-with-MySQL#page=7
So there are a few differences between the formula described in the link above and my current situation: The example was in MySQL, I'm using MS SQL (not a problem, I guess). lat and lon are considered to be database columns with numeric data type, but for some reason the database table was created with two corresponding columns of type varchar.
My problem is: When I want to use an ORDER BY clause, it throws Error converting data type nvarchar to numeric, without it, it works. I did some research on what rubbish was inserted as string and migrated it so that I just have some empty values.
I can't take all because I only need one. But if I do TOP 1 without ORDER BY I don't get any airport rather than the nearest airport. Does anyone know how to fix the query?
Thanks in advance!
SELECT TOP 1
temp.Distance
FROM (
SELECT
(
3956 * 2 * ASIN(
SQRT(
POWER(
SIN((53.6349994 - abs(CAST(latitude_deg AS numeric))) * pi() / 180 / 2), 2) + COS(53.6349994 * pi()/180) * COS(abs(CAST(latitude_deg AS numeric)) * pi()/180) * POWER(SIN((10.0117336 - CAST(longitude_deg AS numeric)) * pi()/180 / 2), 2) ))) AS Distance
FROM Airport_Airports
WHERE
isnumeric(longitude_deg) = 1 AND isnumeric(latitude_deg) = 1 AND
longitude_deg LIKE '%[^0-9.]%' AND latitude_deg LIKE '%[^0-9.]%'
) AS temp
WHERE
temp.Distance < 50000
Order BY
temp.Distance
First, this logic doesn't make sense:
WHERE isnumeric(longitude_deg) = 1 AND
isnumeric(latitude_deg) = 1 AND
longitude_deg LIKE '%[^0-9.]%' AND
latitude_deg LIKE '%[^0-9.]%'
The like is looking for non-numeric characters. I think you intend:
WHERE isnumeric(longitude_deg) = 1 AND
isnumeric(latitude_deg) = 1 AND
longitude_deg NOT LIKE '%[^0-9.]%' AND
latitude_deg NOT LIKE '%[^0-9.]%'
This ensures that the values are numeric.
The solution to your problem -- at least in SQL Server 2012+ -- is to use try_convert() or try_cast():
(3956 * 2 * ASIN(
SQRT(
POWER(
SIN((53.6349994 - abs(try_convert(numeric, latitude_deg))) * pi() / 180 / 2), 2) + COS(53.6349994 * pi()/180) * COS(abs(try_convert(numeric, latitude_deg)) * pi()/180) * POWER(SIN((10.0117336 - try_convert(numeric, longitude_deg)) * pi()/180 / 2), 2) ))) AS Distance
This will prevent any conversion errors.
You shouldn't use just numeric. Use either a floating point representation or something with decimal places, say numeric(20, 10).
The reason this occurs with the order by is because of the SQL optimizer. You clearly have some lat/long values that do not convert correctly to a numeric. SQL Server allows itself to re-arrange operations, so the conversion might take place before the filtering by the where clause. This is part of the overall query optimization.
I am trying to work out this statement but can't get anywhere.
From what I have found so far, I think I need to use an nullif clause for each divisor, but when I try and do this it still does not work.
Here is the statement with no nullif's that produces a divide by zero error.
(1 - (1 - (x1.hup / (x1.hup / (x1.dp / 100)))) / (1 - (x2.sdp / 100)))
There may be simpler ways to express this. I think this is all you need:
(1 - (1 - (x1.hup / (x1.hup / nullif(x1.dp / 100, 0)))) / nullif(1 - (x2.sdp / 100)), 0))
This is Not the Answer
Create a function to check zero like this
CREATE FUNCTION [dbo].[IsZero] (
#Number FLOAT,
)
RETURNS FLOAT
AS
BEGIN
IF (#Number = 0)
BEGIN
SET #Number = 1
END
RETURN (#Number)
END
Not very elegant but will do the job.
SELECT CASE WHEN (1 - (x2.sdp / 100) = 0 THEN NULL
WHEN (x1.dp / 100) = 0 THEN NULL
WHEN (x1.hup / (x1.dp / 100)) = 0 THEN NULL
WHEN (x1.hup / (x1.hup / (x1.dp / 100))) = 0 THEN NULL
ELSE (1 - (1 - (x1.hup / (x1.hup / (x1.dp / 100)))) / (1 - (x2.sdp / 100)))
END AS field
FROM yourtable
Coalescing to zero isn't the magic bullet you're looking for.
Just try to the simplify your problem; if you have 42 / x and x is null, replacing it with zero will just result in a division error.
I don't know about the formula you are applying in this case, but the main thing you are lacking is validation. In my example above, if x equals to 0 we've got a problem.
Similarly, in 23 / (100 - x), x can't ever be 100, so you must check that beforehand and handle the situation accordingly. No expression can result in 0 if it`s a divisor.
So, try to establish which constraints should be watched before processing; for instance, your statement must not accept 0 for x1.dp or x1.hup, and x2.sdp can be anything but 100 (1 - 100/100 = 0, right?). Should one of these situations happen, you could return an error or something.
We have a fairly complicated SQL Server 2008 r2 sp2 query with this as one of the lines :-
SUM((t.Quantity * contract.ValueOfOnePoint) * ((
CASE contract.Style
WHEN 3
THEN 1 / (1.0 + ((100.0 - Val) / 100.0 * 90.0 / 365.0))
WHEN 2
THEN 1000 * (6.0 * (1.0 - (POWER((1.0 / (1.0 + ((100.0 - Val) / 200.0))), 20.0))) / ((100.0 - Val) / 200.0) + (100.0 * (POWER((1.0 / (1.0 + ((100.0 - Val) / 200.0))), 20.0))))
END
) - (
CASE contract.Style
WHEN 3
THEN 1.0 / (1.0 + ((100.0 - t.Price) / 100.0 * 90.0 / 365.0))
WHEN 2
THEN 1000 * (6.0 * (1.0 - (POWER((1.0 / (1.0 + ((100.0 - t.Price) / 200.0))), 20.0))) / ((100.00 - t.Price) / 200.00) + (100.0 * (POWER((1.0 / (1.0 + ((100.0 - t.Price) / 200.0))), 20.0))))
END
)
)) AS NativeAmount
I am testing this on a single row which has a style of 3 so only the first line in the case statement should have any affect yet leaving the "WHEN 2" clause in it reduces the accuracy of the formula.
Eg. if I remove both WHEN 2 conditions I get an answer such 123.45678 but with the WHEN 2 line left in I get 123.46. It seems to be rounding for some reason even though the second WHEN should never be in-play.
Any thoughts would be really appreciated - going mad!
Thanks.
James.
You need to combine two things. The return type of the case statement is the same for all the then and else clauses. This is a quote from the documentation:
[The case statement] returns the highest precedence type from the set of types in
result_expressions and the optional else_result_expression. For more
information, see Data Type Precedence (Transact-SQL).
So, the SQL Engine does care about all the clauses in the query (which is the answer to your question).
I don't fully understand what is happening in this case. When you call the power() function, the compiler has to decide on the precision of the numeric value, based on the constants and column types. Based on this SQL Fiddle, it chooses a precision of 38 and a scale of 1. However, simple arithmetic on the values produces a precision of 36 and a scale of 23. I'm not sure why, in the end, this results in rounding the value to two decimal places. Perhaps the logic for assigning types for the into clause doesn't quite match the logic for typing of expressions.