I need to calculate the remainder of one number divided by another. for instance with these numbers:
271011240311350356232122 % 97
When I just want to do that in a sql statement, it works like a charm:
select 271011240311350356232122 % 97;
But when I have that large number in a varchar variable, I can't seem to get the job done.
I can't convert it into a int or even bigint, because it's too large. I can't convert it into a real because you can't use the modulo operator on a real number.
Any ideas...?
If it is too big for bigint you can use NUMERIC(38,0)
DECLARE #Num VARCHAR(38) = '271011240311350356232122'
SELECT CAST(#Num AS NUMERIC(38,0)) % 97
Related
I am working on the usage of the Cast function in SQL Server. If I use explicit value as written in the code. The output is 38 which is the correct value. But I would need to use field names instead of direct values(There is only value of Base, escalator here ; Base =1.15 and escalator=0.05. But when I use field names, the output is 37. The data type of Base and escalator fields is float. I also tried using round function inside cast, did not solve the issue. Could someone out here help me with this. My query below:
Select CAST((3.05-1.15)/0.05 AS INT) -- returns 38
Select ((3.05-1.15)/0.05) --returns 38
Select cast((3.05-base)/Escalator) as int) from table1 -- I am using field names here. Returns 37
What is likely happening here is that the base and Escalator columns are some kind of non exact floating point type. As a result, the following computation results a value which is slightly less than 38:
(3.05-base) / Escalator = 37.999995 (for example)
Then, when casting to integer, the entire decimal component is being truncated, leaving behind just 37.
One possible workaround to prevent this from happening would be to use NUMERIC or some other exact type for the base and Escalator columns.
You can use Decimal to get rid of the issue
DECLARE #Escalator DECIMAL(7, 5) = 0.05
Select ((3.05-1.15)/0.05) --returns 38
Select CAST(((3.05-1.15)/#Escalator) AS INT) -- returns 38
Demo on db<>fiddle
you can use ceiling or floor inbuilt function based on your requirement
DECLARE #Escalator float = 0.05
DECLARE #Base float = 1.66
Select ((3.05-1.66)/0.05) --returns 27.8
Select ceiling (((3.05-#Base)/#Escalator)) -- returns 28
Select floor (((3.05-#Base)/#Escalator)) -- returns 27
DECLARE #TAX VARCHAR(30)=120.45
DECLARE #TaxRoundOf VARCHAR(30)
SET #TaxRoundOf=ROUND(#TAX,1)
SELECT #TaxRoundOf
This Gives Result (#TaxRoundOf=120.5)
DECLARE #TAX VARCHAR(30)=146.45
DECLARE #TaxRoundOf VARCHAR(30)
SET #TaxRoundOf=ROUND(#TAX,1)
SELECT #TaxRoundOf
This Gives Result (#TaxRoundOf=146.4)
But I need to return 146.50 . why this mismatch between two results?
any one can help plz?
Since you are using VARCHAR to store your numbers, SQL Server is having to do implicit conversion to float behind the scenes, which is having knock on effects on your calculations. You can reproduce this using the below query:
SELECT ROUND(CONVERT(FLOAT, 120.45),1), -- 120.5
ROUND(CONVERT(FLOAT, 146.45),1), -- 146.4
ROUND(CONVERT(DECIMAL(10, 2), 120.45),1), -- 120.50
ROUND(CONVERT(DECIMAL(10, 2), 146.45),1) -- 146.50
Since floating point numbers are not exact, 146.45 cannot be exactly represented as a float, and ends up being stored as a very slightly smaller number, so when this is passed to the round function, it is rounded down, instead of up.
The solution, as demonstrated by the 3rd and 4th columns in the above query, is to use a more precise data type.
You can use this:
SET #TaxRoundOf=ROUND(10 * CAST(#TAX AS FLOAT)) / 10
instead of:
SET #TaxRoundOf=ROUND(#TAX,1)
DEMO
PS as #GarethD already mentioned I wouldn't use #TAX as VARCHAR type.
You can also rely on numeric rounding instead of converting your string to a float, which can lose information.
Cast a string to numeric and then round:
select round(cast('146.45' as numeric(18,2)), 1)
-- 146.50
A decimal constant is already a decimal so there's no need to cast it:
select round(146.45, 1)
-- 146.50
We are undergoing a migration in our tool. Some of the datatypes in our tables have been changed from real to float. To understand the impact I have run the following code:
declare #realVariable as real,
#floatVariable as float
set #realVariable=152304.11999512
set #floatVariable=152304.11999512
select cast(#realVariable AS decimal(15, 4)) as realcol,str(#floatVariable,15,4) as floatcol
The result is different for both real and float variables:
real=152304.1250
float= 152304.1200
How can I make sure that the float result is same as real?
float and real are both approximate-number data types for floating point numeric data.
real is the same as float(24). If you declare float without a suffix, then it means float(53).
So, you are converting from real (which is float(24)) to float(53). You are simply making the numbers more precise.
float(24) has a precision of 7 digits and float(53) has a precision of 15 digits.
See Books Online > float and real (Transact-SQL): https://msdn.microsoft.com/en-GB/library/ms173773.aspx
If we repeat your example and display the variables without conversion, we can see that the real variable is displayed with 7 digits precision, and float variable is displayed with 14 digits precision, which is exactly what you set it to.
DECLARE #realVariable as real,
#floatVariable as float;
-- Set to a number with 14 digits precision
SET #realVariable=152304.11999512;
SET #floatVariable=152304.11999512;
SELECT #realVariable AS '#realVariable', #floatVariable AS '#floatVariable';
The simple answer to your question is that the two numbers are not the same. The one with float(53) is more precise.
Calculating the ABS of the difference of the two numbers and then comparing this to an acceptable threshold is one way to do it. However this would not be satisfactory if the scale can vary widely.
To illustrate why this might not be satisfactory:
r = 0.0001234567
f = 0.00012345678901234
Comparing the difference to a threshold of 0.0005 would not be useful.
So it might be better to use a threshold that is a percentage of the number.
DECLARE #threshold AS float, #delta AS float;
SET #threshold = ABS(#floatVariable / 1000000);
SET #delta = ABS(#floatVariable - CAST(#realVariable AS float));
SELECT #realVariable AS '#realVariable',
#floatVariable AS '#floatVariable',
#threshold AS '#threshold',
#delta AS '#delta',
CASE
WHEN #delta < #threshold THEN N'OK'
ELSE N'Different'
END AS 'Comparison';
Note: This solution might need a bit of adjustment for numbers that are very close to zero.
#RichardCL's description describes the way things work (+1). Now, if you are converting a value stored as a real datatype value into a value stored as a float datatype value, then the entire and precise "original" value will be stored in the resulting value--you will lose no information from your existing set of real values.
Referring to your example, your value of 152304.11999512 cannot be stored precisely as a real value--it will be rounded and stored as 152304.125. Convert this to a float, and you will still have 152304.125.
You need a delta. Compute the absolute value of the difference between the real and float values then make sure that is less than your delta. The delta should be sufficiently small to meet your needs.
For example using your values:
abs(152304.1250 - 152304.1200) <= .0050 so the use .0050 as your delta value.
Using SQL Server 2000
I want to reduce the decimal length
Query
Select 23/12 as total
Output is showing as 1.99999999999
I don't want to round the value, I want to diplay like this 1.99
Tried Query
Select LEFT(23/12, LEN(23/12) - 3) as total
The above query is working only if there is decimal value like 12.444444, but if the total is single digit means like 12 or 4 or 11...., i am getting error at run time.
How to do this.
Need Query Help
There is a very simple solution. You can find it in BOL. Round takes an optional 3rd argument, which is round type. The values are round or truncate.
ROUND numeric_expression , length [ ,function ] )
...
function Is the type of operation to perform. function must be
tinyint, smallint, or int. When function is omitted or has a value of
0 (default), numeric_expression is rounded. When a value other than 0
is specified, numeric_expression is truncated.
So just do
Select ROUND(cast(23 as float)/12, 2, 1) as total
That gives 1.91. Note, if you were really seeing 1.999 - something is really wrong with your computer. 23/12 = 1.916666666(ad infinitum). You need to cast one of the numbers as float since sql is assuming they're integers and doing integer division otherwise. You can of course cast them both as float, but as long as one is float the other will be converted too.
Not terribly elegant, but works for all cases:
Select CONVERT(float,LEFT(CONVERT(nvarchar, 23.0/12.0),CHARINDEX('.',CONVERT(nvarchar, 23.0/12.0)) + 2)) as total
Scalar Function
-- Description: Truncate instead of rounding a float
-- SELECT dbo.TruncateNumber(23.0/12.0,2)
-- =============================================
CREATE FUNCTION TruncateNumber
(
-- Add the parameters for the function here
#inFloat float,
#numDecimals smallint
)
RETURNS float
AS
BEGIN
IF (#numDecimals < 0)
BEGIN
SET #numDecimals = 0
END
-- Declare the return variable here
RETURN CONVERT(float,LEFT(CONVERT(nvarchar, #inFloat),CHARINDEX('.',CONVERT(nvarchar, #inFloat)) + #numDecimals))
END
GO
declare #fieldForceCounter as int
declare #SaleDate as dateTime
declare #RandomNoSeed as decimal
set #fieldForceCounter = 1
set #SaleDate = '1 Jan 2009'
set #RandomNoSeed = 0.0
WHILE #fieldForceCounter <= 3
BEGIN
while #SaleDate <= '1 Dec 2009'
begin
INSERT INTO MonthlySales(FFCode, SaleDate, SaleValue) VALUES(#fieldForceCounter, #SaleDate, RAND(#RandomNoSeed))
set #saleDate = #saleDate + 1
set #RandomNoSeed = Rand(#RandomNoSeed) + 1
end
set #SaleDate = '1 Jan 2009'
set #fieldForceCounter = #fieldForceCounter + 1
END
GO
This T-SQL command was supposed to insert random values in the 'SaleValue'-column in the 'MonthlySales'-table.
But it is inserting '1' every time .
What can be the problem?
Two problems:
Firstly, the rand() function returns a number between 0 and 1.
Secondly, when rand() is called multiple times in the same query (e.g. for multiple rows in an update statement), it usually returns the same number (which I suspect your algorithm above is trying to solve, by splitting it into multiple calls)
My favourite way around the second problem is to use a function that's guaranteed to return a unique value each time, like newid(), convert it to varbinary, and use it as the seed :)
Edit: after some testing, it seems you'll need to try using a different datatype for #RandomNoSeed; float behaves somewhat different to decimal, but still approaches a fixed value, so I'd recommend avoiding the use of #RandomNoSeed altogether, and simply use:
INSERT INTO MonthlySales(FFCode, SaleDate, SaleValue)
VALUES(#fieldForceCounter, #SaleDate, RAND(convert(varbinary,newid())))
You have major issues here...
Decimal issues
The default precision/scale for decimal is 38,0. So you aren't storing any decimal part.
So you are only using RAND(0) for 1st iteration and RAND(1) for all subsequent iterations, which is 0.943597390424144 and 0.713591993212924
I can't recall how rounding/truncation applies, and I don't know what datatype SalesValue is, but rounding would give "1" every time.
Now, if you fix this and declare decimal correctly...
Seeding issues
RAND takes an integer seed. Seeding with 1.0001 or 1.3 or 1.999 gives the same value (0.713591993212924).
So, "Rand(1.713591993212924) + 1" = "RAND(1) + 1" = "1.713591993212924" for every subsequent iteration.
Back to square one...
To fix
Get rid of #RandomNoSeed
Either: Generate a random integer value using CHECKSUM(NEWID())
Or: generate a random float value using RAND() * CHECKSUM(NEWID()) (Don't care about seed now)
Just a guess, but often rand functions generate a number from 0-1. Try multiplying your random number by 10.