I try to make IP to country query.
Simplest way is make query like this:
select Country
from dbip
where 8 * power(255,3) +
8 * power(255,2) +
8 * power(255,1) + 8
between FromIPInt and ToIPInt
It works for small Google's IP, but for greater IP:
Arithmetic overflow error converting expression to data type int.
Which other way I can compute it?
CAST is the solution, but you have do put it in the right place :
select Country
from dbip
where cast(255 as bigint) * power(255,3)
+ 255 * power(255,2)
+ 255 * power(255,1)
+ 255
between FromIPInt and ToIPInt
The error Arithmetic overflow error converting expression to data type int occurs if you CAST the whole calculation:
select cast(255 * power(255,3) as bigint)
But not if you cast your first number
select cast(255 as bigint) * power(255,3)
More details about that: https://msdn.microsoft.com/en-us/library/ms187745.aspx
Integer constants greater than 2,147,483,647 are converted to the decimal data type, not the bigint data type.
SQL Server does not automatically promote other integer data types (tinyint, smallint, and int) to bigint.
Related
I have a field in SQL Server 2014 that I am working with that looks like this:
**RawField**
20060202
20060323
I want to add a split the field and add a decimal point and create a numerical field. This is what I would like to see:
**RawField**
200602.02
200603.23
So I need to split the field, add the decimal point, and convert to a numerical value. I tried some code but was getting an error. Please see my code below:
select top 1000 cast(SUBSTRING(cast(RawField as varchar(6)),1,6) + cast('.' as varchar(1)) + SUBSTRING(cast(RawField as varchar(2)),6,2) as int)
from Table
I get an error of:
Msg 245, Level 16, State 1, Line 11
Conversion failed when converting the varchar value '200602.' to data type int.
Is this a good approach?
you want to convert the string to numeric with 2 decimal places ?
select convert(decimal(10,2), RawField) / 100.0
I guest your RawField contains other alphanumeric after that and you only posted the first 8 characters ?
this should work. Just take the first 8 characters and convert. Simple and direct
select convert(decimal(10,2), left(RawField, 8)) / 100.0
Please try this.
select top 1000 cast(SUBSTRING(cast(RawField as varchar(6)),1,6) + cast('.' as varchar(1)) + SUBSTRING(cast(RawField as varchar(2)),6,2) as numeric(8,2))
from Table
You are trying to cast string with decimal number to int.
Use float/numeric/decimal when casting
select cast(SUBSTRING(RawField ,1,6) + cast('.' as varchar(1)) + SUBSTRING(RawField ,7,2) as numeric(16,2))
So if I have a data (varchar) like say 10.1
I need the value as 0000101000000.
means (000010) whole number and (1000000) decimal value.
Its a 13 character string ,numbers coming before decimal point should be in first 6 characters and numbers coming after decimal point should be in last 7 characters
Maybe..?
DECLARE #d decimal(13,7) = 10.1;
SELECT RIGHT('0000000000000' + CONVERT(varchar(13),CONVERT(bigint,(#d * 10000000))),13);
Using my crystal ball here though.
Edit: As, for some reason, the OP is storing a decimal as a varchar (this is a really bad bad idea on it's own), I have added further logic to attempt to convert the value to a decimal first.
As experience has taught many of us, give a user a non-numeric column to store a numeric value in and they're more than happily store a non-numeric value in it, so i have used TRY_CONVERT and assumed you are using SQL Server 2012+:
DECLARE #d varchar(13) = 10.1;
SELECT RIGHT('0000000000000' + CONVERT(varchar(13),CONVERT(bigint,(TRY_CONVERT(decimal(13,7),#d) * 10000000))),13);
SELECT REPLICATE('0',6-LEN(SUBSTRING(CAST([data] AS VARCHAR), 1,
CHARINDEX('.',CAST([data] AS VARCHAR)) -1)))+SUBSTRING(CAST([data] AS VARCHAR), 1,
CHARINDEX('.',CAST([data] AS VARCHAR)) -1)+
SUBSTRING(CAST([data] AS VARCHAR), CHARINDEX('.',CAST([data] AS VARCHAR)) + 1,
LEN(CAST([data] AS VARCHAR)))+REPLICATE('0',7-LEN(SUBSTRING(CAST([data] AS VARCHAR), CHARINDEX('.',CAST([data] AS VARCHAR)) + 1,
LEN(CAST([data] AS VARCHAR))))) AS Whole
FROM Table1
Output
Whole
0000101000000
Demo
http://sqlfiddle.com/#!18/8649d/16
You can use some math and string operations to do it like below
see live demo
declare #var decimal(10,4)
set #var=10.1
select #var,
right(cast(cast(( floor(#var)+ power(10,7)) as int) as varchar(13)),6)
+
cast(cast(((#var- floor(#var)) * power(10,7)) as int) as varchar(13))
There's a fair amount of string manipulation to be done here. I'll step through what I did.
I used a variable for the base number so I could verify different results:
declare #n decimal(9,3) = 10.1
You need 6 spaces left of the decimal and 7 spaces to the right, so I'm doing all the manipulation on a VARCHAR(13). I didn't create a new variable as a VARCHAR because I'm assuming you want to be able to do this conversion in line on the fly, so I'm using that CAST over and over again.
Start by finding the decimal place.
SELECT CHARINDEX('.',CAST(#n as VARCHAR(13)))
In the sample number, that's a 3, but it could obviously change.
Now, get the portion of the number to the left of the decimal place.
SELECT SUBSTRING(CAST(#n as VARCHAR(13)),1,CHARINDEX('.',CAST(#n as VARCHAR(13)))-1)
Then get the portion to the right of the decimal.
SELECT SUBSTRING(CAST(#n as VARCHAR(13)),CHARINDEX('.',CAST(#n as VARCHAR(13)))+1,LEN(CAST(#n as VARCHAR(13))))
Pad the leading zeroes. Put 6 on, concatenate, and take a RIGHT 6. Accounts for no digits to the left of the decimal.
SELECT RIGHT(REPLICATE(0,6) + SUBSTRING(CAST(#n as VARCHAR(13)),1,CHARINDEX('.',CAST(#n as VARCHAR(13)))-1), 6)
Pad the trailing zeroes. Same idea, but in the other direction.
SELECT LEFT(SUBSTRING(CAST(#n as VARCHAR(13)),CHARINDEX('.',CAST(#n as VARCHAR(13)))+1,LEN(CAST(#n as VARCHAR(13)))) + REPLICATE(0,7),7)
Then put it all together.
SELECT RIGHT(REPLICATE(0,6) + SUBSTRING(CAST(#n as VARCHAR(13)),1,CHARINDEX('.',CAST(#n as VARCHAR(13)))-1), 6)
+
LEFT(SUBSTRING(CAST(#n as VARCHAR(13)),CHARINDEX('.',CAST(#n as VARCHAR(13)))+1,LEN(CAST(#n as VARCHAR(13)))) + REPLICATE(0,7),7)
Results.
0000101000000
declare #var varchar(20) = '10000.112'
SELECT FORMAT (FLOOR(#var), '000000') + left((PARSENAME(#var,1)) + replicate('0',7),7)
I am trying to do the following calculation:
DECLARE #X Decimal(18,6);
DECLARE #Y Decimal(20,16);
SET #X = 0.002700;
SET #Y = 0.0027770000000000;
SELECT 200 * ( POWER(#Y, ( -1 / ( 2 * #X ))) - 1 );
However this results in:
Msg 8115, Level 16, State 2, Line 7
Arithmetic overflow error converting expression to data type float.
I have tried casting to float but to no avail.
Any ideas what I am missing here?
A float data type can store a maximum value of about 1.79E+308, but the result of this expression is approximately 5.16E+475. See http://tinyurl.com/oy7deu8
May be there is a way to do this. You are trying to calculate #y power (-1/2*#x). This is equivalent to calculating e power (log(y)*(-1/2x)).
You could calculate it as
select 200 * POWER(2.718281828, log(#Y) * (-1/(2*#X)))
Note that LOG by default calculates the natural logarithm of the number. And e = 2.718281828
Is it possible to concatenate after using a Math equation?
I have a simple calculation running like so
Value / 100 * 0.5 + Value as Total Value
example would be 3000 / 100 * 0.50 = 3015.
But I would like to include a star with it.
I.e.
Value / 100 * 0.5 + Value + ' *' as Total Value
to display as 3015 *
However I'm getting an error saying
Error converting data type varchar to numeric.
Is here a way around this? Would it be possible to then convert the total value to a varchar and concatenate the star?
You cannot just concatenate together numerical and string values - so you need to do:
SELECT CAST((Value / 100 * 0.5 + Value) AS VARCHAR(20)) + ' *' as Total Value
First determine your result from your computation, then cast it to a varchar large enough to hold the result, and concatenate that casted string with the star string literal.
I have a stored procedure which calculates the distance between two coordinate pairs as a float. I'm trying to use this to filter a list of values but getting an arithmetic overflow error. The query is:
SELECT * FROM Housing h WHERE convert(float, dbo.CalculateDistance(35, -94, h.Latitude, h.Longitude)) <= 30.0
Which errors with:
Msg 8115, Level 16, State 6, Line 1 Arithmetic overflow error
converting float to data type numeric.
The stored procedure for reference:
CREATE FUNCTION [dbo].[CalculateDistance]
(#Longitude1 DECIMAL(8,5),
#Latitude1 DECIMAL(8,5),
#Longitude2 DECIMAL(8,5),
#Latitude2 DECIMAL(8,5))
RETURNS FLOAT
AS
BEGIN
DECLARE #Temp FLOAT
SET #Temp = SIN(#Latitude1/57.2957795130823) * SIN(#Latitude2/57.2957795130823) + COS(#Latitude1/57.2957795130823) * COS(#Latitude2/57.2957795130823) * COS(#Longitude2/57.2957795130823 - #Longitude1/57.2957795130823)
IF #Temp > 1
SET #Temp = 1
ELSE IF #Temp < -1
SET #Temp = -1
RETURN (3958.75586574 * ACOS(#Temp) )
END
've also tried converting the result to decimal with no effect.
Your inputs are DECIMAL(8,5). This means that the equations consist of, for example, SIN(DECIMAL(8,5) / 57.2957795130823). Where 57.2957795130823 can not be represented as a DECIMAL(8,5).
This means that you have an implicat CAST operation due to the different data type. In this case, it would seem that the 57.2957795130823 is being cast to DECIMAL(8,5) [a numeric], and causing the overflow.
I would recommend any of these:
- Altering your function to take the inputs as FLOATS. Even if the function is called with numerics
- Changing 57.2957795130823 to 57.29577
- Explicitly casting the DECIMALs to FLOATs
I would try converting some of my arithmetic just in case
convert(float,(SIN(#Latitude1/57.2957795130823)) * convert(float,(SIN(#Latitude2/57.2957795130823)) + convert(float,(COS(#Latitude1/57.2957795130823)) * convert(float,(COS(#Latitude2/57.2957795130823)) * convert(float,COS(#Longitude2/57.2957795130823 - #Longitude1/57.2957795130823))
another thing you could use is the
IFNULL(convert(float,(SIN(#Latitude1/57.2957795130823)),0.00)
your results may be returning nulls
It's your comparison to <= 30.0
30.0 is decimal(3,2) (Constants with decimal points are decimal in SQL Server) and the float output won't cast. See:
SELECT 30.0 AS What INTO dbo.DataType
Go
SELECT t.name, c.*
FROM sys.columns c JOIN sys.types t ON c.system_type_id = t.system_type_id
WHERE object_id = OBJECT_ID('dbo.DataType')
GO
DROP TABLE dbo.DataType
GO
Try
... <= CAST(30.0 As float)
You're returning a float. Shouldn't you be using floats for the latitude and longitude variables as well?