Parsing/Conversion issue in sql server 2012? - sql

I am getting error while executing below query in sql server 2012:
DECLARE #Id NVARCHAR(max) = 2147483648
DECLARE #Result BIGINT = 0
IF (
#Id = 0
OR #Id = ''
)
BEGIN
SET #Result = NULL
END
ELSE
BEGIN
SET #Result = convert(bigint,#id)
END
Output:
Msg 248, Level 16, State 1, Line 3 The conversion of the nvarchar
value '2147483648' overflowed an int column.
Can someone suggest me why it is happening ?
Thanks in advance

Converting 0 to bigint will work because by default 0 will be treated as INT
DECLARE #Id NVARCHAR(max) = 2147483648
DECLARE #Result BIGINT = 0
IF ( #Id = cast(0 as bigint) -- or = '0'
OR #Id = '' )
BEGIN
SET #Result = NULL
END
ELSE
BEGIN
SET #Result = CONVERT(BIGINT, #id)
END

You have to use '' when setting #id = '0' in following:
DECLARE #Id NVARCHAR(max) = 2147483648
DECLARE #Result BIGINT = 0
IF (
#Id = '0'
OR #Id = ''
)
BEGIN
SET #Result = NULL
END
ELSE
BEGIN
SET #Result = convert(bigint,#id)
END

When you compare varchar with int. varchar is converted to int before comparison.
When an operator combines two expressions of different data types, the
rules for data type precedence specify that the data type with the
lower precedence is converted to the data type with the higher
precedence. If the conversion is not a supported implicit conversion,
an error is returned.
When #ID (= '2147483648') converted to int it exceeds the max limit of int. hence you got the error;
int
-2^31 (-2,147,483,648) to 2^31-1 (2,147,483,647) 4 Bytes
Solution:
Convert to both to varchar
IF (#Id = '0' OR #Id = '' )
Convert #Id to bigint.
IF ( convert(bigint, #Id) = 0 OR #Id = '' )

Related

SQL byte truncate

My input is a varchar payload (that is actually made of hexadecimal characters). I would like to convert it to bits, then truncate the result to decode it.
I already have a built-in function (hexstrtovarbin) that works correctly and that converts varchars into varbinaries.
For example, for an input "4d", I would like to convert it into bits (01001101) then truncate the first 6 digits, before converting them into an integer (and get eventually 19).
DECLARE #payload varchar(4), #binarypayload binary(1), #converted smallint;
SET #payload = '4d';
SET #binarypayload = hexstrtovarbin(#payload);
SET #converted = CAST(SUBSTRING(#binarypayload, 1, 6) AS int)
If I proceed like this, #converted takes 77 as value. This is because #binarypayload value is "TQ==" (and not actual bits), so the substring does not truncate it.
I have tried to use bit data type, but could not store more than 1 of them.
Would anyone know how to get the actual bits in order to truncate them?
it can be messy but this is my answer, according to your data in the question:
declare #intvalue int
set #intvalue= CONVERT(int, CONVERT(varbinary(max), '4d', 2) )
declare #vsresult varchar(16)
declare #inti int
select #inti = 16, #vsresult = ''
declare #Input varchar(16)
--translating ex string in binary digits
while #inti>0
begin
select #vsresult=convert(char(1), #intvalue % 2)+#vsresult
select #intvalue = convert(int, (#intvalue / 2)), #inti=#inti-1
end
set #Input= left(#vsresult,LEN(#vsresult)-2) -- here your input string without last two digits, 00000000010011
--now return integer value
DECLARE #Cnt tinyint = 1
DECLARE #Len tinyint = LEN(#Input)
DECLARE #Output bigint = CAST(SUBSTRING(#Input, #Len, 1) AS bigint)
WHILE(#Cnt < #Len) BEGIN
SET #Output = #Output + POWER(CAST(SUBSTRING(#Input, #Len - #Cnt, 1) * 2 AS bigint), #Cnt)
SET #Cnt = #Cnt + 1
END
select #Output

Convert from bigint to varchar without leading 0?

I need to use a bigint value as part of a string construction, but I can't figure out how to get from bigint to to varchar without having a leading 0
declare #a bigint = 167830720612159876
select convert(varchar(32), convert(varbinary, #a), 2)
This gives me 02544126B47C5184, but I want `2544126B47C5184'
Basically I want the conversion from bigint to varbinary to omit the leading 0 so that it is left out of the string representation.
Try this:
create function dbo.BigintToHex(#X bigint)
returns varchar(20) as
begin
declare #H varchar(20) = ''
declare #D varchar(16) = '0123456789ABCDEF'
if #X = 0
set #H = '0'
if #X < 0
set #H = '-' + dbo.BigintToHex(-#X)
else
while #X > 0
begin
set #H = SUBSTRING(#D,#X % 16+1,1) + #H
set #X = #X / 16
end
return #H
end
go
select dbo.BigintToHex(-167830720612159876)

SQL - Why can't I compare varchar to varchar(15)?

Can anyone explain to me why the following returns 'Not equal'?
DECLARE #X15 varchar(15) = 'ABC'
DECLARE #X varchar = #X15
SELECT CASE WHEN #X = #X15 THEN 'Equal' ELSE 'Not equal' END
I must be missing something obvious.
If you print out #X you'll see the problem:
DECLARE #X15 varchar(15) = 'ABC'
DECLARE #X varchar = #X15
SELECT LEN(#X), #X, CASE WHEN #X = #X15 THEN 'Equal' ELSE 'Not equal' END
If you don't specify a length for varchar it defaults to 1 character so your #X variable can only hold the first character of #X15.
The default value of n is 1 for the char and varchar data types when they are used in variable declaration.
Ex:-
DECLARE #myVariable AS varchar = 'abc';
DECLARE #myNextVariable AS char = 'abc';
DECLARE #myVariableWithLength AS varchar(15) = 'abc';
--The following returns 1
SELECT DATALENGTH(#myVariable), DATALENGTH(#myNextVariable),DATALENGTH(#myVariableWithLength);
GO

SQL 2005 - Variant Parameter Question

I am working on a function that will be used by no less than 10 SProc's, and will probably grow once it is ironed out.
Problem i am running into is that i do not want to develop a function for each Data Type, which is why the SQL_VARIANT data type is looking pretty convenient for the action. I know is can do the ISNULL check on the data type but i also want to check to see if the Value being passed is a valid Number but the ISNUMERIC function does not work with SQL_VARIANT and I'm not too sure about the SQL_VARIANT_PROPERTY function.
Code so far:
CREATE FUNCTION dbo.mpt_Format_Number
(
#value SQL_VARIANT
, #money BIT
)
RETURNS VARCHAR
AS
BEGIN
--Check for NULL value
IF ISNULL(#value) BEGIN
-- Value IS NULL, return NULL
RETURN NULL
END ELSE BEGIN
-- Value is NOT NULL
DECLARE #TMP VARCHAR
END
END
CREATE FUNCTION dbo.mpt_Format_Number
(
#value SQL_VARIANT
, #money BIT
)
RETURNS VARCHAR
AS
BEGIN
--Check for NULL value
IF #value is null
-- Value IS NULL, return NULL
RETURN NULL
ELSE
BEGIN
-- Value is NOT NULL
if isnumeric(convert(varchar(max), #value)) = 1 RETURN 'Y' -- is valid number
--DECLARE #TMP VARCHAR
END
return 'N' --is not valid number
END
You can always test the property type with this syntax. Should be easy to incooperate in your function.
declare #t SQL_VARIANT
set #t = '3'
select SQL_VARIANT_PROPERTY(#t, 'basetype')
Result:
varchar
As a final implementation here is the full function as it was used.
ALTER FUNCTION [dbo].[_mpt_Format_Number]
(
#value SQL_VARIANT
, #money BIT = 0
)
RETURNS VARCHAR(max)
AS
BEGIN
DECLARE #ret VARCHAR(MAX)
--Check for NULL value
IF #value IS NULL BEGIN
-- Value IS NULL, return NULL
SET #ret = 'NULL'
END ELSE BEGIN
-- Value is NOT NULL
--Check for Numeric Value
IF ISNUMERIC(CONVERT(VARCHAR, #value)) = 0 BEGIN
--Value is NOT a Number, return NULL
SET #ret = 'NULL'
END ELSE BEGIN
--Value IS a Number
declare #isNeg BIT
declare #tmp varchar(max)
set #tmp = convert(varchar(max), round(cast(#value as money), 0), 1)
--Check if value is negative
if #value < 0 begin
--Value is Negative
set #isNeg = 1
--Remove the negative sign
set #tmp = replace(#tmp, '-', '')
end
--Remove the decimal plus any digits to the right of the decimal
set #tmp = left(#tmp ,len(#tmp) - 3)
--Is money set to True
if #money = 1 begin
--Pre-pend the dollar sign to value
set #tmp = '$' + #tmp
end
--Is isNeg set to True
if #isNeg = 1 begin
--Encapsulate the value with parenthesis
set #tmp = '(' + #tmp + ')'
end
SET #ret = #tmp
END
END
RETURN #ret
END

Convert integer to hex and hex to integer

So I have this query working (where signal_data is a column) in Sybase but it doesn't work in Microsoft SQL Server:
HEXTOINT(SUBSTRING((INTTOHEX(signal_data)),5,2)) as Signal
I also have it in Excel (where A1 contains the value):
=HEX2DEC(LEFT(DEC2HEX(A1),LEN(DEC2HEX(A1))-2))
Does anyone know how I would do this in SQL Server?
Convert INT to hex:
SELECT CONVERT(VARBINARY(8), 16777215)
Convert hex to INT:
SELECT CONVERT(INT, 0xFFFFFF)
Update 2015-03-16
The above example has the limitation that it only works when the HEX value is given as an integer literal. For completeness, if the value to convert is a hexadecimal string (such as found in a varchar column) use:
-- If the '0x' marker is present:
SELECT CONVERT(INT, CONVERT(VARBINARY, '0x1FFFFF', 1))
-- If the '0x' marker is NOT present:
SELECT CONVERT(INT, CONVERT(VARBINARY, '1FFFFF', 2))
Note: The string must contain an even number of hex digits. An odd number of digits will yield an error.
More details can be found in the "Binary Styles" section of CAST and CONVERT (Transact-SQL). I believe SQL Server 2008 or later is required.
Actually, the built-in function is named master.dbo.fn_varbintohexstr.
So, for example:
SELECT 100, master.dbo.fn_varbintohexstr(100)
Gives you
100 0x00000064
SQL Server equivalents to Excel's string-based DEC2HEX, HEX2DEC functions:
--Convert INT to hex string:
PRINT CONVERT(VARCHAR(8),CONVERT(VARBINARY(4), 16777215),2) --DEC2HEX
--Convert hex string to INT:
PRINT CONVERT(INT,CONVERT(VARBINARY(4),'00FFFFFF',2)) --HEX2DEC
It is possible using the function FORMAT available on SQL Server 2012 and above
select FORMAT(10,'x2')
Results in:
0a
Convert int to hex:
SELECT FORMAT(512+255,'X')
The traditonal 4 bit hex is pretty direct.
Hex String to Integer (Assuming value is stored in field called FHexString) :
CONVERT(BIGINT,CONVERT(varbinary(4),
(SELECT master.dbo.fn_cdc_hexstrtobin(
LEFT(FMEID_ESN,8)
))
))
Integer to Hex String (Assuming value is stored in field called FInteger):
(SELECT master.dbo.fn_varbintohexstr(CONVERT(varbinary,CONVERT(int,
FInteger
))))
Important to note is that when you begin to use bit sizes that cause register sharing, especially on an intel machine, your High and Low and Left and Rights in the registers will be swapped due to the little endian nature of Intel. For example, when using a varbinary(3), we're talking about a 6 character Hex. In this case, your bits are paired as the following indexes from right to left "54,32,10". In an intel system, you would expect "76,54,32,10". Since you are only using 6 of the 8, you need to remember to do the swaps yourself. "76,54" will qualify as your left and "32,10" will qualify as your right. The comma separates your high and low. Intel swaps the high and lows, then the left and rights. So to do a conversion...sigh, you got to swap them yourselves for example, the following converts the first 6 of an 8 character hex:
(SELECT master.dbo.fn_replvarbintoint(
CONVERT(varbinary(3),(SELECT master.dbo.fn_cdc_hexstrtobin(
--intel processors, registers are switched, so reverse them
----second half
RIGHT(FHex8,2)+ --0,1 (0 indexed)
LEFT(RIGHT(FHex8,4),2)+ -- 2,3 (oindex)
--first half
LEFT(RIGHT(FHex8,6),2) --4,5
)))
))
It's a bit complicated, so I would try to keep my conversions to 8 character hex's (varbinary(4)).
In summary, this should answer your question. Comprehensively.
Here is the function for SQL server which converts integer value into its hexadecimal representation as a varchar. It should be easy to adapt to other database types
For example:
SELECT dbo.ToHex(4095) --> FFF
SQL:
CREATE FUNCTION ToHex(#value int)
RETURNS varchar(50)
AS
BEGIN
DECLARE #seq char(16)
DECLARE #result varchar(50)
DECLARE #digit char(1)
SET #seq = '0123456789ABCDEF'
SET #result = SUBSTRING(#seq, (#value%16)+1, 1)
WHILE #value > 0
BEGIN
SET #digit = SUBSTRING(#seq, ((#value/16)%16)+1, 1)
SET #value = #value/16
IF #value <> 0 SET #result = #digit + #result
END
RETURN #result
END
GO
Use master.dbo.fnbintohexstr(16777215) to convert to a varchar representation.
Maksym Kozlenko has a nice solution, and others come close to unlocking it's full potential but then miss completely to realized that you can define any sequence of characters, and use it's length as the Base. Which is why I like this slightly modified version of his solution, because it can work for base 16, or base 17, and etc.
For example, what if you wanted letters and numbers, but don't like I's for looking like 1's and O's for looking like 0's. You can define any sequence this way. Below is a form of a "Base 36" that skips the I and O to create a "modified base 34". Un-comment the hex line instead to run as hex.
declare #value int = 1234567890
DECLARE #seq varchar(100) = '0123456789ABCDEFGHJKLMNPQRSTUVWXYZ' -- modified base 34
--DECLARE #seq varchar(100) = '0123456789ABCDEF' -- hex
DECLARE #result varchar(50)
DECLARE #digit char(1)
DECLARE #baseSize int = len(#seq)
DECLARE #workingValue int = #value
SET #result = SUBSTRING(#seq, (#workingValue%#baseSize)+1, 1)
WHILE #workingValue > 0
BEGIN
SET #digit = SUBSTRING(#seq, ((#workingValue/#baseSize)%#baseSize)+1, 1)
SET #workingValue = #workingValue/#baseSize
IF #workingValue <> 0 SET #result = #digit + #result
END
select #value as Value, #baseSize as BaseSize, #result as Result
Value, BaseSize, Result
1234567890, 34, T5URAA
I also moved value over to a working value, and then work from the working value copy, as a personal preference.
Below is additional for reversing the transformation, for any sequence, with the base defined as the length of the sequence.
declare #value varchar(50) = 'T5URAA'
DECLARE #seq varchar(100) = '0123456789ABCDEFGHJKLMNPQRSTUVWXYZ' -- modified base 34
--DECLARE #seq varchar(100) = '0123456789ABCDEF' -- hex
DECLARE #result int = 0
DECLARE #digit char(1)
DECLARE #baseSize int = len(#seq)
DECLARE #workingValue varchar(50) = #value
DECLARE #PositionMultiplier int = 1
DECLARE #digitPositionInSequence int = 0
WHILE len(#workingValue) > 0
BEGIN
SET #digit = right(#workingValue,1)
SET #digitPositionInSequence = CHARINDEX(#digit,#seq)
SET #result = #result + ( (#digitPositionInSequence -1) * #PositionMultiplier)
--select #digit, #digitPositionInSequence, #PositionMultiplier, #result
SET #workingValue = left(#workingValue,len(#workingValue)-1)
SET #PositionMultiplier = #PositionMultiplier * #baseSize
END
select #value as Value, #baseSize as BaseSize, #result as Result
Declare #Dato xml
Set #Dato = Convert(xml, '<dato>FF</dato>')
Select Cast( rw.value( 'xs:hexBinary( text()[1])' , 'varbinary(max)' ) as int ) From #Dato.nodes('dato') as T(rw)
The answer by Maksym Kozlenko is nice and can be slightly modified to handle encoding a numeric value to any code format. For example:
CREATE FUNCTION [dbo].[IntToAlpha](#Value int)
RETURNS varchar(30)
AS
BEGIN
DECLARE #CodeChars varchar(100)
SET #CodeChars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
DECLARE #CodeLength int = 26
DECLARE #Result varchar(30) = ''
DECLARE #Digit char(1)
SET #Result = SUBSTRING(#CodeChars, (#Value % #CodeLength) + 1, 1)
WHILE #Value > 0
BEGIN
SET #Digit = SUBSTRING(#CodeChars, ((#Value / #CodeLength) % #CodeLength) + 1, 1)
SET #Value = #Value / #CodeLength
IF #Value <> 0 SET #Result = #Digit + #Result
END
RETURN #Result
END
So, a big number like 150 million, becomes only 6 characters (150,000,000 = "MQGJMU")
You could also use different characters in different sequences as an encrypting device. Or pass in the code characters and length of characters and use as a salting method for encrypting.
And the reverse:
CREATE FUNCTION [dbo].[AlphaToInt](#Value varchar(7))
RETURNS int
AS
BEGIN
DECLARE #CodeChars varchar(100)
SET #CodeChars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
DECLARE #CodeLength int = 26
DECLARE #Digit char(1)
DECLARE #Result int = 0
DECLARE #DigitValue int
DECLARE #Index int = 0
DECLARE #Reverse varchar(7)
SET #Reverse = REVERSE(#Value)
WHILE #Index < LEN(#Value)
BEGIN
SET #Digit = SUBSTRING(#Reverse, #Index + 1, 1)
SET #DigitValue = (CHARINDEX(#Digit, #CodeChars) - 1) * POWER(#CodeLength, #Index)
SET #Result = #Result + #DigitValue
SET #Index = #Index + 1
END
RETURN #Result
Given:
declare #hexStr varchar(16), #intVal int
IntToHexStr:
select #hexStr = convert(varbinary, #intVal, 1)
HexStrToInt:
declare
#query varchar(100),
#parameters varchar(50)
select
#query = 'select #result = convert(int,' + #hb + ')',
#parameters = '#result int output'
exec master.dbo.Sp_executesql #query, #parameters, #intVal output
Below are two functions: dbo.HexToInt and dbo.IntToHex, I use them for such conversion:
if OBJECT_ID('dbo.HexToInt') is not null
drop function dbo.HexToInt
GO
create function dbo.HexToInt (#chars varchar(max))
returns int
begin
declare #char varchar(1), #len int, #i int, #r int, #tmp int, #pow int
set #chars = RTRIM(LTRIM(#chars))
set #len = LEN(#chars)
set #i = 1
set #r = 0
while #i <= #len
begin
set #pow = #len - #i
set #char = SUBSTRING(#chars, #i, 1)
if #char = '0'
set #tmp = 0
else if #char = '1'
set #tmp = 1
else if #char = '2'
set #tmp = 2
else if #char = '3'
set #tmp = 3
else if #char = '4'
set #tmp = 4
else if #char = '5'
set #tmp = 5
else if #char = '6'
set #tmp = 6
else if #char = '7'
set #tmp = 7
else if #char = '8'
set #tmp = 8
else if #char = '9'
set #tmp = 9
else if #char = 'A'
set #tmp = 10
else if #char = 'B'
set #tmp = 11
else if #char = 'C'
set #tmp = 12
else if #char = 'D'
set #tmp = 13
else if #char = 'E'
set #tmp = 14
else if #char = 'F'
set #tmp = 15
set #r = #r + #tmp * POWER(16,#pow)
set #i = #i + 1
end
return #r
end
And the second one:
if OBJECT_ID('dbo.IntToHex') is not null
drop function dbo.IntToHex
GO
create function dbo.IntToHex (#val int)
returns varchar(max)
begin
declare #r varchar(max), #tmp int, #v1 int, #v2 int, #char varchar(1)
set #tmp = #val
set #r = ''
while 1=1
begin
set #v1 = #tmp / 16
set #v2 = #tmp % 16
if #v2 = 0
set #char = '0'
else if #v2 = 1
set #char = '1'
else if #v2 = 2
set #char = '2'
else if #v2 = 3
set #char = '3'
else if #v2 = 4
set #char = '4'
else if #v2 = 5
set #char = '5'
else if #v2 = 6
set #char = '6'
else if #v2 = 7
set #char = '7'
else if #v2 = 8
set #char = '8'
else if #v2 = 9
set #char = '9'
else if #v2 = 10
set #char = 'A'
else if #v2 = 11
set #char = 'B'
else if #v2 = 12
set #char = 'C'
else if #v2 = 13
set #char = 'D'
else if #v2 = 14
set #char = 'E'
else if #v2 = 15
set #char = 'F'
set #tmp = #v1
set #r = #char + #r
if #tmp = 0
break
end
return #r
end
IIF(Fields!HIGHLIGHT_COLOUR.Value="","#FFFFFF","#" & hex(Fields!HIGHLIGHT_COLOUR.Value) & StrDup(6-LEN(hex(Fields!HIGHLIGHT_COLOUR.Value)),"0"))
Is working for me as an expression in font colour
To convert Hex strings to INT, I have used this in the past. It can be modified to convert any base to INT in fact (Octal, Binary, whatever)
Declare #Str varchar(200)
Set #str = 'F000BE1A'
Declare #ndx int
Set #ndx = Len(#str)
Declare #RunningTotal BigInt
Set #RunningTotal = 0
While #ndx > 0
Begin
Declare #Exponent BigInt
Set #Exponent = Len(#Str) - #ndx
Set #RunningTotal = #RunningTotal +
Power(16 * 1.0, #Exponent) *
Case Substring(#str, #ndx, 1)
When '0' then 0
When '1' then 1
When '2' then 2
When '3' then 3
When '4' then 4
When '5' then 5
When '6' then 6
When '7' then 7
When '8' then 8
When '9' then 9
When 'A' then 10
When 'B' then 11
When 'C' then 12
When 'D' then 13
When 'E' then 14
When 'F' then 15
End
Set #ndx = #ndx - 1
End
Print #RunningTotal