How to convert an eight hexadecimal bytes (16 digits) into decimal value in SQL Server? - sql

Question

We are attempting to import a SAS7BDAT file into a SQL Server database.
The only issue we're running into is that source decimal values are being read as float.
Using a command line tool named dsread, we found an option that:
Converting the IEEE floating-point numeric values in the SAS7BDAT file to their decimal representation may cause a loss of precision. To get a lossless representation of the data, use the /l flag:
sashelp> dsread /v /l prdsale
ACTUAL,PREDICT,COUNTRY,...
0x0000000000e88c40,0x0000000000908a40,CANADA...
0x0000000000388f40,0x0000000000907240,CANADA...
0x0000000000008340,0x0000000000708a40,CANADA...
0x0000000000108440,0x0000000000a88040,CANADA...
0x0000000000808440,0x0000000000308440,CANADA...
0x0000000000a08d40,0x0000000000607e40,CANADA...
...etc...
The numerics are output as eight hexadecimal bytes (16 digits) giving the internal floating-point representation, which can then be used to reconstruct the exact same value in the receiving software. Use /L to get the bytes in big-endian order
Running some tests, we can see that the decimal value -1.457263 is being represented by the value 0xcbbbea01f350f7bf when we use that /L flag. What we haven't been able to figure out is, how can we convert that hexadecimal value into a SQL Server decimal value?
We've tried many variants, including:
select CONVERT(decimal, convert(varbinary,'0xcbbbea01f350f7bf',1))
but that results in:
Msg 8115, Level 16, State 6, Line 17
Arithmetic overflow error converting varbinary to data type numeric.

Answer 1

If 0xcbbbea01f350f7bf = -1.457263, then it looks like that's a IEEE double-precision floating point number with a big endian byte order.
So reverse the bytes (or get it to export in little endian instead)
0xcbbbea01f350f7bf -> 0xbff750f301eabbcb
Then convert it to a float. You can use CLR or there's a TSQL function you can try here:
Unpacking a binary string with TSQL
Then convert it to a decimal.
select convert(decimal(36,17), dbo.[fnBinaryFloat2Float]( 0xbff750f301eabbcb ), 3)
Which, you can see has preserved a closer approximation to the floating point value
-1.45726299999999998
Is there a way to reconstruct the "exact same value" with no differences?
Then leave the data as float(53) which is exactly the same data type from the source, and don't convert it to decimal at all. decimal and float each store finite subsets of the rational numbers, and many numbers can be exactly represented in either system. But some float values don't have an exact match in decimal, and vice versa.