I am currently outputting values out to 6 decimal places, and would like to round up the 6th place regardless of the integer value.
I have been using a CEILING() function so far which has worked great for values 1-9 on rounding up; however, in situations where I have the 7th decimal as 0 (ex: 2705.1520270), the function does not round up to 2705.152028.
select CEILING(price*1000000)/1000000 as PriceRound
from tc_alcf a (nolock)
Here is one approach:
SELECT ROUND(2705.1520270 + 0.0000005, 6);
2705.1520280
Demo
We can add 0.0000005 to the input and then just use SQL Server's ROUND function to 6 decimal places. This works because values with a sixth decimal place between 0 and 0.4999 (repeating) would become 5 to 0.9999 (repeating), meaning they would round up to the next digit. And values with already have 5 or greater in the sixth decimal place would not be bumped up to the next digit.
This problem should be familiar to many developers as the rounding half up problem.
Add 1 and use FLOOR():
select floor(price*1000000 + 1)/1000000 as PriceRound
from tc_alcf a
Or you can also shift the decimal by multiplying with the power function
CEILING(2705.1520275 * POWER(10,6)) / POWER(10,6)
Related
Projected code is used to convert a date into integer and vice-versa. I want to know the reason why here we have used this specific hexadecimal codes and the number series to get back the date from int. If there is an article about this code sample it would also help me understand this code actually.
I have tried online Hex to Decimal conversion for this codes and found its a 256^1,256^2... even though trying not able to find the exact reason.
declare #dDate date = '2017-10-12'
declare #iDate int = 0
select #iDate = ( (datepart(year,#dDate)*65536 | datepart(month,#dDate)*256 | datepart(dd,#dDate)))
select (#iDate&0xfff0000)/65536 --year
select (#iDate&0xff00)/256 --Month
select (#iDate&0xff) --Date
& is an operator doing bitwise AND. "|" is bitwise OR. See here and here. Also see here for an explanation on using bitwise AND/OR to store multiple number values in a single number column.
This part:
#iDate&0xfff0000
will "mask", or eliminate/replace-with-zeros, the portion of iDate that isn't from 256^2. Then you divide by 65536 -- which is simply reversing the original math of multiplying the year by 65536.
If the concept of bitwise AND is foreign, I'll give an example that DOESN'T WORK in decimal. Bitwise AND converts the whole thing to binary and then masks things (like IP subnetting, if you're familiar with that).
Anyway, consider a decimal number 20171012. If such a thing as a decimal-wise AND existed, it could look like 20171012&11110000. The "1" places are "keepers" and the "0" places are "throw-aways". If you stack them vertically, the result is to keep the values with a "1" beneath them and replace the values with a "0" beneath them with a "0".
number 20171012
dec-wise AND 11110000
result 20170000
now the result isn't 2017, so you'd have to divide by 10000 to get 2017.
For 20171012&1100 you have to use implied leading zeros:
number 20171012
dec-wise AND 00001100
result 1000
I probably would have converted to int by adding the year*10000 and month * 100 and day. Reverting back I would use a combination of integer division and MOD. But I think the bitwise AND is perhaps a bit more elegant (particularly for getting the month).
Based on your comment, I will include how I have converted dates to int and reverted back:
declare #dDate date = '2017-10-12'
declare #iDate int
set #iDate = year(#dDate) * 10000 + month(#dDate) * 100 + day(#dDate)
select #iDate
select 'year', #iDate/10000 -- basic integer division provides the year
select 'month', (#iDate % 10000)/100 -- combine modulo and integer division to get the month
select 'day', #iDate % 100 -- basic modulo arithmetic provides the day
returns:
20171012
year 2017
month 10
day 12
This is bit manipulation.
Bit Shifting
Decimal 3 = Binary 11
If we do a left shift (<<) 4 bits in 3 it will become 48 which is equal to binary 110000 <- 4 zero bits added due to left shift
But since we don't have bit shifting operators in T-SQL therefore we can do the math.
Left Shifting of n bits in number x = x * 2^n
Therefore, multiple a number with 256 is actually left shift 8 bits from that number (2^8 = 256).
Later on when you do bitwise OR between 2 numbers they actually "concatenate" the bits up.
For example, you need to concatenate 2 binary numbers, (3) 11 and (2) 10, the resultant number should be 1110 = 14
So first we'll do 2 left shift in 3 = 3 * 2^2 = 12 and then we will do bitwise OR this number with the next number
12 = 1100
2 = 0010
OR
---------------
14 = 1110
Your example is actually saving the whole date in an integer variable which is actually efficient way of saving a date.
Not understanding this:
Number returned from DataReader: 185549633.66000035
We have a requirement to maintain the number of decimal places per a User Choice.
For example: maintain 7 places.
We are using:
FormatNumber(dr.Item("Field"), 7, TriState.false, , TriState.True)
The result is: 185,549,633.6600000.
We would like to maintain the 3 (or 35) at the end.
When subtracting two numbers from the resulting query we are getting a delta but trying to show these two numbers out to 6,7,8 digits is not working thus indicating a false delta to the user.
Any advice would be appreciated.
Based on my testing, you must be working with Double values rather than Decimal. Not surprisingly, the solution to your problem can be found in the documentation.
For a start, you should not be using FormatNumber. We're not in VB6 anymore ToTo. To format a number in VB.NET, call ToString on that number. I tested this:
Dim dbl = 185549633.66000035R
Dim dec = 185549633.66000035D
Dim dblString = dbl.ToString("n7")
Dim decString = dec.ToString("n7")
Console.WriteLine(dblString)
Console.WriteLine(decString)
and I saw the behaviour you describe, i.e. the output was:
185,549,633.6600000
185,549,633.6600004
I read the documentation for the Double.ToString method (note that FormatNumber would be calling ToString internally) and this is what it says:
By default, the return value only contains 15 digits of precision although a maximum of 17 digits is maintained internally. If the value of this instance has greater than 15 digits, ToString returns PositiveInfinitySymbol or NegativeInfinitySymbol instead of the expected number. If you require more precision, specify format with the "G17" format specification, which always returns 17 digits of precision, or "R", which returns 15 digits if the number can be represented with that precision or 17 digits if the number can only be represented with maximum precision.
I then tested this:
Dim dbl = 185549633.66000035R
Dim dblString16 = dbl.ToString("G16")
Dim dblString17 = dbl.ToString("G17")
Console.WriteLine(dblString16)
Console.WriteLine(dblString17)
and the result was:
185549633.6600004
185549633.66000035
I'm trying to emulate a function in SQL that a client has produced in Excel. In effect, they have a unique, 10-digit numeric value (VARCHAR) as the primary key in one of their enterprise database systems. Within another database, they require a unique, 5-digit alphanumeric identifier. They want that 5-digit alphanumeric value to be a representation of the 10-digit number. So what they did in excel was to split the 10-digit number into pairs, then convert each of those pairs into a hexadecimal value, then stitch them back together.
The EXCEL equation is:
=IF(VALUE(MID(A2,1,4))>0,DEC2HEX(VALUE(MID(A2,3,2)))&DEC2HEX(VALUE(MID(A2,5,2)))&DEC2HEX(VALUE(MID(A2,7,2)))&DEC2HEX(VALUE(MID(A2,9,2))),DEC2HEX(VALUE(MID(A2,5,2)))&DEC2HEX(VALUE(MID(A2,7,2)))&DEC2HEX((VALUE(MID(A2,9,2)))))
I need the SQL equivalent of this. Of course, should someone out there know a better way to accomplish their goal of "a 5-digit alphanumeric identifier" based off the 10-digit number, I'm all ears.
ADDED 8/2/2011
First of all, thank you to everyone for the replies. Nice to see folks willing to help and even enjoying it! Based on all the responses, I'm apt to tell my client they're intent is sound, only their method is off kilter. I'd also like to recommend a solution. So the challenge remains, just modified slightly:
CHALLENGE: Within SQL, take a 10 digit, unique NUMERIC string and represent it ALPHANUMERICALLY in as few characters as possible. The resulting string must also be unique.
Note that the first 3-4 characters in the 10-digit string are likely to be zeros, and that they could be stripped to shorten the resulting alphanumeric string. Not required, but perhaps helpful.
This problem is inherently impossible. You have a 10 digit numeric value that you want to convert to a 5 digit alphanumeric value. Since there are 10 numeric characters, this means that there are 10^10 = 10 000 000 000 unique values for your 10 digit number. Since there are 36 alphanumeric characters (26 letters + 10 numbers), there are 36^5 = 60 466 176 unique values for your 5 digit number. You cannot map a set of 10 billion elements into a set with around 60 million.
Now, lets take a closer look at what your client's code is doing:
So what they did in excel was to split the 10-digit number into pairs, then convert each of those pairs into a hexadecimal value, then stitch them back together.
This isn't 100% accurate. The excel code never uses the first 2 digits, but performs this operation on the remaining 8. There are two main problems with this algorithm which may not be intuitively obvious:
Two 10 digit numbers can map to the same 5 digit number. Consider the numbers 1000000117 and 1000001701. The last four digits of 1000000117 get mapped to 1 11, where the last four digits of 1000001701 get mapped to 11 1. This causes both to map to 00111.
The 5 digit number may not even end up being 5 digits! For example, 1000001616 gets mapped to 001010.
So, what is a possible solution? Well, if you don't care if that 5 digit number is unique or not, in MySQL you can use something like:
hex(<NUMERIC VALUE> % 0xFFFFF)
The log of 10^10 base 2 is 33.219280948874
> return math.log(10 ^ 10) / math.log(2)
33.219280948874
> = 2 ^ 33.21928
9999993422.9114
So, it takes 34 bits to represent this number. In hex this will take 34/4 = 8.5 characters, much more than 5.
> return math.log(10 ^ 10) / math.log(16)
8.3048202372184
The Excel macro is ignoring the first 4 (or 6) characters of the 10 character string.
You could try encoding in base 36 instead of 16. This will get you to 7 characters or less.
> return math.log(10 ^ 10) / math.log(36)
6.4254860446923
The popular base 64 encoding will get you to 6 characters
> return math.log(10 ^ 10) / math.log(64)
5.5365468248123
Even Ascii85 encoding won't get you down to 5.
> return math.log(10 ^ 10) / math.log(85)
5.1829075929158
You need base 100 to get to 5 characters
> return math.log(10 ^ 10) / math.log(100)
5
There aren't 100 printable ASCII characters, so this is not going to work, as zkhr explained as well, unless you're willing to go beyond ASCII.
I found your question interesting (although I don't claim to know the answer) - I googled a bit for you out of interest and found this which may help you http://dpatrickcaldwell.blogspot.com/2009/05/converting-decimal-to-hexadecimal-with.html
I am writing a custom totaling method for a grid view. I am totaling fairly large numbers so I'd like to use a decimal to get the total. The problem is I need to control the maximum length of the total number. To solve this problem I started using float but it doesn't seem to support large enough numbers, I get this in the totals column(1.551538E+07). So is there some formating string I can use in .ToString() to guarentee that I never get more then X characters in the total field? Keep in mind I'm totaling integers and decimals.
If you're fine with all numbers displaying in scientific notation, you could go with "E[numberOfDecimalPlaces]" as your format string.
For example, if you want to cap your strings at, say, 12 characters, then, accounting for the one character for the decimal point and five characters needed to display the exponential part, you could do:
Function FormatDecimal(ByVal value As Decimal) As String
If value >= 0D Then
Return value.ToString("E5")
Else
' negative sign eats up another character '
Return value.ToString("E4")
End If
End Function
Here's a simple demo of this function:
Dim d(5) As Decimal
d(0) = 1.203D
d(1) = 0D
d(2) = 1231234789.432412341239873D
d(3) = 33.3218403820498320498320498234D
d(4) = -0.314453908342094D
d(5) = 000032131231285432940D
For Each value As Decimal in d
Console.WriteLine(FormatDecimal(value))
Next
Output:
1.20300E+000
0.00000E+000
1.23123E+009
3.33218E+001
-3.1445E-001
3.21312E+016
You could use Decimal.Round, but I don't understand the exact question, it sounds like you're saying that if the total adds up to 12345.67, you might only want to show 4 digits and would then show 2345 or do you just mean that you want to remove the decimals?
I'm trying to figure out decimal data type of a column in the SQL Server. I need to be able to store values like 15.5, 26.9, 24.7, 9.8, etc
I assigned decimal(18, 0) to the column data type but this not allowing me to store these values.
What is the right way to do this?
DECIMAL(18,0) will allow 0 digits after the decimal point.
Use something like DECIMAL(18,4) instead that should do just fine!
That gives you a total of 18 digits, 4 of which after the decimal point (and 14 before the decimal point).
You should use is as follows:
DECIMAL(m,a)
m is the number of total digits your decimal can have.
a is the max number of digits you can have after the decimal point.
http://www.tsqltutorials.com/datatypes.php has descriptions for all the datatypes.
The settings for Decimal are its precision and scale or in normal language, how many digits can a number have and how many digits do you want to have to the right of the decimal point.
So if you put PI into a Decimal(18,0) it will be recorded as 3?
If you put PI into a Decimal(18,2) it will be recorded as 3.14?
If you put PI into Decimal(18,10) be recorded as 3.1415926535.
For most of the time, I use decimal(9,2) which takes the least storage (5 bytes) in sql decimal type.
Precision => Storage bytes
1 - 9 => 5
10-19 => 9
20-28 => 13
29-38 => 17
It can store from 0 up to 9 999 999.99 (7 digit infront + 2 digit behind decimal point = total 9 digit), which is big enough for most of the values.
You can try this
decimal(18,1)
The length of numbers should be totally 18. The length of numbers after the decimal point should be 1 only and not more than that.
In MySQL DB decimal(4,2) allows entering only a total of 4 digits. As you see in decimal(4,2), it means you can enter a total of 4 digits out of which two digits are meant for keeping after the decimal point.
So, if you enter 100.0 in MySQL database, it will show an error like "Out of Range Value for column".
So, you can enter in this range only: from 00.00 to 99.99.
The other answers are right. Assuming your examples reflect the full range of possibilities what you want is DECIMAL(3, 1). Or, DECIMAL(14, 1) will allow a total of 14 digits. It's your job to think about what's enough.
request.input("name", sql.Decimal, 155.33) // decimal(18, 0)
request.input("name", sql.Decimal(10), 155.33) // decimal(10, 0)
request.input("name", sql.Decimal(10, 2), 155.33) // decimal(10, 2)