How does redis bitcount command work?
setbit test 1 1
setbit test 3 1
count bits.
bitcount test
returns 2
Range count.
bitcount test 1 2
returns 0
Why? I thought I should get 1 for the bit position between 1 and 2, we have only one bit set.
The start and end arguments refer to bytes. So you're asking for the number of bits in the second and third bytes of the string. You've only set bits in the first byte, so the answer is 0.
It's unfortunate that the documentation for BITCOUNT doesn't explain that, but it is mentioned in the documentation for BITPOS:
It is possible to look for bits only in a specified interval passing the additional arguments start and end.... The range is interpreted as a range of bytes and not a range of bits, so start=0 and end=2 means to look at the first three bytes.
Related
for example
> SETBIT bitmapsarestrings 2 1
> SETBIT bitmapsarestrings 3 1
> SETBIT bitmapsarestrings 5 1
> SETBIT bitmapsarestrings 10 1
> SETBIT bitmapsarestrings 11 1
> SETBIT bitmapsarestrings 14 1
> GET bitmapsarestrings
"42"
Binary storage should not like this: 0010 0110 0001 1100 ?
stored in this way, why value is 42?
These SETBIT operations will make the value as a binary string, whose length is 2 bytes or 16 bits. After the settings, the value will be 0b 00110100 00110010 in binary format.
The first byte (0b 00110100) is 52, which is the ascii code of '4', and the second byte (0b 00110010) is 50, which is the ascii code of '2'. So when you get the value of the string, it returns "42".
What #for_stack said, or just refer to the lines immediately above that example (https://redis.io/commands/setbit):
Bitmaps are not an actual data type, but a set of bit-oriented operations defined on the String type (for more information refer to the Bitmaps section of the Data Types Introduction page). This means that bitmaps can be used with string commands, and most importantly with SET and GET.
Because Redis' strings are binary-safe, a bitmap is trivially encoded as a bytes stream. The first byte of the string corresponds to offsets 0..7 of the bitmap, the second byte to the 8..15 range, and so forth.
I have tried to store bit using setbit in redis like
setbit mykey 123 1 and also use more than maximal int value 2147483647 + 100 as the offset value so it would be like this :
setbit mykey 2147483747 1 this one works.
And also I tried to add severals number until hit the 4547483747 and got
ERR bit offset is not an integer or out of range
my question is : what is the maximal exact value of the offset?
According to the documentation:
The offset argument is required to be greater than or equal to 0, and smaller than 2³²
So the maximum value is 4,294,967,295 (or 2³² - 1). Confirmation via the CLI:
127.0.0.1:6379> setbit mykey 4294967295 1
(integer) 1
127.0.0.1:6379> setbit mykey 4294967296 1
(error) ERR bit offset is not an integer or out of range
I am interested in using Redis to check if a IP address (converted into integer) falls within a range of IPs. It is very likely that the ranges will overlap.
I have found this question/answer, although I am not able to fully understand the logic behind it.
Thank you for your help!
EDIT - Since I got a downvote (a comment to explain why would be nice), I've removed some clutter from my answer.
#DidierSpezia answer in your linked question is a good answer, but it becomes hard to maintain if you are adding/removing ranges.
However it is not trivial (and expensive) to build and maintain it.
I have an answer that is easier to maintain, but it could get slow and memory expensive to compute with many ranges as it requires cloning a set of all ranges.
You need to save all ranges twice, in two sets. The score of each range will be its border values.
Going with the sets in #DidierSpezia example:
A 2-8
B 4-6
C 2-9
D 7-10
Your two sets will be:
ZADD ranges:low 2 "2-8" 4 "4-6" 2 "2-9" 7 "7-10"
ZADD ranges:high 8 "2-8" 6 "4-6" 9 "2-9" 10 "7-10"
To query to which ranges a value belongs, you need to trim the ranges that the lower border is higher than the queried value, and trim the ranges that the higher border is lower.
The most efficient way I can think of is cloning one of the sets, trimming one of it sides by the rules gave above, changing the scores of the ranges to reflect the other border and then trim the second side.
Here's how to find the ranges 5 belongs to:
ZUNIONSTORE tmp 1 ranges:low
ZREMRANGEBYSCORE tmp (5 +inf
ZINTERSTORE tmp 2 tmp ranges:high WEIGHTS 0 1
ZREMRANGEBYSCORE tmp -inf (5
ZRANGE tmp 0 -1
In this discussion, Dvir Volk and #antirez suggested to use a sorted set in which each entry represent a range, and has the following form:
Member = "min-max" range
Score = max value
For example:
ZADD z 10 "0-10"
ZADD z 20 "10-20"
ZADD z 100 "50-100"
And in order to check if a value falls within a range, you can use ZRANGEBYSCORE and parse the member returned.
For example, to check value 5:
ZRANGEBYSCORE z 5 +inf LIMIT 0 1
this will return the "0-10" member, and you only need to parse the string and validate if your value is in between.
To check value 25:
ZRANGEBYSCORE z 25 +inf LIMIT 0 1
will return "50-100", but the value is not between that range.
I'm trying to assemble proper track data given a CVC3 and a bunch of positional parameters. But the EMV C-2 Kernel book is about as obtuse as you could imagine (would it kill somebody to include an example!?!). Can anyone help work this example:
9f62 - pcvc3(t1) - Position of CVC3 in track1: 0x38 (4-6?)
9f63 - punatc(t1) - Unpredictable Number Track1 Pos: 0x3C6 (2-3 7-10?)
9f64 - natc(t1) - Digits in track1 ATC: 4
9f65 - pcvc3(t2) - Position of CVC3 in track2: 0x38 (4-6)
9f66 - punatc(t2) - Unpredictable Number Track2 Pos: 0x3C6 (2-3 7-10?)
9f67 - Digits in track2 ATC: 4
After successful checksum generation:
9f61 - track2 CVC3 - 2EF4
9f60 - track1 CVC3 - 609B
9f36 - ATC - 1E47
assuming the discretionary data field starts out as all 0s, how does it end up? The spec says this:
Convert the binary encoded CVC3 (Track2) to the BCD encoding of the
corresponding number expressed in base 10. Copy the q least significant digits of the
BCD encoded CVC3 (Track2) in the eligible positions of the 'Discretionary Data' in
Track 2 Data. The eligible positions are indicated by the q non-zero bits in
PCVC3(Track2).
I read that as:
CVC3 = 0x609B = 24731 (so copy 731? What does BCD have to do with this? Or are they just saying "copy the 731 as bcd encoded to the byte array"?)
yes you are correct it is rather obtuse. you are correct that you would convert your p values (pCVC3, and PUNATC) to binary. (0011 1000, 0011 1100 0110 for your track1 p values) you then right align the proper values with the discretionary data. example
Bxxxxxxxxxxxxxxxx^ /^14111014010000000000
....000000000000000000CCC000
....00000000000000AAAA000UU0
so you say that you CVC3 for track 1 is 609B which is 24,731, since you PCVC3 is asking for only 3 characters youd set 731 in. your ATC is 1E47 which is 7,751. your PUNATC is asking for 4 digits so you'd use 7751. FYI... if the ATC is lower than the requested characters you'd pad with 0's. Your unpredictable number is even more tricky... so you make a 4 byte random number. convert it to a uint (base 10) and then mark the 8 most significant bytes as 0. example. lets say your random 4 bytes is 29A6 06AE. in base 10 that is 698,746,542. mark out the first 8 characters with 0. and you are left with 000,000,002. you'd place 02 in for you unpredictable number placment.. so.. all that said your track would look like this
Bxxxxxxxxxxxxxxxx^ /^14111014017751731020
the last character is equal to the length of the unpredicatable number (numeric) digits. which was 02.. so the last digit is 2 making your final track data
%Bxxxxxxxxxxxxxxxx^ /^1411014017751731022;
track2 is very similar. good luck with it. I understand your frustrations with this. :)
I'm trying to emulate a function in SQL that a client has produced in Excel. In effect, they have a unique, 10-digit numeric value (VARCHAR) as the primary key in one of their enterprise database systems. Within another database, they require a unique, 5-digit alphanumeric identifier. They want that 5-digit alphanumeric value to be a representation of the 10-digit number. So what they did in excel was to split the 10-digit number into pairs, then convert each of those pairs into a hexadecimal value, then stitch them back together.
The EXCEL equation is:
=IF(VALUE(MID(A2,1,4))>0,DEC2HEX(VALUE(MID(A2,3,2)))&DEC2HEX(VALUE(MID(A2,5,2)))&DEC2HEX(VALUE(MID(A2,7,2)))&DEC2HEX(VALUE(MID(A2,9,2))),DEC2HEX(VALUE(MID(A2,5,2)))&DEC2HEX(VALUE(MID(A2,7,2)))&DEC2HEX((VALUE(MID(A2,9,2)))))
I need the SQL equivalent of this. Of course, should someone out there know a better way to accomplish their goal of "a 5-digit alphanumeric identifier" based off the 10-digit number, I'm all ears.
ADDED 8/2/2011
First of all, thank you to everyone for the replies. Nice to see folks willing to help and even enjoying it! Based on all the responses, I'm apt to tell my client they're intent is sound, only their method is off kilter. I'd also like to recommend a solution. So the challenge remains, just modified slightly:
CHALLENGE: Within SQL, take a 10 digit, unique NUMERIC string and represent it ALPHANUMERICALLY in as few characters as possible. The resulting string must also be unique.
Note that the first 3-4 characters in the 10-digit string are likely to be zeros, and that they could be stripped to shorten the resulting alphanumeric string. Not required, but perhaps helpful.
This problem is inherently impossible. You have a 10 digit numeric value that you want to convert to a 5 digit alphanumeric value. Since there are 10 numeric characters, this means that there are 10^10 = 10 000 000 000 unique values for your 10 digit number. Since there are 36 alphanumeric characters (26 letters + 10 numbers), there are 36^5 = 60 466 176 unique values for your 5 digit number. You cannot map a set of 10 billion elements into a set with around 60 million.
Now, lets take a closer look at what your client's code is doing:
So what they did in excel was to split the 10-digit number into pairs, then convert each of those pairs into a hexadecimal value, then stitch them back together.
This isn't 100% accurate. The excel code never uses the first 2 digits, but performs this operation on the remaining 8. There are two main problems with this algorithm which may not be intuitively obvious:
Two 10 digit numbers can map to the same 5 digit number. Consider the numbers 1000000117 and 1000001701. The last four digits of 1000000117 get mapped to 1 11, where the last four digits of 1000001701 get mapped to 11 1. This causes both to map to 00111.
The 5 digit number may not even end up being 5 digits! For example, 1000001616 gets mapped to 001010.
So, what is a possible solution? Well, if you don't care if that 5 digit number is unique or not, in MySQL you can use something like:
hex(<NUMERIC VALUE> % 0xFFFFF)
The log of 10^10 base 2 is 33.219280948874
> return math.log(10 ^ 10) / math.log(2)
33.219280948874
> = 2 ^ 33.21928
9999993422.9114
So, it takes 34 bits to represent this number. In hex this will take 34/4 = 8.5 characters, much more than 5.
> return math.log(10 ^ 10) / math.log(16)
8.3048202372184
The Excel macro is ignoring the first 4 (or 6) characters of the 10 character string.
You could try encoding in base 36 instead of 16. This will get you to 7 characters or less.
> return math.log(10 ^ 10) / math.log(36)
6.4254860446923
The popular base 64 encoding will get you to 6 characters
> return math.log(10 ^ 10) / math.log(64)
5.5365468248123
Even Ascii85 encoding won't get you down to 5.
> return math.log(10 ^ 10) / math.log(85)
5.1829075929158
You need base 100 to get to 5 characters
> return math.log(10 ^ 10) / math.log(100)
5
There aren't 100 printable ASCII characters, so this is not going to work, as zkhr explained as well, unless you're willing to go beyond ASCII.
I found your question interesting (although I don't claim to know the answer) - I googled a bit for you out of interest and found this which may help you http://dpatrickcaldwell.blogspot.com/2009/05/converting-decimal-to-hexadecimal-with.html