When i add a score for a key using zincrby, it increases the score and puts the element in lexicographical order.
Can i get this list in the order, in which the elements are updated or added ?
e.g>
If I execute
zincrby A 100 g
zincrby A 100 a
zincrby A 100 z
and then
zrange A 0 -1
then the result is
a->g->z
where, i want the result in order the entries are made so,
g->a->z
As score is same for all, redis is placing the elements in lexicographical order. Is there any way to prevent it ?
I don't think it is possible, but if you want to keep the order of insertion with scores, you should manipulate something like this:
<score><timestamp>
instead of
<score>
You will have to define a good time record (millis should be ok). Then you can use
zincrby A 100 * (10^nbdigitsformillis)
For instance:
Score = 100 and timestamps is 1381377600 seconds
That gives: 1001381377600
You incr by 200 the score: 1001381377600 + 200 * 10 = 3001381377600
Be careful with zset as it stores scores with double values (64 bits, but only 52 available for int value) so don't store more than 15-17 digits.
If you can't do that (need for great timestamp precision, and great score precision), you will have to manage two zsets (one for actual score, one for timestamp) and managing your ranking manual with the two values.
Related
I create Redis (v7) Sorted Sets with ZADD. Each set has multiple members. The score I use is a timestamp.
With ZRANGE I'm able to select members with scores between min and max. I want to select - based on one timestamp - one member after the timestamp and one before the timestamp.
Example (score, member) for one key:
(10, A)
(15, B)
(20, C)
I.e. SELECT 17 should give me (15, B) and (20, C).
Looks like Redis has no command to do this. What would be the most efficient way to do that?
You need two commands:
# get the one after the timestamp
zrangebyscore k (17 +inf limit 0 1
# get the one before the timestamp
zrevrangebyscore k (17 -inf limit 0 1
In order to be more efficient and atomic, wrap these two commands into a Lua script.
I'm using Redis sorted set to implement the leaderboard of my game, where I show the user ranking in descending order. I'm stuck in a case where two or more users have the same score. So in this case, I want the higher ranking of the user who gets the score first. For example, I'm adding the following entries in Redis.
127.0.0.1:6379> zadd testing-key 5 a
(integer) 1
127.0.0.1:6379> zadd testing-key 4 b
(integer) 1
127.0.0.1:6379> zadd testing-key 5 c
(integer) 1
and when I'm querying for the rank in reverse order, I'm getting this
127.0.0.1:6379> zrevrange testing-key 0 10
1) "c"
2) "a"
3) "b"
but in my case, the ranking should be like
1) "a"
2) "c"
3) "b"
So is there any provision in Redis to give higher precedence to the entity which entered first in the set with the same score?
I found one solution to this problem. In my case, the score is an integer so I converted it into decimal and added Long.MAX_VALUE - System.nanoTime() after decimal. So the final score code will be like
double finalScore = score.(Long.MAX_VALUE - System.nanoTime());
So the final score of the player who scored first would be higher than the second one. Please let me know if you have any better solution.
If your leaderboard's scores are "small" enough, you may get away with using a combination of the score and the timestamp (e.g. 123.111455234, where 123 is the score). However, since the Sorted Set score is a double floating point, you may lose precision.
Alternatively, keep two Sorted Sets - one with each player's leaderboard score and the other with each player's score timestamp, and use both to determine the order.
Or, use a single sorted set for the leader board, encode the timestamp as part of the member and rely on lexicographical ordering.
If I have 5 members with scores as follows
a - 1
b - 2
c - 3
d - 3
e - 5
ZRANK of c returns 2, ZRANK of d returns 3
Is there a way to get same rank for same scores?
Example: ZRANK c = 2, d = 2, e = 3
If yes, then how to implement that in spring-data-redis?
Any real solution needs to fit the requirements, which are kind of missing in the original question. My 1st answer had assumed a small dataset, but this approach does not scale as dense ranking is done (e.g. via Lua) in O(N) at least.
So, assuming that there are a lot of users with scores, the direction that for_stack suggested is better, in which multiple data structures are combined. I believe this is the gist of his last remark.
To store users' scores you can use a Hash. While conceptually you can use a single key to store a Hash of all users scores, in practice you'd want to hash the Hash so it will scale. To keep this example simple, I'll ignore Hash scaling.
This is how you'd add (update) a user's score in Lua:
local hscores_key = KEYS[1]
local user = ARGV[1]
local increment = ARGV[2]
local new_score = redis.call('HINCRBY', hscores_key, user, increment)
Next, we want to track the current count of users per discrete score value so we keep another hash for that:
local old_score = new_score - increment
local hcounts_key = KEYS[2]
local old_count = redis.call('HINCRBY', hcounts_key, old_score, -1)
local new_count = redis.call('HINCRBY', hcounts_key, new_score, 1)
Now, the last thing we need to maintain is the per score rank, with a sorted set. Every new score is added as a member in the zset, and scores that have no more users are removed:
local zdranks_key = KEYS[3]
if new_count == 1 then
redis.call('ZADD', zdranks_key, new_score, new_score)
end
if old_count == 0 then
redis.call('ZREM', zdranks_key, old_score)
end
This 3-piece-script's complexity is O(logN) due to the use of the Sorted Set, but note that N is the number of discrete score values, not the users in the system. Getting a user's dense ranking is done via another, shorter and simpler script:
local hscores_key = KEYS[1]
local zdranks_key = KEYS[2]
local user = ARGV[1]
local score = redis.call('HGET', hscores_key, user)
return redis.call('ZRANK', zdranks_key, score)
You can achieve the goal with two Sorted Set: one for member to score mapping, and one for score to rank mapping.
Add
Add items to member to score mapping: ZADD mem_2_score 1 a 2 b 3 c 3 d 5 e
Add the scores to score to rank mapping: ZADD score_2_rank 1 1 2 2 3 3 5 5
Search
Get score first: ZSCORE mem_2_score c, this should return the score, i.e. 3.
Get the rank for the score: ZRANK score_2_rank 3, this should return the dense ranking, i.e. 2.
In order to run it atomically, wrap the Add, and Search operations into 2 Lua scripts.
Then there's this Pull Request - https://github.com/antirez/redis/pull/2011 - which is dead, but appears to make dense rankings on the fly. The original issue/feature request (https://github.com/antirez/redis/issues/943) got some interest so perhaps it is worth reviving it /cc #antirez :)
The rank is unique in a sorted set, and elements with the same score are ordered (ranked) lexically.
There is no Redis command that does this "dense ranking"
You could, however, use a Lua script that fetches a range from a sorted set, and reduces it to your requested form. This could work on small data sets, but you'd have to devise something more complex for to scale.
unsigned long zslGetRank(zskiplist *zsl, double score, sds ele) {
zskiplistNode *x;
unsigned long rank = 0;
int i;
x = zsl->header;
for (i = zsl->level-1; i >= 0; i--) {
while (x->level[i].forward &&
(x->level[i].forward->score < score ||
(x->level[i].forward->score == score &&
sdscmp(x->level[i].forward->ele,ele) <= 0))) {
rank += x->level[i].span;
x = x->level[i].forward;
}
/* x might be equal to zsl->header, so test if obj is non-NULL */
if (x->ele && x->score == score && sdscmp(x->ele,ele) == 0) {
return rank;
}
}
return 0;
}
https://github.com/redis/redis/blob/b375f5919ea7458ecf453cbe58f05a6085a954f0/src/t_zset.c#L475
This is the piece of code redis uses to compute the rank in sorted sets. Right now ,it just gives rank based on the position in the Skiplist (which is sorted based on scores).
What does the skiplistnode variable "span" mean in redis.h? (what is span ?)
I am interested in using Redis to check if a IP address (converted into integer) falls within a range of IPs. It is very likely that the ranges will overlap.
I have found this question/answer, although I am not able to fully understand the logic behind it.
Thank you for your help!
EDIT - Since I got a downvote (a comment to explain why would be nice), I've removed some clutter from my answer.
#DidierSpezia answer in your linked question is a good answer, but it becomes hard to maintain if you are adding/removing ranges.
However it is not trivial (and expensive) to build and maintain it.
I have an answer that is easier to maintain, but it could get slow and memory expensive to compute with many ranges as it requires cloning a set of all ranges.
You need to save all ranges twice, in two sets. The score of each range will be its border values.
Going with the sets in #DidierSpezia example:
A 2-8
B 4-6
C 2-9
D 7-10
Your two sets will be:
ZADD ranges:low 2 "2-8" 4 "4-6" 2 "2-9" 7 "7-10"
ZADD ranges:high 8 "2-8" 6 "4-6" 9 "2-9" 10 "7-10"
To query to which ranges a value belongs, you need to trim the ranges that the lower border is higher than the queried value, and trim the ranges that the higher border is lower.
The most efficient way I can think of is cloning one of the sets, trimming one of it sides by the rules gave above, changing the scores of the ranges to reflect the other border and then trim the second side.
Here's how to find the ranges 5 belongs to:
ZUNIONSTORE tmp 1 ranges:low
ZREMRANGEBYSCORE tmp (5 +inf
ZINTERSTORE tmp 2 tmp ranges:high WEIGHTS 0 1
ZREMRANGEBYSCORE tmp -inf (5
ZRANGE tmp 0 -1
In this discussion, Dvir Volk and #antirez suggested to use a sorted set in which each entry represent a range, and has the following form:
Member = "min-max" range
Score = max value
For example:
ZADD z 10 "0-10"
ZADD z 20 "10-20"
ZADD z 100 "50-100"
And in order to check if a value falls within a range, you can use ZRANGEBYSCORE and parse the member returned.
For example, to check value 5:
ZRANGEBYSCORE z 5 +inf LIMIT 0 1
this will return the "0-10" member, and you only need to parse the string and validate if your value is in between.
To check value 25:
ZRANGEBYSCORE z 25 +inf LIMIT 0 1
will return "50-100", but the value is not between that range.
I have a need for a leaderboard where users are stored with a score AND a percentage of level completeness. I only need to be able to sort by score and get ranks by score.
Using Redis' sorted sets a can easily store users' scores like so:
ZADD leaderboard:gamemode1 100 user1
ZADD leaderboard:gamemode1 300 user2
however, I am struggelig to figure out, how to best store the percentage values belonging to the scores of 100 and 300.
Should I do something like this:
ZADD leaderboard:gamemode1 100 user1:29.45
where the :29.45 is the percentage for user1's score of 100 in gamemode1? I think this will make it complicated to update user1's score for gamemode1 later.
Or should I instead create a hashtable as a book-keeping mechanism, that stores all users' scores and percentages for all gamemodes?
My concern about the hashtable approach is, that if I want to show a leaderboard of ~50 users with their score and percentage values, I will then have to query that hashtable 50 times to get the percentages, after pulling the top50 scores with ZRANGE.
Does anyone have any input on how to construct this in a good way?
When working with Sorted Sets, keep in mind that it has member string unicity.
That means:
ZADD game1 100 user1
ZADD game1 90 user1
Leaves you with only 90 user1.
Just a heads-up, I think you've covered that.
For storing the percentage, you could use the decimal part of the score:
ZADD leaderboard:gamemode1 100.0995 user1
ZADD leaderboard:gamemode1 90.0850 user2
Above, I use the decimal part for the percentage like this: 0.0995 = 99.5%.
You can also use the score for score only, and serialize the member string:
ZADD leaderboard:gamemode1 100 [serialized dictionary/dataset]
ZADD leaderboard:gamemode1 90 [serialized dictionary/dataset]
If you use JSON or MsgPack, you can always use server-side scripting to update some data within the serialized string.
I would not recommend bookkeeping for this simple scenario, at least not at this stage. You can always implement that when needed.
Hope this helps, TW