Redis sort by score and date - sql

I'm trying to strategies how to calculate position/rank based on score and date
I have a two pieces of data with the same score shown in the image below.
If i run the command
$ZADD usersScores 11 userId6
It will add it above userId5 which is not something I want
Is it possible to add it below userId1 maybe based on date?
This is my goal essentially where i would like to show the position based on the amount and the oldest would be ontop via date
Thanks!

Sorted Set ordering
Sorted sets are ordered by their scores first, and then lexigraphically by their members if the scores are equivalent. So in your case if you added the following keys:
127.0.0.1:6379> ZADD myset 11 user1
(integer) 1
127.0.0.1:6379> ZADD myset 11 user5
(integer) 1
127.0.0.1:6379> ZADD myset 24 user3
(integer) 1
127.0.0.1:6379> zadd myset 54 user2
(integer) 1
127.0.0.1:6379> zadd myset 11 user6
(integer) 1
and then did a ZRANGE, you'd get them in ascending order with the members ordered in alphabetical order:
127.0.0.1:6379> zrange myset 0 -1 WITHSCORES
1) "user1"
2) "11"
3) "user5"
4) "11"
5) "user6"
6) "11"
7) "user3"
8) "24"
9) "user2"
10) "54"
What it looks like, is your GUI is using, or are just passing in the rev argument to ZRANGE under the hood, which orders everything in reverse (scores descending, reverse alphabetical)
127.0.0.1:6379> ZRANGE myset 0 -1 WITHSCORES REV
1) "user2"
2) "54"
3) "user3"
4) "24"
5) "user6"
6) "11"
7) "user5"
8) "11"
9) "user1"
10) "11"
Second sort-by
There are a couple of ways you could handle the second sort by with a sorted set, let's assume you wanted to do this with a 32 bit unix timestamp
You could prepend the second sortby field to the member name so if user6's score came in at 1659579650, you'd just make the member name be 1659579650:user1, then when it orders it, it would do so lexigraphically
You could combine the score with the second, since 11 is the score, we could shift 11 32 bits to the left and add/or the timestamp to it, which comes out to 48,904,219,906, those will then order correctly, and then when you want to find the score, all you need to do is right-shift the score by 32 bits.

Related

Redis - Sort and filter hash store using string attribute

I have a redis hash store that looks like Item:<id>, with attribute name. I want to filter the hash store by a prefix for name attribute.
What I'm trying to do is store the name (lowercased) in a separate Z-set called Item:::name while setting the score to 0. By doing this, I'm successfully able to get the desired result using ZRANGEBYLEX however I'm unable to map the results back to the original Items. How should I go about implementing something like this?
I've seen multiple autocomplete examples for Redis which require the same functionality but without linking the words back to an actual Item (hash in this case)
In sorted sets the member can't be duplicated, it has to be unique. So different users with the same name will cause problem.
My suggestion requires application layer coding to parse response array and executing hash commands (it will be like secondary indexes);
127.0.0.1:6379> HSET user:1 name jack
(integer) 1
127.0.0.1:6379> HSET user:2 name john
(integer) 1
127.0.0.1:6379> HSET user:3 name keanu
(integer) 1
127.0.0.1:6379> HSET user:4 name jack
(integer) 1
127.0.0.1:6379> ZADD item:names 0 jack::user:1 0 john::user:2 0 keanu::user:3 0 jack::user:4
(integer) 4
127.0.0.1:6379> ZRANGE item:names 0 -1 WITHSCORES
1) "jack::user:1"
2) "0"
3) "jack::user:4"
4) "0"
5) "john::user:2"
6) "0"
7) "keanu::user:3"
8) "0"
127.0.0.1:6379> ZRANGEBYLEX item:names [jack [jo
1) "jack::user:1"
2) "jack::user:4"
At the end you will have name::hash-key formatted array elements. At application layer if you separate each element to two substrings by using ::(any other string such as !!! or || etc) you will have user:1 and user:4.
127.0.0.1:6379> HGETALL user:1
1) "name"
2) "jack"
127.0.0.1:6379> HGETALL user:4
1) "name"
2) "jack"
127.0.0.1:6379>

Can we add Value in Redis List with Expiration Time in C#?

I am using Redis queue and adding the data using ListLeftPush and reading data using ListRightPop. It works fine I am able to get the data. But what if data has not popped out? Can we delete old data? or Can we add Value in Redis List with Expiration Time?
How to add time limit for each value while using ListLeftPush command in C#?
It is not possible to add expire time for individual value for the sake of keeping redis simple and fast.
you can only add expire time for individual keys i.e in ur case it is for whole list.
No redis doesn't support that. Expiration is available only for the top level keys. The closes data type/solution for your case would be sorted sets.
You put your expiration time(timestamp) as score while adding to sorted set(ZADD)
Instead of LPOP you use ZPOPMAX to get "to be last expired" element.
Periodically you may use ZREMRANGEBYSCORE to remove expired elements.
For the demonstration i used smaller numbers as expiration dates.
127.0.0.1:6379> ZADD myset 15 "a"
(integer) 0
127.0.0.1:6379> ZADD myset 25 "b"
(integer) 0
127.0.0.1:6379> ZADD myset 35 "c" 45 "d" 55 "e"
(integer) 0
127.0.0.1:6379> ZRANGE myset 0 -1 WITHSCORES
1) "a"
2) "15"
3) "b"
4) "25"
5) "c"
6) "35"
7) "d"
8) "45"
9) "e"
10) "55"
127.0.0.1:6379> ZPOPMAX myset
1) "e"
2) "55"
127.0.0.1:6379> ZREMRANGEBYSCORE myset -1 15
(integer) 1
127.0.0.1:6379> ZRANGE myset 0 -1
1) "b"
2) "c"
3) "d"
127.0.0.1:6379>

In Redis, is it possible to predicate the inclusion of a member in the result of ZRANGE based on its existence in a different ZSET?

I'm tracking members in multiple Sorted Sets in Redis as a way to do multi-column indexing on the members. As an example, let's say I have two Sorted Sets, lastseen (which is epoch time) and points, and I store usernames as members in these Sorted Sets.
I'm wanting to use ZREVRANGEBYSCORE against lastseen to get a list of users who were recently seen, but I only want a user to be included in the results if its also present in the points Sorted Set.
I looked at using ZINTERSTORE to help me do this, but the available AGGREGATE options are not conducive to my scenario. If one of the AGGREGATE options was to use the score from a specific set provided to the ZINTERSTORE command, then it would work, but that's not an option.
---EDIT---
Example:
redis> ZADD lastseen 12345 "foo"
redis> ZADD lastseen 12346 "bar"
redis> ZADD lastseen 12347 "sucka"
redis> ZADD points 5 "foo"
redis> ZADD points 9 "bar"
Now, if I run:
redis> ZREVRANGEBYSCORE lastseen +inf -inf
1) "sucka"
2) "bar"
3) "foo"
What I'm looking for is a way to only get "foo" and "bar" returned by the call to ZREVRANGEBYSCORE for lastseen since "sucka" isn't resent in the points Sorted Set.
I can do:
redis> ZINTERSTORE out 2 lastseen points
redis> ZRANGE out 0 -1 WITHSCORES
1) "foo"
2) "12350"
3) "bar"
4) "12355"
As you can see, the ZINTERSTORE does cut out the "sucka" member, but the scores for the other two members get aggregated using SUM. There's two other aggregation options (MIN and MAX) available for use with ZINTERSTORE. In this case, using MAX would give me what I'm looking for, but that wouldn't be the case if a user's points were greater than the epoch value for the last time they were seen.
You can get what you're asking by setting the WEIGHT of points to 0 and of lastseen to 1 when running ZINTERSTORE so that only the weight of lastseen is considered:
redis> ZINTERSTORE out 2 lastseen points WEIGHTS 1 0
(integer) 2
Then, depending on what ordering you're going for, you can either:
redis> ZRANGE out 0 -1 WITHSCORES
1) "foo"
2) "12345"
3) "bar"
4) "12346"
or:
redis> ZREVRANGE out 0 -1 WITHSCORES
1) "bar"
2) "12346"
3) "foo"
4) "12345"

ordered sets in redis: random output in case of score ties

I have an ordered set in Redis (I am actually using a python client https://github.com/andymccurdy/redis-py), for example:
zadd myset 1 key1
zadd myset 1 key2
zadd myset 1 key3
zadd myset 0 key4
Note that 3 keys have the same score.
Using ZRANGE, i would like to get the top 2 entries (i.e lowest scores). "key4" will always be the first result as it has a lower value, but I would like the second return value to be randomly selected between the ties: key1,key2,key3. ZRANGE actually returns the keys in the order they are indexed: "keys1" is always my second result:
zrange myset 0 -1 WITHSCORES
1) "key4"
2) "0"
3) "key1"
4) "1"
5) "key2"
6) "1"
7) "key3"
8) "1"
any idea?
thanks,
J.
As kindly requested by Linus G Thiel, here are more details about my usecase:
I would like to use zsets to perform a simple ranking system. I have a list of items, for each one a score representing the relevance of the item. For the cold start of my system, most of the scores will be identical (i.e 0), and I would like to randomly select among the items having the same score. Otherwise I will always return the exact same lexicographic ordering, which will introduce a bias in the system.
The solution you propose, using one specific set for each duplicated score value will work. I will give it a try.
Thanks,

Redis: Is it possible sort ordered set result with same score by other key value

ZADD myset 1 ad1
ZADD myset 1 ad2
SET order:ad1 1
SET order:ad2 2
How to sort firstly by ordered set score then the order value?
If you want to store a result (not altering the way data are stored), then you can use the SORT command. It can be applied to zset.
For instance:
> zadd myset 1 albert
> zadd myset 1 joseph
> zadd myset 4 bertrand
> zadd myset 2 casimir
> zadd myset 3 alfred
You can sort by value:
> sort myset alpha
1) "albert"
2) "alfred"
3) "bertrand"
4) "casimir"
5) "joseph"
You can sort by score AND value:
> zrange myset 0 -1
1) "albert"
2) "joseph"
3) "casimir"
4) "alfred"
5) "bertrand"
Now let's add a new property to these objects:
> set order:albert 5
> set order:alfred 3
> set order:casimir 1
> set order:joseph 4
> set order:bertrand 2
You can sort by the new order property:
> sort myset by order:*
1) "casimir"
2) "bertrand"
3) "alfred"
4) "joseph"
5) "albert"
Now if you need to sort by score AND this order property, then you have no other choice than materialize the order property in the zset OR materialize the score in the order property.
For instance, adding the order property in the zset, we get:
> del myset
> zadd myset 1 5:albert
> zadd myset 1 4:joseph
> zadd myset 4 2:bertrand
> zadd myset 2 1:casimir
> zadd myset 3 3:alfred
> zrange myset 0 -1
1) "4:joseph"
2) "5:albert"
3) "1:casimir"
4) "3:alfred"
5) "2:bertrand"
Beware: in that case, the property field is sorted according to the lexicographic order, so the value may have to be normalized if you need numerical order and multiple digits. There are various tricks to do this. For instance, instead of storing "1", "12" or "123", you can store "11", "212" or "3123", the first digit representing the number of digits of your number.
If both the score and the order property can change, this is not very convenient and you will be better served by sorting on client side. This can be achieved in two roundtrips by retrieving the zset with scores on one side, and the corresponding order values on the other side.
> zrange myset 0 -1 withscores
> sort myset by order:* get order:* get #
Up to the client to join the two results (using some kind of dictionary or associative array), and sort to get the final result.