I have a simple redis set with over 1 millons notes. I add dat using sadd
How can I read this set by partition?
I mean read first 100 000 keys and after 200 000?
I'm not sure what you mean by "by partition", but if you want to read it in chunks, SSCAN is your friend.
SSCAN key cursor [MATCH pattern] [COUNT count]
You start with a value of 0 in the cursor, and you get the next cursor id and COUNT elements each time. When there is nothing more to read you get a cursor of 0.
For example:
# let's add 14 elements to a set
127.0.0.1:6379> SADD myset e1 e2 e3 e4 e5 e6 e7 e8 e9 e10 e11 e12 e13 e14
(integer) 14
# now let's scan it from the "beginning" (notice that it's not ordered)
127.0.0.1:6379> SSCAN myset 0 COUNT 10
1) "3"
2) 1) "e8"
2) "e10"
3) "e2"
4) "e11"
5) "e7"
6) "e3"
7) "e14"
8) "e4"
9) "e6"
10) "e9"
# we got a cursor id of 3, let's give that to the next iteration!
127.0.0.1:6379> SSCAN myset 3 COUNT 10
1) "0"
2) 1) "e13"
2) "e12"
3) "e5"
4) "e1"
# now we got a cursor id of 0, meaning we're done
Related
I'm trying to strategies how to calculate position/rank based on score and date
I have a two pieces of data with the same score shown in the image below.
If i run the command
$ZADD usersScores 11 userId6
It will add it above userId5 which is not something I want
Is it possible to add it below userId1 maybe based on date?
This is my goal essentially where i would like to show the position based on the amount and the oldest would be ontop via date
Thanks!
Sorted Set ordering
Sorted sets are ordered by their scores first, and then lexigraphically by their members if the scores are equivalent. So in your case if you added the following keys:
127.0.0.1:6379> ZADD myset 11 user1
(integer) 1
127.0.0.1:6379> ZADD myset 11 user5
(integer) 1
127.0.0.1:6379> ZADD myset 24 user3
(integer) 1
127.0.0.1:6379> zadd myset 54 user2
(integer) 1
127.0.0.1:6379> zadd myset 11 user6
(integer) 1
and then did a ZRANGE, you'd get them in ascending order with the members ordered in alphabetical order:
127.0.0.1:6379> zrange myset 0 -1 WITHSCORES
1) "user1"
2) "11"
3) "user5"
4) "11"
5) "user6"
6) "11"
7) "user3"
8) "24"
9) "user2"
10) "54"
What it looks like, is your GUI is using, or are just passing in the rev argument to ZRANGE under the hood, which orders everything in reverse (scores descending, reverse alphabetical)
127.0.0.1:6379> ZRANGE myset 0 -1 WITHSCORES REV
1) "user2"
2) "54"
3) "user3"
4) "24"
5) "user6"
6) "11"
7) "user5"
8) "11"
9) "user1"
10) "11"
Second sort-by
There are a couple of ways you could handle the second sort by with a sorted set, let's assume you wanted to do this with a 32 bit unix timestamp
You could prepend the second sortby field to the member name so if user6's score came in at 1659579650, you'd just make the member name be 1659579650:user1, then when it orders it, it would do so lexigraphically
You could combine the score with the second, since 11 is the score, we could shift 11 32 bits to the left and add/or the timestamp to it, which comes out to 48,904,219,906, those will then order correctly, and then when you want to find the score, all you need to do is right-shift the score by 32 bits.
The Redis ZSET Sorted Set (member, score) sorts the set by the score.
The Redis SET are an unordered collection of unique Strings.
What I need is a method that returns the members in a Sorted Set matching a pattern as in ZRANGEBYLEX but with members with different scores.
Is is possible at all with redis?
Well, it seems I did not know about the SCAN suite. In this case ZSCAN solves this issue, however with cost O(N) where N is the number of items in sorted set because it iterates over the whole set.
Example of elements in:
127.0.0.1:6379> zrange l 0 -1 WITHSCORES
1) "foodgood:irene"
2) "1"
3) "foodgood:pep"
4) "1"
5) "sleep:irene"
6) "1"
7) "sleep:juan"
8) "1"
9) "sleep:pep"
10) "1"
11) "sleep:toni"
12) "1"
13) "foodgood:juan"
14) "2"
Now ZSCAN for those with prefix foodgood:
127.0.0.1:6379> ZSCAN l 0 match foodgood:*
1) "0"
2) 1) "foodgood:irene"
2) "1"
3) "foodgood:pep"
4) "1"
5) "foodgood:juan"
6) "2"
The first returned value "0" if zero indicates collection was completely explored.
What I would have liked is that to be O(log(N)+M) where M is the number of elements retrieved similar to Binary Search Tree.
I need to store the lowest score for each key I add to the set, but when I do ZADD, Redis overwrites the score with the new value even if the score is higher.
ZADD myorderset 1 'one' 2 'two' 3 'three'
(integer) 3
ZRANGE myorderset 0 -1 WITHSCORES
1) "one"
2) "1"
3) "two"
4) "2"
5) "three"
6) "3"
ZADD myorderset 5 'three'
(integer) 0
ZRANGE myorderset 0 -1 WITHSCORES
1) "one"
2) "1"
3) "two"
4) "2"
5) "three"
6) "5"
In the example case, I need the key 'three' not be updated since the new score is higher (5) than the existing (3). Is there a way to do this natively or do I need to create a script in Lua?
I've been researching ZADD modifiers (XX, NX, CH) but none of them do what I need.
Thank you very much!
A Lua script for this CAS use case would be the simplest and idiomatic solution:
127.0.0.1:6379> ZADD myorderset 1 'one' 2 'two' 3 'three'
(integer) 3
127.0.0.1:6379> EVAL "local s = redis.call('ZSCORE', KEYS[1], ARGV[2]) if not s or s > ARGV[1] then redis.call('ZADD', KEYS[1], ARGV[1], ARGV[2]) end" 1 myorderset 5 'three'
(nil)
127.0.0.1:6379> ZRANGE myorderset 0 -1 WITHSCORES
1) "one"
2) "1"
3) "two"
4) "2"
5) "three"
6) "3"
127.0.0.1:6379> EVAL "local s = redis.call('ZSCORE', KEYS[1], ARGV[2]) if not s or s > ARGV[1] then redis.call('ZADD', KEYS[1], ARGV[1], ARGV[2]) end" 1 myorderset 2 'three'
(nil)
127.0.0.1:6379> ZRANGE myorderset 0 -1 WITHSCORES
1) "one"
2) "1"
3) "three"
4) "2"
5) "two"
6) "2"
There is no single command or command option to do it both. You can either use combination of ZSCORE with ZADD(in lua). Alternatively("It is/looks like over-engineered") you may use ZUNIONSTORE with the aggregate option MIN.
With the AGGREGATE option, it is possible to specify how the results of the union are aggregated. This option defaults to SUM, where the score of an element is summed across the inputs where it exists. When this option is set to either MIN or MAX, the resulting set will contain the minimum or maximum score of an element across the inputs where it exists.
127.0.0.1:6379> ZADD base 1 a 2 b 5 c
(integer) 3
127.0.0.1:6379> ZADD new 3 c
(integer) 1
127.0.0.1:6379> ZUNIONSTORE base 2 base new AGGREGATE MIN
(integer) 3
127.0.0.1:6379> DEL new
(integer) 1
127.0.0.1:6379> ZRANGE base 0 -1 WITHSCORES
1) "a"
2) "1"
3) "b"
4) "2"
5) "c"
6) "3"
127.0.0.1:6379> ZADD new 5 b
(integer) 1
127.0.0.1:6379> ZUNIONSTORE base 2 base new AGGREGATE MIN
(integer) 3
127.0.0.1:6379> DEL new
(integer) 1
127.0.0.1:6379> ZRANGE base 0 -1 WITHSCORES
1) "a"
2) "1"
3) "b"
4) "2"
5) "c"
6) "3"
127.0.0.1:6379>
If you prefer,
You may generate new set name with random string in application level
Put EXPIRE to this new set, no need to DEL the new key manually after the ZUNIONSTORE, it will be expired eventually.
It can be done in MULTI/EXEC in a single transaction.
I am using Redis queue and adding the data using ListLeftPush and reading data using ListRightPop. It works fine I am able to get the data. But what if data has not popped out? Can we delete old data? or Can we add Value in Redis List with Expiration Time?
How to add time limit for each value while using ListLeftPush command in C#?
It is not possible to add expire time for individual value for the sake of keeping redis simple and fast.
you can only add expire time for individual keys i.e in ur case it is for whole list.
No redis doesn't support that. Expiration is available only for the top level keys. The closes data type/solution for your case would be sorted sets.
You put your expiration time(timestamp) as score while adding to sorted set(ZADD)
Instead of LPOP you use ZPOPMAX to get "to be last expired" element.
Periodically you may use ZREMRANGEBYSCORE to remove expired elements.
For the demonstration i used smaller numbers as expiration dates.
127.0.0.1:6379> ZADD myset 15 "a"
(integer) 0
127.0.0.1:6379> ZADD myset 25 "b"
(integer) 0
127.0.0.1:6379> ZADD myset 35 "c" 45 "d" 55 "e"
(integer) 0
127.0.0.1:6379> ZRANGE myset 0 -1 WITHSCORES
1) "a"
2) "15"
3) "b"
4) "25"
5) "c"
6) "35"
7) "d"
8) "45"
9) "e"
10) "55"
127.0.0.1:6379> ZPOPMAX myset
1) "e"
2) "55"
127.0.0.1:6379> ZREMRANGEBYSCORE myset -1 15
(integer) 1
127.0.0.1:6379> ZRANGE myset 0 -1
1) "b"
2) "c"
3) "d"
127.0.0.1:6379>
Are these 2 Redis commands different, except for the second having optional LIMIT argument?
http://redis.io/commands/zrange
http://redis.io/commands/zrangebyscore
They are different:
ZRANGE key start stop ...: start and stop are zero-based indexes (i.e they correspond to a position of an element within the sorted set),
ZRANGEBYSCORE key min max ...: min and max refer to scores (i.e they are used to specify a score range).
So, the first one operates by indexes while the second one (as its name implies) operates by scores. Thus they are used for different purposes.
say, the sorted set is:
value score
tom 0
bob 1
alice 100
lucy 102
when you use zrangebyscores, and the max score is 2, the min score is 0, then, you will get tom and bob;
when you use zrange, and the start is 0, the stop is 2, then you will get tom, bob and alice.
127.0.0.1:6379> zadd example 0 tom 1 bob 100 alice 102 lucy
127.0.0.1:6379> zrange example 0 2 WITHSCORES
1) "tom"
2) "0"
3) "bob"
4) "1"
5) "alice"
6) "100"
127.0.0.1:6379> zrangebyscore example 0 2 WITHSCORES
1) "tom"
2) "0"
3) "bob"
4) "1"
this is the key difference.