I use sorted set in Redis.
The common value of data in sorted set is over one million. How can I read this sorted set by partitions? I mean first 100 000 rows and the following?
There is only one command to take data: smembers set
You can use the ZRANGE command on your sorted set and specify the start and stop to get 100,000 entries, and then 100,001 to 200,000 for the next ZRANGE.
ZRANGE documentation on Redis.io
You mentioned using smembers set to take data, but that is used only on non-sorted sets. If you are actually using a non-sorted set, then you would need to use SPOP and define your count at 100,000. However, this would simultaneously remove all those entries.
SPOP documentation on Redis.io
You can iterate through the elements of an unsorted set incrementally using SSCAN. Start with cursor 0 and use the returned cursor in subsequent calls, until 0 is returned again.
pantalones:6379> SSCAN five-characters 0 COUNT 3
1) "7"
2) 1) "d"
2) "e"
3) "a"
4) "c"
pantalones:6379> SSCAN five-characters 7 COUNT 3
1) "0"
2) 1) "b"
In this example, the first call to SSCAN returns a cursor of 7, which is then provided to the second call to SSCAN. The second call returns a cursor of 0, so we know the iteration is complete.
See SSCAN documentation on Redis.io.
Related
In the redis documentation, there is this example of ZRANGEBYLEX
ZADD myindex 0 0056:0028.44:90
ZADD myindex 0 0034:0011.00:832
ZRANGEBYLEX myindex [0056:0010.00 [0056:0030.00
1) "0056:0028.44:90"
It is very straightforward.
However if I want to apply the same technique to the following example,
127.0.0.1:6379> zadd feedbacks 1 feedback1 2 feedback2 3 feedback3 1 feedback4
(integer) 4
127.0.0.1:6379> ZRANGEBYLEX feedbacks [feed [feed
(empty list or set)
I get an empty set.
I would expect to see the four values (feedback1 to feedback4)
Why ZRANGEBYLEX failed on my test sample?
It fails because they have different scores. ZRANGEBYLEX works only on same-score subsets.
See https://redis.io/commands/ZRANGEBYLEX
When all the elements in a sorted set are inserted with the same
score, in order to force lexicographical ordering, this command
returns all the elements in the sorted set at key with a value between
min and max.
If the elements in the sorted set have different scores, the returned
elements are unspecified.
Sorted sets have the property of being lexicographically ordered within same-score subsets. This gives them a second use-case, a lexicographically sorted set, but in this case, you add all elements with the same score.
So, you have to choose how to use your sorted set:
Sorted by score (with same-score sorted lex, for predictable order)
Sorted lexicographically, all elements given the same score
You cannot have both. You'd need two sorted sets then.
my redis version:3.0.2
Hash data as below show.
key name:test
contents(values):
1) "xx1"
2) "1"
3) "xx2"
4) "2"
5) "xx3"
6) "3"
7) "xx4"
8) "4"
9) "xx5"
10)"5"
use commond -->HSCAN test 0 COUNT 2
Redis return every key and value, not the first of 2 keys and values!
COUNT option for SCAN does not limit the number of key-values returned.
It is used to force the command to increase the number key-values returned.
Redis COUNT option doc:
When iterating Sets encoded as intsets (small sets composed of just
integers), or Hashes and Sorted Sets encoded as ziplists (small hashes
and sets composed of small individual values), usually all the
elements are returned in the first SCAN call regardless of the COUNT
value.
So, get first two values from the result of HSCAN test 0 command.
I got several sorted sets with a common prefix (itemmovements:) in Redis.
I know we can use ZCOUNT to get the number of items for a single (sorted set) key like this:
127.0.0.1:6379> zcount itemmovements:8 0 1000000000
(integer) 23
(I am able to do this, since I know the range of the item scores.)
How to run this in a loop for all keys prefixed itemmovements:?
Taking hint from How to atomically delete keys matching a pattern using Redis I tried this:
127.0.0.1:6379> EVAL "return redis.call('zcount', unpack(redis.call('keys', ARGV[1])), 0, 1000000000)" 0 itemmovements:*
(integer) 150
but as you can see it just returns a single number (which happens to be the size of itemmovements:0, the first value returned by keys).
I realized I did not understand what that lua code in EVAL was doing. The code below works fine:
eval "local a = {}; for _,k in ipairs(redis.call('keys', 'itemmovements:*')) do table.insert(a, k); table.insert(a, redis.call('zcount', k, 0, 1000000000)); end; return a" 0
I have a table in sql with 3 columns: BIGINT StartNumber, BIGINT EndNumber, BIGINT LocationId, and I need to be able to do something like this
Select LocationId where StartNumber < #number and EndNumber > #number.
for example:
StartNumber EndNumber LocationId
1 5 1
6 9 1
10 16 2
and when I have #number = 7 I should get LocationId = 1
How can I do this in redis?
I was thinking to move this table to redis, use sorted set and ZRANGEBYSCORE but it did't work for me:
1) When I am using ZADD key score member [score] [member], I am unable to add 2 elements with the same member and different score even with nx parameter:
zadd myset nx 1 "17" 2 "17" - it will add one element and then update its score instead of adding two elements.
2) when I am adding this: zadd set1 2 "a" 4 "b" 6 "c" 10 "d" and then trying to do zrangebyscore set1 3 3 (want to get member whose score include 3) I em getting empty result
P.s. All commands are executed on the example pages of redis website.
So as I understood the task, you don't have overlaps and each interval maps to only one location (?) and intervals don't have gaps. Based on this you can use only one sorted list with lower (or upper) bound values:
ZADD StartNumber 1 "1:5:1" 6 "6:9:1" 10 "10:16:2"
Then you can use:
ZREVRANGEBYSCORE StartNumber 7 -inf LIMIT 0 1
And it will be O(log(N)).
Put differently, your question is "how can I map N ranges of numbers to a location". One way of doing this is using two Sorted Sets, one for the StartNumber and the other one for EndNumber. Since members have to be unique, we'll also need to ensure that by using the Start/End values as part of the member. For example, with your example data, this could be done like so:
ZADD StartNumber 1 "1:5:1" 6 "6:9:1" 10 "10:16:2"
ZADD EndNumber 5 "1:5:1" 9 "6:9:1" 16 "10:16:2"
To find the location for #number=7, do ZRANGEBYSCORE StartNumber -inf 7 and ZRANGEBYSCORE EndNumber 7 +inf and intersect the results. All that remains is to split the intesect's result(s) on the colon (:) and use the 3rd element as the location.
Note: if your app ensures that there are no overlapping ranges and that there can be only one location per "number", you can get the same results with only one set.
(this is the first time that I'm giving two answers to the same question - maybe I'll get a badge or sumthin' ;))
The double Sorted Set approach is a generalization and, as such, aims to solve a bigger set of problems than what the OP needs (as put in the comments to the first answer). That approach is also not effective as the query is O(logn)+O(N) so when N is large (e.g. 5M) that's probably not a good idea.
However, to satisfy the requirements and given that the ranges do not overlap, one could actually use only a single Sorted Set and a simpler query. The set's members should be added by concatenating the EndNumber and LocationId and the their scores should be set to their respective StartNumber, so for the sake of the example:
ZADD ranges 1 "5:1" 6 "9:1" 10 "16:2"
Given #number, obtain the relevant LocationId with the following Redis Lua code (O(logn)):
-- rangelookup.lua
-- http://stackoverflow.com/questions/32185898/redis-get-member-where-score-is-between-min-and-max/32186675
-- A **non inclusive** range search on a Sorted Set with the following data:
-- score = <StartNumber>
-- member = <EndNumber>:<LocationId>
--
-- KEYS[1] - Sorted Set key name
-- ARGV[1] - the number to search
--
-- reply - the relevant id, nil if range doesn't exist
--
-- usage example: redis-cli --eval rangelookup.lua ranges , 7
local number = tonumber(ARGV[1])
local data = redis.call('ZREVRANGEBYSCORE', KEYS[1], number, '-inf', 'WITHSCORES', 'LIMIT', 0, 1)
local reply = nil
if data ~= nil and number > tonumber(data[2]) then
local to, id = data[1]:match( '(.*):(.*)' )
if tonumber(to) > number then
reply = id
end
end
return reply
Sample output:
$ redis-cli --eval rangelookup.lua ranges , 7
"1"
$ redis-cli --eval rangelookup.lua ranges , 9
(nil)
$ redis-cli --eval rangelookup.lua ranges , 99
(nil)
For lists I can do the operation:
LLEN KeyName
and it will return the size of a list in Redis. What is the equivalent command for sets? I can't seem to find this in any documentation.
You are looking for the SCARD command:
SCARD key
Returns the set cardinality (number of elements) of the set stored at
Return value
Integer reply: the cardinality (number of elements) of the set, or 0 if key does not exist.
You can view all of the set commands on the documentation webpage.
If it's a sorted set, you can use
ZCOUNT myset -inf +inf
or
ZCARD myset
zCard is short for cardinality (cardinality is the number of elements in a set). It gives you total number of members inside of a "sorted set".
Sometimes you might wanna extract how many members are inside of a range in a sorted set. For that you can use zCount.
ZCOUNT cars 0 50 // inclusive
this will include 0 and 55. 0 <= .... <=50. But if you do not want to include them
ZCOUNT cars (0 (50
if it is regular set
SCARD cars