How do I get the value of multiple keys from redis using a sorted set?
zadd Users 0 David
zadd Users 5 John
zadd Users 15 Linda
zrevrange Users 0 -1 withscores
This will have two users in it.
How can I retrieve the users with key 'David' and 'Linda' in one query?
You can use Redis MGET
redis> MGET key1 key2 nonexisting
1) "Hello"
2) "World"
3) (nil)
More here
http://redis.io/commands/mget
There are multiple ways to do it without introducing a new command in Redis.
For instance, you can fill a temporary set with the names you are interested in, then calculate the intersection between the temporary set and the zset:
multi
sadd tmp David Linda ... and more ...
zinterstore res 2 tmp Users weights 0 1
zrange res 0 -1 withscores
del tmp res
exec
With pipelining, this will only generate one roundtrip and you can fill an arbitrary number of input parameters in tmp.
With Redis 2.6, you can also wrap these lines into a server-side Lua script to finally get a command accepting an input list and returning the result you want:
eval "redis.call( 'sadd', 'tmp', unpack(KEYS) );
redis.call( 'zinterstore', 'res', 2, 'tmp', 'Users', 'weights', 0, 1 );
local res = redis.call( 'zrange', 'res', 0, -1, 'withscores' );
redis.call( 'del', 'res', 'tmp' ) ;
return res
" 2 David Linda
You can safely assume no new command will be added to Redis if it can easily been implemented using scripting.
One uses a sorted set because you want to deal with items that are sorted. What you are asking for is to not use a sorted set as a sorted set. If you don't care about sort order, then perhaps a sorted set is not what you are looking for. You already can retrieve multiple keys, but not arbitrary ones.
If your primary goal is to retrieve multiple arbitrary keys, use a hash and hmget. If your primary need is to access a sorted set, use sorted set and either go the scripting route or pipeline a series of zscore calls.
You cannot get this with one command. The closest you can do to get it in one response:
MULTI
ZSCORE Users David
ZSCORE Users Linda
EXEC
EDIT: Alternatively, you can maintain a parallel hash with users' scores, and query it with
HMGET UserScores David Linda
Related
I am new to Redis, and now I need to update the a sorted set if the key exists in another sorted set.
I think it may be clearer to explain by an example, lets say that there are two sorted sets like the following:
set_1
{key_1:val_1, key_2:val_2, key_3:val_3}
set_2
{key_1:val_new_1, key_3:val_new_3, key_4:val_new_4}
Now I am trying to update the first set if the key exists in the second set, so the result should be:
set_1
{key_1:val_new_1, key_2:val_2, key_3:val_new_3}
I have been reading the Redis documentation for a while, and it seems using he SET command with XX option may help:
The SET command supports a set of options that modify its behavior:
XX -- Only set the key if it already exist.
But is it possible to avoid running this on each entry in the first set? Maybe using something like zunionstore?
The SET command only works for regular keys, not for sorted sets.
In sorted sets, you have score-member pairs, so the key-value pair nomenclature of your example is a little confusing. I'll assume key_1, key_2, key_3, ... are members and val_1, val_2, ... are scores.
Let's create the sorted sets as follows to review the solution:
> ZADD set_1 1 key_1 2 key_2 3 key_3
(integer) 3
> ZADD set_2 1001 key_1 1003 key_3 1004 key_4
(integer) 3
The default AGGREGATE is SUM, it's what we will use all across.
We'll create two sorted sets with the intersection of both, one with the scores of set_1 and one with the scores of set_2.
> ZINTERSTORE intersect_set_1 2 set_1 set_2 WEIGHTS 1 0
(integer) 2
> ZINTERSTORE intersect_set_2 2 set_1 set_2 WEIGHTS 0 1
(integer) 2
Now, we create a middle-step set for set_1, where we set the score to zero for those that are in set_2 as well:
> ZUNIONSTORE pre_set_1 2 set_1 intersect_set_1 WEIGHTS 1 -1
(integer) 3
Now we are ready to update set_1, doing a union of:
pre_set_1: all set_1 but with those also in set_2 set to zero score.
intersect_set_2: the intersection of set_1 and set_2, with the scores of set_2.
Here is the final command:
> ZUNIONSTORE set_1 2 pre_set_1 intersect_set_2
(integer) 3
Let's see the result:
> ZRANGE set_1 0 -1 WITHSCORES
1) "key_2"
2) "2"
3) "key_1"
4) "1001"
5) "key_3"
6) "1003"
Don't forget to clean up:
> UNLINK pre_set_1 intersect_set_1 intersect_set_2
This solution is not optimal as it uses multiple middle-steps, there is a risk for members added to the original sets in between and it uses more memory than necessary.
The optimal solution would be a Lua script:
local set2 = redis.call('ZRANGE', KEYS[1], '0', '-1', 'WITHSCORES')
local set2length = table.getn(set2)
for i=1,set2length,2 do redis.call('ZADD', KEYS[2], 'XX', set2[i+1], set2[i]) end
return set2length/2
This loops through set_2, updating set_1. Note the use of XX in the ZADD command, to only update if it exists.
Use as:
EVAL "local set2 = redis.call('ZRANGE', KEYS[1], '0', '-1', 'WITHSCORES') \n local set2length = table.getn(set2) \n for i=1,set2length,2 do print(1) redis.call('ZADD', KEYS[2], 'XX', set2[i+1], set2[i]) end \n return set2length/2" 2 set_2 set_1
The Lua script is atomic due to the single-threaded nature of Redis.
I want to keep some user feedbacks in redis. Some users may give multiple feedbacks. The users are assigned numerical user id
Here is an example:
zadd feedbacks 1 feedback1 2 feedback2 3 feedback3 1 feedback4
In this case user #1 gave feedbacks feedback1 and feedback4, #2 feedback2 and #3 feedback3.
If I use ZRANGEBYSCORE feedbacks 1 1
I will be able to see the feedback from user #1:
1) "feedback1"
2) "feedback4"
However I want to store more than just the text. I want to be able to retrieve the timestamp for example. Is there any way I can insert a hash vale to the key feedbacks above?
Something likes zadd feedbacks 1 text:feedback1 timestamp:123456
No, you cannot. Lists, sets, hashes and sorted sets only support Redis' string data type for the values.
You can stringify your field-value pairs though, using JSON or your preferred format.
ZADD feedbacks 1 "{\"text\":\"feedback1\",\"timestamp\":\"123456\""
Unless you need to modify a given field atomically, this approach should do.
And even in that case, you can use Lua scripts to achieve server-side JSON manipulation and updates. See How to nest a list into a structure in Redis to reduce top level? for a similar solution.
Secondary indexes
But you may want to query multiple ways: by user id, by timestamp, etc.
In that case, consider using regular keys to store the feedback object, say as a hash.
HSET feedbacks:feedback1 text feedback1 timestamp 123456 user 1 ...
And your indexes:
ZADD feedbacks-by-user 1 feedback1
ZADD feedbacks-by-timestamp 123456 feedback1
...
Say you want all feedbacks of a given user:
ZRANGEBYSCORE feedbacks-by-user 1 1
Then you go get the values for the returned keys. Of course, you may want to avoid the two round trips. Again, Lua script.
The script:
local keys = redis.call('ZRANGEBYSCORE', KEYS[1], ARGV[1], ARGV[1])
for i, v in ipairs(keys) do
local k = {}
k[1] = v
k[2] = redis.call('HGETALL', 'feedbacks:'..v)
keys[i] = k
end
return keys
Usage:
> EVAL "local keys = redis.call('ZRANGEBYSCORE', KEYS[1], ARGV[1], ARGV[1]) \n for i, v in ipairs(keys) do \n local k = {} \n k[1] = v \n k[2] = redis.call('HGETALL', 'feedbacks:'..v) \n keys[i] = k \n end \n return keys" 1 feedbacks-by-user 1
1) 1) "feedback1"
2) 1) "text"
2) "feedback1"
3) "timestamp"
4) "123456"
5) "user"
6) "1"
2) 1) "feedback4"
2) 1) "text"
2) "feedback4"
3) "timestamp"
4) "465465"
5) "user"
6) "1"
You can similarly query for a range of timestamps.
You can mix and match your queries using ZINTERSTORE or ZUNIONSTORE.
You may be interested in How to store in Redis sorted set with server-side timestamp as score?. You can make a nice Lua script to take care of creating the hash, and the secondary index entries, all in one go with redis-server-side timestamping.
Finally, whenever using Lua on Redis, consider to load the script and use EVALSHA.
I have an index for each user which represents their current score.
I want to find all users with the same score.
Provided that the input is as follows, how can I retrieve only users 1 and 2?
ZADD users 0 1:10
ZADD users 0 2:10
ZADD users 0 3:5
I would expect to do something like mentioned in the docs but I cannot use the key as my index since it is too large (for the post I changed to 1,2,3 so it is not an issue), which is why I set it to 0.
I thought the command could be something like ZRANGEBYLEX users [0:10 (:10 but that returns all 3.
1) "1:10"
2) "2:10"
3) "3:5"
I was able to come up with a solution for this problem, which can further refine as:
Retrieve all users with a given score or retrieve a single score for a single user.
What I determined was that using the namespace users is not a sufficient descriptor for both the score and user id. When using 2 different keys, it is clearly possible to get either a range or a single value. The downside is twice the writes I had hoped, but that is overall not a lot of overhead for me and the only way I found to properly retrieve both score and user.
ZADD score:users 0 10:1
ZADD score:users 0 10:2
ZADD score:users 0 5:3
ZADD users:score 0 1:10
ZADD users:score 0 2:10
ZADD users:score 0 3:5
ZRANGEBYLEX users:score [1: [1:\xff # "1:10"
ZRANGEBYLEX score:users [10: [10:\xff # "10:1" "10:2"
I have a large set of ids (around 100000) which I want to store in redis.
I am looking for the most optimal way through which I can check if a given list of ids, what are the ids that are part of my set.
If I use a redis set, I can use SISMEMBER to check if a id is part of my set, but in this case I want to check if, given a list of ids, which one is part of my set.
Example:
redis> SADD myset "1"
(integer) 1
redis> SADD myset "2"
(integer) 2
redis> MYCOMMAND myset "[1,2,4,5]"
(list) 1, 2
Does anything of this sort exist already ?
thanks !
I'm planning to start using hashes insead of regular keys. But I can't find any information about multi get for hash-keys in Redis wiki. Is this kind of command is supported by Redis?
Thank you.
You can query hashes or any keys in pipeline, i.e. in one request to your redis instance. Actual implementation depends on your client, but with redis-py it'd look like this:
pipe = conn.pipeline()
pipe.hgetall('foo')
pipe.hgetall('bar')
pipe.hgetall('zar')
hash1, hash2, hash3 = pipe.execute()
Client will issue one request with 3 commands. This is the same technique that is used to add multiple values to a set at once.
Read more at http://redis.io/topics/pipelining
No MHGETALL but you can Lua it:
local r = {}
for _, v in pairs(KEYS) do
r[#r+1] = redis.call('HGETALL', v)
end
return r
If SORT let you use multiple GETs with the -> syntax, and all your hashes had the same fields, you could get them in a bulk reply by putting their names into a set and sorting that.
SORT names_of_hashes GET *->field1 *->field2 *->field3 *->etc
But it doesn't look like you can do that with the hash access. Plus you'd have to turn the return list back into hashes yourself.
UPDATE: Redis seems to let you fetch multiple fields if you name your hashes nicely:
redis> hset hash:1 name fish
(integer) 1
redis> hset hash:2 name donkey
(integer) 1
redis> hset hash:3 name horse
(integer) 1
redis> hset hash:1 type fish
(integer) 1
redis> hset hash:2 type mammal
(integer) 1
redis> hset hash:3 type mammal
(integer) 1
redis> sadd animals 1
(integer) 1
redis> sadd animals 2
(integer) 1
redis> sadd animals 3
(integer) 1
redis> sort animals get # get hash:*->name get hash:*->type
1. "1"
2. "fish"
3. "fish"
4. "2"
5. "donkey"
6. "mammal"
7. "3"
8. "horse"
9. "mammal"
There is no command to do it on one shot, but there is a way to do it "nicely", using a list (or sorted set) where you would store you hashKeys, and then retrieve them as bulk using multi.
In PHP:
$redis->zAdd("myHashzSet", 1, "myHashKey:1");
$redis->zAdd("myHashzSet", 2, "myHashKey:2");
$redis->zAdd("myHashzSet", 3, "myHashKey:3");
$members = $redis->zRange("myHashzSet", 0, -1);
$redis->multi();
foreach($members as $hashKey) {
$redis->hGetAll($hashKey);
}
$results = $redis->exec();
I recommand using a sorted set, where you use the score as an ID for your hash, it allows to take advantages of all score based command.
Redis has a HMGET command, which returns the values of several hash keys with one command.