I'm currently playing around with Redis and i've got a few questions. Is it possible to get values from an array of keys?
Example:
users:1:name "daniel"
users:1:age "24"
users:2:name "user2"
users:2:age "24"
events:1:attendees "users:1", "users:2"
When i redis.get events:1:attendees it returns "users:1", "users:2". I can loop through this list and get users:1, get users:2. But this feels wrong, is there a way to get all the attendees info on 1 get?!
In rails i would do something like this:
#event.attendees.each do |att|
att.name
end
But in redis i can't because it returns the keys and not the actual object stored at that key.
thanks :)
Doing a loop on the items and synchronously accessing each element is not very efficient. With Redis 2.4, there are various ways to do what you want:
by using the sort command
by using pipelining
by using variadic parameter commands
With Redis 2.6, you can also use Lua scripting, but this is not really required here.
By the way, the data structure you described could be improved by using hashes. Instead of storing user data in separate keys, you could group them in a hash object.
Using the sort command
You can use the Redis sort command to retrieve the data in one roundtrip.
redis> set users:1:name "daniel"
OK
redis> set users:1:age 24
OK
redis> set users:2:name "user2"
OK
redis> set users:2:age 24
OK
redis> sadd events:1:attendees users:1 users:2
(integer) 2
redis> sort events:1:attendees by nosort get *:name get *:age
1) "user2"
2) "24"
3) "daniel"
4) "24"
Using pipelining
The Ruby client support pipelining (i.e. the capability to send several queries to Redis and wait for several replies).
keys = $redis.smembers("events:1:attendees")
res = $redis.pipelined do
keys.each do |x|
$redis.mget(x+":name",x+":age")
end
end
The above code will retrieve the data in two roundtrips only.
Using variadic parameter command
The MGET command can be used to retrieve several data in one shot:
redis> smembers events:1:attendees
1) "users:2"
2) "users:1"
redis> mget users:1:name users:1:age users:2:name users:2:age
1) "daniel"
2) "24"
3) "user2"
4) "24"
The cost here is also two roundtrips. This works if you can guarantee that the number of keys to retrieve is limited. If not, pipelining is a much better solution.
You can use Redis' EVAL command to send it a Lua script that runs a loop "server side" and return the results in a block.
Related
I have multiple hash-keys organized in "folders" like MOC273,
127.0.0.1:6379> HMSET MOC273:123654789 "H_W" "json->H_W->123654789" "H_Y" "json->H_Y->123654789"
OK
127.0.0.1:6379> HMSET MOC273:987654321 "H_W" "json->H_W->987654321" "H_Y" "json->H_Y->987654321"
OK
With HGETALL I can get all the data from a hash
127.0.0.1:6379> HGETALL MOC273:123654789
1) "H_W"
2) "json->H_W->123654789"
3) "H_Y"
4) "json->H_Y->123654789"
How can I get all the hashes contained in MOC273(where the name starts with MOC273) Something like.
127.0.0.1:6379> HGETALL MOC273*
It's possible ?
Current structure of the redis
So you can use Redisearch to create a secondary index around the HASH data structure. This will allow you to search through HASH data more efficiently.
https://oss.redis.com/redisearch/ - Quick start should help you to get started
I am new to Redis I tried to figure out this problem going through Redis documentation but no luck. Here are the details.
Lets say I inserted Strings like below.
Set category:1 "Men"
Set category:2 "Women"
Set category:3 "Kids"
Set category:4 "Home"
Set category:5 "shoes" ...
Now I want to get all the values by querying with keys which follow certain pattern in this case category:*.
Get category:*
Is there any way to get all categories like this?
Use SCAN. SCAN is the only safe way to iterate through the keys in a Redis database. SCAN will chunk out a portion of the keyspace and return a cursor (always the first result) and any values it found in that chunk. You start with a cursor of 0.
> SCAN 0 MATCH "category:*"
1) "1904"
2) (empty list or set)
Then you pass that cursor back into the SCAN command with the same pattern:
> SCAN 1904 MATCH "category:*"
1) "0"
2) 1) "category:3"
2) "category:2"
3) "category:4"
4) "category:1"
In this case the cursor returned is 0 which is the signal that the SCAN command has completed. The second response is an array with the keys found. Note that you need to run the SCAN command in a loop and none or only part of keys that match the pattern will be returned each time.
After you get the keys, you'll need to retrieve the values as normal (GET).
Just a note: From the look of how your data is structured, you're likely using an inappropriate data type: The categories would be better organized into a hash (e.g. HSET categories 1 men then you can use HGETALL)
I need to get in one call all data fields for a set of known REDIS hash keys. I've used MGET for string keys such as :
MGET key [key ...]
Available since 1.0.0.
Time complexity: O(N) where N is the number of keys to retrieve.
Returns the values of all specified keys. For every key that does not hold a string value or does not exist, the special value nil is returned. Because of this, the operation never fails.
HMGET only brings all fields for one key. I need many keys all fields by key.
There is no command like that, redis hashes work within the hash, so HMGET work inside one hash and give all the fields in that hash. There is no way to access all the fields in multiple hashes at ones.
However you can user several HMGET on each hash and get all the fields. you can pipeline these commands to execute in a one go.
Option 1
Ex. implementation in pseudo code
Pipeline p
List<String> = p.hgetall('key1', fields...);
List<String> = p.hgetall('key2', fields...);
List<String> = p.hgetall('key3', fields...);
p.exec();
Option 2
Other option is to write a LUA script and call that using EVAL
local array = {}
local keys = redis.call('KEYS', '<your pattern>')
for _,key in ipairs(keys) do
local val = redis.call('HGETALL', key)
array[#array + 1] = val
end
return array
Call the lua sctipt
redis-cli EVAL "$(cat test.lua)" 0
1) 1) "field1"
2) "val"
2) 1) "field1"
2) "val"
3) "field2"
4) "val2"
As noted in another answer, there's no built in way but there more workarounds besides a transaction.
Option 1: use a Lua script (i.e. EVAL "..." 3 myhash1 myhash2 myhash3 myfield)
local r = {}
while (#KEYS > 0) do
local k = table.remove(KEYS,1)
r[#r+1] = redis.call('HGET', k, ARGV[1])
end
return r
Option 2: write a Redis module
Out of scope as a an answer :)
I've been experimenting with Redis today. I've managed to store cached values from Drupal, but I'm looking to investigate a bit further and view the value stored in cache_my_custom_cache..
127.0.0.1:6379> keys *
1) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:lookup_cache"
2) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:module_implements"
3) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:bootstrap_modules"
4) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:system_list"
5) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:variables"
6) "ff3169bd93659dc31322abc32835ef3e:path:a:und"
7) "myhash"
8) "ff3169bd93659dc31322abc32835ef3e:path:s:und"
9) "ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup"
10) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:hook_info"
127.0.0.1:6379> type ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup
hash
this reveals it's of type hash.. But when I run:
127.0.0.1:6379> HGET ff3169bd93659dc31322abc32835ef3e:cache_qbe:qbe_markup
(error) ERR wrong number of arguments for 'hget' command
It doesn't like it! Completely new to this, can someone offer a solution?
Besides the key you will need to specify the field in the HGET command.
To get a list of all fields in the hash you can run this:
hkeys ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup
Then you can do:
hget ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup FIELD
You can also get all the values in the hash like this:
hvals ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup
HGET expects an additional parameter after the key name that enumerates which field of your hash you would like returned. Something like this:
HGET my_hash_key my_hash_field
If you're trying to retrieve all fields of your hash at once, you should use HGETALL:
HGETALL my_hash_key
Documentation for HGET: here
Documentation for HGETALL: here
Discussion of Redis types (including hashes): here
I'm trying to implement the redis solution suggested here: http://www.slideshare.net/cacois/cois-palkostrata2013: it is the best I have found so far.
I have the following data structure "hash"
{'user': username, 'text': sometext, 'time': 1400543375}
I want to save the data in a log, the slides suggest saving it in a sorted list. When I try this:
zadd mysortedset 1400543375 {'user': username, 'text': sometext, 'time': 1400543375}
I get
Invalid argument(s)
so I thought I should create the hash first and then add it to the set
127.0.0.1:6379> HMSET setmember:1400543375 user username text sometext time 1400543375
OK
and then
127.0.0.1:6379> zadd mysortedset 1400543375 setmember:1400543375
(integer) 1
Is this the way it is supposed to be done?
What’s simply missing are the quotes around your JSON. Otherwithe it tries to interpret your zadd as scove value score value. And „username,“ is no valid score. ;) I tried it with
zadd mysortedset 1234 ”{’user’: foo …}”
and it works.
A log suggest you get the entries in the right order to begin with. In which case use the l* commands (lpush,lpop etc)