Is there a command in Redis for HASH data structure similar to MGET? - redis

I need to get in one call all data fields for a set of known REDIS hash keys. I've used MGET for string keys such as :
MGET key [key ...]
Available since 1.0.0.
Time complexity: O(N) where N is the number of keys to retrieve.
Returns the values of all specified keys. For every key that does not hold a string value or does not exist, the special value nil is returned. Because of this, the operation never fails.
HMGET only brings all fields for one key. I need many keys all fields by key.

There is no command like that, redis hashes work within the hash, so HMGET work inside one hash and give all the fields in that hash. There is no way to access all the fields in multiple hashes at ones.
However you can user several HMGET on each hash and get all the fields. you can pipeline these commands to execute in a one go.
Option 1
Ex. implementation in pseudo code
Pipeline p
List<String> = p.hgetall('key1', fields...);
List<String> = p.hgetall('key2', fields...);
List<String> = p.hgetall('key3', fields...);
p.exec();
Option 2
Other option is to write a LUA script and call that using EVAL
local array = {}
local keys = redis.call('KEYS', '<your pattern>')
for _,key in ipairs(keys) do
local val = redis.call('HGETALL', key)
array[#array + 1] = val
end
return array
Call the lua sctipt
redis-cli EVAL "$(cat test.lua)" 0
1) 1) "field1"
2) "val"
2) 1) "field1"
2) "val"
3) "field2"
4) "val2"

As noted in another answer, there's no built in way but there more workarounds besides a transaction.
Option 1: use a Lua script (i.e. EVAL "..." 3 myhash1 myhash2 myhash3 myfield)
local r = {}
while (#KEYS > 0) do
local k = table.remove(KEYS,1)
r[#r+1] = redis.call('HGET', k, ARGV[1])
end
return r
Option 2: write a Redis module
Out of scope as a an answer :)

Related

Redis using members of sorted list to delete external keys

Using sort one can sort a set and get external keys using results from the sort component of query.
By way of example:
If the external key/value are defined as various keys using the pattern:itemkey:<somestring>
And a sorted list has list of the members then issuing command sort <lists key> by nosort get itemkey:* would get the values of the referenced keys.
I would like to be able to sort through a sorted list and delete these individual keys but it appears that sort <key> by nosort del itemkey:* is not supported.
Any suggestions on how to get list of values stored in a set and then delete the external keys?
Obviously I can do this with two commands, first getting the list of values and then by iterating through list call the delete function - but this is not desirable as I requite atomic operation.
To ensure atomic operation one can use either transactions or redis' lua scripts. For efficiency I decided to go with using script. This way the entire script is completed before next redis action/request is processed.
In code snippet below. I used loadScript in order to store script redis side reducing traffic with every call, the response from loadScript is then used as identifier to Jedis's evalsha command.
Using Scala (Note Jedis is a Java library, hence the .asJava):
val scriptClearIndexRecipe = """local names = redis.call('SORT', KEYS[1]);
| for i, k in ipairs(names) do
| redis.call('DEL', "index:recipe:"..k)
| end;
| redis.call('DEL', KEYS[1]);
| return 0;""".stripMargin
def loadScript(script: String): String = client.scriptLoad(script)
def eval(luaSHA: String, keys: List[String], args: List[String]): AnyRef = {
client.evalsha(luaSHA, keys.asJava, args.asJava)
}

Redis: How to increment hash key when adding data?

I'm iterating through data and dumping some to a Redis DB. Here's an example:
hmset id:1 username "bsmith1" department "accounting"
How can I increment the unique ID on the fly and then use that during the next hmset command? This seems like an obvious ask but I can't quite find the answer.
Use another key, a String, for storing the last ID. Before calling HMSET, call INCR on that key to obtain the next ID. Wrap the two commands in a MULTI/EXEC block or a Lua script to ensure the atomicity of the transaction.
Like Itamar mentions you can store your index/counter in a separate key. In this example I've chosen the name index for that key.
Python 3
KEY_INDEX = 'index'
r = redis.from_url(host)
def store_user(user):
r.incr(KEY_INDEX, 1) # If key doesn't exist it will get created
index = r.get(KEY_INDEX).decode('utf-8') # Decode from byte to string
int_index = int(index) # Convert from string to int
result = r.set('user::%d' % int_index, user)
...
Note that user::<index> is an arbitrary key chosen by me. You can use whatever you want.
If you have multiple machines writing to the same DB you probably want to use pipelines.

Deleting a complete folder in Redis

How can I delete a complete folder in redis, using DEL command OR in C# in StackExchange.Redis.
SomeFolder1:SomeSubfolder1:somekey1
SomeFolder1:SomeSubfolder1:somekey2
SomeFolder1:SomeSubfolder2:somekey1
How can I Del SomeFolder1:SomeSubfolder1 so remaining keys are
SomeFolder1:SomeSubfolder2:somekey1
There's no such thing as a "folder" in redis, and no functionality to del based on a pattern. Options:
use a database; you can have an arbitrary number of databases (each is numbered, the default is 0), and you can discard a database in a single operation: flushdb - so keep all this associated data in one database and you're sorted
use scan to iterate the matching keys, issuing a del for each match
note that scan is a server command, so you need:
const int db = 0;
var server = muxer.GetServer(...);
var db = muxer.GetDatabase(db);
foreach(var key in server.Keys(db, "SomeFolder1:SomeSubfolder1:*",
pageSize: 500))
{
db.KeyDelete(key, flags: CommandFlags.FireAndForget);
}
the FireAndForget allows you to ignore the individual replies, which means you aren't bound by latency and you don't have TPL overheads.
Fundamentally, though: redis is not meant to be used like this - if you find yourself scanning keys, you are doing something wrong. A more typical implementation might be to use a hash to store the folder (which each key/value pair inside the hash being the contents) - then deleting the hash is one operation. Alternatively, use a set to store the keys of the items inside each logical folder.
Hash approach:
hash = SomeFolder1:SomeSubfolder1
key = somekey1, value = ...
key = somekey2, value = ...
hash = SomeFolder1:SomeSubfolder2
key = somekey1, value = ...
Set approach:
set: SomeFolder1:SomeSubfolder1
somekey1
somekey2
set: SomeFolder1:SomeSubfolder2
somekey1
string: SomeFolder1:SomeSubfolder1:somekey1
value
string: SomeFolder1:SomeSubfolder1:somekey2
value
string: SomeFolder1:SomeSubfolder2:somekey1
value

Get redis hash data via cli

I've been experimenting with Redis today. I've managed to store cached values from Drupal, but I'm looking to investigate a bit further and view the value stored in cache_my_custom_cache..
127.0.0.1:6379> keys *
1) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:lookup_cache"
2) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:module_implements"
3) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:bootstrap_modules"
4) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:system_list"
5) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:variables"
6) "ff3169bd93659dc31322abc32835ef3e:path:a:und"
7) "myhash"
8) "ff3169bd93659dc31322abc32835ef3e:path:s:und"
9) "ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup"
10) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:hook_info"
127.0.0.1:6379> type ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup
hash
this reveals it's of type hash.. But when I run:
127.0.0.1:6379> HGET ff3169bd93659dc31322abc32835ef3e:cache_qbe:qbe_markup
(error) ERR wrong number of arguments for 'hget' command
It doesn't like it! Completely new to this, can someone offer a solution?
Besides the key you will need to specify the field in the HGET command.
To get a list of all fields in the hash you can run this:
hkeys ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup
Then you can do:
hget ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup FIELD
You can also get all the values in the hash like this:
hvals ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup
HGET expects an additional parameter after the key name that enumerates which field of your hash you would like returned. Something like this:
HGET my_hash_key my_hash_field
If you're trying to retrieve all fields of your hash at once, you should use HGETALL:
HGETALL my_hash_key
Documentation for HGET: here
Documentation for HGETALL: here
Discussion of Redis types (including hashes): here

How to retrieve all hash values from a list in Redis?

In Redis, to store an array of objects we should use hash for the object and add its key to a list:
HMSET concept:unique_id name "concept"
...
LPUSH concepts concept:unique_id
...
I want to retrieve all hash values (or objects) in the list, but the list contains only hash keys so a two step command is necessary, right? This is how I'm doing in python:
def get_concepts():
list = r.lrange("concepts", 0, -1)
pipe = r.pipeline()
for key in list:
pipe.hgetall(key)
pipe.execute()
Is it necessary to iterate and fetch each individual item? Can it be more optimized?
You can use the SORT command to do this:
SORT concepts BY nosort GET concept:*->name GET concept:*->some_key
Where * will expand to each item in the list.
Add LIMIT offset count for pagination.
Note that you have to enumerate each field in the hash (each field you want to fetch).
Another option is to use the new (in redis 2.6) EVAL command to execute a Lua script in the redis server, which could do what you are suggesting, but server side.