Is there a simple way to know from my whole DB which keys are accessed more frequently? Maybe with command OBECT FREQ? https://redis.io/commands/object
But I can't run the command on every key...
OBJECT FREQ is indeed the way to go, and you can SCAN your database's keyspace and call it on each key.
That said, redis-cli already implements this (SCAN + FREQ) pattern when invoked with the --hotkeys switch, i.e.:
$ redis-cli --hotkeys
# Scanning the entire keyspace to find hot keys as well as
# average sizes per key type. You can use -i 0.1 to sleep 0.1 sec
# per 100 SCAN commands (not usually needed).
...
Related
I have a large number of simple keys to delete from Redis with a prefix and I am trying to find the most efficient way to do this atomically inside a transaction with Lua script:
Iterate with SCAN and DEL keys?
Iterate with SCAN and EXPIRE each key?
Iterate with SCAN and UNLINK keys?
Which of the above is the recommended way to proceed? Shall I have a different approach - like using a hash and multiple keys inside hash? Would any of the above be any problem in case of Redis cluster?
I would suggest to go with unlink with batch processing like following and it will clear the memory in efficient way. I don't suggest expiry as redis will look 10times(this default configuration) within 1second to delete expired keys and it may not be efficient way.
redis-cli --scan --pattern 'prefix:*' | xargs -L 1000 redis-cli unlink
if your keys are simple strings, you can organise your prefixes inside hashes, with keys as name/value pairs.
Then, just drop the entire hash.
PS: Yes you have to rewrite all your read queries, but the performance impact should be negligible?
PS2: Doesn't address the clustered Redis problems.
A Redis Lua script must be provided explicitly with all key names it touches via the KEYS array. All three approaches violate that principle as they fetch the key names with SCAN.
I'm new to redis, have doubt on below commands
In redis-cli i have entered info keyspace, for this i got the following output
# Keyspace
db0:keys=2,expires=0,avg_ttl=0
What does the above lines represent. N how to get the value from key.
You might want to look at https://redis.io/commands/INFO
To be specific the keyspace is, as mention in the official doc:
Database related statistics
The keyspace section provides statistics on the main dictionary of each database. The statistics are the number of keys, and the number of keys with an expiration.
For each database, the following line is added:
dbXXX: keys=XXX,expires=XXX
So in your case you have 2 keys present in redis
I'm have hashes in redis cache like:
Hash Key Value
hashme:1 Hello World
hashme:2 Here Iam
myhash:1 Next One
My goal is to get the Hashes as output in the CLI like:
hashme
myhash
If there's no such option, this is ok too:
hashme:1
hashme:2
myhash:1
I didn't find any relevant command for it in Redis API.
Any suggestions ?
You can use the SCAN command to get all keys from Redis. Then for each key, use the TYPE command to check if it's a hash.
UPDATE:
With Redis 6.0, the SCAN command supports TYPE subcommand, and you can use this subcommand to scan all keys of a specified type:
SCAN 0 TYPE hash
Also never use KEYS command in production environment!!! It's a dangerous command which might block Redis for a long time.
keys *
is work for me. you Can try it.
The idea of redis (and others K/v stores) is for you to build an index. The database won't do it for you. It's a major difference with relational databases, which conributes to better performances.
So when your app creates a hash, put its key into a SET. When your app deletes a hash, remove its key from the SET. And then to get the list of hash IDs, just use SMEMBERS to get the content of the SET.
connection.keys('*') this will bring all the keys irrespective of the data type as everything in redis is stored as key value format
for redis in python, you can use below command to retrieve keys from redis db
def standardize_list(bytelist):
return [x.decode('utf-8') for x in bytelist]
>>> standardize_list(r.keys())
['hat:1236154736', 'hat:56854717', 'hat:1326692461']
here r variable is redis connection object
I have a Redis Cluster consisting of multiple nodes. I want to update 3 different keys in a single atomic operation. My Lua script is like:
local u1 = redis.call('incrby', KEYS[1], ARGV[1])
local u2 = redis.call('incrby', KEYS[2], ARGV[1])
local u3 = redis.call('incrby', KEYS[3], ARGV[1])
And I fired it with:
EVAL script 3 key1 key2 key3 arg
But I got the error message:
WARN Resp(AppErr CROSSSLOT Keys in request don't hash to the same slot)
The above operations cannot be done, and the updates will fail. It seems I cannot modify the keys in different nodes with a single Lua script. But according to the doc:
All Redis commands must be analyzed before execution to determine
which keys the command will operate on. In order for this to be true
for EVAL, keys must be passed explicitly. This is useful in many ways,
but especially to make sure Redis Cluster can forward your request to
the appropriate cluster node.
Note this rule is not enforced in order
to provide the user with opportunities to abuse the Redis single
instance configuration, at the cost of writing scripts not compatible
with Redis Cluster.
So I think as long as I follow the key passing rule, the script should be compatible with Redis Cluster. I wonder what's the problem here and what should I do to update all keys in a single script.
I'm afraid you've misunderstood the documentation. (And I agree that it's not very clear.)
Redis operations, whether commands or Lua scripts, can only work when all the keys are on the same server. The purpose of the key passing rule is to allow Cluster servers to figure out where to send the script and to fail fast if all the keys don't come from the same server (which is what happened in your case).
So it's your responsibility to make sure that all the keys you want to operate on are located on the same server. The way to do that is to use hash tags to force keys to hash to the same slot. See the documentation for more details on that.
I can get some keys with this command:
keys *_products_*
However that command returns all the keys, where I just need the count of those. How can I get It?
DBSIZE
Return the number of keys in the database
http://redis.io/commands/dbsize
You can use DBSIZE or INFO KEYSPACE
But if you want all the keys with a certain pattern in the name you need to use KEYS or SCAN And you need to pay attention to KEYS, running it in production can affect the performance so it should be used with caution.
Both (evil) KEYS and the much preferred SCAN do not return counts, only key names. You could wrap them in a Lua script to just return the count.
However.
Doing KEYS (or SCAN) in real time is very expensive in terms of performance - it means that you're iterating over the entire keyspace. What you want to do is keep the result ready beforehand, and then just fetch it. Usually in such cases, you'd use a Redis Set in which you'd store each relevant key name - SADD to it whenever you create a key that matches the patter *products* and delete from it whenever a key is removed.
But.
Since you're only interesting in a count, you can replace that Set with a simple counter.
Use this command line:
redis-cli --scan --pattern '*_products_*' | wc -l