Redis command to get all available keys? - redis

Is there a Redis command for fetching all keys in the database? I have seen some python-redis libraries fetching them. But was wondering if it is possible from redis-client.

Try to look at KEYS command. KEYS * will list all keys stored in redis.
EDIT: please note the warning at the top of KEYS documentation page:
Time complexity: O(N) with N being the number of keys in the database, under the assumption that the key names in the database and the given pattern have limited length.
UPDATE (V2.8 or greater): SCAN is a superior alternative to KEYS, in the sense that it does not block the server nor does it consume significant resources. Prefer using it.

Updated for Redis 2.8 and above
As noted in the comments of previous answers to this question, KEYS is a potentially dangerous command since your Redis server will be unavailable to do other operations while it serves it. Another risk with KEYS is that it can consume (dependent on the size of your keyspace) a lot of RAM to prepare the response buffer, thus possibly exhausting your server's memory.
Version 2.8 of Redis had introduced the SCAN family of commands that are much more polite and can be used for the same purpose.
The CLI also provides a nice way to work with it:
$ redis-cli --scan --pattern '*'

It can happen that using redis-cli, you connect to your remote redis-server, and then the command:
KEYS *
is not showing anything, or better, it shows:
(empty list or set)
If you are absolutely sure that the Redis server you use is the one you have the data, then maybe your redis-cli is not connecting to the Redis correct database instance.
As it is mentioned in the Redis docs, new connections connect as default to the db 0.
In my case KEYS command was not retrieving results because my database was 1. In order to select the db you want, use SELECT.
The db is identified by an integer.
SELECT 1
KEYS *
I post this info because none of the previous answers was solving my issue.

-->Get all keys from redis-cli
-redis 127.0.0.1:6379> keys *
-->Get list of patterns
-redis 127.0.0.1:6379> keys d??
This will produce keys which start by 'd' with three characters.
-redis 127.0.0.1:6379> keys *t*
This wil get keys with matches 't' character in key
-->Count keys from command line by
-redis-cli keys * |wc -l
-->Or you can use dbsize
-redis-cli dbsize

Get All Keys In Redis
Get all keys using the --scan option:
$ redis-cli --scan --pattern '*'
List all keys using the KEYS command:
$ redis-cli KEYS '*'

Take a look at following Redis Cheat Sheet.
To get a subset of redis keys with the redis-cli i use the command
KEYS "prefix:*"

Yes, you can get all keys by using this
var redis = require('redis');
redisClient = redis.createClient(redis.port, redis.host);
redisClient.keys('*example*', function (err, keys) {
})

SCAN doesn't require the client to load all the keys into memory like KEYS does. SCAN gives you an iterator you can use. I had a 1B records in my redis and I could never get enough memory to return all the keys at once.
Here is a python snippet to get all keys from the store matching a pattern and delete them:
import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
for key in r.scan_iter("key_pattern*"):
print key

redis-cli -h <host> -p <port> keys *
where * is the pattern to list all keys

KEYS pattern
Available since 1.0.0.
Time complexity: O(N) with N being the number
of keys in the database, under the assumption that the key names in
the database and the given pattern have limited length.
Returns all keys matching pattern.
Warning : This command is not recommended to use because it may ruin performance when it is executed against large databases instead of KEYS you can use SCAN or SETS.
Example of KEYS command to use :
redis> MSET firstname Jack lastname Stuntman age 35
"OK"
redis> KEYS *name*
1) "lastname"
2) "firstname"
redis> KEYS a??
1) "age"
redis> KEYS *
1) "lastname"
2) "age"
3) "firstname"

In order to get all the keys available in redis server, you should open redis-cli and type:
KEYS *
In order to get more help please visit this page:
This Link

If your redis is a cluster,you can use this script
#!/usr/bin/env bash
redis_list=("172.23.3.19:7001,172.23.3.19:7002,172.23.3.19:7003,172.23.3.19:7004,172.23.3.19:7005,172.23.3.19:7006")
arr=($(echo "$redis_list" | tr ',' '\n'))
for info in ${arr[#]}; do
echo "start :${info}"
redis_info=($(echo "$info" | tr ':' '\n'))
ip=${redis_info[0]}
port=${redis_info[1]}
echo "ip="${ip}",port="${port}
redis-cli -c -h $ip -p $port set laker$port '湖人总冠军'
redis-cli -c -h $ip -p $port keys \*
done
echo "end"

For the ones that wants a typescript helper (using ioredis)
import Redis from 'ioredis';
import { from, Observable, of } from 'rxjs';
import { first, mergeMap } from 'rxjs/operators';
export function scanKeysFromRedis(redisStore: Redis.Redis, key: string,
target: number = 0, keys: string[] = []): Observable<string[]> {
return from(redisStore.scan(target, 'MATCH', key)).pipe(
first(),
mergeMap((_keys) => {
const _target = Number(_keys[0]);
if (_target !== 0) {
return scanKeysFromRedis(redisStore, key, _target, [...keys, ..._keys[1]]);
}
return of([...keys, ..._keys[1]]);
}),
);
}
and call it with: scanKeysFromRedis(store, 'hello');

If you are using Laravel Framework then you can simply use this:
$allKeyList = Redis::KEYS("*");
print_r($allKeyList);
In Core PHP:
$redis = new Redis();
$redis->connect('hostname', 6379);
$allKeyList = $redis->keys('*');
print_r($allKeyList);

You can simply connect to your redis server using redis-cli, select your database and type KEYS *, please remember it will give you all the keys present in selected redis database.

We should be using --scan --pattern with redis 2.8 and later.
You can try using this wrapper on top of redis-cli.
https://github.com/VijayantSoni/redis-helper

Related

Is there a way to remove the ttl from all keys in redis?

I'm wanting to a redis instance to be used for development purposes without any expirations in it. Is there way to remove the ttl from all the keys in the database?
The following command via redis-cli will loop through all the keys in the selected database and call the persist command which removes the ttl:
EVAL "for i, name in ipairs(redis.call('KEYS', '*')) do redis.call('persist', name); end" 0

Redis delete all keys except keys that start with

My redis collection contains many keys
I want to be able to flush them all except all the keys that start with:
"configurations::"
is this possible?
You can do this
redis-cli KEYS "*" | grep -v "configurations::" | xargs redis-cli DEL
List all keys into the redis, remove from the list keys that contains "configurations::" and delete them from the redis
Edit
As #Sergio Tulentsev notice it keys is not for use in production. I used this python script to remove keys on prodution redis. I stoped replication from master to slave before call the script.
#!/usr/bin/env python
import redis
import time
pattern = "yourpattern*"
poolSlave = redis.ConnectionPool(host='yourslavehost', port=6379, db=0)
redisSlave = redis.Redis(connection_pool=poolSlave)
poolMaster = redis.ConnectionPool(host='yourmasterhost', port=6379, db=0)
redisMaster = redis.Redis(connection_pool=poolMaster)
cursor = '0'
while cursor != 0:
cursor, data = redisSlave.scan(cursor, pattern, 1000)
print "cursor: "+str(cursor)
for key in data:
redisMaster.delete(key)
print "delete key: "+key
# reduce call per second on production server
time.sleep(1)
The SCAN & DEL approach (as proposed by #khanou) is the best ad-hoc solution. Alternatively, you could keep an index of all your configurations:: key names with a Redis Set (simply SADD the key's name to it whenever you create a new configurations:: key). Once you have this set you can SSCAN it to get all the relevant key names more efficiently (don't forget to SREM from it whenever you DEL though).
Yes, it's possible. Enumerate all the keys, evaluate each one and delete if it fits the criteria for deletion.
There is no built-in redis command for this, if this is what you were asking.
It might be possible to cook up a Lua script that will do this (and it'll look to your app that it's a single command), but still it's the same approach under the hood.

get/sum values from wildcard keys in redis

I have a string type key value store in redis having keys like this--
/url-pattern/url-slug-1
/url-pattern/url-slug-2
/url-pattern/url-slug-3
/url-pattern/url-slug-4 ...
I can retrieve all the keys of /url-pattern/ using a wild card query like this --
keys /url-pattern/*
I would like to retrieve the values of all keys corresponding to this wildcard /url-pattern/*
I tried this
mget /url-pattern/*
1) (nil)
but it doesnt returned the array as expected.
How can I retrieve the values of all keys corresponding to /url-pattern/*
I also want to do a sum on the values, but I think there is no such thing called SUM() in redis
MGET accepts multiple arguments where each a key name. It does not do key name patterns.
What you could do is first fetch all the relevant key names (do not use KEYS, use SCAN instead) and then fetch their values with an MGET.
Here is an updated answer for 2015.
If you can upgrade Redis above 2.8, the SCAN command with MATCH will work for this. Before that version, not so much, and do NOT use the KEYS command except in a development environment.
http://redis.io/commands/scan
Example on command line:
$ redis-cli
127.0.0.1:6379> scan match V3.0:*
(error) ERR invalid cursor
127.0.0.1:6379> scan 0 match V3.0:*
1) "0"
2) 1) "V3.0:UNITTEST55660BC7E0C5B"
2) "V3.0:shop.domain.com:route"
3) "V3.0:UNITTEST55660BC4A2548"
127.0.0.1:6379> scan 0 match V1.0:*
1) "0"
2) (empty list or set)
127.0.0.1:6379> scan 0 match V3.0:*
1) "0"
2) 1) "V3.0:UNITTEST55660BC7E0C5B"
2) "V3.0:shop.domain.com:route"
3) "V3.0:UNITTEST55660BC4A2548"
Example in PHP:
// Initialize our iterator to NULL
$iterate = null;
// retry when we get no keys back
$redis->setOption(Redis::OPT_SCAN, Redis::SCAN_RETRY);
while ($arr_keys = $redis->scan($iterate, 'match:*')) {
foreach ($arr_keys as $str_key) {
echo "Here is a key: $str_key\n";
}
echo "No more keys to scan!\n";
}
Note, php code is not tested and from the core documentation for example here. Production code would need to be modified depending on the keys needed to look up.
For those on Ubuntu here are the instructions to upgrade php5-redis:
Download the 2.2.7 package here: http://pecl.php.net/package/redis
$ php -i | grep Redis
Redis Support => enabled
Redis Version => 2.2.4
Follow instructions in README to phpize, configure, make install
Create a symlink for command line cli package: cd /etc/php5/cli/conf.d && sudo ln -s ../../mods-available/redis.ini 20-redis.ini
$ php -i | grep Redis
Redis Support => enabled
Redis Version => 2.2.7
There is NO command available in REDIS which can return values from wildcard keys.
If you see the documentation for KEYS command: http://redis.io/commands/keys, it says
Consider KEYS as a command that should only be used in production
environments with extreme care. It may ruin performance when it is
executed against large databases. This command is intended for
debugging and special operations. Don't use KEYS in your regular
application code.
I don't know your business use case, but looks like you may have to use different data structure for this requirement. You can use list or set to store similar url patterns.

Finding non-expiring keys in Redis

In my setup, the info command shows me the following:
[keys] => 1128
[expires] => 1125
I'd like to find those 3 keys without an expiration date. I've already checked the docs to no avail. Any ideas?
Modified from a site that I can't find now.
redis-cli keys "*" | while read LINE ; do TTL=`redis-cli ttl "$LINE"`; if [ $TTL -eq -1 ]; then echo "$LINE"; fi; done;
edit: Note, this is a blocking call.
#Waynn Lue's answer runs but uses the Redis KEYS command which Redis warns about:
Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases.
Redis documentation recommends using SCAN.
redis-cli --scan | while read LINE ; do TTL=`redis-cli ttl "$LINE"`; if [ $TTL -eq -1 ]; then echo "$LINE"; fi; done;
If you want to scan for a specific key pattern, use:
redis-cli --scan --pattern "something*"
In case somebody is getting bad arguments or wrong number of arguments error,
put double quotes around $LINE.
So,it would be
redis-cli keys "*" | while read LINE ; do TTL=`redis-cli ttl "$LINE"`; if [ $TTL -eq -1 ]; then echo "$LINE"; fi; done;
This happens when there are spaces in the key.
To me the accepted answer appears unusable for a medium-sized dataset, as it will run a redis-cli command for each and every key.
Instead I used this lua script to filter the keys inside the redis server:
local show_persistent = ARGV[1] ~= "expiring"
local keys = {}
for i, name in ipairs(redis.call("keys", "*")) do
local persistent = redis.call("pttl", name) < 0
if persistent == show_persistent then
table.insert(keys, name)
end
end
return keys
This can be called as
$ redis-cli --eval show-persistent-keys.lua
to get all keys without an expiration time. It also can be called as
$ redis-cli --eval show-persistent-keys.lua , expiring
to find the opposite key set of all keys with an expiration time set.
On the downside this may block for too long (appears fine for 1 M keys). I'd use scan instead but I happen to have to run this against a legacy Redis at version 2.6, which does not have scan available.
I needed to extract non-expiring keys from bigger (40GB) dataset, so using keys command was not suitable for me. So in case someone is looking for offline/non-blocking solution, you can use https://github.com/sripathikrishnan/redis-rdb-tools for extraction of non-expiring keys from redis rdb dump.
You can just install libraries via pip:
pip install rdbtools python-lzf
Then create simple parser which extracts keys and values which has expiration set to None:
from rdbtools import RdbParser, RdbCallback
from rdbtools.encodehelpers import bytes_to_unicode
class ParserCallback(RdbCallback):
def __init__(self):
super(ParserCallback, self).__init__(string_escape=None)
def encode_key(self, key):
return bytes_to_unicode(key, self._escape, skip_printable=True)
def encode_value(self, val):
return bytes_to_unicode(val, self._escape)
def set(self, key, value, expiry, info):
if expiry is None:
print('%s = %s' % (self.encode_key(key), self.encode_value(value)))
callback = ParserCallback()
parser = RdbParser(callback)
parser.parse('/path/to/dump.rdb')
#!/usr/bin/env python
import argparse
import redis
p = argparse.ArgumentParser()
p.add_argument("-i", '--host', type=str, default="127.0.0.1", help="redis host", required=False)
p.add_argument("-p", '--port', type=int, default=6379, help="redis port", required=False)
p.add_argument("-n", '--db', type=int, default=0, help="redis database", required=False)
args = p.parse_args()
r = redis.Redis(host=args.host, port=args.port, db=args.db)
try:
keys = r.keys()
for key in keys:
if r.ttl(key) < 0:
print(key)
except KeyboardInterrupt:
pass

Print number of keys in Redis

Is there a way to print the number of keys in Redis?
I am aware of
keys *
But that seems slightly heavy weight. - Given that Redis is a key value store maybe this is the only way to do it. But I would still like to see something along the lines of
count keys *
You can issue the INFO command, which returns information and statistics about the server. See here for an example output.
As mentioned in the comments by mVChr, you can use info keyspace directly on the redis-cli.
redis> INFO
# Server
redis_version:6.0.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:b63575307aaffe0a
redis_mode:standalone
os:Linux 5.4.0-1017-aws x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:9.3.0
process_id:2854672
run_id:90a5246f10e0aeb6b02cc2765b485d841ffc924e
tcp_port:6379
uptime_in_seconds:2593097
uptime_in_days:30
hz:10
configured_hz:10
lru_clock:4030200
executable:/usr/local/bin/redis-server
DBSIZE returns the number of keys and it's easier to parse.
Downside: if a key has expired it may still count.
http://redis.io/commands/dbsize
The DBSIZE command returns the number of keys
> DBSIZE
WARNING: Do not run this on a production machine.
On a Linux box:
redis-cli KEYS "*" | wc -l
Note: As mentioned in comments below, this is an O(N) operation, so on a large DB with many keys you should not use this. For smaller deployments, it should be fine.
Since Redis 2.6, lua is supported, you can get number of wildcard keys like this
eval "return #redis.call('keys', 'prefix-*')" 0
see eval command
Go to redis-cli and use below command
info keyspace
It may help someone
Displays the DB's name along with keys count:
redis-cli info keyspace
# Keyspace
db0:keys=12995,expires=0,avg_ttl=0
db12:keys=5524396,expires=5,avg_ttl=45201
Display the no. of keys in the selected database:
redis-cli dbsize
(integer) 12995
Displays whole Redis's stats:
redis-cli info
dbsize() returns the total number of keys.
You can quickly estimate the number of keys matching a given pattern by sampling keys at random, then checking what fraction of them matches the pattern.
Example in python; counting all keys starting with prefix_:
import redis
r = redis.StrictRedis(host = 'localhost', port=6379)
iter=1000
print 'Approximately', r.dbsize() * float(sum([r.randomkey().startswith('prefix_') for i in xrange(iter)])) / iter
Even iter=100 gives a decent estimate in my case, yet is very fast, compared to keys prefix_.
An improvement is to sample 1000 keys on every request, but keep the total count, so that after two requests you'll divide by 2000, after three requests you'll divide by 3000. Thus, if your application is interested in the total number of matching keys fairly often, then every time it will get closer and closer to the true value.
After Redis 2.6, the result of INFO command are splitted by sections. In the "keyspace" section, there are "keys" and "expired keys" fields to tell how many keys are there.
eval "local count = redis.call('scan', 0, 'match', 'key:*:key', 'count', 10000) if count ~= 0 then return #count[2] end " 0
eval "local count = redis.call('sscan', 'key.key:all', 0, 'match', '*', 'count', 1000000) if count ~= 0 then return #count[2] end " 0