I have a LUA script that fetches a SET data-type and iterates through it's members. This set contains a list of other SET data-types (which ill call the secondary sets for clarity).
For context, the secondary SETs are lists of cache keys, majority of cache keys within these secondary sets reference cache items that no longer exist, these lists grow exponentially in size so my goal is to check each of these cache keys; within each secondary set to see if they have expired and if so, remove them from the secondary set, thus reducing the size and conserving memory.
local pending = redis.call('SMEMBERS', KEYS[1])
for i, key in ipairs(pending) do
redis.call('SREM', KEYS[1], key)
local keys = redis.call('SMEMBERS', key)
local expired = {}
for i, taggedKey in ipairs(keys) do
local ttl = redis.call('ttl', taggedKey)
if ttl == -2 then
table.insert(expired, taggedKey)
end
end
if #expired > 0 then
redis.call('SREM', key, unpack(expired))
end
end
The script above works perfectly, until one of the secondary set keys contain a different hash-slot, the error I receive is:
Lua script attempted to access keys of different hash slots
Looking through the docs, I noticed that Redis allows this to be bypassed with an EVAL flag allow-cross-slot-keys, so following that example I updated my script to the following
#!lua flags=allow-cross-slot-keys
local pending = redis.call('SMEMBERS', KEYS[1])
for i, key in ipairs(pending) do
redis.call('SREM', KEYS[1], key)
local keys = redis.call('SMEMBERS', key)
local expired = {}
for i, taggedKey in ipairs(keys) do
local ttl = redis.call('ttl', taggedKey)
if ttl == -2 then
table.insert(expired, taggedKey)
end
end
if #expired > 0 then
redis.call('SREM', key, unpack(expired))
end
end
I'm now strangely left with the error:
"ERR Error compiling script (new function): user_script:1: unexpected symbol near '#'"
Any help appreciated, have reached out to the pocket burner that is Redis Enterprise but still awaiting a response.
For awareness, this is to clean up the shoddy Laravel Redis implementation where they create these sets to manage tagged cache but never clean them up, over time they amount to gigabytes in wasted space, and if your eviction policy requires allkeys-lfu then all your cache will be pushed out in favour of this messy garbage Laravel leaves behind, leaving you with a worthless caching system or thousands of dollars out of pocket to increase RAM.
Edit: It would seem we're on Redis 6.2 and those flags are Redis 7+ unlikely there is a solution suitable for 6.2 but if there is please let me know.
Since the Redis expires keys in passive and active way,
Is there any way to get a key even if its expire time has passed(but still exists in Redis)?
DEBUG OBJECT myKey will return info about the key even when it is expired timewise - if it has not been garbage-collected (actively-expired) or passively-expired (accessed) yet.
To test this, you can disable the background (active) expiring with DEBUG SET-ACTIVE-EXPIRE 0. Use carefully. Restore with DEBUG SET-ACTIVE-EXPIRE 1
Note the DEBUG OBJECT myKey returns the memory address, so if you really need to see the value, you'll have to explore the RAM contents.
> DEBUG OBJECT myKey
Value at:0x7fff70629920 refcount:1 encoding:embstr serializedlength:4 ...
DEBUG OBJECT myKey returns (error) ERR no such key if it has been expired already, actively or passively, or if it doesn't exist of course.
No, there isn't. The key (and its value) will be expired automatically eventually, or upon trying to access it (i.e. passively or actively).
I've been playing around with redis to keep track of the ratelimit of an external api in a distributed system. I've decided to create a key for each route where a limit is present. The value of the key is how many request I can still make until the limit resets. And the reset is made by setting the TTL of the key to when the limit will reset.
For that I wrote the following lua script:
if redis.call("EXISTS", KEYS[1]) == 1 then
local remaining = redis.call("DECR", KEYS[1])
if remaining < 0 then
local pttl = redis.call("PTTL", KEYS[1])
if pttl > 0 then
--[[
-- We would exceed the limit if we were to do a call now, so let's send back that a limit exists (1)
-- Also let's send back how much we would have exceeded the ratelimit if we were to ignore it (ramaning)
-- and how long we need to wait in ms untill we can try again (pttl)
]]
return {1, remaining, pttl}
elseif pttl == -1 then
-- The key expired the instant after we checked that it existed, so delete it and say there is no ratelimit
redis.call("DEL", KEYS[1])
return {0}
elseif pttl == -2 then
-- The key expired the instant after we decreased it by one. So let's just send back that there is no limit
return {0}
end
else
-- Great we have a ratelimit, but we did not exceed it yet.
return {1, remaining}
end
else
return {0}
end
Since a watched key can expire in the middle of a multi transaction without aborting it. I assume the same is the case for lua scripts. Therefore I put in the cases for when the ttl is -1 or -2.
After I wrote that script I looked a bit more in depth at the eval command page and found out that a lua script has to be a pure function.
In there it says
The script must always evaluates the same Redis write commands with
the same arguments given the same input data set. Operations performed
by the script cannot depend on any hidden (non-explicit) information
or state that may change as script execution proceeds or between
different executions of the script, nor can it depend on any external
input from I/O devices.
With this description I'm not sure if my function is a pure function or not.
After Itamar's answer I wanted to confirm that for myself so I wrote a little lua script to test that. The scripts creates a key with a 10ms TTL and checks the ttl untill it's less then 0:
redis.call("SET", KEYS[1], "someVal","PX", 10)
local tmp = redis.call("PTTL", KEYS[1])
while tmp >= 0
do
tmp = redis.call("PTTL", KEYS[1])
redis.log(redis.LOG_WARNING, "PTTL:" .. tmp)
end
return 0
When I ran this script it never terminated. It just went on to spam my logs until I killed the redis server. However time dosen't stand still while the script runs, instead it just stops once the TTL is 0.
So the key ages, it just never expires.
Since a watched key can expire in the middle of a multi transaction without aborting it. I assume the same is the case for lua scripts. Therefore I put in the cases for when the ttl is -1 or -2.
AFAIR that isn't the case w/ Lua scripts - time kinda stops (in terms of TTL at least) when the script's running.
With this description I'm not sure if my function is a pure function or not.
Your script's great (without actually trying to understand what it does), don't worry :)
My redis server does not delete keys when the time-to-live reaches 0.
Here is a sample code:
redis-cli
>SET mykey "ismykey"
>EXPIRE mykey 20
#check TTL
>TTL mykey
>(integer) 17
> ...
>TTL mykey
>(integer) -1
#mykey chould have expired:
>EXISTS mykey
>(integer) 1
>#oh still there, check its value
>GET mykey
>"ismykey"
If i check the info return by redis, it says 0 keys were expired.
Any idea?
thanks.
Since you're doing a '...' it's hard to say for sure, but I'd say you're setting mykey during that part, which will effectively remove the expiration.
From the EXPIRE manual
The timeout is cleared only when the key is removed using the DEL
command or overwritten using the SET or GETSET commands
Also, regarding the -1 reply from TTL
Return value
Integer reply: TTL in seconds or -1 when key does not
exist or does not have a timeout.
EDIT: Note that this behaviour changed in Redis 2.8
Starting with Redis 2.8 the return value in case of error changed:
The command returns -2 if the key does not exist.
The command returns -1 if the key exists but has no associated expire.
In other words, if your key exists, it would seem to be persistent, ie not have any expiration set.
EDIT: It seems I can reproduce this if I create the key on a REDIS slave server, the slave will not delete the key without master input, since normally you would not create keys locally on a slave. Is this the case here?
However while the slaves connected to a master will not expire keys
independently (but will wait for the DEL coming from the master),
they'll still take the full state of the expires existing in the
dataset, so when a slave is elected to a master it will be able to
expire the keys independently, fully acting as a master.
I want to delete all keys. I want everything wiped out and give me a blank database.
Is there a way to do this in Redis client?
With redis-cli:
FLUSHDB – Deletes all keys from the connection's current database.
FLUSHALL – Deletes all keys from all databases.
For example, in your shell:
redis-cli flushall
Heads up that FLUSHALL may be overkill. FLUSHDB is the one to flush a database only. FLUSHALL will wipe out the entire server. As in every database on the server. Since the question was about flushing a database I think this is an important enough distinction to merit a separate answer.
Answers so far are absolutely correct; they delete all keys.
However, if you also want to delete all Lua scripts from the Redis instance, you should follow it by:
SCRIPT FLUSH
The OP asks two questions; this completes the second question (everything wiped).
FLUSHALL
Remove all keys from all databases
FLUSHDB
Remove all keys from the current database
SCRIPT FLUSH
Remove all the scripts from the script cache.
If you're using the redis-rb gem then you can simply call:
your_redis_client.flushdb
you can use flushall in your terminal
redis-cli> flushall
This method worked for me - delete everything of current connected Database on your Jedis cluster.
public static void resetRedis() {
jedisCluster = RedisManager.getJedis(); // your JedisCluster instance
for (JedisPool pool : jedisCluster.getClusterNodes().values()) {
try (Jedis jedis = pool.getResource()) {
jedis.flushAll();
}
catch (Exception ex){
System.out.println(ex.getMessage());
}
}
}
One more option from my side:
In our production and pre-production databases there are thousands of keys. Time to time we need to delete some keys (by some mask), modify by some criteria etc. Of course, there is no way to do it manually from CLI, especially having sharding (512 logical dbs in each physical).
For this purpose I write java client tool that does all this work. In case of keys deletion the utility can be very simple, only one class there:
public class DataCleaner {
public static void main(String args[]) {
String keyPattern = args[0];
String host = args[1];
int port = Integer.valueOf(args[2]);
int dbIndex = Integer.valueOf(args[3]);
Jedis jedis = new Jedis(host, port);
int deletedKeysNumber = 0;
if(dbIndex >= 0){
deletedKeysNumber += deleteDataFromDB(jedis, keyPattern, dbIndex);
} else {
int dbSize = Integer.valueOf(jedis.configGet("databases").get(1));
for(int i = 0; i < dbSize; i++){
deletedKeysNumber += deleteDataFromDB(jedis, keyPattern, i);
}
}
if(deletedKeysNumber == 0) {
System.out.println("There is no keys with key pattern: " + keyPattern + " was found in database with host: " + host);
}
}
private static int deleteDataFromDB(Jedis jedis, String keyPattern, int dbIndex) {
jedis.select(dbIndex);
Set<String> keys = jedis.keys(keyPattern);
for(String key : keys){
jedis.del(key);
System.out.println("The key: " + key + " has been deleted from database index: " + dbIndex);
}
return keys.size();
}
}
Writing such kind of tools I find very easy and spend no more then 5-10 min.
Use FLUSHALL ASYNC if using (Redis 4.0.0 or greater) else FLUSHALL.
https://redis.io/commands/flushall
Note: Everything before executing FLUSHALL ASYNC will be evicted. The changes made during executing FLUSHALL ASYNC will remain unaffected.
FLUSHALL Deletes all the Keys of All exisiting databases .
FOr Redis version > 4.0 , FLUSHALL ASYNC is supported which runs in a background thread wjthout blocking the server
https://redis.io/commands/flushall
FLUSHDB - Deletes all the keys in the selected Database .
https://redis.io/commands/flushdb
The time complexity to perform the operations will be O(N) where N being the number of keys in the database.
The Response from the redis will be a simple string "OK"
Open redis-cli and type:
FLUSHALL
You can use FLUSHALL which will delete all keys from your every database.
Where as FLUSHDB will delete all keys from our current database.
Stop Redis instance.
Delete RDB file.
Start Redis instance.
redis-cli -h <host> -p <port> flushall
It will remove all data from client connected(with host and port)
After you start the Redis-server using:service redis-server start --port 8000 or redis-server.
Use redis-cli -p 8000 to connect to the server as a client in a different terminal.
You can use either
FLUSHDB - Delete all the keys of the currently selected DB. This command never fails. The time-complexity for this operation is O(N), N being the number of keys in the database.
FLUSHALL - Delete all the keys of all the existing databases, not just the currently selected one. This command never fails. The time-complexity for this operation is O(N), N being the number of keys in all existing databases.
Check the documentation for ASYNC option for both.
If you are using Redis through its python interface, use these two functions for the same functionality:
def flushall(self):
"Delete all keys in all databases on the current host"
return self.execute_command('FLUSHALL')
and
def flushdb(self):
"Delete all keys in the current database"
return self.execute_command('FLUSHDB')
i think sometimes stop the redis-server and delete rdb，aof files。
make sure there’s no data can be reloading.
then start the redis-server,now it's new and empty.
You can use FLUSHDB
e.g
List databases:
127.0.0.1:6379> info keyspace
# Keyspace
List keys
127.0.0.1:6379> keys *
(empty list or set)
Add one value to a key
127.0.0.1:6379> lpush key1 1
(integer) 1
127.0.0.1:6379> keys *
1) "key1"
127.0.0.1:6379> info keyspace
# Keyspace
db0:keys=1,expires=0,avg_ttl=0
Create other key with two values
127.0.0.1:6379> lpush key2 1
(integer) 1
127.0.0.1:6379> lpush key2 2
(integer) 2
127.0.0.1:6379> keys *
1) "key1"
2) "key2"
127.0.0.1:6379> info keyspace
# Keyspace
db0:keys=2,expires=0,avg_ttl=0
List all values in key2
127.0.0.1:6379> lrange key2 0 -1
1) "2"
2) "1"
Do FLUSHDB
127.0.0.1:6379> flushdb
OK
List keys and databases
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> info keyspace
# Keyspace
Your questions seems to be about deleting entire keys in a database. In this case you should try:
Connect to redis. You can use the command redis-cli (if running on port 6379), else you will have to specify the port number also.
Select your database (command select {Index})
Execute the command flushdb
If you want to flush keys in all databases, then you should try flushall.
you can use following approach in python
def redis_clear_cache(self):
try:
redis_keys = self.redis_client.keys('*')
except Exception as e:
# print('redis_client.keys() raised exception => ' + str(e))
return 1
try:
if len(redis_keys) != 0:
self.redis_client.delete(*redis_keys)
except Exception as e:
# print('redis_client.delete() raised exception => ' + str(e))
return 1
# print("cleared cache")
return 0
This works for me: redis-cli KEYS \* | xargs --max-procs=16 -L 100 redis-cli DEL
It list all Keys in redis, then pass using xargs to redis-cli DEL, using max 100 Keys per command, but running 16 command at time, very fast and useful when there is not FLUSHDB or FLUSHALL due to security reasons, for example when using Redis from Bitnami in Docker or Kubernetes. Also, it doesn't require any additional programming language and it just one line.
If you want to clear redis in windows:
find redis-cli in
C:\Program Files\Redis
and run FLUSHALL command.
Its better if you can have RDM (Redis Desktop Manager).
You can connect to your redis server by creating a new connection in RDM.
Once its connected you can check the live data, also you can play around with any redis command.
Opening a cli in RDM.
1) Right click on the connection you will see a console option, just click on it a new console window will open at the bottom of RDM.
Coming back to your question FLUSHALL is the command, you can simply type FLUSHALL in the redis cli.
Moreover if you want to know about any redis command and its proper usage, go to link below.
https://redis.io/commands.
There are different approaches. If you want to do this from remote, issue flushall to that instance, through command line tool redis-cli or whatever tools i.e. telnet, a programming language SDK. Or just log in that server, kill the process, delete its dump.rdb file and appendonly.aof(backup them before deletion).
If you are using Java then from the documentation, you can use any one of them based on your use case.
/**
* Remove all keys from all databases.
*
* #return String simple-string-reply
*/
String flushall();
/**
* Remove all keys asynchronously from all databases.
*
* #return String simple-string-reply
*/
String flushallAsync();
/**
* Remove all keys from the current database.
*
* #return String simple-string-reply
*/
String flushdb();
/**
* Remove all keys asynchronously from the current database.
*
* #return String simple-string-reply
*/
String flushdbAsync();
Code:
RedisAdvancedClusterCommands syncCommands = // get sync() or async() commands
syncCommands.flushdb();
Read more: https://github.com/lettuce-io/lettuce-core/wiki/Redis-Cluster
For anyone wondering how to do this in C# it's the same as the answer provided for Python for this same question.
I am using StackExchange.Redis v2.2.88 for a dot net (core) 5 project. I only need to clear my keys for integration testing and I have no purpose to do this in production.
I checked what is available in intellisense and I don't see a stock way to do this with the existing API. I imagine this is intentional and by design. Luckily the API does expose an Execute method.
I tested this by doing the following:
Opened up a command window. I am using docker, so I did it through docker.
Type in redis-cli which starts the CLI
Type in KEYS * and it shows me all of my keys so I can verify they exist before and after executing the following code below:
//Don't abuse this, use with caution
var cache = ConnectionMultiplexer.Connect(
new ConfigurationOptions
{
EndPoints = { "localhost:6379" }
});
var db = _cache.GetDatabase();
db.Execute("flushdb");
Type in KEYS * again and view that it's empty.
Hope this helps anyone looking for it.