Can I duplicate a key using the redis-cli connected, is there any command predefined in redis or not?
Duplicate FSS_SYSAGENT to FSS_SYSAGENTDuplicate.
10.44.112.213:6403> hgetall FSS_SYSAGENT
1) "SYSTEM_01"
2) "{\"port\":\"4407\",\"ipAddress\":\"10.44.112.213\",\"symbolicName\":\"SYSTEM_01\",\"eventLogEnabled\":\"1110\",\"status\":1,\"wcPort\":\"6029\",\"activeSystem\":\"N\",\"createdBy\":\"\",\"createdDate\":\"2018-11-20 13:11:16\",\"modifiedBy\":\"\",\"modifiedDate\":\"\",\"institution\":\"FSS\",\"delFlag\":0,\"accessID\":0,\"rowCount\":0,\"endCount\":0}"
You can use the DUMP and RESTORE commands to duplicate the key:
use the DUMP command to serialize the value of a key.
use the RESTORE command to restore the serialized value to another key.
You can wrap these two steps into a Lua script:
-- duplicate.lua
local src = KEYS[1]
local dest = KEYS[2]
local val = redis.call('DUMP', src)
if val == false then
return 0
else
-- with RESTORE command, you can also set TTL for the new key, and use the [REPLACE] option to set the new key forcefully.
redis.call('RESTORE', dest, 0, val)
return 1
end
Run the Lua script with redis-cli: ./redis-cli --eval duplicate.lua FSS_SYSAGENT FSS_SYSAGENTDuplicate ,
UPDATE
Since Redis 6.2.0, you can use the COPY command to do the job.
Related
I have a Redis Cluster and I would like to execute a LUA script on a target node.
When I do that I receive an error from Redis Cluster that say that keys must be on the same SLOT, but the script executes two commands on only 1 key.
if redis.call('HEXISTS', 'TEST', KEYS[1]) == 1
then
redis.call('HSET', 'TEST', KEYS[1], ARGV[1])
return 1
end
return 0
I tried to use Hash Tag Key, but it doesn't work.
You are getting this error because you're trying to access a Redis key named TEST, but don't make it known to Redis or your Redis client by listing it in the EVAL command.
Redis Lua scripts take two argument lists, one is a general purpose (ARGV[]) and the other is a strict list of the keys the script is going to access (KEYS[]).
You may have swapped the key name and field names in the HEXISTS call, i.e. you may need to use:
if redis.call('HEXISTS', KEYS[1], 'TEST') == 1
then
redis.call('HSET', KEYS[1], 'TEST', ARGV[1])
return 1
end
return 0
This should work as long as the key name is properly passed on to the EVAL command.
I want to do the following in redis LUA scripting:
SPOP 1+ items from "source" set
SADD elements from #1 into "target" set
I'm using redis 5.
I have the below lua, but this is just for a single element:
local source = KEYS[1]
local target = KEYS[2]
local num = KEYS[3]
local ele = redis.call("SPOP", "source")
redis.call("SADD", target, ele)
return "OK"
How can I update the above with:
handle 1+ elements using the passed in param KEY[3]
ensure if 0 elements were returned from POP, it doesn't try and add to target set.
In Redis v5 and above this should "just work" due to the move to script effects replication as a default.
In v4 you'll have to execute redis.replicate_commands() before any random command in the script.
EDIT: per your edits and comment, here's an example:
-- uncomment the next line for Redis v4
-- redis.replicate_commands()
local source = KEYS[1]
local target = KEYS[2]
local num = ARGV[1]
local elems = redis.call("SPOP", source, num)
if #elems > 0 then
redis.call("SADD", target, unpack(elems))
end
return redis.status_reply("OK")
I have a lot of analytics data that I'm adding to redis. I plan on incrementally moving the data out of redis and into my database.
I know I can use KEYS [the_key]:* to get all keys that match. For example, I can do that to get the following:
127.0.0.1:6379> KEYS c_Track:*
1) "c_Track:6c93a5c1-77e9-4c4a-9232-bf182713a02e"
2) "c_Track:2c9d99c2-af37-4de9-ac64-b48f339e97a9"
3) "c_Track:9e7fd190-86d9-4b4a-9a70-7bf4c7768eef"
4) "c_Track:7f2d2e98-7440-4fd7-a80a-2af309ab15a4"
Is there a recommended way to get these values easily? I can get the keys, but how can I get all the values as well? I can loop through the keys to get the values, but is there some one-shot method for doing this?
Also I know I shouldn't use keys, but this is just an example. Thanks
Thanks
Also I know I shouldn't use keys
So don't. Use SCAN instead.
is there some one-shot method for doing this?
No, not as a core Redis command, but given the need this is fairly simple to achieve with a server-side Lua script. For example, assuming that your values are strings, you could do something like the following:
local cursor = tonumber(ARGV[1])
local pattern = ARGV[2]
local scan = redis.call('SCAN', cursor, 'MATCH', pattern)
for i, v in ipairs(scan[2]) do
local val = redis.call('GET', v)
scan[2][i] = { v, val }
end
return scan
Assuming that this script is saved under "scan.lua", you can run it as follows:
$ redis-cli SET foo bar
OK
$ redis-cli SET baz qaz
OK
$ redis-cli --eval scan.lua , 0 "*"
1) "0"
2) 1) 1) "baz"
2) "qaz"
2) 1) "foo"
2) "bar"
To scan your entire keyspace, call the script with the returned cursor until it returns 0.
Notes:
1) If your keys are of different types, you should change the script accordingly (e.g. https://github.com/itamarhaber/redis-lua-scripts/blob/master/scanfetch.lua).
2) While this script goes against the common recommendation of generating key names inside a script, it is still safe to run as SCAN returns keys that are in the server's keyspace (whether single-instance or clustered).
My redis collection contains many keys
I want to be able to flush them all except all the keys that start with:
"configurations::"
is this possible?
You can do this
redis-cli KEYS "*" | grep -v "configurations::" | xargs redis-cli DEL
List all keys into the redis, remove from the list keys that contains "configurations::" and delete them from the redis
Edit
As #Sergio Tulentsev notice it keys is not for use in production. I used this python script to remove keys on prodution redis. I stoped replication from master to slave before call the script.
#!/usr/bin/env python
import redis
import time
pattern = "yourpattern*"
poolSlave = redis.ConnectionPool(host='yourslavehost', port=6379, db=0)
redisSlave = redis.Redis(connection_pool=poolSlave)
poolMaster = redis.ConnectionPool(host='yourmasterhost', port=6379, db=0)
redisMaster = redis.Redis(connection_pool=poolMaster)
cursor = '0'
while cursor != 0:
cursor, data = redisSlave.scan(cursor, pattern, 1000)
print "cursor: "+str(cursor)
for key in data:
redisMaster.delete(key)
print "delete key: "+key
# reduce call per second on production server
time.sleep(1)
The SCAN & DEL approach (as proposed by #khanou) is the best ad-hoc solution. Alternatively, you could keep an index of all your configurations:: key names with a Redis Set (simply SADD the key's name to it whenever you create a new configurations:: key). Once you have this set you can SSCAN it to get all the relevant key names more efficiently (don't forget to SREM from it whenever you DEL though).
Yes, it's possible. Enumerate all the keys, evaluate each one and delete if it fits the criteria for deletion.
There is no built-in redis command for this, if this is what you were asking.
It might be possible to cook up a Lua script that will do this (and it'll look to your app that it's a single command), but still it's the same approach under the hood.
I have a rather large database (5 dbs of about a million keys each), and each key has the environment namespace in it. For example: "datamine::production::crosswalk==foobar"
I need to sync my development environment with this data copied from the production RDB snapshot.
So what I'm trying to do is batch rename every key, changing the namespace from datamine::production to datamine::development. Is there a good, way to achieve this?
What I've tried so far
redis-cli command of keys "datamine::production*", piped into sed, then back to redis-cli. This takes forever, and bombs for some reason on many keys (combining several in the same line, sporadically). I'd prefer a better option.
Perl search/replace on the .rdb file. My local redis-server flat refuses to load the modified RDB.
The solution:
Ok, here's the script I wrote to solve this problem. It requires the "Redis" gem. Hopefully someone else finds this useful...
#!/usr/bin/env ruby
# A script to translate the current redis database into a namespace for another environment
# GWI's Redis keys are namespaced as "datamine::production", "datamine::development", etc.
# This script connects to redis and translates these key names in-place.
#
# This script does not use Rails, but needs the "redis" gem available
require 'Benchmark'
require 'Redis'
FROM_NAMESPACE = "production"
TO_NAMESPACE = "development"
NAMESPACE_PREFIX = "datamine::"
REDIS_SERVER = "localhost"
REDIS_PORT = "6379"
REDIS_DBS = [0,1,2,3,4,5]
redis = Redis.new(host: REDIS_SERVER, port: REDIS_PORT, timeout: 30)
REDIS_DBS.each do |redis_db|
redis.select(redis_db)
puts "Translating db ##{redis_db}..."
seconds = Benchmark.realtime do
dbsize = redis.dbsize.to_f
inc_threshold = (dbsize/100.0).round
i = 0
old_keys = redis.keys("#{NAMESPACE_PREFIX}#{FROM_NAMESPACE}*")
old_keys.each do |old_key|
new_key = old_key.gsub(FROM_NAMESPACE, TO_NAMESPACE)
redis.rename(old_key, new_key)
print "#{((i/dbsize)*100.0).round}% complete\r" if (i % inc_threshold == 0) # on whole # % only
i += 1
end
end
puts "\nDone. It took #{seconds} seconds"
end
I have a working solution:
EVAL "local old_prefix_len = string.len(ARGV[1])
local keys = redis.call('keys', ARGV[1] .. '*')
for i = 1, #keys do
local old_key = keys[i]
local new_key = ARGV[2] .. string.sub(old_key, old_prefix_len + 1)
redis.call('rename', old_key, new_key)
end" 0 "datamine::production::" "datamine::development::"
Two last parameters are respectively an old prefix and a new prefix.