Lua script on Redis Cluster - redis

I have a Redis Cluster and I would like to execute a LUA script on a target node.
When I do that I receive an error from Redis Cluster that say that keys must be on the same SLOT, but the script executes two commands on only 1 key.
if redis.call('HEXISTS', 'TEST', KEYS[1]) == 1
then
redis.call('HSET', 'TEST', KEYS[1], ARGV[1])
return 1
end
return 0
I tried to use Hash Tag Key, but it doesn't work.

You are getting this error because you're trying to access a Redis key named TEST, but don't make it known to Redis or your Redis client by listing it in the EVAL command.
Redis Lua scripts take two argument lists, one is a general purpose (ARGV[]) and the other is a strict list of the keys the script is going to access (KEYS[]).
You may have swapped the key name and field names in the HEXISTS call, i.e. you may need to use:
if redis.call('HEXISTS', KEYS[1], 'TEST') == 1
then
redis.call('HSET', KEYS[1], 'TEST', ARGV[1])
return 1
end
return 0
This should work as long as the key name is properly passed on to the EVAL command.

Related

How to store in Redis sorted set with server-side timestamp as score?

I want to use a sorted set to store objects using the redis-server timestamp as score.
I know I can use Redis Streams with * id, but Redis Streams have limitations, including I cannot edit the objects, I cannot use rank or lexicographical sorting, I cannot really delete objects in the middle, unions or intersects, etc.
I want to do this atomically, and use the redis-server timestamp so I can use multiple clients to ZADD without worrying about clock-sync.
How to do this?
The solution is to use a Lua script:
local time = redis.call('TIME')
local ts = time[1]..string.format('%06d', time[2])
return redis.call('ZADD', KEYS[1], ts, ARGV[1])
Here we use Redis TIME command. The command returns:
unix time in seconds
microseconds
So we can concatenate these two and use a microsecond-timestamp. We need to zero-pad the microseconds part.
Since sorted sets are good with integer values up to 2^53, our timestamp is safe all the way up to the year 2255.
This is Redis-Cluster-safe as we store in one key. To use multiple keys, make sure to land them on the same node using hash tags if you want to compare timestamps.
You can modify the script to use lower than microsecond resolution.
Here the EVAL command, simple pass key and value as arguments, no need to create the sorted set before hand:
EVAL "local time = redis.call('TIME') local ts = time[1]..string.format('%06d', time[2]) return redis.call('ZADD', KEYS[1], ts, ARGV[1])" 1 ssetKey myVal
As always, you may want to load the script and use EVALSHA.
> SCRIPT LOAD "local time = redis.call('TIME') local ts = time[1]..string.format('%06d', time[2]) return redis.call('ZADD', KEYS[1], ts, ARGV[1])"
"81e366e422d0b09c9b395b5dfe03c03c3b7b3bf7"
> EVALSHA 81e366e422d0b09c9b395b5dfe03c03c3b7b3bf7 1 ssetKey myNewVal
(integer) 1
A note on Redis version. If you are using:
Redis Version before 3.2: sorry, you cannot use TIME (non-deterministic command) and then write with ZADD.
Redis Version greater than 3.2 but < 5.0: Add redis.replicate_commands() on top of the script. See Scripts as pure functions
Redis 5.0 an up: you are good.

'(error) ERR Error running script (call to ...)' when using --eval for LUA script

I am getting an error message when running this command:
redis-cli --eval myscript.lua myzset 3 "one"
Error message:
(error) ERR Error running script (call to f_9c623c243d74e75a4fe64de7a6826b47f8d7
d400): #user_script:1: #user_script: 1: Lua redis() command arguments must be st
rings or integers
Lua script:
local ans = redis.call('ZINCRBY', KEYS[1], ARGV[1], ARGV[2])
if tonumber(ans) <= 0 then
redis.call('ZREM', KEYS[1], ARGV[2])
end
return (tonumber(ans) < 1 and '-1' or ans)
The goal is to remove automatically Zero or Negative value using ZINCRBY.
I am using:
Windows 7 64bit
Redis Version: 3.2.100
Your help will be appreciated.
The problem you're experiencing due to the fact that you're not separating the KEYS and ARGS - use a comma in the command line to do so. Also, note that since the comma is the delimiter, you'll need to use an extra space in order to ensure that it isn't regarded as a part of the key.
Like so:
redis-cli --eval myscript.lua myzset , 3 "one"

Duplicate a key in redis

Can I duplicate a key using the redis-cli connected, is there any command predefined in redis or not?
Duplicate FSS_SYSAGENT to FSS_SYSAGENTDuplicate.
10.44.112.213:6403> hgetall FSS_SYSAGENT
1) "SYSTEM_01"
2) "{\"port\":\"4407\",\"ipAddress\":\"10.44.112.213\",\"symbolicName\":\"SYSTEM_01\",\"eventLogEnabled\":\"1110\",\"status\":1,\"wcPort\":\"6029\",\"activeSystem\":\"N\",\"createdBy\":\"\",\"createdDate\":\"2018-11-20 13:11:16\",\"modifiedBy\":\"\",\"modifiedDate\":\"\",\"institution\":\"FSS\",\"delFlag\":0,\"accessID\":0,\"rowCount\":0,\"endCount\":0}"
You can use the DUMP and RESTORE commands to duplicate the key:
use the DUMP command to serialize the value of a key.
use the RESTORE command to restore the serialized value to another key.
You can wrap these two steps into a Lua script:
-- duplicate.lua
local src = KEYS[1]
local dest = KEYS[2]
local val = redis.call('DUMP', src)
if val == false then
return 0
else
-- with RESTORE command, you can also set TTL for the new key, and use the [REPLACE] option to set the new key forcefully.
redis.call('RESTORE', dest, 0, val)
return 1
end
Run the Lua script with redis-cli: ./redis-cli --eval duplicate.lua FSS_SYSAGENT FSS_SYSAGENTDuplicate ,
UPDATE
Since Redis 6.2.0, you can use the COPY command to do the job.

How to get all keys/values from redis in order to insert them into SQL db?

I have a lot of analytics data that I'm adding to redis. I plan on incrementally moving the data out of redis and into my database.
I know I can use KEYS [the_key]:* to get all keys that match. For example, I can do that to get the following:
127.0.0.1:6379> KEYS c_Track:*
1) "c_Track:6c93a5c1-77e9-4c4a-9232-bf182713a02e"
2) "c_Track:2c9d99c2-af37-4de9-ac64-b48f339e97a9"
3) "c_Track:9e7fd190-86d9-4b4a-9a70-7bf4c7768eef"
4) "c_Track:7f2d2e98-7440-4fd7-a80a-2af309ab15a4"
Is there a recommended way to get these values easily? I can get the keys, but how can I get all the values as well? I can loop through the keys to get the values, but is there some one-shot method for doing this?
Also I know I shouldn't use keys, but this is just an example. Thanks
Thanks
Also I know I shouldn't use keys
So don't. Use SCAN instead.
is there some one-shot method for doing this?
No, not as a core Redis command, but given the need this is fairly simple to achieve with a server-side Lua script. For example, assuming that your values are strings, you could do something like the following:
local cursor = tonumber(ARGV[1])
local pattern = ARGV[2]
local scan = redis.call('SCAN', cursor, 'MATCH', pattern)
for i, v in ipairs(scan[2]) do
local val = redis.call('GET', v)
scan[2][i] = { v, val }
end
return scan
Assuming that this script is saved under "scan.lua", you can run it as follows:
$ redis-cli SET foo bar
OK
$ redis-cli SET baz qaz
OK
$ redis-cli --eval scan.lua , 0 "*"
1) "0"
2) 1) 1) "baz"
2) "qaz"
2) 1) "foo"
2) "bar"
To scan your entire keyspace, call the script with the returned cursor until it returns 0.
Notes:
1) If your keys are of different types, you should change the script accordingly (e.g. https://github.com/itamarhaber/redis-lua-scripts/blob/master/scanfetch.lua).
2) While this script goes against the common recommendation of generating key names inside a script, it is still safe to run as SCAN returns keys that are in the server's keyspace (whether single-instance or clustered).

Redis delete all keys except keys that start with

My redis collection contains many keys
I want to be able to flush them all except all the keys that start with:
"configurations::"
is this possible?
You can do this
redis-cli KEYS "*" | grep -v "configurations::" | xargs redis-cli DEL
List all keys into the redis, remove from the list keys that contains "configurations::" and delete them from the redis
Edit
As #Sergio Tulentsev notice it keys is not for use in production. I used this python script to remove keys on prodution redis. I stoped replication from master to slave before call the script.
#!/usr/bin/env python
import redis
import time
pattern = "yourpattern*"
poolSlave = redis.ConnectionPool(host='yourslavehost', port=6379, db=0)
redisSlave = redis.Redis(connection_pool=poolSlave)
poolMaster = redis.ConnectionPool(host='yourmasterhost', port=6379, db=0)
redisMaster = redis.Redis(connection_pool=poolMaster)
cursor = '0'
while cursor != 0:
cursor, data = redisSlave.scan(cursor, pattern, 1000)
print "cursor: "+str(cursor)
for key in data:
redisMaster.delete(key)
print "delete key: "+key
# reduce call per second on production server
time.sleep(1)
The SCAN & DEL approach (as proposed by #khanou) is the best ad-hoc solution. Alternatively, you could keep an index of all your configurations:: key names with a Redis Set (simply SADD the key's name to it whenever you create a new configurations:: key). Once you have this set you can SSCAN it to get all the relevant key names more efficiently (don't forget to SREM from it whenever you DEL though).
Yes, it's possible. Enumerate all the keys, evaluate each one and delete if it fits the criteria for deletion.
There is no built-in redis command for this, if this is what you were asking.
It might be possible to cook up a Lua script that will do this (and it'll look to your app that it's a single command), but still it's the same approach under the hood.