When getting all the keys from Redis, like this:
redis.server.com:6379> keys *
1) "z13235jxby03knne1w1gucl5"
Instead of manually copying the long key to execute get z13235jxby03knne1w1gucl5, I'd like to run something like get $(1) (pseudo code) to get the value at position 1, as output by the keys command.
Is this possible, if not, is there any workaround to not have to manually copy paste?
Note, I don't want to solve this with a script, then I prefer just copy and paste it
off the top of my head, I'm not aware of a way from inside the cli.
But, Regardless of any performance implications, you can pipe the cli command lines together , but you have to do it from the shell
redis-cli --raw KEYS "*" | sed -n 1p | xargs redis-cli GET
where the 1 in :
sed -n 1p
is the line number (1-based index) inside KEYS output.
but still you need to do your validations; like making sure the index is withing the nuber of keys returned by the KEYS command all keys are of simple string type ; not sets, hash maps, etc...
Related
I am using redis-cli. I would like to add in a set the numbers 1 all the way to 4,500,000. How can I accomplish this? The docs for sadd do not mention how to do this. If this would involve having to type or paste in every single integer individually, this obviously won't work in the CLI, so is there another way programatically?
You can use a bash script for doing this.
Use 'seq' to create a list of numbers, loop over it and add the current iteration inside the set
#!/bin/bash
for i in `seq 10`
do
redis-cli sadd myset $i
done
In case you need to use redis-cli with host and port details, change the command accordingly: redis-cli -h -p
Once you have this ready in a .sh file, you can run it like:
bash redis_script.sh
I have the impression Im always typing again and again the same commands
ZRANGE mykey 0 100 WITHSCORES
and it is quite repetitive as I have to juggle between maps, sets, and sorted sets (and the client is not great, I can't use the same shortcuts that I use in my terminal to delete the previous/next word for instance)
Is there a way, like in bash, to write our own scripts to make our life easier ?
e.g
LISTALL mykey
You can use Redis from bash with redis-cli if that is what you mean?
Then you can make bash aliases. So, in bash:
function LISTALL() { redis-cli "ZRANGE $1 0 100 WITHSCORES"; }
then you can do
LISTALL mykey
and use bash editing.
I have seen this pass results to another command in redis
and using via command line this command works well :
src/redis-cli keys '*' | xargs src/redis-cli mget
However how can we achieve the same effect via Lettuce (i started trying out 4.0.2.Final)
Also a solution to this is particularly important in the following scenario :
Say we are using geolocation capabilities, and we add a set of locations of "my-location-category"
using GEOADD
GEOADD "category-1" 8.6638775 49.5282537 "location-id:1" 8.3796281 48.9978127 "location-id:2" 8.665351 49.553302 "location-id:3"
Next, say we do a GeoRadius to get locations within 10 km radius of 8.6582361 49.5285495 for "category-1"
Now when we get "location-id:1" & "location-id:3"
Given that I already set values for above keys "location-id:1" & "location-id:3"
I want to pipe commands to do the GEORADIUS as well as do mget on all the matching results.
Does Redis provide feature to do that?
and / or how can we achieve this via the Lettuce client library without first manually iterating through results of GEORADIUS and then do manual mget.
That would be more efficient performance for the program that uses it.
Does anyone know how we can do this ?
Update
This is the piped command for the scenario I discussed above :
src/redis-cli GEORADIUS "category-1" 8.6582361 49.5285495 10 km | xargs src/redis-cli mget
Now we need to know how to do this via Lettuce
IMPORTANT: never use KEYS, always use SCAN instead if you must.
This isn't really a question about Lettuce nor Java so I can actually answer it :)
What you're trying to do is use the results from a read operation (GEORADIUS) as input (key names) for another read operation (MGET). This type of flow can't be pipelined, well, just because of that - pipelining means that you don't need the answers for operations right away but in you case you do.
However.
Since you're reading String keys with MGET, you might as well just denormalize everything (remember, we're NoSQL) and store the contents of these keys in the Sorted Set's members, e.g.:
GEOADD "category-1" 8.6638775 49.5282537 "location-id:1:moredata:evenmoredata:{maybe a JSON document here}:orperhapsmsgpack"
This will allow you to get the locations and their "data" with one GEORADIUS call. Of course, any updates to location:1's data will need to be done across all categories.
A note about Lua scripts: while a Lua script could definitely save on the back and forth in this case, any such script will be against best practices/not cluster safe.
After digging around and studying Lua script, my conclusion is that removing round-trips in such a way can only be done via Lua scripts as suggested by Itamar Haber.
I ended up creating a lua script file (myscript.lua) as below
local locationKeys = redis.call('GEORADIUS', 'category-1', '8.6582361', '49.5285495', '10', 'km' )
if unpack(locationKeys) == nil then
return nil
else
return redis.call('MGET', unpack(locationKeys))
end
** of course we should be sending in parameters to this... this is just a poc :)
now you can execute it via command
src/redis-cli EVAL "$(cat myscript.lua)" 0
Then to reduce the network-overhead of sending across the entire script to Redis for execution, we have the option of registering the script with Redis.
Redis will give us a sha1 digested code for future references for that script, which can be used for next calls to that script.
This can be done as below :
src/redis-cli SCRIPT LOAD "$(cat myscript.lua)"
this should give back a sha1 code something like this : 49730aa2ed3034ee48f818e486tpbdf1b500b19e
next calls can be done using this code
eg
src/redis-cli evalsha 49730aa2ed3034ee48f818e486b2bdf1b500b19e 0
The sad part however here is that the sha1 digest is remembered only so long as the instance of redis is running. If it is restarted, that the sha1 digest is lost. Then you do the SCRIPT LOAD once again. And if nothing changes in the script, then the sha1-digest code will be the same.
Ideally while using through client api, we should first attempt evalsha, if that returns a "No matching script" error, then as a fallback do script load, and procure the sha1 code once again, and create an internal map of that and use that sha1 code for further calls.
This can well be done via Lettuce. I could find the methods for those. Hope this gives a good insight into solution for the problem.
My redis collection contains many keys
I want to be able to flush them all except all the keys that start with:
"configurations::"
is this possible?
You can do this
redis-cli KEYS "*" | grep -v "configurations::" | xargs redis-cli DEL
List all keys into the redis, remove from the list keys that contains "configurations::" and delete them from the redis
Edit
As #Sergio Tulentsev notice it keys is not for use in production. I used this python script to remove keys on prodution redis. I stoped replication from master to slave before call the script.
#!/usr/bin/env python
import redis
import time
pattern = "yourpattern*"
poolSlave = redis.ConnectionPool(host='yourslavehost', port=6379, db=0)
redisSlave = redis.Redis(connection_pool=poolSlave)
poolMaster = redis.ConnectionPool(host='yourmasterhost', port=6379, db=0)
redisMaster = redis.Redis(connection_pool=poolMaster)
cursor = '0'
while cursor != 0:
cursor, data = redisSlave.scan(cursor, pattern, 1000)
print "cursor: "+str(cursor)
for key in data:
redisMaster.delete(key)
print "delete key: "+key
# reduce call per second on production server
time.sleep(1)
The SCAN & DEL approach (as proposed by #khanou) is the best ad-hoc solution. Alternatively, you could keep an index of all your configurations:: key names with a Redis Set (simply SADD the key's name to it whenever you create a new configurations:: key). Once you have this set you can SSCAN it to get all the relevant key names more efficiently (don't forget to SREM from it whenever you DEL though).
Yes, it's possible. Enumerate all the keys, evaluate each one and delete if it fits the criteria for deletion.
There is no built-in redis command for this, if this is what you were asking.
It might be possible to cook up a Lua script that will do this (and it'll look to your app that it's a single command), but still it's the same approach under the hood.
I have a string type key value store in redis having keys like this--
/url-pattern/url-slug-1
/url-pattern/url-slug-2
/url-pattern/url-slug-3
/url-pattern/url-slug-4 ...
I can retrieve all the keys of /url-pattern/ using a wild card query like this --
keys /url-pattern/*
I would like to retrieve the values of all keys corresponding to this wildcard /url-pattern/*
I tried this
mget /url-pattern/*
1) (nil)
but it doesnt returned the array as expected.
How can I retrieve the values of all keys corresponding to /url-pattern/*
I also want to do a sum on the values, but I think there is no such thing called SUM() in redis
MGET accepts multiple arguments where each a key name. It does not do key name patterns.
What you could do is first fetch all the relevant key names (do not use KEYS, use SCAN instead) and then fetch their values with an MGET.
Here is an updated answer for 2015.
If you can upgrade Redis above 2.8, the SCAN command with MATCH will work for this. Before that version, not so much, and do NOT use the KEYS command except in a development environment.
http://redis.io/commands/scan
Example on command line:
$ redis-cli
127.0.0.1:6379> scan match V3.0:*
(error) ERR invalid cursor
127.0.0.1:6379> scan 0 match V3.0:*
1) "0"
2) 1) "V3.0:UNITTEST55660BC7E0C5B"
2) "V3.0:shop.domain.com:route"
3) "V3.0:UNITTEST55660BC4A2548"
127.0.0.1:6379> scan 0 match V1.0:*
1) "0"
2) (empty list or set)
127.0.0.1:6379> scan 0 match V3.0:*
1) "0"
2) 1) "V3.0:UNITTEST55660BC7E0C5B"
2) "V3.0:shop.domain.com:route"
3) "V3.0:UNITTEST55660BC4A2548"
Example in PHP:
// Initialize our iterator to NULL
$iterate = null;
// retry when we get no keys back
$redis->setOption(Redis::OPT_SCAN, Redis::SCAN_RETRY);
while ($arr_keys = $redis->scan($iterate, 'match:*')) {
foreach ($arr_keys as $str_key) {
echo "Here is a key: $str_key\n";
}
echo "No more keys to scan!\n";
}
Note, php code is not tested and from the core documentation for example here. Production code would need to be modified depending on the keys needed to look up.
For those on Ubuntu here are the instructions to upgrade php5-redis:
Download the 2.2.7 package here: http://pecl.php.net/package/redis
$ php -i | grep Redis
Redis Support => enabled
Redis Version => 2.2.4
Follow instructions in README to phpize, configure, make install
Create a symlink for command line cli package: cd /etc/php5/cli/conf.d && sudo ln -s ../../mods-available/redis.ini 20-redis.ini
$ php -i | grep Redis
Redis Support => enabled
Redis Version => 2.2.7
There is NO command available in REDIS which can return values from wildcard keys.
If you see the documentation for KEYS command: http://redis.io/commands/keys, it says
Consider KEYS as a command that should only be used in production
environments with extreme care. It may ruin performance when it is
executed against large databases. This command is intended for
debugging and special operations. Don't use KEYS in your regular
application code.
I don't know your business use case, but looks like you may have to use different data structure for this requirement. You can use list or set to store similar url patterns.