I am using RedisJSON and I can't figure out how to get the full JSON values stored for multiple keys, following the documentation on the official site of the plugin.
For example, from the redis-cli:
redis-cli-1234> json.set k1 . '{"a":"a" }'
OK
redis-cli-1234> json.set k2 . '{"b":"b" }'
OK
Now I would like to run a command that would get k1, k2 (the keys of the desired values) and return their values, in one-go:
1) "{\"a\":\"a\"}"
2) "{\"b\":\"b\"}"
But I can't figure out how.
I've tried:
redis-cli-1234> json.mget k1 k2 .
And several other stuff, but I can only get redis to return either one value or the other, bot not both.
Any idea?
https://redis.io/topics/transactions
redis-cloud:6379> multi
OK
redis-cloud:6379(TX)> json.get k1
QUEUED
redis-cloud:6379(TX)> json.get k2
QUEUED
redis-cloud:6379(TX)> exec
1) "{\"a\":\"a\"}"
2) "{\"b\":\"b\"}"
Related
When getting all the keys from Redis, like this:
redis.server.com:6379> keys *
1) "z13235jxby03knne1w1gucl5"
Instead of manually copying the long key to execute get z13235jxby03knne1w1gucl5, I'd like to run something like get $(1) (pseudo code) to get the value at position 1, as output by the keys command.
Is this possible, if not, is there any workaround to not have to manually copy paste?
Note, I don't want to solve this with a script, then I prefer just copy and paste it
off the top of my head, I'm not aware of a way from inside the cli.
But, Regardless of any performance implications, you can pipe the cli command lines together , but you have to do it from the shell
redis-cli --raw KEYS "*" | sed -n 1p | xargs redis-cli GET
where the 1 in :
sed -n 1p
is the line number (1-based index) inside KEYS output.
but still you need to do your validations; like making sure the index is withing the nuber of keys returned by the KEYS command all keys are of simple string type ; not sets, hash maps, etc...
I am trying to run the code from the redis transactions page. Specifically, this part:
WATCH zset
element = ZRANGE zset 0 0
MULTI
ZREM zset element
EXEC
If I try to do it from the cli, line by line, I get this:
localhost:6380> zadd set 1 a
(integer) 1
localhost:6380> WATCH zset
localhost:6380> element = ZRANGE zset 0 0
(error) ERR unknown command 'element'
OK
which probably means I'm doing something wrong? I remember working with lua about 9 years ago, so this doesn't really look like lua either to me.
How does someone run that snippet? Is it only some kind of pseudocode?
As #Dinei said, the example given is pseudocode.
Let's look at it (I added line numbers for us to refer to):
1 WATCH zset
2 element = ZRANGE zset 0 0
3 MULTI
4 ZREM zset element
5 EXEC
The point of the exercise is to solve the race condition that would occur if we only read the key (with ZRANGE, in line 2), and then modify the key (with ZREM in line 4). I assume you understand the problem if we didn't use the "CAS" semantics, so no need to get into it.
As pointed out, redis-cli just gives you the ability to run redis commands and see their replies, but not save values in variables, etc.
So the idea of the example is that in line 2, we are "saving" the result of the "read" operation, into a pseudo-variable element.
Then, in line 4, we are using that value in our "set" operation, and of course lines 1, 3 and 5 are just the "CAS" commands to ensure there is no race condition.
Presumably the actual usage of such commands would be done from a redis client in a programming language that would allow us to save the return value of the ZRANGE and then use it later in the ZREM command.
But if you wanted to run it in redis-cli, you'd see this, where we pretend that our client-side code would have read and saved "a" that was returned from zrange and then passed that value to the zrem command:
127.0.0.1:6379> zadd zset 1 a
(integer) 1
127.0.0.1:6379> watch zset
OK
127.0.0.1:6379> zrange zset 0 0
1) "a"
127.0.0.1:6379> multi
OK
127.0.0.1:6379> zrem zset a
QUEUED
127.0.0.1:6379> exec
1) (integer) 1
127.0.0.1:6379>
Yes, it is some kind of pseudocode.
redis-cli only accepts Redis commands, it is not a full-fledged editor nor supports direct Lua scripting (neither variables like the element variable in the pseudocode).
I remember working with lua about 9 years ago, so this doesn't really look like lua either to me.
This is not Lua, it is pseudocode. Actually, the Redis Transactions page you linked to does not refer to Lua at all (and that's why #Piglet's comment in your post makes sense).
However, it is possible to execute Lua scripts by using Redis' EVAL command.
I have a lot of analytics data that I'm adding to redis. I plan on incrementally moving the data out of redis and into my database.
I know I can use KEYS [the_key]:* to get all keys that match. For example, I can do that to get the following:
127.0.0.1:6379> KEYS c_Track:*
1) "c_Track:6c93a5c1-77e9-4c4a-9232-bf182713a02e"
2) "c_Track:2c9d99c2-af37-4de9-ac64-b48f339e97a9"
3) "c_Track:9e7fd190-86d9-4b4a-9a70-7bf4c7768eef"
4) "c_Track:7f2d2e98-7440-4fd7-a80a-2af309ab15a4"
Is there a recommended way to get these values easily? I can get the keys, but how can I get all the values as well? I can loop through the keys to get the values, but is there some one-shot method for doing this?
Also I know I shouldn't use keys, but this is just an example. Thanks
Thanks
Also I know I shouldn't use keys
So don't. Use SCAN instead.
is there some one-shot method for doing this?
No, not as a core Redis command, but given the need this is fairly simple to achieve with a server-side Lua script. For example, assuming that your values are strings, you could do something like the following:
local cursor = tonumber(ARGV[1])
local pattern = ARGV[2]
local scan = redis.call('SCAN', cursor, 'MATCH', pattern)
for i, v in ipairs(scan[2]) do
local val = redis.call('GET', v)
scan[2][i] = { v, val }
end
return scan
Assuming that this script is saved under "scan.lua", you can run it as follows:
$ redis-cli SET foo bar
OK
$ redis-cli SET baz qaz
OK
$ redis-cli --eval scan.lua , 0 "*"
1) "0"
2) 1) 1) "baz"
2) "qaz"
2) 1) "foo"
2) "bar"
To scan your entire keyspace, call the script with the returned cursor until it returns 0.
Notes:
1) If your keys are of different types, you should change the script accordingly (e.g. https://github.com/itamarhaber/redis-lua-scripts/blob/master/scanfetch.lua).
2) While this script goes against the common recommendation of generating key names inside a script, it is still safe to run as SCAN returns keys that are in the server's keyspace (whether single-instance or clustered).
In Redis, keys user* will print all keys starting with user.
For example:
keys user*
1) "user2"
2) "user1"
Now, I want all keys that don't start with user to be printed.
How could I do that?
IMPORTANT: always use SCAN instead of (the evil) KEYS
Redis' pattern matching is somewhat functionally limited (see the implementation of stringmatchlen in util.c) and does not provide that which you seek ATM. That said, consider the following possible routes:
Extend stringmatchlen to match your requirements, possibly submitting it as a PR.
Consider what you're trying to do - fetching a subset of keys is always going to be inefficient unless you index them, consider tracking the names of all non-user keys (i.e.g. in a Redis Set) instead.
If you are really insistent on scanning the entire keyspace and match against negative patterns, one way to accomplish that is with a little bit of Lua magic.
Consider the following dataset and script:
127.0.0.1:6379> dbsize
(integer) 0
127.0.0.1:6379> set user:1 1
OK
127.0.0.1:6379> set use:the:force luke
OK
127.0.0.1:6379> set non:user a
OK
Lua (save this as scanregex.lua):
local re = ARGV[1]
local nt = ARGV[2]
local cur = 0
local rep = {}
local tmp
if not re then
re = ".*"
end
repeat
tmp = redis.call("SCAN", cur, "MATCH", "*")
cur = tonumber(tmp[1])
if tmp[2] then
for k, v in pairs(tmp[2]) do
local fi = v:find(re)
if (fi and not nt) or (not fi and nt) then
rep[#rep+1] = v
end
end
end
until cur == 0
return rep
Output - first time regular matching, 2nd time the complement:
foo#bar:~$ redis-cli --eval scanregex.lua , "^user"
1) "user:1"
foo#bar:~$ redis-cli --eval scanregex.lua , "^user" 1
1) "use:the:force"
2) "non:user"
#Karthikeyan Gopall you nailed it in your comment above and this saved me a bunch of time. Thanks!
Here's how you can use it in various combinations to get what you want:
redis.domain.com:6379[1]> set "hello" "foo"
OK
redis.domain.com:6379[1]> set "hillo" "bar"
OK
redis.domain.com:6379[1]> set "user" "baz"
OK
redis.domain.com:6379[1]> set "zillo" "bash"
OK
redis.domain.com:6379[1]> scan 0
1) "0"
2) 1) "zillo"
2) "hello"
3) "user"
4) "hillo"
redis.domain.com:6379[1]> scan 0 match "[^u]*"
1) "0"
2) 1) "zillo"
2) "hello"
3) "hillo"
redis.domain.com:6379[1]> scan 0 match "[^u^z]*"
1) "0"
2) 1) "hello"
2) "hillo"
redis.domain.com:6379[1]> scan 0 match "h[^i]*"
1) "0"
2) 1) "hello"
According to redis keys documentation the command supports glob style patterns, not regular expressions.
and if you look at the documentation, you'll see that the "!" character is not special as opposites to regular expressions.
Here is a simple test I ran in my own db:
redis 127.0.0.1:6379> set a 0
OK
redis 127.0.0.1:6379> set b 1
OK
redis 127.0.0.1:6379> keys *
1) "a"
2) "b"
redis 127.0.0.1:6379> keys !a
(empty list or set) // I expected "b" here
redis 127.0.0.1:6379> keys !b
(empty list or set) // I expected "a" here
redis 127.0.0.1:6379> keys [!b]
1) "b"
redis 127.0.0.1:6379> keys [b]
1) "b"
redis 127.0.0.1:6379> keys [ab]
1) "a"
2) "b"
redis 127.0.0.1:6379> keys ![b]
(empty list or set)
So I just don't think what you are trying to achieve is possible via the keys command.
Besides, the keys command is not very suitable for production environment as it locks your whole redis database.
I would recommend getting all the keys with the scan command, store them locally, and then remove them using LUA
Here's a trick to achieve this with native redis commands (no need for Lua scripts or anything).
If you are able to control the timing of when you insert the new keys (the ones you want to keep, deleting all other stuff like in your question), you can:
Before setting the new keys, set the expiration to all existing keys (by pattern or everything) to expire right now (see how)
Load the new keys
Redis will automatically delete all the older keys and you will be left just with the new ones you want.
You also can print all keys and pass it to grep. For example:
redis-cli -a <password> keys "*" | grep -v "user"
I have a string type key value store in redis having keys like this--
/url-pattern/url-slug-1
/url-pattern/url-slug-2
/url-pattern/url-slug-3
/url-pattern/url-slug-4 ...
I can retrieve all the keys of /url-pattern/ using a wild card query like this --
keys /url-pattern/*
I would like to retrieve the values of all keys corresponding to this wildcard /url-pattern/*
I tried this
mget /url-pattern/*
1) (nil)
but it doesnt returned the array as expected.
How can I retrieve the values of all keys corresponding to /url-pattern/*
I also want to do a sum on the values, but I think there is no such thing called SUM() in redis
MGET accepts multiple arguments where each a key name. It does not do key name patterns.
What you could do is first fetch all the relevant key names (do not use KEYS, use SCAN instead) and then fetch their values with an MGET.
Here is an updated answer for 2015.
If you can upgrade Redis above 2.8, the SCAN command with MATCH will work for this. Before that version, not so much, and do NOT use the KEYS command except in a development environment.
http://redis.io/commands/scan
Example on command line:
$ redis-cli
127.0.0.1:6379> scan match V3.0:*
(error) ERR invalid cursor
127.0.0.1:6379> scan 0 match V3.0:*
1) "0"
2) 1) "V3.0:UNITTEST55660BC7E0C5B"
2) "V3.0:shop.domain.com:route"
3) "V3.0:UNITTEST55660BC4A2548"
127.0.0.1:6379> scan 0 match V1.0:*
1) "0"
2) (empty list or set)
127.0.0.1:6379> scan 0 match V3.0:*
1) "0"
2) 1) "V3.0:UNITTEST55660BC7E0C5B"
2) "V3.0:shop.domain.com:route"
3) "V3.0:UNITTEST55660BC4A2548"
Example in PHP:
// Initialize our iterator to NULL
$iterate = null;
// retry when we get no keys back
$redis->setOption(Redis::OPT_SCAN, Redis::SCAN_RETRY);
while ($arr_keys = $redis->scan($iterate, 'match:*')) {
foreach ($arr_keys as $str_key) {
echo "Here is a key: $str_key\n";
}
echo "No more keys to scan!\n";
}
Note, php code is not tested and from the core documentation for example here. Production code would need to be modified depending on the keys needed to look up.
For those on Ubuntu here are the instructions to upgrade php5-redis:
Download the 2.2.7 package here: http://pecl.php.net/package/redis
$ php -i | grep Redis
Redis Support => enabled
Redis Version => 2.2.4
Follow instructions in README to phpize, configure, make install
Create a symlink for command line cli package: cd /etc/php5/cli/conf.d && sudo ln -s ../../mods-available/redis.ini 20-redis.ini
$ php -i | grep Redis
Redis Support => enabled
Redis Version => 2.2.7
There is NO command available in REDIS which can return values from wildcard keys.
If you see the documentation for KEYS command: http://redis.io/commands/keys, it says
Consider KEYS as a command that should only be used in production
environments with extreme care. It may ruin performance when it is
executed against large databases. This command is intended for
debugging and special operations. Don't use KEYS in your regular
application code.
I don't know your business use case, but looks like you may have to use different data structure for this requirement. You can use list or set to store similar url patterns.