I am experimenting with use redis-search + redis-json. But facing problem querying on Nested Json with array. I created following JSON
{
"src":{
"location":[
{
"ref":"/uuid/1/xyz",
"key":"zone"
},
{
"ref":"/uuid/2/abc",
"key":"zone"
}
]
}
}
127.0.0.1:6379> JSON.SET 300:100 $ '{"src":{"location":[{"ref":"/uuid/1/xyz", "key":"zone"},{"ref":"/uuid/2/abc", "key":"zone"}]}}'
JSON.GET 300:100
"{\"src\":{\"location\":[{\"ref\":\"/uuid/1/xyz\",\"key\":\"zone\"},{\"ref\":\"/uuid/2/abc\",\"key\":\"zone\"}]}}"
127.0.0.1:6379> JSON.GET 300:100 $.src.location[*]
"[[{\"ref\":\"/uuid/1/xyz\",\"key\":\"zone\"},{\"ref\":\"/uuid/2/abc\",\"key\":\"zone\"}]]"
Created Index
127.0.0.1:6379> FT.CREATE 300:idx6 ON JSON SCHEMA $.src.location[*].ref as ref TAG
Tried searching with Tag
127.0.0.1:6379> FT.SEARCH 300:idx6 #ref:{/uuid/1/xyz}
1) (integer) 0
But it's not working. But if I replace / in the ref with _ I get the result
127.0.0.1:6379> JSON.SET 300:100 $ '{"src":{"location":[{"ref":"_uuid_1_xyz", "key":"zone"},{"ref":"_uuid_2_abc", "key":"zone"}]}}'
127.0.0.1:6379> FT.CREATE 300:idx7 ON JSON SCHEMA $.src.location[*].ref as ref TAG
OK
127.0.0.1:6379> FT.SEARCH 300:idx7 #ref:{_uuid_1_xyz}
1) (integer) 1
2) "300:100"
3) 1) "$"
2) "{\"src\":{\"location\":[{\"ref\":\"_uuid_1_xyz\",\"key\":\"zone\"},{\"ref\":\"_uuid_2_abc\",\"key\":\"zone\"}]}}"
Is there any issue using \ or how do I escape it ?
You need to escape the punctuation in the query, e.g.,
FT.SEARCH 300:idx6 #ref:{\/uuid\/1\/xyz}
See Including punctuation in tags
Which redis version are you using?
It is working when using Redis Stack docker image, e.g.,
docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest
(currently redis 6.2.6, RediSearch 2.2.10, RedisJSON 2.0.7)
Running the following commands in redis-cli
127.0.0.1:6379> JSON.SET 300:100 $ '{"src":{"location":[{"ref":"/uuid/1/xyz", "key":"zone"},{"ref":"/uuid/2/abc", "key":"zone"}]}}'
OK
127.0.0.1:6379> FT.CREATE 300:idx6 ON JSON SCHEMA $.src.location[*].ref as ref TAG
OK
And escaping the query does not require to replace /:
127.0.0.1:6379> FT.SEARCH 300:idx6 #ref:{\/uuid\/1\/xyz}
1) (integer) 1
2) "300:100"
3) 1) "$"
2) "{\"src\":{\"location\":[{\"ref\":\"/uuid/1/xyz\",\"key\":\"zone\"},{\"ref\":\"/uuid/2/abc\",\"key\":\"zone\"}]}}"
127.0.0.1:6379>
Related
When I directly run the below GET command in my redis cloud,
GET 1000:125:1603875000
I am getting error
WRONGTYPE Operation against a key holding the wrong kind of value redis
When I check
type 1000:125:1603875000
gives me
Hash
But if I execute SET before Get, like this
SET 1000:125:1603875000 11
I get "11" on executing GET command.
Why does the string is considered as Hash? How can I execute GET with the specified string.
That's because when you RUN 'SET 1000:125:1603875000' to 11, you are overwriting the initial '1000:125:1603875000' which was a hash and once you set '1000:125:1603875000' as '11' you can run a GET command to get the value of the key.
To get the value of a redis hash you can run HGETALL to get all the values in the hash or HGET KEYNAME to get a specific key of the hash.
To illustrate the use of these commands:
127.0.0.1:6379> HSET employee name Ankit
(integer) 1
127.0.0.1:6379> GET employee
(error) WRONGTYPE Operation against a key holding the wrong kind of value
127.0.0.1:6379> HGETALL employee
1) "name"
2) "Ankit"
127.0.0.1:6379> HGET employee name
"Ankit"
127.0.0.1:6379> SET employee Ankit
OK
127.0.0.1:6379> GET employee
"Ankit"
127.0.0.1:6379>
I'd like to periodically scan thru a redis instance for keys that changed since the last scan. in between the scans i don't want to process the keys.
eg one key could get a thousand updates between scans. i care for the most recent value only when doing the next periodic scan.
There is no built-in way in Redis to achieve that (yet).
You could, for example, recode your app and add some sort of a way to track updates. For example, wherever you're calling SET foo bar, also call ZADD updated <timestamp> foo. Then, you can use the 'updated' Sorted Set to retrieve updated keys.
Alternatively, you can try using RedisGears to automate the tracking part (for starters). Assuming that you have RedisGears running (i.e. docker run -it -p 6379:6379 redislabs/redisgears), you can do something like the following:
$ cat gear.py
def addToUpdatedZset(x):
import time
now = time.time()
execute('ZADD', 'updated', now, x['key'])
return x
GB().filter(lambda x: x['key'] != 'updated').foreach(addToUpdatedZset).register('*')
$ redis-cli RG.PYEXECUTE "$(cat gear.py)"
OK
$ redis-cli
127.0.0.1:6379> KEYS *
(empty list or set)
127.0.0.1:6379> SET foo bar
OK
127.0.0.1:6379> KEYS *
1) "updated"
2) "foo"
127.0.0.1:6379> ZRANGE updated 0 -1 WITHSCORES
1) "foo"
2) "1559339877.1392548"
127.0.0.1:6379> SET baz qux
OK
127.0.0.1:6379> KEYS *
1) "updated"
2) "baz"
3) "foo"
127.0.0.1:6379> ZRANGE updated 0 -1 WITHSCORES
1) "foo"
2) "1559339877.1392548"
3) "baz"
4) "1559339911.5493586"
In Redis, keys user* will print all keys starting with user.
For example:
keys user*
1) "user2"
2) "user1"
Now, I want all keys that don't start with user to be printed.
How could I do that?
IMPORTANT: always use SCAN instead of (the evil) KEYS
Redis' pattern matching is somewhat functionally limited (see the implementation of stringmatchlen in util.c) and does not provide that which you seek ATM. That said, consider the following possible routes:
Extend stringmatchlen to match your requirements, possibly submitting it as a PR.
Consider what you're trying to do - fetching a subset of keys is always going to be inefficient unless you index them, consider tracking the names of all non-user keys (i.e.g. in a Redis Set) instead.
If you are really insistent on scanning the entire keyspace and match against negative patterns, one way to accomplish that is with a little bit of Lua magic.
Consider the following dataset and script:
127.0.0.1:6379> dbsize
(integer) 0
127.0.0.1:6379> set user:1 1
OK
127.0.0.1:6379> set use:the:force luke
OK
127.0.0.1:6379> set non:user a
OK
Lua (save this as scanregex.lua):
local re = ARGV[1]
local nt = ARGV[2]
local cur = 0
local rep = {}
local tmp
if not re then
re = ".*"
end
repeat
tmp = redis.call("SCAN", cur, "MATCH", "*")
cur = tonumber(tmp[1])
if tmp[2] then
for k, v in pairs(tmp[2]) do
local fi = v:find(re)
if (fi and not nt) or (not fi and nt) then
rep[#rep+1] = v
end
end
end
until cur == 0
return rep
Output - first time regular matching, 2nd time the complement:
foo#bar:~$ redis-cli --eval scanregex.lua , "^user"
1) "user:1"
foo#bar:~$ redis-cli --eval scanregex.lua , "^user" 1
1) "use:the:force"
2) "non:user"
#Karthikeyan Gopall you nailed it in your comment above and this saved me a bunch of time. Thanks!
Here's how you can use it in various combinations to get what you want:
redis.domain.com:6379[1]> set "hello" "foo"
OK
redis.domain.com:6379[1]> set "hillo" "bar"
OK
redis.domain.com:6379[1]> set "user" "baz"
OK
redis.domain.com:6379[1]> set "zillo" "bash"
OK
redis.domain.com:6379[1]> scan 0
1) "0"
2) 1) "zillo"
2) "hello"
3) "user"
4) "hillo"
redis.domain.com:6379[1]> scan 0 match "[^u]*"
1) "0"
2) 1) "zillo"
2) "hello"
3) "hillo"
redis.domain.com:6379[1]> scan 0 match "[^u^z]*"
1) "0"
2) 1) "hello"
2) "hillo"
redis.domain.com:6379[1]> scan 0 match "h[^i]*"
1) "0"
2) 1) "hello"
According to redis keys documentation the command supports glob style patterns, not regular expressions.
and if you look at the documentation, you'll see that the "!" character is not special as opposites to regular expressions.
Here is a simple test I ran in my own db:
redis 127.0.0.1:6379> set a 0
OK
redis 127.0.0.1:6379> set b 1
OK
redis 127.0.0.1:6379> keys *
1) "a"
2) "b"
redis 127.0.0.1:6379> keys !a
(empty list or set) // I expected "b" here
redis 127.0.0.1:6379> keys !b
(empty list or set) // I expected "a" here
redis 127.0.0.1:6379> keys [!b]
1) "b"
redis 127.0.0.1:6379> keys [b]
1) "b"
redis 127.0.0.1:6379> keys [ab]
1) "a"
2) "b"
redis 127.0.0.1:6379> keys ![b]
(empty list or set)
So I just don't think what you are trying to achieve is possible via the keys command.
Besides, the keys command is not very suitable for production environment as it locks your whole redis database.
I would recommend getting all the keys with the scan command, store them locally, and then remove them using LUA
Here's a trick to achieve this with native redis commands (no need for Lua scripts or anything).
If you are able to control the timing of when you insert the new keys (the ones you want to keep, deleting all other stuff like in your question), you can:
Before setting the new keys, set the expiration to all existing keys (by pattern or everything) to expire right now (see how)
Load the new keys
Redis will automatically delete all the older keys and you will be left just with the new ones you want.
You also can print all keys and pass it to grep. For example:
redis-cli -a <password> keys "*" | grep -v "user"
Why do I get the result "raw" from the following?
redis 127.0.0.1:6379> set massage "hello"
OK
redis 127.0.0.1:6379> object encoding massage
"raw"
Does it have anything to do with?
#define REDIS_ENCODING_EMBSTR_SIZE_LIMIT 39
The 39 is to decide if to embed it or not. If you look at the exact piece of code where the define is it explains it all:
https://github.com/antirez/redis/blob/73a809b1591378e1042a1028d0b8e10217e6e7c7/src/object.c#L87
With regards to the raw that is the type for all what you call strings, if it is a valid number representation it is an Int.
Examples:
127.0.0.1:6379> set str "hello"
OK
127.0.0.1:6379> object encoding str
"raw"
127.0.0.1:6379> set int 1
OK
127.0.0.1:6379> object encoding int
"int"
127.0.0.1:6379> lpush list hello
(integer) 1
127.0.0.1:6379> object encoding list
"ziplist"
127.0.0.1:6379> zadd zset 1 1
(integer) 1
127.0.0.1:6379> object encoding zset
"ziplist"
127.0.0.1:6379> sadd set 1
(integer) 1
127.0.0.1:6379> object encoding set
"intset"
127.0.0.1:6379> hset hash field value
(integer) 1
127.0.0.1:6379> object encoding hash
"ziplist"
As you can can see this is how the object is represented internally to Redis.
If you want the actual type you could try the type command.
P.S. Please make your question more clear next time. It may also be worth adding references for where you have looked. You can not simply stumble across that define
I have a long key names in Redis, and would like to access them by their index. For example:
redis XXXX:6379[1]> KEYS *
1) "aaa"
2) "bbb"
3) "ccc"
4) "ddd"
And what I would like to do is something like:
redis XXXX:6379[1]> GET '1'
or
redis XXXX:6379[1]> GET KEYS[1]
To retrieve information about selected key.
Using redis-cli on ubuntu I am able to do following case:
search for keys and sort them
prompt>redis-cli keys a*|sort
ak
aka
asdf
To get first key
redis-cli keys a*|sort|head -n 1
Top 2 keys
redis-cli keys a*|sort|head -n 2
get second element
redis-cli keys a*|sort|head -n 2 |tail -1
This solution could be used in scripts.