I have n keys with the same pattern key:abc:<id> that store unsorted set. I want to do the union for them all, but according to Redis documentation (https://redis.io/commands/sunion) I can only specify separate keys in the command like SUNION key:abc:1 key:abc:2 key:abc:3
I want something like SUNION key:abc:*
Related
I'm looking into redis for my specific need but I know don't see how I could achieve what I need.
I want to store sets of integers (100s-low thousands entries per set) and then "query" by an input set. All sets that contain all the elements as the query should match. (SDIFF query key should return empty set and then iterate over all keys).
I don't see how this can be done efficiently (about 5ms per 10k keys)?
If you will only query your data by integer, consider then storing using the integer as a key. Instead of:
SADD a 3 5
SADD b 3 7
You can:
SADD int:3 a
SADD int:5 a
SADD int:3 b
SADD int:7 b
Then you use SINTER int:3 int:7 ... to get all matching integer set names (what you used for keys originally).
If you do need to query both ways, then you may need to do both. This is like doing a many-to-many relationship in Redis.
Here, you are trading off insert time and memory usage for fast query performance. Every time you add a pair, you need to both SADDs: SADD setName int and SADD prefix:int setName.
If this extra memory and insert effort is not an option on your case, your next option is to use a Lua Script to loop through the keys (pattern matching your set names) and using SISMEMBER to test through the integers of your query. See Lua script for Redis which sums the values of keys for an example of looping through a set of keys using Lua.
A Lua script is like a stored procedure, it will run atomically on your Redis server. However, if it will give perform at 5ms for 10k sets tested for multiple integer members remains to be seen.
I am using Jedis, and new to both that and Redis itself. I have db that stores hashes, and need to find all keys in the db that contain an entry with a specified key and a specified value. EG: "find all hashes in the db that have key/value of STATUS=ERROR". Is this possible in Jedis? From what I can tell from googling, hscan will find keys in a specified hash.
More generally, by way of teaching me to fish, any pointers for where to look this up? It seems there is no real jedis api doc, and not even the Redis doc itself seems to have nothing on hscan.
As you mentioned, you can use HSCAN to find the specified key-value pair from a hash. Also, you need to use the SCAN command to find all hashes.
However, this is NOT an efficient solution. In order to achieve your goal efficiently, you need to build an extra index, i.e. use a Redis SET to save keys of all hashes that have the specified key-value pair.
HSET hash1 STATUS ERROR
// ...
// HSET other members
// ...
// add it to index
SADD status:error hash1
// get all hashes have the specified key-value pair
SMEMBERS status:error
UPDATE:
As #Itamar Haber mentioned in the comments, if you have many records in the SET, you should use SSCAN to get these members. Since in this case, SMEMBERS might block Redis for a long time.
This is a 2 part question.
I have a redis db storing items with the following keys:
record type 1: "site_id:1_item_id:3"
record type 2: "site_id:1_item_id:3_user_id:6"
I've been using KEYS site_id:1_item_id:* to grab record type 1 items (in this case for site 1)
Unfortunately, it returns all type 1 and type 2 items.
Whats the best way to grab all "site_id:1_item_id:3" type records? While avoiding the ones including user_id? Is there an EOL match I can use?
Secondly, I've read using KEYS is a bad choice, can anyone recommend a different approach here? I'm open to editing the key names if I must.
First thing first: unless your are the only redis user on your local developpment machine, you are right using KEYS is wrong. It blocks the redis instance until completed, so anyone querying it while you are using keys will have to wait for you keys query to be finished. Use SCAN instead.
SCAN will iterate over the entries in a non blocking way, and you are guaranteed to get all of them.
I don't know which language you use to query it, but in python it is quite easy to query the keys with scan and filter them on the fly.
But let's say you would like to use keys anyway. Looks to me like either doing KEYS site_id:1_item_id:? or KEYS site_id:1_item_id:3 does the trick.
wether you want the keys finishing with "3" or not (I am not sure I completely understood your question here).
Here is an example that I tried on my local machine:
redis 127.0.0.1:6379> flushall
OK
redis 127.0.0.1:6379> set site_id:1_item_id:3 a
OK
redis 127.0.0.1:6379> set site_id:1_item_id:3_user_id:6 b
OK
redis 127.0.0.1:6379> set site_id:1_item_id:4 c
OK
// ok so we have got the database cleaned and set up
redis 127.0.0.1:6379> keys *
1) "site_id:1_item_id:3"
2) "site_id:1_item_id:4"
3) "site_id:1_item_id:3_user_id:6"
// gets all the keys like site_id:1_item_id:X
redis 127.0.0.1:6379> keys site_id:1_item_id:?
1) "site_id:1_item_id:3"
2) "site_id:1_item_id:4"
// gets all the keys like site_id:1_item_id:3
redis 127.0.0.1:6379> keys site_id:1_item_id:3
1) "site_id:1_item_id:3"
Don't forget that Redis KEYS uses GLOB style pattern, which is not exactly like a regex.
You can check out the keys documentation examples to make sure you understand
The correct approach here, is to use an index of keys - maintained by you. Redis should not be queried in any conventional sense.
How i can find keys with multiple match pattern, for example i've keys with
foo:*, event:*, poi:* and article:* patterns.
how i find keys with redis keys function for match with foo:* or poi:* pattern, its like
find all keys with preffix foo:* or poi:*
You should not do this. KEYS is mainly a debug command. It is not supposed to be used for anything else.
Redis is not a database supporting ad-hoc queries: you are supposed to provide access paths for the data you put into Redis (using extra set or hash or zset indexes).
If you really need to run arbitrary boolean expressions on keys to select data, I would suggest to do it offline by using the rdb-redis-tools package.
Is it possible to search for occurrences of a specific value in redis?
It's easy enough to do the same for keys
SET firstname "John"
KEYS f?rstname
["firstname"]
But can one search for all occurrences of "John" or better yet "J*hn" ?
As far as I know there is no such option in Redis. As you mentioned KEYS pattern can be used to search for the keys with specific pattern, but similar functionality on values would result into search among all of the keys/fields/elements which may not be trivial since Redis has advanced data structures like hashes, sets and lists. Time complexity of this operation would be possibly even greater than O(N) which is why also KEYS command shouldn't be used in production environments.