My goal is to iterate the whole sorted set as fast as possible without blocking other operations. I use ZSCAN command for that and here is my question:
Is it possible to remove scores from ZSCAN output?
Currently, each 2nd output line is a score and it's necessary to filter it out in my Lua script because I need only keys.
Example:
zscan v_ids 0
1) "69632"
2) 1) "88046"
2) "0.00024987506246876561"
3) "47792"
4) "0.00024987506246876561"
5) "91362"
6) "0.00024987506246876561"
...
P.S. I already reviewed all possible ZSCAN parameters and tried to use other versions of the SCAN command.
It seems there is no such option for ZSCAN. You have to switch to other commands like ZRANGE, ZRANGEBYSCORE, etc. They could limit the returned results by score range or by index, but they might not have the incremental mode and performance as ZSCAN.
Related
I have a key in Redis:
travelpayouts-special-offers---origin=MOW-destination=MCX-locale=en
Is that possible to find all such keys where locale=en considering it can be virtually anywhere in the key string? I.e. pattern should be able to find all of those keys:
travelpayouts-special-offers---origin=MOW-destination=MCX-locale=en
travelpayouts-special-offers---origin=MOW-locale=en-destination=MCX
travelpayouts-special-offers---locale=en-origin=MOW-destination=MCX
Yes, you can use *locale=en* as the pattern to feed SCAN / KEYS with.
> SCAN 0 MATCH *locale=en*
1) "0"
2) 1) "travelpayouts-special-offers---origin=MOW-destination=MCX-locale=en"
2) "travelpayouts-special-offers---locale=en-origin=MOW-destination=MCX"
3) "travelpayouts-special-offers---origin=MOW-locale=en-destination=MCX"
As we may know, good naming convention for Redis keys is "object-type:id"
How to get all key namespaces (symbols before :)?
E.g.
127.0.0.1:6379> KEYS *
1) "bar:1"
2) "baz:1"
3) "baz:3"
4) "foo:1"
5) "foo:2"
6) "baz:2"
I want a command like
127.0.0.1:6379> SCAN 0 MATCH "(^\w+):" COUNT 10
1) "6"
2) "foo"
3) "bar"
4) "baz"
But seems like MATCH option of SCAN command does not support PCRE pattern syntax.
In real, I got
127.0.0.1:6379> SCAN 0 MATCH "(^\w+):" COUNT 10
1) "6"
2) (empty list or set)
Any ideas ?
You could create an index key and add each namespace to it every time you add a new key:
SADD "namespaces" "foo". But there would be problem with deleting them, as you'd have to check if the deleted object was the last of it's type.
The second option would be creating a module (if you're using Redis >= 4.0) with a command that you could use to add and remove said keys, that would have it's own type in which it would store object-type counters (so it wouldn't have to call KEYS * which is quite slow). The problem here would be if you had key expiration, or oom key deletion configured. The counter would be incorrect once a key gets deleted by Redis itself.
The third option is to create a module with one command that would call KEYS *, then get the first part of the key and add it to a hash map/dictionary (using RedisModuleDict) and then return it using RedisModule_ReplyWithString(ctx, str) in a loop
I think that you could also write it in Lua instead of creating a module, but a module written in C is probably going to be faster... and I don't know Lua so I won't be able to help you there.
You could always write a simple script in bash that would get the object-type from each key and SADD it to a temporary key then read it, print and delete afterwards, but that's the same as the third option but slower and has to use a temporary key which might be used by some other script or something.
I'm new to Aerospike and am probably missing something fundamental, but I'm trying to see an enumeration of the Keys in a Set (I'm purposefully avoiding the word "list" because it's a datatype).
For example,
To see all the Namespaces, the docs say to use SHOW NAMESPACES
To see all the Sets, we can use SHOW SETS
If I want to see all the unique Keys in a Set ... what command can I use?
It seems like one can use client.scan() ... but that seems like a super heavy way to get just the key (since it fetches all the bin data as well).
Any recommendations are appreciated! As of right now, I'm thinking of inserting (deleting) into (from) a meta-record.
Thank you #pgupta for pointing me in the right direction.
This actually has two parts:
In order to retrieve original keys from the server, one must -- during put() calls -- set policy to save the key value server-side (otherwise, it seems only a digest/hash is stored?).
Here's an example in Python:
aerospike_client.put(key, {'bin': 'value'}, policy={'key': aerospike.POLICY_KEY_SEND})
Then (modified Aerospike's own documentation), you perform a scan and set the policy to not return the bin data. From this, you can extract the keys:
Example:
keys = []
scan = client.scan('namespace', 'set')
scan_opts = { 'concurrent': True, 'nobins': True, 'priority': aerospike.SCAN_PRIORITY_MEDIUM }
for x in (scan.results(policy=scan_opts)): keys.append(x[0][2])
The need to iterate over the result still seems a little clunky to me; I still think that using a 'master-key' Record to store a list of all the other keys will be more performant, in my case -- in this way, I can simply make one get() call to the Aerospike server to retrieve the list.
You can choose not bring the data back by setting includeBinData in ScanPolicy to false.
I'm using jedis, and wish to get the result of a key and modify it and then store it back,
String dataToModify = jedis.get(parent_id);
//some modify eg:
modifiedData = dataToModify + "modify";
jedis.set(parent_id, modifiedData);
but I then realized that if someone else modified data in key parent_id, between this get and set, there will be collision. the watch-multi-exec does not work since I need to get the value of key parent_id during the transaction. Is there any other way to do this atomically? thanks
Not sure regarding the exact jedis syntax, but Redis' WATCH/MULTI/EXEC will let you do that (I.e. run the get/set flow atomically). Alternatively, if you implement your workflow in a server-side Lua script, that would also guarantee atomicity.
How about saving the date in epoch (also known as unix time) value, this way you can treat your date as a number and use INCRBY command e.g.:
jedis.incrby(parent_id, modify);
I tried adding some sample score-value pairs to redis sorted set using below code:
String key = "set";
redis.zadd(key, 5, "1034");
redis.zadd(key, 2, "1030");
redis.zadd(key, 1, "1089");
and tried retrieving it using byteArray and BitSet
byte[] byteArr = redis.get(key.getBytes());
BitSet bitSet = fromByteArrayReverse(byteArr);
System.out.println(bitset.toString()));
also i tried executing
System.out.println(redis.get(key.getBytes()));
which is supposed to give me an address of the byte-array
But for both of these commands i get the error
" ERR Operation against a key holding the wrong kind of value"
So can anyone please tell me why does this error occur in the first place and also the correct redis command/code to retrieve values from a redis sorted-set??
What you want is calling
ZSCORE key "1034"
Or in the case of wanting only elements between two particular scores
ZRANGEBYSCORE key lower upper
Since you also have "rank" (position or index, as in a list) you can also ask for example for the first three elements in your set
ZRANGEBYRANK key 0 2
The error you are getting is because once you assign a value to a key, that value defines the type of the internal structure on redis, and you can only use commands for that particular structure (or generic key commands such as DEL and so on). In your case you are trying to mix sorted sets with byte operations and it doesn't match.
To see all sorted set commands, please refer to http://redis.io/commands#sorted_set