I am using Redis 2.8 key space Pub/Sub notification, I would like to know if it is possible to get notified of which field changed after HSET command?
At the moment I receive the notification for the key as a consequence of the
HSET command, but I would better know which field has been set - I understand I can read again the set to look at differences once I am notified, but I don't find it very efficient.
Standard Redis keyspace notifications do not include data about the data that was changes, and specifically do not include information about the touched Hash field.
While this isn't really what you want, it can still be used as a workaround.
Try to have unique hash key name, for example:
redis.hmset('task:{}'.format(unique_id), status='running')
And when you receive a message, it will look like this:
(b'__keyspace#0__:task:c81b8373-b5ea-4be0-b8f1-b490e7280898', 'hset')
Now, knowing unique task id, you can do:
redis.hget('task:{}'.format(unique_id), 'status')
> running
Related
I am currently looking into using redis to manage a black and whitelist for asterisks to manage spam calls. Since redis is a key value store it is great to check if a phone number is in the db. I can store some additional info in my value part of store and use the phone number as a key. 2 question I have is
A) a phone number could be multiple times in the key as it would be for different users. So since the key has to be unique in db I assume best way would to use a key like user:phone number. Is there a way I could get all records for a given user: in key or can I only get data if I have full key
B) when I store phone number as key, can I use something like user:+2135551212 or will the + in key cause problems ?
To answer your 1st question: No, you don't need full key to fetch all records for a given user. Redis provides wildcard matches which you can use in your case. As example:
127.0.0.1:6379> set john:1234567890 johnOne
OK
127.0.0.1:6379> set john:0987654321 johnTwo
OK
127.0.0.1:6379> keys john*
1) "john:1234567890"
2) "john:0987654321"
127.0.0.1:6379>
I have set two keys both with user 'john' followed by different phone numbers, so when I need all the keys which starts with john, we can use 'keys john*' to get all the keys which starts with 'john'.
To answer your second part, yes you can use '+' in the keys without any problem. Again, as an example:
127.0.0.1:6379> set user:+1234567890 helloagain
OK
I'm currently using Redis and doing an HGET followed by an HDEL to remove the field (and its value) from the hash once I've retrieved the value I need (which only needs to be used once).
Is there any way to do this in one operation (i.e. get a field and remove it from the hash)? I skimmed through the docs and couldn't find the right command.
Transactions will do this. Use the MULTI command to start a transaction, within the transaction HGET what you want, HDEL it, and then EXEC.
You could also write a Lua script to do this.
I am trying to list keys with specific pattern like below:
KEYS "*Team*"
and I am getting resultset with serial number like below:
1) "TeamMetricSummary_google_bps_app_google wfep
league_chambersc2016:04-03-2016_06-04-2016"
2) "\xac\xed\x00\x05t\x00TTeamMetricSummary_google_bps_app_google wfep
league_malini.gto:12-06-2015_04-02-2016"
My problem is that I want to avoid serial number in result set.
Is that possible?
That is not possible. Redis will return the whole key. You can use regex or string operations like split in your application logic to achieve this. For this you must know your input. For example if your key is in a pattern like xTeamNamey. where x and y are some constraints (serial number) you want to avoid, you can insert your key like x:TeamName:y. In retrieval you can use string.split(":")[1] to get the TeamName.
To answer your specific question:
Redis supports Lua scripting. If you're on a version of Redis that is bundled with Lua version 5.0 or above, you Lua script can use regular expressions. Write a Lua script that does redis.call of KEYS command and then does pattern matching to remove serial numbers for you.
Alternative:
By the way, assuming above is part of software that runs in production, here is what I would suggest: Don't use KEYS command on production! As it's documentation says, Redis has to go through all the keys to get keys matching your pattern. Alternatively, you may consider doing shadow writes to a Redis set that's trimmed of serial number every time you add a key. When you need list of keys, you can read the whole set. However, you'll have to manually handle the expiration of keys.
I am trying to insert time based records with multiple fields on the values (with TTL enabled).
For the multiple fields the best way to do it via Redis is using HSET:
HSET user:32 name "johns" timecreated "3333311232" address "somewhere"
I also try to read those values via time range:
for example return all history records (for example user 32) which was inserted in the last day:
so the best for that would be storing via ZADD using scores(this time I am losing the hash-map structure for easy retrieval):
ZADD user:32 3333311232 "name=johns,timecreated=3333311232,address=somewhere"
On the top of the things I want to add TTL for each record
Any idea how I could optimize my design?
I could split into two but that will requires two queries when reading:
ZADD user:32 3333311232 "user:32:3333311232"
HMSET user:32:3333311232 name “johns” timecreated “3333311232” address="somewhere"
than to retrieve ill need:
//some range
ZRANGEBYSCORE user:32 3333311232 333331123
result: 1389772850
now to get all information: HGETALL user:32:1389772850
What do you think?
Thank you,
ray.
The two methods you describe are the two common approaches. If you store the entire object in the ZSET, you would typically store it as a JSON string. If you don't need "random" access to the object, that's a valid approach.
I usually go for the other approach; a ZSET combined with hashes. the two queries are not a big deal. You could even abstract it away with a Lua script; see EVAL.
Regarding the TTL, while you cannot expire individual ZSET values, you could expire the hash, and use keyspace notifications to listen for the expired event, and remove the corresponding value from the ZSET.
Let me know if you need some more specifics.
Im currently having a redis data set with key representing ids and values as a json . I need to add a new entity in the json for every userid(keys). Is there any existing opensource tool? what is the way i should proceed to update for 1M keys of data.
There are a few possibilities:
Here's some pseudo code for doing this with Redis 2.6 Lua scripting.
for userid in users:
EVAL 'local obj = cjson.decode(redis.call("GET", KEY[1])); obj.subobj.newjsonkey = ARGV[1]; redis.call("SET", KEY[1], cjson.encode(obj));' 1 userid "new value!"
Short of that, you may need to stop the service and do this with GETs and SETs since you probably don't have a locking mechanism in place. If you can enforce a lock, see http://redis.io/commands/setnx
There are a few tools for updating an rdb. https://github.com/sripathikrishnan/redis-rdb-tools https://github.com/nrk/redis-rdb
Note, this answer was adapted to my answer to: Working with nested objects in Redis?