I know you can get the idletime of a key in redis to get the rough amount of time since an operation has been performed on that key.
I want to do the same thing for a key in a set.
So I have:
127.0.0.1:6379> smembers celery_tasks
1) "c997f588daa54c07a9bf7c764f08714cEnter Account Number..."
I can get the idletime of celery_tasks, but I want to get the idletime of "c997f588daa54c07a9bf7c764f08714cEnter Account Number..."
Is there a way to do this? Does redis support this behavior?
Thanks!
Nope - the elements in lists and members in sorted and regular sets have no meta data of their own, only key-level idletime is accounted.
Related
I am currently looking into using redis to manage a black and whitelist for asterisks to manage spam calls. Since redis is a key value store it is great to check if a phone number is in the db. I can store some additional info in my value part of store and use the phone number as a key. 2 question I have is
A) a phone number could be multiple times in the key as it would be for different users. So since the key has to be unique in db I assume best way would to use a key like user:phone number. Is there a way I could get all records for a given user: in key or can I only get data if I have full key
B) when I store phone number as key, can I use something like user:+2135551212 or will the + in key cause problems ?
To answer your 1st question: No, you don't need full key to fetch all records for a given user. Redis provides wildcard matches which you can use in your case. As example:
127.0.0.1:6379> set john:1234567890 johnOne
OK
127.0.0.1:6379> set john:0987654321 johnTwo
OK
127.0.0.1:6379> keys john*
1) "john:1234567890"
2) "john:0987654321"
127.0.0.1:6379>
I have set two keys both with user 'john' followed by different phone numbers, so when I need all the keys which starts with john, we can use 'keys john*' to get all the keys which starts with 'john'.
To answer your second part, yes you can use '+' in the keys without any problem. Again, as an example:
127.0.0.1:6379> set user:+1234567890 helloagain
OK
Bussiness Objective
I'm creating a dashboard that will depend on some time-series and I'll use Redis to implement it. I'm new to using Redis and I'm trying to use Redis-Streams to count the elements in a stream.
XADD conversation:9:chat_messages * id 2583 user_type Bot
XADD conversation:9:chat_messages * id 732016 user_type User
XADD conversation:9:chat_messages * id 732017 user_type Staff
XRANGE conversation:9:chat_messages - +
I'm aware that I can get the total count of the elements using the XLEN command like this:
XLEN conversation:9:chat_messages
but I want to also know the elements in a period, for example:
XLEN conversation:9:chat_messages 1579551316273 1579551321872
I know I can use LUA to count those elements but I want some REALLY fast way to achieve this and I know that using Redis markup will be the fastest way.
Is there any way to achieve this with a straight forward Redis command? Or do I have to write a Lua script to do this?
Additional information
I'm limited by AWS' ElastiCache to use the only Redis 5.0.6, I cannot install other modules such as the RedisTimeSeries module. I'd like to use that module but it's not possible at the moment.
While the Redis Stream data structure doesn't support this, you can use a Sorted Set alongside it for keeping track of message ranges.
Basically, for each message ID you get from XADD - e.g. "1579551316273-0" - you need to do a ZADD conversation:9:ids 0 1579551316273-0. Then, you can use ZLEXCOUNT to get the "length" of a range.
Sorry, there is no commands-way to achieve this.
Your best option with Redis Streams would be to use a Lua script. You will get O(N) with N being the number of elements being counted, instead of O(log N) if a command existed.
local T = redis.call('XRANGE', KEYS[1], ARGV[1], ARGV[2])
local count = 0
for _ in pairs(T) do count = count + 1 end
return count
Note the difference between O(N) and O(log(N)) is significant for a large N, but for a chat application, if tracked by conversation, this won't make that big of a difference if chats have hundreds or even thousands of entries, once you account total command time including Round Trip Time which takes most of the time. The Lua script above removes network-payload and client-processing time.
You can switch to sorted sets if you really want O(log N) and you don't need consumer groups and other stream features. See How to store in Redis sorted set with server-side timestamp as score? if you want to use Redis server timestamp atomically.
Then you can use ZCOUNT which is O(log(N)).
If you do need Stream features, then you would need to keep the sorted set as a secondary index.
There are proposals for sorted set item expiration in Redis (see https://groups.google.com/d/msg/redis-db/rXXMCLNkNSs/Bcbd5Ae12qQJ and https://quickleft.com/blog/how-to-create-and-expire-list-items-in-redis/), I tried the worker approach to expire geospatial indexes with ZREMRANGEBYSCORE and ZREMRANGEBYRANK commands unsuccessfully (nothing removed).
I succeded using ZREMRANGEBYLEX.
Is there a way to work with geospatial items score other than Strings?
Update:
For example, if time to live(ttl) of an item is 30sec, I add it as:
geoadd 1 -8.616021 41.154503 30
Now, suppose worker executes after 40sec, I was expecting that
zremrangebyscore 1 0 40
would do the job, but it does not,
ZREMRANGEBYLEX 1 [0 [40
does it. Why is this behavior? That means the score of a geospatial item supports only lexicographical operations?
Sorted Sets have elements (strings), and every element has a score (floating-point). Geosets use the score to encode a coordinate.
Redis doesn't expire members in a Sorted Set (or a Geoset). You have to remove them yourself if that is required.
In your case, you'll need to keep two Sorted Sets - one as your GeoSet and one for managing TTLs as scores.
For example, assuming your member is called 'foo', to add it:
ZADD ttls 30 foo
ZADD elems -8.616021 41.154503 foo
To manually expire, first find the members with a call to ZRANGEBYSCORE ttls, and then remove them from both Sets.
Tip: it is preferable to use a timestamp as score instead of seconds.
In Redis Cache, is it possible to retrieve the original TimeOut that was set on a key? I know that there is a way to retrieve the pending TimeToLive of any key but I want the original value that was set while creating the key.
No, Redis doesn't store the original TTL for keys. It would be interesting to understand the use case that requires this.
You could, however, use a Sorted Set to keep track of the initial TTLs. The idea is that after each call to EXPIRE, call ZADD on that set with the member being the key's name. The score should be a decimal, where the part before the decimal point is the expiration timestamp and the fractional part is the TTL (padded with 0s according to your max TTL).
To retrieve the initial TTL, call ZSCORE on the set with the key's name and extract the part after the decimal point.
Note that by taking this approach you'll have to do some housekeeping, namely removing expired members from the set. To do that, periodically call ZREMBYSCORE from -inf to the current timestamp.
I am trying to insert time based records with multiple fields on the values (with TTL enabled).
For the multiple fields the best way to do it via Redis is using HSET:
HSET user:32 name "johns" timecreated "3333311232" address "somewhere"
I also try to read those values via time range:
for example return all history records (for example user 32) which was inserted in the last day:
so the best for that would be storing via ZADD using scores(this time I am losing the hash-map structure for easy retrieval):
ZADD user:32 3333311232 "name=johns,timecreated=3333311232,address=somewhere"
On the top of the things I want to add TTL for each record
Any idea how I could optimize my design?
I could split into two but that will requires two queries when reading:
ZADD user:32 3333311232 "user:32:3333311232"
HMSET user:32:3333311232 name “johns” timecreated “3333311232” address="somewhere"
than to retrieve ill need:
//some range
ZRANGEBYSCORE user:32 3333311232 333331123
result: 1389772850
now to get all information: HGETALL user:32:1389772850
What do you think?
Thank you,
ray.
The two methods you describe are the two common approaches. If you store the entire object in the ZSET, you would typically store it as a JSON string. If you don't need "random" access to the object, that's a valid approach.
I usually go for the other approach; a ZSET combined with hashes. the two queries are not a big deal. You could even abstract it away with a Lua script; see EVAL.
Regarding the TTL, while you cannot expire individual ZSET values, you could expire the hash, and use keyspace notifications to listen for the expired event, and remove the corresponding value from the ZSET.
Let me know if you need some more specifics.