I'm trying to store phone call logs in Redis in a manner that I can quickly look them up by the phone number and by the timestamp (which is actually the number of hours since the unix epoch).
So the data is something like this:
Phone # | Hour | Data
------------+--------+-------------------
15551231234 | 386615 | "call record 1..."
15551231234 | 386615 | "call record 2..."
And I need to be able to get all the data for the specified phone number, and the hour within a range (ex: the last 24 hours).
What is the best way in Redis to store and retrieve this form of data?
Redis is not very good solution for arbitrary range queries: you would probably better served by a relational database or something like MongoDB.
Now, because your range queries applies on numerical values, you can use a zset (sorted set) to represent your data.
# adding a new call
ZADD <phone> <hour> <call data>
# find calls for a given phone number in a range
ZRANGEBYSCORE <phone> <hour begin> <hour end> WITHSCORES
Each returned item will be a pair with time stamp and call data.
Please note you cannot find all phone calls within a given range with this data structure (only calls for a given phone number within a range).
Related
How can I store collection with key-value pairs in Redis? For example, I want to log time when user tried to login, to some collection. Every user have id, so I want to use it as a key. But I want to store it separatly from other elements, in separate collection
For each user you can have a sorted set. You can use the user id in the name of the sorted set. Just use 1 as the value since you don't need to store something there and use the timestamp as the score.
zadd 'user:' + uid +':logins' currentTimestamp 1
With this you can run queries to grab how many times a user tried to login during certain periods with zcount etc.
I use awesome Redis sorted sets to score users and, then, quickly get user rating by score. Also, my score has "weight", so that one score can give 5 points to user, and another vote can give 2 points, etc. Now if somebody votes for user, I call
ZINCRBY user:votes <vote_weight> <userId>
but now I need to calculate users ratings for the last week, month, year from the current timestamp (like 'moving window')
What is the best way to do it in Redis?
Your current approach would only work if you're interested in counting all votes from the beginning of time until this instant..
Lets focus on the problem of doing this for today - this could be easily addressed by adding a new sorted, e.g. votes:today, and doing the ZINCR on its elements.
What happens when today becomes tomorrow? Simple - either RENAME the key, e.g. to votes:yesterday, or just use the timestamp to begin with so you'll always be updating today's vote key, i.e. votes:<timestamp day value>
If you use the timestamp approach, after a week you'll end up with 7 keys - on for each day - with scores-per-user. Getting the results for the last week is a simple matter of getting these 7 keys' members and summing up their scores. You could even do it on the fly. The same goes for 1m0, 3mo, 6mo, 12mo and so forth... but.
But, if you want to do it for 12mo (~= 365 keys), you'll need more RAM (for storing these keys in Redis) and it will take longer to complete the aggregation, naturally. You can combat this by combining Redis' key expiry capabilities (e.g. set the TTL of a day's key to 12mo to keep only one year of history) and keeping running aggregates that are updated either on the fly (i.e. with every vote) or periodically (daily, weekly, monthly, etc...). Note that the same scripts can do housekeeping and delete/archive old data, potentially solving the need to expire it explicitly.
I use sorted sets type in Redis. Each user adds an event to Redis with a score - current timestamp and key: User:ID_USER
How I can get 10 latest events from redis store sorted by score(timestamp)?
For this there is a function ZREVRANGEBYSCORE, but if use it, I get a problem, because not every user have events at last time score.
Edited:
For example, I want to merge some sorted keys by score(timestamp) and selecting the latest data for last 4 hours for each users.
Problem is, that if user did not post data in Redis in this interval(4 hours) then does not show nothing for this user. But I need select latest data sorted by timestamp from each user. Maybe use any function?
I'm new to Redis and I'm not sure what is the best data structure to use in this situation.
Each user will send a heartbeat (every 5 seconds) to say they are still online.
The data stored with be the userID and the date/time.
Previously in Memcached I would just store a string for each user - say userid-active with datatime as the value.
But I know I will need to query all the active users data at the same time and wanted to know what would be the best method to store this. (maybe an array structure).
Any advise would be very welcome.
thankyou
If you use a sorted set you could then fetch a time interval of active users. Latest hour, day and so on.
ZADD active_users [user-id] [timestamp]
To fetch users active within an hour:
ZREVRANGEBYSCORE active_users +inf [currtime]-3600
I'm implementing a php page that display data in pagination format. My problem is that these data change in real time, so when user request next page, I submit last id and query is executed for retrieve more 10 rows from last id order by a column that value change in real time. For example I have 20 rows:
Id 1 col_real_time 5
Id 2 col_real_time 3
Id 3 col_real_time 11
Etc
I get data sorted by col_real_time in ascending order, so result is
id 2, id 1, id 3
Now in realtime id 2 change in col_real_time 29 before user send request for next page, user now send request for next results and because id 2 now is 29 he already see it.
How can I do?
Now in realtime id 2 change
You basically have to take a snapshot of the data if you don't want the data to appear to change to the user. This isn't something that you can do very efficiently in SQL, so I'd recommend downloading the entire result set into a PHP session variable that can be persisted across pages. That way, you can just get rows on demand. There are Javascript widgets that will effectively do the same thing, but will send the entire result set to the client which is a bad idea if you have a lot of data.
This is not as easy to do as pure SQL pagination, as you will have to take responsibility for cleaning the stored var out when it's no longer needed. Otherwise, you'll rather quickly run out of memory.
If you have just a few pages, you could:
Save it to session and page over it, instead of going back to the
database server.
Save it to a JSON object list and use Jquery to read it and page
over it.
Save it to a temp table indicating generation timestamp, user_id and
session_id, and page over it.