I'm new to Redis and I'm not sure what is the best data structure to use in this situation.
Each user will send a heartbeat (every 5 seconds) to say they are still online.
The data stored with be the userID and the date/time.
Previously in Memcached I would just store a string for each user - say userid-active with datatime as the value.
But I know I will need to query all the active users data at the same time and wanted to know what would be the best method to store this. (maybe an array structure).
Any advise would be very welcome.
thankyou
If you use a sorted set you could then fetch a time interval of active users. Latest hour, day and so on.
ZADD active_users [user-id] [timestamp]
To fetch users active within an hour:
ZREVRANGEBYSCORE active_users +inf [currtime]-3600
Related
I am building an event reminder page where people can set a reminder for certain events. There is an option for the user to set the amount of time before they need to be notified. It is stored in notification_time and notification_unit. notification_time keeps track of the time before they want to be notified and notification_unit keeps track of the PHP date format in which they selected the time, eg. i for minutes, H for hours.
Eg. notification_time - 2 and notification_unit - H means they need to be notified 2 hours before.
I have Cron jobs running in the background for handling the notification. This function is being hit once every minute.
Reminder::where(function ($query) {
$query->where('event_time', '>=', now()->subMinutes(Carbon::createFromFormat('i', 60)->diffInMinutes() - 1)->format('H:i:s'));
$query->where('event_time', '<=', now()->subMinutes(Carbon::createFromFormat('i', 60)->diffInMinutes())->format('H:i:s'));
})
In this function, I am hard coding the 'i', 60 while it should be fetched from the database. event_time is also part of the same table
The table looks something like this -
id event_time ... notification_unit notification_time created_at updated_at
Is there any way to solve this issue? Is it possible to do the same logic with SQL instead?
A direct answer to this question is not possible. I found 2 ways to resolve my issue.
First solution
Mysql has DATEDIFF and DATE_SUB to get timestamp difference and subtract certain intervals from a timestamp. In my case, the function runs every minute. To use them, I have to refactor my database to store the time and unit in seconds in the database. Then do the calculation. I chose not to use this way because both operations are a bit heavy on the server-side since I am running the function every minute.
Second Solution
This is the solution that I personally did in my case. Here I did the calculations while storing it in the database. Meaning? Let me explain. I created a new table notification_settings which is linked to the reminder (one-one relation). The table looks like this
id, unit, time, notify_at, repeating, created_at, updated_at
The unit and time columns are only used while displaying the reminder. What I did is, I calculated when to be notified in the notify_at column. So in the event scheduler, I need to check for the reminders at present (since I am running it every minute). The repeating column is there to keep track of whether the reminder is repeating or not. If it is repeating I re-calculate the notify_at column at the time of scheduling. Once the user is notified notify_at is set to null.
I use awesome Redis sorted sets to score users and, then, quickly get user rating by score. Also, my score has "weight", so that one score can give 5 points to user, and another vote can give 2 points, etc. Now if somebody votes for user, I call
ZINCRBY user:votes <vote_weight> <userId>
but now I need to calculate users ratings for the last week, month, year from the current timestamp (like 'moving window')
What is the best way to do it in Redis?
Your current approach would only work if you're interested in counting all votes from the beginning of time until this instant..
Lets focus on the problem of doing this for today - this could be easily addressed by adding a new sorted, e.g. votes:today, and doing the ZINCR on its elements.
What happens when today becomes tomorrow? Simple - either RENAME the key, e.g. to votes:yesterday, or just use the timestamp to begin with so you'll always be updating today's vote key, i.e. votes:<timestamp day value>
If you use the timestamp approach, after a week you'll end up with 7 keys - on for each day - with scores-per-user. Getting the results for the last week is a simple matter of getting these 7 keys' members and summing up their scores. You could even do it on the fly. The same goes for 1m0, 3mo, 6mo, 12mo and so forth... but.
But, if you want to do it for 12mo (~= 365 keys), you'll need more RAM (for storing these keys in Redis) and it will take longer to complete the aggregation, naturally. You can combat this by combining Redis' key expiry capabilities (e.g. set the TTL of a day's key to 12mo to keep only one year of history) and keeping running aggregates that are updated either on the fly (i.e. with every vote) or periodically (daily, weekly, monthly, etc...). Note that the same scripts can do housekeeping and delete/archive old data, potentially solving the need to expire it explicitly.
I use sorted sets type in Redis. Each user adds an event to Redis with a score - current timestamp and key: User:ID_USER
How I can get 10 latest events from redis store sorted by score(timestamp)?
For this there is a function ZREVRANGEBYSCORE, but if use it, I get a problem, because not every user have events at last time score.
Edited:
For example, I want to merge some sorted keys by score(timestamp) and selecting the latest data for last 4 hours for each users.
Problem is, that if user did not post data in Redis in this interval(4 hours) then does not show nothing for this user. But I need select latest data sorted by timestamp from each user. Maybe use any function?
I'm trying to store phone call logs in Redis in a manner that I can quickly look them up by the phone number and by the timestamp (which is actually the number of hours since the unix epoch).
So the data is something like this:
Phone # | Hour | Data
------------+--------+-------------------
15551231234 | 386615 | "call record 1..."
15551231234 | 386615 | "call record 2..."
And I need to be able to get all the data for the specified phone number, and the hour within a range (ex: the last 24 hours).
What is the best way in Redis to store and retrieve this form of data?
Redis is not very good solution for arbitrary range queries: you would probably better served by a relational database or something like MongoDB.
Now, because your range queries applies on numerical values, you can use a zset (sorted set) to represent your data.
# adding a new call
ZADD <phone> <hour> <call data>
# find calls for a given phone number in a range
ZRANGEBYSCORE <phone> <hour begin> <hour end> WITHSCORES
Each returned item will be a pair with time stamp and call data.
Please note you cannot find all phone calls within a given range with this data structure (only calls for a given phone number within a range).
I was told redis was born for analytic, and I came across some bitmap using cases. They are useful when counting based on yes/no(0/1), but I can't find an efficient way to count the number of user who login at least 4 times during the last 10 days. Because redis runs in memory, I tried using bit map to keep track login flag of each user, and using bitcount to filer, on my laptop, it took a minute to return the count from about 4Million users' login activity.
Is there any way to solve this problem? I guess the round trips between my node redis client and redis server may be the issue, I'll try batch command or lua script to see if it works.
I think you need to use SortedSets with user id in value, and timestamp in score.
When user logs in, score (time stamp) for this user updates to current. Than you can get ether N last logged in users (ZREVRANGE), or users, logged in between some datetime range (ZRANGEBYSCORE)