I have a use case where I'm aggregating until a TTL hits 0 on the key/value in Redis. As far as I can tell in the documentation, retrieval or a background job triggers all expired keys to be deleted immediately.
Is there anyway I can 'halt' that deletion and retrieve the value at the time of expiry? Or something similar to that effect?
My last question contains some context to my use case: Redis - any way to trigger an event when a value is no longer being actively written to?
I believe I found an answer to my question. Every row would have a corresponding row with with the convention expiry:{key}.
uniqueEventHash: [value1, value2, value3] // no expiry
expiry:uniqueEventHash: {no value} // set TTL to 60
Now, whenever a new value for that uniqueEventHash arrives, I do two things. I append it onto the uniqueEventHash row with an append, and then I also subsequently reset the TTL on expiry:uniqueEventHash to 60.
When events for that uniqueEventHash stop arriving, the second expiry:expiry:uniqueEventHash is deleted and a notification is sent out to a subscriber to EXPIRY events. In the message is the key that expired, which in this case is expiry:uniqueEventHash.
I can then do the following:
// pseudo code
onExpiryEvent(message):
[type, key] = eventMessage.split(':');
aggregatedValue = await getByKey(key);
del(key);
Related
If I am using redis to check whether a unique message has been handled historically or is currently being handled (for preventing Pub/Sub duplicate message handling), and I am not expecting to read this kv pair ever again, should I bother deleting the kv pair at the end of the message handler, or just let the LRU eviction eventually delete it? The processing of the message would take far longer than the delay between duplicate messages.
For context, this is some JS pseudocode of what the processing looks like:
// Message comes in
const messageHandler = (message) => {
const duplicate = checkMessageInRedis(message.ID)
if (duplicate) return
registerMessageInRedis(message.ID)
// ... do some stuff to the message
deleteMessageInRedis(message.ID) // DO I BOTHER WITH THIS??? OR LET EVICTION DELETE IT?
}
I guess the question becomes: Is an eviction delete more resource intensive than a DEL transaction? And if so by how much?
Best approach would be to provide a TTL (time to live) when adding the message to Redis.
i.e. registerMessageInRedis(message.ID, ttl)
This will auto-delete the message after ttl expires and will save from an additional network round trip cost for explicit delete.
I've found a possible answer to this question in a Google Group but I'll like to know if it's correct and add a follow-up question if it is correct.
The answer there is
Every time the global update function in your app runs for any reason,
the global subscriptions object is reevaluated as well, and effect
managers receive the new list of current subscriptions
If any time the model is changed subscriptions get called what is the effect on subscriptions such as Time.every second taken from Time Effect Elm Guide - is that means the timer get reset when the model changes? What if that was Time.every minute - if the model changes 20 seconds after it starts will it fire in 60 - 20 = 40 seconds or will it fire in 1 minute?
You can check when update and subscriptions are called by adding a Debug.log statement to each. The subscriptions function is called first at the start (since the messages which will be sent to update may depend on it) and also after each call to update.
The timer interval seems to be unaffected by subsequent calls to subscriptions. For example, if you use the elm clock example, change the subscription to
Time.every (10*Time.second) Tick
and add a button to the view which resets the model value to 0, you will see that the tick still takes place at regular 10s intervals, no matter when you click the button.
TLDR; It will fire in 1 minute, unless you turn your subscription
off and on during the first minute
Every time your update runs, the subscriptions function will run too.
The subscriptions function essentially is a list of things you want your app to subscribe to.
In the example you have a subscription that generates a Tick message every 60 seconds.
The behavior you may expect is:
T= 0s: The first time subscriptions runs, you start your subscription to "receive Tick message every 60 seconds".
T= between 0 AND 60s: As long as that particular subscription remains ON, it doesn't matter how often your update function runs. subscriptions will be run, but as long as your particular subscription to the Tick remains ON, things are fine.
T= 60s: You receive a Tick message from your subscription, which in turn will fire update to be called.
T= 60s: subscriptions will run again (because of previous call to update)
What could be interesting is what happens if the subscription to Tick is canceled along the way and then reinstated:
T= 0: subscription to Tick
T= 20s: suppose something changes in the model, causing subscription to Tick to be canceled
T= 40s: some other change in model, causing subscription to Tick to be turned on again
T= 100s: Tick message is fired, and passed to update function
T= 100s: subscriptions will run again
I have a telegram bot that for any received message runs a program in the server and sends its result back. But there is a problem! If a user sends too many messages to my bot(spamming), it will make server so busy!
Is there any way to block the people whom send more than 5 messages in a second and don't receive their messages anymore? (using telegram api!!)
Firstly I have to say that Telegram Bot API does not have such a capability itself, Therefore you will need to implement it on your own and all you need to do is:
Count the number of the messages that a user sends within a second which won't be so easy without having a database. But if you have a database with a table called Black_List and save all the messages with their sent-time in another table, you'll be able to count the number of messages sent via one specific ChatID in a pre-defined time period(In your case; 1 second) and check if the count is bigger than 5 or not, if the answer was YES you can insert that ChatID to the Black_List table.
Every time the bot receives a message it must run a database query to see that the sender's chatID exists in the Black_List table or not. If it exists it should continue its own job and ignore the message(Or even it can send an alert to the user saying: "You're blocked." which I think can be time consuming).
Note that as I know the current telegram bot API doesn't have the feature to stop receiving messages but as I mentioned above you can ignore the messages from spammers.
In order to save time, You should avoid making a database connection
every time the bot receives an update(message), instead you can load
the ChatIDs that exist in the Black_List to a DataSet and update the
DataSet right after the insertion of a new spammer ChatID to the
Black_List table. This way the number of the queries will reduce
noticeably.
I have achieved it by this mean:
# Using the ttlcache to set a time-limited dict. you can adjust the ttl.
ttl_cache = cachetools.TTLCache(maxsize=128, ttl=60)
def check_user_msg_frequency(message):
print(ttl_cache)
msg_cnt = ttl_cache[message.from_user.id]
if msg_cnt > 3:
now = datetime.now()
until = now + timedelta(seconds=60*10)
bot.restrict_chat_member(message.chat.id, message.from_user.id, until_date=until)
def set_user_msg_frequency(message):
if not ttl_cache.get(message.from_user.id):
ttl_cache[message.from_user.id] = 1
else:
ttl_cache[message.from_user.id] += 1
With these to functions above, you can record how many messages sent by any user in the period. If a user's messages sent more than expected, he would be restricted.
Then, every handler you called should call these two functions:
#bot.message_handler(commands=['start', 'help'])
def handle_start_help(message):
set_user_msg_frequency(message)
check_user_msg_frequency(message)
I'm using pyTelegramBotAPI this module to handle.
I know I'm late to the party, but here is another simple solution that doesn't use a Db:
Create a ConversationState class to attach to each telegram Id when they start to chat with the bot
Then add a LastMessage DateTime variable to the ConversationState class
Now every time you receive a message check if enought time has passed from the LasteMessage DateTime, if not enought time has passed answer with a warning message.
You can also implement a timer that deletes the conversation state class if you are worried about performance.
Good day
I'm trying to perform load testing with LoadRunner 11. Here's an issue:
I've got automatically generated script after actions recording
Need to catch Session ID. I do it with web_reg_save_param() in the next way:
web_reg_save_param("S_ID",
"LB=Set-Cookie: JSESSIONID=",
"RB=; Path=/app/;",
LAST);
web_add_cookie("S_ID; DOMAIN={host}");
I catch ID from the response (Tree View):
D2B6F5B05A1366C395F8E86D8212F324
Compare it with Replay Log and see:
"S_ID = 75C78912AE78D26BDBDE73EBD9ADB510".
Compare 2 IDs above with the next request ID and see 3rd ID (Tree View):
80FE367101229FA34EB6429F4822E595
Why do I have 3 different IDs?
Let me know if I have to provide extra information.
You should Use(Search=All) below Code. Provided your Right and left boundary is correct:
web_reg_save_param("S_ID",
"LB=Set-Cookie: JSESSIONID=",
"RB=; Path=/app/;",
"Search=All",
LAST);
web_add_cookie("{S_ID}; DOMAIN={host}");
For Details refer HP Mannual for web_reg_save_param function.
I do not see what the conflict or controversy is here. Yes, items related to state or session will definitely change from user to user, one recording session to the next. They may even change from one request to the next. You may need to record several times to identify the change and use pattern for when you need to collect and when you need to reuse the collected data from a response in a subsequent request.
Take a listen to this podcast. It should help
http://www.perfbytes.com/dynamic-data-correlation
I want to implement Absolute and Sliding Caching In Redis. Does anyone have any resource link then it will be helpful
Redis already have many commands for this :
EXPIRE : Set a timeout on key.
EXPIREAT : Same as previous but takes an absolute Unix timestamp (seconds since January 1, 1970).
TTL : Returns the remaining time to live of a key that has a timeout
One important thing you have to know about Expiration on Redis : the timeout value is cleared only when the key is removed or overwritten using SET or GETSET. All others commands (INCR, LPUSH, HMSET, ...) will never change the initial timeout.
Absolute expiration is a native feature of Redis using EXPIRE. To implement a sliding expiration you simply need to reset to timeout value after each command.
A basic way to do this could be
MULTI
GET MYKEY
EXPIRE MYKEY 60
EXEC