I'd like to use Redis to cache the most recent piece of data that a user has sent to me. However, I can't just use SET, because the user may send data out of order, I need to condition the SET based on the value of another key, e.g.:
latest_timestamp = GET "latest_timestamp:<new_data.user_id>"
if latest_timestamp < new_data.timestamp {
SET "latest_timestamp:<new_data.user_id>" new_data.timestamp
SET "latest_data:<new_data.user_id>" new_data.to_string()
}
What is the idiomatic way to handle this situation?
A server-side Lua script (see EVAL) is the idiomatic-est approach IMO.
Make sure that your code passes the full names (i.e. does all substitutions) of both keys, as well as the new timestamp and the new data as arguments. The script should look something like this:
local lts = tonumber(redis.call('GET', KEYS[1]))
local nts = tonumber(ARGV[1])
if lts < nts then
redis.call('SET', KEYS[1], nts)
redis.call('SET, KEYS[2], ARGV[2])
end
Related
When we use karate.fork for CLI command and need some information from there to be stored in a variable and using it in the next step.
for EX - karate.fork('java -version')
We need to get only the version data alone.
Then karate.fork() is the wrong choice - just use karate.exec() instead. It does the same thing, but will block, and also return the console output:
* def output = karate.exec('java -version')
Please read this also for advanced examples: https://stackoverflow.com/a/62911366/143475
Look at this lua script:
local clientIds = redis.call('ZRANGEBYSCORE', KEYS[1], '-inf', ARGV[1], 'LIMIT', '0', ARGV[2]);
local prefix = 'lock:';
local lockedClientIds = {};
for _, value in ipairs(clientIds)
do
lockal key = prefix .. tostring(value)
if redis.call('EXISTS', key) == 0 then
redis.call('SET', key, 'PX', ARGV[3]);
table.insert(lockedClientIds, value)
end
end
redis.pcall('ZREM', KEYS[1], unpack(lockedClientIds));
return lockedClientIds;
It takes some values from the sorted set and uses them to create keys (after some simple concatenation). I'm not sure if this is OK because according to Redis Lua tutorials, all keys should be provided in the KEYS array so should be known in compile-time, not the runtime.
All Redis commands must be analyzed before execution to determine
which keys the command will operate on. In order for this to be true
for EVAL, keys must be passed explicitly. This is useful in many ways,
but especially to make sure Redis Cluster can forward your request to
the appropriate cluster node. Note this rule is not enforced in order
to provide the user with opportunities to abuse the Redis single
instance configuration, at the cost of writing scripts not compatible
with Redis Cluster.
So does it mean, there is a risk that this will only work with a single node and when redis is distributed across many nodes it won't work?
YES, it is (highly) possible that the script will not work in cluster mode.
It will continue to work even in cluster mode only if the keys are in same hash slot. The idea of hash tags can be used for this purpose.
Note: I'm assuming, by "redis is distributed across many nodes", you are meaning Redis Cluster mode.
I have defined one global variable in a Pre-request Script.
I want to compare this global variable with variable present in the response.
As the warning message says, you're running a very old version of Postman and it's probably the chrome extension.
This is now several major versions behind and the pm.* functionality is not included in that old version of the chrome extension.
Download the native application and start using the newest version of Postman. By not doing this, you're missing out on so many new features.
As #Danny mentioned, it is recommended to update to the latest version.
Now to your question, if you want to compare the global variable with workkard_number present in response, you need to first parse the response and get the workkard_number in it, which you can then compare with your global variable. You could try something like this in your test script:
var jsonData = JSON.parse(responseBody);
var responseWorkkardNumber = jsonData.wokkard_number;
You can retreive the workkard_number in the response like this(assuming that your response is a json with "workkard_number" as a key in it. Then you can compare it as follows:
tests["workkard_numbers are equal"] = responseWorkkardNumber === globals.workkard_number;
or
tests["workkard_numbers are equal"] = responseWorkkardNumber === pm.globals.get("workkard_number");
Also note - "Warning - Environment and global variables will always be stored as strings. If you're storing objects/arrays, be sure to JSON.stringify() them before storing, and JSON.parse() them while retrieving." - https://www.getpostman.com/docs/v6/postman/environments_and_globals/manage_environments
I was attempting to trigger twisted memcache calls from getattribute and return values to my objects. Is this possible ? My thinking was that gatherResults waits for the call to succeed or fail and then returns the results - which it does but the interpreter returns a deferred to whatever is accessing the attribute.
def __getattribute__(self, key):
exempt = ['__providedBy__', '__class__', '__provides__', '__dict__', 'uid']
if key in exempt:
return object.__getattribute__(self, key)
else:
print key
addr = IPv4Address('TCP', '127.0.0.1', 11211)
mc_pool = MemCachePool(addr, maxClients=10)
uid = object.__getattribute__(self, key)
def return_res(res):
return res
deferred_calls = [mc_pool.get(key)]
d = defer.gatherResults(deferred_calls, consumeErrors = True)
d.addCallback(return_res)
Just a heads to anyone who comes across this. This approach doesn't, can't, won't, should never, and will never work. Twisted will not return a value to your blocking code. So if you run into such a problem you need to rethink your approach. Twisted rocks in so many ways - just not this one.
when using SetEntry, it will automatic generate a set with key "ids:+ objectName" in redis db.
For example:
typedClient.SetEntry("famyly:username:jhon",new Family {FatherName="Jhon",...});
a set with key name of "ids:Family" and a member like "2343443" will be automatic created in redis db,
and each time I update or modify the same key with SetEntry, the set of "ids:Family" will increment with an new auto generated member. And this set will grow extremely large if I update the key frequently.
How can I disable the auto generated set? this set seems useless for the current circumstances.
thanks
I ran into this same problem - I discovered that our database contained a couple dozen of these "ids:XXX" sets, each containing tens of millions of items, which were consuming significant amounts of memory.
The solution is to switch to untyped clients. You can still use typed methods on the client so you're really not giving up any type safety or automatic serialization at all. There's a couple ways to create clients; we tend to use the get-in-get-out Exec shortcuts on RedisClientsManager. You should be able to adapt this to the way you do it.
Typed client - creates "ids" sets:
// set:
redis.ExecAs<T>(c => c.SetEntry(key, value));
// get:
T value = redis.ExecAs<T>(c => c.GetValue(key));
Untyped client - no "ids" sets created:
// set:
redis.Exec(c => c.Set(key, value));
// get:
using (var cli = _redis.GetClient())
{
T value = cli.Get<T>(key);
}
The inferred auto-generated id's are when you use the high-level Redis Typed Client. Use the IRedisClient.SetEntry on the string-based RedisClient API instead.