Is following operation atomic? Cause redis calls it subcommands
BITFIELD key1 GET u8 0 INCRBY u8 0 10 OVERFLOW WRAP
Yes, it is atomic. Every data-path command in Redis is atomic.
A sub-command in Redis simply means a command with an additional 'operation' parameter (e.g., BITFIELD key - where the subcommand can be GET, SET, or INCRBY or CONFIG with GET, HELP, RESETSTAT,REWRITE and SET).
Specifically for BITFIELD, a single command can process multiple operations (each specified by a sub-command). All these operations are processed sequentially, and the whole command is atomic.
Related
I am trying to use redis for a simple process.
Process :-
Read value for a key and update it.
Example :- (a,1)
Read a value and update 1 to 2.
The problem here is that in multithreaded environment ,multiple thread (say 4) read at same time and then update it to 2, actually it should have been 4 .Is there a way where in I can impose locking in redis such that if one thread reads a value it imposes a lock so that other threads are kept in waiting state?
Use INCRBY to do an atomic increment:
redis> SET mykey "10"
"OK"
redis> INCRBY mykey 5
(integer) 15
redis>
Or, if you need something more sophisticated, use an uninterruptible MULTI command.
you do not need the lock. In multithreaded environment, suppose 4, they send
read and update commands to Redis not in right order.
may be:
1:read->2:read->1:update->2:update->3:read->3:update->4:read->4:update
you need combine 1:read and 1:update to an atomic operation, that is Lua Script.
you can write a Lua script for all operation and execute it whit EVAL command, the multithreaded commands will in right order.
I want to increment a redis counter but I want to start counting not from zero but from a defined starting number (for example -5).
I know how this can be achieved via SET/INCR inside a Lua script but I was wondering if I can achieve it only with INCR command. Something similar we define for INCRBY where the increment is defined, can we define the starting point?
Lua is perfectly fine for this procedure, but you can also do it with a transaction:
MULTI
SET counter -5 NX
INCR counter
EXEC
The INCR will run every time, so if you want your first call to set it to -5 you should change the SET value to -6. You can also pipe those 4 commands to avoid the 4 RTTs of sending the commands.
You can't do it with the INCR command alone. I would inspect the value of SETNX and if it returns 0 (meaning the key existed), then increment it.
Notice that if you are talking about non expiring counters, you can achieve atomicity this way without Lua, at the price of two roundtrips: If the key did not exist, we create it, set it to the initial value and that's it, one roundtrip, atomic. If it did exist, we increment it, but we are still consistent (unless the key expired or was deleted between the two calls).
However, there is no reason not to use a Lua script for this, it's the preferred way to do this stuff.
I have following scenario:
Fetch array of numbers (from REDIS) conditionally
For each number do some async stuff (fetch something from DB based on number)
For each thing in result set from DB do another async stuff
Periodically repeat 1. 2. 3. because new numbers will be constantly added to REDIS structure.Those numbers represent unix timestamp in milliseconds so out of the box those numbers will always be sorted in time of addition
Conditionally means fetch those unix timestamp from REDIS that are less or equal to current unix timestamp in milliseconds(Date.now())
Question is what REDIS data type fit the most for this use case having in mind that this code will be scaled up to N instances, so N instances will share access to single REDIS instance. To equally share the load each instance will read for example first(oldest) 5 numbers from REDIS. Numbers are unique (adding same number should fail silently) so REDIS SET seems like a good choice but reading M first elements from REDIS set seems impossible.
To prevent two different instance of the code to read same numbers REDIS read operation should be atomic, it should read the numbers and delete them. If any async operation fail on specific number (steps 2. and 3.), numbers should be added again to REDIS to be handled again. They should be re-added back to the head not to the end to be handled again as soon as possible. As far as i know SADD would push it to the tail.
SMEMBERS key would read everything, it looks like a hammer to me. I would need to include some application logic to get first five than to check what is less or equal to Date.now() and then to delete those and to wrap somehow everything in single transaction. Besides that set cardinality can be huge.
SSCAN sounds interesting but i don't have any clue how it works in "scaled" environment like described above. Besides that, per REDIS docs: The SCAN family of commands only offer limited guarantees about the returned elements since the collection that we incrementally iterate can change during the iteration process. Like described above collection will be changed frequently
A more appropriate data structure would be the Sorted Set - members have a float score that is very suitable for storing a timestamp and you can perform range searches (i.e. anything less or equal a given value).
The relevant starting points are the ZADD, ZRANGEBYSCORE and ZREMRANGEBYSCORE commands.
To ensure the atomicity when reading and removing members, you can choose between the the following options: Redis transactions, Redis Lua script and in the next version (v4) a Redis module.
Transactions
Using transactions simply means doing the following code running on your instances:
MULTI
ZRANGEBYSCORE <keyname> -inf <now-timestamp>
ZREMRANGEBYSCORE <keyname> -inf <now-timestamp>
EXEC
Where <keyname> is your key's name and <now-timestamp> is the current time.
Lua script
A Lua script can be cached and runs embedded in the server, so in some cases it is a preferable approach. It is definitely the best approach for short snippets of atomic logic if you need flow control (remember that a MULTI transaction returns the values only after execution). Such a script would look as follows:
local r = redis.call('ZRANGEBYSCORE', KEYS[1], '-inf', ARGV[1])
redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', ARGV[1])
return r
To run this, first cache it using SCRIPT LOAD and then call it with EVALSHA like so:
EVALSHA <script-sha> 1 <key-name> <now-timestamp>
Where <script-sha> is the sha1 of the script returned by SCRIPT LOAD.
Redis modules
In the near future, once v4 is GA you'll be able to write and use modules. Once this becomes a reality, you'll be able to use this module we've made that provides the ZPOP command and could be extended to cover this use case as well.
I need to calculate how much memory a Redis SortedSet takes assuming my average element of the Sorted Set is X bytes.
If you know the average size of an element before it's stored in redis, just do this:
Clear redis of all data: command flushall (dumps all databases)
Command info, check field used_memory_human (should be zero or close to it)
Add/store data in redis
info again, check used_memory_human, size indicates memory used by redis to store objects.
Hope it helps
According to the Redis benchmarks , Redis can perform 100'000 SET operations/s, and 80'000 GET operations/s. Redis being an in-memory DB, this seems surprising because typically one would expect that memory writes are somewhat slower than reads, e.g. considering that SETs need to allocate memory prior to being able to write a value.
Can someone explain why SET is faster than GET?
Actually this is only an effect that by default you measure more I/O than the actual command execution time. If you start enabling pipelining in the benchmark, it is a bit more the measure of the actual command performance, and the numbers will change:
$ redis-benchmark -q -n 1000000 -P 32 set foo bar
set foo bar: 338964.03
$ redis-benchmark -q -n 1000000 -P 32 get foo
get foo: 432713.09 requests per second
Now GET is faster :-)
We should include pipelining in our benchmark doc page.
EDIT: This is even more evident here:
redis 127.0.0.1:6379> info commandstats
# Commandstats
cmdstat_get:calls=1001568,usec=221845,usec_per_call=0.22
cmdstat_set:calls=831104,usec=498235,usec_per_call=0.60
This command provides CPU time to serve the request internally, without accounting for I/O. SET is three times slower to process.