I am using Redis for cache in my application which is configured in spring beans, spring-data-redis 1.7.1, jedis 2.9.0.
I would like to know how to set the race condition ttl in the configuration.
Please comment if you have any suggestions.
If I understand you right, you want the same as that Ruby repo, but in Java.
For that case you may want to put a technical lock key along the one you need.
get yourkey
(nil)
get <yourkey>::lock
// if (nil) then calculate, if t then wait. assuming (nil) here
setex <yourkey>::lock 30 t
OK
// calcultions
set <yourkey> <result>
OK
del <yourkey>::lock
(integer) 1
Here with setex you set a lock key with TTL of 30 sec. You can put another TTL if you want.
There is one problem with the code above - some time will pass before checking a lock and aquiring it. To properly aquire a lock EVAL can be used: eval "local lk=KEYS[1]..'::lock' local lock=redis.call('get',lk) if (lock==false) then redis.call('setex',lk,KEYS[2],'t') return 1 else return 0 end" 2 <yourkey> 30 This either returns 0 if there is no lock or puts a lock and returns 1.
Related
I am reading https://redis.io/docs/manual/transactions/, and I am not sure I fully understand the importance of WATCH in a Redis transaction. From https://redis.io/docs/manual/transactions/#usage, I am led to believe EXEC is atomic:
> MULTI
OK
> INCR foo
QUEUED
> INCR bar
QUEUED
> EXEC
1) (integer) 1
2) (integer) 1
But between MULTI and EXEC, the values at keys foo and bar could have been changed (e.g. in response to a request from another client). Is that why WATCH is useful? To error out the transaction if e.g. foo or bar changed after MULTI and before EXEC?
Do I have it correct? Anything I am missing?
I am led to believe EXEC is atomic
YES
Is that why WATCH is useful? To error out the transaction if e.g. foo or bar changed after MULTI and before EXEC?
NOT exactly. Redis fails the transaction if some key the client WATCHed before the transaction has been changed (after WATCH and before EXEC). The watched key might even not used in transaction.
MULTI-EXEC ensures commands between them runs atomically, while WATCH provides check-and-set(CAS) behavior.
With MULTI-EXEC, Redis ensures running INCR foo and INCR bar atomically, otherwise, Redis might run some other commands between INCR foo and INCR bar (these commands might and might not change foo or bar). If you watch some key (either foo, or bar, or even any other keys) before the transaction, and the watched key has been modified since the WATCH command, the transaction fails.
UPDATE
The doc you referred has a good example of using WATCH: implement INCR with GET and SET. In order to implement INCR, you need to 1) GET the old counter from Redis, 2) incr it on client side, and 3) SET the new value to Redis. In order to make it work, you need to ensure no other clients update/change the counter after you run step 1, otherwise, you might write incorrect value and should fail the transaction. 2 clients both get the old counter, i.e. 1, and incr it locally, and client 1 update it to 2, then client 2 update it to 2 again (the counter should be 3). In this case, WATCH is helpful.
WATCH mykey
val = GET mykey
val = val + 1
MULTI
SET mykey $val
EXEC
If some other client update the counter after we watch it, the transaction fails.
When do watched keys no longer watched?
EXEC is called.
UNWATCH is called.
DISCARD is called.
connection is closed.
My setting is Redis master-slave replication. I am sure the slaves are read only because when I connect to slave and try to write data, "(error) READONLY You can't write against a read only slave." is returned.
However, when I check the slowlog there are SET commands, eg:
127.0.0.1:6379> slowlog get 1
1) 1) (integer) 1360
2) (integer) 1544276677
3) (integer) 10653
4) 1) "SET"
2) "some value"
Anyone can explain this? Thanks in advance.
The Redis replica is replaying commands sent from the master, so the SET command must have originated from it.
It is unclear why that command ended in the slowlog, but it could be because of any number of reasons (IO or CPU blockage). If this happened once I wouldn't worry about it, but if it is pathological you may want to review your replica's infrastructure and configuration.
I can't find anywhere online what is default TTL in Redis.
I know that I can set TTL for specific SET, but don't know what is default TTL.
Can someone tell me what default time to live is in Redis?
There is no default TTL. By default, keys are set to live forever.
The keys with no expiration time set will not expire.
If you mean TTL command specifically, starting with v2.8, it will return -2 if no EXPIRE value is set.
Edit:
Itamar Haber's comment is true, I recalled false: There is no such setting in redis config for a global TTL. So I deleted the part about that.
Edit2: Also see the link to the official docs about default expiration of keys here: https://redis.io/commands/expire#appendix-redis-expires
I suppose value set to '-1' by default which means forever.
127.0.0.1:6379> set datakey "my-data"
OK
127.0.0.1:6379> TTL datakey
(integer) -1
127.0.0.1:6379>
REDIS Docs says
Starting with Redis 2.8 the return value in case of error changed:
The command returns -2 if the key does not exist.
The command returns -1 if the key exists but has no associated expire.
When I run the info command in redis-cli against a redis 3.2.4 server, it shows me this for expires:
expires=223518
However, when I then run a keys * command and ask for the ttl for each key, and only print out keys with a ttl > 0, I only see a couple hundred.
I thought that the expires is a count of the number of expiring keys but I am not even within an order of magnitude of this number.
Can someone clarify exactly what expires is meant to convey? Does this include both to-be-expired and previously expired but not yet evicted keys?
Update:
Here is how I counted the number of keys expiring:
task count_tmp_keys: :environment do
redis = Redis.new(timeout: 100)
keys = redis.keys '*'
ct_expiring = 0
keys.each do |k|
ttl = redis.ttl(k)
if ttl > 0
ct_expiring += 1
puts "Expiring: #{k}; ttl is #{ttl}; total: #{ct_expiring}"
STDOUT.flush
end
end
puts "Total expiring: #{ct_expiring}"
puts "Done at #{Time.now}"
end
When I ran this script it shows I have a total expiring of 78
When I run info, it says db0:keys=10237963,expires=224098,avg_ttl=0
Because 224098 is so much larger than 78, I am very confused. Is there perhaps a better way for me to obtain a list of all 225k expiring keys?
Also, how is it that my average ttl is 0? Wouldn't you expect it to be nonzero?
UPDATE
I have new information and a simple, 100% repro of this situation locally!
To repro: setup two redis processes locally on your laptop. Make one a slave of the other. On the slave process, set the following:
config set slave-serve-stale-data yes
config set slave-read-only no
Now, connect to the slave (not the master) and run:
set foo 1
expire foo 10
After 10 seconds, you will no longer be able to access foo, but info command will still show that you have 1 key expiring with an average ttl of 0.
Can someone explain this behavior?
expires contains existing keys with TTL which will expire, not including already expired keys.
Example ( with omission of extra information from info command for brevity ):
127.0.0.1:6379> flushall
OK
127.0.0.1:6379> SETEX mykey1 1000 "1"
OK
127.0.0.1:6379> SETEX mykey2 1000 "2"
OK
127.0.0.1:6379> SETEX mykey3 1000 "3"
OK
127.0.0.1:6379> info
# Keyspace
db0:keys=3,expires=3,avg_ttl=992766
127.0.0.1:6379> SETEX mykey4 1 "4"
OK
127.0.0.1:6379> SETEX mykey5 1 "5"
OK
127.0.0.1:6379> info
# Keyspace
db0:keys=3,expires=3,avg_ttl=969898
127.0.0.1:6379> keys *
1) "mykey2"
2) "mykey3"
3) "mykey1"
127.0.0.1:6379>
Given that in your situation you are asking about key expiry on slaves, per https://github.com/antirez/redis/issues/2861:
keys on a slave are not actively expired, and thus the avg_ttl is
never calculated
And per https://groups.google.com/forum/#!topic/redis-db/NFTpdmpOPnc:
avg_ttl is never initialized on a slave and thus it can be what ever
arbitrary value resides in memory at that place.
Thus, it is to be expected that the info command behaves differently on slaves.
The expires just returns the size of keys that will expire not the time.
The source code of 3.2.4
long long keys, vkeys;
keys = dictSize(server.db[j].dict);
vkeys = dictSize(server.db[j].expires);
if (keys || vkeys) {
info = sdscatprintf(info,
"db%d:keys=%lld,expires=%lld,avg_ttl=%lld\r\n",
j, keys, vkeys, server.db[j].avg_ttl);
}
It just calculate the size of server.db[j].expires. (note j is the database index).
My redis server does not delete keys when the time-to-live reaches 0.
Here is a sample code:
redis-cli
>SET mykey "ismykey"
>EXPIRE mykey 20
#check TTL
>TTL mykey
>(integer) 17
> ...
>TTL mykey
>(integer) -1
#mykey chould have expired:
>EXISTS mykey
>(integer) 1
>#oh still there, check its value
>GET mykey
>"ismykey"
If i check the info return by redis, it says 0 keys were expired.
Any idea?
thanks.
Since you're doing a '...' it's hard to say for sure, but I'd say you're setting mykey during that part, which will effectively remove the expiration.
From the EXPIRE manual
The timeout is cleared only when the key is removed using the DEL
command or overwritten using the SET or GETSET commands
Also, regarding the -1 reply from TTL
Return value
Integer reply: TTL in seconds or -1 when key does not
exist or does not have a timeout.
EDIT: Note that this behaviour changed in Redis 2.8
Starting with Redis 2.8 the return value in case of error changed:
The command returns -2 if the key does not exist.
The command returns -1 if the key exists but has no associated expire.
In other words, if your key exists, it would seem to be persistent, ie not have any expiration set.
EDIT: It seems I can reproduce this if I create the key on a REDIS slave server, the slave will not delete the key without master input, since normally you would not create keys locally on a slave. Is this the case here?
However while the slaves connected to a master will not expire keys
independently (but will wait for the DEL coming from the master),
they'll still take the full state of the expires existing in the
dataset, so when a slave is elected to a master it will be able to
expire the keys independently, fully acting as a master.