Number of expiring keys listed by info command on redis slave not consistent with what I see - redis

Question

When I run the info command in redis-cli against a redis 3.2.4 server, it shows me this for expires:
expires=223518
However, when I then run a keys * command and ask for the ttl for each key, and only print out keys with a ttl > 0, I only see a couple hundred.
I thought that the expires is a count of the number of expiring keys but I am not even within an order of magnitude of this number.
Can someone clarify exactly what expires is meant to convey? Does this include both to-be-expired and previously expired but not yet evicted keys?
Update:
Here is how I counted the number of keys expiring:
task count_tmp_keys: :environment do
redis = Redis.new(timeout: 100)
keys = redis.keys '*'
ct_expiring = 0
keys.each do |k|
ttl = redis.ttl(k)
if ttl > 0
ct_expiring += 1
puts "Expiring: #{k}; ttl is #{ttl}; total: #{ct_expiring}"
STDOUT.flush
end
end
puts "Total expiring: #{ct_expiring}"
puts "Done at #{Time.now}"
end
When I ran this script it shows I have a total expiring of 78
When I run info, it says db0:keys=10237963,expires=224098,avg_ttl=0
Because 224098 is so much larger than 78, I am very confused. Is there perhaps a better way for me to obtain a list of all 225k expiring keys?
Also, how is it that my average ttl is 0? Wouldn't you expect it to be nonzero?
UPDATE
I have new information and a simple, 100% repro of this situation locally!
To repro: setup two redis processes locally on your laptop. Make one a slave of the other. On the slave process, set the following:
config set slave-serve-stale-data yes
config set slave-read-only no
Now, connect to the slave (not the master) and run:
set foo 1
expire foo 10
After 10 seconds, you will no longer be able to access foo, but info command will still show that you have 1 key expiring with an average ttl of 0.
Can someone explain this behavior?

Answer 1

expires contains existing keys with TTL which will expire, not including already expired keys.
Example ( with omission of extra information from info command for brevity ):
127.0.0.1:6379> flushall
OK
127.0.0.1:6379> SETEX mykey1 1000 "1"
OK
127.0.0.1:6379> SETEX mykey2 1000 "2"
OK
127.0.0.1:6379> SETEX mykey3 1000 "3"
OK
127.0.0.1:6379> info
# Keyspace
db0:keys=3,expires=3,avg_ttl=992766
127.0.0.1:6379> SETEX mykey4 1 "4"
OK
127.0.0.1:6379> SETEX mykey5 1 "5"
OK
127.0.0.1:6379> info
# Keyspace
db0:keys=3,expires=3,avg_ttl=969898
127.0.0.1:6379> keys *
1) "mykey2"
2) "mykey3"
3) "mykey1"
127.0.0.1:6379>
Given that in your situation you are asking about key expiry on slaves, per https://github.com/antirez/redis/issues/2861:
keys on a slave are not actively expired, and thus the avg_ttl is
never calculated
And per https://groups.google.com/forum/#!topic/redis-db/NFTpdmpOPnc:
avg_ttl is never initialized on a slave and thus it can be what ever
arbitrary value resides in memory at that place.
Thus, it is to be expected that the info command behaves differently on slaves.

Answer 2

The expires just returns the size of keys that will expire not the time.
The source code of 3.2.4
long long keys, vkeys;
keys = dictSize(server.db[j].dict);
vkeys = dictSize(server.db[j].expires);
if (keys || vkeys) {
info = sdscatprintf(info,
"db%d:keys=%lld,expires=%lld,avg_ttl=%lld\r\n",
j, keys, vkeys, server.db[j].avg_ttl);
}
It just calculate the size of server.db[j].expires. (note j is the database index).