I'm working in combined mod - RDB + AOF.
I'm looking for a way to load after a restart from the RDB file - mainly for fast restarts.
On top of that, I want to continue writing the AOF.
Once I know there is a disaster, I will manually load from AOF.
This is my current config: (I know that appendonly yes is saying that the AOF will be loaded after restart, I'm looking for a wait load from RDB and keep writing the AOF.)
aof-use-rdb-preamble yes
aof-load-truncated yes
aof-rewrite-incremental-fsync yes
appendfilename "appendonly.aof"
appendfsync everysec
appendonly yes
Thanks
Redis will always load the AOF if both are enabled, as AOF gives you better durability.
By using aof-use-rdb-preamble yes, you are already getting the best of both worlds. Your AOF is automatically rewritten every now and then automatically, with an RDB file first and a AOF tail. See redis.conf L1157.
Since you want to have a predictable mean time to recovery (MTTR), you want to adjust the parameters for Automatic rewrite of the AOF, as described in redis.conf LL113
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
You can also manually trigger a AOF rewrite with the BGREWRITEAOF command
BGREWRITEAOF would work even if appendonly is configured to no. However, note that every time you call BGREWRITEAOF you are getting pretty much an rdb file inside the appendonly.aof file.
Then, if appendonly is configured to yes, you also get an AOF tail (commands are appended to the appendonly.aof file).
BGREWRITEAOF and BGSAVE are expensive operations and will degrade the performance of your server while running. So I suggest you use only AOF, that already gives you log compaction either automatically or every time you run BGREWRITEAOF.
You can set auto-aof-rewrite-percentage to a low value, say 2 or 5 percent. Then you can test the MTTR (time it takes to restart) with both strategies. I am sure you'll find the difference is too small for it to make sense combining the two strategies separately (RDB and AOF). AOF already gives you RDB-inside if aof-use-rdb-preamble yes
Related
I have configured my redis cluster with the following config:
appendonly no
save ""
But I found out that my redis is taking a back up and has taken a backup recently. My use case doesn't require any rdb save to be done. Am I missing some thing?
For disable all backups in redis go to redis.conf file do the following:
Comment all save directives, by default there are three of them.
save 900 1
save 300 10
save 60 10000
Disable appendonly (set appendonly to no)
Redis doesn't rewrite the appendonly file automatically.
My dump.rdb is 3Gb, but the appendonly file is 36Gb and filled up the disk.
Related configuration:
appendonly yes
appendfilename "appendonly.aof"
appendfsync no
no-appendfsync-on-rewrite yes
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
As I understood this line should make redis to rewrite aof when size has grown 100%.
auto-aof-rewrite-percentage 100
Does it make a new aof in the background and compares with the current size?
I did rewrite manually and aof is ~4GB, then why it didn't rewrite it?
EDIT:
Redis version: 2.8.4
I get these informations:
You open the RDB and AOF at the same time.
Manual rewrite command success.
auto-aof-rewrite-percentage config ok.
Automatic rewrite never triggered.
AOF will delay when RDB is running. I guess, the Redis was busy with RDB bgsave, and AOF rewrite delayed for each check util disk full. The AOF rewrite checking frequency is base on the configration hz.
Check the INFO command for rdb and aof information to assure that.
I configured a Redis instance to operate only in memory (no data are dumped to a persistent storage) by commenting out the save commands:
################################ SNAPSHOTTING ################################
# save 900 1
# save 300 10
# save 60 10000
Now when I start my instance, Redis checks if there is any data the file dump.rdb. If yes, then it loads the data and the execution continues only in-memory.
Is there a way for me to load previous data from appendonly.aof (append only mode) and then continue only in an "in-memory only mode"?
No - AOF loading upon startup (unlike RDB) is done if and only if appendonly is not set to no. What you could do, as a workaround, is set appendonly to yes in the redis.conf file and once the server is up and running issue a CONFIG SET appendonly no to turn it off.
I recently configured Redis to use AOF as well as RDB snapshotting.
However, it does not look like the AOF is replayed correctly on server startup.
I stopped the service. Then I made sure /var/redis/appendonly.aof is valid using redis-check-aof.
Then I started the server again. In this moment, the RDB file was empty. That's another issue I need to look into - Redis started losing all the data from time to time.
In the log file I can see the AOF is supposed to be loaded correctly:
DB loaded from append only file: 1.474 seconds
However, when I try to read a value which I know should be there, I get nothing:
127.0.0.1:6379> get iQube:Live:wordCount:2015:11:13:10:6
(nil)
In the AOF though, there are commands like this:
INCRBY
$36
iQube:Live:wordCount:2015:11:13:10:6
$1
2
*2
$4
Is there something else I need to do to make this work?
My fault. I did not secure the server properly and became target of probably the most typical attack to Redis. In effect, the AOF file contained flushall commands which wiped the DB clean upon loading.
At the very least, I recommend putting these three lines to redis.conf:
rename-command CONFIG someverylongandveryunguessablestring
rename-command FLUSHDB ""
rename-command FLUSHALL ""
How to completely disable RDB and AOF?
I don't care about Persistence and want it to be in mem only.
I have already commented out the:
#save 900 1
#save 300 10
#save 60 10000
But this did not help and I see that Redis still tries to write to disk.
I know that Redis wants to write to disk because I get this error: "Failed opening .rdb for saving: Permission denied"
I don't care about the error, because I want to disable the Persistence altogether.
If you want to change the redis that is running, log into the redis, and
disable the aof:
config set appendonly no
disable the rdb:
config set save ""
If you want to make these changes effective after restarting redis, using
config rewrite
to make these changes to redis conf file.
If your redis have not started, just make some changes to redis.conf,
appendonly no
save ""
make sure there are no sentences like "save 60 1000" after the upper sentences, since the latter would rewrite the former.
Update: please look at Fibonacci's answer. Mine is wrong, although it was accepted.
Commenting the "dbfilename" line in redis.conf should do the trick.