I have redis server 3.0.6 and ubuntu 16.04.
my config file
tcp-keepalive 60
#bind 127.0.0.1
requirepass qwerty
maxmemory-policy noeviction
appendonly yes
appendfilename redis-test.aof
and redis server don't run
Can't open the append-only file: Read-only file system
The error message is pretty clear: The file system on which redis-test.aof resides is mounted as read-only. The whole purpose of this file is to write changes to disk. So the disk must be writable.
Check if you used the ro option while mounting the drive. Run
$ mount
to list all the mountpoints. Check the one on which you want your aof file to reside.
To remount the disk as read-write, use the following command:
$ sudo mount -o remount,rw /partition/identifier /mount/point
If that doesn't help, see the system logs if there are any file system errors. To correct these, you will need to run fsck.
Related
Background
Yesterday our machine crashed unexpectedly and our AOF file for Redis got corrupted.
Upon trying to start the service with sudo systemctl start redis-server we are greeted with the following logs:
Bad file format reading the append only file: make a backup of your
AOF file, then use ./redis-check-aof --fix
Research
Aparently this looks like a simple error to fix, just execute ./redis-check-aof --fix <filename>.
Except I don't have the smallest idea of where that file is.
I have searched the Github discussions for this issue, but unfortunately none provides me with the location for the file:
https://github.com/antirez/redis/issues/4931
The persistence documentation also doesn't make a mention of the location for this file:
https://redis.io/topics/persistence
Specs
These are the specs of the system where I am running Redis:
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
Question
Where is located this file?
You have two choices:
Find the configure file for Redis, normally, it's named redis.conf. The dir and appendfilename configuration specify the directory and file name of the AOF file.
Connect to Redis with redis-cli, and use the CONFIG GET command to get the dir configuration, i.e. CONFIG GET dir. The AOF file should located under this directory.
The path is typically /var/lib/redis/appendonly.aof you will need to run sudo redis-check-aof --fix /var/lib/redis/appendonly.aof
in case if you use docker and append volume to /data then the path to appendonly.aof will be: /data/appendonly.aof
In my case, I was using docker. I started the redis server without using --appendonly yes, then it started without any issues. And then ran CONFIG GET dir like #for-stack said and got this output:
1) "dir"
2) "/data"
So I checked under the /data path and found the file appendonly.aof
Then I ran /usr/local/bin/redis-check-aof --fix /data/appendonly.aof to fix the issue.
I ran /path/redis-check-aof --fix /data/appendonly.aof to fix this.
Thanks all.
Someone before me setup a redis instance (version 2.6).
But for some reason, whoever set this, had
Placed the config file etc like this /etc/redis.conf
The dir config has ./ set, like this dir ./
The instance is being run as non-root.
Like this:
$ ps aux | grep "redis"`
user /home/user/redis-stable/src/redis-server /etc/redis.conf
Logging is going to /dev/null, because daemonize yes && logfile stdout
So it is unable to create backups in /etc/ because it doesn't have permissions (I'm guessing), and I can't even see what is going on because the logs are going to /dev/null.
I want to make a backup so I can turn redis off to fix all these things, without losing any data. Any ideas?
I've tried:
touch /etc/dump.rdb
chown user:users /etc/dump.rdb
But it is still not able to write. I'm guessing it might have a temp file it tries to write to before it moves it to /etc/dump.rdb
After looking at Redis source code, it does seem like there is a temp file: https://github.com/antirez/redis/blob/04542cff92147b9b686a2071c4c53574771f4f88/src/rdb.c#L986
snprintf(tmpfile,256,"temp-%d.rdb", (int) getpid());
Also tried
redis 127.0.0.1:6379> config set logfile /home/user/redis.log
(error) ERR Unsupported CONFIG parameter: logfile
Run:
config get dir
and you would see the directory where redis is saving rdb.
Run:
config set dir /home/user/
to change the rdb dump directory to /home/user.
then run:
redis-cli -p <port> bgsave
This would initiate a rdb dump.
Hope this helps.
Redis went quite on me.
user#mycomputer:~$ redis-cli
Could not connect to Redis at 127.0.0.1:6379: Connection refused
I try to restart the service by doing this
sudo /etc/init.d/redis_6379 stop
/var/run/redis/redis.pid exists, process is already running or crashed
But no luck. Logs didn't show an error as well.
Got it fixed by backing up the redis.rdp file mine is located at
/var/lib/redis
check your config file "/etc/redis/redis.conf" for the rdp file's location and do this
sudo mv /var/lib/redis/redis.rdp /var/lib/redis/redis_backup.rdp
Then recreate the the redis.rdp file
sudo touch redis.rdp
Run the redis-server with the conf and it should work
sudo redis-server /etc/redis/redis.conf
Get it fixed in a tidy way: Recreate the the redis.rdp file as suggested here in one of answer, will purge all the cache recorded so far and redis will start up fresh with no cache data.
This is a warning message to notify system crash / improper shutdown: "/var/run/redis/redis.pid exists, process is already running or crashed"
Just delete /var/run/redis/redis.pid file and restart the server again.
Note: You might have lost latest cache changes due to untidy shutdown, which weren't flushed into the disk. This data loss can be minimized using frequent disk flush configuration in redis conf file(in my case it is #/etc/redis/6379.conf)
save 900 1
save 300 10
save 60 10000
Or try AOF persistence, more details [here][1]
Depends on how you installed redis, the pid can be found on /var/run/redis_6379.pid.
What happened is that redis crashed, but the pid is still there. So you just have to delete it.
sudo rm -f /var/run/redis_6379.pid
Then start redis again:
sudo /etc/init.d/redis_6379 start
If you can't find it, I suggest installing redis "more properly". Follow redis quickstart guide in the Installing Redis more properly section.
You can find it here:
https://redis.io/topics/quickstart
Run the redis-server with config.
sudo redis-server redis.conf
I was wondering how to disable presistence in redis. There is mention of the possibility of doing this here: http://redis.io/topics/persistence. I mean it in the exact same sense as described there. Any help would be very much appreciated!
To disable all data persistence in Redis do the following in the redis.conf file:
Disable AOF by setting the appendonly configuration directive to no (it is the default value). like this:
appendonly no
Disable RDB snapshotting by commenting all of the save configuration directives (there are 3 that are defined by default) and explicitly disabling saving:
#save 900 1
#save 300 10
#save 60 10000
save ""
After change, make sure you restart Redis to apply them.
Alternatively, you can use the CONFIG SET command to apply these changes during runtime (just make sure you also do a CONFIG REWRITE to persist the changes).
Note: depending on your Redis' version, there are other tweaks that prevent Redis from accessing the disk for replication-related tasks.
If you want to avoid playing with redis.conf (dev/test environments), you can do it through the command line with
redis-server --save "" --appendonly no
(tested with redis server 3.2.6 and 5.0.5)
As AOF (appendonly) is disabled by default, there is only one thing that is to be done for disabling persistence without redis service restart is to disable save configuration.
For disabling it on runtime and verifying run below commands
Check current save configuration
pawan#devops:~$ redis-cli config get save
1) "save"
2) "900 1 300 10 60 10000"
Same setting will be present in redis.conf file as well
pawan#devops:~$ grep -w 'save' /etc/redis/redis.conf | grep -v '#'
save 900 1
save 300 10
save 60 10000
Disable save configuration
pawan#devops:~$ redis-cli config set save ""
OK
Modify redis.conf file with the new save configuration so that the configuration remains permanent on redis service restarts
root#ip-172-16-3-114:~# redis-cli config rewrite
OK
Confirm the new save configuration
pawan#devops:~$ redis-cli config get save
1) "save"
2) ""
Now if you will scan the redis.conf file for save configuration there won't be any results
pawan#devops:~$ grep -w 'save' /etc/redis/redis.conf | grep -v '#'
pawan#devops:~$
For RDB snapshotting you can disable it by using
$ sed -e '/save/ s/^#*/#/' -i /etc/redis/redis.conf && sudo service redis-server restart
It will comment the save lines in redis.conf and restarts the redis-server
Context
I'm using redis. The database is < 100 MB.
However, I want to make daily backups.
I'm also running on Ubuntu Server 12.04
When type in:
redis-cli save
I don't know where dump.rdb is saved to (since redis is started as a service and not in my local directory).
Questions:
How do I find where redis is saving my dump.rdb to?
Is there someway that I can specify a filename to 'save', so I type in something like:
redis-cli save ~/db-2012-06-24.rdb
Thanks
To be a little more helpfull... How to find or set where redis is saving the dump.rdb file (ubuntu server):
First find you redis.conf file: In your terminal run:
ps -e aux | grep redis
I found my redis.conf file in:
var/etc/redis/
If yours is the same place then open the file with:
pico var/etc/redis/redis.conf
Look for:
# The filename where to dump the DB
dbfilename dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# Also the Append Only File will be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir /var/lib/redis
Depending on your setting for "dbfilename" and "dir" then that is where you find your redis dump.rdb file.
Update:
To see your redis configurations just run:
redis-cli CONFIG GET *
You can set the file location on the redis.conf file (which you start the server with)
look at the server configuration for that:
# The filename where to dump the DB
dbfilename dump.rdb
finding the location of the currently saved file, it depends on how you start the server - where you have the redis-server file - i think you can find it with ps -e aux | grep redis or ps -e | grep redis
On my (default, Ubuntu) setup the db file is in
/var/lib/redis/redis.rdb
As Christoffer points out, you can see all the settings from the command-line client with
CONFIG GET *
One liner to get both directory and dump file name
echo "CONFIG GET *" | redis-cli | grep -e "dir" -e "dbfilename" -A1
In mac,
the location of dump.rdb is at /usr/local/etc/dump.rdb.
the location of redis.conf is at /usr/local/etc/redis.conf.
In order to find the location use the command find - sudo find / -name "redis.conf"