our Redis server went down (unsure why, not a server outage) and trying to restart with the pre-existing aof file in place and it wouldn't start getting stuck on the line "Reading RDB preamble from AOF file..."
The server was originally started as follows:
docker run -d -p 6379:6379 --name redis -e REDIS_PASSWORD=# -v /var/redis/data:/bitnami/redis/data bitnami/redis:5.0.5-r5
BGREWRITEAOF was enabled so the aof was rewritten by Redis to keep the size lower
Moving the previous AOF file and starting a new container, Redis came up fresh.
I have tried to import the previous AOF using
redis-cli -a # --pipe < previousfile.aof
or
cat previousfil.aof | redis-cli - a # --pipe
But it fails due to the input being too large
redis-check-aof says my previous AOF is invalid, but I wonder it is because the file was compressed with BGREWRITEAOF
I'm stuck with how to move forward.
Can I revert the BGREWRITEAOF to get the raw file?
Can I break the file up in that format and import?
Are there better ways to import Redis data
Thanks
Related
I have redis server 3.0.6 and ubuntu 16.04.
my config file
tcp-keepalive 60
#bind 127.0.0.1
requirepass qwerty
maxmemory-policy noeviction
appendonly yes
appendfilename redis-test.aof
and redis server don't run
Can't open the append-only file: Read-only file system
The error message is pretty clear: The file system on which redis-test.aof resides is mounted as read-only. The whole purpose of this file is to write changes to disk. So the disk must be writable.
Check if you used the ro option while mounting the drive. Run
$ mount
to list all the mountpoints. Check the one on which you want your aof file to reside.
To remount the disk as read-write, use the following command:
$ sudo mount -o remount,rw /partition/identifier /mount/point
If that doesn't help, see the system logs if there are any file system errors. To correct these, you will need to run fsck.
Someone before me setup a redis instance (version 2.6).
But for some reason, whoever set this, had
Placed the config file etc like this /etc/redis.conf
The dir config has ./ set, like this dir ./
The instance is being run as non-root.
Like this:
$ ps aux | grep "redis"`
user /home/user/redis-stable/src/redis-server /etc/redis.conf
Logging is going to /dev/null, because daemonize yes && logfile stdout
So it is unable to create backups in /etc/ because it doesn't have permissions (I'm guessing), and I can't even see what is going on because the logs are going to /dev/null.
I want to make a backup so I can turn redis off to fix all these things, without losing any data. Any ideas?
I've tried:
touch /etc/dump.rdb
chown user:users /etc/dump.rdb
But it is still not able to write. I'm guessing it might have a temp file it tries to write to before it moves it to /etc/dump.rdb
After looking at Redis source code, it does seem like there is a temp file: https://github.com/antirez/redis/blob/04542cff92147b9b686a2071c4c53574771f4f88/src/rdb.c#L986
snprintf(tmpfile,256,"temp-%d.rdb", (int) getpid());
Also tried
redis 127.0.0.1:6379> config set logfile /home/user/redis.log
(error) ERR Unsupported CONFIG parameter: logfile
Run:
config get dir
and you would see the directory where redis is saving rdb.
Run:
config set dir /home/user/
to change the rdb dump directory to /home/user.
then run:
redis-cli -p <port> bgsave
This would initiate a rdb dump.
Hope this helps.
Redis went quite on me.
user#mycomputer:~$ redis-cli
Could not connect to Redis at 127.0.0.1:6379: Connection refused
I try to restart the service by doing this
sudo /etc/init.d/redis_6379 stop
/var/run/redis/redis.pid exists, process is already running or crashed
But no luck. Logs didn't show an error as well.
Got it fixed by backing up the redis.rdp file mine is located at
/var/lib/redis
check your config file "/etc/redis/redis.conf" for the rdp file's location and do this
sudo mv /var/lib/redis/redis.rdp /var/lib/redis/redis_backup.rdp
Then recreate the the redis.rdp file
sudo touch redis.rdp
Run the redis-server with the conf and it should work
sudo redis-server /etc/redis/redis.conf
Get it fixed in a tidy way: Recreate the the redis.rdp file as suggested here in one of answer, will purge all the cache recorded so far and redis will start up fresh with no cache data.
This is a warning message to notify system crash / improper shutdown: "/var/run/redis/redis.pid exists, process is already running or crashed"
Just delete /var/run/redis/redis.pid file and restart the server again.
Note: You might have lost latest cache changes due to untidy shutdown, which weren't flushed into the disk. This data loss can be minimized using frequent disk flush configuration in redis conf file(in my case it is #/etc/redis/6379.conf)
save 900 1
save 300 10
save 60 10000
Or try AOF persistence, more details [here][1]
Depends on how you installed redis, the pid can be found on /var/run/redis_6379.pid.
What happened is that redis crashed, but the pid is still there. So you just have to delete it.
sudo rm -f /var/run/redis_6379.pid
Then start redis again:
sudo /etc/init.d/redis_6379 start
If you can't find it, I suggest installing redis "more properly". Follow redis quickstart guide in the Installing Redis more properly section.
You can find it here:
https://redis.io/topics/quickstart
Run the redis-server with config.
sudo redis-server redis.conf
I was wondering how to disable presistence in redis. There is mention of the possibility of doing this here: http://redis.io/topics/persistence. I mean it in the exact same sense as described there. Any help would be very much appreciated!
To disable all data persistence in Redis do the following in the redis.conf file:
Disable AOF by setting the appendonly configuration directive to no (it is the default value). like this:
appendonly no
Disable RDB snapshotting by commenting all of the save configuration directives (there are 3 that are defined by default) and explicitly disabling saving:
#save 900 1
#save 300 10
#save 60 10000
save ""
After change, make sure you restart Redis to apply them.
Alternatively, you can use the CONFIG SET command to apply these changes during runtime (just make sure you also do a CONFIG REWRITE to persist the changes).
Note: depending on your Redis' version, there are other tweaks that prevent Redis from accessing the disk for replication-related tasks.
If you want to avoid playing with redis.conf (dev/test environments), you can do it through the command line with
redis-server --save "" --appendonly no
(tested with redis server 3.2.6 and 5.0.5)
As AOF (appendonly) is disabled by default, there is only one thing that is to be done for disabling persistence without redis service restart is to disable save configuration.
For disabling it on runtime and verifying run below commands
Check current save configuration
pawan#devops:~$ redis-cli config get save
1) "save"
2) "900 1 300 10 60 10000"
Same setting will be present in redis.conf file as well
pawan#devops:~$ grep -w 'save' /etc/redis/redis.conf | grep -v '#'
save 900 1
save 300 10
save 60 10000
Disable save configuration
pawan#devops:~$ redis-cli config set save ""
OK
Modify redis.conf file with the new save configuration so that the configuration remains permanent on redis service restarts
root#ip-172-16-3-114:~# redis-cli config rewrite
OK
Confirm the new save configuration
pawan#devops:~$ redis-cli config get save
1) "save"
2) ""
Now if you will scan the redis.conf file for save configuration there won't be any results
pawan#devops:~$ grep -w 'save' /etc/redis/redis.conf | grep -v '#'
pawan#devops:~$
For RDB snapshotting you can disable it by using
$ sed -e '/save/ s/^#*/#/' -i /etc/redis/redis.conf && sudo service redis-server restart
It will comment the save lines in redis.conf and restarts the redis-server
I can't find a way to have redis load my own dump.rdb backup file on startup, on dotcloud. I can see in the server logs that redis is loading a file, but I don't know where it is (and I can't find it)
[144] 03 Jul 21:01:18 * DB loaded from disk: 0 seconds
I've tried to put the dump.rdb file in /var/lib/redis directory but it doesn't help
Thanks for any help
I've found what I made wrong: upon restart, redis makes a dump, and was overwriting my dump file with an empty dump and reloading the empty dump on startup. Correct process is:
~$ dotcloud ssh [your service]
~$ sudo /etc/init.d/redis stop
~$ cp [your dump] /var/lib/redis/dump.rdb
~$ sudo /etc/init.d/redis start