Two rdb files in /var/lib/redis dir - redis

I wanted to check what redis dumps are available so in redis-cli I did
127.0.0.1:6379> config get dir
1) "dir"
2) "/var/lib/redis"
also:
127.0.0.1:6379> config get databases
1) "databases"
2) "16"
In my /var/lib/redis dir I discovered 2 .rdb files
root#docker-elk01:/var/lib/redis# ls -lh
total 4.4G
-rw-r--r-- 1 redis redis 4.4G Feb 22 04:27 dump.rdb
-rw-r--r-- 1 redis redis 28M Feb 22 12:10 temp-66388.rdb
What I also noticed is that the temp- files comes and goes.
Why are there 2 files?
I know that in our apps we are using 2 dbs as shown below
127.0.0.1:6379> info keyspace
# Keyspace
db1:keys=262013,expires=0,avg_ttl=0
db10:keys=199,expires=199,avg_ttl=498984788
Does the dump.rdb includes a dump from both databases?
Can I selectively disable dumps from one of those dbs? (or from both?)

Why are there 2 files?
temp-66388.rdb is a temp file used to save RDB file. When saving the database to a RDB file, Redis creates a temp file with the process id, and dumps the database to that file. If it dumps the database successfully, Redis rename this temp file to dump.rdb. That's why the temp file comes and goes.
Does the dump.rdb includes a dump from both databases?
YES.
Can I selectively disable dumps from one of those dbs? (or from both?)
NO, you CANNOT do that. You can either dump all databases, or dump nothing at all. Also check this similar question.

Related

how to import dump.rdb file to redis local server

Hi I'm trying to import the dump.rdb file to my local redis I'm using ubuntu 14.04,
I've tried this solutions :
backup data from server using SAVE command
Locate the location to put the dump.rdb file
Since I install redis using this tutorial, so I copy the imported dump.rdb to my redis root directory, and then started the redis server like this :
src/redis-server
and then connect the client using :
src/redis-cli
But When I tried to get the all keys using KEYS * I got (empty list or set) where did I go wrong? I've been playing this for hours, any help? thank you
If you have followed the steps correctly it will work fine.
1) Make sure the imported dump.rdb contains your data
2) Stop the redis server
3) copy the file in the correct directory (inside redis bin directory)
parallel to redis-server.
4) make sure you have the same data, that is copied. (bcz possibilites
that if your server is still running, it will replace your dump.rdb).
5) start your redis server you will surely find the values.
If it still doesn't work. Check the dbfilename in your redis.conf file.
It must be dbfilename dump.rdb. If there is a change in the location place it in the correct directory.
Hope this works.
I found the problem in my step, in the documentation quick start redis :
Using src/redis-server Redis was started without any explicit configuration file so I need to start the server with the configuration file to make the server read my dump.rdb file like this :
src/redis-server redis.conf
now I can get all the imported data.

Redis Ignoring directory in redis.conf

I recently installed Redis on OS X. I did an initial load of some objects, everything in this regard seems to be working perfectly.
I saved the data to dump.rdb and if I launch redis from this directory and don't use a redis.conf file everything goes according to plan.
The problem is that after settling in and setting up a redis.conf file in /etc/redis/ it seems to completely ignore the dbfilename and dir settings.
To be more clear. I created a data directory in: /Library/Redis_Data/.
In the redis.conf I set dbfilename dump.rdb and dir /Library/Redis_Data
If I launch: redis-server /etc/redis/redis.conf it simply refuses to load the dump.rdb.
I've run redis-cli and can read the config parameters get config dbfilename and get config dir and they return the correct values, but no data gets restored.
I also enabled AOF. The file appendonly.aof which resides in the same directory will be recreated if I delete it and start the server again.
Lastly...if I run save from the cli it will completely wipe out and create a new dump.rdb over my data.
Any thoughts?
FYI: redis server v=2.6.9 sha=00000000:0 malloc=libc bits=64
Thanks.
Solved:
Looking back this morning on this issue I managed to discover the problem.
I originally added my data without enabling appendonly mode. After this test load I setup my environment and decided it would be great to enable appendonly.
What I discovered is this. If you have data in the dump.rdb but the appendonly.aof file is empty and appendonly is turned on; Redis will ignore the dump.rdb instead assuming there is nothing to be done because the appendonly.aof transactions don't exist.
What I did: Turned off appendonly in the config file and restarted Redis with the normal directory setup. This then loaded the dump.rdb as per my config file. I then ran the command from redis-cli config set appendonly yes followed by save.
This then created the appendonly.aof file with all the transactions. Set appendonly back to yes in the config and all was right.
Cheers,
damnabit

How to recover redis data from snapshot(rdb file) copied from another machine?

I transferred my redis snapshot (dump.rdb file) using scp to a remote server. I need to run a redis server on this remote and recover the data from the dump.rdb file. How can I do that?
For databases where the appendonly flag is set to no, you can do the following:
Stop Redis (because Redis overwrites the current rdb file when it exits).
Copy your backup rdb file to the Redis working directory (this is the dir option in your Redis config). Also, make sure your backup filename matches the dbfilename config option.
Start Redis.
If, on the other hand, you need to restore an rdb file to the appendonly database, you should do something along the lines of:
Stop Redis (because Redis overwrites the current rdb file when it exits).
Copy your backup rdb file to the Redis working directory (this is the dir option in your Redis config). Also, make sure your backup filename matches the dbfilename config option.
Change the Redis config appendonly flag to no (otherwise Redis will ignore your rdb file when it starts).
Start Redis.
Run redis-cli BGREWRITEAOF to create a new appendonly file.
Restore the Redis config appendonly flag to yes.
Specifically, this is the relevant bit of documentation from the Redis config file comments:
# Note that you can have both the async dumps and the append only file if you
# like (you have to comment the "save" statements above to disable the dumps).
# >> Still if appendonly mode is enabled Redis will load the data from the
# >> log file at startup ignoring the dump.rdb file.
There is nothing specific to do. Just install the redis server on the new machine, and edit the configuration file. You just need to change the following parameters to point to the location of the dump file you have just copied.
# The filename where to dump the DB
dbfilename mydump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# Also the Append Only File will be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir /data/mydirectory/
Finally, the redis server can be started in the normal way.
Assuming that you run Redis 2.6 or higher, your Redis snapshot filename is dump.rdb, and it exists in the directory /home/user/dbs, the following command would do the trick:
redis-server --dbfilename dump.rdb --dir /home/user/dbs
Relevant section from the official documentation: Passing arguments via the command line
Or you can:
Stop your redis server / instance, eg., service redis6379 stop
Copy the dump.rdb file to the right location, eg., cp /path/to/dump-6379.rdb /var/lib/redis/dump-6379.rdb. Give it the right permissions (user:group should be redis:redis and mode 644)
Start your redis server / instance, eg., service redis6379 start
It is important that you stop the redis server before copying the file to the right location, because Redis saves a snapshot before terminating, so it will replace your file.
Besides, you might want to back up the existing dump.rdb file first.
I would like to add here a tiny detail that did not get mentioned and I will not use config file but instead specify everything in the command line.
When both mydump.rdb and appendonly.aof files are specified when starting redis-server, it will be the appendonly.aof file that wins such that the data from appendonly.aof gets loaded. For example:
redis-server --dbfilename mydump001.rdb --dir /data --appendonly yes
The above start invocation will use the /dir location to find the presence of mydump001.rdb or appendonly.aof files. In this case, redis-server will load the contents from appendonly.aof. If appendonly.aof does not exists, it will create an empty /data/appendonly.aof and the redis-server will be empty.
If you want to load a specific dump file, you can do:
redis-server --dbfilename mydump001.rdb --dir /data
I added this answer coz it is not obvious which is which. In the presence of 2 backup files, and this is often not mentioned.
start redis on your second server, like so:
$ > redis-server /path/to/my/redis/configuration/file/redis.conf
when redis starts, it will find your rdb file because it will look for the name
and file path in the configuration file (redis.conf) that you supply when
start the redis server, as above.
to supply the file name and path, just edit two lines in the redis.conf file template (supplied in the root directory of the redis source. Save your revised version as redis.conf in the directory location that you supplied upon starting the server.
You will find the settings you need in the redis.conf template in the source top-level directory, at lines 127 and 137 (redis version 2.6.9).
# The filename where to dump the DB
dbfilename dump.rdb
# The working directory
dir ./
as you can see, defaults are provided for both settings
so just change the first of these two lines (127) to identify your rdb file
and in the second (137) substitute the default "./" for the actual file path
for your snapshot rdb file; save the redis.conf with your changes, and start redis passing in this new conf file.
This solution work with redis-cluster, but should work too with redis.
Install this dependencie https://github.com/sripathikrishnan/redis-rdb-tools
pip install rdbtools python-lzf
after that execute this
rdb -c protocol /path/to/dump.rdb | redis-cli -h host -p port --pipe
If this is a cluster, the port should the master`s port.
try set appendonly no.
In My case, *.aof file was empty(0 byte), must set appendonly=no then make it load the dump.rdb
Install https://github.com/leonchen83/redis-rdb-cli
rmt -s ./your-dump.rdb -m redis://host:port -r

Update redis server from 1.2.6 to latest

I need to update redis server.
I found a way to save DB on disk and after restore it, but my question is will new redis server have problems with read old DB structure?
The version of the dump file is encoded in the first 9 characters. So the following command can be used to check it:
$ head -1 dump.rdb | cut -c1-9
REDIS0002
Redis 1-2-6 used version 1 of the dump file (it can read and write only version 1)
Redis 2-4-6 is using version 2. However, it is able to read both version 1 and version 2 files. Version 2 happen to be backward compatible with version 1 anyway.
To upgrade, you can just read the version 1 dump file with a recent Redis release, and then dump the file again (it will be written with version 2 format). The new file may be smaller due to some optimizations available with recent Redis versions and the version 2 format.
Optionally, you can check the integrity of the dump file before starting the 2-4 Redis instance by using the redis-check-dump command:
$ ../redis-2.4.4/src/redis-check-dump dump.rdb
==== Processed 19033 valid opcodes (in 639641 bytes) ===========================
This is a pure read-only utility, it cannot harm the dump file.

What should I do if I delete the redis vm file?

Now the redis can not background save.
I just want to dump it and restart.
How could I dump the redis? I've always this error :
"Can't re-open the VM
swap file: /tmp/redis.swap. Exiting."
Delete the dump.rdb file. (or simply move it somewhere e.g ~/saved-rdb/., where redis-server won't find it.)
Start redis-server
Note: dump.rdb is your data. By deleting it you are starting fresh (meaning data lost).