I'm very new to RIAK. I have a cluster with 5 nodes and I want to backup the data on the cluster. I ran the following command to backup data.
[root#PCPRIAK33 local]# riak-admin backup localhost riak /var/local/temp all
However I am getting the following error.
Attempting to restart script through sudo -H -u riak
{"init terminating in do_boot",{{nocatch,{could_not_reach_node,localhost}},[{riak_kv_backup,ensure_connected,1,[{file,"src/riak_kv_backup.erl"},{line,171}]},{riak_kv_backup,backup,3,[{file,"src/riak_kv_backup.erl"},{line,40}]},{erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,572}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
init terminating in do_boot ()
Can you please help me out? :)
Thanks.
I'll answer your immediate question (how to use riak-admin backup) first, but see the comments on preferred methods of backing up, at the end.
The command is:
riak-admin backup <node name> <erlang cookie> <file name with path> all
The node name you can find in your riak vm.args file (look for the line that looks like -name riak#127.0.0.1). It'll be of the form riak#xx.xx.xx.xx with the IP address. So, on my local machine, a single node is named riak#127.0.0.1.
The erlang cookie is also found in the vm.args file, it will most likely be erlang.
The file name parameter should be a fully-qualified path to the actual file name (meaning, you can't give it just a directory name). The filename and extension are arbitrary. So, I would use something like cluster_backup.riak.
So, to put it all together, your backup command should look like:
riak-admin backup riak#<your node ip> riak /var/local/temp/cluster_backup.riak all
Now, having said all that, I don't recommend using the riak-admin backup and restore commands to back up your whole cluster. For several reasons. One, it stores every replica of every object. Meaning, if you're running with the default replica value of n=3, you will be storing 3 copies of each object in your backup file.
Two, the code invoked by that command is single-threaded, and not connection pooled. So all in all, it's going to be SLOW to restore and backup.
Instead, I recommend one of the following approaches:
Take filesystem level snapshots of the data directories of each node. This is the approach currently recommended by Basho, and detailed here: http://docs.basho.com/riak/latest/ops/running/backups/
If you definitely want a "logical" backup (meaning, an export of the objects contained in the cluster), you can use an experimental standalone tool such as the Riak Data Migrator (but see the limitations in the Readme).
I recommend testing out / timing each of these approaches, to see which one is faster for your situation.
Related
I have a problem. I created a pool consisting of single volume of 1 file 2.5Tb just to fight with file duplicates. I copied a folder with photos. Some of the photos were not backed up. Just now I see my pool folder is empty. When I checked with 'sudo zfs list' it said 'No datasets available'.
I thought it was detached and to attach I started again all these commands.
sudo zpool create singlepool -f /home/john/zfsvolumes/zfs_single_volume.dat -m /home/share/zfssinglepool
sudo zfs set dedup=on singlepool
sudo zpool get dedupratio singlepool
sudo zfs set compression=lz4 singlepool
sudo chown -R writer:writer /home/share/zfssinglepool
I see now empty pool!
May I get my folders back which I copied to the pool before I started create pool again?
Unfortunately, use of zpool create -f will recreate the pool from scratch even if ZFS recognizes that a pool has already been created using that storage:
-f Forces use of vdevs, even if they appear in use or specify a
conflicting replication level. Not all devices can be over-
ridden in this manner.
This is similar to reformatting a partition with other file systems, which will leave whatever data is there written in place, but still erase the references the file system needs to find the data. You may be able to pay an expert to reconstruct your data, but otherwise I'm afraid the data will be very hard to get back from your pool. As in any data recovery mission, I'd advise making a copy of the data ASAP on some external media that you can use to do the recovery from, in case further attempts at recovery accidentally corrupt the data even worse.
I use Redis, and today I start to get the following exception:
Can't save in background: fork: Cannot allocate memory
As I understand, this error appears because my DB is too big, and there is no memory for this process.
So I start to delete tables, but the problem is that Redis doesn't success to write it to the disc, and in face it doesn't know about this changes.
I decided to create new .rdb file (in /etc/redis.config), and then change the file path with the new RDB file:
dbfilename dump_cache_new.rdb
Then, I will reload all the data which critical to me (I can do it - its data from my file system), and restart redis service.
The problem is that I can't create this file, because redis is now executing with the old path (and Redis has to run, because other process takes some critical data from it).
How can I create this dump_cache_new.rdb file, while redis is still running with the old path?
If you want to change the snapshot file name (or most other configuration parameters) on a running instance of Redis, use the CONFIG SET command. Based on that documentation page, it looks like dir and dbfilename are both parameters than can be set on a live instance.
Another option to consider is using the synchronous SAVE command, which doesn't require a fork.
You almost never want to call SAVE in production environments where it will block all the other clients. Instead usually BGSAVE is used. However in case of issues preventing Redis to create the background saving child (for instance errors in the fork(2) system call), the SAVE command can be a good last resort to perform the dump of the latest dataset.
It's a pretty severe operation, but if you're already at the point of dumping data to make the save work, this would at least allow you to first make a snapshot.
I am using Redis since last 12 months without any issue, But from last 30 days unknowingly the database getting empty and we couldn't find any logs regarding this. Even it is flushing all the data out randomly after restoration.
We tried following steps to resolve this but result was zero.
We have checked redis logs
Monitored the redis using MONITOR command
We are trying to renaming the critical commands through config but redis is dump after the config change below is example command
rename-command FLUSHDB e0cc96ad2eab73c2c347011806a76b73
We gone made without knowing anything. Helps are appreciated.
Redis Version : 2.8.17
Running under Debian Linux
Renaming the command through config file will work in this case.
Same rename command you have to place inside the config file.
rename-command FLUSHDB e0cc96ad2eab73c2c347011806a76b73
How can I dump a redis that's running on database 0 and restore it in my local machine on a different database (8) ?
I already secure copied the dump file:
scp hostname#/var/lib/redis/dump.rdb .
But if I change my local redis dump.rdb with this one, I'll get the data on database 0. How can I restore it to a specific database?
Firstly note that the use of numbered/shared Redis databases is inadvisable. You really should consider using dedicated Redis servers with a single DB (0) on them (more info at: https://redislabs.com/blog/benchmark-shared-vs-dedicated-redis-instances)
Redis does not offer a straightforward way to do this, but there are two basic ways one could go about it:
Pre-processing: modify the dump.rdb file to load into your database of choosing. You could build a tool for that or perhaps use one of the existing ones. Jan-Erik has done an outstanding job of documenting the RDB v7 format at http://rdb.fnordig.de/file_format.html so all you need to do is basically change the Database Selector byte.
Post-restore: use the MOVE command on the output of SCANing your restored database - should be easily scriptable.
I ended up creating a script in Ruby to dump and restore the keys I wanted. (Please note that this approach is slow, takes around 1 min for 200 keys) .
Get the keys to dump / restore
ssh hostname redis-cli --scan --pattern 'awesome_filter_pattern*'
Open an ssh connection to the production server
Dump the remote key
dump = ssh.exec!("redis-cli dump #{key}").chomp
Restore it on localhost
$redis.connection.restore(key, 0, dump)
I have gone through the docs and I haven't been able to fully understand them.
My question :
For testing purpose we created a replica of a database in hsqldb and would want to use this as inprocess db for unit testing.
Is there a way that I can distribute the replica database so that people connect to this db.I have used the backup command and have the tar file.But how do I open a connection to the db which takes this backed up db... something on the lines of handing a .mdb file in case of Access to another user and asking him/her to use that.
Regards,
Chetan
You need to expand the backed up database using standard gzip / unzip tools, before you can connect to it.
The HSQLDB Jar can be used to extract the database files from the backup file. For example:
java -cp hsqldb.jar org.hsqldb.lib.tar.DbBackup --extract tardir/backup.tar dbdir
In the example, the first file path is the backup file, and the second one, dbdir, is the directory path where the database files are expanded.