After writing some data to a redis server, I could read the data from a client.
However, how can I find the data directory on the file system?
Quickest method: use redis-cli.
redis-cli config get dir
If you have authentication configured, you will need to pass that in using -a password Replacing "password" with your password.
Find your Redis configuration directory, probably /etc/redis. Then look in the config file called redis.conf and find the line that starts dir.
It will look similar to this:
dir /etc/redis/database
This will do the job slowly but surely if you can't be bothered to look :-)
sudo find / -name "redis.conf" -exec grep "^dir" {} \; 2> /dev/null
dir /etc/redis
or if you want the config filename as well:
sudo find / -name "redis.conf" -exec grep -H "^dir" {} \; 2> /dev/null
/private/etc/redis/redis.conf:dir /etc/redis
Other possibilities you can check are whether Redis was started with a custom config file as its first parameter like this:
redis-server /path/to.custom/config-file
or with the dir option set on the commandline like this:
redis-server dir /path/to/data
Use
ps -aef | grep redis
to look for these options.
Because Redis config file can be located in several possible places (depending on the system or container, such as /opt/redis/ in my case), a general solution to find the currently configured location of the RDB file (as set using dir in redis.conf - if config file is used at all*) is:
$ cat $(cd / && find | grep redis.conf) | grep dir
*Note that true to its simplicity Redis by default ships without any config files (using built-in configuration which depends on the version, see docs), and this is indeed the case for the official redis:latest container.
Related
I have about a thousand files on a remote server (all in different directories). I would like to scp them to my local machine. I would not want to run scp command a thousand times in a row, so I have created a text file with a list of file locations on the remote server. It is a simple text file with a path on each line like below:
...
/iscsi/archive/aat/2005/20050801/A/RUN0010.FTS
/iscsi/archive/aat/2006/20060201/A/RUN0062.FTS
/iscsi/archive/aat/2013/20130923/B/RUN0010.FTS
/iscsi/archive/aat/2009/20090709/A/RUN1500.FTS
...
I have searched and found someone trying to do a similar but not the same thing here. The command I would like to edit is below:
cat /location/file.txt | xargs -i scp {} user#server:/location
In my case I need something like:
cat fileList.txt | xargs -i scp user#server:{} .
To download files from a remote server using the list in fileList.txt located in the same directory I run this command from.
When I run this I get an error: xargs: illegal option -- i
How can I get this command to work?
Thanks,
Aina.
You get this error xargs: illegal option -- i because -i was deprecated. Use -I {} instead (you could also use a different replace string but {} is fine).
If the list is remote, the files are remote, you can do this to retrieve it locally and use it with xargs -I {}:
ssh user#server cat fileList.txt | xargs -I {} scp user#server:{} .
But this creates N+1 connections, and more importantly this copies all remote files (scattered in different directories you said) to the same local directory. Probably not what you want.
So, in order to recreate a similar hierarchy locally, let's say everything under /iscsi/archive/aat, you can:
use cut -d/ to extract the part you want to be identical on both sides
use a subshell to create the command that creates the target directory and copies the file there
Thus:
ssh user#server cat fileList.txt \
| cut -d/ -f4- \
| xargs -I {} sh -c 'mkdir -p $(dirname {}); scp user#server:/iscsi/archive/{} ./{}'
Should work, but that's starting to look messy, and you still have N+1 connections, so now rsync looks like a better option. If you have passwordless ssh connection, this should work:
rsync -a --files-from=<(ssh user#server cat fileList.txt) user#server:/ .
The leading / is stripped by rsync and in the end you'll get everything under ./iscsi/archive/....
You can also copy the files locally first, and then:
rsync -a --files-from=localCopyOfFileList.txt user#server:/ .
You can also manipulate that file to remove for example 2 levels:
rsync -a --files-from=localCopyOfFileList2.txt user#server:/iscsi/archive .
etc.
Someone before me setup a redis instance (version 2.6).
But for some reason, whoever set this, had
Placed the config file etc like this /etc/redis.conf
The dir config has ./ set, like this dir ./
The instance is being run as non-root.
Like this:
$ ps aux | grep "redis"`
user /home/user/redis-stable/src/redis-server /etc/redis.conf
Logging is going to /dev/null, because daemonize yes && logfile stdout
So it is unable to create backups in /etc/ because it doesn't have permissions (I'm guessing), and I can't even see what is going on because the logs are going to /dev/null.
I want to make a backup so I can turn redis off to fix all these things, without losing any data. Any ideas?
I've tried:
touch /etc/dump.rdb
chown user:users /etc/dump.rdb
But it is still not able to write. I'm guessing it might have a temp file it tries to write to before it moves it to /etc/dump.rdb
After looking at Redis source code, it does seem like there is a temp file: https://github.com/antirez/redis/blob/04542cff92147b9b686a2071c4c53574771f4f88/src/rdb.c#L986
snprintf(tmpfile,256,"temp-%d.rdb", (int) getpid());
Also tried
redis 127.0.0.1:6379> config set logfile /home/user/redis.log
(error) ERR Unsupported CONFIG parameter: logfile
Run:
config get dir
and you would see the directory where redis is saving rdb.
Run:
config set dir /home/user/
to change the rdb dump directory to /home/user.
then run:
redis-cli -p <port> bgsave
This would initiate a rdb dump.
Hope this helps.
Httpd processes use a non-default configuration file if they are run with the -f flag.
For example
/home/myuser/apache/httpd-2.4.8/bin/httpd -f /confFiles/apache/2.4.8/apache.conf -k start
will use this configuration file: /confFiles/apache/2.4.8/apache.conf
I need to get this location and would rather not have to check for possible -f flags used to start httpd.
The answer here says to run /path/to/httpd -V and concatenate
-D SERVER_CONFIG_FILE="conf/httpd.conf"
with
-D HTTPD_ROOT="/etc/httpd"
to get the final path to the config file.
However, this path will not be the correct one if the -f flag is used to start the httpd process.
Is there a command that can get the config file that is actually being used by the process?
The answer you refer to mentions the paths httpd was compiled with, but as you say those can be manually changed with parameters.
The simple way to check is the command line, if process is called "httpd" (standard name), a simple ps will reveal the config file being used:
ps auxw | grep httpd
Or querying the server if server has mod_info loaded, in command line or with your favourite browser:
curl "http://yourserver.example.com/server-info?server" | grep -i "config file"
Note: mod_info should not be publicaly available for everyone to see.
I'm trying to run rsync over ssh in parallel to transfer files between two machines for evaluation purposes. I wanna see how faster can I get compared to a single rsync process.
I tried these two solutions:
https://wiki.ncsa.illinois.edu/display/~wglick/Parallel+Rsync but with no great success.
https://gist.github.com/rcoup/5358786 (I couldn't make it work)
Based on the first link I run a command like this:
ssh HOST "mkdir -p ~/destdir/basefolder"
cd ./basefolder; ls | xargs -n1 -P 4 -I% rsync -arvuz -e ssh % HOST:~/destdir/basefolder/.
and I get the files transfered, but it doesn't seem to work well... In this case, It will run a process for every file and folder in the basefolder, but when it finds a folder, it will transfer everything inside that folder using only 1 process.
I tried to use find -type f, but I got problems because I loose the file hierarchy.
Does anyone how some methods to do what I want? (Use rsync in parallel over ssh while keeping files and folders hierarchy).
Since you tagged your question 'gnu-parallel' the obvious is to refer you to http://www.gnu.org/software/parallel/man.html#EXAMPLE:-Parallelizing-rsync
cd src-dir; find . -type f -size +100000 | parallel -v ssh fooserver mkdir -p /dest-dir/{//}\;rsync -Havessh {} fooserver:/dest-dir/{}
Context
I'm using redis. The database is < 100 MB.
However, I want to make daily backups.
I'm also running on Ubuntu Server 12.04
When type in:
redis-cli save
I don't know where dump.rdb is saved to (since redis is started as a service and not in my local directory).
Questions:
How do I find where redis is saving my dump.rdb to?
Is there someway that I can specify a filename to 'save', so I type in something like:
redis-cli save ~/db-2012-06-24.rdb
Thanks
To be a little more helpfull... How to find or set where redis is saving the dump.rdb file (ubuntu server):
First find you redis.conf file: In your terminal run:
ps -e aux | grep redis
I found my redis.conf file in:
var/etc/redis/
If yours is the same place then open the file with:
pico var/etc/redis/redis.conf
Look for:
# The filename where to dump the DB
dbfilename dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# Also the Append Only File will be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir /var/lib/redis
Depending on your setting for "dbfilename" and "dir" then that is where you find your redis dump.rdb file.
Update:
To see your redis configurations just run:
redis-cli CONFIG GET *
You can set the file location on the redis.conf file (which you start the server with)
look at the server configuration for that:
# The filename where to dump the DB
dbfilename dump.rdb
finding the location of the currently saved file, it depends on how you start the server - where you have the redis-server file - i think you can find it with ps -e aux | grep redis or ps -e | grep redis
On my (default, Ubuntu) setup the db file is in
/var/lib/redis/redis.rdb
As Christoffer points out, you can see all the settings from the command-line client with
CONFIG GET *
One liner to get both directory and dump file name
echo "CONFIG GET *" | redis-cli | grep -e "dir" -e "dbfilename" -A1
In mac,
the location of dump.rdb is at /usr/local/etc/dump.rdb.
the location of redis.conf is at /usr/local/etc/redis.conf.
In order to find the location use the command find - sudo find / -name "redis.conf"