Our system is configured Ambari with some database. But I am not sure which database is backed up for Ambari.
When I went through the installation procedure, I could see the steps for MariaDB and MySQL is installed.
When I checked ambari.properties file under the path '/etc/ambari-server/conf', I could see the line with server.jdbc.database=mysql
Basically, How to check which database is connected for Ambari and how to confirm?
grep db /etc/ambari-server/conf/ambari.properties | grep name
custom.mysql.jdbc.name=mysql-connector-java.jar
previous.custom.mysql.jdbc.name=mysql-connector-java.jar
server.jdbc.database_name=ambari
server.jdbc.rca.user.name=ambari
server.jdbc.user.name=ambari
grep db /etc/ambari-server/conf/ambari.properties | grep database
server.jdbc.database=mysql
server.jdbc.database_name=ambari
Ambari database name should be in /etc/ambari-server/conf/ambari.properties, along with other connection details to the db.
You can use something like following to grep the db name:-
grep db /etc/ambari-server/conf/ambari.properties | grep name
Related
I'm trying to import data from .csv file to a PostgreSQL database hosted in a Linux Server, using the following command:
COPY areas_brasil FROM 'C:/Temp/RELATORIO_DTB_BRASIL_MUNICIPIO.csv' with delimiter '|' null 'NULL';
But i'm receiving the following error:
ERROR: could not open file
"C:/Temp/RELATORIO_DTB_BRASIL_MUNICIPIO.csv" for reading: No such file
or directory TIP: COPY FROM instructs the PostgreSQL server process
to read a file. You may want a client-side facility such as psql's
\copy.
The .csv file is in a client computer (running on Windows 10) in which i have administrator access to the database hosted in the server (running on Linux - Debian).
Thanks for helping me!
Welcome to SO.
COPY .. FROM 'path' assumes that the file is located in the server. If you wish execute COPY without having the file into the database server, you can either use \copy or just use the STDIN of psql from your client console, e.g. in unix systems (you have to find the cat and | equivalent for Windows):
$ cat file.csv | psql yourdb -c "COPY areas_brasil FROM STDIN DELIMITER '|';"
Using \COPY inside of psql it can be done like this:
\COPY areas_brasil FROM '/home/jones/file.csv' DELIMITER '|';
See this answer for more details.
Files are being written to a directory using the COPY query:
Copy (SELECT * FROM animals) To '/var/lib/postgresql/data/backups/2020-01-01/animals.sql' With CSV DELIMITER ',';
However if the directory 2020-01-01 does not exist, we get the error
could not open file "/var/lib/postgresql/data/backups/2020-01-01/animals.sql" for writing: No such file or directory
PostgeSQL server is running inside a Docker container with the volume mapping /mnt/backups:/var/lib/postgresql/data/backups
The Copy query is being sent from a Node.js app outside of the Docker container.
The mapped host directory /mnt/backups was created by Docker Compose and is owned by root, so the Node.js app sending the COPY query is unable to create the missing directories due to insufficient permissions.
The backup file is meant to be transferred out of the Docker container to the Docker host.
Question: Is it possible to use an SQL query to ask PostgreSQL 11.2 to create a directory if it does not exist? If not, how will you recommend the directory creation be done?
Using Node.js 12.14.1 on Ubuntu 18.04 host. Using PostgreSQL 11.2 inside container, Docker 19.03.5
An easy way to solve it is to create the file directly into the client machine. Using STDOUT from COPY you can let the query output be redirected to the client standard output, which you can catch and save in a file. For instance, using psql in the client machine:
$ psql -U your_user -d your_db -c "COPY (SELECT * FROM animals) TO STDOUT WITH CSV DELIMITER ','" > file.csv
Creating an output directoy in case it does not exist:
$ mkdir -p /mnt/backups/2020-01/ && psql -U your_user -d your_db -c "COPY (SELECT * FROM animals) TO STDOUT WITH CSV DELIMITER ','" > /mnt/backups/2020-01/file.csv
On a side note: try to avoid exporting files into the database server. Although it is possible, I consider it a bad practice. Doing so you will either write a file into the postgres system directories or give the postgres user permission to write somewhere else, and it is something you shouldn't be comfortable with. Export data directly to the client either using COPY as I mentioned or follow the advice from #Schwern. Good luck!
Postgres has its own backup and restore utilities which are likely to be a better choice than rolling your own.
When used with one of the archive file formats and combined with pg_restore, pg_dump provides a flexible archival and transfer mechanism. pg_dump can be used to backup an entire database, then pg_restore can be used to examine the archive and/or select which parts of the database are to be restored. The most flexible output file formats are the “custom” format (-Fc) and the “directory” format (-Fd). They allow for selection and reordering of all archived items, support parallel restoration, and are compressed by default. The “directory” format is the only format that supports parallel dumps.
A simple backup rotation script might look like this:
#!/bin/sh
table='animals'
url='postgres://username#host:port/database_name'
date=`date -Idate`
file="/path/to/your/backups/$date/$table.sql"
mkdir -p `dirname $file`
pg_dump $url -w -Fc --table=$table -f $file
To avoid hard coding the database password, -w means it will not prompt for a password and instead look for a password file. Or you can use any of many Postgres authentication options.
I'm configuring live backup and restore scripts to have "replicated" firebird dbs on main and reserve servers.
Backup doing fine:
"C:\Program Files\Firebird\Firebird_2_5\bin\nbackup" -B 0 "D:\testdb\LABORATORY_DB.FDB" D:\testdb\lab_FULL.fbk -user SYSDBA -pass masterkey -D OFF
Copying file to the remote server as well:
net use R: \\fbserv2\reserve
xcopy /Y D:\testdb\lab_FULL.fbk R:\
But restoring on remote side
"C:\Program Files\Firebird\Firebird_2_5\bin\fbsvcmgr.exe" fbserv2:service_mgr -user SYSDBA -password masterkey -action_nrest -dbname d:\reservedb\LABORATORY_DB.FDB -nbk_file d:\reserve\lab_FULL.fbk
caused an error:
Error (80) creating database file: d:\reservedb\LABORATORY_DB.FDB via copying from: d:\reserve\lab_FULL.fbk
The only way to restore database is to manually delete an old d:\reservedb\LABORATORY_DB.FDB before restoring. GBAK has the option to overwrite restorig db file, while fbsvcmgr seems to be not. Is there any other option? Did I miss something?
You can't restore over an existing database using nbackup. You either need to
delete the old database first and then restore,
or restore under a different name, delete the old database, and rename the new database to its final name.
See also the nbackup documentation, chapter Making and restoring backups:
If the specified database file already exists, the restore fails and you get an error message.
As far as I know it was a design decision to not allow overwriting an existing database. Gbak indeed has that option, but only for historic reasons; if it were built today, it would likely not have that option.
I am creating some databases using PostgreSQL but I want to save them on an external hard drive due to lack of memory in my computer.
How can I do this?
You can store the database on another disk by specifying it as the data_directory setting. You need to specify this at startup and it will apply to all databases.
You can put it in postgresql.conf:
data_directory = '/volume/path/'
Or, specify it on the command line when you start PostgreSQL:
postgres -c data_directory='/volume/path/'
Reference: 18.2. File Locations
STEP 1: If postgresql is running, stop it:
sudo systemctl stop postgresql
STEP 2: Get the path to access your hard drive.
(if Linux) Find and mount your hard drive by:
# Retrieve your device's name with:
sudo fdisk -l
# Then mount your device
sudo mount /dev/DEVICE_NAME YOUR_HD_DIR_PATH
STEP 3: Copy the existing database directory to the new location (in your hard drive) with rsync.
sudo rsync -av /var/lib/postgresql YOUR_HD_DIR_PATH
Then rename the previous postgres main dir with .bak extension to prevent conflicts
sudo mv /var/lib/postgresql/11/main /var/lib/postgresql/11/main.bak
Note: my postgres version was 11. Replace in path with your version.
STEP 4: Edit postgres configuration file:
sudo nano /etc/postgresql/11/main/postgresql.conf
Change the data_directory line with:
data_directory = 'YOUR_HD_DIR_PATH/postgresql/11/main'
STEP 5: Restart Postgres & Check everything is working
sudo systemctl start postgresql
pg_lsclusters
Output should shows status as 'online'
Ver Cluster Port Status Owner Data directory Log file
11 main 5432 online postgres YOUR_HD_DIR_PATH/postgresql/11/main /var/log/postgresql/postgresql-11-main.log
Finally your can access your PostgresSQL with:
sudo -u postgres psql
You can try following the walkthrough here. This worked well for me and is similar to #Antiez's answer.
Currently, I am trying to do the same and the only conflict that I'm having at the moment is that it seems there is an issue with PostgreSQL's incremental backup and point-in-time recovery proccesses. I think it has something to do with folder permissions. If I try uploading a ~30MB csv to the postgres db, it will crash and the server will not start again because files cannot be written to the pg_wal directory. The only file in that directory is 000000010000000000000001 and does not move on to 000000010000000000000002 etc. while writing to a new table.
My stackoverflow post looking for a solution to this issue can be found here.
I have an Apache server that has multiple files located on it. Each file is an updated version from the previous file. Eg. release-2013-07-20 is an older file and release-2013-07-25 is the newer file.
I am looking to query the apache server and have an display of the most updated file via website link.
I am unsure how to go about accomplishing this and I am looking for some suggestions.
If you're running on a unix server, the following command on the relevant directory will return the name of the file that was last modified (most updated):
ls -ltr | grep ^- | awk '{print $9}' | sed -n 1p
you can use this filename in order to copy or generate a link to it