I am profiling a query with cypher-shell, since according to this post it is easier to be compared.
My command is something like the following (the query is different).
sudo cypher-shell -a myip:myport -u myusr -p 'my_pwd' "EXPLAIN MATCH (t1:Node1{field:1})−[:rel1]−>
( : Node2 {ID :"01"})<−[:rel1]−(t2:Trip{Direction:0})
RETURN t1, t2 LIMIT 1;"
This command print the profiling of the query to Terminal. However, it is difficult to read on the Terminal so I was trying to redirect the output to file such as:
sudo cypher-shell -a myip:myport -u myusr -p 'my_pwd' "EXPLAIN MATCH (t1:Node1{field:1})−[:rel1]−>
( : Node2 {ID :"01"})<−[:rel1]−(t2:Node2{field:0})
RETURN t1, t2 LIMIT 1;" >> out.txt
However, the resulting file contains no profiling but only a couple of lines such as:
Plan: "EXPLAIN"
Statement: "READ_ONLY"
Version: "CYPHER 3.5"
Planner: "COST"
Runtime: "INTERPRETED"
Time: 0
How to redirect neo4j profiling output to file?
Add the --format verbose flag to the command.
sudo cypher-shell -a myip:myport -u myusr -p 'my_pwd' --format verbose "EXPLAIN MATCH (t1:Node1{field:1})−[:rel1]−>
( : Node2 {ID :"01"})<−[:rel1]−(t2:Node2{field:0})
RETURN t1, t2 LIMIT 1;" >> out.txt
cypher-shell seems to default to --format plain if output is being redirected to a file.
Example output for a different query but the same command as above - not sure it's that much more readable
Related
I'm not sure if this is possible of if I'm doing something wrong since I'm still pretty new to Docker. Basically, I want to export a query result inside PostgreSQL docker container as a csv file to my local machine.
This is where I got so far. Firstly, I run my PostgreSQL docker container with this command:
sudo docker run --rm --name pg-docker -e POSTGRES_PASSWORD=something -d -p 5432:5432 -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data postgres
Then I access the docker container with docker exec to run PostgreSQL command that would copy the query result to a csv file with specified location like this:
\copy (select id,value from test) to 'test_1.csv' with csv;
I thought that should export the query result as a csv file named test_1.csv in the local machine, but I couldn't find the file anywhere in my local machine, also checked both of these directories: $HOME/docker/volumes/postgres; /var/lib/postgresql/data postgres
You can export the data to the STDOUT and pipe the result to a file in the client machine:
docker exec -it -u database_user_name container_name \
psql -d database_name -c "COPY (SELECT * FROM table) TO STDOUT CSV" > output.csv
-c tells psql you to execute a given SQL statement when the connection is established.
So your command should look like this:
docker exec -it -u postgres pgdocker \
psql -d yourdb -c "COPY (SELECT * FROM test) TO STDOUT CSV" > test_1.csv
The /var/lib/postgresql/data directory is where the database server stores its data files. It isn't a directory that users need to manipulate directly or where nothing interesting can be found.
Paths like test_1.csv are relative to working directory. The default directory when you enter the postgres container with docker exec is / so that's where your file should be. You can also switch to another directory with cd before running psql:
root#b9e5a0572207:/some/other/path# cd /some/other/path/
root#b9e5a0572207:/some/other/path# psql -U postgres
... or you can provide an absolute path:
\copy (select id,value from test) to '/some/other/path/test_1.csv' with csv;
You can use docker cp to transfer a file from the container to the host:
docker cp pg-docker:/some/other/path/test_1.csv /tmp
... or you can create a volume if this is something you do often.
Im running the next postgres query using the next bash command.
sudo -u postgres bash -c "psql -d db -c \"SELECT ip FROM db_accounts;\"" \>/dev/null
The output is a table but before the table is printed, I get the following info prints
> psql: /usr/lib64/libssl.so.10: no version information available
> (required by psql) psql: /usr/lib64/libcrypto.so.10: no version
> information available (required by /usr/pgsql-9.4/lib/libpq.so.5)
> psql: /usr/lib64/libssl.so.10: no version information available
> (required by /usr/pgsql-9.4/lib/libpq.so.5)
I want to run my command without these prints appearing.
I tried to change the end of the command >/dev/null to 2>/dev/null and indeed the prints were disable but my table was not fully displayed (out of 800 rows only 40 were displayed),
Can someone help me please?
Use --quiet when you start psql
OR
It can be set in your postgresql.conf file by adding this
client_min_messages = warning
This blog is really helpful.
To fix want I wanted I added --pset pager=off to the psql to get the whole table and the disable the prints I change the end of the command to 2>/dev/null
final command:
sudo -u postgres bash -c "psql --pset pager=off --quiet -d db -c \"SELECT ip FROM db_accounts;\"" 2>/dev/null
I need to execute the following sql queries from bash/expect script
what is the preferred approach to run these queries from bash script
# psql ambari -U ambari
Password for user ambari:
psql (9.2.24)
Type "help" for help.
ambari=>
ambari=>
ambari=>
ambari=> select
ambari-> sum(case when ulo = 1 then 1 else 0 end) as ulo_1,
ambari-> sum(case when ulo = 2 then 1 else 0 end) as ulo_2,
.
.
.
for access PostgreSQL we do
psql ambari -U ambari
Password for user ambari:bigdata
and when we run this ( /tmp/file include the bach of the query )
psql -U ambari -f /tmp/file ambari
we get
psql: FATAL: no pg_hba.conf entry for host "[local]", user "ambari", database "ambari", SSL off
I'm using this
dbhost=localhost
dbport=5432
dbuser=user
dbpass=pass
dbname=test
export PGPASSWORD="$dbpass"
dbopts="-h $dbhost -p $dbport -U $dbuser -d $dbname"
Then run sql script from file
psql $dbopts < "$path_to_sql_script"
Or from query var
query="
SELECT 1;
...
"
psql $dbopts <<< "$query"
Also pgpass can be set in special file ~/.pgpass like this
echo "$dbhost:$dbport:$dbname:$dbname:$dbpass" > ~/.pgpass
chmod 600 ~/.pgpass
Use switches -c command or -f filename, ie.:
$ psql -U ambari -c "SELECT ... ;" ambari # > result.file
or:
$ cat file.sql
SELECT
... ;
$ psql -U ambari -f file.sql ambari # > result.file
Probably -f as your query seems lengthy. Use > result.file to store the query result to a file.
As for the password, store following kind of entry to .pgpass file in user's home dir:
$ cat >> ~/.pgpass
#hostname:port:database:username:password
localhost:5432:ambari:ambari:t00M4NY53CR3t5
and set its rights to user's eyes only:
$ chmod 600 ~/.pgpass
Also, consider psql -h hostname if the database is not running in localhost (this needs to reflect in .pgpass entry as well).
I am creating a VPS with the API provided for command line. The output of the command comes with several text inside which I don't need. This is my command.
The variables are predefined and work fine.
echo y | /usr/local/bin/CLICMD vm create --hostname=$VMNAME --domain=$srvdomain --cpu 1 --memory 1024 --image $image --datacenter=$dc --billing=hourly -n 100 > /dev/null 1>> /home/logs/createvps.log
When I run it, it gives me the following output in createvps.log file,
This action will incur charges on your account. Continue? [y/N]: id 11232312
created 2015-06-13T14:43:27-05:00
guid xxxxxx-r345-4323-8e3f-c8c04e18fad7
From the above output, I just need to have id (11232312) value stored in a mysql table. I know how to grab the value from log file and save in mysql.
My question is, how do I save just that id in the log file instead of all the other values/strings.
Thank you in advance.
Not sure what is exactly your question, but I guess this should help you:
echo y | /usr/local/bin/CLICMD vm create --hostname=$VMNAME \
--domain=$srvdomain --cpu 1 --memory 1024 --image $image \
--datacenter=$dc --billing=hourly -n 100 | \
grep -oE "id [0-9]+$" | grep -Eo "[0-9]+" >> /home/logs/createvps.log
Few notes to difference in your code and mine:
You do two redirection of stdout, one to /dev/null and one to your log, which is equivalent of doing just one redirection (writing in /dev/null is practically NOP).
I'm looking to be able to run a single query on a remote server in a scripted task.
For example, intuitively, I would imagine it would go something like:
mysql -uroot -p -hslavedb.mydomain.com mydb_production "select * from users;"
mysql -u <user> -p -e 'select * from schema.table'
(Note the use of single quotes rather than double quotes, to avoid the shell expanding the * into filenames)
mysql -uroot -p -hslavedb.mydomain.com mydb_production -e "select * from users;"
From the usage printout:
-e, --execute=name
Execute command and quit. (Disables --force and history file)
here's how you can do it with a cool shell trick:
mysql -uroot -p -hslavedb.mydomain.com mydb_production <<< 'select * from users'
'<<<' instructs the shell to take whatever follows it as stdin, similar to piping from echo.
use the -t flag to enable table-format output
If it's a query you run often, you can store it in a file. Then any time you want to run it:
mysql < thefile
(with all the login and database flags of course)
echo "select * from users;" | mysql -uroot -p -hslavedb.mydomain.com mydb_production
As by the time of the question containerization wasn't that popular, this is how you pass a single query to a dockerized database cluster with Ansible, following #RC.'s answer:
ansible <host | group > -m shell -a "docker exec -it <container_name | container_id> mysql -u<your_user> -p<your_pass> <your_database> -e 'SELECT COUNT(*) FROM my_table;'"
If not using Ansible, just login to the server and use docker exec -it ... part.
MySQL will issue a warning that passing credentials in plain text may be insecure, so be aware of your risks.
From the mysql man page:
You can execute SQL statements in a script file (batch file) like this:
shell> mysql db_name < script.sql > output.tab
Put the query in script.sql and run it.