I want to run query stored file in beeline. This code works OK in putty.
beeline -u "hiveserver" -n "username" -p "password" --outputformat=csv2 --silent=true -e "select * from table;" >output1.txt
When I save sql command to query.hql or query.sql and upload to server where hadoop is, command does not export anything. I get no error.
beeline -u "hiveserver" -n "username" -p "password" --outputformat=csv2 --silent=true -f query.hql >output1.txt
Query in file works when I run it as !run query.hql directly in beeline.
What is wrong with my query in file approach?
Make sure you have a new line character at the end of the file. Otherwise, beeline will not execute that command rather will just print onto the beeline terminal. Please let me know if that works.
Please, check if below is the case.
Related
I am trying to open a random .sql file off the internet using the following command:
psql -h localhost -d database_name -U postgres < file_name.sql
But when I run this command I just get errors like the following:
invalid command 's
invalid command 's
invalid command 'll
invalid command 'Moving
invalid command 's
invalid command "frequently
It just continuously prints out these invalid command error messages. I thought it might be an encoding problem but I confirmed the file is UTF-8 encoded.
Any suggestions on how I can open this file
To expand and clarify on a_horse_with_no_name's comment - the psql command you are running should be run directly in your shell, not inside pgadmin4.
youruser#yourmachine:~$ psql -h localhost -d database_name -U postgres < file_name.sql
That command should load the contents of file_name.sql in to database_name. Once it's complete, you can use pgadmin4 as normal to interact with the database.
One possibility is that the file contains tabulator keys, which are expanded if you read redirect standard input to the SQL script.
Try using the -f option:
psql -h localhost -d database_name -U postgres -f file_name.sql
Apparently the .sql file was generated through a MySQL dump. I thought it would not matter whether I used PostgreSQL or MySQL but it did. Once I installed MySQL my problem got resolved and I now have a Database ready :)
I want to create a shell script which runs a Netezza SQL and create lookup file dbnames.lkp in a unix directory
lookup file name : dbnames.lkp
Could someone help me with nzsql command. I know just to connect to Netezza server using nzsql command and then run the queries. But my requirement is that the shell script should run connect and write the result of SQL query to a lookup file
nzsql -u $user -pw $password -d $db -host $hostname
You can query Netezza System view _v_database to fetch all databases in the connected server, provided the user should have access.
You can create a shell script with below command and this will take care of it.
user=<user_name>
password=<password>
query="select database from _v_database;"
file=dbnames.lkp
dbname=<dbname_to_connect>
hostname=<server>
nzsql -u $user -pw $password -d $dbname -host $hostname -o $file -A -t -c "$query"
-c : will run only a single query and exit nzsql.
You can get more information from below link:
https://www.ibm.com/support/knowledgecenter/en/SSULQD_7.1.0/com.ibm.nz.adm.doc/r_sysadm_nzsql_command.html
I need to execute SQL from batch file.
I am executing following to connect to Postgres and select data from table
C:/pgsql/bin/psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME%
select * from test;
I am able to connect to database, however I'm getting the error
'select' is not recognized as an internal or external command,
operable program or batch file.
Has anyone faced such issue?
This is one of the query i am trying, something similar works in shell script, (please ignore syntax error in the query if there are any)
copy testdata (col1,col2,col3) from '%filepath%/%csv_file%' with csv;
You could pipe it into psql
(
echo select * from test;
) | C:/pgsql/bin/psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME%
When closing parenthesis are part of the SQL query they have to be escaped with three carets.
(
echo insert into testconfig(testid,scenarioid,testname ^^^) values( 1,1,'asdf'^^^);
) | psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME%
Use the -f parameter to pass the batch file name
C:/pgsql/bin/psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME% -f 'sql_batch_file.sql'
http://www.postgresql.org/docs/current/static/app-psql.html
-f filename
--file=filename
Use the file filename as the source of commands instead of reading commands interactively. After the file is processed, psql terminates. This is in many ways equivalent to the meta-command \i.
If filename is - (hyphen), then standard input is read until an EOF indication or \q meta-command. Note however that Readline is not used in this case (much as if -n had been specified).
if running on Linux, this is what worked for me (need to update values below with your user, db name etc)
psql "host=YOUR_HOST port=YOUR_PORT dbname=YOUR_DB_NAME user=YOUR_USER_NAME password=YOUR_PASSWORD" -f "fully_qualified_path_to_your_script.sql"
You cannot put the query on separate line, batch interpreter will assume it's another command instead of a query for psql. I believe you will need to quote it as well.
I agree with Spidey:
1] if you are passing the file with the sql use -f or --file parameter
When you want to execute several commands the best way to do that is to add parameter -f, and after that just type path to your file without any " or ' marks (relative paths works also):
psql -h %host% -p 5432 -U %user% -d %dbname% -f ..\..\folder\Data.txt
It also works in .NET Core. I need it to add basic data to my database after migrations.
Kindly refer to the documentation
1] if you are passing the file with the sql use -f or --file parameter
2] if you are passing individual command use -c or --command parameter
If you are trying the shell script
psql postgresql://$username:$password#$host/$database < /app/sql_script/script.sql
I need to write the results of executing a hive query to a file. how do i do it? currently, it's printing to the console.
beeline -u db_url -n user_name -p password -f query.sql
i tried:
beeline -u db_url -n user_name -p password -f query.sql 2> output.txt
but output.txt just contains when connection started and closed, not the results of the query - which are still being printed to the console.
I assume beeline -u db_url -n user_name -p password -f query.sql > output.txt must be OK. Without 2
"2" in your command is errlog, not the stdout
so "...query.sql 2> output.txt" would put the errlog results into your text file, while "...query.sql > output.txt" would put the actual output into the text file.
In addition to #dds 's answer you can try adding the silent feature to get rid of all the other stuffs like the connection started and closed status being printed in the output file.
beeline -u db_url -n user_name -p password --silent=true -f query.sql > output.txt
I think you meant to type "csv2" instead of "csv 2". Here's the fixed command line:
beeline -u db_url -n user_name -p password -f query.sql2 > output.txt
I have a 123MB sql file which I need to execute in my local PC. But I am getting
Cannot execute script: Insufficient memory to continue the execution of the program
How to solve this issue?
use the command-line tool SQLCMD which is much leaner on memory. It is as simple as:
SQLCMD -d <database-name> -i filename.sql
You need valid credentials to access your SQL Server instance or even to access a database
Taken from here.
It might help you! Please see below steps.
sqlcmd -S server-name -d database-name -i script.sql
Open cmd.exe as Administrator.
Create Documents directory.
Put your SQL Script file(script.sql) in the documents folder.
Type query with sqlcmd, server-name, database-name and script-file-name as like above highlighted query or below command line screen.
For Windows Authentication use this sql cmd
SQLCMD -S TestSQLServer\SQLEXPRESS -d AdventureWorks2018 -i "d:\document\sql document\script.sql"
Note: If there is any space in the sql file path then use " (Quotation marks) "
For SQL Server Authentication use this sql cmd
SQLCMD -S TestSQLServer\SQLEXPRESS -U sa -P sasa -d AdventureWorks2018 -i "d:\document\sql document\script.sql"
-S TestSQLServer\SQLEXPRESS: Here specify SQL Server Name
-U sa: Username (in case of SQL Server Authentication)
-P sasa: Password (in case of SQL Server Authentication)
-d AdventureWorks2018: Database Name come here
-i "d:\document\sql document\script.sql": File Path of SQLFile
You can also simply increase the Minimum memory per query value in server properties. To edit this setting, right click on server name and select Properties > Memory tab.
I encountered this error trying to execute a 30MB SQL script in SSMS 2012. After increasing the value from 1024MB to 2048MB I was able to run the script.
(This is the same answer I provided here)
My database was larger than 500mb, I then used the following
C:\Windows>sqlcmd -S SERVERNAME -U USERNAME -P PASSWORD -d DATABASE -i C:\FILE.sql
It loaded everything including SP's
*NB: Run the cmd as Administrator
If I understand your problem correctly, you are trying to restore (transact sql) xyz.sql - database + schema. You can try this command which worked for me:
SQLCMD -U sa -i xyz.sql
Try this step,
1)Open PowerShell
2)Write this command:
sqlcmd -S PCNAME\SQLEXPRESS -U user -P password -d databanse_name -i C:\script.sql
3)Press Return
:-)
Below script works perfectly:
sqlcmd -s Server_name -d Database_name -E -i c:\Temp\Recovery_script.sql -x
Symptoms:
When executing a recovery script with sqlcmd utility, the ‘Sqlcmd: Error: Syntax error at line XYZ near command ‘X’ in file ‘file_name.sql’.’ error is encountered.
Cause:
This is a sqlcmd utility limitation. If the SQL script contains dollar sign ($) in any form, the utility is unable to properly execute the script, since it is substituting all variables automatically by default.
Resolution:
In order to execute script that has a dollar ($) sign in any form, it is necessary to add “-x” parameter to the command line.
e.g.
Original:
sqlcmd -s Server_name -d Database_name -E -i c:\Temp\Recovery_script.sql
Fixed:
sqlcmd -s Server_name -d Database_name -E -i c:\Temp\Recovery_script.sql -x
Sometimes, due to the heavy size of the script and data, we encounter this type of error. Server needs sufficient memory to execute and give the result. We can simply increase the memory size, per query.
You just need to go to the sql server properties > Memory tab (left side)> Now set the maximum memory limit you want to add.
Also, there is an option at the top, "Results to text", which consume less memory as compare to option "Results to grid", we can also go for Result to Text for less memory execution.
sqlcmd -S mamxxxxxmu\sqlserverr -U sa -P x1123 -d QLDB -i D:\qldbscript.sql
Open command prompt in run as administrator
enter above command
"mamxxxxxmu" is computer name
"sqlserverr" is server name
"sa" is username of server
"x1123" is password of server
"QLDB" is database name
"D:\qldbscript.sql" is sql script file to execute in database
If you need to connect to LocalDB during development, you can use:
sqlcmd -S "(localdb)\MSSQLLocalDB" -d dbname -i file.sql
As in most answers given here use the command-line tool. In my case the script already has database creation code. If your script contains CREATE DATABASE command, for example
USE [master]
GO
CREATE DATABASE [your-database-name]
Then do not use the -d your-database-name, instead use the following command.
For Windows Authentication use the command
sqlcmd -S ServerName\InstanceName -i "script.sql" -x
For SQL Server Authentication use the command
sqlcmd -S ServerName\InstanceName -U usename -P password -i "script.sql" -x