Using BCP from mssql-tools on CentOS7 and trying to copy some TSV data into a local MSSQL DB, BCP fails to do the copy and throws error "BCP copy in failed". The command being run is:
TO_SERVER_ODBCDSN="-D -S MyMSSQLServer"
RECOMMEDED_IMPORT_MODE='-c' # makes a big difference, see https://stackoverflow.com/a/16310219/8236733
/opt/mssql-tools/bin/bcp "$TABLE" in "$filename" \
$TO_SERVER_ODBCDSN \
-U $USER -P $PASSWORD \
-d $DB \
$RECOMMEDED_IMPORT_MODE \
-t "\t" \
-e ${filename}.bcperror.log
Yet the error logs created by the command are empty. What does this mean / imply? Anyone have any further debugging tips for resolving the "copy in failed" error?
The errorlog created when you use the -e option is meant to capture errors regarding the data itself. So, the errorlog will contain errors when there is an overflow of data (too many bytes in a field destined for a column with too few).
Execution errors, or errors with the BCP application itself are not captured in the error file created by the -e option.
In an automated environment, if you want to capture or log such errors you will need to redirect the output of the BCP command to a file for viewing later or even loading into a log table in a SQL table.
Related
I am trying to open a random .sql file off the internet using the following command:
psql -h localhost -d database_name -U postgres < file_name.sql
But when I run this command I just get errors like the following:
invalid command 's
invalid command 's
invalid command 'll
invalid command 'Moving
invalid command 's
invalid command "frequently
It just continuously prints out these invalid command error messages. I thought it might be an encoding problem but I confirmed the file is UTF-8 encoded.
Any suggestions on how I can open this file
To expand and clarify on a_horse_with_no_name's comment - the psql command you are running should be run directly in your shell, not inside pgadmin4.
youruser#yourmachine:~$ psql -h localhost -d database_name -U postgres < file_name.sql
That command should load the contents of file_name.sql in to database_name. Once it's complete, you can use pgadmin4 as normal to interact with the database.
One possibility is that the file contains tabulator keys, which are expanded if you read redirect standard input to the SQL script.
Try using the -f option:
psql -h localhost -d database_name -U postgres -f file_name.sql
Apparently the .sql file was generated through a MySQL dump. I thought it would not matter whether I used PostgreSQL or MySQL but it did. Once I installed MySQL my problem got resolved and I now have a Database ready :)
My problem is that I have a cluster-server with Torque PBS and want to use it to run a sequence-comparison with the program rapsearch.
The normal RapSearch command is:
./rapsearch -q protein.fasta -d database -o output -e 0.001 -v 10 -x t -z 32
Now I want to run it with 2 nodes on the cluster-server.
I've tried with: echo "./rapsearch -q protein.fasta -d database -o output -e 0.001 -v 10 -x t -z 32" | qsub -l nodes=2 but nothing happened.
Do you have any suggestions? Where I'm wrong? Help please.
Standard output (and error output) files are placed in your home directory by default; take a look. You are looking for a file named STDIN.e[numbers], it will contain the error message.
However, I see that you're using ./rapsearch but are not really being explicit about what directory you're in. Your problem is therefore probably a matter of changing directory into the directory that you submitted from. When your terminal is in the directory of the rapsearch executable, try echo "cd \$PBS_O_WORKDIR && ./rapsearch [arguments]" | qsub [arguments] to submit your job to the cluster.
Other tips:
You could add rapsearch to your path if you use it often. Then you can use it like a regular command anywhere. It's a matter of adding the line export PATH=/full/path/to/rapsearch/bin:$PATH to your .bashrc file.
Create a submission script for use with qsub. Here is a good example.
I need to execute SQL from batch file.
I am executing following to connect to Postgres and select data from table
C:/pgsql/bin/psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME%
select * from test;
I am able to connect to database, however I'm getting the error
'select' is not recognized as an internal or external command,
operable program or batch file.
Has anyone faced such issue?
This is one of the query i am trying, something similar works in shell script, (please ignore syntax error in the query if there are any)
copy testdata (col1,col2,col3) from '%filepath%/%csv_file%' with csv;
You could pipe it into psql
(
echo select * from test;
) | C:/pgsql/bin/psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME%
When closing parenthesis are part of the SQL query they have to be escaped with three carets.
(
echo insert into testconfig(testid,scenarioid,testname ^^^) values( 1,1,'asdf'^^^);
) | psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME%
Use the -f parameter to pass the batch file name
C:/pgsql/bin/psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME% -f 'sql_batch_file.sql'
http://www.postgresql.org/docs/current/static/app-psql.html
-f filename
--file=filename
Use the file filename as the source of commands instead of reading commands interactively. After the file is processed, psql terminates. This is in many ways equivalent to the meta-command \i.
If filename is - (hyphen), then standard input is read until an EOF indication or \q meta-command. Note however that Readline is not used in this case (much as if -n had been specified).
if running on Linux, this is what worked for me (need to update values below with your user, db name etc)
psql "host=YOUR_HOST port=YOUR_PORT dbname=YOUR_DB_NAME user=YOUR_USER_NAME password=YOUR_PASSWORD" -f "fully_qualified_path_to_your_script.sql"
You cannot put the query on separate line, batch interpreter will assume it's another command instead of a query for psql. I believe you will need to quote it as well.
I agree with Spidey:
1] if you are passing the file with the sql use -f or --file parameter
When you want to execute several commands the best way to do that is to add parameter -f, and after that just type path to your file without any " or ' marks (relative paths works also):
psql -h %host% -p 5432 -U %user% -d %dbname% -f ..\..\folder\Data.txt
It also works in .NET Core. I need it to add basic data to my database after migrations.
Kindly refer to the documentation
1] if you are passing the file with the sql use -f or --file parameter
2] if you are passing individual command use -c or --command parameter
If you are trying the shell script
psql postgresql://$username:$password#$host/$database < /app/sql_script/script.sql
Example of the contents I require a .cmd to contain
mkdir Output
sqlcmd -S serverName -d dbName -E -i "FILE LOCATION HERE" -o Output\Message.log
sqlcmd -S serverName -d dbName -E -i "FILE LOCATION HERE" >> Output\Messages.log
.
.
.
pause
Specifics: I have a SQL repo and need to generate a code that will take all changes from last revision to this revisionand output the above example.
As far as Mercurial is concerned,
hg status -I re:.*\.sql$ -am --rev 3:7
will give the relation of all files with .sql extension added or modified after changeset 3 and up to changeset 7. You can then massage the output into the desired script with your favourite text processing tools.
N.B.: Blindly running all modified SQL scripts, specially if you intend to use this procedure more than once, seems rather dangerous, unless you are sure all the operations are idempotent.
I'm running a bcp code through command line which looks similar to the below
bcp tempdb..temptable out output.txt -S Servername -i, -U username –P pword –r \n -t
each time I do I get and error saying "Syntax Error in 'úP'
If I remove everything after the username i am able to get the code to work as I am prompted for the password however it gives the table in a format which is imposssible to use.
could anyone advise where the syntax error may be occuring?
From what I can tell, it appears you have a few issues.
-i is not a valid option
-t should specify a field delimeter
You haven't specified a mode (character or native) (-c or -n)
Assuming you are trying to create a csv, here's what you may be looking for:
bcp tempdb..temptable out output.txt -S servername -U username -P password -c -t , -r \n
You may also find this page helpful from the bcp section of the Sybase ASE Utility Guide It's from the ASE 15.5 docs, but the syntax is the same for most versions 12.0 and newer.