Currently What I have is:
bcp "select * from TEST.dbo.HPA_1 WITH (NOLOCK)" queryout D:\Gift_Voucher\HPA\test1.txt -T -c -t
When I run the above code for the first time it works fine and it creates the textfile test1.txt, but When new data is added in the table I want to create new textfile something like test2.txt and test3.txt and so on without changing the code in bcp. Is there anything we can do here?
Change your BAT file to replace the output file path with a command-line parameter:
bcp "select * from TEST.dbo.HPA_1 WITH (NOLOCK)" queryout "%1" -T -c -t
In the SSIS package, create a string variable for the desired output file path. Set the Execute Process Task Executable property to "cmd.exe". On the Expressions page, set the Arguments property to an expression that builds the command with the BAT file path plus the output file argument. The example below also encloses the values in quotes to handle whitespace in the paths:
Set the variable value to the output file path variable in your package prior to executing the task. This can be done in package code or set the value via SSIS configuration.
Note that you could accomplish the same functionality in a Data Flow task instead of shelling out to BCP. That leverages the native export capability of SSIS.
Related
Currently I'm working with Exasol database first time and came across one script which is responsible to run sql script written in .sql file.
Here is the script
C:\Program Files\EXASOL\EXASolution\EXAplus\exaplusx64.exe -configDir EXASolutionConfig -profile profile_PROD_talend -q -f D:/Data/Customer/PROD/EXASolution_SQL/EXASOL_data_script.sql -- databaseName tableName /exasolution/StageArea/fileName.csv
I want to know, how this script is working and what its doing actually ? What I understood so far is below
First "C:\Program Files\EXASOL\EXASolution\EXAplus\exaplusx64.exe " is starting a Exasol on command line and then its pointing to the script where .sql file is located.
Not getting:
1) What this part is doing "-configDir EXASolutionConfig -profile profile_PROD_talend -q -f "?
2) What are these identifiers doing "-q -f "?
3)After launching exaplusx64.exe, Is exasol going to connect with database and table name mentioned in script ? If then How cav file is paying its role in this script ? I mean in .sql there is just an sql statement, If its taking data from file then how ? I'm not getting this ..!!
Please share your comments
1) This where you say to Exasol to read the profile profile_PROD_talend in the folder EXASolutionConfig and execute the file D:/Data/Customer/PROD/EXASolution_SQL/EXASOL_data_script.sql in quiet mode (-q).
From the manual:
-configDir *This is not actually in the EXASOL manual, I assume it's the folder with the profiles, or maybe it does nothing*
-profile Name of connection profile defined in <configDir>/profiles.xml (profiles can be edited in the GUI version). You can use a profile instead of specifying all connection parameters.
-q Quiet mode which suppresses additional output from EXAplus.
-f Name of a text file containing a set of instructions that run and then stop EXAplus.
2) Quiet mode and flag for the name of the file.
3) When you run this command EXAPlus connects to the db using the information provided in the profile and it will execute the .sql file passed.
Now things become interesting, the -- allows you to pass some arguments to the .sql file. So you are passing three parameters (databaseName, tableName, and /exasolution/StageArea/fileName.csv). If you open the sql script you will find &1, &2, and &3, these are the placeholders for the parameters passed by your command.
From the manual again:
-- <args> SQL files can use arguments given over via the parameter “-- ” by evaluating the variables &1, &2 etc. .
For example, the file test.sql including the content
--test.sql
SELECT * FROM &1;
can be called in the following way:
exaplus -f test.sql -- dual
I am new to programming and I am running into a issue. I am calling a table and need to put my results into a csv file in a certain path.
This is what I am doing and the error I get.
dbuser#cbos1:/var/lib/dbspace/bosarc/testing/Abe_Lincoln> cd dbaccess labor32<<?
> UNLOAD TO '/var/lib/dbspace/bosarc/Active_Sites/Cronos_test/Position7'
> select * from informix.position;
> ?
-bash: cd: dbaccess: No such file or directory
dbuser#cbos1:/var/lib/dbspace/bosarc/testing/Abe_Lincoln>
the file path exist but keeps getting message.
Using just $ as the command line prompt, you should be using just:
$ dbaccess labor32 <<?
> UNLOAD TO '/var/lib/dbspace/bosarc/Active_Sites/Cronos_test/Position7'
> select * from informix.position;
> ?
…message(s) from dbaccess
$
This will run the dbaccess program (usually from $INFORMIXDIR/bin) against the database labor32, and generate an UNLOAD format file in the given file name.
The cd command is for changing directory; you don't have a directory called dbaccess (and probably shouldn't), and even if you did have such a directory, you shouldn't provide more options to the cd command, or a here document as standard input — it will ignore them.
Note that the file generated (Position7 will be the base name of the file) will be in Informix's UNLOAD format (pipe delimited fields by default), not CSV. It's certainly possible to convert between the two; I have Perl scripts that can do the conversions — last modified about a decade ago, but not much has changed in the interim. You could also consider using SQLCMD (available as open source from the IIUG Software Repository) which does have support for CSV load and unload formats. (This is the original SQLCMD — or at least an original SQLCMD — and is not Microsoft's Johnny-come-lately program of the same name.)
Create a file unload-table.sh containing:
#!/bin/sh
dbaccess labor32 <<EOF
UNLOAD TO '/var/lib/dbspace/bosarc/Active_Sites/Cronos_test/Position7'
SELECT * FROM informix.position;
EOF
You can then run this as bash unload-table.sh, or make it executable and install it in your $HOME/bin directory (which is on your PATH, isn't it?) so that you can simply run unload-table.sh. Or you can arrange to 'compile' (copy) the file to unload-table (no .sh suffix) so you don't have to type it to execute it: unload-table. You can enhance the script to allow the program (dbacess), database (labor32), table (informix.position) and file (/var/lib/dbspace/bosarc/Active_sites/Cronos_test/Position7) to be set as command line arguments or via environment variables. That requires a bit of fiddling in the script, but nothing outrageous. I'd probably allow the file name to be specified separately from the directory where the file is to be stored so that it is easier to configure on the command line.
I have to run multiple SQL script file in one go.
Like every time i have to write command in SQLPLUS
SQL>#d:\a.txt
SQL>#d:\a2.txt
SQL>#d:\a3.txt
SQL>#d:\a4.txt
is there any way put all file in one folder & run all script file in one go without missing any single file like #d:\final.txt or #d\final.bat
There is no single SQL*Plus command to do that, but you can create a single script that calls all the others:
Put the following into a batch file
#echo off
echo.>"%~dp0all.sql"
for %%i in ("%~dp0"*.sql) do echo #"%%~fi" >> "%~dp0all.sql"
When you run that batch file it will create a new script named all.sql in the same directory where the batch file is located. It will look for all files with the extension .sql in the same directory where the batch file is located.
You can then run all scripts by using sqlplus user/pwd #all.sql (or extend the batch file to call sqlplus after creating the all.sql script)
If you're using gnu linux, you could use process substitution:
sqlplus USERNAME/PASSWORD#DOMAIN < <(cat a.txt a2.txt a3.txt a4.txt)
# ... or a for loop on input files, inside the process substitution
Alternatively, you can create a .pdc file and list your sql scripts:
-- pdc file
#a.txt;
#a2.txt;
#a3.txt;
#a4.txt;
and call sql plus:
sqlplus USERNAME/PASSWORD#DOMAIN < my_scripts.pdc
Some tricks and command can help you to generate master.sql file and you can run from that location.
c:\direcotory_location\dir *.sql /-t /b >master.sql
Go to the parent directory open master.sql open using notepad++
remove master.sql line and use regular expression to replace
\n with \n #
go to cmd
From cmd
C:\root_directory\sqlplus user/password #master.sql
I find this process very convenient if i have 30 to 40 scripts placed in a single directory.
Use *.PDC extension file like this
install.pdc file content
whenever sqlerror exit sql.sqlcode
prompt started!
prompt 1.executing script 1
##install/01.script_1.sql
prompt 2.executing script 2
##install/02.script_2.sql
prompt 3.executing script 3
##install/03.script_3.sql
prompt finished!
where ##install/ points in which directory is the SQL script located
It might be worth the time to write a shell script that runs multiple files.
#!/bin/ksh
sqlplus user/password#instance <<EOF
#a.txt
#a1.txt
exit
EOF
For more on the syntax, look into Here Document
here is similar solution but you do not have to iterate and to have special formated an sql file names. You compose an one sql file and run it once.
cat table_animal.sql > /tmp/temp.sql
cat table_horse.sql >> /tmp/temp.sql
cat table_fish.sql >> /tmp/temp.sql
sqlplus USERNAME/PASSWORD#DOMAIN #/tmp/temp.sql
For Windows try
copy /b *.sql +x final.sql
sqlplus user/password #final.sql
Special Thanks to Joseph Torre
sqlplus login/password#server #filename
reference link
Hi i am trying to set a variable in my .bat file and i want to use the variable in sqlldr code to state the file (infile)
This is what i have in my bat file for the variable:
set directroy_name= D:\Folder\Folder\Folder\File.csv
Then in my command file i have
load data
infile '%directory_name%'
When ever i try to run the .bat file from the command prompt i just receive the SQL_Loader_500: unable to ope file (%directory_name%.dat
I know the files in the correct location?
any ideas why its doing this?
No, you can't do that - you're expecting the Oracle executable to understand Windows environment variable syntax. If it did that it would have to deal with $ variables in Unix, etc.
You can just pass the file name on the command line instead. In your control file omit the INFILE altogether, then when you call SQL*Loader add a a DATA command-line argument:
sqlldr user/password CONTROL=your.ctl DATA=%directory_name% ...
Assuming your variable is just oddly named and does have a full file path as you've shown.
If it's present, the INFILE argument will be overridden by the command-line argument, so you could include a default fixed value if you wanted to, I suppose.
You also appear to have a typo; you set directroy_name, but then use directory_name, which will have no value. You need to change that to:
set directory_name= D:\Folder\Folder\Folder\File.csv
I am creating a batch file where I am restoring a database from an IP address and then executing a couple .sql files onto the database. In a couple of the .sql files there are variables declared and set. But this process has to be done on many machines with different values for each variable in each machine.
So I'm able to restore the database through user input of the IP, but I'm not sure how to use the batch script command to change the variable values.
For example, in one of the .sql files, a variable #store was declared and set to some random number. I want to change that number through the batch file.
I am using windows and sql server express 2008 r2
You can use "scripting variables" with SQLCMD.
Here's an example from that MSDN page:
You can also use the -v option to set a scripting variable that exists in a script. In the following script (the file name is testscript.sql), ColumnName is a scripting variable.
USE AdventureWorks2012;
SELECT x.$(ColumnName)
FROM Person.Person x
WHERE c.BusinessEntityID < 5;
You can then specify the name of the column that you want returned by using the -v option:
sqlcmd -v ColumnName ="FirstName" -i c:\testscript.sql
To return a different column by using the same script, change the value of the ColumnName scripting variable.
sqlcmd -v ColumnName ="LastName" -i c:\testscript.sql
If you are working on a Unix / Linux system, you can use sed to search a string.
Example: Assuming you need to replace 127.0.0.1 to 192.168.1.1, you can use the following instruction:
$ sed 's/127.0.0.1/192.168.1.1/g' script.sql > newScript.sql
This will replace the old ip in script.sql and will save a copy of this script in newScript.sql.
On windows, I don't know how to do it, but you can always download and install Cygwin to do exactly as above.
Hope this helps you.