I'm trying to export data from a table to a CSV file in an automated fashion, hence the VBScript.
Here is my code and error below:
bcp dbname "SELECT * FROM mytable" queryout C:\Test.csv -T -c -Uusername -Ppassword -Sdbservername
I've never used bcp but it looks like a command-line utility. If you want to execute that from a VBScript, you'll need to use WshShell.Run, as in the following example:
With CreateObject("WScript.Shell")
.Run "bcp dbname ""SELECT * FROM mytable"" queryout C:\Test.csv -T -c -Uusername -Ppassword -Sdbservername"
End With
Take note of the doubled quotes around your SQL statement.
bcp "SELECT * FROM mytable" queryout "C:\Test.csv" -T -c -Uusername -Ppassword -Sdbservername
You should not have dbname after bcp. I removed it. This should work.
Related
I want to export all data from sql server table to a csv, I know I can get the desired result by:
sqlcmd -S . -d database -E -s, -W -Q "SELECT * FROM TABLENAME" > file.csv
I have many tables, so I want to create a .bat file that do the work for me, I have this:
set "list = A B C D"
for %%x in (%list%) do (
sqlcmd -S . -d database -E -s, -W -Q "SELECT * FROM %%x" > %%x.csv
)
But I am getting errors I don't know (I am not an expert in bat files). Why this does not work? How can I do what I want?
Spacing is important when using set (unless you're doing math with the /A switch). As written, the variable you're setting isn't %list%. It's %list %. Change your set command as follows:
set "list=A B C D"
How can I save first column from first row from query to file without additional character?
When I save data like this:
EXEC xp_cmdshell 'BCP "SELECT ''xxx'' " queryout D:\file.txt -w -T -S OMD-MG\SQL2008R2'
I've got:
additional \r\n at the end of file
When I save data like that:
EXEC xp_cmdshell 'BCP "SELECT ''xxx'' " queryout D:\file.txt -N -T -S OMD-MG\SQL2008R2'
I've got:
additional characters at front of file I think this is length
I try many parameter without satisfied result
Is there other option to save data to file from query without designer or management studio with correct data?
-N is a native binary format where any nullable or variable length fields are preceeded by their length. If you use -N with a non nullable, fixed-width field it will not be preceeded by its length.
If you want text data without the newlines you could try -r '' to specify the row terminator which is \n by default, e.g.:
bcp "select 'xxx'" queryout test.txt -c -t '' -r ''
..at least in SQL Server 2016 CTP I'm seeing that BCP tries to add padding to varchar columns. If you convert to text it seems to work alright:
bcp "select convert(text, col) from table" queryout file -c -t '' -r ''
I am using below command line to run a SQL query using SQLCMD
sqlcmd -S Server -Q "select top 100 * From people" -d people -t 10
The table has 20 columns and when i look at output command line window wraps the text and makes it difficult to read.
I want my results to be displayed the same way it displays in SQL Server Management Studio (properly formatted). I am not looking for any grids, but i need all my columns to be displayed in row 1 and the results properly beneath.
Thanks in advance.
Answer
We can set the width of each column.
C:/> sqlcmd -S my_server
> :setvar SQLCMDMAXVARTYPEWIDTH 30
> :setvar SQLCMDMAXFIXEDTYPEWIDTH 30
> SELECT * from my_table
> go
We can also set it like this: sqlcmd -S my_server -y 30 -Y 30.
Details
SQLCMDMAXVARTYPEWIDTH (-y)
It limits the number of characters that are returned for the large variable length data type
SQLCMDMAXFIXEDTYPEWIDTH (-Y)
Limits the number of characters that are returned for the following data types
Note: setting -y has serious performance implications.
See https://learn.microsoft.com/en-us/sql/tools/sqlcmd-utility
Formatting issues usually pop up due to your console window.
One solution is to output to the file and use notepad/your favorite editor:
sqlcmd -S myServer -d myDB -E -Q "select top 100 * From people"
-o "output.txt"
This is how I isolated a scalar.
sqlcmd -S xxx.xxx.xxx.xxx,xxxxx -d MyDb -U myUser -P MyPassword -h -1 -W -Q "set NOCOUNT ON; select a from b where b.id='c'"
I need to extract SQL files from multiple tables of a PostgreSQL database. This is what I've come up with so far:
pg_dump -t 'thr_*' -s dbName -U userName > /home/anik/psqlTest/db_dump.sql
However, as you see, all the tables that start with the prefix thr are being exported to a single unified file (db_dump.sql). I have almost 90 tables in total to extract SQL from, so it is a must that the data be stored into separate files.
How can I do it? Thanks in advance.
If you are happy to hard-code the list of tables, but just want each to be in a different file, you could use a shell script loop to run the pg_dump command multiple times, substituting in the table name each time round the loop:
for table in table1 table2 table3 etc;
do pg_dump -t $table -U userName dbName > /home/anik/psqlTest/db_dump_dir/$table.sql;
done;
EDIT: This approach can be extended to get the list of tables dynamically by running a query through psql and feeding the results into the loop instead of a hard-coded list:
for table in $(psql -U userName -d dbName -t -c "Select table_name From information_schema.tables Where table_type='BASE TABLE' and table_name like 'thr_%'");
do pg_dump -t $table -U userName dbName > /home/anik/psqlTest/db_dump_dir/$table.sql;
done;
Here psql -t -c "SQL" runs SQL and outputs the results with no header or footer; since there is only one column selected, there will be a table name on each line of the output captured by $(command), and your shell will loop through them one at a time.
Since version 9.1 of PostgreSQL (Sept. 2011), one can use the directory format output when doing backups
and 2 versions/2 years after (PostgreSQL 9.3), the --jobs/-j makes it even more efficient to backup every single objects in parallel
but what I don't understand in your original question, is that you use the -s option which dumps only the object definitions (schema), not data.
if you want the data, you shall not use -s but rather -a (data-only) or no option to have schema+data
so, to backup all objects (tables...) that begins with 'th' for the database dbName on the directory dbName_objects/ with 10 concurrent jobs/processes (increase load on the server) :
pg_dump -Fd -f dbName_objects -j 10 -t 'thr_*' -U userName dbName
(you can also use the -a/-s if you want the data or the schema of the objects)
as a result the directory will be populated with a toc.dat (table of content of all the objects) and one file per object (.dat.gz) in a compressed form
each file is named after it's object number, and you can retrieve the list with the following pg_restore command:
pg_restore --list -Fd dbName_objects/ | grep 'TABLE DATA'
in order to have each file not compressed (in raw SQL)
pg_dump --data-only --compress=0 --format=directory --file=dbName_objects --jobs=10 --table='thr_*' --username=userName --dbname=dbName
(not enough reputation to comment the right post)
I used your script with some corrections and some modifications for my own use, may be usefull for others:
#!/bin/bash
# Config:
DB=rezopilotdatabase
U=postgres
# tablename searchpattern, if you want all tables enter "":
P=""
# directory to dump files without trailing slash:
DIR=~/psql_db_dump_dir
mkdir -p $DIR
TABLES="$(psql -d $DB -U $U -t -c "SELECT table_name FROM
information_schema.tables WHERE table_type='BASE TABLE' AND table_name
LIKE '%$P%' ORDER BY table_name")"
for table in $TABLES; do
echo backup $table ...
pg_dump $DB -U $U -w -t $table > $DIR/$table.sql;
done;
echo done
(I think you forgot to add $DB in the pg_dumb command, and I added a -w, for an automated script, it is better not to have a psw prompt I guess, for that, I created a ~/.pgpass file with my password in it
I also gave the user for the command to know which password to fetch in .pgpass)
Hope this helps someone someday.
This bash script will do a backup with one file per table:
#!/bin/bash
# Config:
DB=dbName
U=userName
# tablename searchpattern, if you want all tables enter "":
P=""
# directory to dump files without trailing slash:
DIR=~/psql_db_dump_dir
mkdir -p $DIR
AUTH="-d $DB -U $U"
TABLES="$(psql $AUTH -t -c "SELECT table_name FROM information_schema.tables WHERE table_type='BASE TABLE' AND table_name LIKE '%$P%' ORDER BY table_name")"
for table in $TABLES; do
echo backup $table ...
pg_dump $AUTH -t $table > $DIR/$table.sql;
done;
echo done
I woud like to save my MS SQL Server 2005 stored procedures to .sql files automatically (would prefer a tool which I can call via .bat) so I don't have to click each single sproc manually and save it.
I have already found SMOscript from devio IT, but it gathers all tables and sproc which takes some time. Is there any similar tool where I can define which sproc(s) to export? Also I'm missing the USE <DB> clause which SMOScript doesn't add to exported file in contrast to the manuall export as script sproc for CREATE.
Create batch file with script (sorry about formatting, but it's really should be inline to execute batch):
osql -U %1 -P %2 -S %3 -d %4 -h-1 -Q "SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.Routines WHERE ROUTINE_TYPE = 'PROCEDURE'" -n -o "sp_list.txt"
for /f %%a in (sp_list.txt) do osql -U %1 -P %2 -S %3 -d %4 -h-1 -Q "SELECT ROUTINE_DEFINITION FROM INFORMATION_SCHEMA.Routines WHERE ROUTINE_NAME = '%%a'" -n -o "%%a.sql"
Name it "run.bat". Now, to execute batch use params:
run.bat [username] [password] [servername] [database]
on example:
run.bat sa pwd111 localhost\SQLEXPRESS master
first all stored procedure names will be stored in file sp_list.txt, then one by one in separate script files. The only issue - last line of each script with result count - I'm workin' on it :)
edited: bug in query fixed
Removing "Rows affected" line
Ok, now we need to create one more batch:
type %1 | findstr /V /i %2 > xxxtmpfile
copy xxxtmpfile %1 /y /v
del xxxtmpfile
Name it "line_del.bat". See, the first param is file to process, 2nd - string to search lines for removing. Now modify the main batch (again, sorry about formatting):
osql -U %1 -P %2 -S %3 -d %4 -h-1 -Q "SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.Routines WHERE ROUTINE_TYPE = 'PROCEDURE'" -n -o "sp_list.txt"
call line_del sp_list.txt "rows affected"
call line_del sp_list.txt "row affected"
for /f %%a in (sp_list.txt) do osql -U %1 -P %2 -S %3 -d %4 -h-1 -Q "SELECT ROUTINE_DEFINITION FROM INFORMATION_SCHEMA.Routines WHERE ROUTINE_NAME = '%%a'" -n -o "%%a.sql"
for /f %%a in (sp_list.txt) do call line_del %%a.sql "rows affected"
for /f %%a in (sp_list.txt) do call line_del %%a.sql "row affected"
See related articles:
Simple programming commands in a batch environment
osql Utility
MSSQL: How do you script Stored Procedure creation with code?
Delete certain lines in a txt file via a batch file
:) you may notice, last two are from SO!
There is an alternative in SQL Server Management Studio, scripting the database...
Expand the Object Explorer view to find the database, right click and select "Tasks : Generate Scripts"
From there you can script all object, just stored preocedures, of anything in between. There are quite a few options on one page, though the main one I change is:
- "Include IF NOT EXISTS"
By making that option "FALSE" then you just get a whole list of CREATE statements.
You can then choose to script the objects to a new query window, or a file.
Adding the SET NOCOUNT ON did indeed eliminate the need for the line_del.bat, but replacing syscomments with sys.sql_modules resulted in each stored procedure being truncated to 258 characters. So for the best results the code I used was:
sqlcmd -E -S SERVER-d DB -h-1 -Q "SET NOCOUNT ON SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.Routines WHERE ROUTINE_TYPE = 'PROCEDURE'" -o "sp_list.txt"
for /f %%a in (sp_list.txt) do sqlcmd -E -S SERVER-d DB -h-1 -Q "SET NOCOUNT ON SELECT text from dbo.syscomments WHERE id = OBJECT_ID('%%a')" -o "%%a.sql"
Which worked, and did not need the use of the line_del.bat. What I didnt get when I did the manual export using the SSMS wizard (Tasks/Generate Scripts/Stored Procedures/Select All) was the:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
at the beginning of each .sql, and the trailing GO command also. Not incredibly important but something to note. Thanks to Max Gontar, Seansilver and Lee for your contributions! I can now automate the backup of the stored procedures in the database, and apply version control.
I've used sqlcmd and -E instead of user, pass. This works fine so far, just stored procedures longer than 4000 chars will have a line break
sqlcmd -E -S SERVER -d DB -h-1 -Q "SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.Routines WHERE ROUTINE_TYPE = 'PROCEDURE'" -o "sp_list.txt"
call line_del sp_list.txt "rows affected"
call line_del sp_list.txt "row affected"
for /f %%a in (sp_list.txt) do sqlcmd -E -S SERVER -d DB -h-1 -Q "SELECT text from dbo.syscomments WHERE id = OBJECT_ID('%%a')" -o "%%a.sql"
for /f %%a in (sp_list.txt) do call line_del %%a.sql "rows affected"
for /f %%a in (sp_list.txt) do call line_del %%a.sql "row affected"
best regards
sean
I added SET NOCOUNT ON to eliminate the need for line_del.bat. Also replaced syscomments with sys.sql_modules (SQL 2005). I could also have used the OBJECT_DEFINITION function but it was slower than sys.sql_modules.
sqlcmd -E -S SERVER -d DB -h-1 -Q "SET NOCOUNT ON SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.Routines WHERE ROUTINE_TYPE = 'PROCEDURE'" -o "sp_list.txt" for /f %%a in (sp_list.txt) do sqlcmd -E -S SERVER -d DB -h-1 -Q "SET NOCOUNT ON SELECT definition from sys.sql_modules WHERE object_id = OBJECT_ID('%%a')" -o "%%a.sql"
Thanks to all above for help and leaving this here for other sqlcmd noobs like myself to find. Works on sql server 2005. objectName being a procedure, view etc name.
:reset
-- :setvar ObjectName "objectName" -- works
:setvar ObjectName objectName -- works too
declare #sql varchar(max);
set #sql = 'select text from dbo.syscomments where id = object_id(upper("' + '$(ObjectName)' + '"))';
-- exec sp_helptext $(ObjectName) -- quick n easy but gets chopped to 256 width
/* 4000 byte limit
set #sql =
'select view_definition
from information_schema.views
where upper(table_name) = upper("' + '$(ObjectName)' + '")';
*/
:out $(ObjectName).sql
exec(#sql)
go
:out stdout