export sql table data with column names to csv file - sql

I am trying to export the data from sql server tables to a csv file without ssms.
i am trying to achieve it by creating a stored procedure using bcp.
declare #sql nvarchar(4000);
select #sql = 'bcp "select * from table" queryout c:\file.csv -c -t, -T -S'+ ##servername
exec xp_cmdshell #sql
1 ) This query produces the expected results. But what i want is it should also include the column names in the csv files. So how can i achieve that ?
2) I want this result for all the tables in the given database. So how to do that ?
Please give some suggestions or solution as soon as possible
thanks

Only workarounds I know of...
Query the data dictionary and produce a csv field list, then
concatenate the header.csv with the data.csv. This would be querying the columns table, but you would need to take care to generate the SQL to match since you want to remove any and all chances that the column list doesn't match the data.
Create a view with the first row having all field names union all with a select on the data.
Something like:
SELECT 'a', 'b', 'c' UNION ALL SELECT a, b, c FROM table
This may require type conversions for dates, though.

I would suggest doing something along the lines of:
#echo off
bcp "SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '<table_name>' AND TABLE_SCHEMA='<table_schema>'" queryout c:\file.csv -c -r, -T -S <server_name> -d <database_name>
bcp "select * from <table_schema>.<table_name>" queryout c:\data.csv -c -t, -T -S <server_name> -d <database_name>
echo. >> c:\file.csv
type c:\data.csv >> c:\file.csv
del c:\data.csv
in a .bat file.
I think that for what you want to do, it's better to just use the bcp command from a batch file/command line, instead of enabling the xp_cmdshell in SQL Server which could introduce a security issue.
Additionally, I'd like to point out that I'm not sure if the columns will always come out in the same order (in my case it did).
EDIT: Batch file explanation.
I basically ran 2 bcp commands and sent the output to 2 different files, as I couldn't find an option to append the output to another file. I then merely used the type command to add the data to the file already containing the column list, and deleted the file with the data as it is no longer needed.
Feel free to modify and mess around with it and let me know if you run into any problems.

Try this code if you want column name with SQL table data :-
public void ExportExcelFileToFolder()
{
string constr = ConfigurationManager.ConnectionStrings["constr"].ConnectionString;
using (SqlConnection con = new SqlConnection(constr))
{
using (SqlCommand cmd = new SqlCommand("SELECT * FROM MachineMaster"))
{
using (SqlDataAdapter sda = new SqlDataAdapter())
{
cmd.Connection = con;
sda.SelectCommand = cmd;
using (DataTable dt = new DataTable())
{
sda.Fill(dt);
using (XLWorkbook wb = new XLWorkbook())
{
wb.Worksheets.Add(dt, "SheetName");
HttpContext.Current.Response.Clear();
HttpContext.Current.Response.Buffer = true;
HttpContext.Current.Response.Charset = "";
HttpContext.Current.Response.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
HttpContext.Current.Response.AddHeader("content-disposition", "attachment;filename=SqlExport.xlsx");
using (MemoryStream MyMemoryStream = new MemoryStream())
{
wb.SaveAs(MyMemoryStream);
string fileName = Guid.NewGuid() + ".xlsx";
string filePath = Path.Combine(System.Web.Hosting.HostingEnvironment.MapPath("~/ExeclFiles"), fileName);
MyMemoryStream.WriteTo(new FileStream(filePath, FileMode.Create));
HttpContext.Current.Response.Flush();
HttpContext.Current.Response.End();
}
}
}
}
}
}
}
Install-Package ClosedXML -Version 0.92.1

Related

Sql Query Generation Using bat file

I have a requirement where i need to see data for every week using a sql query.
I used query like SELECT * from table Between '27-08-2012' and '30-08-2012'.
Now my requirement is that i need a batch file that can given me this script based on user selection. Like user can give start date and end date and sql query should generate automatically.
A much less error prone way to get a date range is to use a graphical date picker
than manual user input which has to be checked for validity.
(month/day names will match your locale, not my German one)
This PowerShell script:
# Function Pick-Date
[void][System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms")
Function Pick-Date {
$Cal = new-object System.Windows.Forms.MonthCalendar
$Cal.ShowWeekNumbers = $true
$Cal.MaxSelectionCount = 10 # change this value for the max date distance
$Cal.Dock = 'Fill'
$Form = new-object Windows.Forms.Form
$Form.text = "Drag the mouse to select a date range then hit [enter]"
$Form.Size = new-object Drawing.Size #(656,620)
$btnSelect = new-object System.Windows.Forms.Button
$btnSelect.Size = "1,1"
$btnSelect.add_Click({ $Form.close() })
$Form.Controls.Add($btnSelect )
$Form.AcceptButton = $btnSelect
$Form.Controls.Add($Cal)
$Form.Add_Shown({$Form.Activate()})
[void]$Form.showdialog()
return ("SELECT * from table Between '"+
(Get-Date($Cal.SelectionStart) -format 'dd-MM-yyyy')+
"' and '"+
(Get-Date($Cal.SelectionEnd) -format 'dd-MM-yyyy')+
"'")
}
Pick-Date
Will have this output, you can save to a file.sql
PS> .\Pick-Date.ps1
SELECT * from table Between '27-08-2012' and '31-08-2012'
EDIT
This batch wrapper for the powerShell script will store the query in the variable SqlQuery
#Echo off&SetLocal EnableExtensions EnableDelayedExpansion
::Wrap Pick-Date.ps1 in same folder as batch
For /F "delims=" %%%A in (
'Powershell -NoP -NonI -NoLogo -File Pick-Date.ps1 '
) Do Set "SqlQuery=%%A"
Set SqlQuery

SQL Server : drop table with SQL parameters

I'm trying to drop a table using SqlParameters. I have this code .
dbCon.Open();
DataRowView d= (DataRowView) cmbTabele.Items[cmbTabele.SelectedIndex];
string name = (d["table_name"]as string);
SqlCommand com=new SqlCommand("drop table #nume ", dbCon);
com.Parameters.Clear();
SqlParameter param = new SqlParameter("#nume", name);
com.Parameters.Add(param);
com.ExecuteNonQuery(); // ERROR HERE
dbCon.Close();
I receive this error :
Incorrect syntax near '#nume'.
But when I do
SqlCommand com = new SqlCommand("drop table " + name, dbCon);
it works, and I really don't understand this error.
You cannot use a parameter for a table name. Although you'll normally get told off around here for building up a query using string concatenation, this is one occasion where you'll need to!
SqlCommand com=new SqlCommand("drop table " + name, dbCon);
I do not recommand it, but if you really want to use SQLParameter, then it is possible this way.
SqlCommand com=new SqlCommand("EXEC('drop table ''' + #nume + '''')", dbCon);
But really, there is no advantage in doing it this way. This work on SQL Server 2005 and newest version of it.

How to create text files using table content for name and text inside in SQL Server

I have following table:
1 One TEXT_ONE
2 Two TEXT_TWO
3 Three TEXT_Three ...
I want run SQL Query that will creates txt files in specific folder:
C:\Files\One.txt (Text inside - TEXT_ONE)
C:\Files\Two.txt (Text inside - TEXT_TWO)
C:\Files\Three.txt (Text inside - TEXT_Three)...
I'm not so good in SQL, so any help appreciates=)
Thanks
I found my own way. It's probably not absolute answer for my question, but the idea completed.
So, I've created bcp Command that needs to be executed from Command Line
'bcp "SELECT ColumnName FROM DBName.schema.TableName WHERE IDCol = ' + CAST(IDCol as varchar(2)) + '" QUERYOUT .\' + ColumnForFileName + '.txt -FullServerName -T -c'
You can run select statement in SQL. Here is my example:
SELECT sp.[SessionPageID]
,sp.[SummaryBody]
,sp.[MainTableBody]
,sf.[FileName]
,'bcp "SELECT SummaryBody FROM WLS_3E_TIMELOAD_APP.dbo.SessionPage WHERE SessionPageID = ' + CAST(sp.[SessionPageID] as varchar(2)) + '" QUERYOUT .\SB_' + sf.[FileName] + '_Session.txt -SCHAKHAU-T430\CHAKHAU -T -c'
,'bcp "SELECT MainTableBody FROM WLS_3E_TIMELOAD_APP.dbo.SessionPage" QUERYOUT .\MB_' + sf.[FileName] + '_Session.txt -MYLapTOP\LOCAL -T -c'
FROM [SessionPage] sp
JOIN SourceFile sf ON sf.SourceFileID = sp.SourceFileID
If you run this kind of query in result you will get all your bcp commands, so you then can execute them from Command Line

Generate sql insert script from excel worksheet

I have a large excel worksheet that I want to add to my database.
Can I generate an SQL insert script from this excel worksheet?
I think importing using one of the methods mentioned is ideal if it truly is a large file, but you can use Excel to create insert statements:
="INSERT INTO table_name VALUES('"&A1&"','"&B1&"','"&C1&"')"
In MS SQL you can use:
SET NOCOUNT ON
To forego showing all the '1 row affected' comments. And if you are doing a lot of rows and it errors out, put a GO between statements every once in a while
You can create an appropriate table through management studio interface and insert data into the table like it's shown below. It may take some time depending on the amount of data, but it is very handy.
There is a handy tool which saves a lot of time at
http://tools.perceptus.ca/text-wiz.php?ops=7
You just have to feed in the table name, field names and the data - tab separated and hit Go!
You can use the following excel statement:
="INSERT INTO table_name(`"&$A$1&"`,`"&$B$1&"`,`"&$C$1&"`, `"&$D$1&"`) VALUES('"&SUBSTITUTE(A2, "'", "\'")&"','"&SUBSTITUTE(B2, "'", "\'")&"','"&SUBSTITUTE(C2, "'", "\'")&"', "&D2&");"
This improves upon Hart CO's answer as it takes into account column names and gets rid of compile errors due to quotes in the column. The final column is an example of a numeric value column, without quotes.
Depending on the database, you can export to CSV and then use an import method.
MySQL - http://dev.mysql.com/doc/refman/5.1/en/load-data.html
PostgreSQL - http://www.postgresql.org/docs/8.2/static/sql-copy.html
Use the ConvertFrom-ExcelToSQLInsert from the ImportExcel in the PowerShell Gallery
NAME
ConvertFrom-ExcelToSQLInsert
SYNTAX
ConvertFrom-ExcelToSQLInsert [-TableName] <Object> [-Path] <Object>
[[-WorkSheetname] <Object>] [[-HeaderRow] <int>]
[[-Header] <string[]>] [-NoHeader] [-DataOnly] [<CommonParameters>]
PARAMETERS
-DataOnly
-Header <string[]>
-HeaderRow <int>
-NoHeader
-Path <Object>
-TableName <Object>
-WorkSheetname <Object>
<CommonParameters>
This cmdlet supports the common parameters: Verbose, Debug,
ErrorAction, ErrorVariable, WarningAction, WarningVariable,
OutBuffer, PipelineVariable, and OutVariable. For more information, see
about_CommonParameters (http://go.microsoft.com/fwlink/?LinkID=113216).
ALIASES
None
REMARKS
None
EXAMPLE
ConvertFrom-ExcelToSQLInsert MyTable .\testSQLGen.xlsx
You could use VB to write something that will output to a file row by row adding in the appropriate sql statements around your data. I have done this before.
Here is another tool that works very well...
http://www.convertcsv.com/csv-to-sql.htm
It can take tab separated values and generate an INSERT script. Just copy and paste and in the options under step 2 check the box "First row is column names"
Then scroll down and under step 3, enter your table name in the box "Schema.Table or View Name:"
Pay attention to the delete and create table check boxes as well, and make sure you examine the generated script before running it.
This is the quickest and most reliable way I've found.
You can use the below C# Method to generate the insert scripts using Excel sheet just you need import OfficeOpenXml Package from NuGet Package Manager before executing the method.
public string GenerateSQLInsertScripts() {
var outputQuery = new StringBuilder();
var tableName = "Your Table Name";
if (file != null)
{
var filePath = #"D:\FileName.xsls";
using (OfficeOpenXml.ExcelPackage xlPackage = new OfficeOpenXml.ExcelPackage(new FileInfo(filePath)))
{
var myWorksheet = xlPackage.Workbook.Worksheets.First(); //select the first sheet here
var totalRows = myWorksheet.Dimension.End.Row;
var totalColumns = myWorksheet.Dimension.End.Column;
var columns = new StringBuilder(); //this is your columns
var columnRows = myWorksheet.Cells[1, 1, 1, totalColumns].Select(c => c.Value == null ? string.Empty : c.Value.ToString());
columns.Append("INSERT INTO["+ tableName +"] (");
foreach (var colrow in columnRows)
{
columns.Append("[");
columns.Append(colrow);
columns.Append("]");
columns.Append(",");
}
columns.Length--;
columns.Append(") VALUES (");
for (int rowNum = 2; rowNum <= totalRows; rowNum++) //selet starting row here
{
var dataRows = myWorksheet.Cells[rowNum, 1, rowNum, totalColumns].Select(c => c.Value == null ? string.Empty : c.Value.ToString());
var finalQuery = new StringBuilder();
finalQuery.Append(columns);
foreach (var dataRow in dataRows)
{
finalQuery.Append("'");
finalQuery.Append(dataRow);
finalQuery.Append("'");
finalQuery.Append(",");
}
finalQuery.Length--;
finalQuery.Append(");");
outputQuery.Append(finalQuery);
}
}
}
return outputQuery.ToString();}
Here is a link to an Online automator to convert CSV files to SQL Insert Into statements:
CSV-to-SQL
This query i have generated for inserting the Excel file data into database
In this id and price are numeric values and date field as well. This query summarized all the type which I require It may useful to you as well
="insert into product (product_id,name,date,price) values("&A1&",'" &B1& "','" &C1& "'," &D1& ");"
Id Name Date price
7 Product 7 2017-01-05 15:28:37 200
8 Product 8 2017-01-05 15:28:37 40
9 Product 9 2017-01-05 15:32:31 500
10 Product 10 2017-01-05 15:32:31 30
11 Product 11 2017-01-05 15:32:31 99
12 Product 12 2017-01-05 15:32:31 25
I had to make SQL scripts often and add them to source control and send them to DBA.
I used this ExcelIntoSQL App from windows store https://www.microsoft.com/store/apps/9NH0W51XXQRM
It creates complete script with "CREATE TABLE" and INSERTS.
I have a reliable way to generate SQL inserts batly,and you can modify partial parameters in processing.It helps me a lot in my work, for example, copy one hundreds data to database with incompatible structure and fields count.
IntellIJ DataGrip , the powerful tool i use.
DG can batly receive data from WPS office or MS Excel by column or line.
after copying, DG can export data as SQL inserts.

insert MSSQL more than 8000 characters in a nvarchar(max) field

While using sql 2005, it appears I am unable to insert more than 8000 characters in a nvarchar(max) field.
This is really puzzling. I have tried insert, update and update .write with no luck. I can't insert it completely in the console and in .net the error I consistently get is
The data types text and varchar are incompatible in the add operator.
The insert statement is
insert into tablename (columnname) values (' long text');
Update:
update tablename set columnname='long text'
Everything is always truncated at 8000 characters (the text is 11,000 characters). Running a few tests, I see that
select ##textsize
gives 2147483647
Any ideas what I might be doing wrong here?
Your code truncates the value somewhere. You did not include the entire code, so we cannot guess where it truncates. The usual place is parameter declarations. The correct code should be like this:
SqlCommand cmd = new SqlCommand(
#"insert into table (column) values (#param)", conn, trn);
cmd.Paramaters.Add("#param", SqlDbType.NVarChar, -1);
cmd.Parameters["#param"].Value = myLongVariable;
cmd.ExecuteNonQuery();
using (System.Data.SqlClient.SqlConnection con = new
SqlConnection("YourConnection string"))
{
con.Open();
using (System.Data.SqlClient.SqlCommand cmd = new System.Data.SqlClient.SqlCommand())
{
string expression = "long text.................................";
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "Your Stored Procedure";
cmd.Parameters.Add("Your long text Parameter Name ",
SqlDbType.NVarChar).Value = expression;
cmd.Connection = con;
cmd.ExecuteNonQuery();
}
}
I've just had this issue and eventually tracked it down to the ADO constant I was using in the VBA code that calls the stored procedure. Originally I was adVarChar which gave me the "String data, right truncation" message, but when I swapped this out for adLongVarChar it now works fine
cmd.Paramaters.Add("#param", SqlDbType.NVarChar, -1).Value=parametervaluehere;
where -1 means max length for varchar/nvarchar datatype.