So all is in my question. I have an web application in asp.net, with two databases :
The first ASPNETDB.MDF and the second MyApp.sdf
I would like to populate the tables of MyApp.sdf from an external file such as a txt file or something else.
Is this possible in Visual Studio 2010 ? Is there any way to do that ?
Thanks a lot in advance
Write a simple app:
using (SqlConnection sqlcnn = new SqlConnection("Data Source=myapp.sdf"))
{
SqlCommand sqlcmd = new SqlCommand();
sqlcmd.Connection = sqlcnn;
sqlcnn.Open();
StreamReader sr = new StreamReader(#"c:\mydir\myfile.txt");
while (sr.Peek() != -1) // check for eof (end of file)
{
String line = sr.ReadLine();
String[] values = line.Split(' '); // e.g if your values are seperated by space
sqlcmd.CommandText = String.Format("INSERT INTO tableName VALUES ({0},{1},{2})", values[0], values[1], values[2]);
sqlcmd.ExecuteNonQuery();
}
}
You can also use bcp -the BULK INSERT command with a file including records separated by a field terminator. For example:
BULK INSERT tablename
FROM 'c:\file.txt'
WITH
(
FIRSTROW = 2, // if the first row contains table field names
MAXERRORS = 0,
FIELDTERMINATOR = ',', //separator character
ROWTERMINATOR = '\n'
)
Related
I would like export SQL query to csv flatfile in ExecuteProcessTask in SSIS.
I can't see an export with the header, separator and qualify it as text.
I tried with sqlcmd and bcp.
For information, I am obliged to use a SELECT * because the view in FROM is a variable and I have to display all the column.
With sqlcmd :
sqlcmd -S ServerName -d dbName -E -Q "SELECT * FROM vPBI_Tasks WHERE [project Leader] like 'ProjectLeaderName'" -o "exportFile.csv" -W -s";"
Extract Result :
Scope;Project type;Activity type;OBS;Customer;Contr...
-----;------------;-------------;---;--------;-----...
ESP;ESP - Amendment;NULL;NULL;GSA;ESP_Amendment#13;...
ESP;ESP - Amendment;NULL;NULL;GSA;ESP_Amendment#13;...
ESP;ESP - Amendment;NULL;NULL;GSA;ESP_Amendment#13;...
I would like :
"Scope";"Project type";"Activity type";"OBS";"Customer";"Contra..."
ESP";"ESP - Amendment";"NULL";"NULL";"GSA";"ESP_Amendment#13";""
ESP";"ESP - Amendment";"NULL";"NULL";"GSA";"ESP_Amendment#13";""
ESP";"ESP - Amendment";"NULL";"NULL";"GSA";"ESP_Amendment#13";""
With bcp :
bcp "SELECT * FROM vPBI_Resources WHERE [project Leader] like 'ProjectLeaderName'" queryout "exportFile.csv" -c -t ; -S ServerName -T
Result :
I don't have header
I don't have text qualifier
See this answer given to an earlier, similar request:
SQL Server BCP Bulk insert Pipe delimited with text qualifier format file
Essentially, you need to use a BCP format file. When you build your BCP command, include the -f option and specify the location of the format file. In the format file, you specify your delimiter not as just the semi-colon character, but as ";" (that's two dbl-quote chars with a semi-colon between).
Theres a little more to it than that, but the link has the rest.
To get the header included, you just need to use 2 queries essentially. One query will be for the header and another query will be for the detail records. You can "union" the two queries together using "queryout" option of BCP. You will have to convert all your detail data to varchar data types so they can be queried together into a single file. But since you are already going out to a text file anyway, this shouldn't cause an issue. There are other answers that detail how to get a header included in this manner. I'll add one soon as an edit. You can also query out the header and detail records as two separate files (2 separate bcp commands) and just merge them together with a OS/script command.
I had indeed thought about this solution but I was confused by the problem of adding the double-quote at the beginning and ending of the line.
The workaround solution I found is a script in C#.
http://neil037.blogspot.com/2013/07/ssis-script-task-to-export-data-from.html
I put the C# code below, it will be used for other people:).
public void Main()
{
String filePath = Dts.Variables["User::temporyExportFilePath"].Value.ToString();
Dts.TaskResult = (int)ScriptResults.Success;
CreateCSVFile(GetTableData(), filePath);
}
public DataTable GetTableData()
{
String sqlQuery = Dts.Variables["User::sqlQuery"].Value.ToString();
String connectionString = Dts.Variables["User::stringDatabaseConnection"].Value.ToString();
SqlConnection connect = new SqlConnection(connectionString);
SqlCommand cmd = new SqlCommand(sqlQuery, connect);
cmd.CommandType = CommandType.Text;
SqlDataAdapter adap = new SqlDataAdapter(cmd);
DataTable dt = new DataTable();
adap.Fill(dt);
return dt;
}
public void CreateCSVFile(DataTable dt, string strFilePath)
{
StreamWriter sw = new StreamWriter(strFilePath, false);
int iColCount = dt.Columns.Count;
for (int i = 0; i < iColCount; i++)
{
// Write text qualifier double-quote + value + double-quote
sw.Write("\"" + dt.Columns[i] + "\"");
if (i < iColCount - 1)
{
//Parser
sw.Write(";");
}
}
sw.Write(sw.NewLine);
// Now write all the rows.
foreach (DataRow dr in dt.Rows)
{
for (int i = 0; i < iColCount; i++)
{
if (!Convert.IsDBNull(dr[i]))
{
// Write text qualifier double-quote + value + double-quote
sw.Write("\"" + dr[i].ToString() + "\"");
}
if (i < iColCount - 1)
{
//Parser
sw.Write(";");
}
}
sw.Write(sw.NewLine);
}
//Close file and all data writing
sw.Close();
}
I have created a vb.net project and a SQL Server database, I want to store my other created projects to the database with some of there description. I don't know can I do that or I can store my projects files into the database in the first place. I wanted to compress my projects files to a zip file and then store the zip file to the database but I don't know if I can store these files to the database which includes a lot of file types like (jar, java, sln, vb, c#, txt ...etc)?
Thanks for your help.
Yes, you can store files in a SQL Server database table. I highly recommend you to compress them into a zip file.
First step is to create a table:
CREATE TABLE Files
(
Id INT IDENTITY PRIMARY KEY,
FileData VARBINARY(MAX) FILESTREAM NULL,
Name NVARCHAR(300)
)
and then open your zip file as stream and Insert it into your table.
OpenFileDialog openFileDlg = new OpenFileDialog();
openFileDlg.InitialDirectory = Directory.GetCurrentDirectory();
if (openFileDlg.ShowDialog() == DialogResult.OK)
{
FileInfo fi = new FileInfo(openFileDlg.FileName);
FileStream fs = new FileStream(fi.FullName, FileMode.Open, FileAccess.Read);
BinaryReader rdr = new BinaryReader(fs);
byte[] fileData = rdr.ReadBytes((int)fs.Length);
rdr.Close();
fs.Close();
string cs = #"Data Source=<your server>;Initial Catalog=MyFsDb;Integrated Security=TRUE";
using (SqlConnection con = new SqlConnection(cs))
{
con.Open();
string sql = "INSERT INTO Files VALUES (#Data, #Name)";
SqlCommand cmd = new SqlCommand(sql, con);
cmd.Parameters.Add("#Data", SqlDbType.Image, fileData.Length).Value = fileData;
cmd.Parameters.Add("#Name", SqlDbType.NVarChar).Value = fi.Name;
cmd.ExecuteNonQuery();
con.Close();
}
MessageBox.Show(fi.FullName, "File Inserted!", MessageBoxButtons.OK, MessageBoxIcon.Information);
}
Source : MSDN
I have an sql server database table which has xml column name called "MESSAGE" and which will store xml data.
The database table look like,
Now I need to get this "MESSAGE" column data and save into System physical path as xml file(Ex: test.xml etc.,)
Any suggestion how to implement this using c#.net?
You could try something like this (using plain ADO.NET and a very basic SQL query):
static void Main(string[] args)
{
// get connection string from app./web.config
string connectionString = "server=.;database=yourDB;Integrated Security=SSPI;";
// define query
string query = "SELECT MESSAGE FROM dbo.SamTest WHERE ID = 1;";
// set up connection and command
using (SqlConnection conn = new SqlConnection(connectionString))
using (SqlCommand selectCmd = new SqlCommand(query, conn))
{
// open connection, execute query to get XML, close connection
conn.Open();
string xmlContents = selectCmd.ExecuteScalar().ToString();
conn.Close();
// define target file name
string targetFileName = #"C:\tmp\test.xml";
// write XML out to file
File.WriteAllText(targetFileName, xmlContents);
}
}
I'm trying to read data from files and to use bulk copy to insert it in the database table.
When I try to run my code, I get the error: "Cannot Access Denstination Table"
Declaration of FlatTable.
System.Data.DataTable flatTableTempData = new System.Data.DataTable("FlatTable");
DataColumn DistrictColumn = new DataColumn();
DistrictColumn.ColumnName = "DistrictName";
// Create Column 3: TotalSales
DataColumn TownColumn = new DataColumn();
TownColumn.ColumnName = "TownName";
DataColumn FarmerColumn = new DataColumn();
FarmerColumn.ColumnName = "FarmerName";
flatTableTempData.Columns.Add(DistrictColumn);
flatTableTempData.Columns.Add(TownColumn);
flatTableTempData.Columns.Add(FarmerColumn);
This is my code, with the connection string and the insertion using bulk copy:
using (SqlConnection con = new SqlConnection("Data Source=DRTARIQ-PC\\SQLEXPRESS;Integrated Security=SSPI;Initial Catalog=TestDB2"))
{
con.Open();
using (SqlBulkCopy s = new SqlBulkCopy(con))
{
s.DestinationTableName = flatTableTempData.TableName;
foreach (var column in flatTableTempData.Columns)
s.ColumnMappings.Add(column.ToString(), column.ToString());
s.BulkCopyTimeout = 500;
s.WriteToServer(flatTableTempData);
}
}
I've encountered the same problem. The table exists, the SQL user has access but SqlBulkCopy cannot access the table. My problem turned out to be I turned off the indexing to try and insert faster (rebuild index after the bulkcopy), but this made the table inaccessible. After I turned the indexing on again it worked, SqlBulkCopy has access to the table.
The table name in WriteToServer method of SqlBulkCopy must be surrounded with [ ] signs.
I have a very large shape file with hundreds of thousands of rows of polygons and other associated data, like formatted addressing and APN numbers. How do I get this data into a table with geography without using things like Shape2SQL? I can't very well run an insert statement for every row that would take forever, the optimal solution would be to create a csv or a properly formatted bin file and then do a bulk insert, or bcp, or openrowset, but try, try, try as I might I cannot get a csv file or bin file to work. Can anybody help?
The following code is the best I could manage.
SqlGeographyBuilder sql_geography_builder = new SqlGeographyBuilder();
sql_geography_builder.SetSrid(4326);
sql_geography_builder.BeginGeography(OpenGisGeographyType.Polygon);
sql_geography_builder.BeginFigure(-84.576064, 39.414853);
sql_geography_builder.AddLine(-84.576496, 39.414800);
sql_geography_builder.AddLine(-84.576522, 39.414932);
sql_geography_builder.AddLine(-84.576528, 39.414964);
sql_geography_builder.AddLine(-84.576095, 39.415015);
sql_geography_builder.AddLine(-84.576064, 39.414853);
sql_geography_builder.EndFigure();
sql_geography_builder.EndGeography();
SqlGeography sql_geography = new SqlGeography();
sql_geography = sql_geography_builder.ConstructedGeography;
FileStream file_stream = new FileStream("C:\\PROJECTS\\test.bin", FileMode.Create);
BinaryWriter binary_writer = new BinaryWriter(file_stream);
sql_geography.Write(binary_writer);
binary_writer.Flush();
binary_writer.Close();
file_stream.Close();
file_stream.Dispose();
SqlConnection sql_connection = new SqlConnection(connection_string);
sql_connection.Open();
SqlCommand sql_command = new SqlCommand();
sql_command.Connection = sql_connection;
sql_command.CommandTimeout = 0;
sql_command.CommandType = CommandType.Text;
sql_command.CommandText = "INSERT INTO [SPATIAL_TEST].[dbo].[Table_1] ([geo]) " +
"SELECT [ors].* " +
"FROM OPENROWSET(BULK 'C:\\PROJECTS\\AMP\\test.bin', SINGLE_BLOB) AS [ors] ";
sql_command.ExecuteNonQuery();
sql_command.Dispose();
sql_connection.Close();
sql_connection.Dispose();
But this only lets me import singularly the polygon--I need everything else as well.
Well after several days of headache I have come to the conclusion that there is no answer. Not even the mighty ESRI has any clue. Thankfully I did come up with a different soultion. In my table definition I created an NVARCHAR(MAX) column to hold the WFT of my geography and added that WFT to my csv file, and then after the bulk insert I run a table wide update statment to convert tht WFT to the actual geography type. Also adjust the csv file to use a different character besides a , to separate with becuase the WFT contains ,'s
SqlGeographyBuilder sql_geography_builder = new SqlGeographyBuilder();
sql_geography_builder.SetSrid(4326);
sql_geography_builder.BeginGeography(OpenGisGeographyType.Polygon);
sql_geography_builder.BeginFigure(-84.576064, 39.414853);
sql_geography_builder.AddLine(-84.576496, 39.414800);
sql_geography_builder.AddLine(-84.576522, 39.414932);
sql_geography_builder.AddLine(-84.576528, 39.414964);
sql_geography_builder.AddLine(-84.576095, 39.415015);
sql_geography_builder.AddLine(-84.576064, 39.414853);
sql_geography_builder.EndFigure();
sql_geography_builder.EndGeography();
SqlGeography sql_geography = new SqlGeography();
sql_geography = sql_geography_builder.ConstructedGeography;
StreamWriter stream_writer = new StreamWriter("C:\\PROJECTS\\AMP\\test.csv");
stream_writer.AutoFlush = true;
stream_writer.WriteLine("1?123 TEST AVE?" + sql_geography.ToString() + "?");
stream_writer.Flush();
stream_writer.WriteLine("2?456 TEST AVE?" + sql_geography.ToString() + "?");
stream_writer.Flush();
stream_writer.WriteLine("9?789 TEST AVE?" + sql_geography.ToString() + "?");
stream_writer.Flush();
stream_writer.Close();
stream_writer.Dispose();
SqlConnection sql_connection = new SqlConnection(STRING_SQL_CONNECTION);
sql_connection.Open();
SqlCommand sql_command = new SqlCommand();
sql_command.Connection = sql_connection;
sql_command.CommandTimeout = 0;
sql_command.CommandType = CommandType.Text;
sql_command.CommandText = "BULK INSERT [SPATIAL_TEST].[dbo].[Table_1] " +
"FROM 'C:\\PROJECTS\\AMP\\test.csv' " +
"WITH (FIELDTERMINATOR = '?', ROWTERMINATOR = '\n') " +
"" +
"UPDATE [SPATIAL_TEST].[dbo].[Table_1] " +
"SET [geo] = geography::STPolyFromText([geo_string], 4326) ";
sql_command.ExecuteNonQuery();
sql_command.Dispose();
sql_connection.Close();
sql_connection.Dispose();
MessageBox.Show("DONE");
}
catch (Exception ex)
{ MessageBox.Show(ex.Message); }