I have a very large shape file with hundreds of thousands of rows of polygons and other associated data, like formatted addressing and APN numbers. How do I get this data into a table with geography without using things like Shape2SQL? I can't very well run an insert statement for every row that would take forever, the optimal solution would be to create a csv or a properly formatted bin file and then do a bulk insert, or bcp, or openrowset, but try, try, try as I might I cannot get a csv file or bin file to work. Can anybody help?
The following code is the best I could manage.
SqlGeographyBuilder sql_geography_builder = new SqlGeographyBuilder();
sql_geography_builder.SetSrid(4326);
sql_geography_builder.BeginGeography(OpenGisGeographyType.Polygon);
sql_geography_builder.BeginFigure(-84.576064, 39.414853);
sql_geography_builder.AddLine(-84.576496, 39.414800);
sql_geography_builder.AddLine(-84.576522, 39.414932);
sql_geography_builder.AddLine(-84.576528, 39.414964);
sql_geography_builder.AddLine(-84.576095, 39.415015);
sql_geography_builder.AddLine(-84.576064, 39.414853);
sql_geography_builder.EndFigure();
sql_geography_builder.EndGeography();
SqlGeography sql_geography = new SqlGeography();
sql_geography = sql_geography_builder.ConstructedGeography;
FileStream file_stream = new FileStream("C:\\PROJECTS\\test.bin", FileMode.Create);
BinaryWriter binary_writer = new BinaryWriter(file_stream);
sql_geography.Write(binary_writer);
binary_writer.Flush();
binary_writer.Close();
file_stream.Close();
file_stream.Dispose();
SqlConnection sql_connection = new SqlConnection(connection_string);
sql_connection.Open();
SqlCommand sql_command = new SqlCommand();
sql_command.Connection = sql_connection;
sql_command.CommandTimeout = 0;
sql_command.CommandType = CommandType.Text;
sql_command.CommandText = "INSERT INTO [SPATIAL_TEST].[dbo].[Table_1] ([geo]) " +
"SELECT [ors].* " +
"FROM OPENROWSET(BULK 'C:\\PROJECTS\\AMP\\test.bin', SINGLE_BLOB) AS [ors] ";
sql_command.ExecuteNonQuery();
sql_command.Dispose();
sql_connection.Close();
sql_connection.Dispose();
But this only lets me import singularly the polygon--I need everything else as well.
Well after several days of headache I have come to the conclusion that there is no answer. Not even the mighty ESRI has any clue. Thankfully I did come up with a different soultion. In my table definition I created an NVARCHAR(MAX) column to hold the WFT of my geography and added that WFT to my csv file, and then after the bulk insert I run a table wide update statment to convert tht WFT to the actual geography type. Also adjust the csv file to use a different character besides a , to separate with becuase the WFT contains ,'s
SqlGeographyBuilder sql_geography_builder = new SqlGeographyBuilder();
sql_geography_builder.SetSrid(4326);
sql_geography_builder.BeginGeography(OpenGisGeographyType.Polygon);
sql_geography_builder.BeginFigure(-84.576064, 39.414853);
sql_geography_builder.AddLine(-84.576496, 39.414800);
sql_geography_builder.AddLine(-84.576522, 39.414932);
sql_geography_builder.AddLine(-84.576528, 39.414964);
sql_geography_builder.AddLine(-84.576095, 39.415015);
sql_geography_builder.AddLine(-84.576064, 39.414853);
sql_geography_builder.EndFigure();
sql_geography_builder.EndGeography();
SqlGeography sql_geography = new SqlGeography();
sql_geography = sql_geography_builder.ConstructedGeography;
StreamWriter stream_writer = new StreamWriter("C:\\PROJECTS\\AMP\\test.csv");
stream_writer.AutoFlush = true;
stream_writer.WriteLine("1?123 TEST AVE?" + sql_geography.ToString() + "?");
stream_writer.Flush();
stream_writer.WriteLine("2?456 TEST AVE?" + sql_geography.ToString() + "?");
stream_writer.Flush();
stream_writer.WriteLine("9?789 TEST AVE?" + sql_geography.ToString() + "?");
stream_writer.Flush();
stream_writer.Close();
stream_writer.Dispose();
SqlConnection sql_connection = new SqlConnection(STRING_SQL_CONNECTION);
sql_connection.Open();
SqlCommand sql_command = new SqlCommand();
sql_command.Connection = sql_connection;
sql_command.CommandTimeout = 0;
sql_command.CommandType = CommandType.Text;
sql_command.CommandText = "BULK INSERT [SPATIAL_TEST].[dbo].[Table_1] " +
"FROM 'C:\\PROJECTS\\AMP\\test.csv' " +
"WITH (FIELDTERMINATOR = '?', ROWTERMINATOR = '\n') " +
"" +
"UPDATE [SPATIAL_TEST].[dbo].[Table_1] " +
"SET [geo] = geography::STPolyFromText([geo_string], 4326) ";
sql_command.ExecuteNonQuery();
sql_command.Dispose();
sql_connection.Close();
sql_connection.Dispose();
MessageBox.Show("DONE");
}
catch (Exception ex)
{ MessageBox.Show(ex.Message); }
Related
I try to UPDATE data in my SQL Server database and I get this error:
System.Data.SqlClient.SqlException
Incorrect syntax near 'de'
Unclosed quotation mark after the character string ')'
private void BtEnrMod_Click(object sender, EventArgs e)
{
SqlConnection con = new SqlConnection("Data Source=.\\BD4X4;Initial Catalog=BD4X4;Integrated Security=True");
con.Open();
SqlCommand cmd = new SqlCommand("UPDATE Service SET Type = " + TxBxService.Text + ", Prix = " + TxBxPrix.Text + "WHERE Code = " + LbCodeAff.Text + "')", con);
int i = cmd.ExecuteNonQuery();
if (i != 0)
{
MessageBox.Show("Service Modifié");
}
else
{
MessageBox.Show("Erreur");
}
this.Close();
con.Close();
}
Replace the one liner that declares your command with this code block:
SqlCommand cmd = new SqlCommand("UPDATE Service SET Type = #t, Prix = #p WHERE Code = #c", con);
cmd.Parameters.AddWithValue("#t", TxBxService.Text);
cmd.Parameters.AddWithValue("#p", TxBxPrix.Text);
cmd.Parameters.AddWithValue("#t", LbCodeAff.Text);
Always avoid writing an sql where you string concatenate in a value provided by the user in a text box; it's the number one security horror you can make with sql. Always use parameters to put values in, like you see here. For more info on this SQL injection hacking, see http://bobby-tables.com
If you ever fin yourself in a situation where you think you have to concatenate to make an sql, don't concatenate a value in; concatenate a parameter in and add the value into the parameters collection. Here's a hypothetical example:
var cmd = new SqlCommand("","connstr");
strSql = "SELECT * FROM table WHERE col IN (";
string[] vals = new[]{ "a", "b", "c" };
for(int x = 0; x<vals.Length; x++){
strSql += ("#p"+x+",");
cmd.Parameters.AddWithValue("#p"+x, vals[x]);
}
cmd.CommandText = strSql + ")";
This uses concatenation to make an sql of SELECT * FROM table WHERE col IN (#p0, #p1, #p2) and a nicely populated parameters collection
When you're done grokking that, read the link Larnu posted in the comments. There are good reasons to avoid using AddWithValue in various scenarios but it will always be preferable to concatenation of values. Never ditch the use of parameters "because I read a blog one time about how AddWithValue is bad" - form parameters using the new parameter constructor, or use AddWithValue shortcut, but never concat values
Or better still than all of this, use an ORM like Entity Framework, nHibernate or Dapper and leave most of this boring boilerplate low level SQL drudgery behind. These libraries do most of this wrangling for you; EF and nH even write th sql too, dapper you write it yourself but it takes care of everything else
Using a good ORM is like the difference between writing creating a UI manually line by line of position, font, anchor, event code for every button, label and text box versus using the windows forms designer; a world apart and there's no sense in taking hours to create manually what software can do more comprehensively, faster and safer for you in seconds
I am using sap.net web form. In this web form i have a text and a button. user enter name or id and hit search button. Searching with id is working fine but with name it is not working.
What i am missing here help me out please.
String Status = "Active";
String BDstring = ConfigurationManager.ConnectionStrings["CS"].ConnectionString;
using (SqlConnection conn = new SqlConnection(BDstring))
{
try
{
String query = "SELECT * from Driver where(Name LIKE '%' + #search + '%' OR DriverID = #search) AND Status = 'Active'";
SqlCommand cmd = new SqlCommand(query, conn);
cmd.Parameters.AddWithValue("#search", SearchTextBox.Text);
conn.Open();
SqlDataReader SDR = cmd.ExecuteReader();
DataTable DT = new DataTable();
if (SDR.HasRows)
{
DT.Load(SDR);
GridView.DataSource = DT;
GridView.DataBind();
}
}
catch (SqlException exe)
{
throw exe;
}
}
}
The code is generating an exception. The fact that you're unaware of this indicates that you have "error handling" somewhere in your system that is, in fact "error hiding". Remove empty catch blocks, or pointless catch blocks such as the one in your question that just destroys some information in the exception and re-throws it. Those aren't helping you.
The actual problem is that the DriverID column is int and your parameter is varchar. So long as the varchar contains a string that can be converted to a number (which is the direction that the conversion happens in due to precedence), the query is well-formed.
As soon as the parameter contains a string that cannot be implicitly converted to a number, SQL Server generates an error that .NET turns into an exception.
For your LIKE variant, you're forcing a conversion in the opposite direction (numeric -> varchar) since LIKE only operates on strings. That conversion will always succeed, but it means that you're performing textual comparisons rather than numeric, and also means there's no possible index usage here.
I'd suggest that you change your C# code to attempt a int.TryParse on the input text and then uses two separate parameters to pass strings and (optionally) their numeric equivalent to SQL Server. Then use the appropriate parameters in your query for each comparison.
Something like:
String Status = "Active";
String BDstring = ConfigurationManager.ConnectionStrings["CS"].ConnectionString;
using (SqlConnection conn = new SqlConnection(BDstring))
{
String query = "SELECT * from Driver where(Name LIKE '%' + #search + '%' OR " +
"DriverID = #driverId) AND Status = 'Active'";
SqlCommand cmd = new SqlCommand(query, conn);
cmd.Parameters.Add("#search", SqlDbType.VarChar,50).Value = SearchTextBox.Text;
cmd.Parameters.Add("#driverId", SqlDbType.Int);
int driverId;
if(int.TryParse(SearchTextBox.Text, out driverId))
{
cmd.Parameters["#driverId"].Value = driverId;
}
conn.Open();
SqlDataReader SDR = cmd.ExecuteReader();
DataTable DT = new DataTable();
if (SDR.HasRows)
{
DT.Load(SDR);
GridView.DataSource = DT;
GridView.DataBind();
}
}
"SELECT * from Driver where (Name LIKE '%" + #search + "%'
OR DriverID = '" + #search + "' ) AND Status = 'Active'";
how about this?
I face problem for inserting value in Cassandra database.
Please suggest me some code for inserting value in database
var empRecord = new EmployeeEntity()
{
employeeid=2,
age=23,
employeename=txtName.Text,
salary=5001
};
var employeeRecord = new CassandraEntity<List<Column>>().SetColumnFamily(columnFamily).SetKey(empRecord.employeeid).SetData(empRecord);
ctx.ColumnList.InsertOnSubmit(employeeRecord);
ctx.SubmitChanges();
i getting error for inserting value in Cassandra database
Error like -> Not enough bytes to read value of component 0
How to solve it
I'm not sure which driver you are trying to use, but this works with the DataStax C# CQL3 driver.
First, connect to Cassandra and get your session:
cluster = Cluster.Builder().WithCredentials(_user, _password).AddContactPoint(_node).Build();
Session session = cluster.Connect();
Then, insert data using the following:
String strCQL1 = "UPDATE yourKeyspaceName.employee SET age=?, "
+ "employeename=?, Salary=? "
+ "WHERE employeeid=?";
PreparedStatement statement = session.Prepare(strCQL1);
BoundStatement boundStatement = new BoundStatement(statement);
boundStatement.Bind(age, txtName.Text, salary, empRecord.employeeid);
session.Execute(boundStatement);
In this case, Cassandra will perform an "UPSERT," inserting the data if it does not exist and updating if it does. Just make sure that your primary key(s) are in the WHERE clause.
Try This solution ,
string query = " insert into employee (employeeid,age,employeename,salary) values (" + txtId.Text + ",31,'" + strNameHex.Trim() + "'," + myHexNumber + ")";
txtQuery.Text = query;
context.ExecuteNonQuery(query);
context.SaveChanges();
It's help you
I insert the data from CSV file into MySql database, especially into one table.
I use CSVRead, and the CSV file format is :
ts,val
2013-03-31T23:45:00-04:00 New_York,10
And the table is hisdata(ts, val).
Here is my code:
try{
try {
CSVReader reader = new CSVReader(new FileReader(csvFile));
List<String[]> csvList;
csvList = reader.readAll();
System.out.println("Start: size is " + csvList.size());
for(int i = 0; i<csvList.size(); i++){
String[] eachStr = csvList.get(i);
int j = 0;
//insert(ts, val) into hisdata of sql
String sql = "INSERT INTO hisdata" + "(ts, val)" + " VALUES"
+ "('" + eachStr[j] + "', '" + eachStr[j+1] + "')";
Statement st = (Statement) conn.createStatement();
count = st.executeUpdate(sql);
}
System.out.println("access table is inserted: " + count
+ " records");
reader.close();
} catch(IOException e){
System.out.println("Error: " + e.getMessage());
}
conn.close();
} catch (SQLException e) {
System.out.println("insert is failure " + e.getMessage());
}
I think probably, the import data is too large. When I did size(), size is 8835.
Basically, I set connector. Then read CSV file and insert data line by line. Finally, I closed reader and connection.
Here is the Console print out:
Sql Connection starts
Driver loaded
Database connected
Start: size is 8835
insert is failure Data truncated for column 'val' at row 1
Is the problem the data is too large.
Please give help to solve this problem.
Add System.out.println(sql); execute the result in mysql.
if data is too large, increase column length, or get column length then reduce your data with substring.
So all is in my question. I have an web application in asp.net, with two databases :
The first ASPNETDB.MDF and the second MyApp.sdf
I would like to populate the tables of MyApp.sdf from an external file such as a txt file or something else.
Is this possible in Visual Studio 2010 ? Is there any way to do that ?
Thanks a lot in advance
Write a simple app:
using (SqlConnection sqlcnn = new SqlConnection("Data Source=myapp.sdf"))
{
SqlCommand sqlcmd = new SqlCommand();
sqlcmd.Connection = sqlcnn;
sqlcnn.Open();
StreamReader sr = new StreamReader(#"c:\mydir\myfile.txt");
while (sr.Peek() != -1) // check for eof (end of file)
{
String line = sr.ReadLine();
String[] values = line.Split(' '); // e.g if your values are seperated by space
sqlcmd.CommandText = String.Format("INSERT INTO tableName VALUES ({0},{1},{2})", values[0], values[1], values[2]);
sqlcmd.ExecuteNonQuery();
}
}
You can also use bcp -the BULK INSERT command with a file including records separated by a field terminator. For example:
BULK INSERT tablename
FROM 'c:\file.txt'
WITH
(
FIRSTROW = 2, // if the first row contains table field names
MAXERRORS = 0,
FIELDTERMINATOR = ',', //separator character
ROWTERMINATOR = '\n'
)