Import excel sheet to sql database VB Net client - vb.net

I have some code I was hoping someone could look at for performance improvements. I have a spreadsheet that I need to import weekly. The sheet has 112 columns and about 35,000 rows. The code I have works, but it takes about 20 minutes to import the data. The excel column names do not match the database column names (I inherited this). Here is the code I am using. (I removed alot of the fields so it is easier to read)
'Connection String to Excel Workbook
Dim excelConnectionString As String = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & MyFile & ";Extended Properties=""Excel 12.0;HDR=Yes;"""
' Create Connection to Excel Workbook
Using connection As New System.Data.OleDb.OleDbConnection(excelConnectionString)
'List columns you need from the Excel file
Dim command As New System.Data.OleDb.OleDbCommand("Select * FROM [" & txtSheetName.Text & "$]", connection)
connection.Open()
' Create DbDataReader to Data Worksheet
Using dr As System.Data.OleDb.OleDbDataReader = command.ExecuteReader()
Dim strSql As String = ""
strSql = "INSERT INTO MyTestTable (" & _
"State, [Store Code], [Store Name], [Store Zone Code], [Store Zone Code Name], " & _
"WeekProd, YTDNew, " & _
"UpdatedBy, DateUpdated" & _
") VALUES (" & _
"#State, #StoreCode, #StoreName, #StoreZoneCode, #StoreZoneCodeName, " & _
"#WeekProd, #YTDNew, " & _
"#UpdatedBy, #DateUpdated" & _
Try
If dr.HasRows() Then
While dr.Read()
If Convert.ToString(dr.Item(0)) <> "" Then
Dim MyZone As String = Convert.ToString(dr.Item(1))
MyZone = StrConv(MyZone, vbProperCase)
Dim cmd As New SqlClient.SqlCommand
cmd.Connection = cn
cmd.CommandType = CommandType.Text
cmd.CommandText = strSql
cmd.Parameters.Add("#State", SqlDbType.VarChar).Value = ""
cmd.Parameters.Add("#StoreCode", SqlDbType.VarChar).Value = Convert.ToString(dr.Item(0))
cmd.Parameters.Add("#StoreName", SqlDbType.VarChar).Value = MyZone
cmd.Parameters.Add("#StoreZoneCode", SqlDbType.VarChar).Value = Convert.ToString(dr.Item(2))
cmd.Parameters.Add("#StoreZoneCodeName", SqlDbType.VarChar).Value = Convert.ToString(dr.Item(3))
cmd.Parameters.Add("#WeekProd", SqlDbType.Float).Value = Convert.ToDecimal(dr.Item(93))
cmd.Parameters.Add("#YTDNew", SqlDbType.Float).Value = Convert.ToDecimal(dr.Item(94))
cmd.Parameters.Add("#UpdatedBy", SqlDbType.VarChar).Value = MyUser.Substring(MyUser.Length - 4)
cmd.Parameters.Add("#DateUpdated", SqlDbType.Date).Value = Date.Today()
cmd.ExecuteScalar()
End If
End While
End If
Catch ex As Exception
lblMessage.Text = ex.Message
Exit Sub
Finally
cn.Close()
cn = Nothing
End Try
End Using
End Using

Won't promise this will set the world on fire performance-wise, but it might help.
My first thought would be to not create a new SqlCommand object and parameter set for every row in the Excel table. I would create it one time (which should save you the overhead of about 35K object instantiations given your data size), establish the parameter names, all outside the reader loop, and then call SetParameter to set the values for each column as each row in the Excel is traversed. That should trade you the overhead of about 35K * (number of real fields) parameter addition calls for setparameter calls to existing parameters. And I would also change the call type to ExecuteNonQuery rather than ExecuteScalar.
Now, ordinarily, the conventional wisdom would hold that you only open/close a connection to a database as you need it, and I think that is implied in this structure (open/close for each insert) but in this case, for a data update scenario like this, I think this would be a reasonable exception.

Related

how to reduce quantity using vb.net

has an error (An unhandled exception of type 'System.ArgumentOutOfRangeException' occurred in mscorlib.dll
Additional information: Index and length must refer to a location within the string.)
mycmd.Connection = myconnection.open
Dim dami As Integer = quantityt.Text
mycmd.CommandText = "Update inventory set total_quantity=total_quantity-'" & dami & "' where item_code='" & itemcodet.Text & "'"
mycmd.ExecuteNonQuery()
MsgBox("stocks decrease!!", MsgBoxStyle.Information, "Noticed..")
myconnection.close()
I think you have problem with following line
mycmd.CommandText = "Update inventory set total_quantity=total_quantity-'" & dami & "' where item_code='" & itemcodet.Text & "'"
should be
mycmd.CommandText = "Update inventory set total_quantity=total_quantity-" & dami & " where item_code='" & itemcodet.Text & "'"
dont use apostrophe ' when counting with integer.. apostrophes use only in case of string
I used the SQL server provider for demonstration. Change to whatever database you are using. Check the actual data types of your fields in your database. Open the connection at the last minute. See my comment about Using blocks.
Private Sub OPCode()
Using myconnection As New SqlConnection("Your connection string")
Using mycmd As New SqlCommand("Update inventory set total_quantity = total_quantity - #dami where item_code = #itemCode;", myconnection)
mycmd.Parameters.Add("#dami", SqlDbType.Int).Value = CInt(quantityt.Text)
mycmd.Parameters.Add("#itemCode", SqlDbType.VarChar).Value = itemcodet.Text
myconnection.Open()
mycmd.ExecuteNonQuery()
End Using
End Using
MsgBox("stocks decrease!!", MsgBoxStyle.Information, "Noticed..")
End Sub

"Operation must be an updateable query" VB.Net OleDB for Excel

Ive been trying to find a solution for this problem without any success:
I'm using VB.NET and I need to read and update records from an Excel file. I use the OleDB API for that. It works fine for all the reading, but impossible to write to the file (Update or Insert queries)
Here is what I have:
My connection string:
Public connString As String = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\Users\...\Resources\attempt.xls;Extended Properties='Excel 8.0;HDR=YES;IMEX=1;'"
Select query that works fine:
Dim checkQuery = "SELECT * FROM [Sheet1$] WHERE [TravellerPN] = #T"
Try
Using connection As New OleDb.OleDbConnection(Form.connString)
Dim checkCommand As New OleDbCommand(checkQuery, connection)
checkCommand.Parameters.Add("#T", OleDbType.VarChar).Value = PN
connection.Open()
Dim reader As OleDbDataReader = checkCommand.ExecuteReader()
Dim path As String = ""
While reader.Read()
path = reader("Datapath").ToString
End While
reader.Close()
MsgBox(PN & " " & DP & " " & path,,)
If a record for the part number entered doesnt exist, and nothing is returned, insert a new line
If path = "" Then
Dim addQuery = "INSERT INTO [Sheet1$] ([TravellerPN],[Datapath]) VALUES (#T, #D)"
Dim addCommand As New OleDbCommand(addQuery, connection)
addCommand.Parameters.Add("#T", OleDbType.VarChar).Value = PN
addCommand.Parameters.Add("#D", OleDbType.VarChar).Value = DP
Dim rows As Integer = addCommand.ExecuteNonQuery()
And this was where it returns the error.
MsgBox(rows & " row added!",, "") 'Never diplayed
And other query that doesn't work either:
Else 'If does exist, confirm replacement'
Dim replaceResponse = MsgBox("A path already exists for " & PN & "." & vbNewLine & "Do you want to replace " & path & " with " & DP & "?", MsgBoxStyle.YesNo, "Overwrite previous datapath?")
Dim replaceQuery = "UPDATE [Sheet1$] SET [Datapath] = #D WHERE [TravellerPN]=#T"
Dim replaceCommand As New OleDbCommand(replaceQuery, connection)
replaceCommand.Parameters.Add("#D", OleDbType.VarChar).Value = DP
replaceCommand.Parameters.Add("#T", OleDbType.VarChar).Value = PN
Dim rows As Integer = replaceCommand.ExecuteNonQuery()
MsgBox(rows & " row updated!",, "")
End If
connection.Close()
End Using
I've tried to fix the issue: it could be caused by permissions, so I authorized even guests accounts to modify my folder.
If it were a ReadOnly mode in the connection: I tried adding Mode=ReadWrite but my connection String didn't work after that. (That option seems only available for ADO connections?)
I tried running the app as administrator
And finally, I'm posting this here hoping to get some help.
Sorry or the long post, I was trying to give all the elements that could potentially be a problem.
Thanks.

VB.NET updating .accdb records

I'm working on a VB.Net application that interfaces with an .accdb file to create and (eventually) update records on two tables in the same database. I'm able to write new information to a table no problem, however it is updating/changing/adding additional information to that same row in the table I'm having issues with. My code for writing updates to an existing row is at the bottom of my post.
The biggest issue I'm having is, after I execute this subroutine, it fails at the objCmd.ExecuteNonQuery() with the error message IErrorInfo.GetDescription failed with E_FAIL(0x80004005). I've combed through here and Google, trying different methods and moving things around and I cannot figure out what I'm missing. As far as I can tell, I am not using any reserved words in my SQL query. The block under the Else statement does work for creating new rows (I don't have issues with that side of my program), maybe the syntax is different for doing UPDATE commands? Any help/insight is greatly appreciated.
Private Sub WriteToDatabase()
strcs = txtSerialNumber.Text
strOrderType = orderType
strPoRMA = txtPoRMA.Text
strtech = cboTech.Text
strDate = calendarTest.SelectionStart
'Write to database if Production
If strOrderType = "PO" Then
'Check database for duplicate record
strSQL = "SELECT * FROM [New Camera Database] WHERE cameraSer=" & strcs
objCmd = New OleDbCommand(strSQL, dbconn)
dr = objCmd.ExecuteReader
dr.Read()
If dr("calCompleteDate").ToString <> "" Then
MsgBox("Camera S/N " & strcs & " completed " & dr("calCompleteDate") & ". Use Lookup to reprint Cert. of Compliance", vbOK + vbExclamation,
"Camera S/N " & strcs & " already completed")
exitFlag = True
Else
'Write to New Camera Database Table
strSQL = "UPDATE [New Camera Database] SET poNum=#poNum , calCompleteDate=#calCompleteDate, calCompleteTech=#calCompleteTech WHERE cameraSer=" & strcs
objCmd = New OleDbCommand(strSQL, dbconn)
objCmd.Parameters.AddWithValue("#poNum", strPoRMA)
objCmd.Parameters.AddWithValue("#calCompleteDate", strcs)
objCmd.Parameters.AddWithValue("#calCompleteTech", strtech)
objCmd.ExecuteNonQuery()
'Write to up2DateTravelers Table
strSQL = "UPDATE up2DateTravelers SET poRMANum = #poRMANum, calCompleteDate = #calCompleteDate, calCompleteTech = #calCompleteTech WHERE cameraSer=" & strcs
objCmd = New OleDbCommand(strSQL, dbconn)
objCmd.Parameters.AddWithValue("#poRMANum", strPoRMA)
objCmd.Parameters.AddWithValue("#calCompleteDate", strcs)
objCmd.Parameters.AddWithValue("#calCompleteTech", strtech)
objCmd.ExecuteNonQuery()
End If
ElseIf strOrderType = "RMA" Then
'Create new functions, userform, etc (TBD)
End If
btnClear.PerformClick()
End Sub
I guess this line :
objCmd.Parameters.AddWithValue("#calCompleteDate", strcs)
is a mistake and that you wanted to use the Date :
objCmd.Parameters.AddWithValue("#calCompleteDate", strDate)
Also, Use Using and parametrized queries :
'Write to New Camera Database Table
strSQL = "UPDATE [New Camera Database] SET poNum=#poNum , calCompleteDate=#calCompleteDate, calCompleteTech=#calCompleteTech WHERE cameraSer=#cameraSer"
Using objCmd As New OleDbCommand(strSQL, dbconn)
objCmd.Parameters.AddWithValue("#poNum", strPoRMA)
objCmd.Parameters.AddWithValue("#calCompleteDate", strDate)
objCmd.Parameters.AddWithValue("#calCompleteTech", strtech)
objCmd.Parameters.AddWithValue("#cameraSer", strcs)
objCmd.ExecuteNonQuery()
End Using
'Write to up2DateTravelers Table
strSQL = "UPDATE up2DateTravelers SET poRMANum = #poRMANum, calCompleteDate = #calCompleteDate, calCompleteTech = #calCompleteTech WHERE cameraSer=#cameraSer"
Using objCmd As New OleDbCommand(strSQL, dbconn)
objCmd.Parameters.AddWithValue("#poRMANum", strPoRMA)
objCmd.Parameters.AddWithValue("#calCompleteDate", strDate)
objCmd.Parameters.AddWithValue("#calCompleteTech", strtech)
objCmd.Parameters.AddWithValue("#cameraSer", strcs)
objCmd.ExecuteNonQuery()
End Using

how to save all record show in datagridview to the database

i have this code that will save only the top row of the datagridview,
can someone help me to modify this code so that it will save all the row in datagridview. im using vb 2010 and my database is ms access. thankyou in advance.
Try
Dim cnn As New OleDbConnection(conString)
query = "Insert into tblreportlog(EmpID,empname,department,empdate,timeinaM,timeoutam,lateam,timeinpm,timeoutpm,latepm,thw) values ('" & dgvReport.Item(0, dgvReport.CurrentRow.Index).Value & "', '" & dgvReport.Item(1, dgvReport.CurrentRow.Index).Value & "', '" & dgvReport.Item(2, dgvReport.CurrentRow.Index).Value & "','" & dgvReport.Item(3, dgvReport.CurrentRow.Index).Value & "','" & dgvReport.Item(4, dgvReport.CurrentRow.Index).Value & "','" & dgvReport.Item(5, dgvReport.CurrentRow.Index).Value & "','" & dgvReport.Item(6, dgvReport.CurrentRow.Index).Value & "','" & dgvReport.Item(7, dgvReport.CurrentRow.Index).Value & "', '" & dgvReport.Item(8, dgvReport.CurrentRow.Index).Value & "','" & dgvReport.Item(9, dgvReport.CurrentRow.Index).Value & "','" & dgvReport.Item(10, dgvReport.CurrentRow.Index).Value & "')"
cmd = New OleDbCommand(query, cnn)
cnn.Open()
cmd.ExecuteNonQuery()
cnn.Close()
Catch ex As Exception
MsgBox("ERROR: " & ErrorToString(), MsgBoxStyle.Critical)
End Try
Working from what is shown and best practices injected, you should be working from a data source such as a DataTable e.g. if when presented the DataGridView to the user there are no rows then create a new DataTable, set the DataTable as the DataSource of the DataGridView then when you are ready to save these rows in the DataGridView cast the DataSource of the DataGridView to a DataTable and use logic similar to the following
Dim dt As DataTable = CType(DataGridView1.DataSource, DataTable)
If dt.Rows.Count > 0 Then
Using cn As New OleDb.OleDbConnection With {.ConnectionString = "Your connection string"}
' part field list done here
Using cmd As New OleDb.OleDbCommand With
{
.Connection = cn,
.CommandText = "Insert into tblreportlog(EmpID,empname,department) values (#EmpID,#empname,#department)"
}
' TODO - field names, field types
cmd.Parameters.AddRange(
{
{New OleDb.OleDbParameter With {.ParameterName = "#EmpID", .DbType = DbType.Int32}},
{New OleDb.OleDbParameter With {.ParameterName = "#empname", .DbType = DbType.Int32}},
{New OleDb.OleDbParameter With {.ParameterName = "#department", .DbType = DbType.String}}
}
)
Dim Affected As Integer = 0
cn.Open()
Try
For Each row As DataRow In dt.Rows
' this should not be a auto-incrementing key
cmd.Parameters("#EmpID").Value = row.Field(Of Integer)("FieldName goes here")
cmd.Parameters("#empname").Value = row.Field(Of Integer)("FieldName goes here")
cmd.Parameters("#department").Value = row.Field(Of String)("FieldName goes here")
Affected = cmd.ExecuteNonQuery
If Affected <> 1 Then
Console.WriteLine("Error message, insert failed")
End If
Next
Catch ex As Exception
'
' handle exception
'
' for now
MessageBox.Show("Failed with: " & ex.Message)
' decide to continue or not
End Try
End Using
End Using
End If
On the other hand, if there are new rows with current rows we cast the data source as above then check for new rows along with validation as needed.
For Each row As DataRow In dt.Rows
If row.RowState = DataRowState.Added Then
If Not String.IsNullOrWhiteSpace(row.Field(Of String)("CompanyName")) Then
Other options, utilize a DataAdapter or setup data via data wizards in the ide where a BindingNavigator is setup with a save button.
If it's important to get the new primary key back the method for all methods can do this too.
The following code sample is from this MSDN code sample that shows how to get a new key back using OleDb connection and command.
Public Function AddNewRow(ByVal CompanyName As String, ByVal ContactName As String, ByVal ContactTitle As String, ByRef Identfier As Integer) As Boolean
Dim Success As Boolean = True
Try
Using cn As New OleDb.OleDbConnection(Builder.ConnectionString)
Using cmd As New OleDb.OleDbCommand("", cn)
cmd.CommandText = "INSERT INTO Customer (CompanyName,ContactName,ContactTitle) Values (#CompanyName,#ContactName,#ContactTitle)"
cmd.Parameters.AddWithValue("#CompanyName", CompanyName.Trim)
cmd.Parameters.AddWithValue("#ContactName", ContactName.Trim)
cmd.Parameters.AddWithValue("#ContactTitle", ContactTitle.Trim)
cn.Open()
cmd.ExecuteNonQuery()
cmd.CommandText = "Select ##Identity"
Identfier = CInt(cmd.ExecuteScalar)
End Using
End Using
Catch ex As Exception
Success = False
End Try
Return Success
End Function

Insert Multiple Records into Access DB

What is the most efficient way to insert multiple records into an Access DB with VB.net?
I have a list of objects with multiple properties which are the values for an INSERT query and I want to know can I insert them all together instead of looping through the list of objects, building the query string and executing the queries one by one which is very slow.
Rough example of what I have:
For Each Val In ValueList
ValueString = Val.X.ToString & ", "
ValueString += Val.Y.ToString & ", "
ValueString += Val.Z.ToString
SQLValueList.Add(ValueString)
Next
Dim cmd As OleDb.OleDbCommand
Dim strConnection As String
Dim strSql As String = Nothing
strConnection = _
"Provider=Microsoft.ACE.OLEDB.12.0;" & _
"Data Source=C:\db.accdb;" & _
"User ID=Admin;Password=;"
For Each ValueString As String In SQLValueList
strSql = "INSERT INTO Results (FldX, FldY, FldZ)" &
"VALUES ( " & ValueString & ");"
cmd = New OleDb.OleDbCommand(strSql)
cmd.Connection = New OleDb.OleDbConnection(strConnection)
cmd.Connection.Open()
cmd.ExecuteNonQuery()
Next
I'm assuming there is a much better and more efficient way of doing this but I haven't been able to find it!
Yes a parameterized query
Imports System.Data.OleDb
.......
Dim strConnection As String
Dim strSql As String = Nothing
strConnection = _
"Provider=Microsoft.ACE.OLEDB.12.0;" & _
"Data Source=C:\db.accdb;" & _
"User ID=Admin;Password=;"
strSql = "INSERT INTO Results (FldX, FldY, FldZ) VALUES ( ?, ?, ?)"
using cn = new OleDbConnection(strConnection)
using cmd = new OleDbCommand(strSql, cn)
cn.Open()
' HERE all the parameters are added with a string dummy value. '
' This should be changed if one of the underlying field is of different type '
' For example, if FldX is of type integer your need to write '
' cmd.Parameters.AddWithValue("#p1", 0) and then in the loop code '
' '
' cmd.Parameters(0).Value = val.X or '
' cmd.Parameters(0).Value = Convert.ToInt32(val.X) if val.X is not an integer but convertible to... '
cmd.Parameters.AddWithValue("#p1", "")
cmd.Parameters.AddWithValue("#p2", "")
cmd.Parameters.AddWithValue("#p3", "")
For Each val In ValueList
cmd.Parameters(0).Value = val.X.ToString()
cmd.Parameters(1).Value = val.Y.ToString()
cmd.Parameters(2).Value = val.Z.ToString()
cmd.ExecuteNonQuery()
Next
End Using
End Using
This is just an example because it is not clear what kind of data is stored in your ValueList (strings, integers, doubles dates?), but I hope that the idea is clear. Create a command object with 3 parameters (one for each field to insert), add every parameter to the command collection with dummy values (in the example, every parameter contains a string value but you need to add the correct datatype for the underlying field type). At this point just loop one time on your values and execute the query.
Please, stay away to string concatenation to build an sql command, expecially when the string values to concatenate are typed by your user. You risk an Sql Injection attack