I have this generic routine that I wrote that takes a list of sql strings and executes them against the database. Is there any way I can make this work faster? Typically it'll see maybe 200 inserts or deletes or updates at a time. Sometimes there is a mixture of updates, inserts and deletes. Would it be a good idea to separate the queries by type (i.e. group inserts together, then updates and then deletes)?
I am running this against an ms access database and using vb.net 2005.
Public Function ExecuteNonQuery(ByVal sql As List(Of String), ByVal dbConnection as String) As Integer
If sql Is Nothing OrElse sql.Count = 0 Then Return 0
Dim recordCount As Integer = 0
Using connection As New OleDb.OleDbConnection(dbConnection)
connection.Open()
Dim transaction As OleDb.OleDbTransaction = connection.BeginTransaction()
'Using cmd As New OleDb.OleDbCommand()
Using cmd As OleDb.OleDbCommand = connection.CreateCommand
cmd.Connection = connection
cmd.Transaction = transaction
For Each s As String In sql
If Not String.IsNullOrEmpty(s) Then
cmd.CommandText = s
recordCount += cmd.ExecuteNonQuery()
End If
Next
transaction.Commit()
End Using
End Using
Return recordCount
End Function
You can use a data adapter to update the whole dataset at once. It will be faster to run the queries on the ADO object than the database directly. After the batch has cycled, update the whole dataset. That might be faster, but will require some extra coding up front and overhead on the application.
Related
I am a complete beginner to .net and am confused at some basic things. Please help.
First of all the table I create and populate (by right clicking tables in server explorer) disappear once I restart the computer. how do I keep them.
Is there any better place/interface to type SQL queries in vb.net than the command prompt.
In the following code:
Dim cn As SqlConnection = New SqlConnection(strConnection)
cn.Open( )
' Create a data adapter object and set its SELECT command.
Dim strSelect As String = _
"SELECT * FROM Categories"
Dim da As SqlDataAdapter = New SqlDataAdapter(strSelect, cn)
' Load a data set.
Dim ds As DataSet = New DataSet( )
da.Fill(ds, "Categories")
This far the code runs fine but just to gain better understanding, I would like to ask that
while data from SQL Server database was saved into da in accordance to the query, why do we need to save/transfer it in the dataset object ds.
Is there any additional benefit of SqlCommand over SqlDataAdapter besides speed?
Dim autogen As New SqlCommandBuilder(da)
Dim dt As DataTable = ds.Tables("Categories")
' Modify one of the records.
Dim row As DataRow = dt.Select("CategoryName = 'Dairy Products'")(0)
row("Description") = "Milk and stuff"
gives an error when I use it with
da.Update(ds, "Categories")
regarding dt.select not returning any value.
What is the way out?
to answer your questions :
The tables you create with the server explorer are IN MEMORY. Same goes for dataset, they are in-memory representation of your table. As for your 2nd example, the DS you use isnt filled when you try to get the DT. hence why the DT is empty.
If your starting, I would suggest you go look into Linq-to-Sql (http://msdn.microsoft.com/en-us/library/bb425822.aspx) for a more up-to-date way of doing sql in .net ( I think its 4.0 framework)
As for the 2nd point, I'd say normally you should use store procedure for most of your sql commands .. the sqlcommand is use like this
Try
Cmd = New SqlClient.SqlCommand("st_InventoryStatus_Or_AnyStoreProcName_Or_ASqlQuery")
Cmd.CommandTimeout = 300 'not really needed'
Cmd.CommandType = CommandType.StoredProcedure 'you can type CommandType.Text here to use directly your "Select * from Category"'
Cmd.Parameters.Clear() 'just to be sure its empty, its not mandatory'
Cmd.Parameters.Add("#idCategory", SqlDbType.Int).Value = myCategory.Id 'here are the parameters of your store proc, or of your query ("select * from Category where Category.id = #Id")'
Catch ex As Exception
MsgBox(ex.Message, MsgBoxStyle.Information)
End Try
I got an issue that I am completely stumped on.
Part of my application calls a Stored Proc using SQLConnection/SQLCommand. I'm hitting a SQL 2005 database and I am able to make the connection and execute the SP just fine. The problem is it periodically executes the SP multiple times; some times twice, some times three times.
This is basically how I execute the SP...
Dim conString As String = "<Typical Connection String>"
Dim cn As SqlConnection = new SqlConnection(conString)
Dim cmd As SqlCommand = New SqlCommand("dbo.JobStoredProc", cn)
cmd.CommandType = CommandType.StoredProcedure
cmd.Parameters.Add("#Val", SqlDbType.VarChar, 12).Value = "Test Value"
cn.Open()
Dim queryResult As Integer = cmd.ExecuteNonQuery
cn.Close()
cn.Dispose()
I can't figure out why sometimes it executes only once, but other times it executes multiple times. Is there something I'm missing? Is there a better way to go about executing the SP?
Thank you very much in advance!
As it turns out it was because I had two of the same File Watchers looking at the same directory. This was causing the above function to fire twice at the exact same time.
Can anyone give me the source code to add database connectivity in VB2010 express through ADO.Net. Including all the commands to add, update, delete, retrieve and modify the database fields. It would be really helpful if anyone can provide me with a small prototype working model with the source code.
ADO.NET is more or less SQL query based. So for CRUD (Create, Read, Update, Delete) Operations have a look at the SQL-Language (the query syntax might have some smalle differences depending on the database you're using).
The connection uses the specialized provider entities implementing the IDbConnection, IDbCommand, IDbDataAdapter, IDbDataParameter and IDbTransaction Interfaces from the System.Data Namespace.
There are different Database Providers (e.g. Microsoft SQL Server, Oracle, mySQl, OleDb, ODBC etc.). Some of them are natively supported by the .NET Framework (MSSQL=System.Data.SqlClient Namespace, OleDb=System.Data.OleDb, ODBC=System.Data.Odbc Namespace) while others must be added through external libraries (you can also write your own database provider if you like).
With the IDBCommand Objects (e.g. the System.Data.SqlClient.SqlCommand object) you can define your SQL Commands.
Here is a small sample snippet which might help:
Public Class Form1
Sub DBTest()
'** Values to store the database values in
Dim col1 As String = "", col2 As String = ""
'** Open a connection (change the connectionstring to an appropriate value
'** for your database or load it from a config file)
Using conn As New SqlClient.SqlConnection("YourConnectionString")
'** Open the connection
conn.Open()
'** Create a Command object
Using cmd As SqlClient.SqlCommand = conn.CreateCommand()
'** Set the command text (=> SQL Query)
cmd.CommandText = "SELECT ID, Col1, Col2 FROM YourTable WHERE ID = #ID"
'** Add parameters
cmd.Parameters.Add("#ID", SqlDbType.Int).Value = 100 '** Change to variable
'** Execute the value and get the reader object, since we are trying to
'** get a result from the query, for INSERT, UPDATE, DELETE use
'** "ExecuteNonQuery" method which returns an Integer
Using reader As SqlClient.SqlDataReader = cmd.ExecuteReader()
'** Check if the result has returned som results and read the first record
'** If you have multiple records execute the Read() method until it returns false
If reader.HasRows AndAlso reader.Read() Then
'** Read the values of the current columns
col1 = reader("col1")
col2 = reader("col2")
End If
End Using
End Using
Debug.Print("Col1={0},Col2={1}", col1, col2)
'** Close the connection
conn.Close()
End Using
End Sub
End Class
I've submitted a bunch of questions as of late - but this has been a great repository of information. I'm a .NET nub, as you can see, so if I'm missing basics or information please let me know and I'll try and fill in the gaps.
I'm using ASP.NET/VB.NET to create this with SQL 2005. I have a project where I'd like to take a set of records from a table, then send each one through an API, get a result and writeback the result to a table, then move to the next.
Initially, my thought was create a SQLDataSource that grabs all the records, then perform the action on a button to do the action of sending through each record.
Is there a way I can call the recordset from SQLDataSource and perform a loop? I'm thinking something like in Classic ASP/VBScript where you would open a RecordSet, do an action, then Loop until the RS was EoF.
Thanks for the help!
You can may want to put your results in a dataset. After getting the results, you can loop through the returned rows
Dim ds As Dataset = GetSomeDataFromSq
For Each dr As DataRow In ds.Tables(0).Rows
Console.WriteLine (dr("ColName"))
Next
You could also use a sqlDataReader
Using conn As sqlconnection = New sqlconnection("put conn info")
Using MyCommand As SqlCommand = New SqlCommand("SELECT ProductName FROM products", conn)
conn.Open()
Using myDataREader As SqlDataReader = MyCommand.ExecuteReader
While myDataREader.Read
Response.Write("Name: " & myDataREader.Item("ProductName"))
End While
End Using
End Using
End Using
My application requires a user to log in and allows them to edit a list of things. However, it seems that if the same user always logs in and out and edits the list, this user will run into a "System.Data.SqlClient.SqlException: Timeout expired." error. I've read a comment about it possibly caused by uncommitted transactions. And I do have one going in the application.
I'll provide the code I'm working with and there is an IF statement in there that I was a little iffy about but it seemed like a reasonable thing to do.
I'll just go over what's going on here, there is a list of objects to update or add into the database. New objects created in the application are given an ID of 0 while existing objects have their own ID's generated from the DB. If the user chooses to delete some objects, their IDs are stored in a separate list of Integers. Once the user is ready to save their changes, the two lists are passed into this method. By use of the IF statement, objects with ID of 0 are added (using the Add stored procedure) and those objects with non-zero IDs are updated (using the Update stored procedure). After all this, a FOR loop goes through all the integers in the "removal" list and uses the Delete stored procedure to remove them. A transaction is used for all this.
Public Shared Sub UpdateSomethings(ByVal SomethingList As List(Of Something), ByVal RemovalList As List(Of Integer))
Using DBConnection As New SqlConnection(conn)
DBConnection.Open()
Dim MyTransaction As SqlTransaction
MyTransaction = DBConnection.BeginTransaction()
Try
Using MyCommand As New SqlCommand()
MyCommand.Transaction = MyTransaction
MyCommand.CommandType = CommandType.StoredProcedure
For Each SomethingItem As Something In SomethingList
MyCommand.Connection = DBConnection
If SomethingItem.ID > 0 Then
MyCommand.CommandText = "UpdateSomething"
Else
MyCommand.CommandText = "AddSomething"
End If
MyCommand.Parameters.Clear()
With MyCommand.Parameters
If MyCommand.CommandText = "UpdateSomething" Then
.Add("#id", SqlDbType.Int).Value = SomethingItem.ID
End If
.Add("#stuff", SqlDbType.Varchar).Value = SomethingItem.Stuff
End With
MyCommand.ExecuteNonQuery()
Next
MyCommand.CommandText = "DeleteSomething"
For Each ID As Integer In RemovalList
MyCommand.Parameters.Clear()
With MyCommand.Parameters
.Add("#id", SqlDbType.Int).Value = ID
End With
MyCommand.ExecuteNonQuery()
Next
End Using
MyTransaction.Commit()
Catch ex As Exception
MyTransaction.Rollback()
'Exception handling goes here '
End Try
End Using
End Sub
There are three stored procedures used here as well as some looping so I can see how something can be holding everything up if the list is large enough.
I'm using Visual Studio 2008 to debug and am using SQL Server 2000 for the DB.
Edit: I still seem to be getting this error. I've even removed the whole transaction thing and I still encounter it. At this point, I'm assuming there is some kind of leak happening here. I've tried not using the USING statements and explicitly tell the command and connection to dispose itself but no dice. Memory usage by SQL Server also increases quite a bit if this method is called a lot in a short period of time.
I've read that increasing the CommandTimeout property of the SQLCommand would help. I'm wondering if there are any big disadvantages or consequences from doing so.
I would suggest using the following, that way Dispose will always be called and be Rolledback in every non-committed case.
using (SqlConnection sqlCn = new SqlConnection())
{
using (SqlTransaction myTrans = sqlCn.BeginTransaction())
{
...
myTrans.Commit();
}
}
Also, I don't believe you need to make a new SqlCommand for every execution. Just maintain the same one and update the CommandText and Parameters.
If you have a large number of commands, you may want to build them all before opening the connection. After you start the transaction and open the connection, spin through and execute them.
You probably want to use TransactionScope
Using _tx as New System.Transactions.TransactionScope(<add your own timeout here>)
'Do all your sql work'
If _noErrors Then
_tx.Complete()
End If
End Using
With the transaction scope, you can set a timeout of up to 20 minutes without modifying server settings.
I believe I have managed to solve the problem. I have modified the application so that unnecessary calls to the database are not made (i.e. unchanged objects do not need to be updated again) and increased the CommandTimeout property for the SQLCommand object. So far, no problems.
Big thanks for suggestions too.