SqlConnection.Open() Establishing a connection EVERY query? - sql

In my Winforms app which uses a remote database, I have the function below. (I also have two other functions which work similarly: One for scalar queries which returns zero upon failure, the other for updates and inserts which returns false upon failure.)
Currently, ALL data manipulation is pumped through these three functions.
It works fine, but overall please advise if I'd be better off establishing the connection upon launching my app, then closing it as the app is killed? Or at another time? (Again, it's a windows forms app, so it has the potential to be sitting stagnant while a user takes a long lunch break.)
So far, I'm not seeing any ill effects as everything seems to happen "in the blink of an eye"... but am I getting data slower, or are there any other potential hazards, such as memory leaks? Please notice I am closing the connection no matter how the function terminates.
Public Function GetData(ByVal Query As String) As DataTable
Dim Con As New SqlConnection(GlobalConnectionString)
Dim da As New SqlDataAdapter
Dim dt As New DataTable
Try
Con.Open()
da = New SqlDataAdapter(Query, Con)
Con.Close()
da.Fill(dt)
Catch ex As Exception
Debug.Print(Query)
MsgBox("UNABLE TO RETRIEVE DATA" & vbCrLf & vbCrLf & ex.Message, MsgBoxStyle.Critical, "Unable to retrieve data.")
End Try
da.Dispose()
Con.Close()
Return dt
End Function

There are exceptions to this, but best practices in .Net do indeed call for creating a brand new connection object for most queries. Really.
To understand why is, first understand actually connecting to a database involves lots of work in terms of protocol negotiations, authentication, and more. It's not cheap. To help with this, ADO.Net provides a built-in connection pooling feature. Most platforms take advantage of this to help keep connections efficient. The actual SqlConnection, MySqlConnection, or similar object used in your code is comparatively light weight. When you try to re-use that object, you're optimizing for the small thing (the wrapper) at the expense of the much larger thing (the actual underlying connection resources).
Aside from the benefits created from connection pooling, using a new connection object makes it easier for your app to scale to multiple threads. Imagine writing an app which tries to rely on a single global connection object. Later you build a process which wants to spawn separate threads to work on a long-running task in the background, only to find your connection is blocked, or is itself blocking other normal access to the database. Worse, imagine trying to do this for a web app, and getting it wrong such that the single connection is shared for your entire Application Domain (all users to the site). This is a real thing I've seen happen.
So this is something that your existing code does right.
However, there are two serious problems with the existing method.
The first is that the author seems not to understand when to open and when to close a connection. Using the .Fill() method complicates this, because this method will open and close your connection all on its own.1 When using this method, there is no good reason to see a single call to .Open(), .Close(), or .Dispose() anywhere in that method. When not using the .Fill() method, connections should always be closed as part of a Finally block: and the easiest way to do that is with Using blocks.
The second is SQL Injection. The method as written allows no way to include parameter data in the query. It only allows a completed SQL command string. This practically forces you to write code that will be horribly vulnerable to SQL Injection attacks. If you don't already know what SQL Injection attacks are, stop whatever else you're doing and go spend some time Googling that phrase.
Let me suggest an alternative method for you to address these problems:
Public Function GetData(ByVal Query As String, ParamArray parameters() As SqlParameter) As DataTable
Dim result As New DataTable()
Using Con As New SqlConnection(GlobalConnectionString), _
Cmd As New SqlCommand(Query, Con),
da As New SqlDataAdpapter(Cmd)
If parameters IsNot Nothing Then Cmd.Parameters.AddRange(parameters)
Try
da.Fill(result)
Catch ex As Exception
Debug.Print(Query)
'Better to allow higher-level method to handle presenting the error to the user
'Just log it here and Rethrow so presentation tier can catch
Throw
End Try
End Using 'guarantees connection is not left hanging open
Return result
End Function
1See the first paragraph of the "Remarks" section.

This isn't a real "answer" to my own question, but I have something to add and I wanted to add some code.
To Joe: Thank you, my code is well on the way to using parameterized queries almost exclusively. Although I knew what SQL injection attacks were, and that they're a pretty big deal, here's my exuse: In the past I had used stored procedures for parameterized queries, and I kinda hate writing those and for the first year my code will be used only within by my small company of 5 employees who are family members... I had planned to switch everything to stored procedures later if I sold the software. This approach is better and I will probably not need stored procedures at all.
I especially like how elegantly parameterized queries handle dates, as I don't have to convert dates to appropriate text. Much easier.
Anopther advantage I'm seeing: Sometimes a "Save button" must execute either Insert or Update, depending on whether the record displayed is new. Using parameters allows me to write two alternate short basic queries, but to use the same parameters for either with less code.
Overall, this means a whole lot less code-intensive construction of the query string.
The part I didn't have, and I learned to do it elsewhere, was assigning the parameter array, calling the procedure, so I'm including an example here hoping others find it useful:
Dim query As String = "Select Phone from Employees where EmpNo = #EmployeeNumber and Age = #Age"
Dim params As SqlParameter() = {
New SqlParameter("#EmployeeNumber", txtEmployeeNumber.Value),
New SqlParameter("#Age", txtAge.Value)
}
Dim Phone as String = GetData(query, params).Rows.Item(0)

Related

Does OleDbTransaction.Commit() closes the connection?

I need to run a number of queries as part of a transaction.
There's a method in my project, creatively named RunQuery(), that concentrates the database queries, so I'll be using it. It has four parameters:
a string indicating which database to run the query on,
a string containing the query,
a table name for when the result needs to be a data set,
an Enum that indicates what kind of query/results is wanted: Select query returning a data reader, Select query returning a data set, Insert/Update/Delete query.
(If you need more detail, I'll supply it gladly. I'll keep this short-ish for now.)
The point is that when you want to run a query, you don't have to worry about the connection or command objects, you just write your query and call RunQuery(). Fine.
Now, I've added a new ByRef parameter to RunQuery(), an IDbTransaction to track the transaction. It needs to be handled at least partially outside RunQuery(), if only to commit after the very last call, but on the first call it's set at Nothing.
I've also added an item to the Enum so RunQuery() knows it has to deal with a transaction.
My code looks like this:
Dim z_lisQuery As New List(Of String)
' [...] Filling z_lisQuery with queries
Dim z_dtrTransaction As IDbTransaction = Nothing
Dim z_blnExecutionOk As Boolean = True
For Each z_strQuery As String In z_lisQuery
z_blnExecutionOk = z_blnExecutionOk And RunQuery(p_strDbId,
z_strQuery,
"",
z_dtrTransaction,
QueryAction.ExecutionTransaction)
If Not z_blnExecutionOk Then
Exit For
End If
Next
If z_blnExecutionOk Then
z_dtrTransaction.Commit()
End If
z_dtrTransaction?.Connection.Dispose()
z_dtrTransaction?.Dispose()
(A Rollback() is executed within RunQuery() if something goes wrong.)
When I run that and no errors happen, everything goes fine until the penultimate line:
z_dtrTransaction?.Connection.Dispose() throws an exception because z_dtrTransaction.Connection is Nothing.
The obvious workaround is to use z_dtrTransaction?.Connection?.Dispose(), so that's not what I'm puzzled about.
What I'm puzzled about is that z_dtrTransaction.Connection is still a functional OleDbConnection object when execution reaches z_dtrTransaction.Commit(). But after z_dtrTransaction.Commit() is executed, z_dtrTransaction.Connection is Nothing.
I can see the point of that, but the language reference doesn't seem to indicate that it should happen that way and I haven't found a reference to that way of doing things.
Is it officially stated somewhere that this behavior is the normal one? Or is it just with OleDbConnection and not with SqlConnection, for instance?
(Even if it is, obviously I'm keeping z_dtrTransaction?.Connection?.Dispose() because I'm not entirely sure what happens in case of a rollback.)

Use same DataAdapter object for multiple queries causing RaceOnRCWCleanup

I have an application which does quite a few database queries. In order to speed up the application I do some queries which are independent of each other (it doesn't matter which order they get updated in as they aren't referencing one another) on a separate thread.
However, it appears that if two queries are executed at the same time on the different threads, but one finishes earlier, the DataAdapter attempts to close the connection which is still being used on another thread resulting in a RaceOnRCWCleanup warning:
An attempt has been made to free an RCW that is in use. The RCW is in use on the active thread or another thread. Attempting to free an in-use RCW can cause corruption or data loss.
I may be incorrect about this assumption but if I don't call any queries on the secondary thread, or delay them, I don't seem to get this issue. If I force the queries to happen at the same time, I get the warning.
My query functions are, with a few exceptions, all structured as such:
Dim strSQL As String = "SELECT..."
Try
da = New OleDb.OleDbDataAdapter(strSQL, conn)
da.Fill(dsData, "Progress")
Catch ex As Exception
MessageBox.Show(ex.Message)
End Try
Return dsData.Tables("Progress")
And da is declared in a separate module as:
Public da As OleDbDataAdapter
I thought by declaring da = New I force it to be a new object and therefore, closing the connection on one should not effect the other?
Where am I going wrong here?
I believe I have solved this by using a separate OleDbConnection object for each thread (I never call the same functions across different threads).
This way a single connection object is only ever used in serial.
I don't know if this is the correct way to handle this but it seems to work.

Specified cast is not valid with datareader

Hi friends of stackoverflow,
I'm writing a question here because i'm having problems to detect why sometimes a read to a field of a datareader returns a invalid cast exception.
I will give all information possible to understand my situation. I'm working with ASP.NET 3.5
I have a Module that have a function that returns a IDataReader and receives a sql query. something like this
function get_dr(query as String) as IDataReader
dim connection = new SqlClient.SqlConnection("connection string")
connection.open()
dim command = connection.createCommand
command.commandText = query
dim reader = command.executeReader(CommandBehavior.CloseConnection)
return reader
end function
I have a Class with a Shared function that recovers a new dataReader and returns a date. something like this:
public shared function getDate() as Date
using dr = get_dr("SELECT dbo.getDate()")
if dr.read() and dr(0) isnot dbnull.value then
return dr.GetDateTime(0)
end if
end using
end function
when in another code i call the getDate() function, it gives me a call stack like this.
System.InvalidCastException: Specified cast is not valid.
at System.Data.SqlClient.SqlBuffer.get_DateTime()
at System.Data.SqlClient.SqlDataReader.GetDateTime(Int32 i)
Why sometimes i'm getting this error? i was thinking this is because that a lot of users is calling this function in conjunction with another functions of my application (those functions eventually uses get_dr too), mixing the data of the dataReader on another executions, but i need to know if im doing something wrong or maybe to do something better.
Notes:
dbo.getDate is a sql function that ALWAYS returns a date.
don't worry about bad writing code, those are only examples but they have the necessary to understand the scenario.
sorry for my bad english
Really thanks in advance.
One possible reason - you declare connection inside of the function that returns DataReader. When you're out of the function that connection goes out of scope. That means that at some unpredictable point (depends on memory usage etc.) Garbage Collector will collect it. If you try to use the DataReader at that point - all bets are off.
One way to solve it is to declare connection outside of function get_dr and pass it there as a parameter. But also seeing that you're returning a single value and if you don't plan to use the reader for multiple values I suggest using ExecuteScalar instead - it will save you a lot of headaches.

Simultaneous OleDbDataAdapter.Fill Calls on Separate Threads?

First timer here, so go easy on me. Is it theoretically possible to execute two OleDBDataAdapter.Fill calls on separate threads simultaneously - or is that fundamentally flawed?
Consider a form with 2 buttons and 2 datagridviews. Each button click launches a worker thread using an Async \ Await \ Task.Run pattern that calls a method to return a populated datatable and assigns it to one of the datagridviews. The .Fill in the first thread takes 30 seconds to complete. The .Fill in the second thread takes 1 second to complete. When launched individually, both buttons work as expected.
However, if I launch the first worker thread (30 seconds to Fill), then launch the second thread (1 second Fill), the second DataGridView is not populated until the first .Fill call completes. I would expect the second datagridview to populate in 1 second, and the first datagridview to populate ~30 seconds later.
I have duplicated this issue in my sample code with both the OleDBDataAdapter and the SqlDataAdapter. If I replace the long running query with a simple Thread.Sleep(30000), the second datagridview is populated right away. This leads me to be believe that it is not an issue with my design pattern, rather something specific to issuing the .Fill calls simultaneously.
Private Async Sub UltraButton1_Click(sender As Object, e As EventArgs) Handles UltraButton1.Click
Dim Args As New GetDataArguments
Args.ConnectionString = "some connection string"
Args.Query = "SELECT LongRunningQuery from Table"
Dim DT As DataTable = Await Task.Run(Function() FillDataTable(Args))
If DataGridView1.DataSource Is Nothing Then
DataGridView1.DataSource = DT
Else
CType(DataGridView1.DataSource, DataTable).Merge(DT)
End If
End Sub
Function FillDataTable(Args As GetDataArguments) As DataTable
Dim DS As New DataTable
Using Connection As New OleDbConnection(Args.ConnectionString)
Using DBCommand As New OleDbCommand(Args.Query, Connection)
Using DataAdapter As New OleDbDataAdapter(DBCommand)
DataAdapter.Fill(DS)
End Using
End Using
End Using
Return DS
End Function
Private Async Sub UltraButton2_Click(sender As Object, e As EventArgs) Handles UltraButton2.Click
Dim DS As DataTable = Await Task.Run(Function() LoadSecondDGV("1234"))
DataGridView2.DataSource = DS
End Sub
Function LoadSecondDGV(pnum As String) As DataTable
Dim DX As New DataTable
Using xConn As New OleDbConnection("some connection string")
Using DataAdapter As New OleDbDataAdapter("Select name from products where PNUM = """ & pnum & """", xConn)
DataAdapter.Fill(DX)
End Using
End Using
Return DX
End Function
This depends on what the data source is. Some data sources (like Excel) will only allow one connection at a time. Other data sources (like Access) will allow multiple connections, but actually fulfill the results in serial, such that you don't gain anything. Other data sources, like Sql Server, will allow the true parallel activity that you're looking for.
In this case, you mention that you also tried with an SqlDataAdapter, which indicates to me that you're talking to Sql Server, and this should be possible. What's probably going on here is that your first query is locking some of the data you need for the second query. You can get past this by changing your transaction isolation level or through careful use of the with (nolock) hint (the former option is strongly preferred).
One other thing to keep in mind is that this can only work if you're using a separate connection for each query, or if you've specifically enabled the Multiple Active Result Sets feature. It looks like you're using separate connection objects here, so you should be fine, but it's still something I thought was worth bringing up.
Finally, I need to comment on your FillDataTable() method. This method requires you to provide a completed Sql string, which practically forces you to write code that will be horribly vulnerable to sql injection attacks. Continuing to use the method as shown practically guarantees your app will get hacked, probably sooner rather than later. You need to modify this method so that it encourages you to use parameterized queries.

Closing Oracle connection in VB.NET

In my program I got multiple spots where I open a Connection to an Oracle database, read some lines via a stored procedure which returns me a cursor, put them in an IDataReader, close the connection and go on.
Now everything works fine till the point of closing the connection. I've watched the connections opened to the database via TOAD for Oracle and I figured out, that the database seems to keep an connection opened after I said m_Connection.Close and m_Connection.Dispose.
I use Oracle.DataAccess and get an error after a short time, that there are to many connections. I debuged my session and made sure, that vb performs the close() command and it does it, but the database not. Anyone got an idea how to fix this?
In .Net, you need to wrap database connections in a Try/Catch/Finally block, and always close the connections in the Finally section. There is a shorthand for this called a Using block. This means keeping a connection as member of a class (as you seem to be doing) is almost always the wrong way to go about it. .Net is optimized so that it's better to create a new connection and command object for each query.
DataReaders are a little special: if you return a datareader out of using block, the connection can be closed before your DataReader is done with it. In C#, I normally get around the problem with an iterator (yield return). Since VB.Net lacks support for this construct, I might use an Action(Of IDataRecord) instead, like so:
Public Sub MyOracleQuery(ByVal id As Integer, ByVal ProcessRecord As Action(Of IDataRecord))
Dim sql As String = "SELECT <columns> FROM MyTable WHERE ID= #Id"
Using cn As New OracleConnection("connection string"), _
cmd As New OracleCommand(sql, cn)
cmd.Parameters.Add("#Id", SqlDbTypes.Int).Value = id
cn.Open()
Using (rdr As IDataReader = cmd.ExecuteReader())
While rdr.Read()
ProcessRecord(rdr)
End While
End Using
End Using
End Sub
You can now pass an anonymous method to this code when you call it:
Dim id As Integer
If Integer.TryParse(IDTextBox.Text, id) Then
MyOracleQuery(id, _
Function(r)
''#... Do something with each "r" here
End Function _
)
Else
''# Complain to user about invalid ID
End If
Note that this requires Visual Studio 2010 /.Net 4 for the mutli-line anonymous method. For older platforms, you'll want to declare the method.