I am familiar with the VB6 ADO way of dealing with SQL queries and looping through the record set results.
However, what is the correct way to query a server, cycle through the results, and dispose of my query in VB.Net? All the ways I have been using seem to be unstable and crash randomly.
I have been using the following code:
Public Function GetSQLTable(ByVal strSQL As String) As DataTable
Dim table As New DataTable
Dim adapt As SqlDataAdapter
Try
adapt = New SqlDataAdapter(strSQL, gconIntegration)
adapt.Fill(table)
Catch ex As Exception
LogError("GetSQLTable: " & ex.ToString(), "SQL: " & strSQL)
End Try
Return table
End Function
And using it like this:
Dim dt As DataTable
Dim lngRow As Long
Dim current As DataRow
Dim lngContact As long
Try
dt = GetSQLTable(strSQL)
For lngRow = 0 To dt.Rows.Count - 1
current = dt.Rows.Item(lngRow)
lngContact = current.Item("indvid")
DoSomething(lngContact)
Next
Catch ex As Exception
LogError("FindContact: " & ex.ToString(), "SQL: " & strSQL)
lngContact = -1
Finally
current = nothing
dt = nothing
I suspect the problem has to do with how you manage your gconIntegration connection. You're trying too hard to keep using that same connection. It would be helpful to see where it lives.
Better to get "new" connections from the pool and let .Net worry about it for you.
Also, your generic "GetSQLTable" code is missing an important part: it makes no allowance for setting parameters, which tells me you're building them directly into your query strings. That's a recipe for disaster: it will lead to Sql injection security holes.
One more thing: don't set objects to Nothing in .Net. Either dispose them if needed or let them fall out of scope on their own.
Here's my normal method for pulling back a datatable from a datatable:
Function GetSomeData(ByVal Table2ID As Integer)
Dim result As New DataTable
Dim sql As String = "SELECT Column1,Column2 FROM [Table1] WHERE Table2ID= #Table2ID"
Using cn As New SqlConnection( GetConnectionString() ), _
Using cmd As New SqlCommand(sql, cn)
cmd.Parameters.Add("#Table2ID", SqlDbType.Int).Value = Table2ID
Using rdr As SqlDataReader = cmd.ExecuteReader()
result.Load(rdr)
End Using
End Using
return result
End Function
Some notes on that code:
The Using statement will guarantee that the associated object is disposed at the corresponding End Using.
The query parameters are kept strongly typed, and never substituted directly into the query string, even when they are transmitted to the server. Sql Data and Sql Code never mix.
You do need a separate function for each query you need to send. This is really a good thing, as it leads into building a strongly-typed interface for your database. Ideally all of these functions are in the same class, and the GetConnectionString function is private to that class. No database access happens outside of this data layer.
Related
I have a vb.net application program that is suppose to query a oracle/labdaq database and load the dataset into a datatable. For some reason the query works fine and there is no exception thrown, however, when I load the data it seems to be missing the first row. I first noticed the issue when I did a query for a single row and it returned zero rows when I checked the datatable's row amount during debugging. I then compared all my data sets to a data miner application straight from the oracle source and i seems to always be missing one, the first, row of data when I use the application.
here is the code... I changed the query string to something else to maintain company privacy
Private Sub CaqOnSQL(strFileDirect As String)
Try
Dim connString As String = ConfigurationManager.ConnectionStrings("CA_Requisition_Attachments.Internal.ConnectionString").ConnectionString
Dim conn As New OracleConnection With {
.ConnectionString = connString
}
Dim strQuerySQL As String = "SELECT * FROM REQUISITIONS " &
"WHERE DATE BETWEEN TO_DATE('12/10/2020','MM/dd/yyyy') AND " &
"TO_DATE('12/14/2020','MM/dd/yyyy') " &
"ORDER BY ID"
conn.Open()
Dim Cmd As New OracleCommand(strQuerySQL, conn) With {
.CommandType = CommandType.Text
}
Dim dr As OracleDataReader = Cmd.ExecuteReader()
dr.read()
Dim dt As New DataTable
dt.TableName = "RESULTS"
dt.Load(dr)
ExcelFileCreation(dt, strFileDirect)
You should remove the line:
dr.read()
The call to Read is what is causing you to skip the first row, when combined with the DataTable Load method.
In addition, I have taken the liberty to make some additional changes to your code for the purposes of Good Practice. When using Database objects like Connection and Command, you should wrap them in Using blocks to ensure the resources are released as soon as possible.
Dim dt As New DataTable()
dt.TableName = "RESULTS"
Using conn As New OracleConnection(connString)
conn.Open()
Dim strQuerySQL As String = "SELECT * FROM REQUISITIONS " &
"WHERE DATE BETWEEN TO_DATE('12/10/2020','MM/dd/yyyy') AND " &
"TO_DATE('12/14/2020','MM/dd/yyyy') " &
"ORDER BY ID"
Using command = New OracleCommand(strQuerySQL , conn)
Using dataReader = command.ExecuteReader()
dt.Load(dataReader)
End Using
End Using
End Using
Note: Be wary of Using blocks when using a DataReaderas you may find the connection is closed when you don't want it to be. In this case, the DataReader is used entirely within this function and is safe to use.
I am Inserting the data-table into SQLite Database. I am doing like this.
First I Fetch the data with getdata function and insert it into datatable, then with For Each Loop i made the Insert Command and Execute It. I am having 50000 Records it will take 30 Minutes to run.
Please Guide the suitable approach. Here is the Code.
Dim xtable As DataTable = getdata("select * from tablename")
Dim str As String = Nothing
For Each r As DataRow In xtable.Rows ''''HERE IT WILL TAKE TOO MUCH TIME
str = str & ("insert into tablename values(" & r.Item("srno") & "," & r.Item("name"));")
Next
EXECUTEcmd(str)
Public Function getdata(ByVal Query As String) As DataTable
connectionString()
Try
Dim mds As New DataTable
Dim mycommand As New SQLiteCommand(DBConn)
mycommand.CommandText = Query
Dim reader As SQLiteDataReader = mycommand.ExecuteReader()
mds.Load(reader)
Return mds
Catch ex As Exception
MsgBox("DB Error", vbCritical, "")
MsgBox(Err.Description)
Return Nothing
End Try
End Function
Public Sub EXECUTEcmd(ByVal selectcmd As String)
Using cn = New SQLiteConnection(conectionString)
cn.Open()
Using transaction = cn.BeginTransaction()
Using cmd = cn.CreateCommand()
cmd.CommandText = selectcmd
cmd.ExecuteNonQuery()
End Using
transaction.Commit()
End Using
cn.Close()
End Using
End Sub
here the Conncection String is:
conStr = "Data Source=" & dbpath & ";Version=3;Compress=True; UTF8Encoding=True; PRAGMA journal_mode=WAL; cache=shared;"
Use a stringbuilder to build your string, not string concatenation
Dim strB As StringBuilder = New StringBuilder(100 * 50000)
For Each r As DataRow In xtable.Rows
strB.AppendLine($"insert into tablename values({r.Item("srno")},{r.Item("name")});")
Next
Strings cannot be changed in .net. Every time you make a new string VB has to copy everything out of the old string into a new one and add the new bit you want. If each of your insert statements is 100 bytes, that means it copies 100 bytes, then adds 100, then copies 200 bytes and adds 100, then copies 300 bytes, then 400 bytes, then 500 bytes. By the time it has done 10 strings it has made 5.5 kilobytes of copying. By the time it's done 50 thousand strings it has copied 125 gigabytes of data. No wonder it's slow!
Always use a StringBuilder to build massive strings
--
I'm willing to overlook the sql injection hacking nag for this one, because of the nature of the task, but please read http://bobby-tables.com - you should never, ever concatenate values into an SQL as a way of making an sql that has some varying effect.
This entire exercise would be better done as this (pseudocode) kind of thing:
Dim sel as New SQLiteCommand("SELECT a, b FROM table", conn)
Dim ins as New SQLiteCommand("INSERT INTO table VALUES(:a, :b)", conn)
ins.Parameters.Add("a" ...)
ins.Parameters.Add("b" ...)
Dim r = sel.ExecuteReader()
While r.Read()
ins.Parameters("a") = r.GetString(0)
ins.Parameters("b") = r.GetString(1)
ins.ExecuteNonQuery()
End While
That is to say, you minimize your memory by reading rows one at a time out of ther edaer and inserting them one at a time in the insert; the insert command is prepared once, you just change the parameter values, execute it, change them again, execute it ... It's what parameterized queries were designed for (as well as stopping your app getting hacked when someone puts SQL in your variable, or even just stopping it crashing when you have an person named O'Grady
Maybe you must refactor your code like this:
Dim xtable As DataTable = getdata("select * from tablename")
Using cn = New SQLiteConnection(conectionString)
cn.Open()
Using transaction = cn.BeginTransaction()
Try
Using cmd = cn.CreateCommand()
cmd.Transaction = transaction
For Each r As DataRow In xtable.Rows ''''HERE IT WILL TAKE TOO MUCH TIME
cmd.CommandText = "insert into tablename values(" & r.Item("srno") & "," & r.Item("name") & ")"
cmd.ExecuteNonQuery()
Next
End Using
transaction.Commit()
Catch ex As Exception
transaction.Rollback()
End Try
End Using
End Using
Public Function getdata(ByVal Query As String) As DataTable
connectionString()
Try
Dim mds As New DataTable
Dim mycommand As New SQLiteCommand(DBConn)
mycommand.CommandText = Query
Dim reader As SQLiteDataReader = mycommand.ExecuteReader()
mds.Load(reader)
Return mds
Catch ex As Exception
MsgBox("DB Error", vbCritical, "")
MsgBox(Err.Description)
Return Nothing
End Try
End Function
Instead of concatenate an possible giant string, wrap all your inserts into a single transaction, like above. This will reduce the memory used and also make sqlite perform faster.
I have this code in Visual Basic, every time I have a new insert:
Private _conn As SqlConnection
Public Function Include(ByVal pSql As String, Optional timeout As Integer = 120) As Boolean
Try
Dim SQL_Str = "my string of conection... with database. not put on this example"
_conn = New SqlConnection(SQL_Str)
_conn.Open()
_adapter = New SqlDataAdapter
cmd.CommandTimeout = timeout
cmd.Connection = _conn
cmd.CommandText = pSql
cmd.CommandType = CommandType.Text
_adapter.InsertCommand = cmd
_adapter.InsertCommand.ExecuteNonQuery()
Catch ex As Exception
InputBox("New Error on Sql cmd: ", , pSql)
End Try
_conn.Close()
_conn.Dispose()
_conn = Nothing
_adapter.Dispose()
_adapter = Nothing
End Function
Ok this is a straightforward way to update the database.
But supose I have 1000 connections at the same time, do the application would support this kind of approach?
Do this method support simultaneous threads acessing the _conn object?
Is it really necessary to create a pool of connections to handle data?
Do a pool of connections will really improve something?
E.g. with this I'm overloading the application instead of the database?
If so, how would I do it on VbNet/Visual Basic?
Yes, pooled connections really are faster. They keep you from needing to continually re-negotiate login and protocol information. Even better, this is already built into the SqlConnection type, and it's done in a reasonably thread-safe way (where the existing code is not).
The short version is you really do want to create a brand new connection object for most queries, and you do not want to try to share the same connection variable throughout an application or session.
Given that, I see several big problems in that code:
Treating a class-level _conn variable as if it were local, making it impossible to share instances of this class safely across threads.
No mechanism to clean up the connection if an exception is thrown (needs a Finally or Using block. Just closing after the Catch isn't good enough.
No way to pass separate query parameters in the function signature. This will force you to write horribly insecure code elsewhere that's crazy-vulnerable to sql injection attacks. It's the kind of thing where you wake up one morning to find out you were hacked over a year ago, and IMO borders on professional malpractice.
Mixing UI code with utility code.
You want something more like this:
Private cnString As String = "my string of conection... with database. not put on this example"
Public Sub Include(pSql As String, parameters() As SqlParamter, Optional timeout As Integer = 120)
Using conn As New SqlConnectioN(cnString), _
cmd As New SqlCommand(pSql, conn)
If parameters IsNot Nothing AndAlso parameters.Length > 0 Then
cmd.Parameters.AddRange(parameters)
End If
conn.Open()
cmd.ExecuteNonQuery()
End Using
End Sub
And you might call it like this (assuming type or instance name is DB):
Dim pSql As String = "INSERT INTO [ExampleTable] (FirstName, LastName, CreationDate) VALUES (#FirstName, #LastName, #CreationDate)"
Dim parameters = {
New SqlParameter("#FirstName", SqlDbType.NVarChar, 20),
New SqlParameter("#LastName", SqlDbType.NVarChar, 20),
New SqlParameter("#CreationDate", SqlDbType.DateTime)
}
parameters(0).Value = "John"
parameters(1).Value = "Smith"
parameters(2).Value = DateTime.Now
Try
DB.Include(pSql, parameters)
Catch ex As Exception
MessageBox.Show(String.Format("New Error on Sql cmd:{0}{1}{0}{0}Message:{2}",vbCrLf, pSql, ex.Message)
End Try
In my application I create a function that allow the user to change the settings of the app. This settings are stored into a table 'cause there's a lot of records. Anyway, the problem's that if the settings isn't valorized yet, when the application start and load the settings from the table take of course a null GUID field and the message:
GUID format not recognized
appear. A code explaination:
Sub LoadSettings()
Using dbCon As MySqlConnection = establishConnection()
Try
dbCon.Open()
Dim MysqlCommand = New MySqlCommand("SELECT * FROM settings", dbCon)
Dim reader = MysqlCommand.ExecuteReader
For Each row In reader
Select Case row(2)
Case "company_name"
Setting.name.Text = row(3)
Case "company_email"
Setting.email.Text = row(3)
...
End Select
Next
End Sub
This function is called when the settings form is opened. If the settings aren't inserted yet, I get a message of bad format. I want to know how I can avoid this message.
You are not using the DataReader correctly. Consider this code:
Dim reader = MysqlCommand.ExecuteReader
For Each row In reader
... something
Next
MysqlCommand.ExecuteReader returns a DataReader object, but it is not - nor does it contain - a row collection you can iterate. If you hold the mouse over row you should see that it is a Data.Common.DataRecordInternal object which does have an Item property but a reference like row(2) will only compile with Option Strict Off.
Used correctly, when you Read a row the data in that internal object is available via the indexer (Item) and the various Getxxxxx() methods. This just prints the Id and Name from a table in a loop. I cant quite tell what you are trying to do with your results...it sort of looks like a Name/Value pair type thing maybe.
Dim SQL = "SELECT * FROM Demo"
Using dbcon = GetMySQLConnection(),
cmd As MySqlCommand = New MySqlCommand(SQL, dbcon)
dbcon.Open()
Using rdr As MySqlDataReader = cmd.ExecuteReader
If rdr.HasRows Then
Do While rdr.Read()
Console.WriteLine("{0} - {1}", rdr("Id").ToString, rdr("Name").ToString)
Loop
End If
End Using ' dispose of reader
End Using ' dispose of Connection AND command object
Alternatively, you could fill a DataTable and iterate the rows in that. Seems 6:5 and pick-em whether that would gain anything.
Note also that the Connection, Command and DataReader objects are properly disposed of when we are done using them.
I am working on a Mobile 6 Classic phone application for the first time and am having problems with the SqlCeResultSet. I am trying to fill a datagrid with this:
Private Sub LookUpRoutes()
Dim dir As String = Path.GetDirectoryName(Reflection.Assembly _
.GetExecutingAssembly().GetName().CodeBase)
Dim Sql As String = "SELECT RouteID, Name, Description FROM Routes " & _
"Where IsDeleted = 0"
Using con As SqlCeConnection = New SqlCeConnection( _
String.Format("Data Source = '{0}\database\RouteTracker.sdf'", dir))
con.Open()
Using cmd As SqlCeCommand = New SqlCeCommand(Sql, con)
cmd.CommandType = CommandType.Text
Dim resultSet As SqlCeResultSet = _
cmd.ExecuteResultSet(ResultSetOptions.Scrollable)
dgRoutes.DataSource = resultSet
End Using
End Using
End Sub
However I am only getting one filled in row back from that. The remaining rows show x's in the fields instead of the data (image below).
What am I doing wrong?
My best guess is that dgRoutes is actively using the resultSet which is actively using the open connection. Presumably, to make everything operate faster, it only queries results as they are scrolled into view (ResultSetOptions.Scrollable) - therefore the connection needs to remain open so that it can pull additional data on demand.
I am not sure why this works but I got rid of the Using con As SqlCeConnection = New SqlCeConnection and just used con.open and then resisted the urge to add a con.close to the code because doing so makes it not work again.
Can someone fill me in to why I can't close and dispose the connection without breaking the functionality? Is there another way?