So, here is a case where I have more than 40.000 rows in my database.
When I start Select * from Table it gets me result set in less than a second.
But, when I want to fill DataTable with those rows and than show it in DataGridView it takes forever to show (Around 15-20 seconds).
Why is that?
My code:
Public Shared Function FillDTwithSQL(ByVal StoredProc As String, ByVal table As DataTable) As DataTable
Dim cmd As SqlCommand = CreateCommand(SProcedura)
Dim dt As New DataTable("dt")
dt = table
dt.Rows.Clear()
Try
If adoConnection.State = ConnectionState.Closed Then adoConnection.Open()
dt.Load(cmd.ExecuteReader(CommandBehavior.Default))
Catch ex As Exception
MsgBox("Greska: " & ex.ToString)
Error = True
Error_text = ex.ToString
End Try
adoConnection.Close()
Return dt
End Function
Public Shared Function CreateCommand(ByVal SProcedura As String) As SqlCommand
CreateConnection(ConnectionSetup.ConnectionString)
Dim cmd As New SqlCommand(ConnectionSetup.DataBaseName & ".dbo." & SProcedura, adoConnection)
cmd.CommandType = CommandType.StoredProcedure
Return cmd
End Function
Private sub FillDGV()
DataBaseLayer.FillDTwithSQL("SelectProc", ds_Tables.Table)
Me.DataGridView1.DataSource = dds_Tables.Table
Me.DataGridView1.ClearSelection()
End Sub
For large amounts of data it is recommended to use the DataGridView's virtual mode.
Have a look at this link: How to: Implement Virtual Mode in the Windows Forms DataGridView Control
To summarize the link: You have to implement your own data caching by implementing the DataGridView's CellValueNeeded event and set Me.DataGridView1.VirtualMode = true.
Related
For all who assist with this, thank you. My goal is to get the duration of chat conversations for my employees and then calculate their year average. The data is being inserted into a database as short text in the format of (mm:ss or 18:30). I have tried converting and parsing the data several different ways and I have looked through numerous explanations and solutions on SO.com so far with nothing working the way I would like. The below is my display procedure to bring the data into the form. I know I could have done this in an easier way, but I am fairly new to VB.net and coding in general.
Private Sub DisplayJanMetrics()
'Open a connection to the database and then assign the values from the appropriate metric columns to the appropriate labels.
Dim str As String = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=|DataDirectory|\CoachingDB.accdb"
Dim conn As New OleDbConnection(str)
Try
conn.Open()
Dim sql As String = "SELECT Duration, CSAT, Away, Quality, Development FROM January WHERE Employee =" & "'" & cmbEmployee.SelectedItem.ToString & "'"
Dim cmd As New OleDbCommand(sql, conn)
Dim myReader As OleDbDataReader = cmd.ExecuteReader()
While myReader.Read
lblJanDuration.Text = myReader("Duration").ToString
lblJanCSAT.Text = myReader("CSAT").ToString
lblJanAway.Text = myReader("Away").ToString
lblJanQual.Text = myReader("Quality").ToString
lblJanDev.Text = myReader("Development").ToString
End While
Catch ex As OleDbException
MsgBox(ex.ToString)
Finally
conn.Close()
End Try
End Sub
Once the data has been loaded to the correct labels I have a button and click event to calculate the average from the labels - the other ones I was able to do easily because I could parse them to doubles and then do the calculation from there. Here is an image of what the form looks like, I think it will help all of you get an idea of what I am trying to accomplish.
This is what the form layout looks like
It is a good idea to separate your user interface code from your database code. You can use the same data retrieval function for any month by passing the month as TableName.
Database objects Connection, Command, and DataReader all need to be disposed and closed so they are placed in Using blocks. You don't want to hold the connection open while you update the user interface. Just return a DataTable and update the UI with that.
CalculateAverage first creates an array of the labels you want to include in the average. You can include all 12 but the average will not include empty labels. (IsNullOrEmpty)
Separate the string into minutes and seconds. Get the total seconds and add to the list.
Get the average number of seconds by calling Average on List(Of Integer).
Finally, turn the average seconds back into minutes and seconds and format a string to display.
Private Sub DisplayJanMetrics()
Dim dt As DataTable
Try
dt = GetEmployeeData(cmbEmployee.SelectedItem.ToString, "January")
Catch ex As Exception
MsgBox(ex.Message)
Exit Sub
End Try
If dt.Rows.Count > 0 Then
lblJanDuration.Text = dt(0)("Duration").ToString
lblJanCSAT.Text = dt(0)("CSAT").ToString
lblJanAway.Text = dt(0)("Away").ToString
lblJanQual.Text = dt(0)("Quality").ToString
lblJanDev.Text = dt(0)("Development").ToString
End If
End Sub
Private Function GetEmployeeData(Employee As String, TableName As String) As DataTable
Dim str As String = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=|DataDirectory|\CoachingDB.accdb"
Dim sql As String = $"SELECT Duration, CSAT, Away, Quality, Development FROM {TableName} WHERE Employee = #Employee;"
Dim dt As New DataTable
Using conn As New OleDbConnection(str),
cmd As New OleDbCommand(sql, conn)
cmd.Parameters.Add("#Employee", OleDbType.VarChar).Value = Employee
conn.Open()
Using reader = cmd.ExecuteReader
dt.Load(reader)
End Using
End Using
Return dt
End Function
Private Function CalculateAverage() As String
Dim lst As New List(Of Integer)
Dim labels() = {Label1, Label2, Label3, Label4} 'an array of 1 thru 12
For Each label In labels
If Not String.IsNullOrEmpty(label.Text) Then
Dim fields = label.Text.Split(":"c)
Dim mins = CInt(fields(0))
Dim secs = CInt(fields(1))
lst.Add(mins * 60 + secs)
End If
Next
Dim avg = lst.Average
Dim avgMins = Math.Truncate(avg / 60)
Dim remainderSec = avg - avgMins * 60
Return $"{avgMins}:{remainderSec}"
End Function
I have looked at many different code snippets on this site looking that would show me how to do something that should be fairly simple once I have the knowledge.
I want to query a database table for an array of values and then populate a combobox with those results.
Here is what I have so far:
Public Sub getMachines()
Try
Dim SQL As String = "SELECT MachineName from machine"
Form1.machineName.DisplayMember = "Text"
Dim tb As New DataTable
tb.Columns.Add("Text", GetType(String))
Using cn As New MySqlConnection(ConnectionString)
Using cmd As New MySqlCommand(SQL, cn)
For Each cmd As String In cmd
'I want to add each value found in the database to "tb.Rows.Add"
'tb.Rows.Add(???)
Next
Form1.machineName.DataSource = tb
cn.Open()
cmd.ExecuteNonQuery()
End Using
cn.Close()
End Using
Catch ex As MySqlException
MsgBox(ex.Message)
End Try
End Sub
I proceeded much like you did. I used the Load method of the DataTable. It is not necessary to set the column name and type. The name of the column is taken from the Select statement and the datatype is inferred by ADO.net from the first few records.
Luckily a DataTable can be an Enumerable using the .AsEnumnerable method. Then we can use Linq to get all the values from the MachineName column. Calling .ToArray causes the Linq to execute. If you hold your cursor over names on this line you will see that the datatype is String(). Just what we need to fill a combo box.
Code for a class called DataAccess
Private ConnectionString As String = "Your Connection String"
Public Function GetMachineNames() As String()
Dim tb As New DataTable
Dim SQL As String = "SELECT MachineName from machine;"
Using cn As New MySqlConnection(ConnectionString)
Using cmd As New MySqlCommand(SQL, cn)
cn.Open()
dt.Load(cmd.ExecuteReader)
End Using
End Using
Dim names = dt.AsEnumerable().Select(Function(x) x.Field(Of String)("MachineName")).ToArray()
Return names
End Function
In the form load you combo box like this.
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
Dim DatAcc As New DataAccess()
Dim arr = DatAcc.GetMachineNames()
machineName.DataSource = arr
End Sub
If you just want the MachineName to be displayed in the ComboBox, then just use that as the DisplayMember; don't bother creating another column called Text.
Public Sub getMachines()
Try
Dim cmd As String = "SELECT MachineName from machine"
Dim ds As New DataSet()
Using con As New MySqlConnection(ConnectionString)
Using da As New MySqlDataAdapter(cmd, con)
da.Fill(ds)
With Form1.machineName
.DisplayMember = "MachineName"
.ValueMember = "MachineName"
.DataSource = ds
End With
End Using
End Using
Catch ex As MySqlException
MsgBox(ex.Message)
End Try
End Sub
I'll show a few examples, including using parameters, since that is important.
First up, a quick translation to run the existing query and loop through the results:
Public Sub getMachines()
Try
Dim SQL As String = "SELECT MachineName from machine"
Using cn As New MySqlConnection(ConnectionString), _
cmd As New MySqlCommand(SQL, cn)
cn.Open()
Using rdr As MySqlDatareader = cmd.ExecuteReader
While rdr.Read()
Form1.machineName.Items.Add(rdr("MachineName"))
End While
End Using
End Using
Catch ex As MySqlException
MsgBox(ex.Message)
End Try
End Sub
But better practice for a method like this is to isolate data access for the UI. This method should return results to the caller, which can decide what do with them. So I'll show two methods: one to get the data, and the other to loop through it and set up the combobox:
Private Function GetMachines() As DataTable
'No try/catch needed here. Handle it in the UI level, instead
Dim SQL As String = "SELECT MachineName from machine"
Dim result As New DataTable
Using cn As New MySqlConnection(ConnectionString), _
cmd As New MySqlCommand(SQL, cn),
da As New MySqlDataAdapter(cmd)
da.Fill(result)
End Using
Return result
End Function
Public Sub LoadMachines()
Try
For Each item As DataRow in getMachines().Rows
Form1.machineName.Items.Add(item("MachineName"))
Next
Catch ex As MySqlException
MsgBox(ex.Message)
End Try
End Sub
Or, we can use DataBinding:
Private Function GetMachines() As DataTable
Dim SQL As String = "SELECT MachineName from machine"
Dim result As New DataTable
Using cn As New MySqlConnection(ConnectionString), _
cmd As New MySqlCommand(SQL, cn),
da As New MySqlDataAdapter(cmd)
da.Fill(result)
End Using
Return result
End Function
Public Sub LoadMachines()
Try
Form1.machineName.DisplayMember = "FirstName";
Form1.machineName.ValueMember = "City"
Form1.machineName.DataSource = GetMachines()
Catch ex As MySqlException
MsgBox(ex.Message)
End Try
End Sub
If you ever want to use a filter, you might do this (notice the overloading):
Private Function GetMachines(ByVal machineFilter As String) As DataTable
Dim SQL As String = "SELECT MachineName from machine WHERE MachineName LIKE #Filter"
Dim result As New DataTable
Using cn As New MySqlConnection(ConnectionString), _
cmd As New MySqlCommand(SQL, cn),
da As New MySqlDataAdapter(cmd)
'Match the MySqlDbType to your actual database column type and length
cmd.Parameters.Add("#Filter", MySqlDbType.VarString, 30).Value = machineFilter
da.Fill(result)
End Using
Return result
End Function
Private Function GetMachines(ByVal machineFilter As String) As DataTable
Return GetMachines("%")
End Function
Query parameters like that are very important, and if you were doing string concatenation to accomplish this kind of thing on your old platform, you were doing very bad things there, too.
Finally, let's get fancy. A lot of the time, you really don't want to load an entire result set into RAM, as is done with a DataTable. That can be bad. Instead, you'd like be able to stream results into memory and only work with one at a time, minimizing RAM use. In these cases, you get to play with a DataReader... but returning a DataReader object from within a Using block (which is important) doesn't work that well. To get around this, we can use functional programming concepts and advanced language features:
Private Iterator Function GetMachines(ByVal machineFilter As String) As IEnumerable(Of String)
Dim SQL As String = "SELECT MachineName from machine WHERE MachineName LIKE #Filter"
Using cn As New MySqlConnection(ConnectionString), _
cmd As New MySqlCommand(SQL, cn)
'Match the MySqlDbType to your actual database column type and length
cmd.Parameters.Add("#Filter", MySqlDbType.VarString, 30).Value = machineFilter
cn.Open()
Using rdr As MySqlDatareader = cmd.ExecuteReader
While rdr.Read()
Dim result As String = rdr("MachineName")
Yield Return result
End While
End Using
End Using
Return result
End Function
Private Function GetMachines() As IEnumerable(Of String)
Return GetMachines("%")
End Function
Aiming to load a table from a db3 (sqlite) file into a datatable. Then load this datatable into a datagridview to allow editing of it. Then I want to save the edited datatable back to the original db3 file - overwriting any changes.
I've got as far as getting the table from the db3 + into the DGV. I wont include any code around the datatable to dgv. Just needing how to write this datable back to the db3 file. How do I do this? Code:
Imports System.Data.SQLite
Public Class DBOps
Public Function ImportGEDb3(Filepath As String) As DataTable
Dim dt As New DataTable("Data")
Dim cnn As New SQLiteConnection("Data Source='" & Filepath & "'")
cnn.Open()
Dim mycommand As New SQLiteCommand(cnn)
mycommand.CommandText = "Select * from Data"
Dim reader As SQLiteDataReader = mycommand.ExecuteReader()
dt.Load(reader)
reader.Close()
cnn.Close()
Return dt
End Function
Public Sub SaveGEDb3(dt as datatable, filepath as string)
' save passed datatable to File above
end sub
end class
The SQLiteDataAdapter class is the simplest choice for this kind of updates. The method Update of the class scans the DataTable passed and for each row that has the RowState property different from DataRowState.Unchanged executes the appropriate INSERT, DELETE, UPDATE command (if the SELECT command extracts from the DataTable the primary key). So assuming that your SELECT * FROM Data returns also the primary key of the table you could change your code to take advantage of the SQLiteDataAdapter functionality
Imports System.Data.SQLite
Public Class DBOps
' Global because you create it in the ImportGEDb3 and use it in the SaveGEDB3 '
Private daImport As SQLiteDataAdapter
Public Function ImportGEDb3(Filepath As String) As DataTable
Dim dt As New DataTable("Data")
daImport = new SQLiteDataAdapter("Select * from Data", "Data Source='" & Filepath & "'")
daImport.Fill(dt)
' This is critical, it is the SQLiteCommandBuilder that takes '
' the SQLiteDataAdapter SELECT statement and builds the required'
' INSERT,UPDATE,DELETE commands.'
Dim builder = new SQLiteCommandBuilder(daImport)
Return dt
End Function
Public Sub SaveGEDb3(dt as datatable)
if daImport IsNot Nothing Then
daImport.Update(dt)
End If
End Sub
End class
Insert new row inside DataGridView
This answer makes it seem like the database should update with the rows.add
Some other sites have instructions, but form a perspective of creating the database from scratch. I already have a database and just want the stupid thing to accept new data.
Here's what I have done:
Private Sub InitializeDataGridView()
Try
' Set up the DataGridView.
With Me.DataGridView1
' Automatically generate the DataGridView columns.
.AutoGenerateColumns = True
' Set up the data source.
'bindingSource1.DataSource = GetData("SELECT * FROM Places and Stuff")
MyTable = GetData("SELECT * FROM Places and Stuff")
'MyDataSet = bindingSource1.DataSource
'MyTable = MyDataSet.Tables(0)
.DataSource = MyTable
' Automatically resize the visible rows.
.AutoSizeRowsMode = DataGridViewAutoSizeRowsMode.DisplayedCellsExceptHeaders
' Set the DataGridView control's border.
.BorderStyle = BorderStyle.Fixed3D
' Put the cells in edit mode when user enters them.
.EditMode = DataGridViewEditMode.EditOnEnter
' Disables Add New Row
.AllowUserToAddRows = False
End With
Catch ex As SqlException
MessageBox.Show(ex.ToString, _
"ERROR", MessageBoxButtons.OK, MessageBoxIcon.Exclamation)
System.Threading.Thread.CurrentThread.Abort()
End Try
End Sub
I am binding the DGV to a table. It seems like maybe I need a dataset somewhere to update but I cannot figure out how to populate a dataset with a table that is also a sql database. You can also see where I have played around with other datasets/datatables etc.
I also got my datagridview to add a row but the database is being lazy:
Private Sub btnAdd_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnAdd.Click
'Determine Last Row Index
Dim dgvrowCnt As Integer = DataGridView1.Rows.Count - 1
If DataGridView1.Rows.Count > 0 Then
Dim myRow As DataRow = MyTable.NewRow
myRow(0) = DataGridView1.Rows(dgvrowCnt).Cells(0).Value + 1
MyTable.Rows.Add(myRow)
Else
Dim myRow As DataRow = MyTable.NewRow
myRow(0) = 230
MyTable.Rows.Add(myRow)
End If
End Sub
I am a little saddened by not being able to use myRow("<column name here>") = 230 but I'll have to get over it I guess.
I have tried refreshing and checking the table to see if my form needs to be refreshed, but that doesn't seem to be the case.
This page http://support.microsoft.com/kb/301248 has 2 lines and claims it does what I am hoping for:
Dim objCommandBuilder As New SwlCammandBuilder(daAuthors)
daAuthors.Update(dsPubs, "Authors")
I cannot get my table into a dataset as shown in the binding lines of my example.
It seems that you haven't understood a fundamental concept of ADO.NET. The DataTable and other objects like the DataSet are 'disconnected' objects, meaning that adding/updating and removing rows doesn't update/insert/delete the database table.
You need to create an SqlCommand, prepare its command text and then Execute a query to update your db (other methods include using an SqlDataAdapter and its Update method)
For example, to insert a single row in a datatable your code should be something like this
Using con = New SqlConnection(.....constringhere...)
Using cmd = new SqlCommand("INSERT INTO table1 (field1) values (#valueForField)", con)
con.Open()
cmd.Parameters.AddWithValue("#valueForField", newValue)
cmd.ExecuteNonQuery()
End Using
End Using
A more complete tutorial could be found here
Instead this could be a pseudocode to use a SqlDataAdapter and a SqlCommandBuilder to automate the construction of the commands required to store your changes back to the database
' Keep the dataset and the adapter at the global class level'
Dim da As SqlDataAdapter = Nothing
Dim ds As DataSet
Private Function GetData(ByVal sqlCommand As String) As DataSet
ds As New DataSet()
Dim connectionString As String = "Data Source=...."
Using con = New SqlConnection(connectionString)
conn.Open()
Using command = New SqlCommand(sqlCommand, con)
Using da = New SqlDataAdapter(command)
da.Fill(ds)
End Using
End Using
End Using
Return ds
End Function
Use the first table inside the DataSet returned by GetData as Datasource of the grid (or just use the whole dataset)
.DataSource = GetData(.......).Tables(0)
' Add a new button to submit changes back to the database'
Private Sub btnSave_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnAdd.Click
Dim builder = New SqlCommandBuilder(da)
da.UpdateCommand = builder.GetUpdateCommand()
da.InsertCommand = builder.GetInsertCommand()
da.DeleteCommand = builder.GetDeleteCommand()
da.Update(ds)
End Sub
Please, note that I cannot test this code, and I offer it as a pseudocode without any error checking required by a more robust application.
I was asked to do a system that would poll one table in the database every second and if it counters a row that meets a criteria start actions to handle that.
I've done this but every now and then I get a time out exception. I have a WPF application where I have a thread that runs in background. This thread has a loop and sleeps for one second at the end of the loop. The connection to the database is opened inside "using" clause.
Below is my thread sub:
Private Sub PollDatabase()
While m_StopThread = False
Try
Dim listOfRows As List(Of DataObject) = db.GetDataObjects()
... Do something with the rows ...
Catch ex As Exception
m_log.WriteLine(ex.ToString())
End Try
Thread.Sleep(1000)
End While
End Sub
And my SQL function looks like this:
Public Function GetDataObjects() As List(Of DataObject)
Dim result As New List(Of DataObject)
Dim sb As New StringBuilder("... the sql query ...")
Using cnn = New SqlConnection(_connectionString)
cnn.Open()
Using cmd = New SqlCommand(sb.ToString(), cnn)
cmd.CommandTimeout = 0
Using DataReader As SqlDataReader = cmd.ExecuteReader()
Do While DataReader.Read()
... read the columns from table
to the dataobject ...
result.Add(DataObject)
Loop
End Using
End Using
End Using
Return result
End Function
Now what seems randomly my log has a time out exception:
System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
...
at System.Data.SqlClient.SqlConnection.Open()
My questions are: is this at all save way of doing this? Or am I doing something fundamentally wrong here? And of course if anyone have a suggestion to fix this issue.
EDIT:
I tried a bit different approach with the SQL function. I'm now opening a connection once when my application starts and dumped the "using" clauses. So my function looks something like this now:
Public Function GetDataObjects() As List(Of DataObject)
Dim result As New List(Of DataObject)
Dim sb As New StringBuilder("... the sql query ...")
_sqlCmd.CommandText = sb.ToString()
Using DataReader As SqlDataReader = _sqlCmd.ExecuteReader()
Do While DataReader.Read()
... fill the list with objects ...
Loop
End Using
Return result
End Function
My log is clean form errors. So is there something wrong opening a connection to the server once in a second as I do with the using?
EDIT:
I've done a lot of testing now to identify the problem. What I discovered is that just connecting multiple times to the server doesn't cause any problems. Neither does adding a select statement after the connection. But when I actually implement a function where is the complete reader part and return my results I run into the time out problems. Here is two examples.
This isn't causing issues:
Private Sub Window_Loaded(sender As System.Object, e As System.Windows.RoutedEventArgs)
Me.DataContext = Me
m_Thread = New Thread(AddressOf ConnectionTestFunction)
m_Thread.IsBackground = True
m_Thread.Start()
End Sub
Private Sub ConnectionTestFunction()
While m_stopThread = False
Try
m_log.WriteLine("GetData (" & m_ThreadCounter & ")")
Using cnn As SqlConnection = New SqlConnection("Data Source=server;Initial Catalog=db;Integrated Security=True;MultipleActiveResultSets=True")
cnn.Open()
Using cmd As SqlCommand = New SqlCommand("SELECT * FROM Data", cnn)
Using DataReader As SqlDataReader = cmd.ExecuteReader()
Do While DataReader.Read()
Loop
End Using
End Using
End Using
Catch ex As Exception
m_log.WriteLine(ex.ToString())
End Try
m_ThreadCounter += 1
Thread.Sleep(1000)
End While
End Sub
This is causing timeout errors:
Private Sub Window_Loaded(sender As System.Object, e As System.Windows.RoutedEventArgs)
Me.DataContext = Me
m_Thread = New Thread(AddressOf ConnectionTestFunction)
m_Thread.IsBackground = True
m_Thread.Start()
End Sub
Private Sub ConnectionTestFunction()
While m_stopThread = False
Try
m_log.WriteLine("GetData (" & m_ThreadCounter & ")")
Dim datarows As List(Of Data) = Me.GetData()
Catch ex As Exception
m_log.WriteLine(ex.ToString())
End Try
m_ThreadCounter += 1
Thread.Sleep(1000)
End While
End Sub
Private Function GetData() As List(Of Data)
Dim result As New List(Of Data)
Using cnn As SqlConnection = New SqlConnection("Data Source=server;Initial Catalog=db;Integrated Security=True;MultipleActiveResultSets=True")
cnn.Open()
Using cmd As SqlCommand = New SqlCommand("SELECT * FROM Data", cnn)
Using DataReader As SqlDataReader = cmd.ExecuteReader()
Do While DataReader.Read()
Dim d As New Data()
d.DataId = DataReader("DataId")
... etc fields about 10 of them ...
result.Add(d)
Loop
End Using
End Using
End Using
Return result
End Function
I'm really happy if anyone have any thoughts about this... I have to admit I'm really confused now.
Probably, your code is taking longer to complete than the default time-out value for the connection is. Try specifying the time-out when you create your Sql Connection. Make sure that it's longer than the time your code needs to complete.
This approach doesn't seem very good.. Instead of pooling, why not react when new data comes? You can use a trigger or SqlDependency?
http://dotnet.dzone.com/articles/c-sqldependency-monitoring