Slow database query: Using VB.net connecting to Access - vb.net

I have a program which will only allow a certain amount of concurrent users to use the program at one time. To do this I have a single table in an access database which holds each user that is using the program. Now although this does work the query seems to be running very slowly, I am certain it is something to do with the database functions as it was running fine before I implemented them.
Here are my functions:
Public Function openDB() As Boolean
cnn.Open()
Return True
End Function
Public Function closeDB() As Boolean
cnn.Close()
Return True
End Function
Then there is the function for checking the database. This is where I think it may be tripping up because I have 2 queries running here:
Public Function CheckLicence() As Boolean
Dim result As Boolean = HandleRegistry()
If result = True Then
Dim _table As String = "Users"
Dim query As String = "SELECT * FROM " & _table & " WHERE Machine_ID='" & CpuId() & "'"
Dim sizeQuery As String = "SELECT COUNT(*) FROM " & _table
Dim NoUsers As Integer = 0
Dim ds As New DataSet
Dim dr As OleDbDataReader
Dim cmd As New OleDbCommand(query, cnn)
dr = cmd.ExecuteReader
Dim sizeCdm As New OleDbCommand(sizeQuery, cnn)
NoUsers = sizeCdm.ExecuteScalar()
If dr.Read() Then
Return True
Else
If NoUsers < My.Settings.NoUsers Then
addToDB()
Else
MsgBox("Too many users are currently using this program. Clear a user and try again.")
Return False
End If
End If
Else
MsgBox("Your licence has expired, contact support to purchase a new licence.")
Return False
End If
Return True
End Function
And to add and remove I have to get the cpu id, I found the code for that on here somewhere it does work but maybe that could be the slow part, I dont actually know if this is the correct way of getting it.
Public Sub addToDB()
Dim _table As String = "Users"
Dim query As String = "INSERT INTO " & _table & " ([User], [Machine_ID]) VALUES (?,?)"
Dim ds As New DataSet
Dim cmd As New OleDbCommand(query, cnn)
cmd.Parameters.AddWithValue("?", Environment.UserName)
cmd.Parameters.AddWithValue("?", CpuId())
cmd.ExecuteNonQuery()
End Sub
Public Sub RemoveFromDB()
Dim _table As String = "Users"
Dim query As String = "DELETE * FROM " & _table & " WHERE Machine_ID='" & CpuId() & "'"
Dim ds As New DataSet
Dim cmd As New OleDbCommand(query, cnn)
cmd.ExecuteNonQuery()
End Sub
Private Function CpuId() As String
Dim computer As String = "."
Dim wmi As Object = GetObject("winmgmts:" &
"{impersonationLevel=impersonate}!\\" &
computer & "\root\cimv2")
Dim processors As Object = wmi.ExecQuery("Select * from " &
"Win32_Processor")
Dim cpu_ids As String = ""
For Each cpu As Object In processors
cpu_ids = cpu_ids & ", " & cpu.ProcessorId
Next cpu
If cpu_ids.Length > 0 Then cpu_ids =
cpu_ids.Substring(2)
Return cpu_ids
End Function

How about storing the cpuID globally and fetching only once. Also, use the Using construct on anything that needs to be disposed. Lastly, run a few StopWatches around your methods to see which one is slow or just debug through each one to find the slow method.

Answer:
How about storing the cpuID globally and fetching only once. Also, use the Using >construct on anything that needs to be disposed. Lastly, run a few StopWatches >around your methods to see which one is slow or just debug through each one to >find the slow method. – Andrew Mortimer
Moved the call for getting the cpuID to the program initialisation so that it runs on startup. This made it run much faster than it did before.

Related

Performance issue with SQLite database with VB.NET

I am Inserting the data-table into SQLite Database. I am doing like this.
First I Fetch the data with getdata function and insert it into datatable, then with For Each Loop i made the Insert Command and Execute It. I am having 50000 Records it will take 30 Minutes to run.
Please Guide the suitable approach. Here is the Code.
Dim xtable As DataTable = getdata("select * from tablename")
Dim str As String = Nothing
For Each r As DataRow In xtable.Rows ''''HERE IT WILL TAKE TOO MUCH TIME
str = str & ("insert into tablename values(" & r.Item("srno") & "," & r.Item("name"));")
Next
EXECUTEcmd(str)
Public Function getdata(ByVal Query As String) As DataTable
connectionString()
Try
Dim mds As New DataTable
Dim mycommand As New SQLiteCommand(DBConn)
mycommand.CommandText = Query
Dim reader As SQLiteDataReader = mycommand.ExecuteReader()
mds.Load(reader)
Return mds
Catch ex As Exception
MsgBox("DB Error", vbCritical, "")
MsgBox(Err.Description)
Return Nothing
End Try
End Function
Public Sub EXECUTEcmd(ByVal selectcmd As String)
Using cn = New SQLiteConnection(conectionString)
cn.Open()
Using transaction = cn.BeginTransaction()
Using cmd = cn.CreateCommand()
cmd.CommandText = selectcmd
cmd.ExecuteNonQuery()
End Using
transaction.Commit()
End Using
cn.Close()
End Using
End Sub
here the Conncection String is:
conStr = "Data Source=" & dbpath & ";Version=3;Compress=True; UTF8Encoding=True; PRAGMA journal_mode=WAL; cache=shared;"
Use a stringbuilder to build your string, not string concatenation
Dim strB As StringBuilder = New StringBuilder(100 * 50000)
For Each r As DataRow In xtable.Rows
strB.AppendLine($"insert into tablename values({r.Item("srno")},{r.Item("name")});")
Next
Strings cannot be changed in .net. Every time you make a new string VB has to copy everything out of the old string into a new one and add the new bit you want. If each of your insert statements is 100 bytes, that means it copies 100 bytes, then adds 100, then copies 200 bytes and adds 100, then copies 300 bytes, then 400 bytes, then 500 bytes. By the time it has done 10 strings it has made 5.5 kilobytes of copying. By the time it's done 50 thousand strings it has copied 125 gigabytes of data. No wonder it's slow!
Always use a StringBuilder to build massive strings
--
I'm willing to overlook the sql injection hacking nag for this one, because of the nature of the task, but please read http://bobby-tables.com - you should never, ever concatenate values into an SQL as a way of making an sql that has some varying effect.
This entire exercise would be better done as this (pseudocode) kind of thing:
Dim sel as New SQLiteCommand("SELECT a, b FROM table", conn)
Dim ins as New SQLiteCommand("INSERT INTO table VALUES(:a, :b)", conn)
ins.Parameters.Add("a" ...)
ins.Parameters.Add("b" ...)
Dim r = sel.ExecuteReader()
While r.Read()
ins.Parameters("a") = r.GetString(0)
ins.Parameters("b") = r.GetString(1)
ins.ExecuteNonQuery()
End While
That is to say, you minimize your memory by reading rows one at a time out of ther edaer and inserting them one at a time in the insert; the insert command is prepared once, you just change the parameter values, execute it, change them again, execute it ... It's what parameterized queries were designed for (as well as stopping your app getting hacked when someone puts SQL in your variable, or even just stopping it crashing when you have an person named O'Grady
Maybe you must refactor your code like this:
Dim xtable As DataTable = getdata("select * from tablename")
Using cn = New SQLiteConnection(conectionString)
cn.Open()
Using transaction = cn.BeginTransaction()
Try
Using cmd = cn.CreateCommand()
cmd.Transaction = transaction
For Each r As DataRow In xtable.Rows ''''HERE IT WILL TAKE TOO MUCH TIME
cmd.CommandText = "insert into tablename values(" & r.Item("srno") & "," & r.Item("name") & ")"
cmd.ExecuteNonQuery()
Next
End Using
transaction.Commit()
Catch ex As Exception
transaction.Rollback()
End Try
End Using
End Using
Public Function getdata(ByVal Query As String) As DataTable
connectionString()
Try
Dim mds As New DataTable
Dim mycommand As New SQLiteCommand(DBConn)
mycommand.CommandText = Query
Dim reader As SQLiteDataReader = mycommand.ExecuteReader()
mds.Load(reader)
Return mds
Catch ex As Exception
MsgBox("DB Error", vbCritical, "")
MsgBox(Err.Description)
Return Nothing
End Try
End Function
Instead of concatenate an possible giant string, wrap all your inserts into a single transaction, like above. This will reduce the memory used and also make sqlite perform faster.

Can't INSERT INTO access database

I can select the data from an Access database, but I tried many ways to INSERT INTO database. There is no error message, but it didn't insert the data.
Code:
Dim conn As New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & CurDir() & "\fsDB1.accdb")
Dim cmd As OleDbCommand
Dim dr As OleDbDataReader
conn.Open()
Dim CommandString As String = "INSERT INTO tblfile(stdUname,filePw,filePath,status) VALUES('" & userName & "','" & filePw & "','" & filePath & "','A')"
Dim command As New OleDbCommand(CommandString, conn)
Command.Connection = conn
Command.ExecuteNonQuery()
I just want a simple easy way to INSERT INTO an Access database. Is it possible because of the problem of Access database? I can insert this query by running query directly in Access.
Firstly I would check the database settings. If your app copies a new copy of the database each time you run it that would explain why you can select existing data and why your new data is not being saved (Well it is being saved, but the database keeps getting replaced with the old one). Rather set it up to COPY IF NEWER.
Further, you should ALWAYS use parameterized queries to protect your data. It is also is less error prone than string concatenated commands ans is much easier to debug.
Also, I recommend using a USING block to handle database connections so that your code automatically disposes of resources no longer needed, just in case you forget to dispose of your connection when you are done. Here is an example:
Using con As New OleDbConnection
con.ConnectionString = "Provider = Microsoft.ACE.OLEDB.12.0; " & _
"Data Source = "
Dim sql_insert As String = "INSERT INTO Tbl (Code) " & _
"VALUES " & _
"(#code);"
Dim sql_insert_entry As New OleDbCommand
con.Open()
With sql_insert_entry
.Parameters.AddWithValue("#code", txtCode.Text)
.CommandText = sql_insert
.Connection = con
.ExecuteNonQuery()
End With
con.Close()
End Using
Here is an example where data operations are in a separate class from form code.
Calling from a form
Dim ops As New Operations1
Dim newIdentifier As Integer = 0
If ops.AddNewRow("O'brien and company", "Jim O'brien", newIdentifier) Then
MessageBox.Show($"New Id for Jim {newIdentifier}")
End If
Back-end class where the new primary key is set for the last argument to AddNewRow which can be used if AddNewRow returns true.
Public Class Operations1
Private Builder As New OleDbConnectionStringBuilder With
{
.Provider = "Microsoft.ACE.OLEDB.12.0",
.DataSource = IO.Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Database1.accdb")
}
Public Function AddNewRow(
ByVal CompanyName As String,
ByVal ContactName As String,
ByRef Identfier As Integer) As Boolean
Dim Success As Boolean = True
Dim Affected As Integer = 0
Try
Using cn As New OleDbConnection With {.ConnectionString = Builder.ConnectionString}
Using cmd As New OleDbCommand With {.Connection = cn}
cmd.CommandText = "INSERT INTO Customer (CompanyName,ContactName) VALUES (#CompanyName, #ContactName)"
cmd.Parameters.AddWithValue("#CompanyName", CompanyName)
cmd.Parameters.AddWithValue("#ContactName", ContactName)
cn.Open()
Affected = cmd.ExecuteNonQuery()
If Affected = 1 Then
cmd.CommandText = "Select ##Identity"
Identfier = CInt(cmd.ExecuteScalar)
Success = True
End If
End Using
End Using
Catch ex As Exception
Success = False
End Try
Return Success
End Function
End Class

Visual basic - Incrementing the score

Private Sub Button4_Click(sender As Object, e As EventArgs) Handles Button4.Click
Dim READER As MySqlDataReader
Dim Query As String
Dim connection As MySqlConnection
Dim COMMAND As MySqlCommand
Dim item As Object
Try
item = InputBox("What is the item?", "InputBox Test", "Type the item here.")
If item = "shoe" Then
Dim connStr As String = ""
Dim connection As New MySqlConnection(connStr)
connection.Open()
Query = "select * from table where username= '" & Login.txtusername.Text & " '"
COMMAND = New MySqlCommand(Query, connection)
READER = COMMAND.ExecuteReader
If (READER.Read() = True) Then
Query = "UPDATE table set noOfItems = noOfItems+1, week1 = 'found' where username= '" & Login.txtusername.Text & "'"
Dim noOfItems As Integer
Dim username As String
noOfItems = READER("noOfItems") + 1
username = READER("username")
MessageBox.Show(username & "- The number of items you now have is: " & noOfGeocaches)
End If
Else
MsgBox("Unlucky, Incorrect item. Please see hints. Your score still remains the same")
End If
Catch ex As Exception
MessageBox.Show("Error")
End Try
I finally got the message box to display! but now my code does not increment in the database, can anybody help me please :D
Thanks in advance
After fixing your typos (space after the login textbox and name of the field retrieved) you are still missing to execute the sql text that updates the database.
Your code could be simplified understanding that an UPDATE query has no effect if the WHERE condition doesn't find anything to update. Moreover keeping an MySqlDataReader open while you try to execute a MySqlCommand will trigger an error in MySql NET connector. (Not possible to use a connection in use by a datareader). We could try to execute both statements in a single call to ExecuteReader separating each command with a semicolon and, of course, using a parameter and not a string concatenation
' Prepare the string for both commands to execute
Query = "UPDATE table set noOfItems = noOfItems+1, " & _
"week1 = 'found' where username= #name; " & _
"SELECT noOfItems FROM table WHERE username = #name"
' You already know the username, don't you?
Dim username = Login.txtusername.Text
' Create the connection and the command inside a using block to
' facilitate closing and disposing of these objects.. exceptions included
Using connection = New MySqlConnection(connStr)
Using COMMAND = New MySqlCommand(Query, connection)
connection.Open()
' Set the parameter value required by both commands.
COMMAND.Parameters.Add("#name", MySqlDbType.VarChar).Value = username
' Again create the reader in a using block
Using READER = COMMAND.ExecuteReader
If READER.Read() Then
Dim noOfItems As Integer
noOfItems = READER("noOfItems")
MessageBox.Show(username & "- The number of items you now have is: " & noOfItems )
End If
End Using
End Using
End Using

Performance , VB.NET SQL Dependency's Notifications

Ok i have 2 queries that are running SQL Dependecy's to simulate the push - notifications
really my ownly question is why in heck my 2 pieces of code run so much differently ( slower / faster ) then the other
here is the fast code (milisecond updates)
lbnoes.Text = ""
'You must stop the dependency before starting a new one.
'You must start the dependency when creating a new one.
SqlDependency.Stop(getSQLString())
SqlDependency.Start(getSQLString())
Using cn As SqlConnection = New SqlConnection(getSQLString())
Using cmd As SqlCommand = cn.CreateCommand()
cmd.CommandType = CommandType.Text
cmd.CommandText = "SELECT test1, test2 FROM dbo.[ztest]"
cmd.Notification = Nothing
' creates a new dependency for the SqlCommand
Dim dep As SqlDependency = New SqlDependency(cmd)
' creates an event handler for the notification of data changes in the database
AddHandler dep.OnChange, AddressOf dep_onchange1
cn.Open()
Using dr As SqlDataReader = cmd.ExecuteReader()
While dr.Read()
lbnoes.Text = lbnoes.Text & vbCrLf & (dr.GetString(0) & " " & dr.GetString(1))
'PopupNotifier1.ContentText = dr.GetString(0) & " " & dr.GetString(1)
'PopupNotifier1.Popup()
End While
End Using
End Using
End Using
HERE is the slow code (almost 2 minutes /update or so it seems) it is almost acting like its calling the on changes over and over. - hopefully you can tell me why cause i rather use this piece of code
lbnoes.Text = ""
Try
Dim con As New SqlConnection
Dim myConString As String = getSQLString()
Dim objcommand As SqlCommand = New SqlCommand
'con.ConnectionString = myConString
With objcommand
.Connection = con
Dim cmdText As String = "SELECT test1, test2 FROM ztest"
.CommandText = cmdText
End With
con.ConnectionString = myConString
SqlDependency.Stop(getSQLString())
SqlDependency.Start(getSQLString())
Dim dep1 As SqlDependency = New SqlDependency(objcommand)
AddHandler dep1.OnChange, AddressOf dep_onchange1
con.Open()
Using readerObj As SqlClient.SqlDataReader = objcommand.ExecuteReader
'This will loop through all returned records
While readerObj.Read
Dim t1 As String = readerObj("test1").ToString
Dim t2 As String = readerObj("test2").ToString
lbnoes.Text = lbnoes.Text & vbCrLf & (t1 & " " & t2)
' 'PopupNotifier1.ContentText = dr.GetString(0) & " " & dr.GetString(1)
' 'PopupNotifier1.Popup()
End While
End Using
con.Close()
Catch ex As Exception
End Try
Both on Changes are being invoked the same way ( different invoke methods / removehandles ) ...
Please help this confused nub.
Thanks in advance
Well, it's not easy to tell without going further in the application code, but it's much more efficient to access the datareader members trough index than trough column names.
Also, maybe the type conversion has something to do..

Inserting variables into a query string - it won't work!

Basically i have a query string that when i hardcode in the catalogue value its fine. when I try adding it via a variable it just doesn't pick it up.
This works:
Dim WaspConnection As New SqlConnection("Data Source=JURA;Initial Catalog=WaspTrackAsset_NROI;User id=" & ConfigurationManager.AppSettings("WASPDBUserName") & ";Password='" & ConfigurationManager.AppSettings("WASPDBPassword").ToString & "';")
This doesn't:
Public Sub GetWASPAcr()
connection.Open()
Dim dt As New DataTable()
Dim username As String = HttpContext.Current.User.Identity.Name
Dim sqlCmd As New SqlCommand("SELECT WASPDatabase FROM dbo.aspnet_Users WHERE UserName = '" & username & "'", connection)
Dim sqlDa As New SqlDataAdapter(sqlCmd)
sqlDa.Fill(dt)
If dt.Rows.Count > 0 Then
For i As Integer = 0 To dt.Rows.Count - 1
If dt.Rows(i)("WASPDatabase") Is DBNull.Value Then
WASP = ""
Else
WASP = "WaspTrackAsset_" + dt.Rows(i)("WASPDatabase")
End If
Next
End If
connection.Close()
End Sub
Dim WaspConnection As New SqlConnection("Data Source=JURA;Initial Catalog=" & WASP & ";User id=" & ConfigurationManager.AppSettings("WASPDBUserName") & ";Password='" & ConfigurationManager.AppSettings("WASPDBPassword").ToString & "';")
When I debug the catalog is empty in the query string but the WASP variable holds the value "WaspTrackAsset_NROI"
Any idea's why?
Cheers,
jonesy
alt text http://www.freeimagehosting.net/uploads/ba8edc26a1.png
I can see a few problems.
You are using concatenation in a SQL statement. This is a bad practice. Use a parameterized query instead.
You are surrounding the password with single quotes. They are not needed and in fact, I'm surprised it even works assuming the password itself does not have single quotes.
You should surround classes that implement IDisposable with a Using block
You should recreate the WASP connection object in GetWASPcr like so:
Public Sub GetWASPAcr()
Dim username As String = HttpContext.Current.User.Identity.Name
Dim listOfDatabaseConnectionString As String = "..."
Using listOfDatabaseConnection As SqlConnection( listOfDatabaseConnectionString )
Using cmd As New SqlCommand("SELECT WASPDatabase FROM dbo.aspnet_Users WHERE UserName = #Username")
cmd.Parameters.AddWithValue( "#Username", username )
Dim dt As New DataTable()
Using da As New SqlDataAdapter( cmd )
da.Fill( dt )
If dt.Rows.Count = 0 Then
WaspConnection = Null
Else
Dim connString As String = String.Format("Data Source=JURA;Initial Catalog={0};User Id={1};Password={2};" _
, dt.Rows(0)("WASPDatabase") _
, ConfigurationManager.AppSettings("WASPDBUserName") _
, ConfigurationManager.AppSettings("WASPDBPassword"))
WaspConnection = New SqlConnection(connString);
End If
End Using
End Using
End Using
End Sub
In this example, listOfDatabaseConnectionString is the initial connection string to the central database where it can find the catalog name that should be used for subsequent connections.
All that said, why would you need a class level variable to hold a connection? You should make all your database calls open a connection, do a sql statement, close the connection. So, five database calls would open and close a connection five times. This sounds expensive except that .NET gives you connection pooling so when you finish with a connection and another is requested to be opened, it will pull it from the pool.
Your string passed into the constructor for this SqlConnection object will be evaluated when the class is instantiated. Your WASP variable (I'm assuming) won't be set until the method you have shown is called.
Might want to quit looking one you have found your database:
For i As Integer = 0 To dt.Rows.Count - 1
If dt.Rows(i)("WASPDatabase") Is DBNull.Value Then
WASP = ""
Else
WASP = "WaspTrackAsset_" + dt.Rows(i)("WASPDatabase")
break
End If
Next
[link text][1]You are building your string on the fly by adding the value of a column to a string. So, for the row in question for the column "WASPDatabase" was tacked on to your string. So you got what it had. On another note, your earlier query of "select ... from ... where ..." where you are manually concatinating the string of a variable makes you WIDE OPEN to SQL-Injection attacks.
Although this link [1]: how to update a table using oledb parameters? "Sample query using parameterization" is to a C# sample of querying with parameterized values, the similar principles apply to most all SQL databases.
At the time you're creating the new connection, WASP is holding the value you want it to be holding? It is a string data type? Try adding .ToString after WASP and see if that helps anything.
Interesting problem. =-)
The problem is, as Paddy already points out, that the WaspConnection object gets initialized before you even have the chance to call GetWASPAcr. Try this:
Public Sub GetWASPAcr()
'[...]
End Sub
Dim _waspConnection As SqlConnection
Public Readonly Property WaspConnection As SqlConnection
Get
If _waspConnection Is Nothing Then
GetWASPAcr()
_waspConnection = New SqlConnection("Data Source=JURA;Initial Catalog=" & WASP & ";User id=" & ConfigurationManager.AppSettings("WASPDBUserName") & ";Password='" & ConfigurationManager.AppSettings("WASPDBPassword").ToString & "';")
End If
Return _waspConnection
End Get
End Property