Getting hung up in a BackgroundWorker - vb.net

Question

I've got an app that checks every few seconds for new orders.
If it sees a new entry in the database, it fires off a BackgroundWorker to process it.
I have 3 list boxes on my form, pending, completed and failed.
Theres is a second BackgroundWorker process which asynchronously checks with an api to see if the order has been fully processed yet, before moving the item in question to the 'completed' list box.
Everything seems to be working fine until I throw a few orders at a time at the app.
It processes the orders fine and adds them to pending, and it will check back on a few of them but eventually get stuck in the secondbackground worker process.
It will eventually crash out with a "This BackgroundWorker is currently busy" error.
I've gone over my code lots of times and I can't see where it would be crashing out.
Whats the best way for me to diagnose this?
Heres the code for the background worker where I'm getting stuck...
Private Sub BuildProgressCheck_DoWork(sender As Object, e As ComponentModel.DoWorkEventArgs) _
Handles BuildProgressCheck.DoWork
lblstatus.Text = "Checking back on async jobs"
Dim i As Integer
For i = 0 To buildpendinglist.Items.Count - 1
Beep()
Dim cn As New MySqlConnection
cn.ConnectionString = "server=" & dbserver & "; userid=" & dbusername & "; password=" & dbpassword & "; database=" & dbdatabase & ";Convert Zero Datetime=True"
Dim jobcheck As New MySqlDataAdapter("Select * FROM dbcpman_jobs WHERE dbxid='" & buildpendinglist.Items(i) & "'", cn)
Dim jobcheck_table As New DataTable
jobcheck.Fill(jobcheck_table)
Dim jobrow As DataRow
jobrow = jobcheck_table.Select("failed = 'false'").FirstOrDefault()
If Not jobrow Is Nothing Then
Dim job_id As String = jobrow.Item("id")
Dim job_jobid As String = jobrow.Item("jobid")
Dim job_status As String = jobrow.Item("status")
Dim job_dbxid As String = jobrow.Item("dbxid")
Dim jobcommand As String = "command=queryAsyncJobResult&jobId=" & job_jobid
Dim fulljobapicheckurl = cpapiurl & jobcommand
Try
Dim jobapicall As New System.Net.WebClient
jobcheckresult = jobapicall.DownloadString(fulljobapicheckurl)
Catch ex As Exception
''Problem submitting the api request?
End Try
If jobcheckresult.Contains("<jobstatus>1</jobstatus>") Then ''If true, job has completed
Dim doc As New System.Xml.XmlDocument
doc.LoadXml(jobcheckresult) ''api_result contains xml returned from a http request.
If doc.GetElementsByTagName("virtualmachine") IsNot Nothing Then
Dim elem As XmlNodeList = doc.GetElementsByTagName("virtualmachine").Item(0).ChildNodes
For Each item As XmlNode In elem
If item.Name.Equals("state") Then
new_vm_state += ((item.InnerText.ToString()) + Environment.NewLine)
ElseIf item.Name.Equals("hostname") Then
new_vm_hostname += ((item.InnerText.ToString()) + Environment.NewLine)
ElseIf item.Name.Equals("templatename") Then
new_vm_templatename += ((item.InnerText.ToString()) + Environment.NewLine)
ElseIf item.Name.Equals("cpunumber") Then
new_vm_cpunumber += ((item.InnerText.ToString()) + Environment.NewLine)
ElseIf item.Name.Equals("cpuspeed") Then
new_vm_cpuspeed += ((item.InnerText.ToString()) + Environment.NewLine)
ElseIf item.Name.Equals("memory") Then
new_vm_memory += ((item.InnerText.ToString()) + Environment.NewLine)
ElseIf item.Name.Equals("nic") Then
new_vm_netmask += ((item.ChildNodes.Item(3).InnerText.ToString()) + Environment.NewLine)
new_vm_gateway += ((item.ChildNodes.Item(4).InnerText.ToString()) + Environment.NewLine)
new_vm_ipaddress += ((item.ChildNodes.Item(5).InnerText.ToString()) + Environment.NewLine)
new_vm_macaddress += ((item.ChildNodes.Item(11).InnerText.ToString()) + Environment.NewLine)
ElseIf item.Name.Equals("instancename") Then
new_vm_instancename1 += ((item.InnerText.ToString()) + Environment.NewLine)
End If
Next
End If
new_vm_macaddress = new_vm_macaddress.ToUpper
Dim privateip As String = new_vm_ipaddress.Replace(" ", "").ToString
Dim publicip As String = privateip.Replace("172.16.11.", "198.73.112.")
Try
Dim myCommand As New MySqlCommand
Dim myAdapter As New MySqlDataAdapter
Dim SQL As String
myCommand.Connection = cn
cn.Open()
myAdapter.SelectCommand = myCommand
SQL = "DELETE FROM dbcpman_jobs WHERE jobid = '" & job_jobid & "'"
myCommand.CommandText = SQL
myCommand.ExecuteNonQuery()
SQL = "UPDATE dbcpman_vm SET deployresponse = '" & jobcheckresult & "' WHERE dbx_id = '" & job_dbxid & "'"
myCommand.CommandText = SQL
myCommand.ExecuteNonQuery()
SQL = "UPDATE dbcpman_vm SET macaddress = '" & new_vm_macaddress & "' WHERE dbx_id = '" & job_dbxid & "'"
myCommand.CommandText = SQL
myCommand.ExecuteNonQuery()
SQL = "UPDATE dbcpman_vm SET publicip = '" & publicip & "' WHERE dbx_id = '" & job_dbxid & "'"
myCommand.CommandText = SQL
myCommand.ExecuteNonQuery()
SQL = "UPDATE dbcpman_vm SET privateip = '" & privateip & "' WHERE dbx_id = '" & job_dbxid & "'"
myCommand.CommandText = SQL
myCommand.ExecuteNonQuery()
cn.Close()
Dim new_vm_username As String = "clouduser"
Dim new_vm_password As String = GeneratePassword(7)
System.Threading.Thread.Sleep(1000)
Dim new_vm_support_username As String = "dbxsupport"
Dim new_vm_support_password As String = GenerateSupportPassword(7)
cn.Open()
myAdapter.SelectCommand = myCommand
SQL = "INSERT into dbcpman_credentials(username1, username2, password1, password2, type, link) VALUES ('" & new_vm_username & "','" & new_vm_support_username & "','" & new_vm_password & "','" & new_vm_support_password & "','Server root logon','" & job_dbxid & "')"
myCommand.CommandText = SQL
myCommand.ExecuteNonQuery()
SQL = "INSERT into dbcpman_vm_boot(dbxid, ip, macaddress, hostname) VALUES ('" & job_dbxid & "','" & new_vm_ipaddress & "','" & new_vm_macaddress & "','" & job_dbxid & "')"
myCommand.CommandText = SQL
myCommand.ExecuteNonQuery()
cn.Close()
Try ''''add monitoring for this new host
Dim monitorurl As String = "http://192.168.16.32/addhost.php?hostname=" & job_dbxid & "&ipaddr=" & publicip
Dim webClient As New System.Net.WebClient
Dim monresult As String = webClient.DownloadString(monitorurl)
If monresult = "SUCCESS" Then
''success message
Else
''fail message
End If
Catch ex As Exception
End Try
buildcompletedlist.Items.Add(buildpendinglist.Items(i))
buildpendinglist.Items.Remove(buildpendinglist.Items(i))
buildprogresscheckactive = 0
BuildProgressCheck.CancelAsync()
Catch ex As Exception
End Try
ElseIf jobcheckresult.Contains("<jobstatus>0</jobstatus>") Then ''If true, job is still pending
''job still pending - do nothing, will check again on next pass
ElseIf jobcheckresult.Contains("<jobstatus>2</jobstatus>") Then ''If true, job has failed
Try
Dim myCommand As New MySqlCommand
Dim myAdapter As New MySqlDataAdapter
Dim SQL As String
myCommand.Connection = cn
cn.Open()
myAdapter.SelectCommand = myCommand
SQL = "UPDATE dbcpman_jobs SET failed = 'true' WHERE jobid = '" & job_jobid & "'"
myCommand.CommandText = SQL
myCommand.ExecuteNonQuery()
cn.Close()
buildfailedlist.Items.Add(buildpendinglist.Items(i))
buildpendinglist.Items.Remove(buildpendinglist.Items(i))
Catch ex As Exception
BuildProgressCheck.CancelAsync()
buildprogresscheckactive = 0
End Try
End If
End If
Next
buildprogresscheckactive = 0
End Sub

Answer 1

My best guess based on the appended text and code is as follows:
"..checks every few seconds for new orders."
Error: "This BackgroundWorker is currently busy"
So I believe you're starting the asynchronous work in a Tick event of a Timer. So, let's say your timer ticks every 2 seconds.
Friend WithEvents mytimer As New System.Windows.Forms.Timer With {.Interval = 2000, .Enabled = True}
Private Sub _Tick(sender As Object, e As EventArgs) Handles mytimer.tick
Me.BuildProgressCheck.RunWorkerAsync()
End Sub
Now, lets imagine that for 6 ticks BuildProgressCheck_DoWork are executed in less than 2 seconds. No error occurs and everything seems okay. But, at the 7th tick it take 3 seconds. Then, when the timer raise tick for the 8th time, the BackGroundWorker is "currently busy" and throws an error.
What you need to do is to suspend the timer before running the worker and resume it when the worker has completed. Here's an example:
Private Sub _Tick(sender As Object, e As EventArgs) Handles mytimer.Tick
'Suspend the timer:
Me.mytimer.Enabled = False
'Do background work:
Me.BuildProgressCheck.RunWorkerAsync()
End Sub
Private Sub _DoWork(sender As Object, e As DoWorkEventArgs) Handles BuildProgressCheck.DoWork
'Important: Do not invoke any UI controls inside this method.
' (We're inside the background thread)
End Sub
Private Sub _ProgressChanged(sender As Object, e As ProgressChangedEventArgs) Handles BuildProgressCheck.ProgressChanged
'Here you can safely invoke UI controls.
'(We're inside UI thread)
'Me.Label1.Text = "Started..."
End Sub
Private Sub _RunWorkerCompleted(sender As Object, e As RunWorkerCompletedEventArgs) Handles BuildProgressCheck.RunWorkerCompleted
Try
If (e.Error Is Nothing) Then
If (e.Cancelled) Then
'Cancelled
Else
'Success
End If
Else
'Error
End If
Catch ex As Exception
'...
Finally
'Restart the timer.
Me.mytimer.Enabled = True
End Try
End Sub