I have some very basic code that takes a list of strings and a new thread is created per string.
The thread then writes a file named with the string so should be unique but I get access violations.
I try the debugger option in VS 2010 and it doesn't give me the file access problems when I step through but it writes two lines per loop so I am assuming it still has a problem with running the thread twice
Code to call the thread:
For Each x In x_list
Dim trd as new Thread(DirectCast(Sub() tp(x), ThreadStart))
Next
Code for thread:
Private Sub tp(ByVal x_in As String)
Dim res_file As New StreamWriter("C:\result_" + x_in + ".txt", True)
For i = 1 to 5
res_file.WriteLine(DateTime.Now.ToString)
Next
res_file.Close()
End Sub
You're introducing an unnecessary Lambda here:
Sub() tp(x)
And then you're falling foul of the same well known issue from C# - that what you're capturing is the variable that keeps changing until it's set to the last value in the loop.1
Try instead just:
For Each x In x_list
Dim trd as new Thread(DirectCast(tp, ParameterizedThreadStart))
trd.Start(x)
'TODO - Does trd get stored anywhere?
Next
1 See e.g. Closing over the loop variable considered harmful which discusses how there's a change for C#5, but I don't think there's a corresponding change for VB.Net.
Related
I have function with loop
Sub KvmActionForEachVm(CN As MySqlConnection, SshClient As Renci.SshNet.SshClient, Action As Action(Of MySqlConnection, Renci.SshNet.SshClient, String, Integer))
Dim AdmVMList As List(Of AdmVM) = ReadAdmVMList(CN)
For Each One As AdmVM In AdmVMList
Try
Action.Invoke(CN, SshClient, One.Name, CInt(One.Id))
'for example ParseVmConfig(CN, SshClient, One.Name, One.Id)
Catch ex As Exception
Console.WriteLine(One.Name & ": " & ex.Message)
Continue For
End Try
Next
SshClient.Disconnect()
End Sub
and various function with the same signature what can be working in loop like below. Of course, this function require correct VmName(One.Name) and VmId(One.Id)
Sub ParseVmConfig(CN As MySqlConnection, SshClient As Renci.SshNet.SshClient, VmName As String, VmId As Integer)
....
End sub
Without loop I'm usually pass delegates as parameters by the same way
KvmActionForEachVm(CN, SshClient, Sub() ParseVmConfig(CN, SshClient, "", 0))
But in this case I'm confused.
Upd. I'm using NET Core 5.0 and this is screen of my application with this issue.
You didn't indicate which version of .NET / VB you're using. In old versions of .NET, there is an interesting behavior of capture in a loop. It arises from the intersection of the scope of the loop control variable (which is the entire loop, not just an individual iteration) and the fact that variables from the enclosing scope of the lambda are captured by reference and not by value. This can lead to surprising behavior.
If this is the issue you're having (every invocation of the new Action runs as if called by the last loop iteration, or even one past the last loop iteration), then you can fix it by declaring new variables within the loop scope to capture. These variables are scoped by the loop iteration, vs. the loop variable which can have a weird scope of all loop iterations.
e.g.
For Each One As AdmVM In AdmVMList
Dim vmName = One.Name
Dim vmId = One.Id
Call Foo(Sub() Action.Invoke(CN, SshClient, vmName, vmId)
Next
This surprising behavior was addressed in a fairly old version of C# (by making each iteration of the loop get a new copy of the loop control variable), and I'm not sure what its history may be in VB. I would imagine that it was cleaned up ca. 2010 as VB was kept fairly close to C# in that time period, but maybe you're using a very old version of VB, or maybe my imagination is wrong.
Also see:
Related C# question: Lambda variable capture in loop - what happens here?
Eric Lippert on the issue in C# (1): https://ericlippert.com/2009/11/12/closing-over-the-loop-variable-considered-harmful-part-one/
Eric Lippert on the issue in C# (2):
https://ericlippert.com/2009/11/16/closing-over-the-loop-variable-considered-harmful-part-two/
I have a method and I want to run multiple threads to complete it faster.
Public Sub createMaster()
For intLoop = 0 To _MasterCount - 1 Step _Step
_MasterCollection(intLoop) = New Champion(_MasterIDCollection(intLoop), _RRM.returnChampionInformation(_MasterIDCollection(intLoop).Substring(0, _MasterIDCollection(intLoop).Length() - 1), "na"))
Next
End Sub
So, the method basically creates a collection for me. I've tried to do multiple threads to complete it faster, making a champion actually takes a second. Is there a way to make intLoop a specific variable? I tried making intloop = _Start where I increased _Start by one each time and it ended up making _Start all kinds of numbers. My main is like:
Dim thread1 As New System.Threading.Thread(AddressOf createMaster)
thread1.Start()
Dim thread2 As New System.Threading.Thread(AddressOf createMaster)
thread2.Start()
thread1.Join()
thread2.Join()
I tried using a For loop with the thread and it didn't seem to work either. Does anyone know how to make this work in any way?
You could convert your current code to use LINQ to map your _MasterIDCollection to Champion instances:
_MasterCollection = (
From id In _MasterIDCollection
Select New Champion(id, _RRM.returnChampionInformation(id.Substring(0, id.Length() - 1), "na"))
).ToList() ' or ToArray()?
LINQ is easily parallelizable by adding AsParallel, but you also need AsOrdered to maintain the order:
_MasterCollection = (
From id In _MasterIDCollection.AsParallel().AsOrdered()
Select New Champion(id, _RRM.returnChampionInformation(id.Substring(0, id.Length() - 1), "na"))
).ToList() ' or ToArray()?
By default, PLINQ will (I believe) run one thread per CPU core, but you can control that by adding WithDegreeOfParallelism. Whether this is worthwhile depends on the type of work being done (e.g. I/O-bound or CPU-bound):
_MasterCollection = (
From id In _MasterIDCollection.AsParallel().AsOrdered().WithDegreeOfParallelism(20)
Select New Champion(id, _RRM.returnChampionInformation(id.Substring(0, id.Length() - 1), "na"))
).ToList() ' or ToArray()?
This requires .NET 4+, a reference to System.Core and Using System.Linq. See the PLINQ docs for more info.
at the moment I have a small application and need to take information from an object and display it into an excel file, using the Microsoft.office.interop class I've been able to write to the file, and it shows one by one the records being added, however about once every 3 times I try it, the spreadsheet stops filling somewhere between the 300th and 600th record, I have 6,000 in total and it's not breaking every time, I put a check after it finishes to see whether the last record is filled in but the code never reaches that point and I'm unsure of what's happening
I also don't know how to debug the problem as it'd mean going through 6,000 loops to check for it stopping... which might not even happen?
a little section of the code is here
loadExcel(incidents, WorkSheetName)
If WorkSheetName.Cells(DBObject.HighestInci + 1, 6) Is Nothing Then
MessageBox.Show("Failed to fill spreadsheet, Retrying now.")
loadExcel(incidents, WorkSheetName)
End If
above is the code calling and checking the method below
Private Sub loadExcel(ByVal incidents As List(Of Incident), ByRef WorkSheetName As Excel.Worksheet)
Dim i = 2
For Each inc As Incident In incidents
WorkSheetName.Cells(i, 1) = inc.DateLogged
WorkSheetName.Cells(i, 2) = inc.DateClosed
WorkSheetName.Cells(i, 3) = Convert.ToString(inc.DateLogged).Substring(3, 2)
i += 1
Next
End Sub
Thanks in advance
EDIT
I'm thinking loading it to a buffer of some sort then writing once they have all been updated would be the way to go instead of it currently loading and writing each separately? however I have no idea where to start for that?
I've fixed my problem, with what I had above Excel was opened and it started printing into the spreadsheet line by line, the problem is that any interactions with excel would cause the process to freeze
By adding an
ExcelApp.visible = false
before carrying out the process and an
ExcelApp.visible = true
afterwards, it all works and then opens the file afterwards
I've read a lot of different questions on SO about multithreaded applications and how to split work up between them, but none really seem to fit what I need for this. Here's how my program currently basically works:
Module Module1
'string X declared out here
Sub Main()
'Start given number of threads of Main2()
End Sub
Sub Main2()
'Loops forever
'Call X = nextvalue(X), display info as needed
End Sub
Function nextvalue(byval Y as string)
'Determines the next Y in the sequence
End Function
End Module
This is only a rough outline of what actually happens in my code by the way.
My problem being that if multiple threads start running Main2(), they're dealing with the same X value as in the other threads. The loop inside of main2 executes multiple times per millisecond, so I can't just stagger the loops. There is often duplication of work done.
How can I properly divide up the work so that the two threads running simultaneously never have the same work to run?
You should synchronize the generation and storage of X so that the composite operation appears atomic to all threads.
Module Module1
Private X As String
Private LockObj As Object = New Object()
Private Sub Main2()
Do While True
' This will be used to store a snapshot of X that can be used safely by the current thread.
Dim copy As String
' Generate and store the next value atomically.
SyncLock LockObj
X = nextValue(X)
copy = X
End SyncLock
' Now you can perform operations against the local copy.
' Do not access X outside of the lock above.
Console.WriteLine(copy)
Loop
End Sub
End Module
A thread manager is required to manage the threads and the work that they do. Say it is desirable to split up the work into 10 threads.
Start the manager
Manager creates 10 threads
Assign work to the manager (queue up the work, let's say it queues up 10000 work items)
Manager assigns a work item to complete for each of the 10 threads.
As threads finish thier work, they report back to the manager that they are done and recieve another work item. The queue of work should be thread safe so that items can be enqueued and dequeued. The manager handles the management of work items. The threads just execute the work.
Once this is in place, work items should never be duplicated amongst threads.
Use a lock so that only one thread can access X at a time. Once one thread is done with it, another thread is able to use it. This will prevent two threads from calling nextvalue(x) with the same value.
I have been creating multiple background threads to parse xml files and recreate new xml files. Now the problem I am having is that even though I use synclock on global variables, I will still at times get errors and I am sure that this is just the crude way of coding I am doing, but I was wondering if someone had a better option.
program flow =
access local folder and upload all files into list
strip each file into xml entries and put these entries into an arraylist
parse for specific values and enter these values into a database table
now create a thread and take the arraylist of entries and the thread will reparse
thread parses and creates a new xml file
main thread continues with another function and then goes and get a file from list
I will add some code to show problem areas but if I have declared global variable in use does the different threads overwrite that value in the variable causing contamination.
For Each g In resultsList
gXmlList.Add(g)
Next
Dim bgw As New BackgroundWorker
bgw.WorkerSupportsCancellation = True
AddHandler bgw.DoWork, New DoWorkEventHandler(AddressOf createXML)
AddHandler bgw.RunWorkerCompleted, AddressOf WorkComplete
threadlist.Add(bgw)
bgw.RunWorkerAsync()
Private Sub createXML()
num += 1
Dim file As String = Module1.infile
xmlfile = directoryPath & "\New" & dateTime.Now.ToUniversalTime.ToString("yyyyMMddhhmmss") & endExtension
Thread.Sleep(2000)
Dim doc As XmlDocument = New XmlDocument
**xwriter = New XmlTextWriter(xmlfile, Encoding.UTF8)** this is where ioexception error
xwriter.Formatting = Formatting.Indented
xwriter.Indentation = 2
xwriter.WriteStartDocument(True)
xwriter.WriteStartElement("Posts")
I have global variables through out the app and should I be locking each one and does this not make using threads then useless.
Dim j As Integer = 0
I believe your biggest problem is not knowing what features in .Net are thread safe. A list for example is not (a dictionary is). While you may get away with it you will eventually run into problems with locking, etc.
Your using classes and variables that are not thread safe. Any time you are working with threads you have to be Extremely careful with locking. To answer your question, yes, you have to lock and unlock everything you are working with unless the type / method specifically handles it for you.
There are a lot of multi threading (PLINQ for example) in .Net 4.0 which handle a lot of the "grunt work" for you. While you should learn and understand how to do thread safe code yourself it will give you a head start.
Try passing the data into the createXML() method. That may help isolate the code from other data being accessed. I would suggest reading up on threading and learning how to do it without a background worker.
Global variables are generally a bad idea. Given your VB code I'm guessing this is a carry over from the VB6 world for you. That's not in any way intended to be insulting, just trying to help advance your skills forward. Variable scope should be as confined as possible.
Another thought looking at your code is to learn how to use String.Format() when building strings / paths.
Simple manual thread in VB to get you started:
Dim bThread As New Threading.Thread(AddressOf createXML)
bThread.IsBackground = True
bThread.Start()
Well if you are having issues with thread locking then you can simply wrap your action in the following manor.
'This will need to be out of scope so that all threads have access to it
Dim readerWriterLock As New Threading.ReaderWriterLockSlim
readerWriterLock.EnterWriteLock()
xwriter = New XmlTextWriter(xmlfile, Encoding.UTF8)
'other logic
readerWriterLock.ExitWriteLock()
'anything reading from this would need to have the following
readerWriterLock.EnterReadLock()
'logic
readerWriterLock.ExitReadLock()
Try this and then if not successful post the exception message and any other information that you can.