I am working on a project where I have to load assemblies (lets call them tasks) at runtime, run the task and then be able to kill off the whole assembly so that the dll can be replaced without interrupting the main application.
There are many of these tasks running within the main application, some run sequentially and some run in parallel. Occasionally one or to of these tasks need to be updated and then re-added to the queue. Currently we are stopping the entire application and interrupting the other tasks at whatever stage they are at, which is not ideal.
I have figured out that I will have to load each assembly into a separate AppDomain, but loading the assemblies in those domains is proving difficult, especially when I need to actually run the tasks and receive events from them. I have been looking into this problem for a couple of days and have still not managed to get a working proof-of-concept.
I have an Interface for the tasks which includes a 'run' and 'kill' method (subs) and a 'taskstep', 'complete' and 'killed' event. 'taskstep' returns an object to be cached later, 'complete' fires when the whole task is done and 'killed' fires when it is ready to be unloaded. There should also be a timeout on the whole process of two hours and a timeout of 2 minutes on the killed event in case it gets stuck, at which point I would like to be able to unload it, forcing any threads to terminate (which is what 'kill' should do anyway). Each assembly may contain several tasks, all of which should loadable be unloadable.
I have no problems loading these tasks as 'plugins' but am lost when trying to both use them and unload them. If I have to create some elaborate wrapper then so be it but is what I need even possible?
I have tried inheriting from MarshalByRefObject but I do not even know the assembly fullname unless I load it first, which then locks the file. I have tried loading from a byte array of the assembly. This means that the file is not locked but a copy of it remains in the current appdomain. This will become problematic over the following months / years!
For Each key As String In assemblies.Keys
Dim ad As AppDomain = AppDomainHelper.BuildChildAppDomain(AppDomain.CurrentDomain, key)
AddHandler ad.AssemblyResolve, AddressOf AssemblyResolve
_l.Add(ad)
For Each value As String In assemblies(key)
Dim item As IScanner = CType(ad.CreateInstanceAndUnwrap(key, value), IScanner)
ListBox1.Items.Add(item)
Next
Next
Private Function AssemblyResolve(sender As Object, args As ResolveEventArgs) As Assembly
Return GetType(IScanner).Assembly
End Function
Consider using Managed Extensibility Framework.
It's part of .Net 4.0, and you can read a short overview here.
Loading and unloading assemblies can be a very bad idea - things doesn't work as your might expect (ever try to catch an exception thrown in another AppDomain? BAM you can't!)
It appears that exception is thrown from the new (secondary, if you will) AppDomain. Which means that the ResolveEventArgs have to be serialize to cross appdomain boundary. You should handle the event from within the secondary appdomain so you dont have to cross that boundary.
so... create a class that handles the resolutions.. something like...
Public Class AssemblyResolver
Inherits MarshalByRefObject
Public Property SearchPath As String = String.Empty
Public Function ResolveAssembly(sender as Object, e As ResolveEventArgs) As Assembly
Dim tPath As String = Path.Combine(SearchPath, e.Name & ".dll")
If File.Exists(tPath) Then
Return Assembly.LoadFrom(tPath)
End If
Return Nothing
End Function
End Class
Now use it from your plugin loader...
Dim tPluginDomain As AppDomain = AppDomainHelper.BuildChildAppDomain(AppDomain.CurrentDomain, key)
Dim tResolver As AssemblyResolver = tPluginDomain.CreateInstanceAndUnwrap(GetType(AssemblyResolver).Assembly.FullName, GetType(AssemblyResolver).FullName)
AddHandler tPluginDomain.AssemblyResolve, AddressOf tResolver.ResolveAssembly
That should get you pointed down the right road.
All code was off the top of my head with reference to MSDN. DOnt know if it will compile.. expect to typo/debug it.
EDIT
Whoops.. forgot to tag the resolver into the plugin domain. Fixed.
Related
I have a bug in my application which is the same as here which this person was running into the same problem. My application is multi threaded where the worker thread is updating the Waveformgraph on the UI. I believe that is where my problem is and why, periodically, and on occassion I get a big red X in at least one of my waveformgraph objects when running the application. From reading and research, I need to use an Invoke or BeginInvoke method? Can someone please explain better and provide a sample code that is relevant to my code? The samples that I've found so far still have me hazy on how I need to do this or what I need to do. Thank you for your help.
This code is on the swScopeOnOff click event, main thread.
thread2 = New System.Threading.Thread(AddressOf dataAcquiring)
thread2.Start()
This code is in dataAcquiring Sub
Public Sub dataAcquiring()
'While Scope switch is on, stream each Ai channel's data continuously to its respective WaveForm graph
Do While swScopeOnOff.Value = True
data = reader.ReadWaveform(readRate)
i = 0
For Each WaveformGraph In WFGS
WaveformGraph.PlotWaveformAppend(data(i)) 'This line is updating the UI's waveform graphs
i += 1
Next
i = 0
Loop
End Sub
Proper, thread-safe invocation is actually not as hard as one might think (not even for thread-safe events, but that's irrelevant for this question).
I would recommend you to use the normal Invoke method, such as Me.Invoke() (where Me is the current form, if not, use Form1 or whatever it's called instead). Using BeginInvoke() may be asynchronous but it stacks memory usage and can cause memory leaks if EndInvoke() is not called correctly.
If you target .NET 4.0 or higher you can simply do like this:
Me.Invoke(Sub() WaveformGraph.PlotWaveformAppend(data(i)))
However if you target .NET 3.5 or lower it requires a few more lines of code.
'Outside your Sub.
Delegate Sub WaveformAppendDelegate(ByRef WaveformGraph, ByRef data)
'Create a new sub.
Public Sub AppendData(ByRef WaveformGraph, ByRef data)
WaveformGraph.PlotWaveformAppend(data)
End Sub
'Inside your sub, when you're going to invoke.
Me.Invoke(New WaveformAppendDelegate(AddressOf AppendData), WaveformGraph, data(i))
I am using Point Grey's FlyCapture API to drive some cameras.
In a public class, I implemented all the starting and initializing code ; in the following _cam refers to a ManagedGigECamera.
Because I have 16 cameras, I want the code to be as fast as possible, so I wanted to use tasks.
Here is the code I use:
_cam.StartCapture(AddressOf OnImageGrabbed)
.../...
Public Sub OnImageGrabbed(ByVal raw_image As ManagedImage)
Dim t As Task = Task.Run(Sub()
'save image to disk or whatever
End Sub)
t.Wait()
End Sub
The above gives -sort of- satisfaction. By viewing image timestamps, I can see that some images are saved seconds after they are grabbed, and even some images are skipped altogether...
I wanted to make sure each call to OnImageGrabbed would start a new task, and tried the following, but it crashes right away with 'object not set to an instance of an object' (can't really debug, the code is running on a remote machine)
_cam.StartCapture(AddressOf OnImageGrabbed)
.../...
Public Async Sub OnImageGrabbed(ByVal raw_image As ManagedImage)
Await Task.Run(Sub()
'save image to disk or whatever
End Sub)
End Sub
All in all, my questions are:
how can I run an event handler asynchronously ?
why, using the first code, do I get (what appears to be) random delays between each call
to OnImageGrabbed ? I mean the differences in time between image timestamps is never the same, and tend to increase on the long run (first few images are almost synchronized, but after 1 minute or so, each image is separated by more and more time). Memory leak ? GC ?
Thanks in advance for any hint !
EDIT:
In the end I changed the way the system works: I fire a software trigger on each camera using a timer, and each trigger is fired 'in parallel':
Parallel.ForEach(Of ListOfCameras)(SingleCamera,
Sub(aCamera, loopstate, num)
aCamera.FireTrigger()
End Sub)
Starting a task and then immediately blocking on it (via Wait) nets you nothing. You may as well just run the saving-image code directly.
The second example is actually asynchronous. You're probably getting an exception because the ManagedImage argument or one of its child objects is being disposed. Remember that the code raising the event has no idea that your code is asynchronous; it's up to you to copy out what you need from the event arguments if you're going to use it asynchronously.
I have an in-application service which allows me to feed it messages from various sources, which will be put into a simple list. The service, running in its own thread, will, periodically, process all messages in the list into various files; one file for each source, which are then managed for size.
My question is about the proper way to check for messages and performing a lock around the code which accesses the list. There are only two places which access the list; one is where a message is added to the list and the other is where the messages are dumped from the list into a processing list.
Adding a message to the list:
Public Sub WriteMessage(ByVal messageProvider As IEventLogMessageProvider, ByVal logLevel As EventLogLevel, ByVal message As String)
SyncLock _SyncLockObject
_LogMessages.Add(New EventLogMessage(messageProvider, logLevel, Now, message))
End SyncLock
End Sub
Processing the list:
Dim localList As New List(Of EventLogMessage)
SyncLock _SyncLockObject
If (_LogMessages.Count > 0) Then
localList.AddRange(_LogMessages)
_LogMessages.Clear()
End If
End SyncLock
' process list into files...
My questions are: should I do a double check when I am processing the list, see below? And why? Or why not? And are there any dangers in accessing the list’s count property outside of the lock? Are either of the methods better or more efficient? And why? Or why not?
Dim localList As New List(Of EventLogMessage)
If (_LogMessages.Count > 0) Then
SyncLock _SyncLockObject
If (_LogMessages.Count > 0) Then
localList.AddRange(_LogMessages)
_LogMessages.Clear()
End If
End SyncLock
End If
' process list into files...
I understand that in this particular case, it may not matter if I do a double check given the fact that, outside of the processing function, the list can only grow. But this is my working example and I’m trying to learn about the finer details of threading.
Thank you in advance for any insights…
After some further research, thank you 'the coon', and some experimental programming, I have some further thoughts.
Concerning the ReaderWriterLockSlim, I have the following example which seems to work fine. It allows me to read the number of messages in the list without interfering with other code which may be trying to read the number of messages in the list, or the messages themselves. And when I desire to process the list, I can upgrade my lock to write mode, dump the messages into a processing list and process them outside of any read/write locks, thus not blocking any other threads which may want to add, or read, more messages.
Please note, that this example uses a simpler construct for the message, a String, as opposed to the previous example which used a Type along with some other metadata.
Private _ReadWriteLock As New Threading.ReaderWriterLockSlim()
Private Sub Process()
' create local processing list
Dim processList As New List(Of String)
Try
' enter read lock mode
_ReadWriteLock.EnterUpgradeableReadLock()
' if there are any messages in the 'global' list
' then dump them into the local processing list
If (_Messages.Count > 0) Then
Try
' upgrade to a write lock to prevent others from writing to
' the 'global' list while this reads and clears the 'global' list
_ReadWriteLock.EnterWriteLock()
processList.AddRange(_Messages)
_Messages.Clear()
Finally
' alway release the write lock
_ReadWriteLock.ExitWriteLock()
End Try
End If
Finally
' always release the read lock
_ReadWriteLock.ExitUpgradeableReadLock()
End Try
' if any messages were dumped into the local processing list, process them
If (processList.Count > 0) Then
ProcessMessages(processList)
End If
End Sub
Private Sub AddMessage(ByVal message As String)
Try
' enter write lock mode
_ReadWriteLock.EnterWriteLock()
_Messages.Add(message)
Finally
' always release the write lock
_ReadWriteLock.ExitWriteLock()
End Try
End Sub
The only problem I see with this technique is that the developer must be diligent about acquiring and releasing the locks. Otherwise, deadlocks will occur.
As to whether this is more efficient than using a SyncLock, I really could not say. For this particular example and its usage, I believe either would suffice. I would not do the double check for the very reasons ‘the coon’ gave about reading the count while someone else is changing it. Given this example, the SyncLock would provide the same functionality. However, in a slightly more complex system, one where multiple sources might read and write to the list, the ReaderWriterLockSlim would be ideal.
Concerning the BlockingCollection list, the following example works like the one above.
Private _Messages As New System.Collections.Concurrent.BlockingCollection(Of String)
Private Sub Process()
' process each message in the list
For Each item In _Messages
ProcessMessage(_Messages.Take())
Next
End Sub
Private Sub AddMessage(ByVal message As String)
' add a message to the 'global' list
_Messages.Add(message)
End Sub
Simplicity itself…
Theory:
Once a thread acquires the _SyncLockObject lock all other threads reentering that method will have to wait for the lock to be released.
So the check before and after the lock is irrelevant. In other words, it will have no effect. It is also not safe, because you're not using a concurrent list.
If one thread happens to check the Count in the first test while another is clearing or adding to the collection, then you'll get an exception with Collection was modified; enumeration operation may not execute.. Also, the second check can only be executed by one thread at a time (since it's synced).
This applies for your Add method as well. While the lock is owned by one thread (meaning the execution flow has reached that line), no other threads will be able to process or add to the list.
You should be careful to also lock if you are just reading from the list in some other places in your application. For more complex read/write scenarios (such as a custom concurrent collection), I recommend using ReaderWriterLockSlim.
Practice:
Use a BlockingCollection, since it is thread safe (i.e. it handles concurrency internally).
Thanks for reading!
I am currently trying to develop an application which will backup large folders to a specified destination. To find all the files in the specified 'backup' directory I am using the following code.
Public Shared Function GetFilesRecursive(ByVal initial As String) As List(Of String)
Dim result As New List(Of String)
Dim stack As New Stack(Of String)
stack.Push(initial)
Do While (stack.Count > 0)
Dim dir As String = stack.Pop
Try
result.AddRange(Directory.GetFiles(dir, "*.*"))
Dim directoryName As String
For Each directoryName In Directory.GetDirectories(dir)
stack.Push(directoryName)
Next
Catch ex As Exception
'stay quiet
End Try
Loop
Return result
End Function
From here I am using BackgroundWorker to copy each file and report the progress of the completed list to the GUI via a progress bar.
This is fine and works great until I come to a large directory, say C:\Windows where it hangs and freezes the GUI until it completes, this is horrible!
I have tried putting my GetFilesRecursive function into a separate background worker to run first so I can update the GUI however I am struggling on how to return the List of found files back to my application (I get cross-threading exception) and how to update the progress bar to show the progress of the GetFileRecursive function so the user knows it is processing the list of files and has not crashed/frozen.
Thank you very much in advance for your input! :)
Steve
For reference, here's the BackgroundWorker manual page.
First, have a look at the RunWorkerAsync(Object) method. With this method, you can pass an object to your 'worker' function. This might be an empty stack, for example.
In your RunWorkerCompleted event handler, you can grab that same object (now populated with file paths) from the RunWorkerCompletedEventArgs and do what you will with it. The reason I suggest passing an object is that you avoid the need for a static variable. In this case, a static variable is a bit of a code smell and should be avoided.
If you want to report the progress of the routine, you need to call the ReportProgress(int) function from within your worker function. This will fire the ProgressChanged event which you can use to drive your progress bar.
MSDN also has a pretty good example of the pattern here. They don't do anything to demonstrate progress reporting but there's an example of that on the ReportProgress page.
Edit:
If it's a plain Stack you're using, then I suggest creating the stack, then passing it to RunWorkerAsync( ). From here, you can get the stack from the event args of BackgroundWorker_DoWork, which you can then pass to your actual worker function, in this case, GetFilesRecursive( ). So GetFilesRecursive( ) does not actually return anything, but it's able to fill the stack, which you can then access again from your ProgressCompleted handler. Mind you, you'll need to cast that object back to a Stack (or whatever type of object you decide to use) before you actually use it.
Big picture: the stack is like a bucket which you pass around, and gets filled. You first create the bucket when you call RunWorkerAsync( ), passing the bucket as an argument.
I have been creating multiple background threads to parse xml files and recreate new xml files. Now the problem I am having is that even though I use synclock on global variables, I will still at times get errors and I am sure that this is just the crude way of coding I am doing, but I was wondering if someone had a better option.
program flow =
access local folder and upload all files into list
strip each file into xml entries and put these entries into an arraylist
parse for specific values and enter these values into a database table
now create a thread and take the arraylist of entries and the thread will reparse
thread parses and creates a new xml file
main thread continues with another function and then goes and get a file from list
I will add some code to show problem areas but if I have declared global variable in use does the different threads overwrite that value in the variable causing contamination.
For Each g In resultsList
gXmlList.Add(g)
Next
Dim bgw As New BackgroundWorker
bgw.WorkerSupportsCancellation = True
AddHandler bgw.DoWork, New DoWorkEventHandler(AddressOf createXML)
AddHandler bgw.RunWorkerCompleted, AddressOf WorkComplete
threadlist.Add(bgw)
bgw.RunWorkerAsync()
Private Sub createXML()
num += 1
Dim file As String = Module1.infile
xmlfile = directoryPath & "\New" & dateTime.Now.ToUniversalTime.ToString("yyyyMMddhhmmss") & endExtension
Thread.Sleep(2000)
Dim doc As XmlDocument = New XmlDocument
**xwriter = New XmlTextWriter(xmlfile, Encoding.UTF8)** this is where ioexception error
xwriter.Formatting = Formatting.Indented
xwriter.Indentation = 2
xwriter.WriteStartDocument(True)
xwriter.WriteStartElement("Posts")
I have global variables through out the app and should I be locking each one and does this not make using threads then useless.
Dim j As Integer = 0
I believe your biggest problem is not knowing what features in .Net are thread safe. A list for example is not (a dictionary is). While you may get away with it you will eventually run into problems with locking, etc.
Your using classes and variables that are not thread safe. Any time you are working with threads you have to be Extremely careful with locking. To answer your question, yes, you have to lock and unlock everything you are working with unless the type / method specifically handles it for you.
There are a lot of multi threading (PLINQ for example) in .Net 4.0 which handle a lot of the "grunt work" for you. While you should learn and understand how to do thread safe code yourself it will give you a head start.
Try passing the data into the createXML() method. That may help isolate the code from other data being accessed. I would suggest reading up on threading and learning how to do it without a background worker.
Global variables are generally a bad idea. Given your VB code I'm guessing this is a carry over from the VB6 world for you. That's not in any way intended to be insulting, just trying to help advance your skills forward. Variable scope should be as confined as possible.
Another thought looking at your code is to learn how to use String.Format() when building strings / paths.
Simple manual thread in VB to get you started:
Dim bThread As New Threading.Thread(AddressOf createXML)
bThread.IsBackground = True
bThread.Start()
Well if you are having issues with thread locking then you can simply wrap your action in the following manor.
'This will need to be out of scope so that all threads have access to it
Dim readerWriterLock As New Threading.ReaderWriterLockSlim
readerWriterLock.EnterWriteLock()
xwriter = New XmlTextWriter(xmlfile, Encoding.UTF8)
'other logic
readerWriterLock.ExitWriteLock()
'anything reading from this would need to have the following
readerWriterLock.EnterReadLock()
'logic
readerWriterLock.ExitReadLock()
Try this and then if not successful post the exception message and any other information that you can.