In my VB.NET program is a time consuming function that gets data and updates the UI at a periodic interval. I moved this function to another thread, but it now takes much longer to execute. Using the stopwatch class, I calculated that when it is part of the main thread, it takes 130 ms, but in the separate thread it takes 542 ms, so that's more than 4 times slower.
My CPU is a Core I5 M520 (2 cores), so I don't now why is it taking so much longer.
I am using the System.Threading.Thread class. I also tried to set the new thread's priority higher, but this had no effect.
Why is the separate thread taking so much longer and is there a way I can speed it up?
Thanks
The code:
Public Sub update(ByVal temp As Visual)
SyncLock mUpdateQueue
If Not mUpdateQueue.Contains(temp) Then
mUpdateQueue.Enqueue(temp)
End If
End SyncLock
If Not mainThread.IsAlive Then ' moet hierdie beter doen
mainThread = New Thread(AddressOf DataFetchThread)
mainThread.Start()
End If
End Sub
Private Sub DataFetchThread()
Dim s As New Stopwatch()
s.Start()
Dim temp As Visual = Nothing
While mUpdateQueue.Count > 0
SyncLock mUpdateQueue
temp = mUpdateQueue.Peek()
End SyncLock
mDataCollector.updateV(temp)
SyncLock mUpdateQueue
mUpdateQueue.Dequeue()
End SyncLock
End While
s.Stop()
Debug.WriteLine("thread run time: " & s.ElapsedMilliseconds)
End Sub
mDataCollector.updateV(temp): This function get data from a database and plots the points on a picturebox to create a graph. It wouldn't make a lot of sense to add all of the code here.
To ask this question in another way: Is it normal that the second thread takes much longer to execute or is there something wrong with my code?
You are accessing the mUpdateQueue variable from multiple threads and using locks to gaurd access to it. This is fine, but using locks has an overhead (to aquire the lock, and during the time that the other threads wait to aquire the lock). This is probably why your new thread is taking longer: it is waiting on the locking.
You could try using the ReaderWriterLockSlim class which may provide faster access to your variables. Just remember that it implements IDisposable so you need to call Dispose on it when you're done with it.
Related
I want to add workers into a queue, but only have the first N workers processing in parallel. All samples I find are in C#.
This is probably simple for a programmer, but I'm not one. I know enough about VB to write simple programs.
But my first application runs fine until it suddenly hits 100% CPU and then crashes. Help, please (Yes, I've wasted 5 hours of work time searching before posting this...)
More Context: Performing a recursive inventory of directory structures, files, and permissions across file servers with over 1 million directories/subdirectories.
Process runs serially, but will take months to complete. Management already breathing on my neck. When I try using Tasks, it goes to about 1000 threads, then hits 100% CPU, stops responding, then crashes. This is on a 16 core server with 112 GB RAM.
--Added
So, with the sample provided on using Semaphores, this is what I've put in:
Public Class InvDir
Private mSm as Semaphore
Public Sub New(ByVal maxPrc As Integer)
mSm = New Semaphore(maxPrc, maxPrc)
End Sub
Public Sub GetInventory(ByVal Path As String, ByRef Totals As Object, ByRef MyData As Object)
mSm.WaitOne()
Task.Factory.StartNew(Sub()
Dim CurDir As New IO.DirectoryInfo(Path)
Totals.SubDirectoryCount += CurDir.GetDirectories().Count
Totals.FilesCount += CurDir.GetFiles().Count
For Each CurFile As IO.FileInfo in CurDir.EnumerateFiles()
MyData.AddFile(CurFile.FileName, CurFile.Extension, CurFile.FullName, CurFile.Length)
Next
End Sub).ContinueWith(Function(x) mSm.Release())
End Sub
End Class
You're attempting multithreading with disk I/O. It might be getting slower because you're throwing more threads at it. No matter how many threads there are, the disk can physically only seek one position at a time. (In fact, you mentioned that it works serially.)
If you did want to limit the number of concurrent threads you could use a Semaphore. A semaphore is like a syncLock except you can specify how many threads are allowed to execute the code at a time. In the example below, the semaphore allows three threads to execute. Any more than that have to wait until one finishes. Some modified code from the MSDN page:
Public Class Example
' A semaphore that simulates a limited resource pool.
'
Private Shared _pool As Semaphore
<MTAThread> _
Public Shared Sub Main()
' Create a semaphore that can satisfy up to three
' concurrent requests. Use an initial count of zero,
' so that the entire semaphore count is initially
' owned by the main program thread.
'
_pool = New Semaphore(0, 3)
End Sub
Private Sub SomeWorkerMethod()
'This is the method that would be called using a Task.
_pool.WaitOne()
Try
'Do whatever
Finally
_pool.Release()
End Try
End Sub
End Class
Every new thread must call _pool.WaitOne(). That tells it to wait its turn until there are fewer than three threads executing. Every thread blocks until the semaphore allows it to pass.
Every thread must also call _pool.Release() to let the semaphore know that it can allow the next waiting thread to begin. That's important, even if there's an exception. If threads don't call Release() then the semaphore will just block them forever.
If it's really going to take five months, what about cloning the drive and running the check on multiple instances of the same drive, each looking at different sections?
In a project I’ve recently taken over, there is a call to a function which does some calculations; this is called in a row, several times (between 1 and 10 times usually).
While dr.read ‘depending on a db call, loop 1 or more times
Dim calc As New CalcClass
Dim newDoStuff As New System.Threading.Thread(New System.Threading.ParameterizedThreadStart(AddressOf DoStuff))
newDoStuff.Start(calc)
End while
Private Sub DoStuff(ByVal calc As Object)
‘do something that takes between 5-10 seconds
End sub
In order to speed this up, i am trying to add asynchronous processing (see above example), this works in my code, all tasks are done at the same time, but what I don’t understand is how to then wait for all these threads to finish (there is no set amount of threads, it can be between 1 and 10 depending on some other data) before finishing up with a final task that needs to run after all tasks are completed.
Can anyone suggest a way to do this – I’m looking for an easy way to basically say “O.k, all tasks are finished at this point, call another task”
Cliffs
Several tasks need to run at the same time (between 1 and 10)
Each task takes several seconds
Code currently works - it does them all at the same time
Once all tasks (between 1-10) are
finished, fire off some other code (only when all tasks are finished) - stuck on best method to do the following
Put all your thread in an List
Dim threads As new List(Of System.Threading.Thread)
While dr.read ‘depending on a db call, loop 1 or more times
Dim calc As New CalcClass
Dim newDoStuff As New System.Threading.Thread(New System.Threading.ParameterizedThreadStart(AddressOf DoStuff))
threads.Add(newDoStuff)
newDoStuff.Start(calc)
End while
finally join all your threads
For Each thread In threads Do
thread.Join()
Next
The easiest way is to put all of your new threads into a list, then iterate over that list and call .Join() on each one. The join method blocks the current thread until the thread you are Joining completes:
Apologies if there are any syntactic errors in the following code- I don't have VB handy and my memory of the syntax is pretty rusty:
Dim threadList as New List(Of Thread)
While dr.read ‘depending on a db call, loop 1 or more times
Dim calc As New CalcClass
Dim newDoStuff As New System.Threading.Thread(New System.Threading.ParameterizedThreadStart(AddressOf DoStuff))
newDoStuff.Start(calc)
threadList.Add(newDoStuff)
End while
For Each t as Thread in ThreadList
t.Join
End For
With that said, I'd strongly encourage you to look into using the classes in the System.Threading.Tasks namespace, as that provides a much better paradigm than starting and managing your own threads.
I've read a lot of different questions on SO about multithreaded applications and how to split work up between them, but none really seem to fit what I need for this. Here's how my program currently basically works:
Module Module1
'string X declared out here
Sub Main()
'Start given number of threads of Main2()
End Sub
Sub Main2()
'Loops forever
'Call X = nextvalue(X), display info as needed
End Sub
Function nextvalue(byval Y as string)
'Determines the next Y in the sequence
End Function
End Module
This is only a rough outline of what actually happens in my code by the way.
My problem being that if multiple threads start running Main2(), they're dealing with the same X value as in the other threads. The loop inside of main2 executes multiple times per millisecond, so I can't just stagger the loops. There is often duplication of work done.
How can I properly divide up the work so that the two threads running simultaneously never have the same work to run?
You should synchronize the generation and storage of X so that the composite operation appears atomic to all threads.
Module Module1
Private X As String
Private LockObj As Object = New Object()
Private Sub Main2()
Do While True
' This will be used to store a snapshot of X that can be used safely by the current thread.
Dim copy As String
' Generate and store the next value atomically.
SyncLock LockObj
X = nextValue(X)
copy = X
End SyncLock
' Now you can perform operations against the local copy.
' Do not access X outside of the lock above.
Console.WriteLine(copy)
Loop
End Sub
End Module
A thread manager is required to manage the threads and the work that they do. Say it is desirable to split up the work into 10 threads.
Start the manager
Manager creates 10 threads
Assign work to the manager (queue up the work, let's say it queues up 10000 work items)
Manager assigns a work item to complete for each of the 10 threads.
As threads finish thier work, they report back to the manager that they are done and recieve another work item. The queue of work should be thread safe so that items can be enqueued and dequeued. The manager handles the management of work items. The threads just execute the work.
Once this is in place, work items should never be duplicated amongst threads.
Use a lock so that only one thread can access X at a time. Once one thread is done with it, another thread is able to use it. This will prevent two threads from calling nextvalue(x) with the same value.
I have written below console application in VB.net.
My intention is to write an application that triggers every one minute and perform some task. But
when I run this application it is consuming 50% of CPU.
How can I make it to consume less CPU?
Am I calling the timer in the right place (In the main method)?
Later I would like to make a windows service with this same task and install on the server.
How can I make the application consume less CPU?
Module Module1
Dim inputPath As String = "C:\Input"
Dim outputPath As String = "C:\Output"
Dim folder As Directory
Sub Main()
Dim tmr As Timer = New Timer(New TimerCallback(AddressOf Upload), Nothing, 1000, 60000)
While Not tmr Is Nothing
End While
End Sub
Public Sub Upload(ByVal o As Object)
Dim sr As StreamReader
Dim conStr1 As String = "Data Source=TNS Name;User ID=xx; Password=xx;"
'Look up for pending requests in RQST_TBL
Dim cnn1 As New OracleConnection(conStr1)
Dim datReader As OracleDataReader
Dim cmd1 As New OracleCommand
cnn1.Open()
.....
.....
End Sub
End Module
Thank you..
I'm guessing you have two CPUs? The infinite while loop is consuming 100% of one CPU; leaving you with 50% total CPU consumption.
From the look of your code - the loop is completely unneeded. Your timer class is going to call the Upload() method when it is complete.
Remove the while loop...
While Not tmr Is Nothing
End While
And use something like Console.Readline to keep the application from closing.
Alternatively stick a thread.sleep() call inside the while loop if you really like the loop.
While Not tmr Is Nothing
End While
You were already warned about this in a previous question. Delete that code.
While Not tmr Is Nothing
End While
This is just an infinite loop. You're not actually allowing anything to get done.
As this is a console application, you probably only need a loop that sleeps for a minute, then performs your task.
As Rob said, the 50% load probably means you're using 100% of one of your CPUs cores.
Instead of the infinite loop, you can use Console.ReadLine() to keep the console application running.
While the console is waiting for input, your timer will still work as you intend it.
How do you limit the CPU of a while loop?
In this case, the code which is inside the while loop:
Private Sub wait(ByVal time)
Dim sw As New Stopwatch
sw.Start()
Do While sw.ElapsedMilliseconds < time And StillOpen = True
Application.DoEvents()
Loop
sw.Stop()
End Sub
But now, here is the issue. This loop is allowing the while loop to run every second, once a second, and the wait sub is causing this delay, as it should.
How can I limit the CPU that this is taking up? For some reason, my task manager says it is taking 50 CPUs to run this simple task, yet it should probably take no more than 1 or 2. Though the manager says it is taking that much CPU, my computer speed is not being affected at all, which is odd considering it is a two-year-old laptop.
I don't want any users to freak out about it, but knowing how people are these days....
Anyway, the language is vb.net. Can someone please help me?
Thanks!
EDIT: To clarify, that code is not inside the while loop itself, but a call for the subroutine is, i.e. wait(1000)
Use a timer event !!! Nearly no cpu effort.
You could always perform some kind of sleep between iterations of the loop...
I'm not familiar with VB.NET but a duration of 100-200ms will probably be more than enough to drop the CPU usage.
Eg:
Do while (...)
Application.blah();
System.Threading.Thread.Sleep(150);
End
Edit After some research, I think the function you want is: System.Threading.Thread.Sleep()
Your code is executing Application.DoEvents() constantly in the while loop, for the time duration specified in your time parameter. This will consume one core of your CPU, which is why you're seeing 50% processor usage (you have a dual-core processor, correct?). This is an ugly way to wait. You could instead call Thread.Sleep(), passing it the number of milliseconds you'd like your thread to wait.
If you'd like your application to stay responsive, you might also spin off a timer, and block the UI from any action until the timer triggers. Something like (lightly tested):
// constructor or designer code
System.Windows.Forms.Timer timer = new System.Windows.Forms.Timer();
timer.Tick += new EventHandler(timer_Tick);
void Wait(int interval)
{
timer.Interval = interval;
timer.Start();
BlockUIOperations(); // implement yourself
}
void timer_Tick(object sender, EventArgs e)
{
timer.Stop();
EnableUIOperations(); // implement yourself
}
Here's my attempt at a translation into VB:
'' Add a Timer object to the form named "Timer".
'' Hook its Tick event to Timer_Tick
Private Sub Wait(ByVal interval As Integer)
Timer.Interval = interval
Timer.Start()
BlockUIOperations() '' implement yourself
End Sub
Private Sub Timer_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer.Tick
Timer.Stop()
EnableUIOperations() '' implement yourself
End Sub
Well, the CPU is always running at 100% when it's running, so the only practical way to limit the CPU usage is to run bursts or loop and sleeping in between.
Laptop CPUs usually have some SpeedStep technology or equievalent that will slow down the CPU when it's not working hard, but it's not reasonable to assume that your application would have access to control that, at least not directly. You might be able to affect it indirectly by measuring the CPU usage and adjust the length of the work and sleep cycles to get the desired result.
If you don't mind blocking the current thread, you could use a WaitHandle.
Public Sub Wait(ByVal ms As Integer)
Using wh As New ManualResetEvent(False)
wh.WaitOne(ms)
End Using
End Sub
Sub Main()
Console.WriteLine("Hello World!")
Wait(5000)
Console.WriteLine("Good-Bye!")
End Sub
Of course, something more complex can be constructed depending on what you are trying to accomplish.
This is perfect as a VB.net sleep replacement. Now my console app is NOT reported as non responsive since I have no sleep commands!
Just add Imports System.Threading above your module and place this just above your sub main
Public Sub Wait(ByVal ms As Integer)
Using wh As New ManualResetEvent(False)
wh.WaitOne(ms)
End Using
End Sub
Then, in your sub main, use
wait(100)
to pause your app for 100 miliseconds.
Have fun
You should take note of if you are doing this in the main UI Thread or a thread you have spun off.
For Threads the easiest way is to just Thread.Sleep(x miliseconds)
On the main UI thread I tend to use a DoEvents function in vb.net and vb6 like this
Public Sub TimeKiller(byval secondstowait as integer)
dim tmptime as datetime = datetime.now
do while datetime.now < dateadd("s",secondstowait,tmptime)
Application.Doevents
end while
End Sub
On the question of CPU usage I look at it like this.... if you make just a hard loop that like
while true
end while
I would expect to see very high cpu usage over 50% because the UI thread is hard blocking on this.... in most cases the windows system will limit the cpu usage of any given program so that its threads dont block the entire system.
The DoEvents ensure that windows message pumps fire correct and respond to correct. It also ensures that the garbage collector fires on time.
Also if you have other threads spun up off of your UI.Thread your UI.Thread can respond to events fired from these other threads....
In such cases where your calling form controls from other threads and do form.InvokeRequired routines will be able to respond correctly.
Also The only time you should be hard looping on the MainUI thread is when it is in response to some user activity and you need to put waits in for the user to see progress of something....
If it is some kind of automated process that is always running... look to moving it to another thread.
Or if its something that runs periodically on a timer or a time that kicks off a thread.
Somebody please tell me if I am wrong on these assumptions....
Not sure about the Using wh As New ManualResetEvent(False) wh.WaitOne(ms) as I have never heard of that and have no idea what that does.