VB.NET Constantly polling a directory for files is causing an ongoing increase in memory usage - vb.net

I have a small application that on startup instantiates a class, reads in a JSON array from a text file and builds an object. It then initialises a new system.timers.timer and scans a directory every xx minute(s) to get a file count.
When my application starts up, Windows Task Manager shows it at around 13MB of Memory, however, leaving it to run for about 12 hours now shows it at 700+MB of Memory when no files were even detected in this directory. I'm worried that this is just going to grow forever and eventually crash. It doesn't seem like anything is getting cleaned up by the GC (probably because my timer and objects are always active).
Although this class is quite simple in theory, I do have other functions and timers involved that don't get called until required. I am happy to post all my code somewhere, or even just the class, but it's a little too long to add to this post.
I have included the main elements of my class however:
Public Sub Initialize()
Try
If AgentSettings.DebugMode Then CreateLog("Entered Module: Initialize()")
CreateLog("Initializing Alerting Rule: <" & Me.Name & ">")
CreateLog("Validating all Rule Profiles")
'Loop through all the Rule Profiles and make sure they are all valid
'If not, disable the profile so it never scans
For Each profile As RuleProfile In Me.RuleProfiles
CreateLog("Verifying Profile: <" & profile.Name & ">")
'Check to make sure the Profile is functionally valid
If ProfileIsValid(profile) Then
'Profile is valid so setup an event handler to handle the isBroken event
profile.Active = True
CreateLog("Rule Profile: <" & profile.Name & "> Enabled")
AddHandler profile.BrokenStatusChanged, AddressOf CheckIfRuleIsBroken
Else
'Disable the Profile
CreateLog("Rule Profile: <" & profile.Name & "> Disabled")
profile.Active = False
End If
Next
'Setup the checking timer and set the interval to the default checking time
If Me.ruleCheckTimer Is Nothing Then
Me.ruleCheckTimer = New System.Timers.Timer
Me.ruleCheckTimer.AutoReset = True
Me.ruleCheckTimer.Interval = Me.DefaultMinutesToCheck * 60000
AddHandler Me.ruleCheckTimer.Elapsed, AddressOf CheckRule
End If
'If there is an escalation plan then setup a timer for it
If Me.EscalationPlan.PlanItems.Count > 0 Then
'Setup the escalation timer and set the interval to the first plan item
If Me.escalationTimer Is Nothing Then
Me.escalationTimer = New System.Timers.Timer
Me.escalationTimer.AutoReset = True
AddHandler Me.escalationTimer.Elapsed, AddressOf Escalate
End If
End If
Catch ex As Exception
CreateLog("Module: Initialize()" & vbNewLine & "Exception Error: " & ex.Message)
End Try
End Sub
Private Sub CheckRule(ByVal sender As Object, ByVal e As EventArgs)
Try
If AgentSettings.DebugMode Then CreateLog("Entered Module: CheckRule()")
'Loop through all the profiles
For Each profile As RuleProfile In Me.RuleProfiles
CreateLog("Scanning for active profiles")
'Make sure the Profile is active
If profile.Active Then
Select Case profile.Name
Case "FileCount"
CreateLog("Checking Rule Profile: <" & profile.Name & "> for isBroken status")
'Get file count of the specified directory
Dim files As String()
files = Directory.GetFiles(profile.DirectoryPath)
'Determine if we're checking for files greater than or less than
If profile.GreaterThan Then
If files.Count > profile.FileCount Then
profile.isBroken = True
Else
profile.isBroken = False
End If
Else
If files.Count < profile.FileCount Then
profile.isBroken = True
Else
profile.isBroken = False
End If
End If
files = Nothing
End Select
End If
Next
Catch ex As Exception
CreateLog("Module: CheckRule()" & vbNewLine & "Exception Error: " & ex.Message)
End Try
End Sub
When profile.isBroken receives an update, it triggers an event which then loops through all profiles to determine which ones are broken. If all are broken, then it begins its alerting and escalation process. However, none of this is occurring and my memory is still increasing constantly.
Can anyone see anything wrong with this? Or maybe a better way (and more efficient way) of polling this directory?
The theory behind my application is to instantiate multiple classes to allow various directories or files to be scanned, but if the memory has increased this much with just the one instance, I'm concerned about running multiple.
EDITED
My log handling is as follows:
Public Sub CreateLog(ByVal text As String)
Try
'Create Log Directory if it doesn't exist
If (Not System.IO.Directory.Exists(GetFolderPath(SpecialFolder.CommonApplicationData) & "\MyCompany\Agent\Log")) Then System.IO.Directory.CreateDirectory(GetFolderPath(SpecialFolder.CommonApplicationData) & "\MyCompany\Agent\Log\")
'Write log entry
LogFile = GetFolderPath(Environment.SpecialFolder.CommonApplicationData) & "\MyCompany\Agent\Log\" & "Logfile - " & Format(Now, "ddMMyyyy") & ".txt"
File.AppendAllText(LogFile, DateTime.Now.ToString("dd/MM/yyyy HH:mm:ss") & " " & text & Environment.NewLine)
'Update the activity text on the main UI
frmMain.UpdateActivityText(text)
Catch ex As Exception
'No need to create a log entry
End Try
End Sub

Related

Multiple timers, same handler? MultiThreading, Server requests

I am trying to make a program that will test a small server.
The task is to make cyclic readings at some intervals.
For this, I thought to create separate timers (each one with its own interval), but on the Tick event, to have same handler.
Some code:
If testStarted Then
Dim timer As New CustomTimer(tempCyclicReading.ReadInterval)
timer.AssignedNodeId = tempCyclicReading.NodeId
timer.DisplayName = tempCyclicReading.DisplayName
AddHandler timer.Elapsed, AddressOf CyclicReadingTimer_Tick
timer.Enabled = True
cyclicReadingTimers.Add(timer)
End If
Private Sub CyclicReadingTimer_Tick(sender As Object, e As EventArgs)
Try
Dim obj As CustomTimer = TryCast(sender, CustomTimer)
Dim info As DataValue = ServerObj.ReadValue(tEO.Session, obj.AssignedNodeId)
Dim temp As String = ""
temp &= " Cyclic reading: "
temp &= "ItemName: " & obj.DisplayName & ", "
temp &= "Value: " & CStr(info.Value) & ", "
LogInBox(rtbLiveData, Color.ForestGreen, temp)
Catch ex As Exception
Utils.ReadException(ex)
logger.Error("Error when ticking cycling reading timer!" & vbCrLf & Utils.ReadException(ex))
End Try
End Sub
Now I have some questions:
Is it ok to have same handler for all timers? What happens if 2 or timers will call the function ServerObj.ReadValue (which connects to the server and read a value from a node) ? I know that each timer is created in each own thread. BUt when the tick event happens, and the handler function is called, is a new instance (corresponding to each timer) of the handler, created ?
DO I need to provide a lock mechanism or the server will handle this itself?
I also have a LogInBox function, that writes some results in a richtexbox. Again, does the richtextBox have a buffer or a queue and knows to prioritize and server each call to display data?
Thank you so much!

Making my Multi-threaded Task Asynchronous VB.net

So I have a task which loops through a sub as many times as there are people in my array. I know that making this Async will be a lot better, but I am not sure the best way to go about doing this. Here is my code:
Private Sub status()
Dim selectedStreamers As New List(Of String)
For Each item In streamerList.CheckedItems
selectedStreamers.Add(item)
Next
Dim tasks As New List(Of Task)()
For Each streamerName In selectedStreamers
Dim t As Task = Task.Run(Sub()
streamerOnline(streamerName)
Select Case condition
Case StreamersCondition.Streaming
Me.Invoke(Sub() console.Text &= streamerName + " is online." & vbNewLine)
Case StreamersCondition.Offline
Me.Invoke(Sub() console.Text &= streamerName + " is offline." & vbNewLine)
Case StreamersCondition.UserNotFound
Me.Invoke(Sub() console.Text &= streamerName + " hasn't been found." & vbNewLine)
End Select
End Sub)
tasks.Add(t)
Next
Task.WaitAll(tasks.ToArray())
End Sub
Also, nothing related to the question at hand, but if I wanted to constantly check that the streamers were streaming. I would set up a timer, right? Except I can't seem to get this working whilst running this sub? So is there any good way to keep constantly checking (so running this sub) twice per minute?

Many instances of the same process writing to the same log file

I am kicking off a number of instances of the same process and the issue is that they all write to the same log file. I know it is not a good practice and was wondering what can I do to avoid possible issues. Here is the procedure I use to write to file:
Sub WriteToErrorLog(ByVal Msg As String)
Dim path As String
path = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location)
Dim strFile As String = System.IO.Path.Combine(path, "Log_" & DateTime.Today.ToString("dd-MMM-yyyy") & ".txt")
Dim sw As StreamWriter
Dim fs As FileStream = Nothing
Try
If (Not File.Exists(strFile)) Then
fs = File.Create(strFile)
fs.Close()
End If
sw = File.AppendText(strFile)
sw.WriteLine(Msg & vbcrlf)
Catch ex As Exception
MsgBox("Error Creating Log File")
MsgBox(ex.Message & " - " & ex.StackTrace)
Finally
sw.Close()
End Try
End Sub
I would appreciate any suggestions/improvements. thanks!
As I have said in my comment, the scenario of multiple access to the same file resource should be handled carefully and probably the best solution is to use a well tested log library like Log4Net or NLog.
In any case you could improve your code in a couple of point
Sub WriteToErrorLog(ByVal Msg As String)
Dim path As String
path = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location)
Dim strFile As String = System.IO.Path.Combine(path, "Log_" & DateTime.Today.ToString("dd-MMM-yyyy") & ".txt")
Dim retry as Integer = 3 ' this could be changed if you experience a lot of collisions.'
Dim sw As StreamWriter = Nothing
While retry > 0
Try
Using sw = File.AppendText(strFile)
sw.WriteLine(Msg & vbcrlf)
End Using
Exit While
Catch ex as Exception
retry -= 1
End Try
End While
' If retry has reached zero then we have exausted our tentatives and give up....'
if retry = 0 Then
MessageBox.Show("Error writing to Log File")
End if
End Sub
I have removed all the part that check if file exists and then create it. This is not necessary because as the documentation explains, File.Append is the same that calling StreamWriter(file, true) and this means that if the file doesn't exist it will be created.
Next, to try to handle possible collision with other process writing to the same file concurrently, I have added a retry loop that could get access to the log file just after another process finishes.
(this is really a poor-man solution but then it is better to use a well tested library)
It is important to enclose the opening and writing of the file inside a using statement that closes and disposes the Stream also in case of exceptions. This is mandatory to be sure to leave the file always closed for the other processes to work.

Stop complete process tree on vb.net

I've writte this simple algorithm to stop a complete process tree from vb.net:
Private Sub TerminateProcessTree2(P As Process)
Dim Tree = GenerateProcessTree(P)
For Each childproc As Process In Tree
Try
If childproc.HasExited = False Then childproc.Kill()
Catch ex As Exception
AddError("Could not delete process " & childproc.ProcessName & ". " & ex.Message)
End Try
Next
Dim pName As String = "<unknown>"
Try
If P IsNot Nothing Then
pName = P.ProcessName
If P.HasExited = False Then P.Kill()
End If
Catch ex As Exception
AddError("Error killing process " & pName & ". " & ex.Message)
End Try
End Sub
Function GenerateProcessTree(p As Process) As Collections.Generic.HashSet(Of Process)
Dim hash As New Collections.Generic.HashSet(Of Process)
GenerateProcessTreeNode(p, hash)
Return hash
End Function
Private Sub GenerateProcessTreeNode(parent As Process, hash As Collections.Generic.HashSet(Of Process))
Dim searcher As New ManagementObjectSearcher("Select * From Win32_Process Where ParentProcessID=" & parent.Id)
Dim moc As ManagementObjectCollection = searcher.[Get]()
For Each mo As ManagementObject In moc
Dim i As Integer = CInt(mo("ProcessID"))
Dim childP As Process
Try
childP = Process.GetProcessById(i)
If childP IsNot Nothing AndAlso hash.Contains(childP) = False Then
hash.Add(childP)
GenerateProcessTreeNode(childP, hash)
End If
Catch ex As Exception
AddError("Could not get process ID for " & mo.ToString)
Continue For
End Try
Next
End Sub
But, some of my program users are telling me that, every once in a while (like one or two percent of the times), this algorithm closes ALL processes, and not only child process from the given process. How can this be possible? and does anything need to be fixed from the algorithm? I suppose there are easiest ways to do this, but I want to know why this one fails.
Your code works as expected and is correct. IMO the problem occurs because of the WMI property ParentProcessId. MSDN says:
ParentProcessId
Data type: uint32
Access type: Read-only
Unique identifier of the process that creates a process.
Process identifier numbers are reused, so they only identify
a process for the lifetime of that process. It is possible that
the process identified by ParentProcessId is terminated, so
ParentProcessId may not refer to a running process. It is also
possible that ParentProcessId incorrectly refers to a process
that reuses a process identifier. You can use the CreationDate
property to determine whether the specified parent was created
after the process represented by this Win32_Process instance
was created.
I assume, that your HashSet holds at some point ProcessId's that where replaced by the system with new processes and the new processes and not child processes anymore but are still in the collection and are terminated when fetched from the list.
You could extensively log every call of the process.Kill() (name, process id, timestamp, and so on) and then try to track the problem using the log.

Process Start and Errors from Slow Applications

I made an application that our company uses to launch databases and updates them on the users machine, when needed.
I am having a slight problem when it comes to launching databases and the database starts up slow. When this occurs my application throws an exception, as I assuming its awaiting some kind of response back.
As of now the error thrown is: The system cannot find the file specified
I am trying to prevent this exception logging for cases like this(Slow Application), but still allow the logging if a real error occurs while opening a database.
Current Code I am using:
Private Sub OpenApplication()
If File.Exists(LocalPathString) Then ' File Found. Open the File.
Try
Dim ps As New Process
ps = Process.Start(LocalPathString)
Catch ex As Exception
ex.Source += " | " & LocalPathString
RaiseEvent ShowError(ex)
Finally
RaiseEvent CancelIt() ' Thread Complete. Close the ActionForm
End Try
Else
If LocalPathString = vbNullString Then
RaiseEvent CancelIt() ' No file exits. Cancel thread.
Else
RaiseEvent ShowError(New Exception("Database Not Located: " & LocalPathString))
End If
End If
End Sub
StackTrace:
System.Diagnostics.Process.StartWithShellExecuteEx(startInfo As ProcessStartInfo)
App.exe: N 00912
System.Diagnostics.Process.Start()
App.exe: N 00136
System.Diagnostics.Process.Start(startInfo As ProcessStartInfo)
App.exe: N 00049
SAMi.ActionClass.OpenApplication()
App.exe: N 00117
Maybe I'm missing something, but why don't you simply omit the logging if you found that specific exception?
Catch ex As Exception
ex.Source += " | " & LocalPathString
if not ex.Message.Contains("The system cannot find the file specified") Then
RaiseEvent ShowError(ex)
end if