How to avoid "Out Of Memory" exception when reading large files using File.ReadAllText(x) - vb.net

Question

This is my code to search for a string for all files and folders in the drive "G:\" that contains the string "hello":
Dim path = "g:\"
Dim fileList As New List(Of String)
GetAllAccessibleFiles(path, fileList)
'Convert List<T> to string array if you want
Dim files As String() = fileList.ToArray
For Each s As String In fileList
Dim text As String = File.ReadAllText(s)
Dim index As Integer = text.IndexOf("hello")
If index >= 0 Then
MsgBox("FOUND!")
' String is in file, starting at character "index"
End If
Next
This code will also results in memory leak/out of memory exception (as I read file as big as 5GB!). Perhaps it will bring the whole file to the RAM, then went for the string check.
Dim text As String = File.ReadAllText("C:\Users\Farizluqman\mybigmovie.mp4")
' this file sized as big as 5GB!
Dim index As Integer = text.IndexOf("hello")
If index >= 0 Then
MsgBox("FOUND!")
' String is in file, starting at character "index"
End If
But, the problem is: This code is really DANGEROUS, that may lead to memory leak or using 100% of the RAM. The question is, is there any way or workaround for the code above? Maybe chunking or reading part of the file and then dispose to avoid memory leak/out of memory? Or is there any way to minimize the memory usage when using the code? As I felt responsible for other's computer stability. Please Help :)

Answer 1

You should use System.IO.StreamReader, which reads line by line instead all the lines at the same time (here you have a similar post in C#); I personally never use ReadAll*** unless under very specific conditions. Sample adaptation of your code:
Dim index As Integer = -1
Dim lineCount As Integer = -1
Using reader As System.IO.StreamReader = New System.IO.StreamReader("C:\Users\Farizluqman\mybigmovie.mp4")
Dim line As String
line = reader.ReadLine
If (line IsNot Nothing AndAlso line.Contains("hello")) Then
index = line.IndexOf("hello")
Else
If (line IsNot Nothing) Then lineCount = line.Length
Do While (Not line Is Nothing)
line = reader.ReadLine
If (line IsNot Nothing) Then
lineCount = lineCount + line.Length
If (line.Contains("hello")) Then
index = lineCount - line.Length + line.IndexOf("hello")
Exit Do
End If
End If
Loop
End If
End Using
If index >= 0 Then
MsgBox("FOUND!")
' String is in file, starting at character "index"
End If