System.IO.File.ReadAllLines() method with progress bar in vb.net - vb.net

I am reading a large text file in vb.net using System.IO.File.ReadAllLines(TextFileURL). Since the process takes a few seconds to finish, would there be a possibility to use a progress bar?
.
RawFile = System.IO.File.ReadAllLines(TextFileURL)
lines = RawFile.ToList
If arg = "" Then MsgBox("IMPORTER IS DONE")
.
There is no loop or anything that could be used to update the value of the progress bar. Any thoughts or workaround would be appreciated.

The following reads a pretty big .TXT file line by line and reports progress:
Code:
Imports System.IO
Public Class Form1
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
End Sub
Private Async Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim dialog As New OpenFileDialog
dialog.Filter = "Text|*.txt"
Dim result = dialog.ShowDialog()
If result <> DialogResult.OK Then
Return
End If
Dim stream = File.OpenRead(dialog.FileName)
Dim reader As New StreamReader(stream)
Dim percentage As Integer
While True
Dim line As String = Await reader.ReadLineAsync()
If line Is Nothing Then
Exit While
End If
' TODO do something with your line
Dim percentD As Double = 1D / stream.Length * stream.Position * 100D
Dim percentI As Integer = Math.Floor(percentD)
If percentI > percentage Then
ProgressBar1.Value = percentI
percentage = percentI
End If
End While
Await stream.DisposeAsync()
End Sub
End Class
Result:
Notes:
this puts a burden on the stream as ultimately reading a line is small data
try using a buffered stream to reduce pressure
notice that I only report when integer percentage is greater than previous
you'd overwhelm UI when updating the progress bar otherwise
there is trivial async usage, you may want to improve that overall
the progress bar doesn't exactly reach 100%, I let you fix that, it's pretty EZ to do

You can use ReadLines instead of ReadAllLines
as docs said, when you are working with very large files, ReadLines can be more efficient:
Dim lstOflines as List(Of String)
For Each line As String In File.ReadLines(TextFileURL)
lstOflines.Add(line)
Next line
For get the total number of lines, you can make a guess based on file Size instead processing two times the file
Code for getting filesize: (use before start processing)
Dim myFile As New FileInfo(TextFileURL)
Dim sizeInBytes As Long = myFile.Length

Related

VB.NET - System.AccessViolationException: 'Attempted to read or write protected memory...' Occurs sporadically when using autoCompleteSource

System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'
I started getting this exception after trying to implement AutoComplete into one of my text boxes. It seems to occur somewhat sporadically? It often occurs when trying to use AutoComplete within a couple seconds of loading it's panel but has occured upwards of 5/6 seconds afterwards too. Additionally if it doesn't occur, the AutoComplete can be used endlessly without crashing.
I've had the same error occur on two different machines so I'm really unsure of what could be causing this and would be endlessly grateful if someone could point me in the right direction!
I've tried to strip out all the parts that definitely aren't causing it to save space.
This code runs first:
Private Sub btnManagerEditItem_Click(sender As Object, e As EventArgs) Handles btnManagerEditItem.Click
refreshEditItemsScreen(False) ' clears all on screen data and repopulates DataGrid
txtManagerEditItemsSearchName.AutoCompleteMode = AutoCompleteMode.SuggestAppend
txtManagerEditItemsSearchName.AutoCompleteSource = AutoCompleteSource.CustomSource
End Sub
Private Sub refreshEditItemsScreen(refreshDatabase As Boolean)
dtgrdManagerEditItems.Rows.Clear() ' DataGridView displays items to user.
If refreshDatabase Then
updateItemDatabase() ' reads the database from file and updates _itemDatabase with it
End If
For Each entry In _itemDatabase ' Global variable holding the database
dtgrdManagerEditItems.Rows.Add(entry.Name, entry.Price.ToString("c"), entry.UniqueID, getStaffDictionary(entry.StaffID))
Next
End Sub
Private Sub updateItemDatabase()
_itemDatabase = readItemDatabaseFromFile()
End Sub
Private Function readItemDatabaseFromFile() As List(Of Item)
Dim pathtofile As String = IO.Path.Combine(My.Application.Info.DirectoryPath.Substring(0, My.Application.Info.DirectoryPath.Length - 21), "Item Database.xml")
Dim itemDatabaseFromFile As New List(Of Item)
If System.IO.File.Exists(pathtofile) Then
Dim objStreamReader As New StreamReader(pathtofile)
Dim xsSerialize As New XmlSerializer(itemDatabaseFromFile.GetType)
itemDatabaseFromFile = xsSerialize.Deserialize(objStreamReader)
objStreamReader.Close()
End If
Return itemDatabaseFromFile
End Function
Then this code runs when you start typing in the AutoCorrect box:
Private Sub txtManagerEditItemsName_TextChanged(sender As Object, e As EventArgs) Handles txtManagerEditItemsSearchName.TextChanged
txtManagerEditItemsSearchName.AutoCompleteCustomSource = _autoCompleteSource
End Sub
_autoCompleteSource gets it's value when this code is ran on form load (updateItemDatabase() is also ran on form load):
Private Sub updateAutoCompleteSource()
' reads the id dictionary from file and adds it to the auto complete source
For Each Itm In readItemIDDictionaryFromFile().Keys
_autoCompleteSource.Add(Itm)
Next
End Sub
Private Function readItemIDDictionaryFromFile() As Dictionary(Of String, String)
Dim pathtofile As String = IO.Path.Combine(My.Application.Info.DirectoryPath.Substring(0, My.Application.Info.DirectoryPath.Length - 21), "ID Dictionary.txt")
Dim output As New Dictionary(Of String, String)
If File.Exists(pathtofile) Then
For Each line In IO.File.ReadAllLines(pathtofile)
Dim parts() As String = line.Split(",")
output.Add(parts(1), parts(0))
Next
Return output
End If
End Function
Sorry if some of this code was unnecessary to post, I'm pretty lost as to what's causing the error so I just posted everything that runs before it occurs that I think could be related to it.
Thank you

reading from txt file and writing in textbox

Can somebody help me with this, im stuck no idea what to do next
give a text file at any location in the computer it contains 15 different integers that can be repeated
make the program so that multiple repetitions of one number are not printed separately, but each number is printed exactly once
and next to it are written in which places he appeared
Imports System.IO
Public Class form1
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
Dim povratnaVrijednost As DialogResult
Dim nazivDatoteke As String
Try
OpenFileDialog1.AddExtension = True
OpenFileDialog1.Multiselect = False
OpenFileDialog1.Filter = "Tekst datoteke (*.txt)|*.txt;"
povratnaVrijednost = OpenFileDialog1.ShowDialog()
If povratnaVrijednost = Windows.Forms.DialogResult.OK Then
If OpenFileDialog1.CheckFileExists = True And
OpenFileDialog1.CheckPathExists = True Then
nazivDatoteke = OpenFileDialog1.FileName
TextBox1.Text = nazivDatoteke
Dim citac As New StreamReader(nazivDatoteke)
Dim redTeksta As String = ""
Do
redTeksta = citac.ReadLine()
If Not redTeksta Is Nothing Then
RichTextBox1.Text = RichTextBox1.Text + redTeksta
End If
Loop Until redTeksta Is Nothing
citac.Close()
End If
End If
Catch ex As Exception
MsgBox("Greska prilikom otvaranja" + ex.StackTrace.ToString)
End Try
End Sub
End Class
123
Requirement #1:
give a text file at any location in the computer it contains 15 different integers that can be repeated
This requirement implies a couple of other requirements. First, you're expected to read the text file. Second, you're expected to parse the values into numbers (presumably integers).
You can use an OpenFileDialog (documentation) and specify that it can only accept text files:
Using browseFileDialog = New OpenFileDialog()
With browseFileDialog
.Filter = "*.txt|*.txt"
If (.ShowDialog() = DialogResult.Ok) Then
'.FileName will be the text file that the user picked
End If
End With
End Using
To read the text file, assuming you want each line, use the File.ReadAllLines method (documentation):
Using browseFileDialog = New OpenFileDialog()
With browseFileDialog
.Filter = "*.txt|*.txt"
If (.ShowDialog() = DialogResult.Ok) Then
Dim lines = IO.File.ReadAllLines(.FileName)
End If
End With
End Using
To parse the values, use the Array.ConvertAll method (documentation) and inside of the predicate use the Integer.Parse method (documentation):
Using browseFileDialog = New OpenFileDialog()
With browseFileDialog
.Filter = "*.txt|*.txt"
If (.ShowDialog() = DialogResult.Ok) Then
Dim lines = IO.File.ReadAllLines(.FileName)
Dim values = Array.ConvertAll(lines, Function(line) Integer.Parse(line))
End If
End With
End Using
Keep in mind that if you wanted to validate that the lines are all valid numbers, instead of assuming that they are, this step would be different. However, you didn't specify that requirement in your original post so I'm not including how to do that.
Requirement #2:
make the program so that multiple repetitions of one number are not printed separately, but each number is printed exactly once
You can use the Random.Next method (documentation) to generate random values. Be sure to declare the new instance of the random object once so that the seed is only set once. To randomly order the values, use the OrderBy method (documentation) passing the random value in the predicate:
Private ReadOnly _random As New Random()
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Using browseFileDialog = New OpenFileDialog()
With browseFileDialog
.Filter = "*.txt|*.txt"
If (.ShowDialog() = DialogResult.Ok) Then
Dim lines = IO.File.ReadAllLines(.FileName)
Dim values = Array.ConvertAll(lines, Function(line) Integer.Parse(line))
Dim randomlyOrderedValues = values.OrderBy(Function(value) _random.Next())
RichTextBox1.Text = String.Join(", ", randomlyOrderedValues.ToArray())
End If
End With
End Using
End Sub
Default for .AddExtension is True. Default for .Multiselect is False. I have simplified the If statement by comparing the return value of .Showdialog directly to the desired DialogResult. If the path of file don't exist the dialog will show a waring.
Always scope variables as narrowly as possible. I move the Dim nazivDatoteke to inside the If since it is not used anywhere else. StreamReaders require a Using block since they need to be disposed.
Why a RichTextBox for a text file?
As you can see getting the distince items in the array is easy, a single line of code. What is more difficult is finding the indexes in the original array, lines.
I looped through all the items in the distinct array (actually it is an IEnumerable(Of String) but that is not important to this code, it is just easier to type array in my explanation). I created a List(Of Integer) to hold the indexes of each item in the original array, lines. The startIndex where the FindIndex method of an Array is the location of the first occurrence of the String.
On each iteration of the Do loop I look for the matches to the item from the lines array. The FindIndex takes parameters (original array, index to start at, what we are looking for). It stops as soon as it finds a match and returns -1 if no matches are found. You can see the operation of this method be setting a break point and checking the variable values.
If a match is found I added that value to the list and change the startIndex to the position following the found index.
Next I used a StringBuilder to create the string to display. I chose the immediate window for testing purposes but you could show sb.ToString in a text box or add it to a ListBox.
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
OpenFileDialog1.Filter = "Tekst datoteke (*.txt)|*.txt;"
If DialogResult.OK = OpenFileDialog1.ShowDialog Then
Dim nazivDatoteke = OpenFileDialog1.FileName
TextBox1.Text = nazivDatoteke
Dim lines = File.ReadAllLines(nazivDatoteke)
Dim distinct = lines.Distinct
For Each item In distinct
Dim lst As New List(Of Integer)
Dim startIndex = Array.IndexOf(lines, item)
Do
Dim FoundIndex = Array.FindIndex(lines, startIndex, Function(line) line = item)
If FoundIndex = -1 Then
Exit Do
Else
lst.Add(FoundIndex)
startIndex = FoundIndex + 1
End If
Loop
Dim sb As New StringBuilder
sb.Append(item)
For Each i In lst
sb.Append($" ({i})")
Next
Debug.Print(sb.ToString)
Next
End If
End Sub
This code displays the distinct items in the list and follow it with the positions (indexes) in the original list.

ListView displays incorrect images for each item

I have a listview that lists all items from a directory like so:
But as you can see in this image:
They are not in order, because one image text has been skipped entirely, however the image itself is still in order.
But, when it gets further down the list, starting from about half way they start becoming completely mixed up like in this example:
Image 3
When clicking an image it shows the preview of the correct image on the right side.
image 4
This is the code I'm using to load in all the images:
Dim imgList As New ImageList
Dim imgSize As New Size
Dim count As Integer = 0
Dim imgFilename As String
Private Sub BackgroundWorker1_DoWork(sender As Object, e As System.ComponentModel.DoWorkEventArgs) Handles BackgroundWorker1.DoWork
'Dim imlTemp As New ImageList
Dim dirFiles() As String = IO.Directory.GetFiles(My.Settings.GalleryLocation)
'Dim item As New ListViewItem
For Each dirFile As String In dirFiles
imgFileName = IO.Path.GetFileName(dirFile)
Dim img As New System.Drawing.Bitmap(dirFile)
Dim imgImage As Image = Image.FromFile(dirFile)
'Dim imgHeight As Integer
'imgHeight = imgImage.Height
imgSize.Width = 120
imgSize.Height = 174
Threading.Thread.Sleep(10)
BackgroundWorker1.ReportProgress(100 / ((dirFiles.Count + 1) - count), img)
Next
End Sub
Private Sub BackgroundWorker1_ProgressChanged(ByVal sender As Object, ByVal e As System.ComponentModel.ProgressChangedEventArgs) Handles BackgroundWorker1.ProgressChanged
ListView1.SmallImageList = imgList
ListView1.LargeImageList = imgList
imgList.ImageSize = imgSize
imgList.ColorDepth = ColorDepth.Depth32Bit
ListView1.Items.Add(imgFilename, count)
imgList.Images.Add(e.UserState.Clone)
count += 1
'ListView1.EnsureVisible(ListView1.Items.Count - 1)
End Sub
Before I added in the backgroundworker it had tremendous loading times for a large amount of images, so I thought I'd implement the backgroundworker to allow async work to be done. However, something is going completely wrong at the start of the task and multiple times part way through the list, where it completely messes up as shown in Image 3.
Does anybody have any idea what is going wrong, or any alternative solutions to what I'm aiming to do?
Since ReportProgress() is not blocking (as far as I know), there is a chance that the files are iterated though faster than the UI can update.
To keep this synchronized you should make a custom class holding everything you want to update and pass that to the ReportProgress() method (even making some steps automated).
For example:
Public Class GalleryImage
Public Property FullPath As String
Public Property FileName As String
Public Property [Image] As Image
Public Sub New(ByVal File As String)
Me.Image = Image.FromFile(File) 'Get the image.
Me.FullPath = File 'Save the full path of the image.
Me.FileName = Path.GetFileName(File) 'Get the file name.
End Sub
End Class
(Also, if you want the file name but not the extension you can use Path.GetFileNameWithoutExtension())
Now to your code. First of all the counter variable isn't really necessary as it can be replaced by either ListView1.Items.Count or imgList.Images.Count.
Secondly, you shouldn't constantly keep setting the imgSize variable nor the ListView's Small-/LargeImageList properties. Doing that is completely unnecessary and will slow things down. For the ListView just set the image list once, and for the imgSize variable you can do like this:
Dim ReadOnly imgSize As New Size(120, 174)
Making the variable ReadOnly does what it sounds like; you can read from it, but not modify it.
Now to fix these other things we'll start in the BackgroundWorker's For Each loop:
For Each dirFile As String In dirFiles
Threading.Thread.Sleep(10) 'You don't need this Sleep if you don't want to.
'Report the progress, declare an in-line version of our class and send it with.
BackgroundWorker1.ReportProgress(100 / ((dirFiles.Count + 1) - ListView1.Items.Count), New GalleryImage(dirFile)) 'A new GalleryImage is created. The property setting is handled by the class itself.
Next
As you see we have now narrowed the code down rather much.
Now we are going to handle the ReportProgess() event:
Private Sub BackgroundWorker1_ProgressChanged(ByVal sender As Object, ByVal e As System.ComponentModel.ProgressChangedEventArgs) Handles BackgroundWorker1.ProgressChanged
Dim img As GalleryImage = e.UserState 'Retrieve our class, no need to clone it.
imgList.Images.Add(img.Image) 'Add the image first (I don't know for sure, but if you add the image after, wouldn't it require to redraw the ListView an extra time?).
ListView1.Items.Add(img.FileName, imgList.Images.Count - 1) 'Add the new item with the values from our class.
End Sub
And lastly, the initialization and initial setting of stuff should be done in for example the Form Load event:
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
'In here we do all the standard settings and initializations.
ListView1.SmallImageList = imgList
ListView1.LargeImageList = imgList
imgList.ImageSize = imgSize
imgList.ColorDepth = ColorDepth.Depth32Bit
End Sub
And by that I think I should have covered it all.
Hope this helps!

Why VB.NET simple app takes forever to load several lines

I have a super simple script that I am using to separate a long list of phone numbers we've gathered from donors over the years into separate area code files.
Obviously, when you have almost 1 million lines it's going to take a while - BUT - if I put in 1,000 it takes less than a second. Once I put in 1 million it takes 10 seconds to do only 5 lines. How could this be?
Imports System.IO
Public Class Form1
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
System.Windows.Forms.Control.CheckForIllegalCrossThreadCalls = False
BackgroundWorker1.RunWorkerAsync()
End Sub
Private Sub BackgroundWorker1_DoWork(ByVal sender As System.Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles BackgroundWorker1.DoWork
System.Windows.Forms.Control.CheckForIllegalCrossThreadCalls = False
Dim lines As String
lines = RichTextBox1.Lines.Count
Dim looper As String = 0
Dim path As String = "c:\acs\"
MsgBox("I have " + lines + " lines to do.")
If (Not System.IO.Directory.Exists(path)) Then
System.IO.Directory.CreateDirectory(path)
End If
Dim i As String = 0
For loops As Integer = 0 To RichTextBox1.Lines.Count - 1
Dim ac As String = RichTextBox1.Lines(looper).ToString
ac = ac.Substring(0, 3)
Dim strFile As String = path + ac + ".txt"
Dim sw As StreamWriter
sw = File.AppendText(strFile)
sw.WriteLine(RichTextBox1.Lines(looper))
sw.Close()
Label1.Text = String.Format("Processing item {0} of {1}", looper, lines)
looper = looper + 1
Next
MsgBox("done now")
End Sub
End Class
Each time you use the RichTextBox.Lines properties, VB.Net will need to split the content by CR+LF pair. Thus your For loops As Integer = - To RichTextBox1.Lines.Count-1 is really a performance hit.
Try to use:
For Each vsLine As String in RichTextBox1.Lines
instead. It will be a lot faster. Alternatively, if you must use For loop, then get it once:
Dim vasLines() As String = RichTextBox1.Lines
For viLines As Integer = 0 to UBound(vasLines.Count)
'....
Next
instead.
First, you're performing UI updates in your For loop. That will take time.
You're updating the UI in a thread that is not the main thread which might impact performance. You should not use the CheckForIllegalCrossThreadCalls method. You should update the UI properly using the ReportProgress method of the BackgroundWorker.
You are opening and closing a file for each iteration of the loop. That will take time as well.
I think a better method would be to add the data to a Dictionary(Of String, List(Of String)), with the area code as the key and the list would hold all the number for that area code. Once the Dictionary is filled, loop through the keys and write the numbers out.

Populating a combo box with the first word of a textfile

So I feel like im pretty close, but I also have a feeling I am mixing up StreamReader and ReadAllLines
....................................................................................
Option Strict On
Imports System.IO
Public Class Form4
Dim file As System.IO.StreamWriter
Private Sub Form4_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
file = My.Computer.FileSystem.OpenTextFileWriter("c:\devices.bat", False)
file.WriteLine("#echo off")
file.WriteLine("cd " & Form1.TextBox2.Text)
file.WriteLine("adb devices > C:\devices.txt")
file.Close()
Shell("C:\devices.bat", AppWinStyle.Hide, True, 500)
Dim output() = System.IO.File.ReadAllLines("C:\deviceinfo2.txt")
Dim Devices As String = ""
Dim line() As String = {}
For X = 1 To output.Count = -1
line = output(X).Split(New Char() {(" ")})
Devices = line(0)
ComboBox1.Items.Add(Devices)
Next
output.Close()
output.Dispose()
End Sub
End Class
........................................................................
What I am trying to have it do is to start reading on line two of devices.txt and then read the first word from each line until the text file is done.
It seems simple enough, but like I said, I think I am mixing streamreader with readalllines
Any help is appreciated
Class Test
Public Sub Main()
Try
' Create an instance of StreamReader to read from a file.
' The using statement also closes the StreamReader.
Using sr As New StreamReader("TestFile.txt")
Dim line, firstWord As String
Dim i as Integer = 0
' Read and display lines from the file until the end of
' the file is reached.
Do
line = sr.ReadLine()
If Not (line Is Nothing) AndAlso i > 0 Then
firstWord = line.Split(" ")(i)
'do your logic
End If
i += 1
Loop Until line Is Nothing
End Using
Catch e As Exception
' Let the user know what went wrong.
End Try
End Sub
End Class
Grabbed this from MSDN and modified it. It should compile, but I didn't test it. This will loop through the lines, 1 by 1, skip the first line and grab each line's first word after. Hope this helps.