Replace Memory Issue VB.net - vb.net

i was kindly helped before with this code, however I have hit a stumbling block, and im not sure the correct way to go. I have the code below which does a find and replace on over 120k of find and replaces. The problem is the text file is HUGE easily over 5 gig of log files so i get a memory issue which is not surprise. So do i load the data in blocks if that is even possible?, if so how.
Private Sub CmdBtnTestReplace_Click(sender As System.Object, e As System.EventArgs) Handles CmdBtnTestReplace.Click
Dim fName As String = "c:\backup\logs\master.txt"
Dim wrtFile As String = "c:\backup\logs\masterUserFormatted.txt"
Dim strRead As New System.IO.StreamReader(fName)
Dim strWrite As New System.IO.StreamWriter(wrtFile)
Dim s As String
s = strRead.ReadToEnd()
'runs through over 120k of find and replaces
For Each row As DataGridViewRow In DataGridView1.Rows
If Not row.IsNewRow Then
Dim Find1 As String = row.Cells(0).Value.ToString
Dim Replace1 As String = row.Cells(1).Value.ToString
Cursor.Current = Cursors.WaitCursor
'replace using string from 1st column and replaces with string from 2nd column.
s = s.Replace(Find1, Replace1)
End If
Next
strWrite.Write(s)
strRead.Close()
strWrite.Close()
Cursor.Current = Cursors.Default
MessageBox.Show("Finished Replacing")
End Sub

If the input file is a simple multi-line text file, where no individual line is too big to load into memory at once, and the search string is never going to span multiple lines, then reading only one line at a time will be the simplest solution. For instance:
Dim fName As String = "c:\backup\logs\master.txt"
Dim wrtFile As String = "c:\backup\logs\masterUserFormatted.txt"
Dim strRead As New System.IO.StreamReader(fName)
Dim strWrite As New System.IO.StreamWriter(wrtFile)
Cursor.Current = Cursors.WaitCursor
While True
Dim line As String = strRead.ReadLine()
If line IsNot Nothing Then
For Each row As DataGridViewRow In DataGridView1.Rows
If Not row.IsNewRow Then
Dim Find1 As String = row.Cells(0).Value.ToString
Dim Replace1 As String = row.Cells(1).Value.ToString
line = line.Replace(Find1, Replace1)
End If
Next
strWrite.WriteLine(line)
Else
Exit While
End If
End While
strRead.Close()
strWrite.Close()
Cursor.Current = Cursors.Default
MessageBox.Show("Finished Replacing")
It's worth mentioning that the StreamReader and StreamReader implement IDisposable. As such, it would be preferable to enclose them in a Using block rather than explicitly calling Close yourself.

Related

How many count lines duplicates in text files

please how can I get count of duplicate lines?
Source data: line e.g. user_id;name;surname;3400;44711;30.05.2022 7:00:00;30.05.2022 15:30:00;0;480;0;1;682;10000120;9
Private Sub remove_duplicite(sender As Object, e As EventArgs)
Dim sFiles() As String
sFiles = Directory.GetFiles(filesPath1, remove_dupl)
Dim path As String = String.Join("", sFiles)
'MessageBox.Show(path)
Dim lines As New HashSet(Of String)()
'Read to file
Using sr As StreamReader = New StreamReader(path)
Do While sr.Peek() >= 0
lines.Add(sr.ReadLine())
Loop
End Using
'Write to file
Using sw As StreamWriter = New StreamWriter(path)
For Each line As String In lines
sw.WriteLine(line)
Next
End Using
Close()
End Sub
I try some answers but no success.But I think that will be easy.
Thank you
Dim sList As New List(of String)
sList.Add("1")
sList.Add("2")
sList.Add("2")
sList.Add("3")
Dim sListDistinct As List(Of String) = sList.Distinct().ToList()
Dim iCount as Integer = sList.Count - sListDistinct.Count
But depending on the size of your file, this isn't the best performance way.
Maybe check in your HashSet with .Contains and count if entry already exists

How to combine all csv files from the same folder into one data

I want merge multiple csv files with same layout from the same folder
example :
csv1.csv
ID, PINA,PINB,PCS
1,100,200,450
2,99,285,300
csv2.csv
ID, PINA,PINB,PCS
1,100,200,999
2,99,285,998
out.csv (The file I want make by VB.net)
ID, PINA,PINB,PCS,PCS
1,100,200,450,999
2,99,285,300,998
my problem code :
Dim FileReader As StreamReader
Dim i As Integer = 0
Dim temp As String
For i = 0 To LstFiles.Items.Count - 1
FileReader = File.OpenText(LstFiles.Items.Item(i))
temp = FileReader.ReadToEnd
File.AppendAllText(SaveFileDialog1.FileName, temp)
Next
Please guide me.
Thanks a lot !
Looks to me like each line in the input files has an identifier based on the first value in that row. You want to combine all the numbers after that identifier, from all the files in your ListBox, into one list of numbers that is sorted and has no duplicates. Then you want to generate an output file that has all those identifiers followed by each set of sorted, unique numbers.
If that is correct, then try this out:
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
If SaveFileDialog1.ShowDialog = DialogResult.OK Then
Dim header As String = ""
Dim combinedLines As New SortedList(Of Integer, List(Of Integer))
For Each filename As String In LstFiles.Items
Dim lines = File.ReadLines(filename)
If header = "" Then
header = lines.First
End If
lines = lines.Skip(1)
For Each line As String In lines
Dim strValues = line.Split(",").AsEnumerable
Try
Dim lineNumber As Integer = Integer.Parse(strValues.First)
strValues = strValues.Skip(1)
Dim numbers = strValues.ToList.ConvertAll(Of Integer)(Function(x) Integer.Parse(x))
If Not combinedLines.ContainsKey(lineNumber) Then
combinedLines.Add(lineNumber, New List(Of Integer)(numbers))
Else
combinedLines(lineNumber).AddRange(numbers)
End If
Catch ex As Exception
MessageBox.Show("Error Parsing Line: " & line)
End Try
Next
Next
Using sw As New StreamWriter(SaveFileDialog1.FileName, False)
sw.WriteLine(header)
For Each numberSet In combinedLines
Dim numbers = numberSet.Value.Distinct.ToList
numbers.Sort()
sw.WriteLine(numberSet.Key & "," & String.Join(",", numbers))
Next
End Using
End If
End Sub

Replacing multiple lines in a textfile in VB 2017

I'm stuck a little here, what I want to do is read a text file find the lines that contain X= and Y= and change X= to X=1 and Y= to Y=1
So far I got this code
Dim MyPath As String = "C:\Users\kiko4\Desktop\text.txt"
Dim Rdr As New StreamReader(MyPath)
Dim ln As String
Dim NewFile As New StringBuilder
ln = Rdr.ReadLine
Do Until ln Is Nothing
If ln.StartsWith("X=", "Y=") And ComboBox1.Text = "1" Then ln = "X=1" & "Y=1"
NewFile.AppendLine(ln)
ln = Rdr.ReadLine
Loop
Rdr.Close()
File.WriteAllText(MyPath, NewFile.ToString)
MsgBox("Successfully changed the resolution.", MsgBoxStyle.Information)
End Sub
So this results to only changing X and leaving Y like it is, is it possible to change the two lines at the same time?
Try going for a different approach when you're stuck... try this :
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim SR As New StreamReader("C:\Users\kiko4\Desktop\text.txt")
Dim Result As String = SR.ReadToEnd()
SR.Close()
Dim Lines As String() = Result.Split(Environment.NewLine)
Dim TestLine As String
Dim ReplacedLine As String
For Each line As String In Lines
TestLine = line
If TestLine.Contains("X=") Then
ReplacedLine = TestLine.Replace("X=", "X=" & ComboBox1.Text)
Result = Result.Replace(TestLine, ReplacedLine)
ElseIf TestLine.Contains("Y=") Then
ReplacedLine = TestLine.Replace("Y=", "Y=" & ComboBox1.Text)
Result = Result.Replace(TestLine, ReplacedLine)
End If
Next
Dim SW As New StreamWriter("C:\Users\kiko4\Desktop\text.txt")
SW.Write(Result)
SW.Close()
End Sub
Please feel free to ask about anything you didn't understand.
Hope that was useful
Using StreamReaders/Writers is nice and all, and it surely is the best approach
when you are trying to iterate over multiple lines, one by one.
However, in this case this really isn't neccessary, since OP's just trying to replace all occurences within the file, lines do not matter at all.
From a readability/length perspective, something like this would be preferable:
Dim inPath As String = "C:\Users\kiko4\Desktop\text.txt"
Dim trText As String = File.ReadAllText(inPath)
If trText.Contains("X=") AndAlso trText.Contains("Y=") Then File.WriteAllText(inPath, trText.Replace("X=", "X=1").Replace("Y=", "Y=1"))
(all text in the file is read and saved as a single string to trText, if trText contains the string to be replaced write the contents of trText with replaced values to the file)
Or, if you are 100% sure that every file passed will contain the search-values, you can even do it in one Line:
File.WriteAllText("C:\Users\kiko4\Desktop\text.txt", File.ReadAllText("C:\Users\kiko4\Desktop\text.txt").Replace("X=", "X=1").Replace("Y=", "Y=1"))
This one will blow up if the file doesn't contain "X=" AND "Y=".
This is how I fixed it, works as i intended to, which changes both values, thanks everyone for their help :)
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim MyPath As String "C:\Users\kiko4\AppData\Local\FortniteGame\Saved\Config\WindowsClient\text.txt")
Dim Rdr As New StreamReader(MyPath)
Dim ln As String
Dim ac As String
Dim NewFile As New StringBuilder
ln = Rdr.ReadLine
ac = Rdr.ReadLine
Do Until ln Is Nothing
If ln.StartsWith("ResolutionSizeX=") Then ln = "ResolutionSizeX=" + TextBox1.Text
NewFile.AppendLine(ln)
ln = Rdr.ReadLine
If ac.StartsWith("ResolutionSizeY=") Then ac = "ResolutionSizeY=" + TextBox2.Text
NewFile.AppendLine(ac)
ac = Rdr.ReadLine
Loop
Rdr.Close()
File.WriteAllText(MyPath, NewFile.ToString)
MsgBox("Successfully changed the resolution.", MsgBoxStyle.Information)
End Sub
End Class

How to replace multiple consective lines in a line and skip the header for that section

I need to find a section header, in this case "[Store Hours]", in a text file that I'm using to save the settings for the program. I need to skip the header and replace the next 7 lines with the text that is in the text boxes. The code below currently deletes the header "[Store Hours]" and does not replace any of the lines.
Dim objFileName As String = "Settings.txt"
Private Sub BtnAdd_Click(sender As System.Object, e As System.EventArgs) Handles btnSaveHours.Click
Dim OutPutLine As New List(Of String)()
Dim matchFound As Boolean
For Each line As String In System.IO.File.ReadAllLines(objFileName)
matchFound = line.Contains("[Store Hours]")
If matchFound Then
'does not skip the header line
line.Skip(line.Length)
'Need to loop through this 7 times (for each day of the week)
'without reading the header again
For intCount = 0 To 6
Dim aryLabelDay() As String = {"Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"}
Dim varLabelNameIn As String = "txt" & aryLabelDay(intCount).ToString & "In"
Dim varDropNameIn As String = "drp" & aryLabelDay(intCount).ToString & "In"
Dim varLabelNameOut As String = "txt" & aryLabelDay(intCount).ToString & "Out"
Dim varDropNameOut As String = "drp" & aryLabelDay(intCount).ToString & "Out"
Dim varTextBoxInControl() As Control = Me.Controls.Find(varLabelNameIn, True)
Dim varDropBoxInControl() As Control = Me.Controls.Find(varDropNameIn, True)
Dim varTextBoxOutControl() As Control = Me.Controls.Find(varLabelNameOut, True)
Dim varDropBoxOutControl() As Control = Me.Controls.Find(varDropNameOut, True)
Dim dymTextNameIn As TextBox = DirectCast(varTextBoxInControl(0), TextBox)
Dim dymDropNameIn As ComboBox = DirectCast(varDropBoxInControl(0), ComboBox)
Dim dymTextNameOut As TextBox = DirectCast(varTextBoxOutControl(0), TextBox)
Dim dymDropNameOut As ComboBox = DirectCast(varDropBoxOutControl(0), ComboBox)
Dim ReplaceLine As String
ReplaceLine = dymTextNameIn.Text & "," & dymDropNameIn.Text & "," & dymTextNameOut.Text & "," & dymDropNameOut.Text
'this doesn't replace anything
line.Replace(line, ReplaceLine)
intCount += 1
Next intCount
Else
OutPutLine.Add(line)
End If
Next
End Sub
Instead of using ReadAllLines simply use a streamreader and read the file line by line, like this;
Dim line As String
Using reader As StreamReader = New StreamReader("file.txt")
' Read one line from file
line = reader.ReadLine
If(line.Contains("[Store Hours]") Then
'The current line is the store hours header, so we skip it (read the next line)
line = reader.ReadLine
'Process the line like you want, and keep processing through the lines by doing a readline each time you want to progress to the next line.
End If
End Using
More importantly though, you should not be saving the settings for your program in a text file. They should be stored in app.config or web.config. See this question for further guidance on that.
Part of your confusion might be coming from the fact that you can't just replace part of a text file without copying it and overwriting it. One way, to do this, is to copy the file to memory changing the appropriate lines and overwriting the existing file with the new information. Here's one way that can be done:
Private Sub BtnAdd_Click(sender As System.Object, e As System.EventArgs) Handles btnSaveHours.Click
Dim OutPutLine As New List(Of String)()
Dim sr As New StreamReader(objFileName)
While Not sr.EndOfStream
Dim line = sr.ReadLine
'Found the header so let's save that line to memory and add all the other _
info after it.
If line.Contains("[Store Hours]") Then
OutPutLine.Add(line)
'Need to loop through this 7 times (for each day of the week)
'without reading the header again
Dim aryLabelDay() As String = {"Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"}
For intCount = 0 To 6
'even though we're not using the info from the file, calling _
Readline advances the file pointer so that the next iteration _
of the while loop we can finish reading the file.
If Not sr.EndOfStream Then
line = sr.ReadLine
Dim dymTextNameIn As TextBox = DirectCast(Me.Controls("txt" & aryLabelDay(intCount) & "In"), TextBox)
Dim dymDropNameIn As ComboBox = DirectCast(Me.Controls("drp" & aryLabelDay(intCount) & "In"), ComboBox)
Dim dymTextNameOut As TextBox = DirectCast(Me.Controls("txt" & aryLabelDay(intCount) & "Out"), TextBox)
Dim dymDropNameOut As ComboBox = DirectCast(Me.Controls("drp" & aryLabelDay(intCount) & "Out"), ComboBox)
OutPutLine.Add(dymTextNameIn.Text & "," & dymDropNameIn.Text & "," & dymTextNameOut.Text & "," & dymDropNameOut.Text)
End If
Next
Else
'Any line that isn't in that section gets copied as is.
OutPutLine.Add(line)
End If
End While
'Copy all the new info to the same file overwriting the old info.
File.WriteAllLines(objFileName, OutPutLine)
End Sub
On a side note. The Controls collection is indexed by number or name which makes it fairly simple to access the appropriate control just by knowing its name.
Thanks for the help I actually figured it out and this is the final code I used
Private Sub btnSaveHours_Click(sender As System.Object, e As System.EventArgs) Handles btnSaveHours.Click
Dim intOutPutLine As New List(Of String)()
Dim blnSearchString As Boolean
Dim intLineCount As Integer = -1
Dim intLoopCount As Integer
For Each line As String In System.IO.File.ReadAllLines(objFileName)
blnSearchString = line.Contains("[Store Hours]")
If blnSearchString Then
intLineCount = intOutPutLine.Count
line.Remove(0)
intLoopCount = 0
ElseIf intLineCount = intOutPutLine.Count And intLoopCount < 7 Then
line.Length.ToString()
line.Remove(0)
intLoopCount += 1
Else
intOutPutLine.Add(line)
End If
Next
System.IO.File.WriteAllLines(objFileName, intOutPutLine.ToArray())
Dim objFileWrite As New StreamWriter(objFileName, True)
If File.Exists(objFileName) Then
objFileWrite.WriteLine("[Store Hours]")
Dim varMe As Control = Me
Call subConvertFrom12to24Hours(objFileWrite, varMe)
Else
objFileWrite.WriteLine("[Store Hours]")
Dim varMe As Control = Me
Call subConvertFrom12to24Hours(objFileWrite, varMe)
End If
Call btnClear_Click(sender, e)
End Sub

Reading Text line by line to make string manipulation

So to start this is the code I have already written:
Dim MyFile As String = "Path"
Dim str_new As String
Dim str_old As String = File.ReadAllText(MyFile)
Dim sr As New StreamReader(MyFile)
Dim strLines As String() = Strings.Split(sr.ReadToEnd, Environment.NewLine)
Dim Character As Integer = 5 'Line 1 always has 5 characters
For i = 2 To strLines.Length
Dim PreviousLine As String = sr.ReadLine(i - 1)
Dim CurrentLine As String = sr.ReadLine(i)
If CurrentLine.Contains(TextBox1.Text / 100) Then
If PreviousLine.Contains("divide") Then
Exit For
End If
End If
Character = Character + CurrentLine.Length
Next
sr.Close()
str_new = Replace(str_old, (TextBox1.Text / 100), (TextBox3.Text / 100), Character, 1)
Dim objWriter3 As New System.IO.StreamWriter(MyFile, False)
objWriter3.Write(str_new)
objWriter3.Flush()
objWriter3.Close()
I am trying to figure out a way to break a long code file into lines then check each line for certain strings. If the current line contains the string then I will do additional check on above and/or below lines to make sure This is the correct instance of the string. Finally I want to replace just that instance of the string with a different string.
An example: text file
class
...
0.3
divide <-- Previous Line
0.3 <-- TextBox1.Text is 30
.5
end
I want the code to go past the first instance of 0.3
Find the second instance
Check previous line for divide
Exit Loop
Replace second instance of 0.3 to some value
I have been looking into this for a while now and any help would be greatly appreciated!
~Matt
Revised: Code
Dim MyFile As String = "Path"
Dim NewFile As String = "Temporary Path"
Dim PreviousLine As String = ""
Dim CurrentLine As String = ""
Using sr As StreamReader = New StreamReader(MyFile)
Using sw As StreamWriter = New StreamWriter(NewFile)
CurrentLine = sr.ReadLine
Do While (Not CurrentLine Is Nothing)
Dim LinetoWrite = CurrentLine
If CurrentLine.Contains(TextBox1.Text) Then
If PreviousLine.Contains("divide") Then
LinetoWrite = Replace(CurrentLine, TextBox1.Text, TextBox3.Text)
End If
End If
sw.WriteLine(LinetoWrite)
PreviousLine = CurrentLine
CurrentLine = sr.ReadLine
Loop
End Using
End Using
My.Computer.FileSystem.CopyFile(NewFile, MyFile, True)
You are facing the problem in the wrong way. You have to set both, reader and writer, and generate a new file with all the modifications. Your code has various parts which should be improved; in this answer, I am just showing how to use StreamReader/StreamWriter properly. Sample code:
Dim MyFile As String = "input path"
Dim OutputFile As String = "output path"
Dim PreviousLine As String = ""
Dim CurrentLine As String = ""
Using sr As StreamReader = New StreamReader(MyFile)
Using sw As StreamWriter = New StreamWriter(OutputFile)
CurrentLine = sr.ReadLine
Do While (Not CurrentLine Is Nothing)
Dim lineToWrite = CurrentLine
'Perform the analysis you wish by involving the current line, the previous one and as many other (previous) lines as you wish; and store the changes in lineToWrite. You should call a function here to perform this analysis
sw.WriteLine(lineToWrite) 'Writing lineToWrite to the new file
PreviousLine = CurrentLine 'Future previous line
CurrentLine = sr.ReadLine 'Reading the line for the next iteration
Loop
End Using
End Using