(VB.NET) System.IO.IOException: The process cannot access the file - vb.net

Question

I am fetching about 600 000 rows into a DataGridView (through SQL) and splitting them all into files by the date.
I tried several ways to do this, and while I found some ways to make it not crash and do it safely and securely, it's EXTREMELY slow.
The fastest way, however, I found to be simply cutting the date only out of the Date column, e.g. turning 2020/06/01 13:50:43 into 0601, and using that as the file name and then just append text to that file. (As opposed to either creating all the rows in a temp object and writing it all at once etc).
It's fast and everything is good, however after writing about, maybe 30? out of about 150 files, it crashes. It's very random, I tried running it a couple of times. It could crash after a few seconds or after half a minute. Hence, I believe it's caused by it tripping itself when trying to write so many times so fast.
I have tried changing the method to write several times, making sure to use Using blocks, as well as trying file share.
Here's my code:
Private Sub btnExec_Click(sender As Object, e As EventArgs) Handles btnExec.Click
Static start_time As DateTime
Static stop_time As DateTime
Dim elapsed_time As TimeSpan
start_time = Now
If IsNumeric(TextBox1.Text) Then columnNumber = CInt(TextBox1.Text) - 1
Dim dgw = Form1.DataGridView1
Dim curcell As String = ""
Dim ToWrite As String = ""
Dim Written As Integer = 0
Try
ProgressBar1.Maximum = dgw.Rows.Count
For i = 0 To dgw.Rows.Count - 1
'For each row
If IsDBNull(dgw.Rows(i).Cells(columnNumber).Value.ToString()) Then
curcell = "" 'if null replace with a blank value
Else
curcell = CStr(dgw.Rows(i).Cells(columnNumber).Value)
End If
'RB FORMATTING
If RadioButton1.Checked = True Then '2020/01/01 00:00:00
'2020/06/16 19:38:33
curcell = curcell.Remove(0, 5)
curcell = curcell.Remove(curcell.Length - 9, 9)
curcell = curcell.Replace("/", "")
'0616
End If
'Now we have 0616 in a string which is the date of the current row
Dim currow As String = ""
For Each c As DataGridViewCell In dgw.Rows(i).Cells
If IsDBNull(c.Value) Then
currow += Chr(34) + "" + Chr(34) + ","
Else
currow += Chr(34) + CStr(c.Value) + Chr(34) + ","
End If
Next
currow = currow.Remove(currow.Length - 1, 1) 'Remove last comma
'Now we have the full row to print as well as the date in a string
Dim FILENAME As String = TextBox2.Text + "\" + curcell + ".csv"
' Using fs As FileStream = File.OpenWrite(FILENAME)
' AddText(fs, currow + vbCrLf)
' End Using
Dim fs As FileStream = New FileStream(FILENAME, FileMode.OpenOrCreate, FileAccess.Write)
Dim w As StreamWriter = New StreamWriter(fs)
w.BaseStream.Seek(0, SeekOrigin.[End])
w.Write(currow + vbCrLf)
w.Close()
'Using writer As StreamWriter = File.AppendText(FILENAME)
'writer.WriteLine(currow)
' End Using
'My.Computer.FileSystem.WriteAllText(FILENAME, currow + vbCrLf, 1) 'Gets used by its own process error some times
Written += 1
Next 'Next row
stop_time = Now
elapsed_time = stop_time.Subtract(start_time)
Log("Wrote " + CStr(Written) + " lines in " + elapsed_time.TotalSeconds.ToString("0.00" & " seconds."))
Catch ex As Exception
Log(ex.ToString())
End Try
End Sub
Here's the error:
System.IO.IOException: The process cannot access the file 'C:\Users\username\Desktop\TEST2\0416.csv' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access)
at TicketsSQL.frmSplitFiles.btnExec_Click(Object sender, EventArgs e) in C:\Users\username\source\repos\TicketsSQL\TicketsSQL\frmSplitFiles.vb:line 63
I commented out two other ways than the current one that I tried in the code. I tried the following three ways:
Dim fs As FileStream = New FileStream(FILENAME, FileMode.OpenOrCreate, FileAccess.Write)
Dim w As StreamWriter = New StreamWriter(fs)
w.BaseStream.Seek(0, SeekOrigin.[End])
w.Write(currow + vbCrLf)
w.Close()
Using writer As StreamWriter = File.AppendText(FILENAME)
writer.WriteLine(currow)
End Using
My.Computer.FileSystem.WriteAllText(FILENAME, currow + vbCrLf, 1)
It seems like no matter what way I try to prevent it using an already open file, or even allowing it to do so... it trips anyway.
Anyone have any work arounds or ways to solve this problem?

Answer 1

Try this and let us know what errors happen
Static start_time As DateTime
Static stop_time As DateTime
Dim elapsed_time As TimeSpan
start_time = Now
If IsNumeric(TextBox1.Text) Then columnNumber = CInt(TextBox1.Text) - 1
Dim dgw = Form1.DataGridView1
Dim curcell As String = ""
Dim ToWrite As String = ""
Dim Written As Integer = 0
Try
ProgressBar1.Maximum = dgw.Rows.Count
For i = 0 To dgw.Rows.Count - 1
'For each row
If IsDBNull(dgw.Rows(i).Cells(columnNumber).Value.ToString()) Then
curcell = "" 'if null replace with a blank value
Else
curcell = CStr(dgw.Rows(i).Cells(columnNumber).Value)
End If
'RB FORMATTING
If RadioButton1.Checked = True Then '2020/01/01 00:00:00
'2020/06/16 19:38:33
curcell = curcell.Remove(0, 5)
curcell = curcell.Remove(curcell.Length - 9, 9)
curcell = curcell.Replace("/", "")
'0616
End If
'Now we have 0616 in a string which is the date of the current row
Dim currow As String = ""
For Each c As DataGridViewCell In dgw.Rows(i).Cells
If IsDBNull(c.Value) Then
currow += Chr(34) + "" + Chr(34) + ","
Else
currow += Chr(34) + CStr(c.Value) + Chr(34) + ","
End If
Next
currow = currow.Remove(currow.Length - 1, 1) 'Remove last comma
'>>>>>>>>>>>>>>>>>>>>>>>>>
'Now we have the full row to print as well as the date in a string
Dim FILENAME As String
FILENAME = IO.Path.Combine(TextBox2.Text, curcell)
FILENAME = IO.Path.ChangeExtension(FILENAME, ".csv")
Try
IO.File.WriteAllText(FILENAME, currow & ControlChars.CrLf)
Catch ex As Exception
Stop ' what is the error <<<<<<<<<<<<<<<<<<<<<<
End Try
'<<<<<<<<<<<<<<<<<<<<<<<<<
Written += 1
Next 'Next row
stop_time = Now
elapsed_time = stop_time.Subtract(start_time)
Log("Wrote " + CStr(Written) + " lines in " + elapsed_time.TotalSeconds.ToString("0.00" & " seconds."))
Catch ex As Exception
Log(ex.ToString())
End Try