How can I ignore a new line character when reading CSV file in VB.NET? - vb.net

Question

I wrote a utility in VB.NET that reads an input CSV file, does some processing (specifically it ignores the first 5 lines of the input file and replaces them with a header row saved in another file) and writes the information from the input file into a new output CSV file.
Where my program fails is when the input data includes new line characters within one column value within the CSV.
I would like to ignore the new line character within a CSV data row when I load it into my string array.
Here is my code (its embedded in a form)
Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click
Dim incsvPath = strFileName
Dim outcsvPath = fi.DirectoryName & "\" & outfilename
Dim headerPath = fi.DirectoryName & "\ACTIVITY_HISTORY_HEADER.csv"
Dim fileP As String = incsvPath
Dim fileheader As String = headerPath
Dim CSVheaderIn As New ArrayList
Dim CSVlinesIn As New ArrayList
Dim CSVout As New List(Of String)
CSVheaderIn.AddRange(IO.File.ReadAllLines(fileheader))
CSVlinesIn.AddRange(IO.File.ReadAllLines(fileP))
messageTB.AppendText(vbCrLf & vbCrLf)
For Each line As String In CSVheaderIn
Dim nameANDnumber As String() = line.Split(",")
messageTB.AppendText("csv file header row = " & line & vbCrLf & vbCrLf & "csv file contents follow ..." & vbCrLf)
CSVout.Add(line)
Next
Dim mySubAL As ArrayList = CSVlinesIn.GetRange(5, CSVlinesIn.Count - 5)
For Each line As String In mySubAL 'CSVlinesIn
messageTB.AppendText(line & vbCrLf)
CSVout.Add(line)
Next
IO.File.WriteAllLines(outcsvPath, CSVout.ToArray)
End Sub

Answer 1

This is fairly hard work actually; it'll be easier to use a library that knows how to read and write CSV with newlines in the data than roll your own - not saying you couldn't, but it's a wheel that has already been invented so why do it again?
I used Steve Hansen's Csv - right click your project in solution explorer, choose Manage Nuget Packages, click Browse, Search csv, install the right one
Imports System.Text
Imports Csv
Imports System.IO
Module Module1
Sub Main(args As String())
'open the headers file
Using hIn = File.OpenText("C:\temp\h.csv")
'setup instruction to the csv reader with headersabsent flag so we can get the first line as data
Dim hOptions = New CsvOptions With {.HeaderMode = HeaderMode.HeaderAbsent}
'take the first line into an array - these are our headers
Dim headers = CsvReader.Read(hIn, hOptions)(0).Values
'open the data file,
Using fIn = File.OpenText("C:\temp\a.csv")
'setup instruction for the reader to skip 5 rows, treat first row as data, and allow newlines in quoted fields
Dim fOptions = New CsvOptions With {.RowsToSkip = 5, .HeaderMode = HeaderMode.HeaderAbsent, .AllowNewLineInEnclosedFieldValues = True}
Using fOut = File.CreateText("C:\temp\a_out.csv")
'convert the ICsvLine rows in the reader to rows of String() that the writer will accept, and write them under the headers
CsvWriter.Write(fOut, headers, CsvReader.Read(fIn, fOptions).Select(Function(line) line.Values))
End Using
End Using
End Using
End Sub
End Module
You don't have to use this lib to read the headers; you could just file.ReadText().ReadLine().Split(","c) it
If you want to perform per-line processing on the elements, do this:
CsvWriter.Write(fOut, headers, CsvReader.Read(fIn, fOptions).Select(Function(line) ProcessLine(line.Values)))
...
Function ProcessLine(input As String()) As String()
'Note: If(input(8), "") returns input(8) unless it is nothing in which case "" is returned instead
If If(input(8), "").Length > 10 Then input(8) = input(8).Remove(10) 'Trim if over 10
If If(input(14), "").Length > 10 Then input(14) = input(14).Remove(10)
Return input 'Always return
End Function