I convert CSV file to DataTable using following code. It is working fine but sometimes it adds invalid character between the string.
Private Shared Function convertToDT(filePath As String) As DataTable
Dim detailDataTable As New DataTable()
Using csvReader As New TextFieldParser(filePath)
csvReader.SetDelimiters(New String() {","})
csvReader.HasFieldsEnclosedInQuotes = True
Dim colFields As String() = csvReader.ReadFields()
For Each column As String In colFields
Dim datecolumn As New DataColumn(column.ToLowerInvariant)
datecolumn.AllowDBNull = True
detailDataTable.Columns.Add(datecolumn)
Next
While Not csvReader.EndOfData
Dim rowData As String() = csvReader.ReadFields //sometimes rowData convert invalid name
For i As Integer = 0 To rowData.Length - 1
//perform operation
Next
If rowData IsNot Nothing AndAlso rowData(0) IsNot Nothing Then
If Not (rowData(0).StartsWith("#") Or String.IsNullOrEmpty(rowData(0))) Then
detailDataTable.Rows.Add(rowData)
End If
End If
End While
End Using
Return detailDataTable
End Function
Sometime it add invalid name. Can anybody please suggest
Related
please how can I get count of duplicate lines?
Source data: line e.g. user_id;name;surname;3400;44711;30.05.2022 7:00:00;30.05.2022 15:30:00;0;480;0;1;682;10000120;9
Private Sub remove_duplicite(sender As Object, e As EventArgs)
Dim sFiles() As String
sFiles = Directory.GetFiles(filesPath1, remove_dupl)
Dim path As String = String.Join("", sFiles)
'MessageBox.Show(path)
Dim lines As New HashSet(Of String)()
'Read to file
Using sr As StreamReader = New StreamReader(path)
Do While sr.Peek() >= 0
lines.Add(sr.ReadLine())
Loop
End Using
'Write to file
Using sw As StreamWriter = New StreamWriter(path)
For Each line As String In lines
sw.WriteLine(line)
Next
End Using
Close()
End Sub
I try some answers but no success.But I think that will be easy.
Thank you
Dim sList As New List(of String)
sList.Add("1")
sList.Add("2")
sList.Add("2")
sList.Add("3")
Dim sListDistinct As List(Of String) = sList.Distinct().ToList()
Dim iCount as Integer = sList.Count - sListDistinct.Count
But depending on the size of your file, this isn't the best performance way.
Maybe check in your HashSet with .Contains and count if entry already exists
ask permission,
I created a bot to input data to the web using vb.net and selenium.
Retrieve data from csv .
How to retrieve data from csv as needed, for example, there are 100 rows, only 30-50 rows are taken, for example. The loop code should not be looped at all.
Dim textFieldParser As TextFieldParser = New TextFieldParser(TextBox1.Text) With
{
.TextFieldType = FieldType.Delimited,
.Delimiters = New String() {","}
}
drv = New ChromeDriver(options)
While Not textFieldParser.EndOfData
Try
Dim strArrays As String() = textFieldParser.ReadFields()
Dim name As String = strArrays(0)
Dim alamat As String = strArrays(1)
Dim notlp As String = strArrays(2)
drv.Navigate().GoToUrl("URL")
Dim Nm = drv.FindElement(By.XPath("/html/body/div[1]/div[3]/form/div[1]/div[1]/div[1]/div/div[2]/input"))
Nm.SendKeys(name)
Threading.Thread.Sleep(3000)
Catch ex As Exception
MsgBox("Line " & ex.Message & "is not valid and will be skipped.")
End Try
End While
Thank you
Here's an example of using TextFieldParser to read one specific line and a specific range of lines. Note that I am using zero-based indexes for the lines. You can adjust as required if you want to use 1-based line numbers.
Public Function GetLine(filePath As String, index As Integer) As String()
Using parser As New TextFieldParser(filePath) With {.Delimiters = {","}}
Dim linesDiscarded = 0
Do Until linesDiscarded = index
parser.ReadLine()
linesDiscarded += 1
Loop
Return parser.ReadFields()
End Using
End Function
Public Function GetLines(filePath As String, startIndex As Integer, count As Integer) As List(Of String())
Using parser As New TextFieldParser(filePath) With {.Delimiters = {","}}
Dim linesDiscarded = 0
Do Until linesDiscarded = startIndex
parser.ReadLine()
linesDiscarded += 1
Loop
Dim lines As New List(Of String())
Do Until lines.Count = count
lines.Add(parser.ReadFields())
Loop
Return lines
End Using
End Function
Simple loops to skip and to take lines.
I've adapted the code from the #tim-schmelter answer to question convert csv data to DataTable in VB.net (see below)
I would like to parse in the column titles from row 0 of the csv file
DT|Meter Number|Customer Account Number|Serial Number|Port...
but I'm not having any luck trying to figure out how to do this. any suggestions would be very appreciated.
Public Function csvToDatatable_2(ByVal filename As String, ByVal separator As String)
'////////////////////////////////////////
'Reads a selected txt or csv file into a datatable
'based on code from http://stackoverflow.com/questions/11118678/convert-csv-data-to-datatable-in-vb-net
'////////////////////////////////////////
Dim dt As System.Data.DataTable
Try
dt = New System.Data.DataTable
Dim lines = IO.File.ReadAllLines(filename)
Dim colCount = lines.First.Split(separator).Length
For i As Int32 = 1 To colCount
dt.Columns.Add(New DataColumn("Column_" & i, GetType(String)))
Next
For Each line In lines
Dim objFields = From field In line.Split(separator)
Dim newRow = dt.Rows.Add()
newRow.ItemArray = objFields.ToArray()
Next
Catch ex As Exception
Main.Msg2User(ex.Message.ToString)
Return Nothing
End Try
Return dt
End Function
Just loop thru all the line of the file. Use a boolean to check for the first row.
Public Function csvToDatatable_2(ByVal filename As String, ByVal separator As String)
Dim dt As New System.Data.DataTable
Dim firstLine As Boolean = True
If IO.File.Exists(filename) Then
Using sr As New StreamReader(filename)
While Not sr.EndOfStream
If firstLine Then
firstLine = False
Dim cols = sr.ReadLine.Split(separator)
For Each col In cols
dt.Columns.Add(New DataColumn(col, GetType(String)))
Next
Else
Dim data() As String = sr.Readline.Split(separator)
dt.Rows.Add(data.ToArray)
End If
End While
End Using
End If
Return dt
End Function
Here is a hybrid of the two solutions above, with a few other changes:
Public Shared Function FileToTable(ByVal fileName As String, ByVal separator As String, isFirstRowHeader As Boolean) As DataTable
Dim result As DataTable = Nothing
Try
If Not System.IO.File.Exists(fileName) Then Throw New ArgumentException("fileName", String.Format("The file does not exist : {0}", fileName))
Dim dt As New System.Data.DataTable
Dim isFirstLine As Boolean = True
Using sr As New System.IO.StreamReader(fileName)
While Not sr.EndOfStream
Dim data() As String = sr.ReadLine.Split(separator, StringSplitOptions.None)
If isFirstLine Then
If isFirstRowHeader Then
For Each columnName As String In data
dt.Columns.Add(New DataColumn(columnName, GetType(String)))
Next
isFirstLine = True ' Signal that this row is NOT to be considered as data.
Else
For i As Integer = 1 To data.Length
dt.Columns.Add(New DataColumn(String.Format("Column_{0}", i), GetType(String)))
Next
isFirstLine = False ' Signal that this row IS to be considered as data.
End If
End If
If Not isFirstLine Then
dt.Rows.Add(data.ToArray)
End If
isFirstLine = False ' All subsequent lines shall be considered as data.
End While
End Using
Catch ex As Exception
Throw New Exception(String.Format("{0}.CSVToDatatable Error", GetType(Table).FullName), ex)
End Try
Return result
End Function
I'm trying to populate a List using a DataTable, I have a for each loop that checks every row and adds the item to the list. But the code isn't working, I keep getting the error..
System.NullReferenceException: {"Object reference not set to an
instance of an object."}
-Data: {System.Collections.ListDictionaryInternal}
-HelpLink: Nothing -Inner Exception: Nothing
-TargetSite: {System.Collections.Generic.List`1[System.String] getListOfUsers()}
This is my code...
Function getListOfUsers() As List(Of String)
'Dim i As Integer = 0
Dim lUserNames As List(Of String) = Nothing
Dim dt As DataTable = getDataTable(db_Config, "SELECT * FROM tblUsers")
If dt.Rows.Count > 0 Then
Try
For Each dRowItem As DataRow In dt.Rows
'i = i + 1
'If IsDBNull(dt.Rows(0)("fldUserName").ToString) = False Then
' lUserNames.Add(dt.Rows(0)("fldUserName").ToString)
'End If
If dRowItem.Item("fldUserName").ToString <> "" Then
lUserNames.Add(dRowItem.Item("fldUserName").ToString)
End If
Next dRowItem
Catch ex As Exception
MsgBox(ex.ToString)
End Try
End If
Return lUserNames
End Function
Currently lUserNames has not been initialized. You need to do this:
Dim lUserNames As New List(Of String)
You have declared the List, but you haven't initialized it.
So change
Dim lUserNames As List(Of String) = Nothing
to
Dim lUserNames As List(Of String) = New List(Of String)
Of course you would also get a NullReferenceException if getDataTable returns Nothing instead of an empty DataTable if tblUsers would be empty at If dt.Rows.Count > 0.
You're not initializing your List on line 4. It should be:
Dim lUserNames As List(Of String) = New List(Of String)()
Edit: apparently this is a bit of a CSharp'ism :) This also works:
Dim lUserNames As New List(Of String)
I have the following piece of code that I am using to try and rip a csv file and turn it into a datatable. My problem is that the debugger never makes it to the return statement. Everything is appending to the datatable correctly, so I know that part works. Any idea's on what I can do to trouble shoot this further. Also, if you know of a simpler way to turn a import a csv file to a datatable I'd be very interested in learning about it.
Thanks!
Public Function loadCSVTableII() As DataTable
Dim dt As New DataTable("TableII")
Dim line As String = String.Empty
Dim counter As Integer = 0
Dim reader As New StreamReader(pathTableTwo)
Try
While Not IsNothing(line)
line = reader.ReadLine()
Dim lineSep As String() = line.Split(New Char() {","c})
If Not counter = 0 Then
dt.Rows.Add(lineSep)
counter += 1
Else
For Each value As String In lineSep
dt.Columns.Add(value)
Next
counter += 1
End If
End While
'cursor never gets to this code block...
Dim primarykey(0) As DataColumn
primarykey(0) = dt.Columns("Ages")
dt.PrimaryKey = primarykey
Return dt
Catch ex As Exception
Throw
End Try
End Function
Update: It is erroring out on this line in the code.
Dim lineSep As String() = line.Split(New Char() {","c})
It say that the Object reference is not set to an instance of an object. What's weird though is that it works through the whole data table fine. Could it be that the while loop is not terminating at the end of the file?
Try changing your While loop to handle the end of stream condition. It's not very clear what the IsNothing function is doing in your code.
While Not reader.EndOfStream
line = reader.ReadLine
'// Dim lineSep As String() = line.Split(New Char() {","c})
For your line split, in VB.Net, it's simple to just do this:
Dim lineSep As String() = line.Split(",")
You can use OLEDB provider for this.
string query = "SELECT Symbol, [Name of Company], FROM [just file name with extension]";
string connStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + [csv file path without file name] + ";" + "Extended Properties=’text;HDR=YES;’";
//create dataadapter object
OleDbDataAdapter adapter = new OleDbDataAdapter(query, connStr);
// create table
DataTable dtSymbolDetails = new DataTable("ScriptDetails");
dtSymbolDetails.Columns.Add("Symbol");
dtSymbolDetails.Columns.Add("Name of Company");
// fill the table with data using adapter
adapter.Fill(dtDetails);