VB: Count number of columns in csv - vb.net

So, quite simple.
I am importing CSVs into a datagrid, though the csv always has to have a variable amount of columns.
For 3 Columns, I use this code:
Dim sr As New IO.StreamReader("E:\test.txt")
Dim dt As New DataTable
Dim newline() As String = sr.ReadLine.Split(";"c)
dt.Columns.AddRange({New DataColumn(newline(0)), _
New DataColumn(newline(1)), _
New DataColumn(newline(2))})
While (Not sr.EndOfStream)
newline = sr.ReadLine.Split(";"c)
Dim newrow As DataRow = dt.NewRow
newrow.ItemArray = {newline(0), newline(1), newline(2)}
dt.Rows.Add(newrow)
End While
DG1.DataSource = dt
This works perfectly. But how do I count the number of "newline"s ?
Can I issue a count on the number of newlines somehow? Any other example code doesn't issue column heads.
If my csv file has 5 columns, I would need an Addrange of 5 instead of 3 and so on..
Thanks in advance

Dim sr As New IO.StreamReader(path)
Dim dt As New DataTable
Dim newline() As String = sr.ReadLine.Split(","c)
' MsgBox(newline.Count)
' dt.Columns.AddRange({New DataColumn(newline(0)),
' New DataColumn(newline(1)),
' New DataColumn(newline(2))})
Dim i As Integer
For i = 0 To newline.Count - 1
dt.Columns.AddRange({New DataColumn(newline(i))})
Next
While (Not sr.EndOfStream)
newline = sr.ReadLine.Split(","c)
Dim newrow As DataRow = dt.NewRow
newrow.ItemArray = {newline(0), newline(1)}
dt.Rows.Add(newrow)
End While
dgv.DataSource = dt
End Sub

Columns and item values can be added to a DataTable individually, using dt.Columns.Add and newrow.Item, so that these can be done in a loop instead of hard-coding for a specific number of columns. e.g. (this code assumes Option Infer On, so adjust as needed):
Public Function CsvToDataTable(csvName As String, Optional delimiter As Char = ","c) As DataTable
Dim dt = New DataTable()
For Each line In File.ReadLines(csvName)
If dt.Columns.Count = 0 Then
For Each part In line.Split({delimiter})
dt.Columns.Add(New DataColumn(part))
Next
Else
Dim row = dt.NewRow()
Dim parts = line.Split({delimiter})
For i = 0 To parts.Length - 1
row(i) = parts(i)
Next
dt.Rows.Add(row)
End If
Next
Return dt
End Function
You could then use it like:
Dim dt = CsvToDataTable("E:\test.txt", ";"c)
DG1.DataSource = dt

Related

how to Flip dataset and display in datagridview

I try to flip dataset to display column as rows by using this code but it does not work :
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button.Click
Dim ds2 As New DataSet
Dim dt2 As New DataTable
Dim com1 As String = "select col1,col2,col3 from table1"
ds2 = FlipDataSet(ds2)
Dim dp As New SqlDataAdapter(com1, conn)
dp.Fill(dt2)
DGV_lev1.DataSource = dt2.DefaultView
End Sub
and use this function to flip dataset :
Private Function FlipDataSet(old_DataSet As DataSet) As DataSet
Dim ds As New DataSet()
For Each dt As DataTable In old_DataSet.Tables
Dim table As New DataTable()
For i As Integer = 0 To dt.Rows.Count
table.Columns.Add(Convert.ToString(i))
table.Columns(0).ColumnName = "Fields"
If i = 0 Then
Continue For
Else
table.Columns(i).ColumnName = "Customer " & i
End If
Next
Dim r As DataRow
For k As Integer = 0 To dt.Columns.Count - 1
r = table.NewRow()
r(0) = dt.Columns(k).ToString()
For j As Integer = 1 To dt.Rows.Count
r(j) = dt.Rows(j - 1)(k)
Next
table.Rows.Add(r)
Next
ds.Tables.Add(table)
Next
Return ds
End Function
to make datagirdview display from this :
to this :
can anyone help me
thank you
Try this, it worked in a quick test I did:
Private Function Transpose(ByVal table As DataTable) As DataTable
Dim flippedTable As New DataTable
'creates as many columns as rows in source table
flippedTable.Columns.AddRange(
table.Select.Select(
Function(dr) New DataColumn("col" & table.Rows.IndexOf(dr), GetType(Object))
).ToArray)
'iterates columns in source table
For Each dc As DataColumn In table.Columns
'get array of values of column in each row and add as new row in target table
flippedTable.Rows.Add(table.Select.Select(Function(dr) dr(dc)).ToArray)
Next
Return flippedTable
End Function

Insert headers of a datatable VB.NET

I need your help for my code vb
In fact, i created an new datatable and i want to copy the headers from another datatable
here is my code
Dim name(de.Tables(0).Columns.Count) As String
Dim p As Integer = 0
For Each column As DataColumn In de.Tables(0).Columns
name(p) = column.ColumnName
p += 1
Next
Dim m As Integer = 0
For m = 0 To de.Tables(0).Columns.Count - 1
dt.Columns(m).ColumnName = name(p)
Next
If you only want to "copy" the schema of a DataTable(so the columns and constraints) without it's content(DataRows) you can use DataTable.Clone:
Dim clonedTable As DataTable = originalTable.Clone()
If you also want to copy the DataRows you have to use DataTable.Copy.
Try this one
Dim dt As New DataTable()
Dim name(de.Tables(0).Columns.Count) As String
For Each column As DataColumn In de.Tables(0).Columns
dt.Columns.Add(New DataColumn(column.ColumnName))
Next

Performance improvement on vb.net code

I need to write 50 million records with 72 columns into text file, the file size is growing as 9.7gb .
I need to check each and every column length need to format as according to the length as defined in XML file.
Reading records from oracle one by one and checking the format and writing into text file.
To write 5 crores records it is taking more than 24 hours. how to increase the performance in the below code.
Dim valString As String = Nothing
Dim valName As String = Nothing
Dim valLength As String = Nothing
Dim valDataType As String = Nothing
Dim validationsArray As ArrayList = GetValidations(Directory.GetCurrentDirectory() + "\ReportFormat.xml")
Console.WriteLine("passed xml")
Dim k As Integer = 1
Try
Console.WriteLine(System.DateTime.Now())
Dim selectSql As String = "select * from table where
" record_date >= To_Date('01-01-2014','DD-MM-YYYY') and record_date <= To_Date('31-12-2014','DD-MM-YYYY')"
Dim dataTable As New DataTable
Dim oracleAccess As New OracleConnection(System.Configuration.ConfigurationManager.AppSettings("OracleConnection"))
Dim cmd As New OracleCommand()
cmd.Connection = oracleAccess
cmd.CommandType = CommandType.Text
cmd.CommandText = selectSql
oracleAccess.Open()
Dim Tablecolumns As New DataTable()
Using oracleAccess
Using writer = New StreamWriter(Directory.GetCurrentDirectory() + "\FileName.txt")
Using odr As OracleDataReader = cmd.ExecuteReader()
Dim sbHeaderData As New StringBuilder
For i As Integer = 0 To odr.FieldCount - 1
sbHeaderData.Append(odr.GetName(i))
sbHeaderData.Append("|")
Next
writer.WriteLine(sbHeaderData)
While odr.Read()
Dim sbColumnData As New StringBuilder
Dim values(odr.FieldCount - 1) As Object
Dim fieldCount As Integer = odr.GetValues(values)
For i As Integer = 0 To fieldCount - 1
Dim vals As Array = validationsArray(i).ToString.ToUpper.Split("|")
valName = vals(0).trim
valDataType = vals(1).trim
valLength = vals(2).trim
Select Case valDataType
Case "VARCHAR2"
If values(i).ToString().Length = valLength Then
sbColumnData.Append(values(i).ToString())
'sbColumnData.Append("|")
ElseIf values(i).ToString().Length > valLength Then
sbColumnData.Append(values(i).ToString().Substring(0, valLength))
'sbColumnData.Append("|")
Else
sbColumnData.Append(values(i).ToString().PadRight(valLength))
'sbColumnData.Append("|")
End If
Case "NUMERIC"
valLength = valLength.Substring(0, valLength.IndexOf(","))
If values(i).ToString().Length = valLength Then
sbColumnData.Append(values(i).ToString())
'sbColumnData.Append("|")
Else
sbColumnData.Append(values(i).ToString().PadLeft(valLength, "0"c))
'sbColumnData.Append("|")
End If
'sbColumnData.Append((values(i).ToString()))
End Select
Next
writer.WriteLine(sbColumnData)
k = k + 1
Console.WriteLine(k)
End While
End Using
writer.WriteLine(System.DateTime.Now())
End Using
End Using
Console.WriteLine(System.DateTime.Now())
'Dim Adpt As New OracleDataAdapter(selectSql, oracleAccess)
'Adpt.Fill(dataTable)
Return Tablecolumns
Catch ex As Exception
Console.WriteLine(System.DateTime.Now())
Console.WriteLine("Error: " & ex.Message)
Console.ReadLine()
Return Nothing
End Try

Trimming a datagridview duplicat rows except the recent one

i'm in VS2008 Studio, i have this datagridview with multiple columns which the last column contains a date and time value.
lot's of rows are pretty the same except by they're date column.
what i wanted to do is to trim the whole datagridview duplicate rows except they're most recent ones based on they're date column.
i have sth like this:
Administrator,192.168.137.221,2,file://C:\WMPub\WMRoot\industrial.wmv , 07.Jul.2014 - 23:11:59
Administrator,192.168.137.221,2,file://C:\WMPub\WMRoot\industrial.wmv , 07.Jul.2014 -
21:11:59
Administrator,192.168.137.221,2,file://C:\WMPub\WMRoot\industrial.wmv , 07.Jul.2014 - 22:11:59
Administrator,192.168.137.221,2,file://C:\WMPub\WMRoot\industrial.wmv , 07.Jul.2014 - 20:11:59
Administrator,192.168.137.221,2,file://C:\WMPub\WMRoot\industrial.wmv , 07.Jul.2014 - 11:11:59
Everyone ,192.168.137.221,2,file://C:\WMPub\WMRoot\industrial.wmv , 07.Jul.2014 - 17:11:59
Everyone ,192.168.137.221,2,file://C:\WMPub\WMRoot\industrial.wmv , 07.Jul.2014 - 14:11:59
the output i want should be like this:
Administrator 192.168.137.221 2 file://C:\WMPub\WMRoot\industrial.wmv 07.Jul.2014 - 23:11:59
Everyone 192.168.137.201 2 file://C:\WMPub\WMRoot\industrial.wmv 07.Jul.2014 - 17:11:59
....
please consider "," as column seprators! (i dont know how to draw a table here, sorry again)!
i have this snippet that trim the duplicate lines in a datagridview but it lacks preserving the latest entry:
Public Function RemoveDuplicateRows(ByVal dTable As DataTable, ByVal colName As String) As DataTable
Dim hTable As New Hashtable()
Dim duplicateList As New ArrayList()
For Each dtRow As DataRow In dTable.Rows
If hTable.Contains(dtRow(colName)) Then
duplicateList.Add(dtRow)
Else
hTable.Add(dtRow(colName), String.Empty)
End If
Next
For Each dtRow As DataRow In duplicateList
dTable.Rows.Remove(dtRow)
Next
Return dTable
End Function
what should i do?
thanks in advance
Here is some code that illustrates the approach:
Dim dict As New dictionary(Of String, DataRow)
For Each dtRow As DataRow In dTable.Rows
Dim key As String = dtRow("column1") + "," + dtRow("column2") ' + etc.
Dim dictRow As DataRow = Nothing
If dict.TryGetValue(key, dictRow) Then
'check and update date
'you can skip this part, if your data is sorted
If dtRow("dateColumn") > dictRow("dateColumn") Then
dictRow("dateColumn") = dtRow("dateColumn")
End If
Else
dict.Add(key, dtRow)
End If
Next
In the end dict contains the rows you need, you can get them via dict.Values.ToArray()
EDIT: I found the error - dictRow should be dtRow in the above code (now fixed). Then it should work. Here is a full version of self contained example (console app), since I wrote it anyway - focus on RemoveDuplicates, the rest is just prepwork:
Sub Main()
Dim dt As New DataTable
With dt.Columns
.Add("PublishingPoint")
.Add("Username")
.Add("IP")
.Add("Status")
.Add("Req URL")
.Add("Last seen", GetType(Date))
End With
'this populates the initial data table, use your method
Dim _assembly As Assembly = Assembly.GetExecutingAssembly()
Dim _textStreamReader As New StreamReader(_assembly.GetManifestResourceStream("ConsoleApplication16.data.csv"))
While Not _textStreamReader.EndOfStream
Dim sLine As String = _textStreamReader.ReadLine().TrimEnd
If String.IsNullOrEmpty(sLine) Then Exit While
Dim values() As String = sLine.Split(",")
Dim newRow As DataRow = dt.NewRow
For iColumnIndex As Integer = 0 To dt.Columns.Count - 1
Dim columnName As String = dt.Columns(iColumnIndex).ColumnName
newRow.Item(columnName) = values(iColumnIndex)
Next
dt.Rows.Add(newRow)
End While
Console.WriteLine("Old count: " & dt.Rows.Count)
Dim newDt As DataTable = RemoveDuplicates(dt, "Last seen")
Console.WriteLine("New count: " & newDt.Rows.Count)
Console.ReadLine()
End Sub
Private Function RemoveDuplicates(dt As DataTable, colName As String) As DataTable
Dim keyColumnNames As New List(Of String)
Dim exceptColumnsHash As New HashSet(Of String)({colName})
For Each col As DataColumn In dt.Columns
Dim columnName As String = col.ColumnName
If Not exceptColumnsHash.Contains(col.ColumnName) Then
keyColumnNames.Add(columnName)
End If
Next
Dim dict As New Dictionary(Of String, DataRow)
For Each dtRow As DataRow In dt.Rows
Dim keyColumnValues As New List(Of String)
For Each keyColumnName In keyColumnNames
keyColumnValues.Add(dtRow.Item(keyColumnName))
Next
Dim key As String = String.Join(",", keyColumnValues)
Dim dictRow As DataRow = Nothing
If dict.TryGetValue(key, dictRow) Then
If dtRow(colName) > dictRow(colName) Then
dictRow(colName) = dtRow(colName)
End If
Else
dict.Add(key, dtRow)
End If
Next
Dim dtReturn As DataTable = dt.Clone
For Each dtRow As DataRow In dict.Values
dtReturn.ImportRow(dtRow)
Next
Return dtReturn
End Function
To make this code run, you need to manually add a file to the project and set build action to "Embedded resource".

Stored list of array using For Each loop

I want to store the "zone_check_value" to a array of string then while inserting into array i will check the array of string if the next value is repeated or have duplicate.
Example.
1st Example
1st loop = zone_check_value = ZD1/01/2014
2nd loop = zone_check_value = ZD1/01/2014
2nd Example
1st loop = zone_check_value = ZD1/01/2014
2nd loop = zone_check_value = ZD2/02/2014
3rd loop = zone_check_value = ZD1/01/2014
Code:
For Each dt As DataTable In xls.Tables
Dim array_of_string as String() 'i want to put the value in here
For Each dr As DataRow In dt.Rows
Dim zone_destination As String = dr(2).ToString
Dim affected_date As String = dr(7).ToString
Dim zone_check_value = zone_destination + affected_date
''''''How can i store zone_check_value in a string array?
Next
Next
EDIT
What if i add select case in my loop? the array_of_string value become NULL. I need to check the value of the current value in array_of_string .
Example Code
For Each dt As DataTable In xls.Tables
Dim array_of_string as String() 'i want to put the value in here
select Case dt.tablename
case "Sheet1"
For Each dr As DataRow In dt.Rows
Dim zone_destination As String = dr(2).ToString
Dim affected_date As String = dr(7).ToString
Dim zone_check_value = zone_destination + affected_date
''''''How can i store zone_check_value in a string array?
Next
case "Sheet 2"
For Each dr As DataRow In dt.Rows
Dim check_value as Boolean = array_of_string.Contains(dr(0).ToString)
'but when i got in sheet 2 the array_of_string is null
Next
Next
The easiest way is to use a List(Of String).
For Each dt As DataTable In xls.Tables
Dim array_of_string as List(Of String) = New List(Of String) 'i want to put the value in here
For Each dr As DataRow In dt.Rows
Dim zone_destination As String = dr(2).ToString
Dim affected_date As String = dr(7).ToString
Dim zone_check_value = zone_destination & affected_date
''''''How can i store zone_check_value in a string array?
array_of_string.Add(zone_check_value)
Next
''' Now if you really need it in array form you can cast it via:
''' Dim values() As String = array_of_string.ToArray()
Next
You might even consider:
For Each dt As DataTable In xls.Tables
Dim values As List(Of String) = New List(Of String)
dt.Rows.ForEach( Sub(item) values.Add(item(2).ToString & item(7).ToString) )
''' Now do something with values.
Next
Either way, make sure you always use the string concatenation operator & to concatenate strings. The arithmetic addition operator + will cause you problems from time to time if you use it on strings.