I have a scenario, in which I have to apply distinct filter onto DataTable and find the rows only which are distinct,
I am using dt.DefaultView.ToTable(True, Columns) this statement but no effect.
Here is my chunk of code..
Try
Dim dTable As New DataTable()
dTable.Columns.Add("AutoID")
dTable.Columns.Add("AnotherID")
dTable.Columns.Add("CitY")
Dim row As DataRow = Nothing
For i As Integer = 0 To 4
row = dTable.NewRow()
row("AutoID") = i + 1
row("AnotherID") = i + 10
row("City") = "Vetican"
dTable.Rows.Add(row)
Next
dTable.Rows.Add(6, "11", "Oslo")
dTable.Rows.Add(7, "12", "Toronto")
Dim TobeDistinct As String() = {"AnotherID"}
Dim dtDistinct As DataTable = GetDistinctRecords(dTable, TobeDistinct)
Catch ex As Exception
End Try
and the method ..
Public Shared Function GetDistinctRecords(ByVal dt As DataTable, ByVal Columns As String()) As DataTable
Dim dtURecords As New DataTable()
dtURecords = dt.DefaultView.ToTable(True, Columns)
Return dtURecords
End Function
Here is the screen shot , which I want..
Which rows do you want to keep and which rows should be removed? If you just want to keep one row per AnotherID it seems to be arbitrary to keep Vetican instead of Oslo. Maybe you want to concat both as in Vetican, Oslo.
I would use Linq instead:
Dim resultTable = dTable.Clone() ' empty table same columns
Dim idGroups = dTable.AsEnumerable().GroupBy(Function(r) r.Field(Of String)("AnotherID"))
For Each grp In idGroups
Dim r As DataRow = resultTable.Rows.Add()
r.SetField("AutoID", grp.First().Field(Of String)("AutoID"))
r.SetField("AnotherID", grp.Key)
Dim cities = From row In grp Select row.Field(Of String)("City")
r.SetField("City", String.Join(", ", cities))
Next
I need to write 50 million records with 72 columns into text file, the file size is growing as 9.7gb .
I need to check each and every column length need to format as according to the length as defined in XML file.
Reading records from oracle one by one and checking the format and writing into text file.
To write 5 crores records it is taking more than 24 hours. how to increase the performance in the below code.
Dim valString As String = Nothing
Dim valName As String = Nothing
Dim valLength As String = Nothing
Dim valDataType As String = Nothing
Dim validationsArray As ArrayList = GetValidations(Directory.GetCurrentDirectory() + "\ReportFormat.xml")
Console.WriteLine("passed xml")
Dim k As Integer = 1
Try
Console.WriteLine(System.DateTime.Now())
Dim selectSql As String = "select * from table where
" record_date >= To_Date('01-01-2014','DD-MM-YYYY') and record_date <= To_Date('31-12-2014','DD-MM-YYYY')"
Dim dataTable As New DataTable
Dim oracleAccess As New OracleConnection(System.Configuration.ConfigurationManager.AppSettings("OracleConnection"))
Dim cmd As New OracleCommand()
cmd.Connection = oracleAccess
cmd.CommandType = CommandType.Text
cmd.CommandText = selectSql
oracleAccess.Open()
Dim Tablecolumns As New DataTable()
Using oracleAccess
Using writer = New StreamWriter(Directory.GetCurrentDirectory() + "\FileName.txt")
Using odr As OracleDataReader = cmd.ExecuteReader()
Dim sbHeaderData As New StringBuilder
For i As Integer = 0 To odr.FieldCount - 1
sbHeaderData.Append(odr.GetName(i))
sbHeaderData.Append("|")
Next
writer.WriteLine(sbHeaderData)
While odr.Read()
Dim sbColumnData As New StringBuilder
Dim values(odr.FieldCount - 1) As Object
Dim fieldCount As Integer = odr.GetValues(values)
For i As Integer = 0 To fieldCount - 1
Dim vals As Array = validationsArray(i).ToString.ToUpper.Split("|")
valName = vals(0).trim
valDataType = vals(1).trim
valLength = vals(2).trim
Select Case valDataType
Case "VARCHAR2"
If values(i).ToString().Length = valLength Then
sbColumnData.Append(values(i).ToString())
'sbColumnData.Append("|")
ElseIf values(i).ToString().Length > valLength Then
sbColumnData.Append(values(i).ToString().Substring(0, valLength))
'sbColumnData.Append("|")
Else
sbColumnData.Append(values(i).ToString().PadRight(valLength))
'sbColumnData.Append("|")
End If
Case "NUMERIC"
valLength = valLength.Substring(0, valLength.IndexOf(","))
If values(i).ToString().Length = valLength Then
sbColumnData.Append(values(i).ToString())
'sbColumnData.Append("|")
Else
sbColumnData.Append(values(i).ToString().PadLeft(valLength, "0"c))
'sbColumnData.Append("|")
End If
'sbColumnData.Append((values(i).ToString()))
End Select
Next
writer.WriteLine(sbColumnData)
k = k + 1
Console.WriteLine(k)
End While
End Using
writer.WriteLine(System.DateTime.Now())
End Using
End Using
Console.WriteLine(System.DateTime.Now())
'Dim Adpt As New OracleDataAdapter(selectSql, oracleAccess)
'Adpt.Fill(dataTable)
Return Tablecolumns
Catch ex As Exception
Console.WriteLine(System.DateTime.Now())
Console.WriteLine("Error: " & ex.Message)
Console.ReadLine()
Return Nothing
End Try
So, quite simple.
I am importing CSVs into a datagrid, though the csv always has to have a variable amount of columns.
For 3 Columns, I use this code:
Dim sr As New IO.StreamReader("E:\test.txt")
Dim dt As New DataTable
Dim newline() As String = sr.ReadLine.Split(";"c)
dt.Columns.AddRange({New DataColumn(newline(0)), _
New DataColumn(newline(1)), _
New DataColumn(newline(2))})
While (Not sr.EndOfStream)
newline = sr.ReadLine.Split(";"c)
Dim newrow As DataRow = dt.NewRow
newrow.ItemArray = {newline(0), newline(1), newline(2)}
dt.Rows.Add(newrow)
End While
DG1.DataSource = dt
This works perfectly. But how do I count the number of "newline"s ?
Can I issue a count on the number of newlines somehow? Any other example code doesn't issue column heads.
If my csv file has 5 columns, I would need an Addrange of 5 instead of 3 and so on..
Thanks in advance
Dim sr As New IO.StreamReader(path)
Dim dt As New DataTable
Dim newline() As String = sr.ReadLine.Split(","c)
' MsgBox(newline.Count)
' dt.Columns.AddRange({New DataColumn(newline(0)),
' New DataColumn(newline(1)),
' New DataColumn(newline(2))})
Dim i As Integer
For i = 0 To newline.Count - 1
dt.Columns.AddRange({New DataColumn(newline(i))})
Next
While (Not sr.EndOfStream)
newline = sr.ReadLine.Split(","c)
Dim newrow As DataRow = dt.NewRow
newrow.ItemArray = {newline(0), newline(1)}
dt.Rows.Add(newrow)
End While
dgv.DataSource = dt
End Sub
Columns and item values can be added to a DataTable individually, using dt.Columns.Add and newrow.Item, so that these can be done in a loop instead of hard-coding for a specific number of columns. e.g. (this code assumes Option Infer On, so adjust as needed):
Public Function CsvToDataTable(csvName As String, Optional delimiter As Char = ","c) As DataTable
Dim dt = New DataTable()
For Each line In File.ReadLines(csvName)
If dt.Columns.Count = 0 Then
For Each part In line.Split({delimiter})
dt.Columns.Add(New DataColumn(part))
Next
Else
Dim row = dt.NewRow()
Dim parts = line.Split({delimiter})
For i = 0 To parts.Length - 1
row(i) = parts(i)
Next
dt.Rows.Add(row)
End If
Next
Return dt
End Function
You could then use it like:
Dim dt = CsvToDataTable("E:\test.txt", ";"c)
DG1.DataSource = dt
Dim dt As DataTable = catheter.FetchCatheter()
Dim ct As New EntityObjects.CatheterTypeBO
Dim cList As New List(Of EntityObjects.CatheterTypeBO)
For i As Integer = 0 To dt.Rows.Count - 1
ct.ID = dt.Rows(i)("ID")
ct.Type = dt.Rows(i)("CTYPE")
ct.Active = dt.Rows(i)("ACTIVE").ToString()
cList.Add(ct)
Next
updating all index of cList by last row of dt. Need to convert datatable to List
Try to use AsEnumerable then Last() like:
Dim dt As DataTable = catheter.FetchCatheter()
Dim ct As New EntityObjects.CatheterTypeBO
Dim cList As New List(Of EntityObjects.CatheterTypeBO)
Dim dr as DataRow = dt.AsEnumerable().Last()
ct.ID = dr("ID")
ct.Type = dr("CTYPE")
ct.Active = dr("ACTIVE").ToString()
cList.Add(ct)
I have a datatable like this
X,Y,Z
0,0,A
0,2,B
0,0,C
1,0,A
1,0,C
2,2,A
2,2,B
2,0,C
3,2,B
3,1,C
4,3,A
4,0,B
4,1,C
5,3,A
5,2,B
5,0,C
and I want to convert it to something like this:
X,A,B,C
0,0,2,0
1,0, ,0
2,2,2,0
3, ,2,1
4,3,0,1
5,3,2,0
I tried with dataset and linq but not I wasn't lucky.
My code for linq:
Dim q = (From c In dt _
Select c("Z") Distinct) 'I found out which categories I have in Z column (my example :A,B,C)
Dim ldt(q.Count) As DataTable
For i = 0 To q.Count - 1
Dim sfil As String = q(i).ToString
Dim r = (From c In dt _
Select c Where c("Z") = sfil)
ldt(i) = r.CopyToDataTable
Next
So now I have 3 tables (ldt(0) with values for A, ldt(1) with values for B, ldt(2) with values for C)
and I was thinking to do something like leftJoin but anything that I tried is fail.
Any solution or even a better idea?
Thanks
So a new example it would be:
I have this table:
id,Price,Item
0,0,Laptop
0,2,Tablet
0,0,Cellphone
1,0,Laptop
1,0,Tablet
2,2,Laptop
2,2,Cellphone
2,0,Tablet
3,2,Cellphone
3,1,Tablet
4,3,Laptop
4,0,Cellphone
4,1,Tablet
5,3,Laptop
5,2,Cellphone
5,0,Tablet
and I would like to convert it to this:
X,Laptop,Tablet,Cellphone
0,0,2,0
1,0, ,0
2,2,2,0
3, ,2,1
4,3,0,1
5,3,2,0
The values for each of the columns Laptop, Tablet, Cellphone are the Y values from the first table.
I hope it make more sense now.
I believe you can create a DataTable with column names corresponding to the item names. Then you group the previous DataTable by id and use each grouping to populate a row. Forgive me if I get anything wrong. I don't work with VB or DataTables that much.
Dim itemNames = (From c In dt _
Select c("Item") Distinct)
Dim newDt as DataTable = new DataTable()
Dim idColumn As DataColumn = new DataColumn()
idColumn.DataType = System.Type.GetType("System.Int32")
idColumn.ColumnName = "id"
idColumn.ReadOnly = True
idColumn.Unique = True
newDt.Columns.Add(idColumn)
For Each itemName As String In itemNames
Dim column As DataColumn = new DataColumn()
column.DataType = GetType(Nullable(Of Integer))
column.ColumnName = itemName
column.ReadOnly = True
column.Unique = False
newDt.Columns.Add(column)
Next
Dim groupingById = From row in dt
Group By Id = row("id")
Into RowsForId = Group
For Each grouping In groupingById
Dim row as DataRow = newDt.NewRow()
row("id") = grouping.Id
For Each rowForId in grouping.RowsForId
row(rowForId("Item")) = rowForId("Price")
Next
newDt.Rows.Add(row)
Next