Display only certain rows in datatable - vb.net

So I have a dataset like below
Col1 Col2 Col3 Col4
Apple 1 10 Orange
Apple 2 20 Orange
Apple 3 30 Orange
Apple 1 10 Pear
Apple 2 20 Pear
Apple 3 30 Pear
Orange 1 10 grapes
Orange 2 20 grapes
ORange 1 10 kiwi
Berries 1 10 apple
Berries 1 20 Kiwi
I just need something like
Col1 Col2 Col3 Col4
Apple 1 10 Orange
Apple 2 20 Orange
Apple 3 30 Orange
ORange 1 10 Grapes
Orange 2 20 Grapes
Berries 1 10 Apple
So basically it is the col1 and col4, if col4 changes for col1 as I read through the rows, I should'nt display that
Could someone please help me
After first's suggestion
For i As Integer = 0 To dtResults.Rows.Count - 1
Dim firstItem As String = dtResults.Rows(i)("col1").ToString()
Dim firstToB As String = dtResults.Rows(i)("col4").ToString()
dtResults.DefaultView.RowFilter = "col4= '" + firstToB + "'"
Dim tempTable As DataTable = dtResults.DefaultView.ToTable()
Dim Total As Integer = 0
For Each dr As DataRow In tempTable.Rows
'Dim firstItem As String = dr("col1").ToString()
'If (dr("col1") = firstItem) AndAlso (firstToBin = dr("col4")) Then
If item Is Nothing OrElse item <> dr("col1") Then
If Not item Is Nothing Then
dgv.Rows.Add()
End If
itemnum = dr("col1")
Else
itemnum = ""
desc = ""
size = ""
Total += dr("col3")
End If
item = dr("col1")
dgv.Rows.Add(dr('',dr('',dr('')...)
count += 1
'End If
Next
Next

You will need to filter through your DataTable's rows. You can use the DefaultView in order to do this.
For example:
Dim ds1 As New DataSet1 'Create an instance of your DataSet1
ds1.yourDataTable.DefaultView.RowFilter = "Col4='Orange'"
If your DataSet's DataTables are created in the Visual Studio's designer, you can specify the DataType (string, integer, boolean) for each column. By default the DataType is set to a string.
You can do this in two ways
Filtering and Sorting using DataViews
Filtering and Sorting using DataTables

So to achieve as mentioned in the question here is what I did. I can always make changes in Stored procedure but I should be able to see those rows also in dataset for some other purpose. And so to bind the Grid I took row's item like below loop through
For i As Integer = 0 To dtResults.Rows.Count - 1
Dim firstItem As String = dtResults.Rows(i)("col1").ToString()
If acceptitem Is Nothing OrElse acceptitem <> firstItem Then
Dim firstToB As String = dtResults.Rows(i)("col4").ToString()
dtResults.DefaultView.RowFilter = "col4 = '" + firstToB + "' AND col1 = '" + firstItem + "'"
Dim tempTable As DataTable = dtResults.DefaultView.ToTable()
Dim Total As Integer = 0 ' dtResults.Rows(i)("col3").ToString()
For Each dr As DataRow In tempTable.Rows
acceptitem = dr("col1").ToString()
If item Is Nothing OrElse item <> dr("col1") Then
If Not item Is Nothing Then
dgv.Rows.Add()
End If
itemnum = dr("col1")
Total += Convert.ToInt16(dr("col3").ToString())
Else
itemnum = ""
Total += Convert.ToInt16(dr("col3").ToString())
End If
item = dr("col1")
dgv.Rows.Add(dr('',dr('',dr('').......)
count += 1
Next
End If
Next
Thanks so much for all the other suggestions...

Related

Faster way to loop through DataTable elements

Description of the current situation:
I have an excel file of approximately 315 columns and 4000 rows. The file contains the answers to a 300-question questionnaire. The data format is as follows:
(Headers) A | B | C | D | E | F | Q.1 | Q.2 | ... | Q.300 |
(FirstRow) Info of first participant | AnswerCode for every Q |
The columns A to F contain contain info on every participant, while the columns Q.1 to Q.300 contain the respective answer code to each question. After storing the file as a large DataTable:
I need to load all 4000 rows on an existing database table, but before I do that I must edit the data format. The end result must become:
ParticipantCode | QuestionCode | AnswerCode | DateOfRegistration
00001 | 0001 | 1234567 | yyyy-MM-dd HH:mm:ss
... | ... | ... | ...
00001 | 0300 | 1234567 | yyyy-MM-dd HH:mm:ss
00002 | 0001 | 1234567 | yyyy-MM-dd HH:mm:ss
... | ... | ... | ...
04000 | 0300 | 1234567 | yyyy-MM-dd HH:mm:ss
So every row of the original ExcelDataTable is transformed into 300 rows in the FinalDataTable. In this way, the FinalDataTable will have about 1.2 million rows.
What Have I implemented so far:
Private Function MyFunction()
For Each ExcelRow As DataRow In ExcelDataTable.Rows
For Each ExcelColumn As DataColumn In ExcelDataTable.Columns
QuestionCodeFound = False
ExcelColumnNameRaw = ExcelColumn.ColumnName.ToString.Trim
If ExcelColumnNameRaw.StartsWith("Q") Then
' Correct the headers
ExcelColumnSplit = ExcelColumnNameRaw.Split("#")
ExcelColumnName = String.Concat(ExcelColumnSplit(0), ExcelColumnSplit(1))
SelectedRowFromDT = QuestionCodeAndQuestionIDDataTable.Select("QuestionID = '" + ExcelColumnName + "'")
' Search for "_", because some questions are different
If SelectedRowFromDT.Length > 0 Then
QuestionCodeFound = True
Else
Dim ExcelColumnSplitForMult As String()
ExcelColumnSplitForMult = ExcelColumnName.Split("_")
SelectedRowFromDT = QuestionCodeAndQuestionIDDataTable.Select("QuestionID = '" + ExcelColumnSplitForMult(0).ToString + "'")
If SelectedRowFromDT.Length > 0 Then
QuestionCodeFound = True
End If
End If
If QuestionCodeFound Then
Dim QuestionCode As String
Dim QuestionTypeDataTable As DataTable
Dim QuestionType As String
' Get the Question Type from the respective table
QuestionType = String.Empty
QuestionCode = SelectedRowFromDT(0).Item("QuestionCode").ToString
QuestionTypeDataTable = SearchInSql(My.Settings.ConnectionString, SQLString)
If QuestionTypeDataTable.Rows.Count > 0 Then
QuestionType = QuestionTypeDataTable.Rows(0).Item(0).ToString.Trim
End If
' Fix the Date Format
DateRaw = ExcelRow.Item(1).ToString
DateSplit = DateRaw.Split("/")
If DateSplit(0).Length = 1 Then
DateSplit(0) = String.Concat("0", DateSplit(0))
End If
If DateSplit(1).Length = 1 Then
DateSplit(1) = String.Concat("0", DateSplit(1))
End If
DateText = String.Concat(DateSplit(0), "/", DateSplit(1), "/", DateSplit(2))
DateRegistration = DateTime.ParseExact(DateText, "MM/dd/yyyy", CultureInfo.InvariantCulture)
DateRegistrationReformed = DateRegistration.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)
DateRegFinal = DateTime.ParseExact((DateRegistrationReformed + " " + "10:00:00").ToString, "yyyy-MM-dd HH:mm:ss", CultureInfo.InvariantCulture)
Dim AnswerValue As String
Dim AnswerCode As String
Dim AnswerCodeDataTable As DataTable
Dim QuestionWasAnswer As String
Dim AnswerValueRow() As DataRow = ExcelDataTable.Select("ParticipantCode = '" + ExcelRow.Item(2).ToString + "'")
AnswerCodeDataTable = New DataTable
AnswerValue = ""
QuestionWasAnswer = "0"
' Complete "QuestionWasAnswer" field for all questions and retrieve the AnswerCode for the answer given by each participant
If AnswerValueRow.Length > 0 And AnswerValueRow(0).Item(ExcelColumnNameRaw).GetType IsNot GetType(DBNull) Then
If Not (QuestionType.Equals("02") Or QuestionType.Equals("03")) Then
AnswerValue = AnswerValueRow(0).Item(ExcelColumnNameRaw)
QuestionWasAnswer = "1"
ElseIf QuestionType.Equals("02") Or QuestionType.Equals("03") Then
Dim ExcelColumnSplitForMultSecond As String()
Dim MultAnswerValue As String
ExcelColumnSplitForMultSecond = ExcelColumnName.Split("_")
MultAnswerValue = AnswerValueRow(0).Item(ExcelColumnNameRaw).ToString.Trim
AnswerValue = ExcelColumnSplitForMultSecond(1).ToString
If MultAnswerValue.Equals("1") Then
QuestionWasAnswer = "1"
ElseIf MultAnswerValue.Equals("2") Then
QuestionWasAnswer = "2"
End If
End If
' Search in the Answers table for the existing AnswerCode
SQLString = String.Format("SELECT Answers.AnswerCode
FROM Answers
WHERE Answers.QuestionCode = '{0}'
AND (Answers.AnswerNumber = '{1}' OR Answers.Answer = '{1}')", QuestionCode, AnswerValue)
AnswerCodeDataTable = SearchInSql(My.Settings.ConnectionString, SQLString)
If AnswerCodeDataTable.Rows.Count > 0 Then
AnswerCode = AnswerCodeDataTable.Rows(0).Item(0).ToString
FormattedDataTable.Rows.Add(ParticipantAnswerCode, ExcelRow.Item(2), QuestionCode, AnswerCode, QuestionWasAnswer, DateRegFinal)
ParticipantAnswerCode = Convert.ToInt32(ParticipantAnswerCode + 1).ToString.PadLeft(ParticipantAnswerCodeFieldLength, "0")
Else
' If a given answer does not exist, save it in the respective table and then try again
Dim AnswerCodeLength = GetLengthFromSqlDataBase(My.Settings.ConnectionString, "Answers", "AnswerCode")
Dim NextAnswerCode = CalculateNextAnswerCode(AnswerCodeLength)
Dim NestAnswerNumber = CalculateNextAnswerNumber(QuestionCode)
SaveNewAnswer(NextAnswerCode, QuestionCode, NestAnswerNumber, AnswerValue)
SQLString = String.Format("SELECT Answers.AnswerCode
FROM Answers
WHERE Answers.QuestionCode = '{0}'
AND Answers.Answer = '{1}'", QuestionCode, AnswerValue)
AnswerCodeDataTable = SearchInSql(My.Settings.ConnectionString, SQLString)
If AnswerCodeDataTable.Rows.Count > 0 Then
AnswerCode = AnswerCodeDataTable.Rows(0).Item(0).ToString
FormattedDataTable.Rows.Add(ParticipantAnswerCode, ExcelRow.Item(2), QuestionCode, AnswerCode, QuestionWasAnswer, DateRegFinal)
ParticipantAnswerCode = Convert.ToInt32(ParticipantAnswerCode + 1).ToString.PadLeft(ParticipantAnswerCodeFieldLength, "0")
End If
End If
End If
End If
End If
Next
Next
Return FormattedDataTable
End Function
After that, I bulk insert the FinalDataTable on the DB.
The problem I am facing:
Using the current program I built, every row in the ExcelDataTable takes about 40 seconds to transform into 300 rows in the FinalDataTable. If I try to load all 4000 rows, it will take more than 40 hours to transform the entire datatable. I need to find a faster way to do this.
As mentioned, there isn't much to go off of on this with what has been provided.
I'm sure there are more helpful fixes to consider but I wanted to put my two cents in about the For Loops.
I recommend switching the
For Each
statements with
For i as integer = 0 to ExcelDataTable.Rows.Count - 1
I've read that For Each is not as performance-friendly as it gathers each "row" as a collection, therefore increasing the overhead per loop.
Here is a SO post about this subject:
Major difference between 'for each' and 'for' loop in .NET
Not sure if that will make a difference for you but thought I would recommend it anyway.

Remove duplicates from DataGridView but keep the count?

I have a datagridview with data I pull from an SQL Server. It comes back with some data that are duplicates. I want to remove all the duplicates but keep count of how many there are of each unique items. Here is an example of what the data looks like...
For intI = DataGridView1.Rows.Count - 1 To 0 Step -1
For intJ = intI - 1 To 0 Step -1
If DataGridView1.Rows(intI).Cells(1).Value =
DataGridView1.Rows(intJ).Cells(1).Value AndAlso
DataGridView1.Rows(intI).Cells(3).Value =
DataGridView1.Rows(intJ).Cells(3).Value Then
DataGridView1.Rows.RemoveAt(intI)
Exit For
End If
Next
Next
So I can remove all the duplicates, but I want to be able to have a count of all the items in the end, including the duplicates. For example, There are 5 CA, I want to remove 4 leaving just 1 unique one, but I want to show that there are 5 CA in my datagridview next to California.
So in the end, I want the datagridview to have something like :
State | Short
California | 5
Here is my Query:
DECLARE #StartDate DATETIME
DECLARE #EndDate DATETIME
SELECT
States , Shorts
FROM
DAT.States
INNER JOIN
DATR.[Shorts]
WHERE #StartDate IS NULL
Thank you for your help!
EDIT:
I came up with a method of counting the GridView1 into GridView2 but I'm not exactly sure on the code. Maybe I could get some help with this part instead?
For i As Integer = DataGridView1.RowCount - 1 To 1 Step -1
For j As Integer = DataGridView2.RowCount - 1 To 1 Step -1
'If Grid2.Row(j).Cells(1).Value = Grid1.Rows(i).Cells(1).Value Then
' Grid2 already has that value, +1 to Column 2 of Cell(j)
' Delete row(i)
'Else
'Value does not exist, add row(i) into Grid2, and in the 2nd column of Grid2, the count is 1
'End If
Next
Next
The end result for GridView2 would be something like :
Here is my code I came up with to try and count the States in Column 1 and then delete it and add it to the Grid2, but it's not working...
Dim Counts As Integer = 0
Dim State As String = ""
Dim CurLoop As Integer = 0
Do Until CurLoop > DataGridView1.RowCount - 1
State = DataGridView1.Rows(CurLoop).Cells(0).Value
For i As Integer = DataGridView1.RowCount - 1 To 1 Step -1
If DataGridView1.Rows(i).Cells(0).Value = State Then
Counts += 1
DataGridView1.Rows.RemoveAt(i)
End If
Next
DataGridView2.Rows.Add(State, Counts)
State = ""
Counts = 0
CurLoop += 1
Loop
I have figured out the answer to my own question. It's not the prettiest solution but it works. If anyone else is in need of something like this, here is what I did...
Dim Counts As Integer = 0
Dim State As String = ""
Dim CurLoop As Integer = 0
Do Until CurLoop > DataGridView1.RowCount - 1
State = DataGridView1.Rows(CurLoop).Cells(0).Value
For i As Integer = DataGridView1.RowCount - 1 To 1 Step -1
If DataGridView1.Rows(i).Cells(0).Value = State Then
Counts += 1
DataGridView1.Rows.RemoveAt(i)
End If
Next
DataGridView2.Rows.Add(State, Counts)
State = ""
Counts = 0
CurLoop += 1
Loop
Counts = DataGridView2.Rows(0).Cells(1).Value
Counts += 1
DataGridView2.Rows(0).Cells(1).Value = Counts
For some reason, whatever in the first row of Grid1 is, it's always 1 lower than what it's supposed to be, so I add 1 to it in the end.

datatable sum column and concatenate rows using LINQ and group by on multiple columns

I Have a datatable with following records
ID NAME VALUE CONTENT
1 AAA 10 SYS, LKE
2 BBB 20 NOM
1 AAA 15 BST
3 CCC 30 DSR
2 BBB 05 EFG
I want to write a VB.NET/LINQ query to have a output like below table: -
ID NAME SUM CONTENT (as CSV)
1 AAA 25 SYS, LKE, BST
2 BBB 25 NOM, EFG
3 CCC 30 DSR
Please provide me LINQ query to get the desired result. Thanks.
I have tried concatenation using below query
Dim grouped = From row In dtTgt.AsEnumerable() _
Group row By New With {row.Field(Of Int16)("ID"), row.Field(Of String)("Name")} _
Into grp() _
Select ID, Name, CONTENT= String.Join(",", From i In grp Select i.Field(Of String)("CONTENT"))
This query will give you the expected output:-
Dim result = From row In dt.AsEnumerable()
Group row By _group = New With {Key .Id = row.Field(Of Integer)("Id"),
Key .Name = row.Field(Of String)("Name")} Into g = Group
Select New With {Key .Id = _group.Id, Key .Name = _group.Name,
Key .Sum = g.Sum(Function(x) x.Field(Of Integer)("Value")),
Key .Content = String.Join(",", g.Select(Function(x) x.Field(Of String)("Content")))}
Thanks for your answers.
However, I have managed to get the desired result using simple code (Without LINQ): -
Dim dt2 As New DataTable
dt2 = dt.Clone()
For Each dRow As DataRow In dt.Rows
Dim iID As Integer = dRow("ID")
Dim sName As String = dRow("Name")
Dim sContt As String = dRow("Content")
Dim iValue As Integer = dRow("Value")
Dim rwTgt() As DataRow = dt2.Select("ID=" & iID)
If rwTgt.Length > 0 Then
rwTgt(0)("Value") += iValue
rwTgt(0)("Content") += ", " & sContt
Else
rw = dt2.NewRow()
rw("ID") = iID
rw("Name") = sName
rw("Value") = iValue
rw("Content") = sContt
dt2.Rows.Add(rw)
End If
Next

Cross table VB.NET & SQL Server & Linq

I have a table like this:
MAName feldtext
------------------
karl fieldtext1
karl fieldtext2
karl fieldtext1
karl fieldtext3
karl fieldtext4
karl fieldtext2
karl fieldtext5
karl fieldtext3
karl fieldtext3
susi fieldtext1
susi fieldtext4
john fieldtext2
john fieldtext5
john fieldtext5
and I need:
MAName fieldtext1 fieldtext2 fieldtext3 fieldtext4 fieldtext5 FehlerJeMA
karl 2 2 3 1 1 9
susi 1 0 0 1 0 2
john 0 1 0 0 2 3
The columns fieldtext can go from fieldtext1 to fieldtextn, it's dynamic, depending on query.
I was looking here for solutions and found, so my approach:
Dim dt2 As New DataTable
Dim nn As Integer = 0
Dim Zeile As DataRow
dt2.Columns.Add("MAName")
' fieldtext distinct
Dim query2 = (From dr In (From d In newTable2.AsEnumerable Select New With {.feldtext1 = d("feldtext")}) Select dr.feldtext1 Distinct)
For Each Feldtext In query2
dt2.Columns.Add(Feldtext)
Next
column = New DataColumn()
column.DataType = System.Type.GetType("System.Int32")
column.ColumnName = "FehlerJeMA"
dt2.Columns.Add(column)
' MAName distinct
Dim query3 = (From dr In (From d In newTable2.AsEnumerable Select New With {.MAName2 = d("MAName")}) Select dr.MAName2.ToString.ToLower Distinct)
For Each Mitarbeiter In query3
Zeile = dt2.NewRow()
Zeile(0) = Mitarbeiter.ToString.ToLower
MA2 = Mitarbeiter.ToString.ToLower
nn = 1
For Each colName2 In query2
Fehler2 = colName2
Dim AnzahlFehler As String = (From row In newTable2.Rows Select row Where row("MAName").ToString.ToLower = MA2 And row("feldtext") = Fehler2).Count
If AnzahlFehler = 0 Then
AnzahlFehler = ""
End If
Zeile(nn) = AnzahlFehler
nn += 1
If AnzahlFehler <> "" Then
FehlerJeMA += CInt(AnzahlFehler)
End If
Next
Zeile(nn) = FehlerJeMA
dt2.Rows.Add(Zeile)
Next
This works, but is very slow...
It could be the case that in my table has more than 10.000 rows...
So my question is: what is fastest approach to get the result?
Is it some kind of cross table with linq? Other approaches?
In C# you will be able to use the code, try to translate it for your problem:
var pivotData = data.GroupBy(x => new {x.MAName, x.feldtext}, (key, group) => new { MAName = key.Column1, feldtext = key.Column2, count = group.Count() });

Is it possible to peek at the next value in a DataTable Row?

I have a DataTable that contains 4 rows. I want to compare a value in one column in row 1 with the value in row 2. Something similar to this:
For Each row As DataRow in drRows
If row("column") <> row("column") 'I want the second row("column") to be the next row.
'do something else
End If
Next
You keep track of the last item:
Dim last As DataRow = Nothing
For Each row As DataRow In drRows
If last IsNot Nothing Then
' Compare last with row
End If
last = row
Next
You can always access a DataRow with its index (DataRowCollection.Item):
For i As Integer = 0 To tbl.Rows.Count - 1
Dim row As DataRow = tbl.Rows(i)
If i <> tbl.Rows.Count - 1 Then
Dim nextRow As DataRow = tbl(i + 1)
If row("column").Equals(nextRow("column")) Then
'do something"
End If
End If
Next
Say you have the following table:
A B C D
1 2 3 4
5 6 7 8
9 10 11 12
12 12 13 14
For i As Integer = 0 To dt.Rows.Count - 2
If dt.Rows(i)("ColName") <> dt.Rows(i + 1)("ColName") Then
'Do something
End If
Next