Sorting a DataTable using LINQ, when sort-by-columns may vary - vb.net

I need to sort DataTables, however the sort-by-columns vary.
Scenario #1, DataTable1 should be sorted by "Column1".
Scenario #2, DataTable2 should be sorted by "Column1, Column2".
Below is my first attempt at creating a helper function for this purpose. This works ok.
Private Sub SortDataTable(ByRef dataTable As DataTable, ByVal sortColumnNames As List(Of String))
'Validation (not shown here)
Dim sortOrder = String.Join(", ", sortColumnNames)
dataTable.DefaultView.Sort = sortOrder
dataTable = dataTable.DefaultView.Table
End Sub
I tried implementing this in LINQ, however, I don't know how to pass multiple sort-by-columns to the lambda function. Work-in-progress code shown below.
Private Sub SortDataTable(ByRef dataTable As DataTable, ByVal sortColumnNames As List(Of String))
'Validation (not shown here)
dataTable.AsEnumerable().OrderBy(Function (row) row(sortColumnNames(0))).ThenBy(...)
End Sub
How should I pass multiple sort-by-columns to the OrderBy/ThenBy extension methods?

Something like that:
Private Function SortDataTable(table As DataTable, ParamArray columns As String()) As DataTable
If columns.Length = 0 Then
Return table
End If
firstColumn = columns.First()
Dim result = table.AsEnumerable().OrderBy(Function(r) r(firstColumn))
For Each columnName As var In columns.Skip(1)
result = result.ThenBy(Function(r) r(columnName))
Next
Return result.AsDataView().ToTable()
End Function
Converted from this C# code ( I've written this in C# and then used http://www.developerfusion.com/tools/convert/csharp-to-vb/ ):
DataTable SortDataTable(DataTable table, params string[] columns)
{
if (columns.Length == 0)
{
return table;
}
firstColumn = columns.First();
var result = table.AsEnumerable().OrderBy(r => r[firstColumn]);
foreach (var columnName in columns.Skip(1))
{
result = result.ThenBy(r => r[columnName]);
}
return result.AsDataView().ToTable();
}
PS: didn't test that. But that's very simple, so should be no problems.

Related

VB.NET Index out of Range exception related to text file

I have some code I have used many times over which has always worked great for me. The latest use, however, throws an exception under certain circumstances that I cannot seem to resolve. Here it is:
I read from a text file to an array, use it as a binding source for some of my controls (it autofills 3 controls based on the selection of a single control). I created a Student class with 4 properties (Name, ID, DOB and DOE). Here is the code I use:
Private Sub autoFill()
Dim rost As String = "Roster.txt"
Dim lines As List(Of String) = File.ReadAllLines(rost).ToList
Dim list As List(Of Student) = New List(Of Student)
For i As Integer = 0 To lines.Count - 1
Dim data As String() = lines(i).Split(":")
list.Add(New Student() With {
.StudentName = data(0),
.StudentID = data(1),
.StudentDOB = data(2),
.StudentDOE = data(3)
})
Next
StudentBindingSource.DataSource = list
End Sub
Now here is the problem. In the "For" loop when I set i to 0 to lines.count -1 it throws this error:
VB>NET EXCEPTION
However...If I change i to 1 instead of 0 it works OR if I take away data(2) and data(3) it works with i = 0. I would prefer to use 0 so that I can have a blank line in the combobox or "--choose--", etc. The only thing I have thought that might be useful is that my first row in the text file has nothing to split. Here is the line format of the text file:
Student Name ID# DOB DOE <-----This header row is NOT in the text file
Last Name, First Name : 0000000 : 01/01/2021 : 01/01/2021
I'm going to assume I'm missing something really simple here. Any guidance would be greatly appreciated! Thank you.
Before we get to the actual problem, let's re-work some things.
A better way to structure code, especially when working with data loading, is to have a method that accepts an input and returns a result. Additionally, calling ToList() or ToArray() is a very expensive operation for performance. Very often you can improve performance dramatically by working with a lower-level IEnumerable for as long as possible.
With those principles in mind, consider this code:
Private Function ReadStudentData(fileName As String) As IEnumerable(Of Student)
Dim lines As IEnumerable(Of String) = File.ReadLines(fileName)
Return lines.
Select(Function(line) line.Split(":")).
Select(Function(data)
Return New Student() With {
.StudentName = data(0),
.StudentID = data(1),
.StudentDOB = data(2),
.StudentDOE = data(3)
}
End Function)
End Function
Private Sub autoFill()
StudentBindingSource.DataSource = ReadStudentData("Roster.txt")
End Sub
Now on to the actual issue. The problem was not from looping through the list variable. The problem is the data array. At some point you have a line that doesn't have enough elements. This is common, for example, as the last line in a file.
There are many ways to address this. In some cases, the exception is already the appropriate result, because if you have bad data you really don't want to continue. In other cases you want to log the bad records, perhaps to a report you can easily review later. Or maybe you just want to ignore the error, or pre-filter for rows with the right number of columns. Here is an example of the last option:
Private Function ReadStudentData(fileName As String) As IEnumerable(Of Student)
Return File.ReadLines(fileName).
Select(Function(line) line.Split(":")).
Where(Function(data) data.Length = 4).
Select(Function(data)
Return New Student() With {
.StudentName = data(0),
.StudentID = data(1),
.StudentDOB = data(2),
.StudentDOE = data(3)
}
End Function)
End Function
Private Sub autoFill()
StudentBindingSource.DataSource = ReadStudentData("Roster.txt")
End Sub
The problem is that you didn't check 'data' to have enough elements to create the 'Student'. A simple check should fix it.
Private Sub autoFill()
Dim rost As String = "Roster.txt"
Dim lines As List(Of String) = File.ReadAllLines(rost).ToList
Dim list As List(Of Student) = New List(Of Student)
For i As Integer = 0 To lines.Count - 1
Dim data As String() = lines(i).Split(":"c)
'Check data
If data.Length >= 4 Then '
list.Add(New Student() With {
.StudentName = data(0),
.StudentID = data(1),
.StudentDOB = data(2),
.StudentDOE = data(3)
})
End If
Next
StudentBindingSource.DataSource = list
End Sub
try this code:
Dim list As List(Of Student) = New List(Of Student)(100)
basically initialize the student list with a capacity. This is the capacity of the list, not the count/length.

How to get IEnumerable(Of DataRow) From DataTable query

All the examples I have found of complex grouping of DataTable results that use linq query commands look to have no problems getting an IEnumerable(Of DataRow) object as the result.
However I seem to only get a AnonymousType Enumerator return that I cannot cast to DataTable.
I have workarounds, but would prefer to convert the results to a DataTable, as it looks possible and I may be doing something wrong.
It's a simple table with Many ClientID and ClientName columns and other columns with login timestamps.
Dim dtMatrix As DataTable = New DataTable()
... (populate DataTable)
Dim qClients = From row In dtMatrix
Group row By client = New With {Key .ClientID = row("ClientID"), Key .ClientName = row("ClientName")} Into Group
Select New With {Key .ClientID = client.ClientID, Key .ClientName = client.ClientName}
This returns the generic Enumerator result, however
Dim qClients As IEnumerable(Of DataRow) = From row In dtMatrix
Group row By client = New With {Key .ClientID = row("ClientID"), Key .ClientName = row("ClientName")} Into Group
Select New With {Key .ClientID = client.ClientID, Key .ClientName = client.ClientName}
Throws an exception
Unable to cast object of type... to type
'System.Collections.Generic.IEnumerable`1[System.Data.DataRow]'.
I will be happy to paste the whole error message if it will add more clarity.
My base assumption is that the DataTable should allow the cast to occur inherently as it is the object being queried. However this does not seem to be the case. Have I constructed my query incorrectly? (Framework 4.6.2)
You can use OfType on the Rows property of the DataTable:
Dim dtMatrix As DataTable = New DataTable()
'' Populate code goes here...
Dim dtRows As IEnumerable(Of DataRow) = dtMatrix.Rows.OfType(Of DataRow)()
The Rows property returns a DataRowCollection, that implements (through inheritance) the IEnumerable interface but not the IEnumerable(Of T) interface, that's why you can't use most of linq over it directly.
The following extension uses Reflection to create a new DataTable and create DataColumns in it that match the properties and fields of the type passed in. In general, if you are creating anonymous types in LINQ, you can't just convert to a DataRow which must be tied to a DataTable which must already have matching columns. I went ahead and wrote a second extension to DataTable that adds an IEnumerable<T> with matching field/property names to it.
Public Module Ext
<Extension()>
Public Function GetValue(member As MemberInfo, srcObject As Object) As Object
If TypeOf member Is FieldInfo Then
Return DirectCast(member, FieldInfo).GetValue(srcObject)
ElseIf TypeOf member Is PropertyInfo Then
Return DirectCast(member, PropertyInfo).GetValue(srcObject)
Else
Throw New ArgumentException("MemberInfo must be of type FieldInfo or PropertyInfo", Nameof(member))
End If
End Function
<Extension()>
Public Function GetMemberType(member As MemberInfo) As Type
If TypeOf member Is FieldInfo Then
Return DirectCast(member, FieldInfo).FieldType
ElseIf TypeOf member Is PropertyInfo Then
Return DirectCast(member, PropertyInfo).PropertyType
ElseIf TypeOf member Is EventInfo Then
Return DirectCast(member, EventInfo).EventHandlerType
Else
Throw New ArgumentException("MemberInfo must be of type FieldInfo, PropertyInfo or EventInfo", Nameof(member))
End If
End Function
<Extension()>
Public Function ToDataTable(Of T)(rows As IEnumerable(Of T)) As DataTable
Dim dt = New DataTable
If (rows.Any()) Then
Dim rowType = rows.First().GetType()
Dim memberInfos = rowType.GetProperties.Cast(Of MemberInfo)().Concat(rowType.GetFields).ToArray()
For Each info In memberInfos
dt.Columns.Add(New DataColumn(info.Name, info.GetMemberType()))
Next
For Each r In rows
dt.Rows.Add(memberInfos.Select(Function (i) i.GetValue(r)).ToArray())
Next
End If
Return dt
End Function
<Extension()>
Public Function AddObjects(Of T)(dt As DataTable, rows As IEnumerable(Of T))
If (rows.Any()) Then
Dim rowType = rows.First().GetType()
Dim memberInfos = rowType.GetProperties().Cast(Of MemberInfo)().Concat(rowType.GetFields()).ToArray()
For Each r In rows
Dim newRow = dt.NewRow()
For Each memberInfo In memberInfos
newRow(memberInfo.Name) = memberInfo.GetValue(r)
Next
dt.Rows.Add(newRow)
Next
End If
Return dt
End Function
End Module
Note that I write in C# and translated this from my C# extension. It is untested but compiles.
Using the extension, you should be able to get a DataTable from your qClients by:
Dim dtClients = qClients.ToDataTable()

How to make a generic database record to object conversion function?

I am using the following function to retrieve records from a database and convert the records to a collection of strongly typed objects.
Private Function GetPlantSettingsFiltered(parameters As Dictionary(Of String, Object), queryCondition As String) As PlantSettings
Dim query As String
query = " SELECT * FROM Plant_Settings " _
+ queryCondition
Dim settings As New PlantSettings
Dim table As DataTable = GetQueryResults(parameters, query, GetConnectionString("WeighScaleDB"))
If table Is Nothing Then
Return settings
End If
For Each row As DataRow In table.Rows
settings.Add(New PlantSetting With {
.Setting_ID = ConvertByteArrayToString(TryCast(row("Setting_ID"), Byte())),
.Plant_ID = ConvertByteArrayToString(TryCast(row("Plant_ID"), Byte())),
.Value = row("Setting_Value").ToString(),
.Comments = row("Setting_Comments").ToString()
})
Next
Return settings
End Function
I would like to create a generic version of this function that would work for any of my objects without me creating this function for each object.
For example, if the caller could specify the type, then some other details, the function would return a collection of that type.
Private Function GetObjects(Of T)(parameters As Dictionary(Of String, Object), query As String) As WSAEntityCollection(Of T)
Dim objectCollection As New WSAEntityCollection(Of T)
Dim table As DataTable = GetQueryResults(parameters, query, GetConnectionString("WeighScaleDB"))
If table Is Nothing Then
Return objectCollection
End If
For Each row As DataRow In table.Rows
' Here is my problem
objectCollection.Add(New T With {})
Next
Return objectCollection
End Function
My current problem with this new function is that I do not know how to dynamically match the column names with the parameters of the generic object. Any ideas on how this could be done?

InvalidCastException when trying to Sum datatable rows with LINQ

Hi i am trying to sum all my datatable values to one row. but i retrieve a InvalidCastException:
Failed to convert an object of
typeWhereSelectEnumerableIterator2[System.Linq.IGrouping2[System.Object,System.Data.DataRow],VB$AnonymousType_0`4[System.Object,System.Double,System.Decimal,System.Decimal]]
to type System.Data.DataTable.
SQL Datatypes:
NAME_AGE string
LON money
sal_tjformon money
sal_sjuklon money
Private Function GroupByName(dataTable As DataTable) As DataTable
Dim result = dataTable.AsEnumerable().GroupBy(
Function(row) row.Item("NAME_AGE")).Select(Function(group) New With {
.Grp = group.Key,
.LON = group.Sum(Function(r) Decimal.Parse(r.Item("LON"))),
.sal_tjformon = group.Sum(Function(r) Decimal.Parse(r.Item("sal_tjformon"))),
.sal_sjuklon = group.Sum(Function(r) Decimal.Parse(r.Item("sal_sjuklon")))
})
Return result
The LINQ statement returns an IEnumerable(Of <anonymous_type>). There are two problems with this. First of all, your function returns a DataTable, which your object definitely is not. Secondly of all, you can't return an anonymous type from a function call.
If you want to return the select result, you have to create an explicit type (a class) and return the IEnumerable(Of MyType), like in the code below. I strongly advice to set an explicit type to the Grp property (like String?).
Class GroupNameAgeResult
Public Property Grp As Object
Public Property LON As Decimal
Public Property sal_tjformon As Decimal
Public Property sal_sjuklon As Decimal
End Class
Private Function GroupByName(dataTable As DataTable) As IEnumerable(Of GroupNameAgeResult)
Dim result = dataTable.AsEnumerable().GroupBy(Function(row) row.Item("NAME_AGE")) _
.Select(Function(grp) New GroupNameAgeResult() With
{.Grp = grp.Key,
.LON = grp.Sum(Function(r) Decimal.Parse(r.Item("LON").ToString)),
.sal_tjformon = grp.Sum(Function(r) Decimal.Parse(r.Item("sal_tjformon").ToString)),
.sal_sjuklon = grp.Sum(Function(r) Decimal.Parse(r.Item("sal_sjuklon").ToString))})
Return result
End Function
If you want to return a DataTable, you can define this, loop over the groups and add a row. You can return afterwards the result. See example code below.
Private Function GroupByName(dataTable As DataTable) As DataTable
Dim result As New DataTable()
result.Columns.Add("Grp", GetType(Object))
result.Columns.Add("LON", GetType(Decimal))
result.Columns.Add("sal_tjformon", GetType(Decimal))
result.Columns.Add("sal_sjuklon", GetType(Decimal))
For Each grp In dataTable.AsEnumerable().GroupBy(Function(row) row.Item("NAME_AGE"))
Dim row As DataRow = result.NewRow()
row.Item("Grp") = grp.Key
row.Item("LON") = grp.Sum(Function(r) Decimal.Parse(r.Item("LON").ToString))
row.Item("sal_tjformon") = grp.Sum(Function(r) Decimal.Parse(r.Item("sal_tjformon").ToString))
row.Item("sal_sjuklon") = grp.Sum(Function(r) Decimal.Parse(r.Item("sal_sjuklon").ToString))
result.Rows.Add(row)
Next
Return result
End Function
Last but not least. I strongly advice you to turn on "Option strict" (you can set this in the project properties -> Compile). You'll notice many more (possible) errors with your code (even the small function from this question).

How can we query on a generic columnName?

So far we have a method (OnSearchButtonClick) which searches on the tenantName column:
Private Sub OnSearchButtonClick(ByVal searchValue As String)
_filteredTenants = New ListCollectionView(_allTenants.Cast(Of TenantOverviewDto).Where(Function(p) p.TenantName.ToUpper().StartsWith(searchValue.ToUpper())).ToList())
End Sub
In this method we want to pass another string as a parameter with the name of a certain column of TenantOverviewDto.
For example if we want to look for a tennant in a group called "Runners" we want to pass "Runners" in the searchValue parameter and group in the new parameter and then execute a query which looks in the group column.
(If the second parameter is TenantName we should look in the TenantName column)
Does anyone have an idea of how we can accomplish this? All help is very much appreciated.
My collegue has found a solution with reflection.
the method looks like this:
Private Sub OnSearchButtonClick(ByVal searchValue As String, ByVal columnName As String)
_filteredTenants = New ListCollectionView(_allTenants.Cast(Of TenantOverviewDto).Where(Function(p) GetPropertyValue(p, columnName).ToUpper().StartsWith(searchValue.ToUpper())).ToList())
End Sub
In the GetPropertyValue method we'll use reflection:
Private Function GetPropertyValue(ByVal o As Object, ByVal propertyName As String) As Object
Dim type As Type = o.GetType()
Dim info As PropertyInfo = type.GetProperty(propertyName)
Dim value As Object = info.GetValue(o, Nothing)
Return value
End Function
First we get the type of object we passed (in this case TenantOverviewDto). Second we get the propertyInfo of the columnName we've passed. Then we get the corresponding value and we send this one back.
In this case, the reflection solution is simpler and more concise, therefore I'd prefer to use it.
However, I thought it was interesting to post a C# solution using dynamic expressions:
private void OnSearchButtonClick(string propertyName, string searchValue)
{
var parameter = Expression.Parameter(typeof(TenantOverviewDto), "p");
var toUpperMethodInfo = typeof(string).GetMethods()
.Single(m => m.Name == "ToUpper" && !m.GetParameters().Any());
var startsWithMethodInfo = typeof(string).GetMethods()
.Single(m => m.Name == "StartsWith" && m.GetParameters().Count() == 1);
var expression =
Expression.Call(
Expression.Call(
Expression.Property(parameter, propertyName),
toUpperMethodInfo),
startsWithMethodInfo,
Expression.Constant(searchValue.ToUpper())
);
var compiledCondition = Expression.Lambda<Func<TenantOverviewDto, bool>>
(expression, parameter).Compile();
_filteredTenants = new ListCollectionView(
_allTenants.Cast<TenantOverviewDto>().Where(compiledCondition).ToList());
}
This approach is much too complex for what's needed in this case. However, it could come in handy when the condition more complex than just comparing a variable property to a constant string.
try This solution :
Private Sub OnSearchButtonClick(ByVal searchValue As String, ByVal columnName As String)
_filteredTenants = _filteredTenants.findAll(f=>f[columnName].toString().StartsWith(searchValue.ToUpper())).toList()
End Sub