VB.Net Use XmlNodeList in a parallel ForEach - vb.net

I have a piece of code that iterates over the nodes in an XmlNodeList and creates a different object for each one depending on the node name and adds it to a list for printing.
For Each node as XmlNode In nodeList
Select Case node.Name.ToUpper()
Case "SHAPE"
_items.Add(New ShapeTemplate(node, Me))
Case "TEXTBLOCK"
_items.Add(New TextblockTemplate(node, Me))
End Select
Next
This code works fine, but because of all the work that has to be done by the ShapeTemplate and TextblockTemplate constructors, it is VERY slow. Since the order objects are added to _items doesn't matter, I thought a good way to speed it up would be to use a parallel.ForEach loop. The problem is XmlNodeList can't be used with parallel.ForEach because it is a non-generic collection. I've been looking into ways to convert XmlNodeList to List(Of XmlNode) with no luck. The answer I keep seeing come up is
Dim nodes as New List(Of xmlNode)(nodeList.Cast(Of xmlNode)())
But when I try it, I get an error telling me that 'Cast' is not a member of XmlNodeList.
I've also tried using TryCast like this
Dim nodes as List(Of XmlNode) = TryCast(CObj(nodeList), List(Of XmlNode))
but it results in nodes being Nothing because the object can't be cast.
Does anyone know how I can use XmlNodeList in a parallel.ForEach loop?
EDIT: I'm trying to avoid using a loop for the conversion if I can

You could use Parallel LINQ instead of Parallel.ForEach, which seems like a more natural fit for this sort of transformation. This looks just like a normal LINQ query, but with .AsParallel() added.
Imports System.Linq
' _items will not be in same order as nodes in nodeList.
' Add .AsOrdered() after .AsParallel() to maintain order, if needed.
_items = (
From node In nodeList.Cast(Of XmlNode)().AsParallel()
Select item = CreateTemplate(node)
Where item IsNot Nothing
).ToList()
Function CreateTemplate(node As XmlNode) As ITemplate ' interface or base class for your types
Dim item As ITemplate
Select Case node.Name.ToUpper()
Case "SHAPE"
item = New ShapeTemplate(node, Me)
Case "TEXTBLOCK"
item = New TextblockTemplate(node, Me)
Case Else
item = Nothing
End Select
Return item
End Function

As seen here, the XmlNodeList can be converted to a generic List(Of XmlNode) by passing it to the constructor with OfType.
' I added so your code could compile.
' I assumed an interface shared by ShapeTemplate and TextblockTemplate
Dim nodeList As XmlNodeList
Dim _items As New List(Of ITemplate)
Dim _nodes As New List(Of XmlNode)(nodeList.OfType(Of XmlNode))
Now the parallel loop. Note, if you are adding to a non-threadsafe collection such as List, you will need to synchronize adding the objects to the list. So i separated the time consuming portion (Template constructor) from the fast operation (adding to the list). This should improve your performance.
Dim oLock As New Object
Parallel.ForEach(
_nodes,
Sub(node)
Dim item As ITemplate
Select Case node.Name.ToUpper()
Case "SHAPE"
item = New ShapeTemplate(node, Me)
Case "TEXTBLOCK"
item = New TextblockTemplate(node, Me)
Case Else
item = Nothing ' or, do something else?
End Select
SyncLock oLock
_items.Add(item)
End SyncLock
End Sub)

Related

Reflection Optimization

I've recently implemented reflection to replace the more tedious aspects of our data retrieval from a SQL database. The old code would look something like this:
_dr = _cmd.ExecuteReader (_dr is the SQLDataReader)
While _dr.Read (_row is a class object with public properties)
_row.Property1 = Convert.ToInt16(_dr("Prop1"))
_row.Property2 = Convert.ToInt16(_dr("Prop2"))
_row.Property3 = Convert.ToInt16(_dr("Prop3"))
If IsDBNull(_dr("Prop4")) = False Then _row.Prop4 = _dr("Prop4")
...
Since my code base has a lot of functionality like this, reflection seemed like a good bet to simplify it and make future coding easier. How to assign datareader data into generic List ( of T ) has a great answer that was practically like magic for my needs and easily translated into VB. For easy reference:
Public Shared Function GenericGet(Of T As {Class, New})(ByVal reader As SqlDataReader, ByVal typeString As String)
'Dim results As New List(Of T)()
Dim results As Object
If typeString = "List" Then
results = New List(Of T)()
End If
Dim type As Type = GetType(T)
Try
If reader.Read() Then
' at least one row: resolve the properties
Dim props As PropertyInfo() = New PropertyInfo(reader.FieldCount - 1) {}
For i As Integer = 0 To props.Length - 1
Dim prop = type.GetProperty(reader.GetName(i), BindingFlags.Instance Or BindingFlags.[Public])
If prop IsNot Nothing AndAlso prop.CanWrite Then
props(i) = prop
End If
Next
Do
Dim obj = New T()
For i As Integer = 0 To props.Length - 1
Dim prop = props(i)
If prop Is Nothing Then
Continue For
End If
' not mapped
Dim val As Object = If(reader.IsDBNull(i), Nothing, reader(i))
If val IsNot Nothing Then SetValue(obj, prop, val)
Next
If typeString = "List" Then
results.Add(obj)
Else
results = obj
End If
Loop While reader.Read()
End If
Catch ex As Exception
Helpers.LogMessage("Error: " + ex.Message + ". Stacktrace: " + ex.StackTrace)
End Try
Return results
End Function
The only caveat with this is that it is somewhat slower.
My question is how to optimize. Sample code I find online is all in C# and does not convert neatly into VB. Scenario 4 here seems like exactly what I want, but converting it to VB gives all kinds of errors (Using CodeFusion or converter.Telerik.com).
Has anyone done this in VB before? Or can anyone translate what's in that last link?
Any help's appreciated.
Couple ideas for you.
Don't use the DataReader when reading ALL records at once, it is slower than using a DataAdapter.
When you use the DataAdapter to fill a DataSet, you can iterate through the rows and columns which does NOT use reflection and will be much faster.
I have a program I created (and many other programmers do this too) that generate the code from a database for me. Each table and row is a class that is specifically named an generated in such a way that I can use intellisense and prevents many run-time errors by making them compile-time errors when data changes. This is very much like the EntityFramework but lighter because it fits MY specific needs.

Flatten Treeview Nodes collection to List without recursion

Hopefully this is not a repeat question. I looked and found similar questions but not sufficient in this case.
I have many tree view controls, and can traverse the Nodes recursively for various reasons.
However, I often need to traverse the Nodes as if they are in a List.
I would like to make a function that creates a Generic.List(of TreeNode) from the Nodes collection, without recursion if at all possible.
(Without recursion purely for the exercise of doing it without recursion - I understand it may not be possible)
This function would save alot of time with repeated use across a massive solution, where the coders could use a simple For Each paradigm to traverse the Nodes.
I have seen a technique to 'flatten' the Nodes collection using C#, which uses LINQ and recursion, but I am not sure that the syntax can be converted to VB.NET cleanly. So if there are any clever VB functions out there can that I can mold to this task - would be very helpful.
There are many similar questions and very informative answers on SO, like this one:
Enumerating Collections that are not inherently IEnumerable?
...which highlights stack overflow errors in very deep trees using some algorithms. I hope that a method that does not use recursion will not suffer from Stack overflow errors - however, I am prepared that it might be long and clumsy and slow.
I am also prepared for the answer that 'It is not possible to do this without recursion'. However, I would like to confirm or deny this claim using the power of SO (this forum)
It's possible, and not very hard at all....
Public Class Form1
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
TreeView1.ExpandAll()
For Each TN As TreeNode In TreeView1.NodesToListWithoutRecursionBecauseWhyNot(TraverseType.BreadthFirst)
Debug.Print(TN.Text)
Next
End Sub
End Class
Public Module Extensions
Public Enum TraverseType
BreadthFirst
DepthFirst
End Enum
<Runtime.CompilerServices.Extension()> _
Public Function NodesToListWithoutRecursionBecauseWhyNot(ByVal TV As TreeView, Optional ByVal Traverse As TraverseType = TraverseType.DepthFirst) As List(Of TreeNode)
Dim nodes As New List(Of TreeNode)
Select Case Traverse
Case TraverseType.BreadthFirst
Dim Q As New Queue(Of TreeNode)
For Each TN As TreeNode In TV.Nodes
Q.Enqueue(TN)
Next
While Q.Count > 0
Dim TN As TreeNode = Q.Dequeue
nodes.Add(TN)
For Each subTN As TreeNode In TN.Nodes
Q.Enqueue(subTN)
Next
End While
Case TraverseType.DepthFirst
Dim L As New List(Of TreeNode)
For Each TN As TreeNode In TV.Nodes
L.Add(TN)
Next
While L.Count > 0
Dim TN As TreeNode = L.Item(0)
L.RemoveAt(0)
nodes.Add(TN)
For i As Integer = TN.Nodes.Count - 1 To 0 Step -1
L.Insert(0, TN.Nodes(i))
Next
End While
End Select
Return nodes
End Function
End Module
Just add the nodes to the list but at the same time keep the position of the last node you processed. A node is considered process when its immediate children are added to the list.
Public Function GetAllNodes(ByVal topNode As TreeNode)
Dim allNodes As New List(Of TreeNode)
Dim lastProcessPosition As Integer = 0
allNodes.Add(topNode)
Do While lastProcessPosition < allNodes.Count
allNodes.AddRange(allNodes(lastProcessPosition).Nodes)
lastProcessPosition += 1
Loop
Return allNodes
End Function
If you don't have a top node then just substitute the parameter for a list of nodes to start with.
Public Function GetAllNodes(ByVal topNodes As TreeNodeCollection)
Dim allNodes As New List(Of TreeNode)
Dim lastProcessPosition As Integer = 0
allNodes.AddRange(topNodes)
Do While lastProcessPosition < allNodes.Count
allNodes.AddRange(allNodes(lastProcessPosition).Nodes)
lastProcessPosition += 1
Loop
Return allNodes
End Function
I'm not sure if a check for Nothing must be done on the Nodes property before using it.
Note: I was able to remove this for loop and replace it with AddRange
'For Each node As TreeNode In allNodes(lastProcessPosition).Nodes
' allNodes.Add(node)
'Next

variable is referenced from scope, but not defined LINQ expression tree

I'm trying to get this LINQ expression:
Result = Result.Where(Function(Row) _WhereExpressions(0).InElements.Contains(Convert.ToString(Row(0))))
I have this code for it:
convertMethod = GetType(System.Convert).GetMethod("ToString", New Type() {GetType(Object)})
containsMethod = GetType(System.Collections.Generic.List(Of String)).GetMethod("Contains", New Type() {GetType(String)})
Dim listParameter = Expression.Parameter(GetType(List(Of String)), "_WhereExpressions(0).InElements")
expr = Expression.Call(whereMethod, Result.AsQueryable.Expression,
Expression.Lambda(Expression.Call(listParameter, containsMethod,
Expression.Call(convertMethod, Expression.ArrayAccess(rowParameter, Expression.Constant(index)))), rowParameter))
I get the desired expression, but if I compile, I get the error:
variable '_WhereExpressions(0).InElements' of type 'System.Collections.Generic.List`1[System.String]' referenced from scope '', but it is not defined
The _WhereExpressions(0).InElements is of course declared.
How can I fix it?
Thanks.
EDIT: here are all the declarations:
Dim whereMethod = GetType(Queryable).GetMethods(BindingFlags.Public Or BindingFlags.Static).First(Function(m) m.Name = "Where").MakeGenericMethod(GetType(Object()))
Dim convertMethod As MethodInfo = Nothing
Dim containsMethod As MethodInfo = Nothing
Dim rowParameter = Expression.Parameter(GetType(Object()), "Row")
The _WhereExpressions(0).InElements is a simple list of string, like this here:
Dim idlist As New List(Of String)
idlist.Add("1")
idlist.Add("2")
I read the linked post, but I can't really figure out, how I should solve my problem.
Expression trees have a lot capability, but looks a bit difficult for me.
EDIT2:
This is an example, what exactly I would like to achieve. Just copy and paste in vs:
Dim dt As New DataTable
dt.Columns.Add("f1", Type.GetType("System.String"))
dt.Columns.Add("f2", Type.GetType("System.Int32"))
For i = 0 To 100
dt.Rows.Add(i.ToString, i * 2)
Next
Dim indexes As New List(Of Integer)
indexes.Add(0)
indexes.Add(1)
Dim lst As New List(Of String)
lst.Add("10")
lst.Add("11")
Dim datarows As New List(Of DataRow)
For i = 0 To dt.Rows.Count - 1
datarows.Add(dt.Rows(i))
Next
Dim result As IEnumerable(Of Object())
result = datarows.Select(Function(row) indexes.Select(Function(index) row(index)).ToArray)
'I would like this as an expression:
result = result.Where(Function(row) lst.Contains(Convert.ToString(row(0))))
EDIT3: I got it:
Dim lst As Expression = Expression.Constant(list, GetType(System.Collections.Generic.List(Of String)))
It is hard to replicate code without knowing the full variables. I think your mistake though is in your understanding of Expression.Parameter. This is mainly used as a way to pass explicit parameters into the lambda: In your example, Row is a good example for when Expression.Parameter should be used. _WhereExpressions isn't an explicit parameter, it is a variable that I assume is in scope that you want to create a closure around.
You should also note that the second variable of the Expression.Parameter method is optional, and for debug purposes only: If you changed it to say: Expression.Parameter(GetType(List(Of String)), "Nonsense.nonsense"), you would probably see the error message change accordingly.
It sounds like you're trying to introduce a closure around _WhereExpressions. Using raw expression trees, this is fairly hard to do. The closest thing to an easy way to do this is to wrap Expression.Constant around _WhereExpressions.InElements. However, you're going to run into trouble if you're executing the compiled expression when _WhereExpressions is out of scope. See Issue with closure variable capture in c# expression.

Cast(Of ?) from UIElement

In silverlight my custom controls are in theUIElementCollection of my StackPanel. I want to get a list of them by a specific value. There only DivElements in the container. It returns Nothing when I know I have one or more. I know I can make a simple loop and cast types inline, but I want to get better with LINQ and Cast(Of TResult). My attempt at casting:
Dim myList = TryCast(spDivs.Children.Where(Function(o) DirectCast(o, DivElement).ElementParent Is bComm).Cast(Of DivElement)(), List(Of DivElement))
The problem is you can't cast into a List(Of DivElement). The collection is a UIElementCollection, not a List(Of T).
You could build a new list, though. This can also be simplified by using OfType instead of casting manually:
Dim myList = spDivs.Children.OfType(Of DivElement)()
.Where(Function(o) o.ElementParent Is bComm)
.ToList()

Generic Lists copying references rather than creating a copiedList

I was developing a small function when trying to run an enumerator across a list and then carry out some action. (Below is an idea of what I was trying to do.
When trying to remove I got a "Collection cannot be modified" which after I had actually woken up I realised that tempList must have just been assigned myLists reference rather than a copy of myLists. After that I tried to find a way to say
tempList = myList.copy
However nothing seems to exist?? I ended up writing a small for loop that then just added each item from myLsit into tempList but I would have thought there would have been another mechanism (like clone??)
So my question(s):
is my assumption about tempList receiving a reference to myList correct
How should a list be copied to another list?
private myList as List (Of something)
sub new()
myList.add(new Something)
end sub
sub myCalledFunction()
dim tempList as new List (Of Something)
tempList = myList
Using i as IEnumerator = myList.getEnumarator
while i.moveNext
'if some critria is met then
tempList.remove(i.current)
end
end using
end sub
By writing tempList = myList you don't make a copy oh the collection, you only make tempList reference myList. Try this instead : dim tempList as new List (Of Something)(myList)
I think if you called myCalledFunction(byVal aListCopy as Something) you can let the framework do the work.
If your list consists of value types you can just create a new list with the old list passed in the constructor. If you are going to be doing a deep copy of a reference object your best bet is to have your reference type implement ICloneable (example). You can then loop through and clone each object or you could add an extension method (like this c# example).
Try this - use LINQ to create a new list from the original, for example:
Sub Main()
Dim nums As New List(Of Integer)
nums.Add(1)
nums.Add(2)
nums.Add(3)
nums.Add(4)
Dim k = (From i In nums _
Select i).ToList()
For Each number As Integer In nums
k.Remove(number)
Next
End Sub
k will then be a new list of numbers which are not linked to the source.