VB optimize loop avoid with an Evaluate() - vb.net

Question

I am running a loop that evaluates input given by the user over more than 150k objects. The user sets the info to read like "obj.Name", "obj.Path", "obj.Date"... which may contain also logic evaluations such as "IIf(obj.Params>5,1,0)". It is then provided into the programm as a string.
I used the Evaluate() functions and it does work well, just that it is slow. It takes almost 6h to go over all elements. I was thinking if there is a way in which I can take the requested info and turn it into a straight-forward executable somehow, and avoid using the Evaluate expression in the whole loop (it runs for the number of requested data by the user * 150k).
This is a schematic of the loop I am running:
For Each Object in ObjectsList
For Each UserRequest in Requests
ResultsMatrix(i,j) = Evaluate(Requests(i))
j += 1
Next
i += 1
Next
I store then the results in a Matrix which is pasted in an Excel file at the end. Is there a way in which I can do sort of working the string to be evaluated into a function's return? I'd like to avoid usig the Eval function and parse directly the string in an executable and dont evaluate it for each object. Any tips on speeding up the loop?

Answer 1

It might be worth considering writing the requests into a set of functions and using the .NET CodeDom compilers to build it into a DLL. You can then load the assembly, find the right functions using reflection and put them into an array, then call them using reflection - that way you'll be calling .NET code and it should be far faster. Some (incomplete) code to get you started from a project where I have done this...
Private Function CombineCode() As String
Dim ret As New System.Text.StringBuilder
ret.AppendLine("Imports System")
ret.AppendLine("Imports Microsoft.VisualBasic")
ret.AppendLine()
ret.AppendLine("Namespace " & MainNamespace)
ret.AppendLine("Public Class " & MainClassName)
For Each e In _Entries
ret.AppendLine(e.Value.Code)
Next
ret.AppendLine("End Class")
ret.AppendLine("End Namespace")
Return ret.ToString
End Function
Private Function Compile(Code As String) As Assembly
'Dim d As New Dictionary(Of String, String)
'd.Add("langversion", "14")
Dim VBP As New Microsoft.CodeDom.Providers.DotNetCompilerPlatform.VBCodeProvider()
Dim PM As New System.CodeDom.Compiler.CompilerParameters
'PM.GenerateInMemory = True
PM.GenerateExecutable = False
PM.OutputAssembly = IO.Path.Combine(_Path, GenerateFileName() & ".dll") ' "Generated.dll"
PM.MainClass = MainClassName
PM.IncludeDebugInformation = True
Dim ASM As System.Reflection.Assembly
For Each ASM In AppDomain.CurrentDomain.GetAssemblies()
Try
If ASM.Location <> "" Then PM.ReferencedAssemblies.Add(ASM.Location)
Catch
End Try
Next
PM.ReferencedAssemblies.Add("System.Web.dll")
'Get compilation results
Dim Results As System.CodeDom.Compiler.CompilerResults
Results = VBP.CompileAssemblyFromSource(PM, Code)
'Show possible compilation errors
Dim Err As System.CodeDom.Compiler.CompilerError
For Each Err In Results.Errors
Throw New SyntaxErrorException("Error N. " & Err.ErrorNumber &
" Message: " & Err.ErrorText & " Line " & Err.Line & " in code " & vbCrLf & Code)
Next
Return Results.CompiledAssembly
End Function
Private Sub FindMethods()
Dim dt = (From t In _LatestAssembly.GetTypes() Where t.Name = MainClassName).Single
For Each e In _Entries.Values
e.Method = dt.GetMethod(e.MethodName)
Next
End Sub
Assembly = Assembly.LoadFrom(System.IO.Path.Combine(Path, sd.LatestAssemblyFile))

Answer 2

The Evaluate function is just resources on the computer itself. It's a great candidate for using Parallel.For.
In this case, j is the implied index.
For Each Object in ObjectsList
Parallel.For(0, Requests.Length, New ParallelOptions(), Sub(j, loopState)
ResultsMatrix(i,j) = Evaluate(Requests(j))
End Sub
)
i += 1
Next
Note, that Requests(i) is getting called repeatedly and produces the same result, so I assume you mean Requests(j).