According to the language specification guide for VB.NET Section 10.9.3
The enumerator expression in a for each loop is copied over into
memory.
If I have a list of 10000 objects that list will be in memory twice for the code below?
dim myList as new list(of bobs)
'put 10000 bobs in my list
for each x In myList
'do something
next
If I were generating the list from a linqQuery or some other such query it would make sense to generate that list at the for each loop statement thus not having the list in memory twice for example.
for each x in myList.where(function(x) x.name = Y)
'do something
next
If the LINQ query is unreadable on the for each loop, do I forgo readability and just put it on the for each loop declaration line?
Should I declare the list in its own variable and just bite the bullet and have the list exist twice in memory?
that list will be in memory twice for the code below
No, it won't. In your case, the spec is talking about the variable "x" here - not the entire collection. Remember, and enumerator (any IEnumerable<T> or similar) doesn't necessarily even have items in memory. When created via an iterator in C#, for example, you can have "collections" that are generated as you enumerate over them. There isn't a "list" of objects (necessarily) that could be copied, even if the language wanted to do so.
Is the linq query is unreadable on the for each loop
In many cases, I prefer filtering this way. You can just as easily move this outside of the loop, if you want to make it more clear, as well:
Dim filteredCollection = myList.Where(Function(x) x.name = Y)
For Each x in filteredCollection
There is no disadvantage to doing this if you find it more readable.
Related
I am trying to program a way to read a text file and match all the values and their quantites. For example if the text file is like this:
Bread-10 Flour-2 Orange-2 Bread-3
I want to create a list with the total quantity of all the common words. I began my code, but I am having trouble understanding to to sum the values. I'm not asking for anyone to write the code for me but I am having trouble finding resources. I have the following code:
Dim query = From data In IO.File.ReadAllLines("C:\User\Desktop\doc.txt")
Let name As String = data.Split("-")(0)
Let quantity As Integer = CInt(data.Split("-")(1))
Let sum As Integer = 0
For i As Integer = 0 To query.Count - 1
For j As Integer = i To
Next
Thanks
Ok, lets break this down. And I not seen the LET command used for a long time (back in the GWBASIC days!).
But, that's ok.
So, first up, we going to assume your text file is like this:
Bread-10
Flour-2
Orange-2
Bread-3
As opposed to this:
Bread-10 Flour-2 Orange-2 Bread-3
Now, we could read one line, and then process the information. Or we can read all lines of text, and THEN process the data. If the file is not huge (say a few 100 lines), then performance is not much of a issue, so lets just read in the whole file in one shot (and your code also had this idea).
Your start code is good. So, lets keep it (well ok, very close).
A few things:
We don't need the LET for assignment. While older BASIC languages had this, and vb.net still supports this? We don't need it. (but you will see examples of that still floating around in vb.net - especially for what we call "class" module code, or "custom classes". But again lets just leave that for another day.
Now the next part? We could start building up a array, look for the existing value, and then add it. However, this would require a few extra arrays, and a few extra loops.
However, in .net land, we have a cool thing called a dictionary.
And that's just a fancy term of for a collection VERY much like an array, but it has some extra "fancy" features. The fancy feature is that it allows one to put into the handly list things by a "key" name, and then pull that "value" out by the key.
This saves us a good number of extra looping type of code.
And it also means we don't need a array for the results.
This key system is ALSO very fast (behind the scene it uses some cool concepts - hash coding).
So, our code to do this would look like this:
Note I could have saved a few lines here or there - but that would make this code hard to read.
Given that you look to have Fortran, or older BASIC language experience, then lets try to keep the code style somewhat similar. it is stunning that vb.net seems to consume even 40 year old GWBASIC type of syntax here.
Do note that arrays() in vb.net do have some fancy "find" options, but the dictionary structure is even nicer. It also means we can often traverse the results with out say needing a for i = 1 to end of array, and having to pull out values that way.
We can use for each.
So this would work:
Dim MyData() As String ' an array() of strings - one line per array
MyData = File.ReadAllLines("c:\test5\doc.txt") ' read each line to array()
Dim colSums As New Dictionary(Of String, Integer) ' to hold our values and sum them
Dim sKey As String
Dim sValue As Integer
For Each strLine As String In MyData
sKey = Split(strLine, "-")(0)
sValue = Split(strLine, "-")(1)
If colSums.ContainsKey(sKey) Then
colSums(sKey) = colSums(sKey) + sValue
Else
colSums.Add(sKey, sValue)
End If
Next
' display results
Dim KeyPair As KeyValuePair(Of String, Integer)
For Each KeyPair In colSums
Debug.Print(KeyPair.Key & " = " & KeyPair.Value)
Next
The above results in this output in the debug window:
Bread = 13
Flour = 2
Orange = 2
I was tempted here to write this code using just pure array() in vb.net, as that would give you a good idea of the "older" types of coding and syntax we could use here, and a approach that harks all the way back to those older PC basic systems.
While the dictionary feature is more advanced, it is worth the learning curve here, and it makes this problem a lot easier. I mean, if this was for a longer list? Then I would start to consider introduction of some kind of data base system.
However, without some data system, then the dictionary feature is a welcome approach due to that "key" value lookup ability, and not having to loop. It also a very high speed system, so the result is not much looping code, and better yet we write less code.
The following VB .net code gives me an out of memory exception. Does anybody knows why?
Dim vArray As ILArray(Of Double) = ILMath.rand(10000000)
Using ILScope.Enter(vArray)
For i As Integer = 1 To 100
vArray = ILMath.add(vArray, vArray)
Next
End Using
Thank you very much.
In this toy example you can simply remove the artificial scope and it will run fine:
Dim vArray As ILArray(Of Double) = ILMath.rand(10000000)
For i As Integer = 1 To 100
vArray = ILMath.add(vArray, vArray)
Next
Console.WriteLine("OK: " + vArray(0).ToString())
Console.ReadKey()
However, in a more serious situation, ILScope will be your friend. As stated on the ILNumerics page an artificial scope ensures a deterministic memory management:
All arrays created inside the scope are disposed once the block was
left.
Otherwise one had to rely on the GC for cleanup. And, as you know, this involves a gen 2 collection for large objects – with all disadvantages in terms of performance.
In order to be able to dispose the arrays they need to be collected and tracked somehow. Whether or not this qualifies for the term 'memory leak' is rather a philosophical question. I will not go into it here. The deal is: after the instruction pointer runs out of the scope these arrays are taken care of: their memory is put into the memory pool and will be reused. As a consequence, no GC will be triggered.
The scheme is especially useful for long running operations and for large data. Currently, the arrays are released only AFTER the scope block was left. So if you create an algorithm/ loop which requires more memory than available on your machine you need to clean up DURING the loop already:
Dim vArray As ILArray(Of Double) = ILMath.rand(10000000)
For i As Integer = 1 To 100
Using ILScope.Enter
vArray.a = ILMath.add(vArray, vArray)
' ...
End Using
Next
Here, the scope cleans up the memory after each iteration of the loop. This affects all local arrays assigned within the loop body. If we want an array value to survive the loop iteration we can assign to its .a property as shown with vArray.a.
there are two lists
Myobjects list contains all items, all have property "oid" of
ObjectID
The list of ObjectID types is list of ALL objects to be
deleted
Problem is to find any item in Myobjects that has ObjectID in oid property.
I'm having a hard time finding equivalent of this two "for each" loops in LINQ.
This should be easy, but I'm doing it wrong. It takes 20 sec to perform this LINQ on 30 000 items, and under one sec using for-each loop. Below is working for-each loop, and my attempt to make LINQ of it (this one is slow). Linq solution is what I would like to have.
For Each i As LengthAreaObject In myobjects
For Each o As ObjectId In oidsToRem
If i.oid = o Then
returnlist.Add(i)
oidsToRem.Remove(o)
Exit For
End If
Next
Next
and LINQ attempt here
Dim rlist As List(Of LengthAreaObject) = (From i As LengthAreaObject In myobjects.AsParallel Where oidsToRem.Contains(i.oid) Select i).ToList
How much items have the collections?
For small collections, AsParallel will be slower because of the extra method call.
I have found similiar question on stackoverflow and modified it.
So the solution to speed things up is nothing more but to hash collection where the objects are you test for "contains". Otherwise, contains is slow!
Dim setToRemove = New HashSet(Of ObjectId)(oidstoRem)
Dim returnlist As List(Of LengthAreaObject) = lengthAreaList.AsParallel.Where(Function(x) setToRemove.Contains(x.oid)).ToList
Simplified, I have a List(Of MyObj), and I want to iterate through that list and compare each element to all other elements in the same list, excluding (if possible) the same element. I have a solution that works, but it's slow and uses double For loops. It may possibly have also summoned Cthulhu from his sleep.
Is there a better approach? Linq, perhaps? Or some fancy algorithm? This below is a sanitized version of what I have:
Dim MyList As New List(Of MyObj)({Obj1, Obj2, Obj3, Obj4, Obj5, Obj6})
If MyList.Count > 0 Then
For i = 0 To (MyList.Count - 1) Step 1
For j As Int32 = 0 To (MyList.Count - 1) Step 1
If MyList(i).GetHashCode = MyList(j).GetHashCode Then
Continue For
Else
If MyList(i).SomeFunction(MyList(j)) Then
Call DoSomething()
End If
End If
Next j
Next i
Else
' Error Code Here.
End If
This will work in O(M*N) where N is ObjCount and M is the number of non-duplicate objects. Your current solution is O(N^2).
You need a Hash Function. You can determine whether GetHashCode will suffice or whether you need to implement Sha1.
Instantiate a HashSet (or HashTable, depending on your Hash Function)
Add each object, if it does not already exist, into the HashSet or HashTable.
For each object in the HashSet, execute SomeFunction() against every other object in the HashSet. If you dump into an array and iterate via indexes, you only have to compare indexes, rather than objects.
For i as integer = 0 to MyHashResultsArray.Count - 1
For j as integer = 0 to MyHashResultsArray.Count - 1
if i <> j then
MyHashResultsArray(i).DoSomething(j)
end if
next
next
Important
This is only a good solution IF there exists a significant amount of duplicates, perhaps a duplicate-level of 10% would be necessary to consider this solution, except for very large values of N. If N gets too large, a re-engineering of the application may be necessary to hopefully avoid the need for M actions against M objects.
Edit
Much of the comment discussion was based upon my misunderstanding of the Author's needs regarding the DoSomething() function.
Barring any potential problems with using GetHashCode to check for object equality (best not to do this - it'll only bite you at some point - and it's probably this that has awakened Cthulhu!), your solution is about as fast as it's likely to get.
Sure, you can tweak it, but it will remain O(N^2), that is, the runtime will be of the order of the square of the number of elements in your list. If you double the number of elements, your runtime will increase by a factor of 4.
See if this will work
MyList.Select(Function(x) MyList.Except(New () {x}).ToList().ForEach(Sub(y) Do
If x.SomeFunction(y) Then
DoSomething()
End If
End Sub))
I would like to loop through two lists using a For each loop.
dim data as list(of pointpairlist)
For each recLine in records
For Each chan In recLine.channels and d in data
d.add( func(chan) )
Next
next
note: each record line has one sample from each channel recorded. ie each record line is a slice of a 32 sensor recordings. I want to build up a x,y list of data points for each channel (the x axis is common to all channels)
Is there some way to do it similar to what i have above (avoiding indexing variables)
The best way to iterate over two different collections in a single loop is to use indexers.
For index As Integer = 0 To recLine.channels.Count - 1
data(index) = func(chan(index))
Next
As #0xA3 comments, you can also a while loop, calling GetEnumerator, MoveNext and Current directly on each collection, but this buys you nothing.
I realize you want to avoid using indexers, but there is no simple language support for anything else.
This is the case also for C#.
Why do you need to avoid indexers?
Nest the loops. I cannot reverse-engineer the actual declarations from the snippet, but something like this:
Dim data As List(Of PointPairList)
''...
For Each point In data
For Each chan In point.recLine
func(chan)
Next
Next