Is it possible to speed up this code using Parallel.For loop?
I have List of few millions Integer arrays and need to remove all arrays that doesn't meet compare criteria. How can I use multi-threading to speed-up the loop through the List if I need to remove some items from it?
(This is simple code example, real code has more complex criteria checks inside the loop)
Function NewList(iList As List(Of Integer())) As List(Of Integer())
Dim i As Integer
Dim j As Integer
Dim compareArray As Integer() = {0, 1, 2, 3, 4}
Dim item As Integer()
For i = iList.Count - 1 To 0 Step -1
item = iList.Item(i)
For j = 0 To UBound(compareArray )
If compareArray(j) > 0 AndAlso Not item.Contains(compareArray(j)) Then
iList.RemoveAt(i)
GoTo nextIteration
End If
Next j
nextIteration:
Next i
Return iList
End Function
I would try something like this (untested):
Public Shared Function GetFilteredList(valueList As List(Of Int32()), mustIncludeList As Int32()) As List(Of Int32())
'Check args
If (valueList Is Nothing) Then Throw New ArgumentNullException(NameOf(valueList))
If (mustIncludeList Is Nothing) OrElse (Not mustIncludeList.Any()) Then Return New List(Of Int32())(valueList) 'A new instance to avoid side effects. Could be optimized if not needed
'Normalize args
mustIncludeList = (From e In mustIncludeList.Distinct() Where e > 0 Order By e).ToArray() 'Normalize it, remove duplicates and values < 1 (don't do it 1 million times inside the loop)
'Initialize the filter
Dim myFilter As Func(Of Int32(), Boolean) = Function(valueArray As Int32())
'As all of the values must be included we can skip all arrays that are shorter
If (valueArray Is Nothing) OrElse (valueArray.Length < mustIncludeList.Length) Then Return False
'Try out if the next line gives a better performance (depends a bit the size of the arrays)
Dim myValues As New HashSet(Of Int32)(valueArray)
For Each myMustInclude As Int32 In mustIncludeList
If (Not myValues.Contains(myMustInclude)) Then Return False
Next
Return True
End Function
'Apply filter and return new list
Return valueList.AsParallel().Where(myFilter).ToList()
End Function
EDITED
Thanks to everybody and especially for #Christoph advice to assign "Nothing" to items instead of deleting them and then to clear "Nothing"s from List. Also useful advice to normalize/prepare compare array once outside the loop and not check it millions times [this advice is not implemented here )]
Result speed increased from "minutes" to "seconds"
The following code returns new list in milliseconds with over 2 millions items.
Function NewList(iList As List(Of Integer())) As List(Of Integer())
Dim compareArray As Integer() = {12, 15, 24}
'//Note that "j" and "item" should be declared inside enclosed Sub
'Parallel.For(0, iList.Count - 1, Sub(i)
Parallel.For(0, iList.Count, Sub(i)
'//changed upper limit for loop /see comment below code/
Dim j As Integer
Dim item As Integer()
item = iList.Item(i)
For j = 0 To UBound(compareArray)
If compareArray(j) > 0 AndAlso Not item.Contains(compareArray(j)) Then
iList(i) = Nothing
GoTo nextIteration
End If
Next j
nextIteration:
End Sub)
'//parallel.for looped instantly with 2,118,760 items
'//then removing "Nothing" in usual loop have taken more than a minute,
'//so speed-up failed
'Dim k As Integer
'For k = iList.Count - 1 To 0 Step -1
' If iList(k) Is Nothing Then
' iList.RemoveAt(k)
' End If
'Next
'//but using RemoveAll Function cleared List instantly
iList.RemoveAll(Function(itm) IsNothing(itm))
Return iList
End Function
I've tested Parallel.For approach (more complex than in my own answer) but with passing parameters to be checked to main Function. And result was not always correct. Using #Christoph Function (with AsParallel) - results are ok.
With loop until iList.Count - 1: the last list was not checked, I changed upper loop limit to iList.Count -> and results are correct.
Have to read documentation attentively...(upper loop value is Exclusive)
public static System.Threading.Tasks.ParallelLoopResult For (int
fromInclusive, int toExclusive,
System.Threading.Tasks.ParallelOptions parallelOptions,
Action<int,System.Threading.Tasks.ParallelLoopState> body);
Related
enter image description hereWrite a program to display 1000 8-character random user IDs in a text box after you click a button. Make sure the program verifies that none of the IDs are identical. Each userid should include a mixture of alphabetic characters and numbers.
What I have so far, which could be enough to get by as a random ID generator alone, but it does not loop 1000 times to be displayed in the label, and doesn't check for repeated ID's.
Public Class Form1
Private Sub btnGenerateRandomID_Click(sender As Object, e As EventArgs) Handles btnGenerateRandomID.Click
Dim rand As New Random
Dim char1 As String
Dim char2 As String
Dim char3 As String
Dim char4 As String
Dim char5 As String
Dim char6 As String
Dim char7 As String
Dim char8 As String
char1 = ChrW(rand.Next(Asc("A"), Asc("Z") + 1))
char2 = ChrW(rand.Next(Asc("A"), Asc("Z") + 1))
char3 = ChrW(rand.Next(Asc("A"), Asc("Z") + 1))
char4 = ChrW(rand.Next(Asc("A"), Asc("Z") + 1))
char5 = rand.Next(0, 9)
char6 = rand.Next(0, 9)
char7 = rand.Next(0, 9)
char8 = rand.Next(0, 9)
lblRandomId.Text = char1 + char2 + char3 + char4 + char5 + char6 + char7 + char8
End Sub
End Class
Thanks.
EDIT:
Public Class Form1
'Write a program to display 1000 8-character random user IDs in a text
'box after you click a button. Make sure the program verifies that none Of the IDs are identical.
'Each userid should include a mixture Of alphabetic characters And numbers.
Private Sub btnGenerateRandomID_Click(sender As Object, e As EventArgs) Handles btnGenerateRandomID.Click
Dim strChar As String = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
Dim rand As New Random
Dim strID As String = ""
For count_ids As Integer = 0 To 999
For count_chars As Integer = 0 To 7
strID += strChar(rand.Next(0, 62))
Next count_chars
lblRandomId.Text = strID
Next
End Sub
End Class
Here is the actual question "Q. Write a program to display 1000 8-character random user IDs in a text box after you click a button. Make sure the program verifies that none of the IDs are identical. Each userid should include a mixture of alphabetic characters and numbers.
Explain your program in your essay, with screenshots. Include a paragraph on the random number generator used in Visual Basic, and answer the question: how should the random number generator be best seeded?"
First of all, Welcome to StackOverflow,
From what i understand from your post you want to generate 1000 8 Character long Unique ID's
First of all we declare our variables
Dim Characters As String = "AZERTYUIOPQSDFGHJKLMWXCVBN0123456789" 'Declare All Characters on one string
Dim IDsList As New List(Of String) With {.Capacity = 1000} 'You can limit the capacity to 1000
Dim Id As String = ""
Dim Rand As New Random
Now we begin to genearte ID's and add them to IDsList using a nested loop
For i As Integer = 0 To 999 'This is a Zero Based Index so 999 is actually 1000
For _i As Integer = 0 To 7 'This also means 7 is 8
Id += Characters(Rand.Next(0, 36))
Next
IDsList.Add(Id)
Id = ""
Next
'You can Check for duplicates by adding this piece of code or use the next one instead
Dim NoDupesIDsList As New List(Of String)
NoDupesIDsList = IDsList.Distinct.ToList
For i As Integer = 0 To 999 'This is a Zero Based Index so 999 is actually 1000
For _i As Integer = 0 To 7 'This also means 7 is 8
Id += Characters(Rand.Next(0, 36))
Next
If not IDsList.contains(Id) then
IDsList.Add(Id)
End if
Id = ""
Next
Use a Textbox with multiline and scroll bar for better experience
Feel free to ask me anything via comments
PS: I tested my method and it's working , enjoy it !
This function will return the ASCII characters between 48 an 122. That will include some characters that are not letters or numbers but this makes the ids even more unique and simplifies the code.
The For loop builds the ids with a length of 8. Then the id is checked for uniqueness with the .Contains method. Only if it is not in the list, it is added to the list and the counter is incremented.
Private Rnd As New Random
Private Function GetListOfIDs() As List(Of String)
Dim Counter As Integer
Dim lst As New List(Of String)
Dim id As String = ""
Do While Counter < 1000
id = ""
For i = 0 To 7
id &= Chr(Rnd.Next(48, 123))
Next
If Not lst.Contains(id) Then
lst.Add(id)
Counter += 1
End If
Loop
Return lst
End Function
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim lst = GetListOfIDs()
TextBox1.Text = String.Join(vbCrLf, lst)
End Sub
EDIT
As per suggestion by Caius Jard in comments I have added a solution using a HashSet. Same idea only a HashSet will not accept duplicate entries. The .Add method returns a boolean telling you if the Add was successful.
Private Function GetListOfIDs() As HashSet(Of String)
Dim hs As New HashSet(Of String)
Dim id As String = ""
Do While hs.Count < 1001
id = ""
For i = 0 To 7
id &= Chr(Rnd.Next(48, 123))
Next
If hs.Add(id) Then
Counter += 1
End If
Loop
Return hs
End Function
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim hs = BuildHashSet()
TextBox1.Text = String.Join(vbCrLf, hs)
End Sub
This method is probably more efficient because a HashSet is designed for hight performance. For 1000 items I did not notice any difference in the user interface. The text box updated immediately with both methods.
This will create the names:
Function RandomName(length As Integer) As String
Shared rnd As New Random()
Shared corpus() As Char = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789".ToCharArray()
Return New String(Enumerable.Range(0, length).Select(Function(c) corpus(rnd.Next(corpus.Length))).ToArray())
End Function
Iterator Function GetUniqueNames(length As Integer, count As Integer) As IEnumerable(Of String)
Dim names As New HashSet(Of String)
For i As Integer = 0 To count
Dim name As String = RandomName(length)
While names.Contains(name)
'Potential (but unlikely) halting problem
name = RandomName(length)
End While
names.Add(name)
Yield name
Next
End Function
Dim uniqueNames = GetUniqueNames(8, 1000)
But do you really want to display 1000 of them in a single label?
lblRandomId.Text = string.Join(vbCrLf, uniqueNames)
Let's talk about your code.
Dim strChar As String = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
This assignment will work in your loop because a String is really an array of Char. This is why you can refer to an index in the string.
Dim rand As New Random()
The .net Random class provides a seed for Random based on system time. Only rarely would you want to provide a seed.
From the docs https://learn.microsoft.com/en-us/dotnet/api/system.random?view=netcore-3.1
CONSTRUCTORS
Random() Initializes a new instance of the Random class
using a default seed value.
Random(Int32) Initializes a new instance of the Random class, using
the specified seed value.
For count_ids As Integer = 0 To 999
A For loop is not appropriate for this problem. We only want to increment the counter if we get a unique ID. The above For loop will increment on every iteration.
Do While Counter < 1000
'Code here
Loop
Notice in my code the counter is only incremented if the ID is unique.
The following loop should work fine but above this code we need to reset strID to an empty string. Otherwise we will have a very long string.
For count_chars As Integer = 0 To 7
strID += strChar(rand.Next(0, 62))
Next count_chars
You should not update the user interface (the label) on each iteration of the loop. Repainting the screen is one of the slowest operations in code. This line will overwrite the string in the label on every iteration.
lblRandomId.Text = strID
Move the update of the UI to after both loops are complete
Your code never checks for duplicates.
Now let's fix it.
We use a List(Of T) to accumulate the IDs. The T stands for Type; in our case it is String. A List is like an array only you don't have to know ahead of time how many entries there will be. No ReDim Preserve required.
Moved Dim strID As String = "" to the beginning of the inner loop so we will get a new strID on each iteration of the outer loop.
Changed strID &= strChar(rand.Next(0, 62)). &= instead of +=. In VB.net we use the ampersand to concatenate strings although the plus sign will work. The .Next method of the Random class has an overload that takes 2 Integers. It is inclusive of the first and exclusive of the second which means (0, 62) will return numbers form 0 to 61. Your string with a length of 62 has indexes of 0 to 61.
Next we check for duplicates with the .Contains method of the List class. This method does just what you would expect. We pass the new ID to the method and it returns a Boolean telling if the string is already in the list.
If it is not in the list, we add it to the list and increment the counter. The counter is only incremented if we have a successful add.
Finally we update the user interface by using a method of the String class. the .Join method takes a separator and the list to join. The TextBox must be multiline and have scroll bars.
I changed the names of the controls so I could test it in my testing program.
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim lst As New List(Of String)
Dim strChar As String = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
Dim rand As New Random()
Dim counter As Integer
Do While counter < 1000
Dim strID As String = ""
For count_chars = 0 To 7
strID &= strChar(rand.Next(0, 62))
Next
If Not lst.Contains(strID) Then
lst.Add(strID)
counter += 1
End If
Loop
TextBox1.Text = String.Join(vbCrLf, lst)
End Sub
EDIT
The official documentation for this method can be found here.
https://learn.microsoft.com/en-us/dotnet/api/system.string.join?view=netcore-3.1
I know it will be difficult to read but just take a look. Eventually you will be able to understand it and will be using these docs a lot.
String.Join(separator, List(Of String))
The separator is what to put between the strings in the list. In our case that is vbCrLf which is a vb constant for new line. Cr = Carraiage Return and Lf = Line Feed. Other types of collections that contain strings can also use this method.
The method takes each string in the list and adds the separator. After it is done with the list it returns one long string which we display in the TextBox.
I would like all possible combinations from two string arrays.
Both arrays must have same length.
Result must keep order
For example :
dim lStr1() as string = {"One", "Two", "Three"}
dim lStr2() as string = {"EditOne", "EditTwo", "EditThree"}
dim res() as string = myAwesomeFunction(lStr1, lStr2)
// res :
One Two Three
One Two EditThree
One EditTwo Three
One EditTwo EditThree
EditOne Two Three
EditOne Two EditThree
EditOne EditTwo Three
EditOne EditTwo EditThree
It's like the binary composition of 2 arrays of strings.
Here's another solution. Since only 2 arrays are involved, we can bit-fiddle to get all of the "combinations". The & " " is just to format the output to match the example.
Private Function myAwesomeFunction(Array1() As String, Array2() As String) As String()
If Array1.Length <> Array2.Length Then
Throw New ArgumentException("Array lengths must be equal.")
End If
Dim combos(CInt(2 ^ Array1.Length) - 1) As String
For i As Integer = 0 To combos.Count - 1
For j = 0 To Array1.Length - 1
If (i And (1 << j)) > 0 Then
combos(i) += Array2(j) & " "
Else
combos(i) += Array1(j) & " "
End If
Next
Next
Return combos
End Function
The following code will produce the array in your example. It should work for any pair of input arrays. The function checks that the input arrays are of the same length.
The GetPermutations function is taken from a more general class I use for generating permutations of numbers. It returns arrays of total Integers between 0 and choose - 1, and being an Iterator function, it returns the next array each time it is called.
In order to match your example, I returned an array of String where each element is a single string consisting of each of the selected strings separated by spaces. You may find it more useful to return a List(Of String()) or even a List(Of List(Of String))
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim lStr1() As String = {"One", "Two", "Three"}
Dim lStr2() As String = {"EditOne", "EditTwo", "EditThree"}
Dim res() As String = myAwesomeFunction(lStr1, lStr2)
End Sub
Function MyAwesomeFunction(lStr1() As String, lStr2() As String) As String()
Dim combos As New List(Of String)
If lStr1.Length <> lStr2.Length Then Throw New ArgumentException("Arrays must have the same length")
For Each combo() As Integer In GetPermutations(lStr1.Length, 2)
Dim elem As New List(Of String)
For i As Integer = 0 To combo.Length - 1
elem.Add(If(combo(i) = 0, lStr1(i), lStr2(i)))
Next
combos.Add(String.Join(" ", elem))
Next
Return combos.ToArray
End Function
Public Iterator Function GetPermutations(choose As Integer, total As Integer) As IEnumerable(Of Integer())
Dim totals() As Integer = Enumerable.Repeat(Of Integer)(total, choose).ToArray
Dim value(choose - 1) As Integer
Do
Yield value
For index As Integer = choose - 1 To 0 Step -1
value(index) += 1
If value(index) < totals(index) Then Continue Do
value(index) = 0
Next
Exit Do
Loop
End Function
the vb.net code below permutates a given word...the problem i have is that it does not accept larger words like "photosynthesis", "Calendar", etc but accepts smaller words like "book", "land", etc ...what is missing...Pls help
Module Module1
Sub Main()
Dim strInputString As String = String.Empty
Dim lstPermutations As List(Of String)
'Loop until exit character is read
While strInputString <> "x"
Console.Write("Please enter a string or x to exit: ")
strInputString = Console.ReadLine()
If strInputString = "x" Then
Continue While
End If
'Create a new list and append all possible permutations to it.
lstPermutations = New List(Of String)
Append(strInputString, lstPermutations)
'Sort and display list+stats
lstPermutations.Sort()
For Each strPermutation As String In lstPermutations
Console.WriteLine("Permutation: " + strPermutation)
Next
Console.WriteLine("Total: " + lstPermutations.Count.ToString)
Console.WriteLine("")
End While
End Sub
Public Sub Append(ByVal pString As String, ByRef pList As List(Of String))
Dim strInsertValue As String
Dim strBase As String
Dim strComposed As String
'Add the base string to the list if it doesn't exist
If pList.Contains(pString) = False Then
pList.Add(pString)
End If
'Iterate through every possible set of characters
For intLoop As Integer = 1 To pString.Length - 1
'we need to slide and call an interative function.
For intInnerLoop As Integer = 0 To pString.Length - intLoop
'Get a base insert value, example (a,ab,abc)
strInsertValue = pString.Substring(intInnerLoop, intLoop)
'Remove the base insert value from the string eg (bcd,cd,d)
strBase = pString.Remove(intInnerLoop, intLoop)
'insert the value from the string into spot and check
For intCharLoop As Integer = 0 To strBase.Length - 1
strComposed = strBase.Insert(intCharLoop, strInsertValue)
If pList.Contains(strComposed) = False Then
pList.Add(strComposed)
'Call the same function to review any sub-permutations.
Append(strComposed, pList)
End If
Next
Next
Next
End Sub
End Module
Without actually creating a project to run this code, nor knowing how it 'doesn't accept' long words, my answer would be that there are a lot of permutations for long words and your program is just taking much longer than you're expecting to run. So you probably think it has crashed.
UPDATE:
The problem is the recursion, it's blowing up the stack. You'll have to rewrite your code to use an iteration instead of recursion. Generally explained here
http://www.refactoring.com/catalog/replaceRecursionWithIteration.html
Psuedo code here uses iteration instead of recursion
Generate list of all possible permutations of a string
Hi I am new to VB and I have problems with using array. My code is like that.
This is class FindFactorsObject.vb
Public Sub FindFactors()
count = 0
temp = Convert.ToInt32(Math.Sqrt(_x))
For i As Integer = 1 To temp
If _x Mod i = 0 Then
ReDim array(count)
array(count) = i
count += 1
End If
Next
So I created an array and stored the results. Now I want to display each value in my array in Form.vb and if it is possible can somebody teach me how to make delay for each of the displayed value. Thanks very much
Always declare your variables, if possible to their exact types, you think they will take care of. When you say 'Now I want to display each value in my array in Form.vb' I understood literally: in the Form so, we will print them on your form
Public Sub FindFactors(_x As Integer)
Dim temp As Integer = Convert.ToInt32(Math.Sqrt(_x))
Dim l As New List(Of Integer)
For i As Integer = 1 To temp
If _x Mod i = 0 Then
l.add(i)
End If
Next
Dim pf As New PointF(20, 20)
For Each i As Integer In l
creategraphics.drawstring(i.ToString, New font(font.fontFamily, 24), brushes.cadetblue, pf)
pf = New PointF(pf.X, pf.Y + 30)
Next
End Sub
There are some good examples on how to calculate word frequencies in C#, but none of them are comprehensive and I really need one in VB.NET.
My current approach is limited to one word per frequency count. What is the best way to change this so that I can get a completely accurate word frequency listing?
wordFreq = New Hashtable()
Dim words As String() = Regex.Split(inputText, "(\W)")
For i As Integer = 0 To words.Length - 1
If words(i) <> "" Then
Dim realWord As Boolean = True
For j As Integer = 0 To words(i).Length - 1
If Char.IsLetter(words(i).Chars(j)) = False Then
realWord = False
End If
Next j
If realWord = True Then
If wordFreq.Contains(words(i).ToLower()) Then
wordFreq(words(i).ToLower()) += 1
Else
wordFreq.Add(words(i).ToLower, 1)
End If
End If
End If
Next
Me.wordCount = New SortedList
For Each de As DictionaryEntry In wordFreq
If wordCount.ContainsKey(de.Value) = False Then
wordCount.Add(de.Value, de.Key)
End If
Next
I'd prefer an actual code snippet, but generic 'oh yeah...use this and run that' would work as well.
This might be what your looking for:
Dim Words = "Hello World ))))) This is a test Hello World"
Dim CountTheWords = From str In Words.Split(" ") _
Where Char.IsLetter(str) _
Group By str Into Count()
I have just tested it and it does work
EDIT! I have added code to make sure that it counts only letters and not symbols.
FYI: I found an article on how to use LINQ and target 2.0, its a feels bit dirty but it might help someone http://weblogs.asp.net/fmarguerie/archive/2007/09/05/linq-support-on-net-2-0.aspx
Public Class CountWords
Public Function WordCount(ByVal str As String) As Dictionary(Of String, Integer)
Dim ret As Dictionary(Of String, Integer) = New Dictionary(Of String, Integer)
Dim word As String = ""
Dim add As Boolean = True
Dim ch As Char
str = str.ToLower
For index As Integer = 1 To str.Length - 1 Step index + 1
ch = str(index)
If Char.IsLetter(ch) Then
add = True
word += ch
ElseIf add And word.Length Then
If Not ret.ContainsKey(word) Then
ret(word) = 1
Else
ret(word) += 1
End If
word = ""
End If
Next
Return ret
End Function
End Class
Then for a quick demo application, create a winforms app with one multiline textbox called InputBox, one listview called OutputList and one button called CountBtn. In the list view create two columns - "Word" and "Freq." Select the "details" list type. Add an event handler for CountBtn. Then use this code:
Imports System.Windows.Forms.ListViewItem
Public Class MainForm
Private WordCounts As CountWords = New CountWords
Private Sub CountBtn_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles CountBtn.Click
OutputList.Items.Clear()
Dim ret As Dictionary(Of String, Integer) = Me.WordCounts.WordCount(InputBox.Text)
For Each item As String In ret.Keys
Dim litem As ListViewItem = New ListViewItem
litem.Text = item
Dim csitem As ListViewSubItem = New ListViewSubItem(litem, ret.Item(item).ToString())
litem.SubItems.Add(csitem)
OutputList.Items.Add(litem)
Word.Width = -1
Freq.Width = -1
Next
End Sub
End Class
You did a terrible terrible thing to make me write this in VB and I will never forgive you.
:p
Good luck!
EDIT
Fixed blank string bug and case bug
This might be helpful:
Word frequency algorithm for natural language processing
Pretty close, but \w+ is a good regex to match with (matches word characters only).
Public Function CountWords(ByVal inputText as String) As Dictionary(Of String, Integer)
Dim frequency As New Dictionary(Of String, Integer)
For Each wordMatch as Match in Regex.Match(inputText, "\w+")
If frequency.ContainsKey(wordMatch.Value.ToLower()) Then
frequency(wordMatch.Value.ToLower()) += 1
Else
frequency.Add(wordMatch.Value.ToLower(), 1)
End If
Next
Return frequency
End Function