Sort 3 x one dimensional arrays in parallell - vb.net

I have 3 x one dimensional arrays of type integer, each array is the same length (between 5 - 20 million values) and I would like to sort them in parallel (ie keeping the relative position) by the second array, then the third array, then the first array. Does anyone have any ideas on the most efficient way to do this in vb.net?
If it helps, the first array is just a record of the initial position (the three arrays will be re-sorted to this order once various calculations are done). I need to sort them to determine the number of unique combinations of the second and third arrays (a combination is determined by the position in the array - in the example below the combinations are (0-1-4), (1-1-6) etc). Once that is determined I will resort them back based on the first array.
I looked at array.sort but that only covers 2 parallel arrays. I'm a bit wary of putting the values in tuples (or any other format), as it would (I assume - perhaps not) be a big overhead when converting 5 - 20 million records before processing.
For example:
Record Number Array: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
Link Array: {1, 1, 2, 1, 1, 2, 2, 2, 1, 2}
Line Number Array: {4, 6, 3, 5, 6, 7, 3, 2, 3, 4}
Having sorted / ordered by the second, then third, then first array, the expected output would be:
Record Number (1st Array): {8, 0, 3, 1, 4, 7, 2, 6, 9, 5}
Link Array (2nd Array): {1, 1, 1, 1, 1, 2, 2, 2, 2, 2}
Line Number Array (3rd Array): {3, 4, 5, 6, 6, 2, 3, 3, 4, 7}
Array.sort only allows you to sort 2 arrays in parallel and i'm a little confused by the options available in LinQ.
Does anyone have any suggestions on the best way to solve this problem?
Cheers,

I think you can get a backup of your Link Array. Then sort Link Array with Record Array, then you can sort un-sorted Link Array with Line Number Array. Can you please try it?

You didn't say if you want efficient in terms of time or space, but this takes about 3 seconds to transfer 20 million records from arrays to a list, and then about 30 seconds to do the sort on them, using less than 1GB of RAM. On my computer.
Option Infer On
Module Module1
Class Grouped
Property RecNo As Integer
Property Link As Integer
Property LineNo As Integer
Public Overrides Function ToString() As String
Return String.Format("({0}, {1}, {2})", Me.RecNo, Me.Link, Me.LineNo)
End Function
End Class
Sub Main()
' First try with the test data to show the correct result is obtained
Dim recNos = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
Dim links = {1, 1, 2, 1, 1, 2, 2, 2, 1, 2}
Dim lineNos = {4, 6, 3, 5, 6, 7, 3, 2, 3, 4}
' transfer the arrays to a List
Dim xs As New List(Of Grouped)
xs.Capacity = recNos.Length
For i = 0 To recNos.Length() - 1
xs.Add(New Grouped With {.RecNo = recNos(i), .Link = links(i), .LineNo = lineNos(i)})
Next
' sort the data
Dim ys = xs.OrderBy(Function(x) x.Link).ThenBy(Function(x) x.LineNo).ToList()
Console.WriteLine(String.Join(", ", ys))
' Now try with twenty million records
Dim rand = New Random()
Dim nRecs As Integer = 20000000
recNos = Enumerable.Range(0, nRecs - 1).ToArray()
ReDim links(nRecs - 1)
ReDim lineNos(nRecs - 1)
For i = 0 To nRecs - 1
links(i) = rand.Next(0, 9)
lineNos(i) = rand.Next(1, 9)
Next
Dim sw As New Stopwatch
sw.Start()
xs.Clear()
xs.Capacity = nRecs
For i = 0 To recNos.Length() - 1
xs.Add(New Grouped With {.RecNo = recNos(i), .Link = links(i), .LineNo = lineNos(i)})
Next
sw.Stop()
Console.WriteLine(sw.ElapsedMilliseconds.ToString())
sw.Restart()
ys = xs.OrderBy(Function(x) x.Link).ThenBy(Function(x) x.LineNo).ToList()
sw.Stop()
Console.WriteLine(sw.ElapsedMilliseconds.ToString())
Console.ReadLine()
End Sub
End Module

Related

Is there a way to sort a list of numbers whilst maintaining their original indexes in VB.net?

I need to sort a list of numbers but I need to keep their initial indexes.
I had previously just created an array of the numbers and then another array of the indexes which I sorted at the same time like so:
Dim AverageSuccess(23) As Decimal
Dim intervalList() As Integer = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}
x = 2
Do
Sorted = True
For i = 23 To x Step -1
If AverageSuccess(i) < AverageSuccess(i - 1) Then
TempNum = AverageSuccess(i)
AverageSuccess(i) = AverageSuccess(i - 1)
AverageSuccess(i - 1) = TempNum
TempIndex = intervalList(i)
intervalList(i) = intervalList(i - 1)
intervalList(i - 1) = TempIndex
Sorted = False
End If
Next
x += 1
Loop Until Sorted
however this is for a project and my teacher informed me that this is bad programming practise and I should be using a list instead.
I have struggled to find a simple example of how to use a list in VB.net for this purpose, so if someone could give me an example I would appreciate it.
I don't know how much you have covered about lists in class...
Let us create a list with some integers in:
Dim nums As New List(Of Integer) From {9, 8, 4, 5}
Now, we want to store the original indices of those numbers. We can do that with the Select method, which has an optional parameter that will give the index of the current item, and create a new entity with items which we can give names to, say "Num" and "Idx":
Dim numsWithIndex = nums.Select(Function(n, i) New With {.Num = n, .Idx = i})
Then we can use the LINQ method OrderBy to get those entities in the desired order:
Dim sortedNums = numsWithIndex.OrderBy(Function(nwi) nwi.Num)
And we can have a look at what we have constructed with
Console.WriteLine(String.Join(vbCrLf, sortedNums))
which outputs:
{ Num = 4, Idx = 2 }
{ Num = 5, Idx = 3 }
{ Num = 8, Idx = 1 }
{ Num = 9, Idx = 0 }
(It shows the names we gave to the properties of the anonymous type created with New earlier.)
Here is the whole thing as a console app you can copy-and-paste to investigate with on your computer:
Module Module1
Sub Main()
Dim nums As New List(Of Integer) From {9, 8, 4, 5}
Dim numsWithIndex = nums.Select(Function(n, i) New With {.Num = n, .Idx = i})
Dim sortedNums = numsWithIndex.OrderBy(Function(nwi) nwi.Num)
Console.WriteLine(String.Join(vbCrLf, sortedNums))
Console.ReadLine()
End Sub
End Module

Assign an integer variable to an integer variable and change name dynamic

Dim LastNumber as Integer = 1
Dim num_0() as Integer = {1, 2, 3, 4, 5}
Dim num_1() as Integer = {6, 7, 8, 9, 10}
Dim num_2() as Integer = {20, 21, 14, 36, 0}
Dim y() As Integer
y(0) = num_0(2)
When I use this code it executes perfectly
But the problem is I want to change the "0" in num_0(2)
When I do...
y(0) = num_ & LastNumber & (2)
This doesnt work
Or
y(0) = ("num_" & LastNumber & "(2)")
This gives me an error that converting a string to an integer is not possible
My question is How can I replace the "0" in num_0(2) with the LastNumber integer variable... so it reads the "8" out of the array num_1(2)
You can use multidimensional array (AKA rectangular array):
Dim num As Integer(,) = { {1, 2, 3, 4, 5}, {6, 7, 8, 9, 10}, {20, 21, 14, 36, 0} }
y(0) = num(LastNumber, 2)
or jagged array (array of arrays) :
Dim num As Integer()() = { ({1, 2, 3, 4, 5}), ({6, 7, 8, 9, 10}), ({20, 21, 14, 36, 0}) }
y(0) = num(LastNumber)(2)

VB.NET - How to calculate the parity bit of a byte array

What is the most efficient way calculate the parity bit (if the number of active bits are odd or even) in a byte array? I have though about iterating through all the bits and summing up the active bits, but that would be very impractical purely based on the number of iterations required on larger byte arrays/files.
For your convenience (and my curiosity), I have done some timing tests with a parity lookup table compared to the other two methods suggested so far:
Module Module1
Dim rand As New Random
Dim parityLookup(255) As Integer
Sub SetUpParityLookup()
' setBitsCount data from http://stackoverflow.com/questions/109023/how-to-count-the-number-of-set-bits-in-a-32-bit-integer
Dim setBitsCount = {
0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4,
1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5,
1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5,
2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,
1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5,
2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,
2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,
3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7,
1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5,
2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,
2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,
3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7,
2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,
3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7,
3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7,
4, 5, 5, 6, 5, 6, 6, 7, 5, 6, 6, 7, 6, 7, 7, 8
}
For i = 0 To 255
parityLookup(i) = setBitsCount(i) And 1
Next
End Sub
' Method using lookup table
Function ParityOfArray(a() As Byte) As Integer
Dim parity As Integer = 0 ' use an Integer because they are faster
For i = 0 To a.Length - 1
parity = parity Xor parityLookup(a(i))
Next
Return parity
End Function
' Method by Alireza
Function ComputeParity(bytes() As Byte) As Byte
Dim parity As Boolean = False
For i As Integer = 0 To bytes.Length - 1
Dim b As Byte = bytes(i)
While b <> 0
parity = Not parity
b = CByte(b And (b - 1))
End While
Next
Return Convert.ToByte(parity)
End Function
' Method by dbasnett
Function CountBits(byteArray As Byte()) As Integer
Dim rv As Integer = 0
For Each b As Byte In byteArray
Dim count As Integer = b
count = ((count >> 1) And &H55) + (count And &H55)
count = ((count >> 2) And &H33) + (count And &H33)
count = ((count >> 4) And &HF) + (count And &HF)
rv += count
Next
Return rv
End Function
Sub FillWithRandomBytes(ByRef a() As Byte)
rand.NextBytes(a)
End Sub
Sub Main()
SetUpParityLookup()
Dim nBytes = 10000
Dim a(nBytes - 1) As Byte
FillWithRandomBytes(a)
Dim p As Integer
Dim sw As New Stopwatch
sw.Start()
p = ParityOfArray(a)
sw.Stop()
Console.WriteLine("ParityOfArray - Parity: {0} Time: {1}", p, sw.ElapsedTicks)
sw.Restart()
p = ComputeParity(a)
sw.Stop()
Console.WriteLine("ComputeParity - Parity: {0} Time: {1}", p, sw.ElapsedTicks)
sw.Restart()
p = CountBits(a)
sw.Stop()
' Note that the value returned from CountBits should be And-ed with 1.
Console.WriteLine("CountBits - Parity: {0} Time: {1}", p And 1, sw.ElapsedTicks)
Console.ReadLine()
End Sub
End Module
Typical ouput:
ParityOfArray - Parity: 0 Time: 386
ComputeParity - Parity: 0 Time: 1014
CountBits - Parity: 0 Time: 695
An efficient way to do this is to use the x & (x - 1) operation in a loop, until x becomes zero. This way you will loop only by the number of bits set to 1.
In VB.NET for a byte array:
Function ComputeParity(bytes() As Byte) As Byte
Dim parity As Boolean = False
For i As Integer = 0 To bytes.Length - 1
Dim b As Byte = bytes(i)
While b <> 0
parity = Not parity
b = b And (b - 1)
End While
Next
Return Convert.ToByte(parity)
End Function
Here is a function that counts bits.
Private Function CountBits(byteArray As Byte()) As Integer
Dim rv As Integer = 0
For x As Integer = 0 To byteArray.Length - 1
Dim b As Byte = byteArray(x)
Dim count As Integer = b
count = ((count >> 1) And &H55) + (count And &H55)
count = ((count >> 2) And &H33) + (count And &H33)
count = ((count >> 4) And &HF) + (count And &HF)
rv += count
Next
Return rv
End Function
Note: this code came from a collection of bit twiddling hacks I found some years ago. I converted it to VB.

Unique Combination of Set

I'm trying to write vb.net code to return unique combinations for set
My set contains 3 different elements. I've found similar post this Post but couldn't find any VB solution to get this results
Example:
Elements: 1, 2, 3
{ 1, 2, 3}
Result must be
1
2
3
12
13
23
123
...........
>...................
i,m trying to achieve this by using following code
Function GetCombinations(ByVal depth As Integer, ByVal values As String()) As IEnumerable(Of String)
If depth > values.Count + 1 Then Return New List(Of String)
Dim result = New List(Of String)
For i = 0 To depth - 1
For y = 0 To values.Count - 1
If i = 0 Then
result.Add(values(y))
Else
result.Add(values(i - 1) + values(y))
End If
Next
Next
Return result
End Function
To Get the result
Dim reslt = GetCombinations(4, data_array)
?reslt
Count = 12
(0): "1"
(1): "2"
(2): "3"
(3): "11"
(4): "12"
(5): "13"
(6): "21"
(7): "22"
(8): "23"
(9): "31"
(10): "32"
(11): "33"
Hint:
I work with Mathematics and manage to calculate no of combinations. i can test the out with
this formula
Example
there is this formula called nCr. it means out of n number of element how many way of taking r number of elements with unique combinations of r.
nPr = n!/(n-r)!
n! = 1 * 2 * 3 * 4* ... (n-1) * n
Elements: 1, 2, 3
In this case n = 3 and r can be 1, 2, and 3 all
number of combinations = 3P1 + 3P2 + 3P3
= 3!/2! + 3!/1! + 3!/0!
= 6/2 + 6/1 + 6/1 (0!=1)
= 3+6+6
= 15
Knowing the term makes it easier to find existing algorithms. What you're looking for is a power set. Here's a quick VB.NET translation of a C# implementation I found on Rosetta Code:
Public Function GetPowerSet(Of T)(ByVal input As IEnumerable(Of T)) As IEnumerable(Of IEnumerable(Of T))
Dim seed As IEnumerable(Of IEnumerable(Of T)) = {Enumerable.Empty(Of T)()}
Return input.Aggregate(seed, Function(a, b) a.Concat(a.Select(Function(x) x.Concat({b}))))
End Function
Testing:
For Each x In GetPowerSet({1, 2, 3})
Console.WriteLine(String.Join(", ", x))
Next
Output:
1
2
1, 2
3
1, 3
2, 3
1, 2, 3
EDIT - Based on your latest explanation, I think you need a different approach. It seems you want combinations with repetitions / replacement, for all sizes up to the input size. You could simply call one of those algorithms with parameters (S, k) for each value of k from 1 to n and join all the results into a single set of results.
Translating Python's algorithm:
Public Iterator Function GetCombinationsWithReplacement(Of T)(source As IEnumerable(Of T), size As Integer) As IEnumerable(Of IEnumerable(Of T))
Dim pool = source.ToList()
Dim n = pool.Count
If n = 0 AndAlso size > 0 Then
Return
End If
Dim indices = Enumerable.Repeat(0, size).ToList()
Yield indices.Select(Function(i) pool.Item(i))
While True
Dim index As Nullable(Of Integer) = Nothing
For i = size - 1 To 0 Step -1
If indices.Item(i) <> n - 1 Then
index = i
Exit For
End If
Next
If Not index.HasValue Then
Return
End If
indices = indices.Take(index.Value).Concat(Enumerable.Repeat(indices.Item(index.Value) + 1, size - index.Value)).ToList()
Yield indices.Select(Function(i) pool.Item(i))
End While
End Function
(You will need to modify this if your VB.NET compiler doesn't support Yield.)
The results of calling this with different sizes are:
GetCombinationsWithReplacement({1, 2, 3}, 1):
{1}
{2}
{3}
GetCombinationsWithReplacement({1, 2, 3}, 2):
{1, 1}
{1, 2}
{1, 3}
{2, 2}
{2, 3}
{3, 3}
GetCombinationsWithReplacement({1, 2, 3}, 3):
{1, 1, 1}
{1, 1, 2}
{1, 1, 3}
{1, 2, 2}
{1, 2, 3}
{1, 3, 3}
{2, 2, 2}
{2, 2, 3}
{2, 3, 3}
{3, 3, 3}
We can join these into a single sequence with all 19 subsets:
Public Iterator Function GetCombinationsWithReplacementAllSizes(Of T)(source As IEnumerable(Of T)) As IEnumerable(Of IEnumerable(Of T))
Dim pool = source.ToList()
For size = 1 To pool.Count
For Each subset In GetCombinationsWithReplacement(pool, size)
Yield subset
Next
Next
End Function
Here is some pseudo code for any number of elements to help you (I make no claim this is the fastest way to do this, it is just a way to do it.)
Given elementList is a list of elements given
Given tempList is a list as a temporary holder
Given resultList is a list of the result
loop by # items in elementlist
{
if tempList is empty // special case for first iteration
add each element of elementList to tempList and resultlist
else
{
for each element in templist
for each element2 in elementlist
add combo to result list
copy elements added to result list for this iteration to templist
}
}

Shifting Byte Array

If have an array of Bytes in visual basic:
Dim data() As Byte = {0, 128, 0, 4, 9, 9, 32, 0, 0, 0, 0, 0, 0, 0, 0}
Is there a quick and easy way to insert two data values to the front of this array, and knock off the last two values?
Dim data() As Byte = {128, 128, 0, 128, 0, 4, 9, 9, 32, 0, 0, 0, 0, 0, 0}
Yes. First you need to move all of the existing values up 2 places in your array. Doing so will overwrite the last 2 values. You'll then want to set the first two values of your array.
'Move data up 2 spots. This needs to be done in reverse order so we don't lose any data
For i as Integer = data.Length - 1 To 2 Step -1
data(i) = data(i - 2)
End
'Assign the new values
data(0) = 128
data(1) = 128
You could load the bytes into a vb.net stack with a loop then use the stack.push then rewrite the data back