Lempel-Ziv-Welch algorithm - vb.net

Heys guys,
I'm trying to implement the Lempel-Ziv-Welch algorithm on byte level in VB.NET. Here is what the Code looks like so far:
Dim FileBytes As Byte() = {Convert.ToByte(12), Convert.ToByte(13), Convert.ToByte(12), Convert.ToByte(13), Convert.ToByte(12), Convert.ToByte(13)}
Dim bw As New BitWriter(New BinaryWriter(New FileStream(output, FileMode.Create)))
Dim ByteDictionary As New Dictionary(Of Byte(), Integer)
Dim DictionaryN As Integer = 1
Dim DictionaryMax As Integer = 4048
Dim outputs As New List(Of Integer)
Dim bts As New List(Of Byte)
For index As Integer = 0 To FileBytes.Length - 1
Dim currentbyte As Byte = FileBytes(index)
If ContainsByte(ByteDictionary, AddToList(bts, currentbyte).ToArray) Then
bts.Add(currentbyte)
Else
If bts.Count > 1 Then
ByteDictionary.Add(bts.ToArray, 255 + DictionaryN)
DictionaryN += 1
Else
ByteDictionary.Add(New Byte() {currentbyte}, currentbyte)
End If
Console.WriteLine(GetByteValue(ByteDictionary, bts.ToArray))
bts.Clear()
bts.Add(currentbyte)
End If
Next
End Sub
Public Function ContainsByte(ByVal dic As Dictionary(Of Byte(), Integer), ByVal bt As Byte()) As Boolean
Dim flag = True
For Each kp As KeyValuePair(Of Byte(), Integer) In dic
If ByteArrayEquals(kp.Key, bt) Then
Return True
End If
Next
Return False
End Function
Public Function AddToList(ByVal list As List(Of Byte), ByVal bt As Byte) As List(Of Byte)
Dim newlist = New List(Of Byte)(list)
newlist.Add(bt)
Return newlist
End Function
Public Function ByteArrayEquals(ByVal first As Byte(), ByVal second As Byte()) As Boolean
If first.Length = second.Length Then
Dim flag = True
For index As Integer = 0 To first.Length - 1
If first(index) <> second(index) Then
flag = False
End If
Next
Return flag
Else
Return False
End If
End Function
Public Function GetByteValue(ByVal dic As Dictionary(Of Byte(), Integer), ByVal bt As Byte()) As Integer
For Each kp As KeyValuePair(Of Byte(), Integer) In dic
If ByteArrayEquals(kp.Key, bt) Then
Return kp.Value
End If
Next
End Function
The idea behind my implementation is from https://www.cs.cf.ac.uk/Dave/Multimedia/node214.html . But somehow it doesn't work, it simply puts out the input bytes. What is wrong with it?

The main problem is these two lines:
If bts.Count > 1 Then
ByteDictionary.Add(bts.ToArray, 255 + DictionaryN)
In both lines here you're using bts where you should be using bts + currentByte, i.e. you want something like
Dim currentbyte As Byte = FileBytes(index)
Dim btsPlusCurrentByte As List(Of Byte) = AddToList(bts, currentbyte)
If ContainsByte(ByteDictionary, btsPlusCurrentByte.ToArray) Then
bts.Add(currentbyte)
Else
If btsPlusCurrentByte.Count > 1 Then
ByteDictionary.Add(btsPlusCurrentByte.ToArray, 255 + DictionaryN)
The other problem is that you'll complete the loop with data left in bts that you haven't output, so I think you'll need a block to do that afterwards. It may be safe to just do
Console.WriteLine(GetByteValue(ByteDictionary, bts.ToArray))
again after the Next but I haven't thought about that very carefully.
I also think you should be able to use .NET's own built-ins rather than your four helper functions.

Related

Cross Correlation in .NET

I am working on an app that takes two audio channels and compares them to find the phase difference (delay). I came across this post;
Calculating FFT Correlation Coefficient
which refers to this sample code;
https://dotnetfiddle.net/1nWIgQ
I got the code working with correct results after translating it to VB.NET (its the language I started with years ago in this app).
The problem I see is that when I change the signal generated from random noise to a sine wave, then the code gives crazy unrelated results. Any suggestions would be appreciated.
Code below;
Imports System
Imports System.Collections.Generic
Imports System.Linq
Imports AForge.Math 'NuGet Package Aforge.Math module
Imports System.Runtime.InteropServices
Imports System.IO
Public Class Form2
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
Dim c = New Xcorr4_2()
Dim size As Integer = 2048
Dim delay As Double
delay = 2
Dim signal1 = c.GenerateSignal(size)
Dim signal2 = c.GenerateSignal(size, delay)
Dim signal1Complex = c.ToComplexWithPadding(signal1, 2)
Dim fftSignal1 = c.FFT(signal1Complex)
Dim signal2Complex = c.ToComplexWithPadding(signal2, 2)
Dim fftSignal2 = c.FFT(signal2Complex)
Dim cc = c.CorrelationCoefficient(fftSignal1.ToArray(), fftSignal2.ToArray())
MsgBox(cc.Item1) 'correlation coeff
MsgBox(cc.Item2) 'recovered delay
End Sub
End Class
Class Xcorr4_2
Public Function CrossCorrelation(ByVal ffta As Complex(), ByVal fftb As Complex()) As Complex()
Dim conj = ffta.[Select](Function(i) New Complex(i.Re, -i.Im)).ToArray()
conj = conj.Zip(fftb, Function(v1, v2) Complex.Multiply(v1, v2)).ToArray()
FourierTransform.FFT(conj, FourierTransform.Direction.Backward)
Dim rr As Double() = New Double(conj.Length - 1) {}
rr = conj.[Select](Function(i) i.Magnitude).ToArray()
Return conj
End Function
Public Function CorrelationCoefficient(ByVal ffta As Complex(), ByVal fftb As Complex()) As Tuple(Of Double, Integer)
Dim tuble As Tuple(Of Double, Integer)
Dim correlation = CrossCorrelation(ffta, fftb)
Dim seq = correlation.[Select](Function(i) i.Magnitude)
Dim maxCoeff = seq.Max()
Dim maxIndex As Integer = seq.ToList().IndexOf(maxCoeff)
tuble = New Tuple(Of Double, Integer)(maxCoeff, maxIndex)
Return tuble
End Function
Public Function FFT(ByVal signal As Complex()) As Complex()
FourierTransform.FFT(signal, FourierTransform.Direction.Forward)
Return signal
End Function
Public Function IFFT(ByVal signal As Complex()) As Complex()
FourierTransform.FFT(signal, FourierTransform.Direction.Backward)
Return signal
End Function
Public Function ToComplexWithPadding(ByVal sample As Double(), ByVal Optional padding As Integer = 1) As Complex()
Dim logLength As Double = Math.Ceiling(Math.Log(sample.Length * padding, 2.0))
Dim paddedLength As Integer = CInt(Math.Pow(2.0, Math.Min(Math.Max(1.0, logLength), 14.0)))
Dim complex As Complex() = New Complex(paddedLength - 1) {}
Dim samples = sample.ToArray()
Dim i As Integer = 0
While i < sample.Length
complex(i) = New Complex(samples(i), 0)
i += 1
End While
While i < paddedLength
complex(i) = New Complex(0, 0)
i += 1
End While
Return complex
End Function
Public Function GenerateSignal(ByVal size As Integer, ByVal Optional shift As Integer = 0) As Double()
Dim list As List(Of Double) = New List(Of Double)()
Dim generator = New AForge.Math.Random.StandardGenerator()
' Changed original random signal to Sine wave below
For i As Integer = 0 To size - 1
'Dim randomNumber As Double = generator.[Next]()
'list.Add(randomNumber)
list.Add(Math.Sin(2 * Math.PI / 200 * i + shift))
Next
Dim list2 As List(Of Double) = New List(Of Double)()
For i As Integer = 0 To shift - 1
list2.Add(0)
Next
Dim ar = list.ToArray()
For i As Integer = 0 To size - shift - 1
list2.Add(ar(i))
Next
Return list2.ToArray()
End Function
End Class
Below is the output for original code vs the modified code with sine wave.
Original code random signal
Modified with sine wave inputs

How do I get the value from a specific Key with Key Value Pair and VB.Net?

Public _SecurityLevel As List(Of KeyValuePair(Of Integer, Integer))
Public Sub New()
_SecurityLevel.Add(New KeyValuePair(Of Integer, Integer)(1, 0))
_SecurityLevel.Add(New KeyValuePair(Of Integer, Integer)(2, 1))
_SecurityLevel.Add(New KeyValuePair(Of Integer, Integer)(3, 2))
_SecurityLevel.Add(New KeyValuePair(Of Integer, Integer)(4, 3))
_SecurityLevel.Add(New KeyValuePair(Of Integer, Integer)(5, 4))
_SecurityLevel.Add(New KeyValuePair(Of Integer, Integer)(6, 5))
End Sub
I want to get the value (3) when I specify the Key (4).
I don't want to iterate through the entire KVP until I find it.
Is this possible? If not what should I be using?
You can get it by using LINQ:
Dim theValue As Integer = _SecurityLevel.Find(Function(kvp) kvp.Key = 4).Value
But I would recommend to use a Dictionary instead, if no duplicate keys should occur.
Dim _SecurityLevel As New Dictionary(Of Integer, Integer)
_SecurityLevel.Add(1, 0)
_SecurityLevel.Add(2, 1)
_SecurityLevel.Add(3, 2)
_SecurityLevel.Add(4, 3)
_SecurityLevel.Add(5, 4)
_SecurityLevel.Add(6, 5)
Dim result As Integer
_SecurityLevel.TryGetValue(4, result)
Edit
As stated in the comments, FindIndex would be the better joice here, as Find will return a default KeyValuePair with key/value (0,0) if no entry is found in the list.
Something like the following would be more reliable:
Dim index As Integer = _SecurityLevel.FindIndex(Function(kvp) kvp.Key = 4)
If index > -1 Then
MsgBox(_SecurityLevel(index).Value)
Else
'Not found
End If
A Dictionary seems more appropriate for your application.
Public Class SomeClass
Public _SecurityLevel As New Dictionary(Of Integer, Integer)
Public Sub New()
_SecurityLevel.Add(1, 0)
_SecurityLevel.Add(2, 1)
_SecurityLevel.Add(3, 2)
_SecurityLevel.Add(4, 3)
_SecurityLevel.Add(5, 4)
_SecurityLevel.Add(6, 5)
End Sub
End Class
Private Sub OPCode()
Dim sc As New SomeClass
Dim v = sc._SecurityLevel(4)
Debug.Print(v.ToString)
'Prints 3
End Sub
It would be unusual to have a public field in a class.
Try this:
Dim valueFromKey As Integer = (From currentValue In _SecurityLevel.AsEnumerable
Where currentValue.Key = 4
Select currentValue.Value).FirstOrDefault

How to most efficiently replace this code to vb.net version?

Public Function fileToColHarvest(ByRef stream As Scripting.TextStream, Optional ByRef limit As Integer = 2000000, Optional ByRef unique As Boolean = False, Optional ByRef FirstSectionAsKey As Boolean = False, Optional ByRef prob As Double = 1) As Generic.List(Of String)
Dim buffer As String
Dim i As Integer
If prob < 1 Then
End If
fileToColHarvest = New Generic.List(Of String)
Do While (Not (stream.AtEndOfStream))
i = i + 1
System.Windows.Forms.Application.DoEvents()
'If Microsoft.VisualBasic.Rnd < 0.01 Then
' appendToTextFile CStr(fileToColHarvest.Count) + "|" + microsoft.visualbasic.str(i) + "|" + buffer, RESULT, ForWriting
'End If
buffer = stream.ReadLine
'buffer = Microsoft.VisualBasic.Replace(buffer, " ", "+")
If Microsoft.VisualBasic.Rnd() < prob Then
If unique Then
If Not FirstSectionAsKey Then
fileToColHarvest.AddIfNotExist(buffer)
Else
fileToColHarvest.AddIfNotExist(buffer)
End If
Else
fileToColHarvest.Add(buffer)
End If
End If
If fileToColHarvest.Count() >= limit Then
Exit Do
End If
Loop
End Function
Basically I want to get rid Scripting.TextStream.
Also I want to read the text line by line
You can use StreamReader and asynchronous ReadLineAsync method.
Asynchronous approach will replace "ugly" Application.DoEvents()
Public Async Function FileToColHarvest(
pathToFile As String,
limit As Integer,
isUnique As Boolean,
isFirstSectionAsKey As Boolean,
prob As Single) As Task(Of List(Of String))
Dim lines = New List(Of String)()
Dim uniqueLines = New HashSet(Of String)()
Using stream As New FileStream(pathToFile, FileMode.Open)
Using reader As New StreamReader(stream)
While reader.EndOfStream = False
'Await will prevent blocking UI thread
var line = Await reader.ReadLineAsync()
If prob < VBMath.Rnd() Then Continue While
' I have removed check of isFirstSectionAsKey
' because based on your code it does same thing
If isUnique Then
uniqueLines.Add(line)
If uniqueLines.Count >= limit Then Return uniqueLines.ToList()
Else
lines.Add(line)
If lines.Count >= limit Then Return lines
End If
End While
End Using
End Using
Return If(isUnique, uniqueLines.ToList(), lines)
End Function
Not related but isUnique argument divide this method in two different logic - so I suggest instead of parameter introduce two different methods
FileToColHarvest(...)
FileToColHarvestWithUniqueOnly(...)

Filter a Dictionary to return a list

I know I can do this with a for loop cause that's how i'm doing it now. I was hoping for a more efficient way to accomplish the task.
I have a dictionary(Of Integer, Boolean)
or Of String, Boolean.
i want to get a list(of integer) or Of String from the dictionary where all the values are true(or false depending on what i need at the time)
and to generalize it or "black box" it, it could be any dictionary(of whatever, whatever)
and return a list(of whatever) where the value = whatever i'm looking for at the time.
string, string where value = "Closed"
in short: i want all list of all the keys who's value = some criteria
my current code:
Public Function FindInDict(Of tx, ty)(thedict As Dictionary(Of tx, ty), criteria As ty) As List(Of tx)
Dim tmpList As New List(Of tx)
For xloop As Integer = 0 To thedict.Count - 1
If CType(thedict(thedict.Keys(xloop)), ty).Equals(criteria) Then
tmpList.Add(thedict.Keys(xloop))
End If
Next
Return tmpList
End Function
You can do this easily with Linq:
Public Function FindInDict(Of tx, ty)(thedict As Dictionary(Of tx, ty), criteria As ty) As List(Of tx)
Return (From kvp In thedict
Where kvp.Value.Equals(criteria)
Select kvp.key).ToList()
End Function
Use LINQ, like so:
Dim tStorage As Dictionary(Of String, String) = New Dictionary(Of String, String)
Dim tKeys As List(Of String) = New List(Of String)
Dim tCriteria As List(Of String) = New List(Of String)
tStorage.Add("One", "Uno")
tStorage.Add("Two", "Dos")
tStorage.Add("Three", "Tres")
tStorage.Add("Four", "Quatro")
tCriteria.Add("Dos")
tCriteria.Add("Quatro")
tKeys = (From k In tStorage.Keys Where tCriteria.Contains(tStorage(k)) Select k).ToList
For Each tKey As String In tKeys
Console.WriteLine(tKey)
Next
Console.ReadKey()

How to retrieve just unread emails using pop3?

I'm using open source component to retrieve emails from my mail server using vb.net (pop3)
but because i have a lot of messages it gives me response Time out and i think if i just got the new messages it will make reading faster.
this is my code:
Dim popp As New Pop3Client("user#mail.com", "*******", "pop3.mail.com")
popp.AuthenticateMode = Pop3AuthenticateMode.Pop
popp.Port = 110
'popp.Ssl = True
popp.Authenticate()
Dim msglist As New List(Of String)
If popp.State = Pop3ConnectionState.Authenticated Then
Dim totalmsgs As Integer = popp.GetTotalMessageCount()
If totalmsgs > 0 Then
For index As Integer = 1 To totalmsgs
Dim msg As Pop3Message = popp.GetMessage(index)
msglist.Add(msg.Subject)
Next
popp.Close()
End If
End If
Return msglist
please i need some help if i'm using the component in a wrong way or if there is another component do what i'm looking for.
b.s. : my component name is "Higuchi.Mail.dll" or "OpenPOP.dll" and the two are same.
thanks
POP3 does not have the capibility to track whether messages are read or unread. I would suggest you set your limit to a finite number such as 50 or 100. Perhaps you could do some sort of pagination system.
This code needs to be within a function so that you can call it like so:
Sub Main
Dim start As Integer = Integer.parse(Request.QueryString("start"))
Dim count As Integer = Integer.parse(Request.QueryString("count"))
Dim subjects As New List(Of String)
subjects = getSubjects(start, count)
'Do whatever with the results...
'
End Sub
Function getSubjects(ByVal startItem As Integer, ByVal endItem as Integer) As List(Of String)
Dim popp As New Pop3Client("user#mail.com", "*******", "pop3.mail.com")
popp.AuthenticateMode = Pop3AuthenticateMode.Pop
popp.Port = 110
popp.Authenticate()
Dim msglist As New List(Of String)
If popp.State = Pop3ConnectionState.Authenticated Then
Dim totalmsgs As Integer = popp.GetTotalMessageCount()
Dim endItem As Integer = countItems + startItem
If endItem > totalmsgs Then
endItem = totalmsgs
End If
If totalmsgs > 0 Then
For index As Integer = startItem To endItem
Dim msg As Pop3Message = popp.GetMessage(index)
msglist.Add(msg.Subject)
Next
popp.Close()
End If
End If
Return msglist
End Function
Just have the program change the value for startItem to 50 get the next fifty (items 50-100)
POP3 protocol does not have the notion of seen/unseen messages.
Can't you use IMAP?
It would give you more features (like searching, flagging, folder management) than POP3.