How to implement murmurhash3 in VBNET - vb.net

I'm trying to implement murmurhash3 in vb.net and trying to convert from this C# implementation
first part of the function in c#
public static SqlInt32 MurmurHash3(SqlBinary data)
{
const UInt32 c1 = 0xcc9e2d51;
const UInt32 c2 = 0x1b873593;
int curLength = data.Length; /* Current position in byte array */
int length = curLength; /* the const length we need to fix tail */
UInt32 h1 = seed;
UInt32 k1 = 0;
/* body, eat stream a 32-bit int at a time */
Int32 currentIndex = 0;
while (curLength >= 4)
{
/* Get four bytes from the input into an UInt32 */
k1 = (UInt32)(data[currentIndex++]
| data[currentIndex++] << 8
| data[currentIndex++] << 16
| data[currentIndex++] << 24);
/* bitmagic hash */
k1 *= c1;
k1 = rotl32(k1, 15);
k1 *= c2;
h1 ^= k1;
h1 = rotl32(h1, 13);
h1 = h1 * 5 + 0xe6546b64;
curLength -= 4;
}
And same in VB.net:
Public Shared Function MurmurHash3(data As Byte()) As Int32
Const c1 As UInt32 = &HCC9E2D51UI
Const c2 As UInt32 = &H1B873593
Dim curLength As Integer = data.Length
' Current position in byte array
Dim length As Integer = curLength
' the const length we need to fix tail
Dim h1 As UInt32 = seed
Dim k1 As UInt32 = 0
' body, eat stream a 32-bit int at a time
Dim dBytes As Byte()
Dim currentIndex As Int32 = 0
While curLength >= 4
' Get four bytes from the input into an UInt32
dBytes = New Byte() {data(currentIndex), data(currentIndex + 1), data(currentIndex + 2), data(currentIndex + 3)}
k1 = BitConverter.ToUInt32(dBytes, 0)
currentIndex += 4
' bitmagic hash
k1 *= c1
k1 = rotl32(k1, 15)
k1 *= c2
h1 = h1 Xor k1
h1 = rotl32(h1, 13)
h1 = h1 * 5 + &HE6546B64UI
curLength -= 4
End While
Private Shared Function rotl32(x As UInt32, r As Byte) As UInt32
Return (x << r) Or (x >> (32 - r))
End Function
k1 *= c1
Throws error Arithmetic operation resulted in an overflow.
Any suggestions how this should be implemented? I'm Not sure how to do the Get four bytes from the input into an UInt32 part if that is the problem or is it related to something else since there are some differences in bitwise operations between C# and VB.
For the reference Java implementation also exists
https://github.com/yonik/java_util/blob/master/src/util/hash/MurmurHash3.java

I'd first convert the 32-bit k1 to a 64-bit variant first, e.g:
k1_64 = CType(k1, UInt64)
for modulo-32bit calculation, do
k1_64 = (k1_64 * c1) And &HFFFFFFFFUI
finally, recast back to 32-bit
k1 = CType(k1_64 And $HFFFFFFFFUI, UInt32)
to add more performance, you might want to consider replacing the BitConverter.ToUInt call with something else.
EDIT : Here's a simpler version without additional variable (but with a 'helper constant')
Const LOW_32 as UInt32 = &HFFFFFFFFUI
' ... intervening code ...
k1 = (1L * k1 * c1) And LOW_32
' ... later on ...
h1 = (h1 * 5L + &HE6546B64UL) And LOW_32
the 1L forces the calculation within the parens to be performed as Long (Int64). The And LOW_32 pares down the number of non-zero bits to 32, and the overall result is then automatically casted to UInt32. Similar thing happens on the h1 line.
Reference: http://www.undermyhat.org/blog/2009/08/secrets-and-lies-of-type-suffixes-in-c-and-vb-net/ (scroll down to the section "Secrets of constants and type suffixes")

Unfortunately, it possible to do the equivalent of unchecked {} in VB.NET? You could use a try/catch blocked and do the shift manually if you overflow. Just be careful, putting an error handler in there will slow down the hash calculation.

Related

Bitcoin blockchain parser c# snippet to vb

I am trying to get this snippet to vb but keep getting errors in the ide
var header = new byte[8];
int index = 0;
var magic_signature = ((uint)header[index++] << 0) | ((uint)header[index++] << 8) | ((uint)header[index++] << 16) | ((uint)header[index++] << 24);
I tried an online coverter and got this (which produces errors)
Dim magic_signature = (CUInt(header(Math.Max(Threading.Interlocked.Increment(index), index - 1))) << 0) Or (CUInt(header(Math.Max(Threading.Interlocked.Increment(index), index - 1))) << 8) Or (CUInt(header(Math.Max(Threading.Interlocked.Increment(index), index - 1))) << 16) Or (CUInt(header(Math.Max(Threading.Interlocked.Increment(index), index - 1))) << 24)
The header variable is filled after declaration.
Could someone help me convert the above to vb.net?
EDIT:
Populating header via reading a stream
Dim header = New Byte(8) {}
If stream.Read(header, 0, 8) < 8 Then
Throw New ApplicationException("Incomplete data.")
End If
As Mike_OBrien has pointed out, VB doesn't have a built-in ++ - but we can fake one easily enough:
Function PostIncr(ByRef x As Integer) As Integer
' PostIncr(i) works like i++
Dim x0 As Integer = x : x += 1 : Return x0
End Function
After that, conversion is straightforward:
Dim header(7) As Byte
' Make believe this is a stream.Read
Array.Copy({CByte(1), CByte(2), CByte(3), CByte(4)},
header, 4)
Dim index As Integer = 0
Dim magic_signature = CUInt(header(PostIncr(index))) << 0 Or
CUInt(header(PostIncr(index))) << 8 Or
CUInt(header(PostIncr(index))) << 16 Or
CUInt(header(PostIncr(index))) << 24
And for fun:
Function PreIncr(ByRef x As Integer) As Integer
' PreIncr(i) works like ++i
x += 1 : Return x
End Function
I believe David Wilson is on the right path but as rskar pointed out the Index + 1 portion does not produce the same results as index++ would in c#. The easiest way to get the behavior from c# would be to break the single command into multiple commands and store the results in temp variables, incrementing Index between each step and then evaluating the temp variables at the end. This would result in a lot more code however.
Ex:
Dim temp1 = (CUInt(header(Index)) << 0)
Index += 1
Dim temp2 = (CUInt(header(Index)) << 8)
Index += 1
Dim temp3 = (CUInt(header(Index)) << 16)
Index += 1
Dim magic_signature As UInteger = temp1 Or temp2 Or temp3 Or (CUInt(header(Index)) << 24)
Unfortunately to my knowledge there isn't anything in vb.net that behaves the same as the ++ operator in c#.

vb.net arithmetic operation resulted in an overflow in decryption

I am working on new decryption functions for password recovery tools. I tried code in c++ and worked:
void poco_pwd(u_char *pwd, int type) {
int len,
tmp;
u_char *out;
short azz;
if(type) azz = 0x2537; // encrypt message
else azz = 0x2a9a; // other passwords
len = strlen(pwd) >> 1;
out = pwd;
while(len--) {
sscanf(pwd, "%02X", &tmp);
pwd += 2;
*out++ = tmp ^ (azz >> 8);
azz = ((tmp + azz) * 0x8141) + 0x3171;
}
*out = 0;
}
I tried to convert this code to vb.net and c# but it throws arithmetic overflow operation. This functions new value is put to "azz" variable. "azz" is a short variable but this these values are very high. Strange is that it works in c++.
I converted this code to vb.net:
Dim encpass As String = "1EF66D8BD3C32476CEC8CF"
Dim encpassByte As Byte() = Encoding.UTF8.GetBytes(HexToString(encpass))
Dim azz As Integer = &H2A9A
Dim len As Integer = encpassByte.Length >> 1
Dim storage(len) As Char
For i = 0 To len
storage(i) = (ChrW(encpassByte(i) Xor (azz >> 8)))
azz = ((encpassByte(i) + azz) * &H8141) + &H3171 //Error: arithmetic operation resulted in an overflow.
Next
Console.WriteLine(storage.ToString)
Console.ReadKey()
Hex to string function:
Function HexToString(ByVal hex As String) As String
Dim text As New System.Text.StringBuilder(hex.Length \ 2)
For i As Integer = 0 To hex.Length - 2 Step 2
text.Append(Chr(Convert.ToByte(hex.Substring(i, 2), 16)))
Next
Return text.ToString
End Function
This code throws this error: arithmetic operation resulted in an overflow.

Fibonacci shift register pseudo-random number generator

I am attempting to get the following code working for a Fibonacci shift register to generate pseudo-random numbers. Can't seem to get it working, so is(are) there any obvious issues(?)
Shared Function Main() As Integer
Dim start_state As UShort = &HACE1UI ' Any nonzero start state will work.
Dim lfsr As UShort = start_state
Dim bit As UInteger
Dim period As UInteger = 0
Do While lfsr <> start_state
' taps: 16 14 13 11; feedback polynomial: x^16 + x^14 + x^13 + x^11 + 1
bit = ((lfsr >> 0) Xor (lfsr >> 2) Xor (lfsr >> 3) Xor (lfsr >> 5)) And 1
lfsr = (lfsr >> 1) Or (bit << 15)
period += 1
Loop
Return 0
End Function
Last, does "period" need to be divided by a large integer to get U(0,1)'s?
Below is the original C++ code:
# include <stdint.h>
int main(void)
{
uint16_t start_state = 0xACE1u; /* Any nonzero start state will work. */
uint16_t lfsr = start_state;
uint16_t bit; /* Must be 16bit to allow bit<<15 later in the code */
unsigned period = 0;
do
{
/* taps: 16 14 13 11; feedback polynomial: x^16 + x^14 + x^13 + x^11 + 1 */
bit = ((lfsr >> 0) ^ (lfsr >> 2) ^ (lfsr >> 3) ^ (lfsr >> 5) ) & 1;
lfsr = (lfsr >> 1) | (bit << 15);
++period;
} while (lfsr != start_state);
return 0;
}
As in #dummy's comment,
Do While lfsr <> start_state
...
Loop
doesn't run because lfsr = start_state at the beginning.
The code equivalent to C++
do {
...
} while (lfsr != start_state);
in VB.NET is
Do
...
Loop While lfsr <> start_state

VB.NET Bit manipulation: how to extract byte from short?

Given this Short (signed):
&Hxxxx
I want to:
Extract the most right &HxxFF as SByte (signed)
Extract the left &H7Fxx as Byte (unsigned)
Identify if the most left &H8xxx is positive or negative (bool result)
Extract the most right 0xxxff
myShort & 0x00FF
Extract the left 0xffxx
(myShort & 0xFF00) >> 8
Identify if the most left 0xfxxx is
positive or negative (it's a signed
short).
(myShort & 0xF000) >= 0;
Dim test As UInt16 = &HD 'a test value 1101
Dim rb As Byte 'lsb
Dim lb As Byte 'msb - 7 bits
Dim rm As UInt16 = &HFF 'lsb mask
Dim lm As UInt16 = &H7F00 'msb mask
Dim sgn As Byte = &H80 'sign mask
For x As Integer = 0 To 15 'shift the test value one bit at a time
rb = CByte(test And rm) 'get lsb
lb = CByte((test And lm) >> 8) 'get msb
Dim lbS, rbS As Boolean 'sign
'set signs
If (rb And sgn) = sgn Then rbS = True _
Else rbS = False
If (lb And sgn) = sgn Then lbS = True _
Else lbS = False 'should always be false based on mask
Console.WriteLine(String.Format("{0} {1} {2} {3} {4}",
x.ToString.PadLeft(2, " "c),
Convert.ToString(lb, 2).PadLeft(8, "0"c),
Convert.ToString(rb, 2).PadLeft(8, "0"c),
lbS.ToString, rbS.ToString))
test = test << 1
Next
inline char getLsb(short s)
{
return s & 0xff;
}
inline char getMsb(short s)
{
return (s & 0xff00) >> 8;
}
inline bool isBitSet(short s, unsigned pos)
{
return (s & (1 << pos)) > 0;
}
Uh...
value & 0x00ff
(value & 0xff00) >> 8
(value & 0xf000) >= 0
EDIT: I suppose you want the byte value and not just the upper 8 bits.
Extract the most right &HxxFF as SByte (signed)
CType(s AND &H00FF, SByte)
Extract the left &H7Fxx as Byte (unsigned)
CType((s AND &H7F00) >> 8, Byte)
Identify if the most left &H8xxx is positive or negative (bool result)
s AND &H8000 > 0
I think those work, been a while since I have worked in VB

permutations gone wrong

I have written code to implement an algorithm I found on string permutations. What I have is an arraylist of words ( up to 200) and I need to permutate the list in levels of 5. Basically group the string words in fives and permutated them. What I have takes the first 5 words generates the permutations and ignores the rest of the arraylist?
Any ideas appreciated.
Private Function permute(ByVal chunks As ArrayList, ByVal k As Long) As ArrayList
ReDim ItemUsed(k)
pno = 0
Permutate(k, 1)
Return chunks
End Function
Private Shared Sub Permutate(ByVal K As Long, ByVal pLevel As Long)
Dim i As Long, Perm As String
Perm = pString ' Save the current Perm
' for each value currently available
For i = 1 To K
If Not ItemUsed(i) Then
If pLevel = 1 Then
pString = chunks.Item(i)
'pString = inChars(i)
Else
pString = pString & chunks.Item(i)
'pString += inChars(i)
End If
If pLevel = K Then 'got next Perm
pno = pno + 1
SyncLock outfile
outfile.WriteLine(pno & " = " & pString & vbCrLf)
End SyncLock
outfile.Flush()
Exit Sub
End If
' Mark this item unavailable
ItemUsed(i) = True
' gen all Perms at next level
Permutate(K, pLevel + 1)
' Mark this item free again
ItemUsed(i) = False
' Restore the current Perm
pString = Perm
End If
Next
K above is = to 5 for the number of words in one permutation but when I change the for loop to the arraylist size I get an error of index out of bounds
Index out of bounds error usually happens when you start the loop from 1 to length. The the for loop as following.
For i = 0 to array.length - 1
You will get this error.
When you do
For i = 1 To K
The last value of i will be the size of your array.
chunks.Item(i)
Will crash when i equals the size of the array since the index starts at 0.
I would suggest you change your for loop to
For i = 0 To K - 1
Or you change the way you access the values in your arrays to
chunks.Item(i-1)
C++ Permutation
#include <stdio.h>
void print(const int *v, const int size)
{
if (v != 0)
{
for (int i = 0; i < size; i++)
{
printf("%4d", v[i] );
}
printf("\n");
}
} // print
void permute(int *v, const int start, const int n)
{
if (start == n-1) {
print(v, n);
}
else {
for (int i = start; i < n; i++) {
int tmp = v[i];
v[i] = v[start];
v[start] = tmp;
permute(v, start+1, n);
v[start] = v[i];
v[i] = tmp;
}
}
}
main()
{
int v[] = {1, 2, 3, 4};
permute(v, 0, sizeof(v)/sizeof(int));
}