nodejs and vb.net - zlib deflate compression non-compatibility between nodejs and vb - vb.net

Question

I'm trying to communicate between a client and a server, with the client being on VB.NET and the server on nodejs.
It works, but I'm now trying to use the deflate compression method which seems to not work exactly the same in vb.net and nodejs.
Here's the code. I will explain the problem next in order to be understandable.
deflate compression on (VB.NET)
Shared Function Compression(buffer As Byte()) As Byte()
Try
Dim ms As New MemoryStream()
Using zipStream As New Compression.DeflateStream(ms, IO.Compression.CompressionMode.Compress, True)
zipStream.Write(buffer, 0, buffer.Length)
End Using
ms.Position = 0
Dim outStream As New MemoryStream()
Dim compressed As Byte() = New Byte(ms.Length - 1) {}
ms.Read(compressed, 0, compressed.Length)
Dim gzBuffer As Byte() = New Byte(compressed.Length + 3) {}
System.Buffer.BlockCopy(compressed, 0, gzBuffer, 4, compressed.Length)
System.Buffer.BlockCopy(BitConverter.GetBytes(buffer.Length), 0, gzBuffer, 0, 4)
Return gzBuffer
Catch
Return buffer
End Try
End Function`
I use it like this:
Compression(stringtobyte("testTESTtestTESTtestTESTtestTEST"))
It returns this byte() data: [32,0,0,0,43,73,45,46,9,113,13,14,41,193,65,3,0].
deflate compression in (nodejs) (using zlib)
zlib.deflateSync(Buffer.from("testTESTtestTESTtestTESTtestTEST"))
That returns: [120, 156, 43, 73, 45, 46, 9, 113, 13, 14, 41, 193, 65, 3, 0, 201, 232, 12, 1]
As you can see, the "inside" compression bytes are the same in nodejs and VB (they are bolded). In VB there are four leading bytes [32,0,0,0]. I understand that that shows the length of uncompressed original data. That's true since the length of "testTESTtestTESTtestTESTtestTEST" is 32 chars.
However in NodeJS there is always [120,156] at the start, and some bytes at the end (always 4?). Here [201, 232, 12, 1]. They are never the same if I change the string that I want to compress.
So what i tried:
In nodejs, using zlib.inflate(Buffer.from([120, 156, 43, 73, 45, 46, 9, 113, 13, 14, 41, 193, 65, 3, 0, 201, 232, 12, 1])).toString() actually returns my input string: "testTESTtestTESTtestTESTtestTEST".
However I must compress a string in VB (without the 2 firsts chars that you get in the nodejs compression [120,156], and without also the 4 chars at the end : [201, 232, 12, 1]) , and then decompress/uncompress it in Node.js.
I tried to add the [120,156] characters in the VB compressed string, while removing the first four chars in the VB compressed string that indicate the length of uncompressed data, but I don't have any clue as to what the four last chars are in the Node.js compression function. So the inflate(uncompress function) in Node.js doesn't work with my modified VB compressed string:
vb compressed data: [32,0,0,0,43,73,45,46,9,113,13,14,41,193,65,3,0] that I must turn into: [120, 156, 43,73,45,46,9,113,13,14,41,193,65,3,0,201,232,12,1]
//but I can't get those 4 last chars in vb
zlib.inflateSync(Buffer.from([120, 156, 43,73,45,46,9,113,13,14,41,193,65,3,0]))
//i remove the first 4 chars and add 120,156 "header" that i always got in nodejs, but i can't add the last 4 chars as I don't know what there are about, i only get them in nodejs zlib.deflateSync but not in my VB.net compression function.
The inflateSync function returns an error: "unexpected end of file" (the end that precisely I don't know how to get).
How can I find out what the four last chars from the Node.js deflate function are? (I imagine the two first chars 120,156 are about specifying that it's the deflate compression mode) I would like to add them in my VB.NET return string to make it compatible with Node.js.

Answer 1

Actually I found a solution, it's using GZIP compression mode instead of DEFLATE, it seems more stable and cross-platform compatible.
here are my VB.Net compression and decompression
Shared Function compression(buf As Byte()) As Byte()
Using ms As New MemoryStream()
Using gzipstream As New GZipStream(ms, CompressionMode.Compress) 'si m arche pas, ajouter dernière param
gzipstream.Write(buf, 0, buf.Length)
End Using
Return ms.ToArray()
End Using
End Function
Shared Function decompression(buf As Byte()) As Byte()
Using ms As New MemoryStream(buf)
Using outputstream As New MemoryStream
Using decstream As New GZipStream(ms, CompressionMode.Decompress)
decstream.CopyTo(outputstream)
End Using
Return outputstream.ToArray()
End Using
End Using
End Function
and in node instead of deflateSync and inflateSync, i simply use gzipSync and gunzipSync... it works perfectly
bye

Answer 2

You are using two different formats on the two platforms, plus the extra twist of the leading uncompressed size on the VB side. On the VB side you are using raw deflate format. On the Node.js side you are using the zlib format. The zlib format is a deflate stream with a two-byte zlib header and a four-byte Adler-32 integrity check value, which is a function of the uncompressed data.
On the VB side can use ZLibStream instead of DeflateStream, and you can get rid of the leading four bytes by simply removing that from your VB code: just have one BlockCopy line: System.Buffer.BlockCopy(compressed, 0, gzBuffer, 0, compressed.Length). That will then be compatible with the use of inflate on the Node.js side.