Screen sharing through sockets - vb.net

I'm making my first screen sharing application in VB.NET using sockets to establish the connections.
This is the client side receiving screen images from the server (they are both running in a thread):
Private Sub startscreen()
Using imgstream As NetworkStream = imgclient.GetStream()
Using ms As New MemoryStream
Dim read As Double
Do
If (imgstream.DataAvailable) Then
read = 0
ms.Seek(0, SeekOrigin.Begin)
While imgclient.Available
Dim buffer(imgclient.Available - 1) As Byte
imgstream.Read(buffer, 0, buffer.Length)
ms.Write(buffer, 0, buffer.Length)
read += buffer.Length
End While
Me.Text = "Frame bytes read: " & read
PictureBox1.Image = Image.FromStream(ms)
ms.Flush()
End If
Thread.Sleep(34) 'about 30 FPS
Loop
End Using
End Using
End Sub
And this is the server side:
Private Sub screen()
Using imgstream As NetworkStream = imgclient.GetStream()
Using ms As New MemoryStream
Do
Thread.Sleep(34) 'about 30 FPS
ms.Seek(0, SeekOrigin.Begin)
Using img = ScreenCap()
img.Save(ms, Imaging.ImageFormat.Jpeg)
End Using
ms.WriteTo(imgstream)
Loop
End Using
End Using
End Sub
Public Function ScreenCap() As Image
Dim screenSize As Size = New Size(My.Computer.Screen.Bounds.Width, My.Computer.Screen.Bounds.Height)
Dim screenGrab As New Bitmap(My.Computer.Screen.Bounds.Width, My.Computer.Screen.Bounds.Height) ', Imaging.PixelFormat.Format16bppRgb555)
Dim g As Graphics = Graphics.FromImage(screenGrab)
g.CopyFromScreen(New Point(0, 0), New Point(0, 0), screenSize)
g.Dispose()
Return screenGrab
End Function
The main problem is when I call the "Image.FromStream(ms)" function, it sometimes works and others doesn't depending on how many milliseconds I set the thread to wait. Tested on 2 different computers in my LAN, on around 1 second it seems OK but always with a high CPU-Network usage. If I set, as the example says around 34 milliseconds to get all more "LIVE", that function throw an exception because of the MemoryStream. How can I speed it up? Is there any smarter way I'm missing right now? I've also tried putting a delimiter byte (like a char = "*") at the and of the MemoryStream and then send it to the client who read one byte at a time until it found a char equal to the delimiter. But it turned out to be a bad solution because a single byte of the image could represent the delimiter if converted to char. Another question I have is: How can I change the image quality and the color depth? Is it a good approach using what the comment says: "Imaging.PixelFormat.Format16bppRgb555"
Thank you!

Related

Parse HEX file Parrallel with Filestream

So i'm trying to speed up a For i as integer to get max preformace. I'm using more and more Parrallel and Async method in my code and it helps me alot. However currently i'm stuck with this one. I simply what to loop trhough a file an reading specific index position to see what's in the file so i can do certain things with it later on.
This is an example of the current For:
Using fs As New FileStream(PathToFile, FileMode.Open, FileAccess.Read, FileShare.Read)
For i As Long = 0 To fs.Length Step 1024
'Go to the calculated index in the file
fs.Seek(i, SeekOrigin.Begin)
'Now get 24 bytes at the current index
fs.Read(buffer, 0, 24)
'Do some stuff with it
List.Add(Buffer)
Next
End Using
The files can be 15MB to 4GB in size. Currently i'm stuck at how to use a Step 1024 in a Parrallel.For and also how to approach it thread-safe. Hopefully someone could help me out with this.
This might be marked down, but if memory usage isn't a huge problem, then I would suggest reading the whole file and storing just those 24 byte sequences in an array of byte(23). Then using Parallel.For processing the array. On my pc the array takes up about 160mb for a 4gb. The speed of reading will of course depend on the system used. On my PC it take around 25 seconds.
Try this..
Imports System.Math
Imports System.IO
Public Class Form1
'create the data array, with just 1 element to start with
'so that it can be resized then you know how many 1024 byte
'chunks you have in your file
Dim DataArray(1)() As Byte
Private Sub ReadByteSequences(pathtoFile As String)
Using fs As New FileStream(pathtoFile, FileMode.Open, FileAccess.Read, FileShare.Read)
'resize the array when you have the file size
ReDim DataArray(CInt(Math.Floor(fs.Length) / 1024))
For i As Long = 0 To fs.Length Step 1024
Dim buffer(23) As Byte
fs.Seek(i, SeekOrigin.Begin)
fs.Read(buffer, 0, 24)
'store each 24 byte sequence in order in the
'DataArray for later processing
DataArray(CInt(Math.Floor(i / 1024))) = buffer
Next
End Using
End Sub
Private Sub ProcessDataArray()
Parallel.For(0, DataArray.Length, Sub(i As Integer)
'do atuff in parallel
End Sub)
End Sub
End Class

Read from MemoryStream using StreamReader without closing it VB.Net

I'm trying to create a class for handling multiple Tcp connections. The problem is that I write the data sent from client in a MemoryStream. The I have to read the first line from the stream and see if it contains "[ID]". It seems that once I try to read it using:
Using sr as new StreamReader(data_stream)
If sr.readLine="[id]" then
' Code
End If
End Using
it closes the stream.I tried even creating a dublicate (I don't want too many dublicates because I will send large-size data over tcp so it will fill the RAM). I have then to send the memorystream in a event. Even if the stream was sent ByVal it still closed it when I used a StreamReader. I don't want to leave the GC to take care of the un-closed Stream Readers( I tried another class made by me-also for Tcp- and the GC was being called MANY times in a row..and the application used 27 MB of RAM because of dublicates-I was sending multiple pictures). Is it ok to use a buffer (an array of bytes) and leave the memory stream? Or should I let the GC to do the job for me?
Here's a part from the class:
Sub receive_header(ar As IAsyncResult)
received_size += client_stream.EndRead(ar)
If received_size = 0 Then
RaiseEvent Internal_Disconnected(New Exception("Connection closed normally"))
Else
If received_size < client_header_size Then
client_stream.BeginRead(buffer, received_size, client_header_size - received_size, New AsyncCallback(AddressOf receive_header), client_stream)
Else
received_size = 0
client_header = readHeader(buffer)
Array.Clear(buffer, 0, client_header_size)
client_stream.BeginRead(buffer, 0, buffer.Length, New AsyncCallback(AddressOf receive_data), client_stream)
End If
End If
End Sub
Sub receive_data(ar As IAsyncResult)
received_size += client_stream.EndRead(ar)
If received_size = 0 Then
RaiseEvent Internal_Disconnected(New Exception("Connection closed normally"))
Else
data_stream.Write(buffer, 0, received_size)
If received_size < client_header.PACK_SIZE Then
'TODO: Here can be made an optimization
' Buffer has a length of client_stream.EndRead(ar). So take that instead of adding to received_size
Array.Clear(buffer, 0, buffer.Length)
client_stream.BeginRead(buffer, 0, buffer.Length, New AsyncCallback(AddressOf receive_data), client_stream)
Else
received_size = 0
' Send data to user if canSend
If canSend Then
data_stream.Position = 0
' If in this event I use stream reader it closes the main stream althought it is passed byVal
RaiseEvent New_Message(Me, data_stream)
End If
' Check first line
data_stream.Position = 0
Dim sr As New StreamReader(data_stream)
If sr.ReadLine = "[ID]" Then
canSend = True
With client
.ID = sr.ReadLine.Split("=")(1)
.Computer_Name = sr.ReadLine.Split("=")(1)
.Win_Ver = sr.ReadLine.Split("=")(1)
End With
End If
' Reset
data_stream.Close()
data_stream = New MemoryStream
Array.Clear(buffer, 0, buffer.Length)
client_stream.BeginRead(buffer, 0, client_header_size, New AsyncCallback(AddressOf receive_header), client_stream)
End If
End If
End Sub
As you can see in receive_data function I only 2048 bytes from the buffer. If the message is longer I reuse the buffer but not before I save the data in data_stream. How can I pass data_stream throught the event and read it without closing it?

JPEG Compression causes GDI+ Exception

So, I'm currently working on a LAN-Video-Streaming program, which records single images and sends them over. Because it would be too much to send 30 1920x1080 pictures per second, to get 30FPS, I did some research and found JPEG-Compression. The problem is, that when I try to save the compressed JPEG, it throws an System.Runtime.InteropServices.ExternalException, with the additional information: General error in GDI+.
Here's my code:
Private Sub Stream() Handles StreamTimer.Tick
If Streaming = True Then
Try
ScreenCap = New Bitmap(Bounds.Width, Bounds.Height)
GFX = Graphics.FromImage(ScreenCap)
GFX.CopyFromScreen(0, 0, 0, 0, ScreenCap.Size)
Dim Frame As New Bitmap(ScreenCap, Resolution.Split(";")(0), Resolution.Split(";")(1))
Dim jpgEncoder As ImageCodecInfo = GetEncoder(ImageFormat.Jpeg)
Dim myEncoder As Encoder = Encoder.Quality
Dim myEncoderParameters As New EncoderParameters(1)
Dim myEncoderParameter As New EncoderParameter(myEncoder, Compression)
myEncoderParameters.Param(0) = myEncoderParameter
Frame.Save(My.Computer.FileSystem.SpecialDirectories.Temp & "\LSSFrame.jpg", jpgEncoder, myEncoderParameters) 'Error occurs in this line
Using FS As New FileStream(My.Computer.FileSystem.SpecialDirectories.Temp & "\LSSFrame.jpg", FileMode.Open)
Frame = Image.FromStream(FS)
FrameSizeStatus.Text = Math.Round(FS.Length / 1000) & "KB"
FS.Close()
End Using
PreviewBox.Image = Frame
FPSStat += 1
FlushMemory()
If ViewerIPs.Count > 0 Then
For i = 0 To ViewerIPs.Count - 1
SendFrame(ViewerIPs(i), Frame)
Next
End If
Catch ex As Exception
LostFrames += 1
End Try
End If
End Sub
Any help is appreciated!
In part, you are not disposing of the Graphics or Bitmap objects you create. The error message is not very helpful, but not disposing of those will leave resources unrecovered.
There is also a lot going on in that procedure. If it were broken into parts it might be easier to fine-tune for performance and such.
' form level objects
Private jEncParams As EncoderParameters
Private jpgEncoder As ImageCodecInfo
...
' inititalize somewhere when the process starts:
Dim quality As Int64 = 95
jpgEncoder = GetEncoder(ImageFormat.Jpeg)
Dim myjEnc As Imaging.Encoder = Imaging.Encoder.Quality
jEncParams = New EncoderParameters(1)
' quality is inverse to compression
jEncParams.Param(0) = New EncoderParameter(myjEnc, quality)
Since the encoder and quality elements are not going to change for each screen snap, create them once and resuse them. Then your timer event:
Dim scrBytes = GetScreenSnap(1280, 720)
' do something to send them....maybe queue them?
Console.WriteLine("image size: {0}k", (scrBytes.Length / 1024).ToString)
Optimizing SendFrame is outside the scope of this Q/A, but getting the screen shot is separate from sending.
Private Function GetScreenSnap(w As Int32, h As Int32) As Byte()
Using bmpScrn As New Bitmap(My.Computer.Screen.Bounds.Width, My.Computer.Screen.Bounds.Height)
Using g As Graphics = Graphics.FromImage(bmpScrn)
g.CopyFromScreen(0, 0, 0, 0, bmpScrn.Size)
End Using ' done with graphics
Using bmpThumb As New Bitmap(bmpScrn, w, h),
ms As New MemoryStream
bmpThumb.Save(ms, jpgEncoder, jEncParams)
Return ms.ToArray
End Using ' dispose of bmp
End Using ' dispose of bmpScrn
End Function
For no particular reason, I am thumbnailing the entire screen. Yours seems off using Bounds.Width, Bounds.Height since that would refer to the form. It would only work as a screen snapshot if the form is maximized. keypoints:
This is predicated on the idea that you can/will be sending the byte array in a stream. As such, I leave it as a byte array rather than creating a BMP only to (presumably) convert it back later.
There is no need to create a disk file to get the size. If the array contains encoded bytes, it will be the same size.
I've never used the COMPRESSION param, but I know Quality is inverse to compression.
Resulting sizes for various Quality factors:
100 = 462
95 = 254
90 = 195
80 = 147
Strictly speaking you do not need the encoder, bmpThumb.Save(ms, Imaging.ImageFormat.Jpeg) will also work but a single encoder object offers more fine tuning.
For the sending part, you might want a Stack, Queue or LinkedList to store the byte array. This would further isolate getting images from sending them. The collection would be a sort of ToDo/ToSend list.
Then, if there are multiple recipients, I'd look into perhaps doing SendFrame as a Task, perhaps sending 2-3 at a time. There might be a point where the number of recievers interferes with how fast you can grab new ones.

Sending and Receiving Bitmaps per TcpClients

So, I'm currently making a LAN-Streaming-Program, which should record the screen of the streamer and display it at the viewer's screen.
My problem is, that when I try to convert the received ByteArray-Image to an image, it gives me a System.ArgumentException.
Here's the code for sending:
Private Sub SendFrame(ByRef IP As String, ByRef Frame As Bitmap)
FrameClient = New TcpClient(IP, 7009)
Dim FrameStream As New MemoryStream
Dim FrameBytes() As Byte
Frame.Save(FrameStream, ImageFormat.Bmp)
FrameBytes = FrameStream.GetBuffer()
SendMessage(IP, "NextFrameSize;" & LocalIP & ";" & FrameBytes.Length)
Using NS As NetworkStream = FrameClient.GetStream
NS.Write(FrameBytes, 0, FrameBytes.Length)
NS.Close()
End Using
End Sub
And here for receiving:
Private Sub CheckForFrames() 'This Sub is called everytime the viewer receives the size of the next Bitmap.
If FrameListener.Pending = True Then
FrameClient = FrameListener.AcceptTcpClient
Dim ImageBytes(NextFrameSize) As Byte
FrameClient.GetStream.Read(ImageBytes, 0, NextFrameSize)
Dim MS As New MemoryStream(ImageBytes)
StreamBox.Image = Image.FromStream(MS, False)
MS.Close()
FlushMemory()
End If
End Sub
I would be thankful for any answers!
Dim ImageBytes(FrameClient.ReceiveBufferSize) As Byte
This returns the size of te buffer, not the actual size of the bitmap:
The size of the receive buffer, in bytes. The default value is 8192
bytes.
So, ImageBytes will probably be 8192 bytes, not the size of your bitmap.
You might want to send the size before you send the bitmap data, and on the receive side read in the size first to initialize your MemoryStream.

VB.NET Display progress of file decryption?

I'm using this code to encrypt/decrypt files:
Public Shared Sub encryptordecryptfile(ByVal strinputfile As String, _
ByVal stroutputfile As String, _
ByVal bytkey() As Byte, _
ByVal bytiv() As Byte, _
ByVal direction As CryptoAction)
Try
fsInput = New System.IO.FileStream(strinputfile, FileMode.Open, FileAccess.Read)
fsOutput = New System.IO.FileStream(stroutputfile, FileMode.OpenOrCreate, FileAccess.Write)
fsOutput.SetLength(0)
Dim bytbuffer(4096) As Byte
Dim lngbytesprocessed As Long = 0
Dim lngfilelength As Long = fsInput.Length
Dim intbytesincurrentblock As Integer
Dim cscryptostream As CryptoStream
Dim csprijndael As New System.Security.Cryptography.RijndaelManaged
Select Case direction
Case CryptoAction.ActionEncrypt
cscryptostream = New CryptoStream(fsOutput, _
csprijndael.CreateEncryptor(bytkey, bytiv), _
CryptoStreamMode.Write)
Case CryptoAction.ActionDecrypt
cscryptostream = New CryptoStream(fsOutput, _
csprijndael.CreateDecryptor(bytkey, bytiv), _
CryptoStreamMode.Write)
End Select
While lngbytesprocessed < lngfilelength
intbytesincurrentblock = fsInput.Read(bytbuffer, 0, 4096)
cscryptostream.Write(bytbuffer, 0, intbytesincurrentblock)
lngbytesprocessed = lngbytesprocessed + CLng(intbytesincurrentblock)
End While
cscryptostream.Close()
fsInput.Close()
fsOutput.Close()
Catch ex As Exception
End Try
End Sub
Is I need to get the percentage of this process being done as an integer. I am going to use a background worker, so I need to call for this sub from the background worker and be able to keep refreshing a progress bar that the background worker reports to. Is this possible?
Thanks in advance.
There are a couple of things you can do to make your cryptor more efficient and other issues:
A method like encryptordecryptfile which then requires a "mode" argument to know which action to take means it really might be better off as 2 methods
The way you are going, you will be raising a blizzard of ProgressChanged events which the ProgressBar wont be able to keep up with given the animation. A 700K file will result in 170 or so progress reports of tiny amounts
Some of the crypto steps can be incorporated
You have a lot of things not being disposed of; you could run out of resources if you run a number of files thru it in a loop.
It might be worth noting that you can replace the entire While block with fsInput.CopyTo(cscryptostream) to process the file all at once. This doesnt allow progress reporting though. Its also not any faster.
Rather than a BackgroundWorker (which will work fine), you might want to implement it as a Task. The reason for this is that all those variables need to make their way from something like a button click to the DoWork event where your method is actually called. Rather than using global variables or a class to hold them, a Task works a bit more directly (but does involve one extra step when reporting progress). First, a revised EncryptFile method:
Private Sub EncryptFile(inFile As String,
outFile As String,
pass As String,
Optional reporter As ProgressReportDelegate = Nothing)
Const BLOCKSIZE = 4096
Dim percentDone As Integer = 0
Dim totalBytes As Int64 = 0
Dim buffSize As Int32
' Note A
Dim key = GetHashedBytes(pass)
Dim iv = GetRandomBytes(16)
Dim cryptor As ICryptoTransform
' Note B
Using fsIn As New FileStream(inFile, FileMode.Open, FileAccess.Read),
fsOut As New FileStream(outFile, FileMode.OpenOrCreate, FileAccess.Write)
fsOut.SetLength(0)
' Note C
'ToDo: work out optimal block size for Lg vs Sm files
If fsIn.Length > (2 * BLOCKSIZE) Then
' use buffer size to limit to 20 progress reports
buffSize = CInt(fsIn.Length \ 20)
' to multiple of 4096
buffSize = CInt(((buffSize + BLOCKSIZE - 1) / BLOCKSIZE) * BLOCKSIZE)
' optional, limit to some max size like 256k?
'buffSize = Math.Min(buffSize, BLOCK256K)
Else
buffSize = BLOCKSIZE
End If
Dim buffer(buffSize-1) As Byte
' Note D
' write the IV to "naked" fs
fsOut.Write(iv, 0, iv.Length)
Using rij = Rijndael.Create()
rij.Padding = PaddingMode.ISO10126
Try
cryptor = rij.CreateEncryptor(key, iv)
Using cs As New CryptoStream(fsOut, cryptor, CryptoStreamMode.Write)
Dim bytesRead As Int32
Do Until fsIn.Position = fsIn.Length
bytesRead = fsIn.Read(buffer, 0, buffSize)
cs.Write(buffer, 0, bytesRead)
If reporter IsNot Nothing Then
totalBytes += bytesRead
percentDone = CInt(Math.Floor((totalBytes / fsIn.Length) * 100))
reporter(percentDone)
End If
Loop
End Using
Catch crEx As CryptographicException
' ToDo: Set breakpoint and inspect message
Catch ex As Exception
' ToDo: Set breakpoint and inspect message
End Try
End Using
End Using
End Sub
Note A
One of the standard crypto tasks it could handle is creating the Key and IV arrays for you. These are pretty simple and could be shared/static members.
Public Shared Function GetHashedBytes(data As String) As Byte()
Dim hBytes As Byte()
' or SHA512Managed
Using hash As HashAlgorithm = New SHA256Managed()
' convert data to bytes:
Dim dBytes = Encoding.UTF8.GetBytes(data)
' hash the result:
hBytes = hash.ComputeHash(dBytes)
End Using
Return hBytes
End Function
Public Shared Function GetRandomBytes(size As Integer) As Byte()
Dim data(size - 1) As Byte
Using rng As New RNGCryptoServiceProvider
' fill the array
rng.GetBytes(data)
End Using
Return data
End Function
As will be seen later, you can store the IV in the encrypted file rather than saving and managing it in code.
Note B
Using blocks close and dispose of resources for you. Basically, if something has a Dispose method, then you should wrap it in a Using block.
Note C
You dont want to report progress for every block read, that will just overwhelm the ProgressBar. Rather than another variable to keep track of when the progress has changed by some amount, this code starts by creating a buffer size which is 5% of the input file size so there will be about 20 reports (every 5%).
As the comments indicate, you may want to add some code to set minimum/maximum buffer size. Doing so would change the progress report frequency.
Note D
You can write the IV() to the filestream before you wrap it in the CryptoStream (and of course read it back first when Decrypting). This prevents you from having to store the IV.
The last part is kicking this off as a Task:
Dim t As Task
t = Task.Run(Sub() EncryptFile(inFile, oFile, "MyWeakPassword",
AddressOf ReportProgress))
...
What a BGW does is execute the work on one thread, but progress is reported on the UI thread. As a Task, all we need to do is use Invoke:
Delegate Sub ProgressReportDelegate(value As Int32)
Private Sub ReportProgress(v As Int32)
If progBar.InvokeRequired Then
progBar.Invoke(Sub() progBar.Value = v)
Else
progBar.Value = v
progBar.Invalidate()
End If
End Sub
The Encryptor will work either directly or as a Task. For small files, you can omit the progress report entirely:
' small file, no progress report:
EncryptFile(ifile, oFile, "MyWeakPassword")
' report progress, but run on UI thread
EncryptFile(ifile, oFile, "MyWeakPassword",
AddressOf ReportProgress)
' run as task
Dim t As Task
t = Task.Run(Sub() EncryptFile(ifile, oFile, "MyWeakPassword",
AddressOf ReportProgress))
...and if you had a list of files to do, you could run them all at once and perhaps report total progress.