Download Image under the Mouse pointer via WebBrowser - vb.net

Question

I'm navigating to Google Images using a WebBrowser control. The aim is to be able to right click on any image and download and populate a PictureBox background.
I have my own ContextMenuStrip with Copy on it and have disabled the built in context menu.
The issue I am having is that the coordinate returned from CurrentDocument.MouseMove are always relative to the first (top left) image.
So my code works correctly if the Image I want is the very first image on the page, however clicking on any other Images always returns the coordinates of the first image.
It would appear that the coordinates are relative to each Image rather than the page.
Private WithEvents CurrentDocument As HtmlDocument
Dim MousePoint As Point
Dim Ele As HtmlElement
Private Sub Google_covers_Load(sender As Object, e As EventArgs) Handles MyBase.Load
WebBrowser1.IsWebBrowserContextMenuEnabled = False
WebBrowser1.ContextMenuStrip = ContextMenuStrip1
End Sub
Private Sub WebBrowser1_Navigated(sender As Object, e As WebBrowserNavigatedEventArgs) Handles WebBrowser1.Navigated
CurrentDocument = WebBrowser1.Document
End Sub
Private Sub CurrentDocument_MouseMove(sender As Object, e As HtmlElementEventArgs) Handles CurrentDocument.MouseMove
MousePoint = New Point(e.MousePosition.X, e.MousePosition.Y)
Me.Text = e.MousePosition.X & " | " & e.MousePosition.Y
End Sub
Private Sub ContextMenuStrip1_Opening(sender As Object, e As System.ComponentModel.CancelEventArgs) Handles ContextMenuStrip1.Opening
Ele = CurrentDocument.GetElementFromPoint(MousePoint)
If Ele.TagName = "IMG" Then
CopyToolStripMenuItem.Visible = True
Else
CopyToolStripMenuItem.Visible = False
End If
End Sub
Private Sub CopyToolStripMenuItem_Click(sender As System.Object, e As System.EventArgs) Handles CopyToolStripMenuItem.Click
Dim ToImg = Ele.GetAttribute("src")
mp3_row_edit.PictureBox1.BackgroundImage = New System.Drawing.Bitmap(New IO.MemoryStream(New System.Net.WebClient().DownloadData(ToImg)))
ToImg = Nothing
End Sub

Answer 1

This code allow to use a standard WebBrowser control to navigate to the Google Image search page and select/download an Image with a right-click of the Mouse.
To test it, drop a WebBrowser Control and a FlowLayoutPanel on a Form and navigate to a Google Image search page.
Things to know:
WebBrowser.DocumentCompleted: This event is raised each time one of the Sub-Documents inside a main HtmlDocument page is completed. Thus, it can be raised multiple times. We need to check whether the WebBrowser.ReadyState = WebBrowserReadyState.Complete.
Read these note about this: How to get an HtmlElement value inside Frames/IFrames?
The images in the Google search page can be inserted in the Document in 2 different manners: both using a Base64Encoded string and using the classic src=[URI] format. We need to be ready to get both.
The mouse click position can be espressed in either absolute or relative coordinates, referenced by the e.ClientMousePosition or e.OffsetMousePosition.
Read the notes about this feature here: Getting mouse click coordinates in a WebBrowser Document
The WebBrowser emulation mode can be important. We should use the most recent compatible mode available in the current machine.
Read this answer and apply the modifications needed to have the most recent Internet Explorer mode available: How can I get the WebBrowser control to show modern contents?.
Note that an event handler is wired up when the current Document is completed and is removed when the Browser navigates to another page. This prevents undesired calls to the DocumentCompleted event.
When the current Document is complete, clicking with the right button of the Mouse on an Image, creates a new PictureBox control that is added to a FlowLayouPanel for presentation.
The code in the Mouse click handler (Protected Sub OnHtmlDocumentClick()) detects whether the current image is a Base64Encoded string or an external source URI.
In the first case, it calls Convert.FromBase64String to convert the string into a Byte array, in the second case, it uses a WebClient class to download the Image as a Byte array.
In both cases, the array is then passed to another method (Private Function GetBitmapFromByteArray()) that returns an Image from the array, using Image.FromStream() and a MemoryStream initialized with the Byte array.
The code here is not performing null checks and similar fail-proof tests. It ought to, that's up to you.
Public Class frmBrowser
Private WebBrowserDocumentEventSet As Boolean = False
Private base64Pattern As String = "base64,"
Private Sub frmBrowser_Load(sender As Object, e As EventArgs) Handles MyBase.Load
WebBrowser1.ScriptErrorsSuppressed = True
WebBrowser1.IsWebBrowserContextMenuEnabled = False
End Sub
Private Sub WebBrowser1_DocumentCompleted(sender As Object, e As WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
If WebBrowser1.ReadyState = WebBrowserReadyState.Complete AndAlso WebBrowserDocumentEventSet = False Then
WebBrowserDocumentEventSet = True
AddHandler WebBrowser1.Document.MouseDown, AddressOf OnHtmlDocumentClick
End If
End Sub
Protected Sub OnHtmlDocumentClick(sender As Object, e As HtmlElementEventArgs)
Dim currentImage As Image = Nothing
If Not (e.MouseButtonsPressed = MouseButtons.Right) Then Return
Dim source As String = WebBrowser1.Document.GetElementFromPoint(e.ClientMousePosition).GetAttribute("src")
If source.Contains(base64Pattern) Then
Dim base64 As String = source.Substring(source.IndexOf(base64Pattern) + base64Pattern.Length)
currentImage = GetBitmapFromByteArray(Convert.FromBase64String(base64))
Else
Using wc As WebClient = New WebClient()
currentImage = GetBitmapFromByteArray(wc.DownloadData(source))
End Using
End If
Dim p As PictureBox = New PictureBox() With {
.Image = currentImage,
.Height = Math.Min(FlowLayoutPanel1.ClientRectangle.Height, FlowLayoutPanel1.ClientRectangle.Width)
.Width = .Height,
.SizeMode = PictureBoxSizeMode.Zoom
}
FlowLayoutPanel1.Controls.Add(p)
End Sub
Private Sub WebBrowser1_Navigating(sender As Object, e As WebBrowserNavigatingEventArgs) Handles WebBrowser1.Navigating
If WebBrowser1.Document IsNot Nothing Then
RemoveHandler WebBrowser1.Document.MouseDown, AddressOf OnHtmlDocumentClick
WebBrowserDocumentEventSet = False
End If
End Sub
Private Function GetBitmapFromByteArray(imageBytes As Byte()) As Image
Using ms As MemoryStream = New MemoryStream(imageBytes)
Return DirectCast(Image.FromStream(ms).Clone(), Image)
End Using
End Function
End Class