Programmatically download file (PDF) in GeckoFX VB.NET console application - vb.net

I use GeckoFX to navigate to a certain website which requires me to login. So far the navigating and logging in looks something like this:
Public Flag_Completed As Boolean = False
....
Navigate("http://www.website.com/loginpage/")
While Not Flag_Completed
Application.DoEvents()
End While
Dim User As GeckoInputElement = GeckoWebBrowser1.Document.GetHtmlElementById("edit-name")
Dim Pass As GeckoInputElement = GeckoWebBrowser1.Document.GetHtmlElementById("edit-pass")
User.Value = "myUsername"
Pass.Value = "myPassword"
Dim Form = CType(GeckoWebBrowser1.Document.Forms(1), GeckoFormElement)
Form.submit()
Flag_Completed = false
While Not Flag_Completed
Application.DoEvents()
End While
...
Public Sub Navigate(ByVal URL As String, Optional ByVal LimitTimeinMinutes As Integer = 1)
Flag_Completed = False
GeckoWebBrowser1.Width = 1920
Application.DoEvents()
Try
GeckoWebBrowser1.Navigate(URL)
Catch ex As Exception
End Try
End Sub
Everything seems to work so far but I can't find a way to download the file properly. I tried using WebClient in combination with the DownloadFile() Method like so:
Dim myWebClient As New WebClient()
myWebClient.DownloadFile("http://www.website.com/path/file.pdf", "C:\Path\to\local\file.pdf")
The problem is that the WebClient is not logged in (in contrary to my GeckoFX browser (GeckoWebBrowser1). What happens is that I end up with a file that has a .pdf file extension but opening it in a text editor makes clear that the file is actually the HTML webpage that would show up on the screen when you enter the link without being logged-in. (Makes sense)
Unfortunately I have searched for more than a day now and can't find an answer to my particular problem. There doesn't seem to be a method within the GeckoFX library what could take the DownloadFile()-method's place. I found the following question on here: How to handle downloading in GeckoFX 29 which seems to be similar to my problem. Sadly for me, the solution is aimed at a Windows.Forms application in C# and I can't seem to make use of that in my own VB.NET Console-Application. Would this be the right way to approach this? If yes, any ideas? If not, how exactly?
UPDATE:
For completeness sake I will mention that I solved my particular problem (downloading a PDF behind a login) but I didn't use GeckoFX but instead used a WebClient and HttpRequests to download the file. I highly recommend the following Tutorial that explains just that: http://odetocode.com/Articles/162.aspx

Related

Error thrown trying launch a webpage from a button

I created a button and a menu item in vb.net (.NET 6). I found several answers here on SO that say the process to launching a webpage from such an event can be launched with this code:
Dim webAddress As String = "http://www.example.com/"
Process.Start(webAddress)
However, trying launch the code, I'm given the error of "system cannot find the file specified".
Looking more into it, I know that .NET 6 is running a bit differently and changed the code to the following:
Using link As New Process()
link.StartInfo.UseShellExecute = True
link.Start(New ProcessStartInfo("https://example.com"))
End Using
But still to no avail, and I am given the same error. "System cannot find the file specified." I can run addresses via the regular Windows Run prompt... but the program still cannot launch.
Following Jimi's comment to my original question, I changed the Sub to the following:
Sub LaunchWebsite(strWebpageURL As String)
Using Process.Start(New ProcessStartInfo(strWebpageURL) With {.UseShellExecute = True})
End Using
End Sub
Using this, the webpage launched in my desktop's default browser with no problem.
You can use
Respone.Redirect("http://www.example.com/")
or use javascript in server side code
Dim url As String = "http://www.example.com"
Dim s As String = "window.open('" & url + "', 'popup_window', 'width=300,height=100,left=100,top=100,resizable=yes');"
ClientScript.RegisterStartupScript(Me.GetType(), "script", s, True)
Above code open webpage in new popup window.
Regards
Aravind

vb.net downloading a file without popups

I am trying to download tracking information from FedEx's website. First I have to pass login credentials and then navigate to a webpage which triggers an automatic download. My code so far works fine for this. However; as soon as the download is triggered a popup comes up asking if I want to save the file and then when I click save I asks where I want to save the file to. I want to disable the popups and have the file just automatically saved in my downloads folder. Is there a way to do this? Thanks in advance for any help!
Here is my code so far.
Dim WebBrowser1 As New WebBrowser
Dim url As String = "https://www.fedex.com/insight/manifest/manifest.jsp?&__QS=252D0F3B4E380B211B122B1A09251510050E0F5C273A223E34360539237976645E45745E57776C&"
WebBrowser1.Navigate(url)
WebBrowser1.ScriptErrorsSuppressed = True
Threading.Thread.Sleep(2000)
Application.DoEvents()
Do Until WebBrowser1.IsBusy = False
Threading.Thread.Sleep(1000)
Application.DoEvents()
Loop
WebBrowser1.Document.GetElementById("username").SetAttribute("value", "MyUsername")
WebBrowser1.Document.GetElementById("password").SetAttribute("value", "MyP#ssw0rd")
WebBrowser1.Document.GetElementById("login").InvokeMember("click")
Threading.Thread.Sleep(1000)
Application.DoEvents()
Do Until WebBrowser1.IsBusy = False
Threading.Thread.Sleep(1000)
Application.DoEvents()
Loop
WebBrowser1.Navigate("https://www.fedex.com/insight/manifest/download_post.jsp?VIEW=|Outbound_View&INFOTYPE=STATUS")
Do Until WebBrowser1.IsBusy = False
Threading.Thread.Sleep(1000)
Application.DoEvents()
Loop
So I tried this code but all this does is still just download the page's source.
Using client As New WebClient()
client.Headers("User-Agent") = "Mozilla/4.0"
client.Credentials = New NetworkCredential("Username", "P#sword")
client.Credentials = CredentialCache.DefaultCredentials ' << if Windows Authentication
Dim content As String = client.DownloadString("https://www.fedex.com/insight/manifest/download.jsp?VIEW=/Outbound_View")
Console.WriteLine(content.Substring(0, 15))
End Using
I checked the webpage's source again to try to find a different url but also could not find anything else. I was wondering if it would be easier to download using WebBrowser.Navigate and then suppress the popups. Is there away to suppress or get rid of the download popups in vb? I am just not having much success with the WebClient. Thanks for all the help as I am still pretty new at vb.net!

Identifying if a JPG file is open

I've been trying to set an error trap that will detect if a file is already open. This is no problem when the file is a text file using the following code:
Private Function FILEOPEN(ByVal sFile As String) As Boolean
Dim THISFILEOPEN As Boolean = False
Try
Using f As New IO.FileStream(sFile, IO.FileMode.Open)
THISFILEOPEN = False
End Using
Catch
THISFILEOPEN = True
End Try
Return THISFILEOPEN
End Function
My problem is that when the file is an open JPG file, not a text file, the above function returns False indicating that it is not open? I have tried different variations of the function but still cannot find a function that can tell if a JPG file is open.
You should NOT do this kind of behavior. Simple answer is because after you check, but before you do anything with it, the file may become unavailable. A proper way is to handle an exception as you access the file. You may find this answer helpful:
https://stackoverflow.com/a/11288781/897326

Download URL Contents Directly into String (VB6) WITHOUT Saving to Disk

Basically, I want to download the contents of a particular URL (basically, just HTML codes in the form of a String) into my VB6 String variable. However, there are some conditions.
I know about the URLDownloadToFile Function - however, this requires that you save the downloaded file/HTML onto a file location on disk before you can read it into a String variable, this is not an option for me and I do not want to do this.
The other thing is, if I need to use an external library, it must already come with all versions of Windows from XP and onwards, I cannot use a control or library that I am required to ship, package and distribute even if it is free, this is not an option and I do not want to do this. So, I cannot use the MSINET.OCX (Internet Transfer) Control's .OpenURL() function (which simply returns contents into a String), as it does not come with Windows.
Is there a way to be able to do this with the Windows API, URLMON or something else that is pre-loaded into or comes with Windows, or a way to do it in VB6 (SP6) entirely?
If so, I would appreciate direction, because even after one hour of googling, the only examples I've found are references to URLDownloadToFile (which requires saving on disk before being ale to place into a String) and MsInet.OpenURL (which requires that I ship and distribute MSINET.OCX, which I cannot and don't want to do).
Surely there has got to be an elegant way to be able to do this? I can do it in VB.NET without an issue, but obviously don't have the luxury of the .NET framework in VB6 - any ideas?
Update:
I have found this: http://www.freevbcode.com/ShowCode.asp?ID=1252
however it says that the displayed function may not return the entire
page and links to a Microsoft bug report or kb article explaining
this. Also, I understand this is based off wininet.dll - and I'm
wondering which versions of Windows does WinInet.dll come packaged
with? Windows XP & beyond? Does it come with Windows 7 and/or Windows
8?
This is how I did it with VB6 a few years ago:
Private Function GetHTMLSource(ByVal sURL As String) As String
Dim xmlHttp As Object
Set xmlHttp = CreateObject("MSXML2.XmlHttp")
xmlHttp.Open "GET", sURL, False
xmlHttp.send
GetHTMLSource = xmlHttp.responseText
Set xmlHttp = Nothing
End Function
If you want to do this with pure VB, and no IE, then you can take advantage of a little-used features of the VB UserControl - async properties.
Create a new UserControl, and call it something like UrlDownloader. Set the InvisibleAtRuntime property to True. Add the following code to it:
Option Explicit
Private Const m_ksProp_Data As String = "Data"
Private m_bAsync As Boolean
Private m_sURL As String
Public Event AsyncReadProgress(ByRef the_abytData() As Byte)
Public Event AsyncReadComplete(ByRef the_abytData() As Byte)
Public Property Let Async(ByVal the_bValue As Boolean)
m_bAsync = the_bValue
End Property
Public Property Get Async() As Boolean
Async = m_bAsync
End Property
Public Property Let URL(ByVal the_sValue As String)
m_sURL = the_sValue
End Property
Public Property Get URL() As String
URL = m_sURL
End Property
Public Sub Download()
UserControl.AsyncRead m_sURL, vbAsyncTypeByteArray, m_ksProp_Data, IIf(m_bAsync, 0&, vbAsyncReadSynchronousDownload)
End Sub
Private Sub UserControl_AsyncReadComplete(AsyncProp As AsyncProperty)
If AsyncProp.PropertyName = m_ksProp_Data Then
RaiseEvent AsyncReadComplete(AsyncProp.Value)
End If
End Sub
Private Sub UserControl_AsyncReadProgress(AsyncProp As AsyncProperty)
If AsyncProp.PropertyName = m_ksProp_Data Then
Select Case AsyncProp.StatusCode
Case vbAsyncStatusCodeBeginDownloadData, vbAsyncStatusCodeDownloadingData, vbAsyncStatusCodeEndDownloadData
RaiseEvent AsyncReadProgress(AsyncProp.Value)
End Select
End If
End Sub
To use this control, stick it on a form and use the following code:
Option Explicit
Private Sub Command1_Click()
XDownload1.Async = False
XDownload1.URL = "http://www.google.co.uk"
XDownload1.Download
End Sub
Private Sub XDownload1_AsyncReadProgress(the_abytData() As Byte)
Debug.Print StrConv(the_abytData(), vbUnicode)
End Sub
Suffice to say, you can customise this to your hearts content. It can tell (using the AyncProp object) whether the file is cached, and other useful information. It even has a special mode in which you can download GIF, JPG and BMP files and return them as a StdPicture object!
One alternative is using Internet Explorer.
Dim ex As InternetExplorer
Dim hd As HTMLDocument
Dim s As String
Set ex = New InternetExplorer
With ex
.Navigate "http://donttrack.us/"
.Visible = 1
Set hd = .Document
s = hd.body.innerText ' assuming you just want the text
's = hd.body.innerHTML ' if you want the HTML
End With
EDIT: For the above early binding to work you need to set references to "Microsoft Internet Controls" and "Microsoft HTML Object Library" (Tools > References). You could also use late binding, but to be honest, I forget what the proper class names are; maybe someone smart will edit this answer :-)

FtpWebRequest.GetRequestStream hang up and fails.

I have wrote a web service, in a nutshell it uses openpop to get email messages does stuff with the content to insert into databases and saves attachments which are images. That works fine when i save images locally, it does exactley what it is suppose to. Now an added requirment was to save images to an FTP directory, so i can create my folders dynamically (they are created based upon timestamp) and that works well. My problem comes from when i try to save them to the ftp. Yes my user name and password are correct, otherwise i wouldn't be creating the directory.
Private Sub UploadFile(ByVal fileToSave As FileInfo, ByVal path As String)
Dim UploadRequest As FtpWebRequest = DirectCast(WebRequest.Create("ftp://UserName:Passowrd#999.99.999.9" & path), FtpWebRequest)
UploadRequest.Credentials = New NetworkCredential("PicService", "grean.matching18")
UploadRequest.Method = System.Net.WebRequestMethods.Ftp.UploadFile
UploadRequest.UseBinary = True
UploadRequest.UsePassive = True
' Const BufferSize As Integer = 2048
' Dim content(BufferSize - 1) As Byte, dataRead As Integer
Dim bFile() As Byte = System.IO.File.ReadAllBytes(fileToSave.ToString)
'UploadRequest.ContentLength = content.Length
Using FileStream1 As FileStream = fileToSave.OpenRead()
Try
'open request to send
Using RequestStream As Stream = UploadRequest.GetRequestStream
End Using
Catch ex As Exception
Finally
'ensure file closed
FileStream1.Close()
End Try
End Using
End Sub
I have tried using Passive False and Binary False as well, i did more research on my stack trace.
And found this article but no solution as of yet. Any input would be appreciated, i am also posting another question on windows services for different issue. If you would like to take a shot at it, the other question isnt about ftp but permissions for a service on windows server 2003
This may not be the solution but I've found that the URI string has to be 'just right' and that what is 'just right' varies by the ftp server.
So ftp://server/directory/file works on some servers but needs to be ftp://server//directory/file to work on others (note the double slash after the server name)
Aso, your URI has 'password' spelled incorrectly: ftp://UserName:Passowrd#999.99.999.9 and you are supplying the credentials in a separate code line as well.