Populate Web Form From VB Application - vb.net

I have created a simple FORM in VB.NET that takes some details and then needs to log in to 3 locations using this information.
At the moment I have the code so it takes this data from the textBoxs and assigns them to 4 different variables. From there I have also opened up the three different websites.
I am having difficulties finding how I will take the variables and then populate the corresponding field on the web application. Any suggestions?
My Code:
Public Class Form1
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
'Define Store variable
Dim Store As String
Store = Me.TextBox1.Text
'Define IP Address variable
Dim IPAddress As String
IPAddress = Me.TextBox2.Text
'Define Username variable
Dim Username As String
Username = Me.TextBox3.Text
'Define Password variable
Dim Password As String
Password = Me.TextBox4.Text
' Open Store Specific URL 1
Dim WebAddress1 As String = "http://" & IPAddress & ":"
Process.Start(WebAddress1)
getElementByName
' Open Store Specific URL 2
Dim WebAddress2 As String = "http://somedomain2.com"
Process.Start(WebAddress2)
' Open Store Specific URL 3
Dim WebAddress3 As String = "http://somedomain3.com"
Process.Start(WebAddress3)
End Sub
End Class

What you need to do is identify the element name that you want to populate. This can typically done by going to the web page, and pressing View Source (changes by web browser, some you can right click and it will be there, some you can access through the settings button.)
Once looking at the source, you will want to find the object (usually a text box or something along those lines) where you want to send the information. Usually these boxes have titles, like Username, or Password. So I would recommend doing a Ctrl + F search based on the information you can see on the site. I see in your code you have GetElementByName, and that's exactly what you'll do. You will want to store
Here's an example code:
Dim IE As Object 'Internet explorer object
Dim objCollection As Object 'Variable used for cycling through different elements
'Create IE Object
IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.Navigate("https://somewebsite.com/") 'Your website
Do While IE.Busy
Application.DoEvents() 'This allows the site to load first
Loop
'Find the field you are looking for and store it into the objCollection variable
objCollection = IE.document.getelementsbyname("CustomerInfo.AccountNumber") 'The "CustomerInfo.AccountNumber" is the name of the element I looked for in this case.
'Call element, and set value equal to the data you have from your form
objCollection(0).Value = MainForm.tbLoan.Text
' Clean up
IE = Nothing
objCollection = Nothing
This should be a good start for you. There are multiple resources on this site that might be able to give you additional information when it comes to entering data into websites using vb.net.
Hopefully this helps!

Related

Is there a way to retrieve some text from a webpage to a textbox in VB?

I'm trying to make it so I can have several text boxes in my form show pieces of information from a specific webpage. For example, would there be a way I would be able to retrieve the title of this question to a variable with the click of a button in Visual Basic?
It not hard but you have to look at the source page, and identify the elements.
In nicely formed pages, usually the div elements have a tag ID, but often they don't, so you have to grab say by a attribute name - often you can use the class name of the div in question.
So, to grab the Title, and you question text of this post?
This works:
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim xDoc As New Xml.XmlDocument
Dim strURL As String = "https://stackoverflow.com/questions/55753982"
Dim xWeb As New WebBrowser
xWeb.ScriptErrorsSuppressed = True
xWeb.Navigate(strURL)
Do Until xWeb.ReadyState = WebBrowserReadyState.Complete
Application.DoEvents()
Loop
Dim HDoc As HtmlDocument = xWeb.Document
Debug.Print(HDoc.GetElementById("question-header").FirstChild.InnerText)
Debug.Print(FindClass(HDoc, "post-text"))
End Sub
Function FindClass(Hdoc As HtmlDocument, strClass As String) As String
' get all Divs, and search by class name
Dim OneElement As HtmlElement
For Each OneElement In Hdoc.GetElementsByTagName("div")
If OneElement.GetAttribute("classname") = strClass Then
Return OneElement.InnerText
End If
Next
' we get here, not found, so return a empty stirng
Return "not found"
End Function
OutPut:
(first part is title question)
Is there a way to retrieve some text from a webpage to a textbox in VB?
(second part is question text)
I'm trying to make it so I can have several text boxes in my form show pieces of
information from a specific webpage. For example, would there be a way I would be
able to retrieve the title of this question to a variable with the click of a button
in Visual Basic?

Controling Web browser through VB.net application

I want to open separate instances of internet explorer to specified websites. After they are open I would like to cycle through them being displayed on a timer.
I have the following code, but I am not able to switch to the specified IE process:
Dim rotatethrough As Boolean = True
For i = 0 To ListBox1.Items.Count - 1 'I have a list box that contains the website URLs
Dim Processname As New List(Of String)
Dim processnum(Environment.ProcessorCount) As Process
processnum(i) = New Process
processnum(i) = System.Diagnostics.Process.Start(ListBox1.Items(i)) 'start up seperate instances of IE for each website
Next
Do While rotatethrough = True
For n = 0 To ListBox1.Items.Count - 1
AppActivate(processnum(n).Id) 'activate the websites
Threading.Thread.Sleep(1000)
Next
Loop
So far the code opens up the separate instance of IE but fails on appactivate because " Object reference not set to an instance of an object.". I have tried creating IE as an object, but then I am not sure how to get the correct process ID for the corresponding listbox item.
ANY help would be awesome, I just can seem to figure this one out.

System.UnauthorizedAccessException only using multithreading

I wrote a code to parse some Web tables.
I get some web tables into an IHTMLElementCollection using Internet Explorer with this code:
TabWeb = IE.document.getelementsbytagname("table")
Then I use a sub who gets an object containing the IHTMLElementCollection and some other data:
Private Sub TblParsing(ByVal ArrVal() As Object)
Dim WTab As mshtml.IHTMLElementCollection = ArrVal(0)
'some code
End sub
My issue is: if I simply "call" this code, it works correctly:
Call TblParsing({WTab, LiRow})
but, if I try to run it into a threadpool:
ThreadPool.QueueUserWorkItem(New WaitCallback(AddressOf TblParsing), {WTab, LiRow})
the code fails and give me multiple
System.UnauthorizedAccessException
This happens on (each of) these code rows:
Rws = WTab(RifWT("Disc")).Rows.Length
If Not IsError(WTab(6).Cells(1).innertext) Then
Ogg_W = WTab(6).Cells(1).innertext
My goal is to navigate to another web page while my sub perform parsing.
I want to clarify that:
1) I've tryed to send the entire HTML to the sub and get it into a webbrowser but it didn't work because it isn't possible to cast from System.Windows.Forms.HtmlElementCollection to mshtml.IHTMLElementCollection (or I wasn't able to do it);
2) I can't use WebRequest and similar: I'm forced to use InternetExplorer;
3) I can't use System.Windows.Forms.HtmlElementCollection because my parsing code uses Cells, Rows and so on that are unavailable (and I don't want to rewrite all my parsing code)
EDIT:
Ok, I modified my code using answer hints as below:
'This in the caller sub
Dim IE As Object = CreateObject("internetexplorer.application")
'...some code
Dim IE_Body As String = IE.document.body.innerhtml
ThreadPool.QueueUserWorkItem(New WaitCallback(AddressOf TblParsing_2), {IE_Body, LiRow})
'...some code
'This is the called sub
Private Sub TblParsing_2(ByVal ArrVal() As Object)
Dim domDoc As New mshtml.HTMLDocument
Dim domDoc2 As mshtml.IHTMLDocument2 = CType(domDoc, mshtml.IHTMLDocument2)
domDoc2.write(ArrVal(0))
Dim body As mshtml.IHTMLElement2 = CType(domDoc2.body, mshtml.IHTMLElement2)
Dim TabWeb As mshtml.IHTMLElementCollection = body.getElementsByTagName("TABLE")
'...some code
I get no errors but I'm not sure that it's all right because I tryed to use IE_Body string into webbrowser and it throws errors in the webpage (it shows a popup and I can ignore errors).
Am I using the right way to get Html from Internet Explorer into a string?
EDIT2:
I changed my code to:
Dim IE As New SHDocVw.InternetExplorer
'... some code
Dim sourceIDoc3 As mshtml.IHTMLDocument3 = CType(IE.Document, mshtml.IHTMLDocument3)
Dim html As String = sourceIDoc3.documentElement.outerHTML
ThreadPool.QueueUserWorkItem(New WaitCallback(AddressOf TblParsing_2), {html, LiRow})
'... some code
Private Sub TblParsing_2(ByVal ArrVal() As Object)
Dim domDoc As New mshtml.HTMLDocument
Dim domDoc2 As mshtml.IHTMLDocument2 = CType(domDoc, mshtml.IHTMLDocument2)
domDoc2.write(ArrVal(0))
Dim body As mshtml.IHTMLElement2 = CType(domDoc2.body, mshtml.IHTMLElement2)
Dim TabWeb As mshtml.IHTMLElementCollection = body.getElementsByTagName("TABLE")
But I get an error PopUp like (I tryed to translate it):
Title:
Web page error
Text:
Debug this page?
This page contains errors that might prevent the proper display or function properly.
If you are not testing the web page, click No.
two checkboxes
do not show this message again
Use script debugger built-in Internet Explorer
It's the same error I got trying to get Html text into a WebBrowser.
But, If I could ignore this error, I think the code could work!
While the pop is showing I get error on
Dim domDoc As New mshtml.HTMLDocument
Error text translated is:
Retrieving the COM class factory for component with CLSID {25336920-03F9-11CF-8FD0-00AA00686F13} failed due to the following error: The 8,001,010th message filter indicated that the application is busy. (Exception from HRESULT: 0x8001010A (RPC_E_SERVERCALL_RETRYLATER)).
Note that I've alredy set IE.silent = True
Edit: There was confusion as to what the OP meant by "Internet Explorer". I originally assumed that it meant the WinForm Webbrowser control; however the OP is creating the COM browser directly instead of using the .Net wrapper.
To get the browser document's defining HTML, you can cast the document against the mshtml.IHTMLDocument3 interface to expose the documentElement property.
Dim ie As New SHDocVw.InternetExplorer ' Proj COM Ref: Microsoft Internet Controls
ie.Navigate("some url")
' ... other stuff
Dim sourceIDoc3 As mshtml.IHTMLDocument3 = CType(ie.Document, mshtml.IHTMLDocument3)
Dim html As String = sourceIDoc3.documentElement.outerHTML
End Edit.
The following is based on my comment above. You use the WebBrowser.DocumentText property to create a mshtml.HTMLDocument.
Use this property when you want to manipulate the contents of an HTML page displayed in the WebBrowser control using string processing tools.
Once you extract this property as a String, there is no connection to the WebBrowser control and you can process the data in any thread you want.
Dim html As String = WebBrowser1.DocumentText
Dim domDoc As New mshtml.HTMLDocument
Dim domDoc2 As mshtml.IHTMLDocument2 = CType(domDoc, mshtml.IHTMLDocument2)
domDoc2.write(html)
Dim body As mshtml.IHTMLElement2 = CType(domDoc2.body, mshtml.IHTMLElement2)
Dim tables As mshtml.IHTMLElementCollection = body.getElementsByTagName("TABLE")
' ... do something
' cleanup COM objects
System.Runtime.InteropServices.Marshal.FinalReleaseComObject(body)
System.Runtime.InteropServices.Marshal.FinalReleaseComObject(tables)
System.Runtime.InteropServices.Marshal.FinalReleaseComObject(domDoc)
System.Runtime.InteropServices.Marshal.FinalReleaseComObject(domDoc2)

How to Post & Retrieve Data from Website

I am working with a Windows form application. I have a textbox called "tbPhoneNumber" which contains a phone number.
I want to go on the website http://canada411.com and enter in the number that was in my textbox, into the website textbox ID: "c411PeopleReverseWhat" and then somehow send a click on "Find" (which is an input belonging to class "c411ButtonImg").
After that, I want to retrieve what is in between the asterixs of the following HTML section:
<div id="contact" class="vcard">
<span><h1 class="fn c411ListedName">**Full Name**</h1></span>
<span class="c411Phone">**(###)-###-####**</span>
<span class="c411Address">**Address**</span>
<span class="adr">
<span class="locality">**City**</span>
<span class="region">**Province**</span>
<span class="postal-code">**L#L#L#**</span>
</span>
So basically I am trying to send data into an input box, click the input button and store the values retrieved into variables. I want to do this seemlessly so I would need to do something like an HTTPWebRequest? Or do I use a WebBrowser object? I just don't want the user to see that the application is going on a website.
I do a good amount of website scraping and I will show you how I do it. Feel free to skip ahead if I am being too specific, but this is a commonly requested theme and should be made specific.
URL Simplification
The library I use for this is htmlagilitypack (It is a dll, make a new project and add a reference to it). The first thing to check is if we have to go to take any special steps to get to a page by using a phone number. I searched for John Smith and found quite a few. I entered 2 of these results and noticed that the url formatting is very simple. Those results were..
http://www.canada411.ca/res/7056736767/John-Smith/138223109.html
http://www.canada411.ca/res/7052355273/John-Smith/172439951.html
I tested to see if I can remove some of the values from the url that I don't know and just leave the phone number. The result was that I can...
http://www.canada411.ca/search/re/1/7056736767/-
http://www.canada411.ca/search/re/1/7052355273/-
You can see by the url that there are some static areas in the url and our phone number. From this lets construct a string for the url.
Dim phoneNumber as string = "7056736767" 'this could be TextBox1.Text or whatever
Dim URL as string = "http://www.canada411.ca/search/re/1/" + phoneNumber +"/-"
Value Extraction with XPath
Now that we have the page dialed in, lets examine the html you provided above. You need 6 values from the page so we will create them now...
Dim FullName As String
Dim Phone As String
Dim Address As String
Dim Locality As String
Dim Region As String
Dim PostalCode As String
As mentioned above, we will be using htmlagilitypack which uses Xpath. The cool thing about this is that once we can find some unique identifier in the html, we can use Xpath to find our values. I know it may be confusing, but it will become clearer.
All of the values you need are within tags that have a class name. Lets use the class name in our Xpath to find them.
Dim FullNameXPath As String = "//*[#class='fn c411ListedName']"
Dim PhoneXPath As String = "//*[#class='c411Phone']"
Dim AddressXPath As String = "//*[#class='c411Address']"
Dim LocalityXPath As String = "//*[#class='locality']"
Dim RegionXPath As String = "//*[#class='region']"
Dim PostalCodeXPath As String = "//*[#class='postal-code']"
Essentially what we are looking at is a string that will inform htmlagilitypack what to look for. In our case, text contained within the classes we named. There is a lot to XPath and it could take a while to explain all of it. On a side note though...If you use Google Chrome and highlight a value on a page, you can right click inspect element. In the code that appears below, you can right click the value and copy to XPath!!! Very useful.
Basic HTMLAgilityPack Template
Now, all that is left is to connect to the page and get those variables populated.
Dim Web As New HtmlAgilityPack.HtmlWeb
Dim Doc As New HtmlAgilityPack.HtmlDocument
Doc = Web.Load(URL)
For Each nameResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(FullNameXPath)
Msgbox(nameResult.InnerText)
Next
In the above example we create an HtmlWeb object named Web. This is the actual crawler of our project. We then define a HtmlDocument which will consist of our converted and searchable page source. All of this is done behind the scenes. We then send Web to get the page source and assign it to the Doc object we created. Doc is reusable, which thankfully requires us to connect to the page only once.
The for loop looks for any nodes in our Doc that match FullNameXPath which was defined previously as the XPath value for finding the name. When a Node is found, it is assigned to the nameResult variable and from within the loop we call a message box to display the inner text of our node.
So when we put it all together
Complete Working Code (As of 2/17/2013)
Dim phoneNumber As String = "7056736767" 'this could be TextBox1.Text or whatever
Dim URL As String = "http://www.canada411.ca/search/re/1/" + phoneNumber + "/-"
Dim FullName As String
Dim Phone As String
Dim Address As String
Dim Locality As String
Dim Region As String
Dim PostalCode As String
Dim FullNameXPath As String = "//*[#class='fn c411ListedName']"
Dim PhoneXPath As String = "//*[#class='c411Phone']"
Dim AddressXPath As String = "//*[#class='c411Address']"
Dim LocalityXPath As String = "//*[#class='locality']"
Dim RegionXPath As String = "//*[#class='region']"
Dim PostalCodeXPath As String = "//*[#class='postal-code']"
Dim Web As New HtmlAgilityPack.HtmlWeb
Dim Doc As New HtmlAgilityPack.HtmlDocument
Doc = Web.Load(URL)
For Each nameResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(FullNameXPath)
FullName = nameResult.InnerText
MsgBox(FullName)
Next
For Each PhoneResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(PhoneXPath)
Phone = PhoneResult.InnerText
MsgBox(Phone)
Next
For Each ADDRResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(AddressXPath)
Address = ADDRResult.InnerText
MsgBox(Address)
Next
For Each LocalResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(LocalityXPath)
Locality = LocalResult.InnerText
MsgBox(Locality)
Next
For Each RegionResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(RegionXPath)
Region = RegionResult.InnerText
MsgBox(Region)
Next
For Each postalCodeResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(PostalCodeXPath)
PostalCode = postalCodeResult.InnerText
MsgBox(PostalCode)
Next
Yes it is possible, I've done this using the selenium framework, which is aimed for testing automation. However, it provides you with the tools to do exactly that.
Download for .net here:
http://docs.seleniumhq.org/download/

how to refer back to created browser instance

I create a browser like so, and manually navigate to the web page I need to be. I intend to automatically pull certain elements once I get to the page I need to be on via a seperate macro
Sub Test()
Set CAS = New SHDocVw.InternetExplorer ' create a browser
CAS.Visible = True ' make it visible
CAS.navigate "http://intraneturl"
Do Until CAS.readyState = 4
DoEvents
Loop
This works fine, then I do
Public Sub Gather
Set HTMLDoc2 = CAS.document.frames("top").document
Call Timer1
With HTMLDoc2
.getElementById("tab4").FirstChild.Click
End With
Call Timer2
Dim fir, las, add1, add2, cit, stat, zi As String
Dim First As Variant
Dim Last As Variant
Dim addr1 As Variant
Dim addr2 As Variant
Dim city As Variant
Dim Thisstate As Variant
Dim Zip As Variant
Call Timer2
Set HTMLDoc = CAS.document.frames("MainFrame").document
Call Timer2
With HTMLDoc
First = .getElementsByName("IndFirst")
Last = .getElementsByName("IndLast")
addr1 = .getElementsByName("txtAdd_Line1")
addr2 = .getElementsByName("txtAdd_Line2")
city = .getElementsByName("txtAdd_City")
Thisstate = .getElementsByName("cmb_Add_State")
Zip = .getElementsByName("txtAdd_Zip")
End With
fir = First.Value
las = Last.Value
add1 = addr1.Value
add2 = addr2.Value
cit = city.Value
stat = Thisstate.Value
zi = Zip.Value
'navigate back to start page
With HTMLDoc2
.getElementById("tab3").FirstChild.Click
End With
End Sub
This works the first time, but after the first time, I get "Object variable or with block variable not set" when trying to run the gather() sub again, on a different web page that contains similar information. Any Ideas as to what im doing wrong?
"The error "object variable or with block variable not set" occurs on: Set HTMLDoc2 = CAS.document.frames("top").document the second time i try running Gather()."
This is probably one of three things:
CAS is no longer an object
To check this, set a breakpoint on the line, press ctr+G in the VBA Editor and type ?CAS Is Nothing in the Immediate Window; the result should be False; if it is True CAS is no longer an object
Like Daniel Dusek suggested, make sure CAS.document.frames("top") is an actual element on the page.
To check this, open the webpage you are trying to script, press F12 in Internet Explorer, click on the arrow in the toolbar and click on the "top" frame element in the webpage, switch back to the Developer Tool and look at the line highlighted. Make sure the frame element is named "top".
The HTML hasn't fully loaded when you try to reference the frame element. Set a longer delay or a loop.
i.e. (untested):
Do Until HtmlDoc2 Is Nothing = false
Set HTMLDoc2 = CAS.document.frames("top").document
Loop
Maybe the more important question is why manually navigate to another page? Can't you automate that part of your process too?