Ive been trying to get this value from wb forever now.. And can't do it.
What I am trying to get is "388399"
However "ember557" changes constantly to "ember'random number'" and the class is not unique.
<div id="ember557" class="ember-view">388399</div>
Maybe you could try to get it with the CLASS instead of the ID
you can call GetElementByTagName and then iterate through this to find the element of the relevant class
And if that doesnt work then you could also try to use regex to get the string you want. Then replace the unwanted characters.
Dim myString As String = "your html string here"
Dim regex = New Regex("<div id=""ember[0-9]*"" class=""ember-view"">[0-9]*</div>")
Dim match = regex.Match(myString)
If match.Success Then
'replace the unwanted characters.
End If
Related
Is there any way to search only the first line of a Multiline Textbox without knowing exactly at what position the text is you're looking for?
If I knew the position of the text I was looking for I could do something like:
Dim myNotes As String = "The book has a lot of text"
Dim myText As String = "text"
If Not myNotes.Substring(0,4) = myText Then
' Do Something
End If
Or if I wanted to search the entire textbox I could do something like:
Dim myNotes As String = "The book has a lot of text"
Dim myText As String = "text"
If Not myNotes.Contains(myText) Then
' Do Something
End If
But I want to search only the first line of the textbox and I'm not sure at what position the text may be. Is there anyway to do a search like that?
This is another example of why you should ALWAYS read the relevant documentation. If you had read the documentation for the TextBox class then you'd know that it has a Lines property. To get the first line of text, you simply get the first element of that array:
Dim firstLine = myTextBox.Lines(0)
If Not filrstLine.Contains(myText) Then
'Do something
End If
Note that this only applies where the user has explicitly added a line break to the text. I assume that that is what you want, given that you have accepted another answer that does the same thing. If you mean the first line based on automatic word-wrap then that requires a bit more effort.
You could take the text and extract the first line.
int pos = text.IndexOfAny('\r', '\n');
if (pos >= 0)
text = text.SubString(0, pos);
// text now contains only the first line
Then you can search the resulting string.
I am making a small "home" application using VB. As the title says, I want to grab a part of text from a local html file and use it as variable, or put it in a textbox.
I have tried something like this...
Private Sub Open_Button_Click(sender As Object, e As EventArgs) Handles Open_Button.Click
Dim openFileDialog As New OpenFileDialog()
openFileDialog.CheckFileExists = True
openFileDialog.CheckPathExists = True
openFileDialog.FileName = ""
openFileDialog.Filter = "All|*.*"
openFileDialog.Multiselect = False
openFileDialog.Title = "Open"
If openFileDialog.ShowDialog = Windows.Forms.DialogResult.OK Then
Dim fileReader As String = My.Computer.FileSystem.ReadAllText(openFileDialog1.FileName)
TextBox.Text = fileReader
End If
End Sub
The result is to load the whole html code inside this textbox. What should I do so to grab a specific part of html files's code? Let's say I want to grab only the word text from this span...<span id="something">This is a text!!!</a>
I make the following assumptions on this answer.
Your html is valid - i.e. the id is completely unique in the document.
You will always have an id on your html tag
You'll always be using the same tag (e.g. span)
I'd do something like this:
' get the html document
Dim fileReader As String = My.Computer.FileSystem.ReadAllText(openFileDialog1.FileName)
' split the html text based on the span element
Dim fileSplit as string() = fileReader.Split(New String () {"<span id=""something"">"}, StringSplitOptions.None)
' get the last part of the text
fileReader = fileSplit.last
' we now need to trim everything after the close tag
fileSplit = fileReader.Split(New String () {"</span>"}, StringSplitOptions.None)
' get the first part of the text
fileReader = fileSplit.first
' the fileReader variable should now contain the contents of the span tag with id "something"
Note: this code is untested and I've typed it on the stack exchange mobile app, so there might be some auto correct typos in it.
You might want to add in some error validation such as making sure that the span element only occurs once, etc.
Using an HTML parser is highly recommended due to the HTML language's many nested tags (see this question for example).
However, finding the contents of a single tag using Regex is possible with no bigger problems if the HTML is formatted correctly.
This would be what you need (the function is case-insensitive):
Public Function FindTextInSpan(ByVal HTML As String, ByVal SpanId As String, ByVal LookFor As String) As String
Dim m As Match = Regex.Match(HTML, "(?<=<span.+id=""" & SpanId & """.*>.*)" & LookFor & "(?=.*<\/span>)", RegexOptions.IgnoreCase)
Return If(m IsNot Nothing, m.Value, "")
End Function
The parameters of the function are:
HTML: The HTML code as string.
SpanId: The id of the span (ex. <span id="hello"> - hello is the id)
LookFor: What text to look for inside the span.
Online test: http://ideone.com/luGw1V
I'm writing an app in vb.net that needs public IP address in text only format. I'd knew there is lots of site that give you your IP in text format. But where is always a chance of being closed or getting out of service. But the Google will not ever stop! Now I want to get my ip from google searching. For example if you search "my ip" in google it will bring your ip like this:
Sample of search
Is anyway to get IP from Google?
Thanks guys but I found a way:
At first import some namespaces:
Imports System.Net
Imports System.Text.RegularExpressions
Now lets write a function:
Dim client As New WebClient
Dim To_Match As String = "<div class=""_h4c _rGd vk_h"">(.*)"
Dim recived As String = client.DownloadString("https://www.google.com/search?sclient=psy-ab&site=&source=hp&btnG=Search&q=my+ip")
Dim m As Match = Regex.Match(recived, To_Match)
Dim text_with_divs As String = m.Groups(1).Value
Dim finalize As String() = text_with_divs.Split("<")
Return finalize(0)
It's now working and live!
Hard-coded Div Class names make me a bit nervous because they can easily change at any time, so, I expanded on Hirod Behnam's example a little bit.
I removed the Div Class pattern, replacing it with a simpler IP address search, and it'll return only the first one found, which for this search, should be the first one shown on the page (your external IP).
That also removed the need for splitting the results into the array, and those related vars. I also simplified the Google search string to the bare minimum.
It'd still probably be a nice touch to include a timeout or two for the .DownloadString() and the .Match() respectively, if speed is of the essence.
Private Function GetExternalIP() As String
Dim m As Match = Match.Empty
Try
Dim wClient As New System.Net.WebClient
Dim strURL As String = wClient.DownloadString("https://www.google.com/search?q=my+ip")
Dim strPattern As String = "\b(?:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\b"
' Look for the IP
m = Regex.Match(strURL, strPattern)
Catch ex As Exception
Debug.WriteLine(String.Format("GetExternalIP Error: {0}", ex.Message))
End Try
' Failed getting the IP
If m.Success = False Then Return "IP: N/A"
' Got the IP
Return m.value
End Function
I am working with a Windows form application. I have a textbox called "tbPhoneNumber" which contains a phone number.
I want to go on the website http://canada411.com and enter in the number that was in my textbox, into the website textbox ID: "c411PeopleReverseWhat" and then somehow send a click on "Find" (which is an input belonging to class "c411ButtonImg").
After that, I want to retrieve what is in between the asterixs of the following HTML section:
<div id="contact" class="vcard">
<span><h1 class="fn c411ListedName">**Full Name**</h1></span>
<span class="c411Phone">**(###)-###-####**</span>
<span class="c411Address">**Address**</span>
<span class="adr">
<span class="locality">**City**</span>
<span class="region">**Province**</span>
<span class="postal-code">**L#L#L#**</span>
</span>
So basically I am trying to send data into an input box, click the input button and store the values retrieved into variables. I want to do this seemlessly so I would need to do something like an HTTPWebRequest? Or do I use a WebBrowser object? I just don't want the user to see that the application is going on a website.
I do a good amount of website scraping and I will show you how I do it. Feel free to skip ahead if I am being too specific, but this is a commonly requested theme and should be made specific.
URL Simplification
The library I use for this is htmlagilitypack (It is a dll, make a new project and add a reference to it). The first thing to check is if we have to go to take any special steps to get to a page by using a phone number. I searched for John Smith and found quite a few. I entered 2 of these results and noticed that the url formatting is very simple. Those results were..
http://www.canada411.ca/res/7056736767/John-Smith/138223109.html
http://www.canada411.ca/res/7052355273/John-Smith/172439951.html
I tested to see if I can remove some of the values from the url that I don't know and just leave the phone number. The result was that I can...
http://www.canada411.ca/search/re/1/7056736767/-
http://www.canada411.ca/search/re/1/7052355273/-
You can see by the url that there are some static areas in the url and our phone number. From this lets construct a string for the url.
Dim phoneNumber as string = "7056736767" 'this could be TextBox1.Text or whatever
Dim URL as string = "http://www.canada411.ca/search/re/1/" + phoneNumber +"/-"
Value Extraction with XPath
Now that we have the page dialed in, lets examine the html you provided above. You need 6 values from the page so we will create them now...
Dim FullName As String
Dim Phone As String
Dim Address As String
Dim Locality As String
Dim Region As String
Dim PostalCode As String
As mentioned above, we will be using htmlagilitypack which uses Xpath. The cool thing about this is that once we can find some unique identifier in the html, we can use Xpath to find our values. I know it may be confusing, but it will become clearer.
All of the values you need are within tags that have a class name. Lets use the class name in our Xpath to find them.
Dim FullNameXPath As String = "//*[#class='fn c411ListedName']"
Dim PhoneXPath As String = "//*[#class='c411Phone']"
Dim AddressXPath As String = "//*[#class='c411Address']"
Dim LocalityXPath As String = "//*[#class='locality']"
Dim RegionXPath As String = "//*[#class='region']"
Dim PostalCodeXPath As String = "//*[#class='postal-code']"
Essentially what we are looking at is a string that will inform htmlagilitypack what to look for. In our case, text contained within the classes we named. There is a lot to XPath and it could take a while to explain all of it. On a side note though...If you use Google Chrome and highlight a value on a page, you can right click inspect element. In the code that appears below, you can right click the value and copy to XPath!!! Very useful.
Basic HTMLAgilityPack Template
Now, all that is left is to connect to the page and get those variables populated.
Dim Web As New HtmlAgilityPack.HtmlWeb
Dim Doc As New HtmlAgilityPack.HtmlDocument
Doc = Web.Load(URL)
For Each nameResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(FullNameXPath)
Msgbox(nameResult.InnerText)
Next
In the above example we create an HtmlWeb object named Web. This is the actual crawler of our project. We then define a HtmlDocument which will consist of our converted and searchable page source. All of this is done behind the scenes. We then send Web to get the page source and assign it to the Doc object we created. Doc is reusable, which thankfully requires us to connect to the page only once.
The for loop looks for any nodes in our Doc that match FullNameXPath which was defined previously as the XPath value for finding the name. When a Node is found, it is assigned to the nameResult variable and from within the loop we call a message box to display the inner text of our node.
So when we put it all together
Complete Working Code (As of 2/17/2013)
Dim phoneNumber As String = "7056736767" 'this could be TextBox1.Text or whatever
Dim URL As String = "http://www.canada411.ca/search/re/1/" + phoneNumber + "/-"
Dim FullName As String
Dim Phone As String
Dim Address As String
Dim Locality As String
Dim Region As String
Dim PostalCode As String
Dim FullNameXPath As String = "//*[#class='fn c411ListedName']"
Dim PhoneXPath As String = "//*[#class='c411Phone']"
Dim AddressXPath As String = "//*[#class='c411Address']"
Dim LocalityXPath As String = "//*[#class='locality']"
Dim RegionXPath As String = "//*[#class='region']"
Dim PostalCodeXPath As String = "//*[#class='postal-code']"
Dim Web As New HtmlAgilityPack.HtmlWeb
Dim Doc As New HtmlAgilityPack.HtmlDocument
Doc = Web.Load(URL)
For Each nameResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(FullNameXPath)
FullName = nameResult.InnerText
MsgBox(FullName)
Next
For Each PhoneResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(PhoneXPath)
Phone = PhoneResult.InnerText
MsgBox(Phone)
Next
For Each ADDRResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(AddressXPath)
Address = ADDRResult.InnerText
MsgBox(Address)
Next
For Each LocalResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(LocalityXPath)
Locality = LocalResult.InnerText
MsgBox(Locality)
Next
For Each RegionResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(RegionXPath)
Region = RegionResult.InnerText
MsgBox(Region)
Next
For Each postalCodeResult As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes(PostalCodeXPath)
PostalCode = postalCodeResult.InnerText
MsgBox(PostalCode)
Next
Yes it is possible, I've done this using the selenium framework, which is aimed for testing automation. However, it provides you with the tools to do exactly that.
Download for .net here:
http://docs.seleniumhq.org/download/
I have been making some progress on this but still have some issues to resolve.
Hopefully, this one won't be that hard.
I have this:
For Each item In Request.QueryString("doc").Split(","c)
sb.Append("http://default.html?k=")
sb.Append(item)
sb.Append("&p=2&o=m</p>")
Next
When I test this code:
Response.Write(sb.ToString())
I get:
http://default.html?k=122&p=2&o=m
http://default.html?k=123&p=2&o=m
That's exactly what we are looking for
When we assign it to a variable like:
Dim linkList As String = sb.ToString()
However, when I loop through linkList
and write it to the screen, it is spitting out only the letter h.
Any ideas what I am doing wrong and how to fix it if possible?
Dim link As String
For Each link I linkList
'let me know if I am still getting the links
response.write link
'we will save all the links later
Next
As always, thanks a lot for your help
You can not loop through a string and get another string (note: linkList is a String - Dim linkList As String = sb.ToString()). That's why you get the h it's trying to pick the Chars in the string. Place the strings in an array then loop through the array. Try this:
Dim linkArray() As String
For Each item In Request.QueryString("doc").Split(","c)
Dim stb As New StringBuilder
stb.Append("http://default.html?k=")
stb.Append(item)
stb.Append("&p=2&o=m</p>")
linkArray.add(stb.toString())
Next
For Each link As String In linkArray
response.write link
Next
It looks like you are not closing your <a> tag
&p=2&o=m</p>
Notice the </p> to close a paragraph without the closing <a> tag with </a>.
Try to view source and you'll probably see the HTML is malformed in that way.