VB.NET | HttpWebRequest, How to see if the host is online? - vb.net

I am coding a whitelist for my application in VB.NET.
I am using the HttpWebRequest Method and HttpWebResponse.
If the Whitelist Host is down, the whitelist is bypassed and the program is available to anyone which is a vulnerability.
Public Function GetWhitelist(ByVal PageURL As String) As String
Dim S As String = ""
Try
Dim Request As HttpWebRequest = WebRequest.Create("WHITELIST URL HERE")
Dim Response As HttpWebResponse = Request.GetResponse()
Using Reader As StreamReader = New StreamReader(Response.GetResponseStream())
S = Reader.ReadToEnd
End Using
Catch ex As Exception
Debug.WriteLine("Start Program Error. Handle:0")
End Try
Return S
End Function
I want to give the user an error if the website is down, any ideas?
Regards.

What is your error condition? This function alone doesn't prevent anything from happening in the event of an exception. The function still exits with returning a string. So any consuming code would have to look for error conditions and handle them accordingly.
If the request fails, you should meaningfully handle the exception. Two ideas off the top of my head would include:
1) Let the exception bubble up the stack:
Public Function GetWhitelist(ByVal PageURL As String) As String
Dim S As String = ""
Dim Request As HttpWebRequest = WebRequest.Create("WHITELIST URL HERE")
Dim Response As HttpWebResponse = Request.GetResponse()
Using Reader As StreamReader = New StreamReader(Response.GetResponseStream())
S = Reader.ReadToEnd
End Using
Return S
End Function
This would make the attempt to contact the host and, if that attempt failed, throw an exception instead of return a string.
2) Throw a custom exception:
Public Function GetWhitelist(ByVal PageURL As String) As String
Dim S As String = ""
Try
Dim Request As HttpWebRequest = WebRequest.Create("WHITELIST URL HERE")
Dim Response As HttpWebResponse = Request.GetResponse()
Using Reader As StreamReader = New StreamReader(Response.GetResponseStream())
S = Reader.ReadToEnd
End Using
Catch ex As Exception
Debug.WriteLine("Start Program Error. Handle:0")
Throw New SomeCustomException(String.Format("Unable to contact host: {0}", PageURL), ex)
End Try
Return S
End Function
This would provide a more targeted exception instead of whatever comes out of the response reader, would provide useful runtime information about the error for logging and analysis (namely the runtime value of the PageURL), and takes a step toward hiding the implementation details from code outside of this object (since that code doesn't really care about the HttpWebRequest and HttpWebResponse, it just wants to know if the URL is good or not).
Remember that throwing an exception is a perfectly acceptable exit path for a function. It doesn't always have to return a value. An exception is an appropriate way of indicating an error condition. Your current implementation, however, "swallows" the exception and provides no indication to the consuming code that anything went wrong. Instead it returns a "magic value" of String.Empty which consuming code may or may not ignore.

Change your exception handler. The correct way to do this is to just try the request and handle the exception if it fails.

Related

Using Webrequest/response

Can someone explain to me the "workings" of webrequest/response in this situation, i have this function:
Public Shared Function DoRequests(ByVal url As String) As String
Dim sr As StreamReader = Nothing
Try
Dim request As WebRequest = WebRequest.Create(url)
Dim response As WebResponse = request.GetResponse()
sr = New StreamReader(response.GetResponseStream())
Return sr.ReadToEnd()
Catch
Return Nothing
Finally
If sr IsNot Nothing Then sr.Dispose()
End Try
End Function
Why .net 2.0 creates 5 threads to run the request with this usage?:
DoRequests("HTTPS LINK HERE")
The problem is as said above, .net creates 5 threads expecifically 3 here:
Dim request As WebRequest = WebRequest.Create(url)
and 2 here:
Dim response As WebResponse = request.GetResponse()
The process is fast but the threads doesn't get cleaned.. at least not right away, i'm asking why.. or if there is somewhat a "better" way of doing it, since though i understand why 2 threads are created, i don't understand why 5 are needed for a simple request/response, and i still need to run it on a thread if i want to be sure the UI doesn't freeze...
I know Garbage collector exists but in this case i think this wouldn't be the case for letting him cleanup since it's the requests that creates it's thread

Check if folder on web exists or not

I'm creating desktop aplication and when I write a username in TextBox1 and on Button1.Click event it should check does folder on web exists.
As far I've tried this:
username = Me.TextBox1.Text
password = Me.TextBox2.Text
Dim dir As Boolean = IO.Directory.Exists("http://www.mywebsite.com/" + username)
If dir = true Then
Dim response As String = web.DownloadString("http://www.mywebsite.com/" + username + "/Password.txt")
If response.Contains(password) Then
MsgBox("You've logged in succesfully", MsgBoxStyle.Information)
Exit Sub
Else
MsgBox("Password is incorect!")
End If
Else
MsgBox("Username is wrong, try again!")
End If
First problem is that my boolean is giving FALSE as answer (directory exists for sure and all permissions are granted to see folder). I tried to solve that with setting dir = false and after that I go into first IF (but that's not what I want, since it should be TRUE, not FALSE)
There we come to second problem, in this line: Dim response As String=web.DownloadString("http://www.mywebsite.com/" + username + "/Password.txt") I get this error message: The remote server returned an error: (404) Not Found.
Anyone more experienced with this kind of things who can help me?
IO.Directory.Exists will not work in this case. That method only works to check for a folder on a disk somewhere (locally or network) ; you can't use it to check for the existence of a resource over HTTP. (i.e a URI)
But even if it did work this way, it's actually pointless to call it before attempting to download - the method DownloadString will throw an exception if something goes wrong - as you have seen, in this case it's telling you 404 Not Found which means "This resource does not exist as far as you are concerned". **
So you should try/catch the operation, you need to catch exceptions of type WebException, cast its Response member to HttpWebException, and check the StatusCode property.
An good example (albeit in C#) is here
** I say "as far as you are concerned" because for all you know, the resource may very well exist on the server, but it has decided to hide it from you because you do not have access to it, etc, and the developer of that site decided to return 404 in this case instead of 401 Unauthorised. The point being that from your point of view, the resource is not available.
Update:
here is the code from the answer I linked to, translated via this online tool because my VB is dodgy enough :). This code runs just fine for me in LinqPad, and produces the output "testlozinka"
Sub Main
Try
Dim myString As String
Using wc As New WebClient()
myString = wc.DownloadString("http://dota2world.org/HDS/Klijenti/TestKlijent/password.txt")
Console.WriteLine(myString)
End Using
Catch ex As WebException
Console.WriteLine(ex.ToString())
If ex.Status = WebExceptionStatus.ProtocolError AndAlso ex.Response IsNot Nothing Then
Dim resp = DirectCast(ex.Response, HttpWebResponse)
If resp.StatusCode = HttpStatusCode.NotFound Then
' HTTP 404
'the page was not found, continue with next in the for loop
Console.WriteLine("Page not found")
End If
End If
'throw any other exception - this should not occur
Throw
End Try
End Sub
Hope that helps.

My url checker function is hanging application in vb.net

Here is vb.net 2008 code is:
Public Function CheckURL(ByVal URL As String) As Boolean
Try
Dim Response As Net.WebResponse = Nothing
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Response = WebReq.GetResponse
Response.Close()
Return True
Catch ex As Exception
End Try
End Function
when a url is processing in checking it hangs my application for a while. Is this possible it checks smoothly all url list without hanging my application..
Is there any other fastest way to check urls?
Note: I have about 800 urls in file to check all links a valid by website responce or not.
If an exception occurs, the WebResponse object isn't properly disposed of. This can lead to your app running out of connections. Something like this will work better:
Try
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Using Response = WebReq.GetResponse()
Return True
End Using
Catch ex as WebException
Return False
End Try
This using the Using keyword ensures that the response is closed and finalized whenever that block exits.
If it's the server itself that's taking awhile to respond, look into the BeginGetResponse method on the HttpWebRequest. Check MSDN for a sample on how to use it. But be warned, that way also lies madness if you are not careful.
The answer is two fold:
Most of the waiting time is due to downloading content you don't need. If you request to only return the header, you will receive substantially less data, which will make your process faster.
As Matt identified, you aren't disposing of your connections, which may slow your process.
Expanding on Matt's answer, do the following:
Try
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
WebReq.Method = "HEAD" 'This is the important line.
Using Response = WebReq.GetResponse()
Return True
End Using
Catch ex as WebException
Return False
End Try
GetResponse delivers you the whole content to your request. If this is what you want, there's not many room to speed up the request on the client side, since it mostly depends on the URLs server how fast to reply and how much data will be send over. If you just want to check if the URL is valid (or responding at all), it might be better to just ping it.
Keep in mind GetResponse isn't disposed when it runs into an error, so use the code posted by Matt to avoid this.
For your other problem, hanging application, you might avoid this be running this code as a thread.
This works basically like this (from here):
rem at the top of the code
Imports System.Threading
...
rem your event handler, p.e. button click or whatever
trd = New Thread(AddressOf ThreadTask)
trd.IsBackground = True
trd.Start()
rem your code
Private Sub ThreadTask()
dim i as long
Do
i += 1
Thread.Sleep(100)
Loop
End Sub

VB.net httpwebrequest For loop hangs every 10 or so iterations

I am trying to loop through an array and perform an httpwebrequest in each iteration.
The code seems to work, however it pauses for a while (eg alot longer than the set timeout. Tried setting that to 100 to check and it still pauses) after every 10 or so iterations, then carries on working.
Here is what i have so far:
For i As Integer = 0 To numberOfProxies - 1
Try
'create request to a proxyJudge php page using proxy
Dim request As HttpWebRequest = HttpWebRequest.Create("http://www.pr0.net/deny/azenv.php")
request.Proxy = New Net.WebProxy(proxies(i)) 'select the current proxie from the proxies array
request.Timeout = 10000 'set timeout
Dim response As HttpWebResponse = request.GetResponse()
Dim sr As StreamReader = New StreamReader(response.GetResponseStream())
Dim pageSourceCode As String = sr.ReadToEnd()
'check the downloaded source for certain phrases, each identifies a type of proxy
'HTTP_X_FORWARDED_FOR identifies a transparent proxy
If pageSourceCode.Contains("HTTP_X_FORWARDED_FOR") Then
'delegate method for cross thread safe
UpdateListbox(ListBox3, proxies(i))
ElseIf pageSourceCode.Contains("HTTP_VIA") Then
UpdateListbox(ListBox2, proxies(i))
Else
UpdateListbox(ListBox1, proxies(i))
End If
Catch ex As Exception
'MessageBox.Show(ex.ToString) used in testing
UpdateListbox(ListBox4, proxies(i))
End Try
completedProxyCheck += 1
lblTotalProxiesChecked.CustomInvoke(Sub(l) l.Text = completedProxyCheck)
Next
I have searched all over this site and via google, and most responses to this type of question say the response must be closed. I have tried a using block, eg:
Using response As HttpWebResponse = request.GetResponse()
Using sr As StreamReader = New StreamReader(response.GetResponseStream())
Dim pageSourceCode As String = sr.ReadToEnd()
'check the downloaded source for certain phrases, each identifies a type of proxy
'HTTP_X_FORWARDED_FOR identifies a transparent proxy
If pageSourceCode.Contains("HTTP_X_FORWARDED_FOR") Then
'delegate method for cross thread safe
UpdateListbox(ListBox3, proxies(i))
ElseIf pageSourceCode.Contains("HTTP_VIA") Then
UpdateListbox(ListBox2, proxies(i))
Else
UpdateListbox(ListBox1, proxies(i))
End If
End Using
End Using
And it makes no difference (though i may have implemented it wrong) As you can tell im very new to VB or any OOP so its probably a simple problem but i cant work it out.
Any suggestions or just tips on how to diagnose these types of problems would be really appreciated.
EDIT:
Now im thoroughly confused. Does the try catch statement automatically close the response, or do i need to put something in Finally? If so, what? i cant use response.close() because its declared in the try block.
Perhaps im just using really badly structured code and there is a much better way to do this? Or something else is causing the pause/hang?
Yeah, you need to close the response after you are done with it, as .net enforces a maximum number of concurrent requests
so just add
response.close()
at the end of your code block
Because, it's a very difficult to write code in comment I will continue as answer.
For i As Integer = 0 To numberOfProxies - 1
Dim response As HttpWebResponse
Try
'create request to a proxyJudge php page using proxy
Dim request As HttpWebRequest = HttpWebRequest.Create("http://www.pr0.net/deny/azenv.php")
request.Proxy = New Net.WebProxy(proxies(i)) 'select the current proxie from the proxies array
request.Timeout = 10000 'set timeout
response = request.GetResponse()
Dim sr As StreamReader = New StreamReader(response.GetResponseStream())
Dim pageSourceCode As String = sr.ReadToEnd()
'check the downloaded source for certain phrases, each identifies a type of proxy
'HTTP_X_FORWARDED_FOR identifies a transparent proxy
If pageSourceCode.Contains("HTTP_X_FORWARDED_FOR") Then
'delegate method for cross thread safe
UpdateListbox(ListBox3, proxies(i))
ElseIf pageSourceCode.Contains("HTTP_VIA") Then
UpdateListbox(ListBox2, proxies(i))
Else
UpdateListbox(ListBox1, proxies(i))
End If
Catch ex As Exception
'MessageBox.Show(ex.ToString) used in testing
UpdateListbox(ListBox4, proxies(i))
Finally
response.Close()
End Try
completedProxyCheck += 1
lblTotalProxiesChecked.CustomInvoke(Sub(l) l.Text = completedProxyCheck)
Next

How can I use VB.Net to read the content returned from a URL?

Below is the example code I'm using to get this to work and it does work if I try to read yahoo.com.
Here is the problem. The address I need to read is a java servlet that processes parameters passed in, generates a text document on the server, and then redirects to another URL and returns the address of the text file on the server. I then need to download that text file and process it. I'm having problems connecting to the first URL with the parameters and I think it has to do with the redirect.
I'm using the WebRequest object and I've tried using the HttpWebRequest object. Are there any other objects that support redirects?
TIA
Dim reader As StreamReader
Dim request As WebRequest
Dim response As WebResponse
Dim data As String = ""
Try
request = WebRequest.Create("URL Here")
request.Timeout = 30000
response = request.GetResponse()
reader = New StreamReader(response.GetResponseStream())
data = reader.ReadToEnd()
Catch ex As Exception
MsgBox(ex.Message)
End Try
Return data
Edit
I just tested out HttpWebRequest.Create() and that does handle the 301 and 302 fine with out extra code.
Can you post the error you are seeing
You could cast the WebResponse to a HttpWebResponse:
I need to convert this to VB... but it might help you start:
var response = request.GetResponse() as HttpWebResponse;
if (response.StatusCode == HttpStatusCode.Moved || response.StatusCode == HttpStatusCode.Redirect)
{
// Follow Redirect, new request based off Redirect
}
// Read Data
I believe you just have to set the AutoRedirect property.
request.AutoRedirect = true;
webRequest = webRequest.Create(URL)
webresponse = webRequest.GetResponse()
inStream = New StreamReader(webresponse.GetResponseStream())
Read URL full source code
winston
I think I found something that will work.
I used a WebBrowser control instead.
Have a button that runs this code...
WebBrowser1.Navigate("URL Here")
And this function to process once the request returns.
Private Sub WebBrowser1_Navigated(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserNavigatedEventArgs) Handles WebBrowser1.Navigated
MsgBox(WebBrowser1.DocumentText)
End Sub