Can someone explain to me the "workings" of webrequest/response in this situation, i have this function:
Public Shared Function DoRequests(ByVal url As String) As String
Dim sr As StreamReader = Nothing
Try
Dim request As WebRequest = WebRequest.Create(url)
Dim response As WebResponse = request.GetResponse()
sr = New StreamReader(response.GetResponseStream())
Return sr.ReadToEnd()
Catch
Return Nothing
Finally
If sr IsNot Nothing Then sr.Dispose()
End Try
End Function
Why .net 2.0 creates 5 threads to run the request with this usage?:
DoRequests("HTTPS LINK HERE")
The problem is as said above, .net creates 5 threads expecifically 3 here:
Dim request As WebRequest = WebRequest.Create(url)
and 2 here:
Dim response As WebResponse = request.GetResponse()
The process is fast but the threads doesn't get cleaned.. at least not right away, i'm asking why.. or if there is somewhat a "better" way of doing it, since though i understand why 2 threads are created, i don't understand why 5 are needed for a simple request/response, and i still need to run it on a thread if i want to be sure the UI doesn't freeze...
I know Garbage collector exists but in this case i think this wouldn't be the case for letting him cleanup since it's the requests that creates it's thread
Related
I have built a Windows Phone 8 app which compiles and runs without any problems. However, when I run the built-in VS Store Test Kit, it brings back LOTS of errors under XAP package requirements, formatted as such:
[ERROR]: Unsupported API cannot be used by a background agent.
Assembly Microsoft.Phone.Controls.Toolkit.dll was trying to use
Microsoft.Phone.Shell.ApplicationBarIconButton.
I'm getting errors for my project dll ([projectname].dll) and the Toolkit dll. It seems to be bringing back everything, even references I'm not using (such as add_OrientationChanged).
Now I've had a look through the Unsupported APIs listed by Microsoft, and I'm not using any of them. At the bottom of the page is a list of "noteworthy APIs", the one of which I;m using is HTTPWebRequest to connect to a web service. I'm assuming this is where the problem lies, as I use this throughout the app.
Here's an example of what I'm doing with HTTPWebRequest:
Public Sub LoadRequest()
Dim request As HttpWebRequest = CType(WebRequest.Create(_ServiceURL), HttpWebRequest)
request.ContentType = "text/xml;charset=utf-8"
request.Method = "POST"
request.Headers("SOAPAction") = "<web service address>"
Start the asynchronous operation.
Dim result As IAsyncResult = _
CType(request.BeginGetRequestStream(AddressOf GetRequestsStreamCallback, request), _
IAsyncResult)
End Sub
Public Sub GetRequestsStreamCallback(ByVal asyncResult As IAsyncResult)
Dim request As HttpWebRequest = CType(asyncResult.AsyncState, HttpWebRequest)
Dim soapBody As System.Xml.Linq.XDocument = System.Xml.Linq.XDocument.Parse( _
"<soap:Envelope xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns:xsd='http://www.w3.org/2001/XMLSchema' " & _
"xmlns:soap='http://schemas.xmlsoap.org/soap/envelope/'><soap:Header><[SOAP Header Here]></soap:Header>" & _
"<soap:Body><[SOAP Body Here]></soap:Body></soap:Envelope>")
'End the operation
Dim postStream As Stream = request.EndGetRequestStream(asyncResult)
'Convert the string into byte array.
Dim byteArray As Byte() = Encoding.UTF8.GetBytes(soapBody.ToString())
'Write to the stream.
postStream.Write(byteArray, 0, soapBody.ToString().Length)
postStream.Close()
Dispatcher.BeginInvoke(Sub()
Dim result As IAsyncResult = _
CType(request.BeginGetResponse(AddressOf GetRequestsResponseCallback, request), IAsyncResult)
End Sub)
End Sub
Public Sub GetRequestsResponseCallback(ByVal Result As IAsyncResult)
'Get the response
Dim request As HttpWebRequest = CType(Result.AsyncState, HttpWebRequest)
Dim response As HttpWebResponse = CType(request.EndGetResponse(Result), HttpWebResponse)
Dim streamResponse As Stream = response.GetResponseStream()
Dim streamRead As New StreamReader(streamResponse)
Dim responseString As String = streamRead.ReadToEnd()
streamResponse.Close()
streamRead.Close()
'Take action on the response
Dim Xml As XDocument = XDocument.Parse(responseString)
Dim ns As XNamespace = "<Service Address>"
etc...
End Sub
I then proceed to pull apart the XML response for the data I need for the app. Any ideas as to what the test kit isn't happy about here? I'm assuming that as it doesn't pass validation at this stage, it's likely to be rejected from the store.
Thanks in advance for any ideas.
The problem was an existing background agent I was unaware of (I took over the project from another developer). There was a reference in the WMAppManifest.xml file, which was apparently part of an early test using the Wallet, which is no longer required. Removing this reference fixed the problem.
<ExtendedTask Name="BackgroundTask">
<BackgroundServiceAgent Specifier="WalletAgent" Name="WalletAgent" Source="<ProjectName>" Type="<ProjectName>.WPWalletAgent" />
</ExtendedTask>
I am coding a whitelist for my application in VB.NET.
I am using the HttpWebRequest Method and HttpWebResponse.
If the Whitelist Host is down, the whitelist is bypassed and the program is available to anyone which is a vulnerability.
Public Function GetWhitelist(ByVal PageURL As String) As String
Dim S As String = ""
Try
Dim Request As HttpWebRequest = WebRequest.Create("WHITELIST URL HERE")
Dim Response As HttpWebResponse = Request.GetResponse()
Using Reader As StreamReader = New StreamReader(Response.GetResponseStream())
S = Reader.ReadToEnd
End Using
Catch ex As Exception
Debug.WriteLine("Start Program Error. Handle:0")
End Try
Return S
End Function
I want to give the user an error if the website is down, any ideas?
Regards.
What is your error condition? This function alone doesn't prevent anything from happening in the event of an exception. The function still exits with returning a string. So any consuming code would have to look for error conditions and handle them accordingly.
If the request fails, you should meaningfully handle the exception. Two ideas off the top of my head would include:
1) Let the exception bubble up the stack:
Public Function GetWhitelist(ByVal PageURL As String) As String
Dim S As String = ""
Dim Request As HttpWebRequest = WebRequest.Create("WHITELIST URL HERE")
Dim Response As HttpWebResponse = Request.GetResponse()
Using Reader As StreamReader = New StreamReader(Response.GetResponseStream())
S = Reader.ReadToEnd
End Using
Return S
End Function
This would make the attempt to contact the host and, if that attempt failed, throw an exception instead of return a string.
2) Throw a custom exception:
Public Function GetWhitelist(ByVal PageURL As String) As String
Dim S As String = ""
Try
Dim Request As HttpWebRequest = WebRequest.Create("WHITELIST URL HERE")
Dim Response As HttpWebResponse = Request.GetResponse()
Using Reader As StreamReader = New StreamReader(Response.GetResponseStream())
S = Reader.ReadToEnd
End Using
Catch ex As Exception
Debug.WriteLine("Start Program Error. Handle:0")
Throw New SomeCustomException(String.Format("Unable to contact host: {0}", PageURL), ex)
End Try
Return S
End Function
This would provide a more targeted exception instead of whatever comes out of the response reader, would provide useful runtime information about the error for logging and analysis (namely the runtime value of the PageURL), and takes a step toward hiding the implementation details from code outside of this object (since that code doesn't really care about the HttpWebRequest and HttpWebResponse, it just wants to know if the URL is good or not).
Remember that throwing an exception is a perfectly acceptable exit path for a function. It doesn't always have to return a value. An exception is an appropriate way of indicating an error condition. Your current implementation, however, "swallows" the exception and provides no indication to the consuming code that anything went wrong. Instead it returns a "magic value" of String.Empty which consuming code may or may not ignore.
Change your exception handler. The correct way to do this is to just try the request and handle the exception if it fails.
Trying to download file in code.
Current code:
Dim uri As New UriBuilder
uri.UserName = "xxx"
uri.Password = "xxx"
uri.Host = "xxx"
uri.Path = "xxx.aspx?q=65"
Dim request As HttpWebRequest = DirectCast(WebRequest.Create(uri.Uri), HttpWebRequest)
request.AllowAutoRedirect = True
request = DirectCast(WebRequest.Create(DownloadUrlIn), HttpWebRequest)
request.Timeout = 10000
'request.AllowWriteStreamBuffering = True
Dim response As HttpWebResponse = Nothing
response = DirectCast(request.GetResponse(), HttpWebResponse)
Dim s As Stream = response.GetResponseStream()
'Write to disk
Dim fs As New FileStream("c:\xxx.pdf", FileMode.Create)
Dim read As Byte() = New Byte(255) {}
Dim count As Integer = s.Read(read, 0, read.Length)
While count > 0
fs.Write(read, 0, count)
count = s.Read(read, 0, read.Length)
End While
'Close everything
fs.Close()
s.Close()
response.Close()
Running this code and checking the response.ResponseUri indicates im being redirected back to the login page and not to the pdf file.
For some reason its not authorising access what could I be missing as Im sending the user name and password in the uri? Thanks for your help
You don't need all of that code to download a file from the net
just use the WebClient class and its DownloadFile method
you should check and see if the site requires cookies (most do), i'd use a packet analyzer and run your code and see exactly what the server is returning. use fiddler or http analyzer to log packets
With UWP, this has become a more pertinent question as UWP does not have a WebClient. The correct answer to this question is if you are being re-directed to the login page, then there must be an issue with your credentials OR the setting (or lack of) header for the HttpWebRequest.
According to Microsoft, the request for downloading is sent with the call to GetResponse() on the HttpWebRequest, therefore the downloaded file SHOULD be in the stream in the response (returned by the GetResponse() call mentioned above).
I am trying to loop through an array and perform an httpwebrequest in each iteration.
The code seems to work, however it pauses for a while (eg alot longer than the set timeout. Tried setting that to 100 to check and it still pauses) after every 10 or so iterations, then carries on working.
Here is what i have so far:
For i As Integer = 0 To numberOfProxies - 1
Try
'create request to a proxyJudge php page using proxy
Dim request As HttpWebRequest = HttpWebRequest.Create("http://www.pr0.net/deny/azenv.php")
request.Proxy = New Net.WebProxy(proxies(i)) 'select the current proxie from the proxies array
request.Timeout = 10000 'set timeout
Dim response As HttpWebResponse = request.GetResponse()
Dim sr As StreamReader = New StreamReader(response.GetResponseStream())
Dim pageSourceCode As String = sr.ReadToEnd()
'check the downloaded source for certain phrases, each identifies a type of proxy
'HTTP_X_FORWARDED_FOR identifies a transparent proxy
If pageSourceCode.Contains("HTTP_X_FORWARDED_FOR") Then
'delegate method for cross thread safe
UpdateListbox(ListBox3, proxies(i))
ElseIf pageSourceCode.Contains("HTTP_VIA") Then
UpdateListbox(ListBox2, proxies(i))
Else
UpdateListbox(ListBox1, proxies(i))
End If
Catch ex As Exception
'MessageBox.Show(ex.ToString) used in testing
UpdateListbox(ListBox4, proxies(i))
End Try
completedProxyCheck += 1
lblTotalProxiesChecked.CustomInvoke(Sub(l) l.Text = completedProxyCheck)
Next
I have searched all over this site and via google, and most responses to this type of question say the response must be closed. I have tried a using block, eg:
Using response As HttpWebResponse = request.GetResponse()
Using sr As StreamReader = New StreamReader(response.GetResponseStream())
Dim pageSourceCode As String = sr.ReadToEnd()
'check the downloaded source for certain phrases, each identifies a type of proxy
'HTTP_X_FORWARDED_FOR identifies a transparent proxy
If pageSourceCode.Contains("HTTP_X_FORWARDED_FOR") Then
'delegate method for cross thread safe
UpdateListbox(ListBox3, proxies(i))
ElseIf pageSourceCode.Contains("HTTP_VIA") Then
UpdateListbox(ListBox2, proxies(i))
Else
UpdateListbox(ListBox1, proxies(i))
End If
End Using
End Using
And it makes no difference (though i may have implemented it wrong) As you can tell im very new to VB or any OOP so its probably a simple problem but i cant work it out.
Any suggestions or just tips on how to diagnose these types of problems would be really appreciated.
EDIT:
Now im thoroughly confused. Does the try catch statement automatically close the response, or do i need to put something in Finally? If so, what? i cant use response.close() because its declared in the try block.
Perhaps im just using really badly structured code and there is a much better way to do this? Or something else is causing the pause/hang?
Yeah, you need to close the response after you are done with it, as .net enforces a maximum number of concurrent requests
so just add
response.close()
at the end of your code block
Because, it's a very difficult to write code in comment I will continue as answer.
For i As Integer = 0 To numberOfProxies - 1
Dim response As HttpWebResponse
Try
'create request to a proxyJudge php page using proxy
Dim request As HttpWebRequest = HttpWebRequest.Create("http://www.pr0.net/deny/azenv.php")
request.Proxy = New Net.WebProxy(proxies(i)) 'select the current proxie from the proxies array
request.Timeout = 10000 'set timeout
response = request.GetResponse()
Dim sr As StreamReader = New StreamReader(response.GetResponseStream())
Dim pageSourceCode As String = sr.ReadToEnd()
'check the downloaded source for certain phrases, each identifies a type of proxy
'HTTP_X_FORWARDED_FOR identifies a transparent proxy
If pageSourceCode.Contains("HTTP_X_FORWARDED_FOR") Then
'delegate method for cross thread safe
UpdateListbox(ListBox3, proxies(i))
ElseIf pageSourceCode.Contains("HTTP_VIA") Then
UpdateListbox(ListBox2, proxies(i))
Else
UpdateListbox(ListBox1, proxies(i))
End If
Catch ex As Exception
'MessageBox.Show(ex.ToString) used in testing
UpdateListbox(ListBox4, proxies(i))
Finally
response.Close()
End Try
completedProxyCheck += 1
lblTotalProxiesChecked.CustomInvoke(Sub(l) l.Text = completedProxyCheck)
Next
Below is the example code I'm using to get this to work and it does work if I try to read yahoo.com.
Here is the problem. The address I need to read is a java servlet that processes parameters passed in, generates a text document on the server, and then redirects to another URL and returns the address of the text file on the server. I then need to download that text file and process it. I'm having problems connecting to the first URL with the parameters and I think it has to do with the redirect.
I'm using the WebRequest object and I've tried using the HttpWebRequest object. Are there any other objects that support redirects?
TIA
Dim reader As StreamReader
Dim request As WebRequest
Dim response As WebResponse
Dim data As String = ""
Try
request = WebRequest.Create("URL Here")
request.Timeout = 30000
response = request.GetResponse()
reader = New StreamReader(response.GetResponseStream())
data = reader.ReadToEnd()
Catch ex As Exception
MsgBox(ex.Message)
End Try
Return data
Edit
I just tested out HttpWebRequest.Create() and that does handle the 301 and 302 fine with out extra code.
Can you post the error you are seeing
You could cast the WebResponse to a HttpWebResponse:
I need to convert this to VB... but it might help you start:
var response = request.GetResponse() as HttpWebResponse;
if (response.StatusCode == HttpStatusCode.Moved || response.StatusCode == HttpStatusCode.Redirect)
{
// Follow Redirect, new request based off Redirect
}
// Read Data
I believe you just have to set the AutoRedirect property.
request.AutoRedirect = true;
webRequest = webRequest.Create(URL)
webresponse = webRequest.GetResponse()
inStream = New StreamReader(webresponse.GetResponseStream())
Read URL full source code
winston
I think I found something that will work.
I used a WebBrowser control instead.
Have a button that runs this code...
WebBrowser1.Navigate("URL Here")
And this function to process once the request returns.
Private Sub WebBrowser1_Navigated(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserNavigatedEventArgs) Handles WebBrowser1.Navigated
MsgBox(WebBrowser1.DocumentText)
End Sub