Trying to download file in code.
Current code:
Dim uri As New UriBuilder
uri.UserName = "xxx"
uri.Password = "xxx"
uri.Host = "xxx"
uri.Path = "xxx.aspx?q=65"
Dim request As HttpWebRequest = DirectCast(WebRequest.Create(uri.Uri), HttpWebRequest)
request.AllowAutoRedirect = True
request = DirectCast(WebRequest.Create(DownloadUrlIn), HttpWebRequest)
request.Timeout = 10000
'request.AllowWriteStreamBuffering = True
Dim response As HttpWebResponse = Nothing
response = DirectCast(request.GetResponse(), HttpWebResponse)
Dim s As Stream = response.GetResponseStream()
'Write to disk
Dim fs As New FileStream("c:\xxx.pdf", FileMode.Create)
Dim read As Byte() = New Byte(255) {}
Dim count As Integer = s.Read(read, 0, read.Length)
While count > 0
fs.Write(read, 0, count)
count = s.Read(read, 0, read.Length)
End While
'Close everything
fs.Close()
s.Close()
response.Close()
Running this code and checking the response.ResponseUri indicates im being redirected back to the login page and not to the pdf file.
For some reason its not authorising access what could I be missing as Im sending the user name and password in the uri? Thanks for your help
You don't need all of that code to download a file from the net
just use the WebClient class and its DownloadFile method
you should check and see if the site requires cookies (most do), i'd use a packet analyzer and run your code and see exactly what the server is returning. use fiddler or http analyzer to log packets
With UWP, this has become a more pertinent question as UWP does not have a WebClient. The correct answer to this question is if you are being re-directed to the login page, then there must be an issue with your credentials OR the setting (or lack of) header for the HttpWebRequest.
According to Microsoft, the request for downloading is sent with the call to GetResponse() on the HttpWebRequest, therefore the downloaded file SHOULD be in the stream in the response (returned by the GetResponse() call mentioned above).
Related
The below code works on some URLs, but some other URLs having parameters return the error: The remote server returned an error: (405) Method Not Allowed.
my work:
Dim objHttpWebRequest As HttpWebRequest = Nothing
Dim objHttpWebResponse As HttpWebResponse = Nothing
Dim objRequestStream As Stream = Nothing
Dim objResponseStream As Stream = Nothing
Dim objXMLReader As XmlTextReader
Try
objHttpWebRequest = WebRequest.Create(URL)
'Start HttpRequest
objHttpWebRequest.Method = "POST"
objHttpWebRequest.ContentType = "application/xml"
'Get Stream Object
objRequestStream = objHttpWebRequest.GetRequestStream()
objRequestStream.Close()
'Start HTTP Response
objHttpWebResponse = objHttpWebRequest.GetResponse()
If objHttpWebResponse.StatusCode = HttpStatusCode.OK Then
objResponseStream = objHttpWebResponse.GetResponseStream()
objXMLReader = New XmlTextReader(objResponseStream)
Dim xmldoc As XmlDocument = New XmlDocument
xmldoc.Load(objXMLReader)
XMLResponse = xmldoc
objXMLReader.Close()
End If
Is the problem in the method I am using? or the content type?
Based on the status code, the problem is in the method. Not all of the URLs might be able to respond to POST requests.
Wikipedia states
405 Method Not Allowed
A request was made of a resource using a request method not supported
by that resource; for example, using GET on a form which requires data
to be presented via POST, or using PUT on a read-only resource.
I'm trying to upload a file from an FTP site to Basecamp using the Basecamp API. I'm using a simple console application. Here's my code:
Try
Dim accountID As String = ConfigurationManager.AppSettings("BaseCampID")
Dim projectID As Integer = 9999999
Dim folderName As String = "XXXXX/XXXXX"
Dim fileName As String = "XXX.zip"
'The URL to access the attachment method of the API
Dim apiURL = String.Format("https://basecamp.com/{0}/api/v1/projects/{1}/attachments.json", accountID, projectID)
'Get the file from the FTP server as a byte array
Dim fileBytes As Byte() = GetFileBytes(String.Format("{0}\\{1}", folderName, fileName))
'Initialize the WebClient object
Dim client As New WebClient()
client.Headers.Add("Content-Type", "application/zip")
'Need to provide a user-agent with a URL or email address
client.Headers.Add("User-Agent", "Basecamp Upload (email#email.com)")
'Keep the connection alive so it doesn't close
client.Headers.Add("Keep-Alive", "true")
'Provide the Basecamp credentials
client.Credentials = New NetworkCredential("username", "password")
'Upload the file as a byte array to the API, and get the response
Dim responseStr As Byte() = client.UploadData(apiURL, "POST", fileBytes)
'Convert the JSON response to a BaseCampAttachment object
Dim attachment As BaseCampAttachment
attachment = JSonHelper.FromJSon(Of BaseCampAttachment)(Encoding.Default.GetString(responseStr))
Catch ex As Exception
Console.WriteLine(ex.Message)
Finally
Console.ReadLine()
End Try
But whenever it calls client.UploadData, I get the error message "The underlying connection was closed: The connection was closed unexpectedly." I ran into this issue earlier and thought I solved it by adding the "Keep-Alive" header, but it's not working anymore. The API works if I upload a local file with client.UploadFile, but I'd like to just upload the file from they byte array from the FTP rather than downloading the file locally then uploading it to Basecamp.
Any thoughts would be greatly appreciated. Thanks!
I never figured out what was wrong with the WebClient call, but I ended up using a Basecamp API wrapper from https://basecampwrapper.codeplex.com. That wrapper uses HTTPRequest and HTTPResponse instead of WebClient.UploadData. It's also much easier to just use that wrapper than to try writing my own code from scratch.
I am making a Windows Phone app and I am trying to get JSON data from a URL. The user needs to be logged into the website (which hosts the JSON data) in order to get JSON data and I cannot use the Web Browser control to display the data and then extract the string since the browser doesn't recognize it (for some weird reason) and asks to search for an app on Store which can handle that JSON file type. (If I open the URL in desktop browser on my Windows PC, I can see the raw JSON data). I can't use normal HTTPWebRequest or WebClient as to get JSON data the login cookies needs to be set (the JSON data is user-specific) and I can't extract the cookies from Web browser control and use it with WebClient or HTTPWebRequest. So the best thing I can do is use a special internal instance of IWebRequestCreate that is used internally by the WebBrowser. By opening background HTTP requests with that class, the cookies get automatically set as if they were created/sent by the WebBrowser control. But my code is not returning the JSON data, I get blank response, as in the string resp is empty.
Below is the code:
Dim browser = New WebBrowser()
Dim brwhttp = GetType(WebRequestCreator).GetProperty("BrowserHttp")
Dim requestFactory = TryCast(brwhttp.GetValue(Browser, Nothing), IWebRequestCreate)
Dim uri = New Uri("http://api.quora.com/api/logged_in_user?fields=inbox,notifs,following,followers")
Dim req = requestFactory.Create(uri)
req.Method = "GET"
req.BeginGetResponse(New AsyncCallback(AddressOf request_Callback), req)
Private Sub request_Callback(asyncResult As IAsyncResult)
Dim webRequest As HttpWebRequest = DirectCast(asyncResult.AsyncState, HttpWebRequest)
Dim webResponse As HttpWebResponse = DirectCast(webRequest.EndGetResponse(asyncResult), HttpWebResponse)
Dim tempStream As New MemoryStream()
webResponse.GetResponseStream().CopyTo(tempStream)
Dim sr As New StreamReader(tempStream)
Dim resp As String = sr.ReadToEnd
End Sub
What's wrong?
I found that CopyTo can leave the Stream's pointer at the end of the buffer, you probably need to reset tempStream's pointer to the beginning before attempting to read it with the StreamReader, here's the code...
webResponse.GetResponseStream().CopyTo(tempStream);
tempStream.Seek(0, SeekOrigin.Begin);
Dim sr As New StreamReader(tempStream);
I created http request application to test my web site qulatiy (see below).
Dim Request As HttpWebRequest = WebRequest.Create(webAddress)
Dim Response As HttpWebResponse = Request.GetResponse()
Request.Method = "Get"
Dim Reader As New StreamReader(Response.GetResponseStream)
Dim Html As String = Reader.ReadToEnd()
In this case, I would like to create anonymous request without catching the response. How can I do that?
To do so, u have to get a little low level , working with sockets
TcpCient in this case
Sample code
Imports System.Net.Sockets
Module Module1
Sub Main()
Dim tcpcli = New TcpClient()
tcpcli.Connect("google.co.in", 80)
Dim stream As NetworkStream = tcpcli.GetStream()
Dim reqdata As String = String.Format("GET / HTTP/1.1{0}Host: www.google.co.in{0}Connection: Close{0}{0}", vbCrLf)
Dim reqbytes() As Byte = Text.Encoding.ASCII.GetBytes(reqdata)
stream.Write(reqbytes, 0, reqbytes.Length)
stream.Close()
stream.Dispose()
tcpcli.Close()
End Sub
End Module
Network capture via wireshark (no response received)
You can make a web request anonymously by using ProxySharp. It basically makes the web request behind a random vpn each time. This makes it look like the request is coming from a different IP address on each request.
Below is the example code I'm using to get this to work and it does work if I try to read yahoo.com.
Here is the problem. The address I need to read is a java servlet that processes parameters passed in, generates a text document on the server, and then redirects to another URL and returns the address of the text file on the server. I then need to download that text file and process it. I'm having problems connecting to the first URL with the parameters and I think it has to do with the redirect.
I'm using the WebRequest object and I've tried using the HttpWebRequest object. Are there any other objects that support redirects?
TIA
Dim reader As StreamReader
Dim request As WebRequest
Dim response As WebResponse
Dim data As String = ""
Try
request = WebRequest.Create("URL Here")
request.Timeout = 30000
response = request.GetResponse()
reader = New StreamReader(response.GetResponseStream())
data = reader.ReadToEnd()
Catch ex As Exception
MsgBox(ex.Message)
End Try
Return data
Edit
I just tested out HttpWebRequest.Create() and that does handle the 301 and 302 fine with out extra code.
Can you post the error you are seeing
You could cast the WebResponse to a HttpWebResponse:
I need to convert this to VB... but it might help you start:
var response = request.GetResponse() as HttpWebResponse;
if (response.StatusCode == HttpStatusCode.Moved || response.StatusCode == HttpStatusCode.Redirect)
{
// Follow Redirect, new request based off Redirect
}
// Read Data
I believe you just have to set the AutoRedirect property.
request.AutoRedirect = true;
webRequest = webRequest.Create(URL)
webresponse = webRequest.GetResponse()
inStream = New StreamReader(webresponse.GetResponseStream())
Read URL full source code
winston
I think I found something that will work.
I used a WebBrowser control instead.
Have a button that runs this code...
WebBrowser1.Navigate("URL Here")
And this function to process once the request returns.
Private Sub WebBrowser1_Navigated(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserNavigatedEventArgs) Handles WebBrowser1.Navigated
MsgBox(WebBrowser1.DocumentText)
End Sub