I'm trying to send a request to a website but it keeps telling me my cookies are not enabled. (But i did used a cookiecontainer in my code)
Here is the code :
Private Function does_it_asks_for_cookies() As Boolean
Dim url As String = "https://www.myhabit.com/?hash="
Dim req As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
req.CookieContainer = New CookieContainer
req.ContentType = "application/x-www-form-urlencoded"
Dim res As Net.HttpWebResponse = req.GetResponse()
Dim strm As New IO.StreamReader(res.GetResponseStream())
Dim html As String = strm.ReadToEnd()
strm.Dispose()
res.Dispose()
If html.Contains("To continue shopping at Amazon.com, please enable cookies in") Then
Return True
Else
Return False
End If
End Function
Usage : MsgBox(does_it_asks_for_cookies)
While you have a CookieContainer the code in your example doesn't actually put anything in it. HttpWebRequest doesn't know anything about previous requests and certainly doesn't magically store the contents of any cookies in your container, you will have to do that yourself. Having a CookieContainer isn't enough. Most (especially e-commerce) websites are very specific in what they expect there to be present in the form of a cookie, often reporting that cookies aren't turned on when the cookie doesn't contain the proper values. You will have to inspect the web site in a browser to figure out what cookies are set and what values they contain. Google "inspect cookies" to learn how to view cookies in the browser of your choice.
Related
I'm trying to make a stripe payment work from a VB website. I know, I know, "I should use C#". I can't because the site is already in VB. Nothing I can do about it.
Anyway, I have most of it figured out:
User clicks submit button with valid info
Form submits to Stripe
Stripe sends a token back
A jQuery ajax function posts the data to donate/pay-by-stripe
I have this line of code in my Global.asax.vb
routes.MapRoute("pay-by-stripe", "donate/pay-by-stripe", New With{.controller = "Dynamic", .action = "PayByStripe"})
So my PayByStripe function in the Dynamic Controller looks like this:
Function PayByStripe()
''The Stripe Account API Token
Dim STR_Stripe_API_Token As String = "sk_test_*****"
''The Stripe API URL
Dim STR_Stripe_API_URL As [String] = "https://api.stripe.com/v1/charges"
''The Stripe Card Token
Dim token As String = HttpContext.Request.Form("token")
Dim description As String = HttpContext.Request.Form("description")
Dim amount As Single = HttpContext.Request.Form("amount")
''Creates a Web Client
Dim OBJ_Webclient As New System.Net.WebClient()
''Creates Credentials
Dim OBJ_Credentials As New System.Net.NetworkCredential(STR_Stripe_API_Token, "")
''Sets the Credentials on the Web Client
OBJ_Webclient.Credentials = OBJ_Credentials
''Creates a Transaction with Data that Will be Sent to Stripe
''Dim OBJ_Transaction As New System.Collections.Specialized.NameValueCollection()
Dim OBJ_Transaction As NameValueCollection = New NameValueCollection()
OBJ_Transaction.Add("amount", amount)
OBJ_Transaction.Add("currency", "usd")
OBJ_Transaction.Add("address-country", "US")
OBJ_Transaction.Add("description", "")
OBJ_Transaction.Add("card", token)
''The Stripe Response String
Dim STR_Response As String = Encoding.ASCII.GetString(OBJ_Webclient.UploadValues(STR_Stripe_API_URL, OBJ_Transaction))
'Response.Redirect("/donate/?transaction=success");
Return STR_Response
End Function
I'm getting a 400 bad request error on the STR_Response line:
Dim STR_Response As String = Encoding.ASCII.GetString(OBJ_Webclient.UploadValues(STR_Stripe_API_URL, OBJ_Transaction))
I'm a VB and Stripe noob, and not sure what this means. My main theory now is that it's because I don't have a /donate/pay-by-stripe/ page, but I don't know what I'd even put in there if I did create it.
Any help would be great!
That's a webservice you are calling, right?
A 400 Bad Request with a webservice means your XML request is malformed.
Example, in my request, part of it is a UTC in a certain date format. Example: <pp:utc>2013-05-24 2025</pp:utc>
So, if I were to malform my request to this <pp:utc>2013-05-24 2025</pp:utc2> it would result in:
HTTP/1.1 400 Bad Request
Cache-Control: private
Server: Microsoft-IIS/7.5
X-AspNet-Version: 2.0.5
So, check your request and make sure everything is properly formatted.
EDIT: just noticed I put the "incorrect" utc tags incorrectly.
Please notice the opening tag <pp:utc> is being closed with a </pp:utc2>, which is the reason why you see 400 bad request
I had to put my password in System.Net.NetworkCredentials, and address-country is not a usable field. The only usable fields when submitting a charge are amount, currency, description, and card (which is actually the token). This is the final, working version of my PayByStripe Function in my Dynamic Controller:
Function PayByStripe()
'' The Stripe Account API Token - change this for testing
Dim STR_Stripe_API_Token As String = ""
If (this_is_a_test) Then
' Test Secret Key
STR_Stripe_API_Token = "sk_test_***"
Else
' Prod Secret Key
STR_Stripe_API_Token = "sk_live_***"
End If
''The Stripe API URL
Dim STR_Stripe_API_URL As [String] = "https://api.stripe.com/v1/charges"
''The Stripe Card Token
Dim token As String = HttpContext.Request.Form("token")
Dim description As String = HttpContext.Request.Form("description")
Dim amount As Single = HttpContext.Request.Form("amount")
''Creates a Web Client
Dim OBJ_Webclient As New System.Net.WebClient()
''Creates Credentials
Dim OBJ_Credentials As New System.Net.NetworkCredential(STR_Stripe_API_Token, "YOUR PASSWORD FOR STRIPE")
''Sets the Credentials on the Web Client
OBJ_Webclient.Credentials = OBJ_Credentials
''Creates a Transaction with Data that Will be Sent to Stripe
Dim OBJ_Transaction As New System.Collections.Specialized.NameValueCollection()
OBJ_Transaction.Add("amount", amount)
OBJ_Transaction.Add("currency", "usd")
OBJ_Transaction.Add("description", description)
OBJ_Transaction.Add("card", token)
''The Stripe Response String
Dim STR_Response As String = Encoding.ASCII.GetString(OBJ_Webclient.UploadValues(STR_Stripe_API_URL, OBJ_Transaction))
Return STR_Response
End Function
I've never had to pass in my password when connecting to Stripe's API. Simply pass in your private API Key through an authorization header with no password. It may also help to pass in a version header as well, something Stripe recommends. The following lines of card are in C#, I know your question was in VB, but I'm sure you can easily adaptive this:
webrequest.Headers.Add("Stripe-Version", "2014-12-22");
webrequest.Headers.Add("Authorization", String.Concat("Basic ", (Convert.ToBase64String(Encoding.UTF8.GetBytes(string.Format("{0}:", "sk_test_XXXXXXXXXXXXXXXXXXX"))))));
Also, it may help to know that Stripe sends a 400 Bad Request when an expired or invalid card token is sent.
I am making a Windows Phone app and I am trying to get JSON data from a URL. The user needs to be logged into the website (which hosts the JSON data) in order to get JSON data and I cannot use the Web Browser control to display the data and then extract the string since the browser doesn't recognize it (for some weird reason) and asks to search for an app on Store which can handle that JSON file type. (If I open the URL in desktop browser on my Windows PC, I can see the raw JSON data). I can't use normal HTTPWebRequest or WebClient as to get JSON data the login cookies needs to be set (the JSON data is user-specific) and I can't extract the cookies from Web browser control and use it with WebClient or HTTPWebRequest. So the best thing I can do is use a special internal instance of IWebRequestCreate that is used internally by the WebBrowser. By opening background HTTP requests with that class, the cookies get automatically set as if they were created/sent by the WebBrowser control. But my code is not returning the JSON data, I get blank response, as in the string resp is empty.
Below is the code:
Dim browser = New WebBrowser()
Dim brwhttp = GetType(WebRequestCreator).GetProperty("BrowserHttp")
Dim requestFactory = TryCast(brwhttp.GetValue(Browser, Nothing), IWebRequestCreate)
Dim uri = New Uri("http://api.quora.com/api/logged_in_user?fields=inbox,notifs,following,followers")
Dim req = requestFactory.Create(uri)
req.Method = "GET"
req.BeginGetResponse(New AsyncCallback(AddressOf request_Callback), req)
Private Sub request_Callback(asyncResult As IAsyncResult)
Dim webRequest As HttpWebRequest = DirectCast(asyncResult.AsyncState, HttpWebRequest)
Dim webResponse As HttpWebResponse = DirectCast(webRequest.EndGetResponse(asyncResult), HttpWebResponse)
Dim tempStream As New MemoryStream()
webResponse.GetResponseStream().CopyTo(tempStream)
Dim sr As New StreamReader(tempStream)
Dim resp As String = sr.ReadToEnd
End Sub
What's wrong?
I found that CopyTo can leave the Stream's pointer at the end of the buffer, you probably need to reset tempStream's pointer to the beginning before attempting to read it with the StreamReader, here's the code...
webResponse.GetResponseStream().CopyTo(tempStream);
tempStream.Seek(0, SeekOrigin.Begin);
Dim sr As New StreamReader(tempStream);
I'm new to WebRequest authentication and have been researching how to authenticate w/ a couple websites to pull some excel data from them. Couple things I'm confused about is
a.) how to properly read a log from Fiddler (using this to pick up get/post data from the website authentication)
b.) how do use the data from Fiddler to program the VB.NET WebRequest properly.
I've been able to authenticate w/ websites that use simple authentication HTTPS, but any site that does any redirects/REST/cookie auth I'm lost...
Let me know if I can provide anymore detail.
Dim req As Net.HttpWebRequest = Net.WebRequest.Create(Url)
If Not Login = Nothing AndAlso Not Password = Nothing Then
Dim myCache As New System.Net.CredentialCache()
myCache.Add(New Uri(Url), "Basic", New System.Net.NetworkCredential(Login, Password))
req.Credentials = myCache
End If
Dim sr As New StreamReader(req.GetResponse().GetResponseStream())
Dim ss as string = sr.ReadToEnd
'Save it as excel & close stream
sr.Close()
Trying to download file in code.
Current code:
Dim uri As New UriBuilder
uri.UserName = "xxx"
uri.Password = "xxx"
uri.Host = "xxx"
uri.Path = "xxx.aspx?q=65"
Dim request As HttpWebRequest = DirectCast(WebRequest.Create(uri.Uri), HttpWebRequest)
request.AllowAutoRedirect = True
request = DirectCast(WebRequest.Create(DownloadUrlIn), HttpWebRequest)
request.Timeout = 10000
'request.AllowWriteStreamBuffering = True
Dim response As HttpWebResponse = Nothing
response = DirectCast(request.GetResponse(), HttpWebResponse)
Dim s As Stream = response.GetResponseStream()
'Write to disk
Dim fs As New FileStream("c:\xxx.pdf", FileMode.Create)
Dim read As Byte() = New Byte(255) {}
Dim count As Integer = s.Read(read, 0, read.Length)
While count > 0
fs.Write(read, 0, count)
count = s.Read(read, 0, read.Length)
End While
'Close everything
fs.Close()
s.Close()
response.Close()
Running this code and checking the response.ResponseUri indicates im being redirected back to the login page and not to the pdf file.
For some reason its not authorising access what could I be missing as Im sending the user name and password in the uri? Thanks for your help
You don't need all of that code to download a file from the net
just use the WebClient class and its DownloadFile method
you should check and see if the site requires cookies (most do), i'd use a packet analyzer and run your code and see exactly what the server is returning. use fiddler or http analyzer to log packets
With UWP, this has become a more pertinent question as UWP does not have a WebClient. The correct answer to this question is if you are being re-directed to the login page, then there must be an issue with your credentials OR the setting (or lack of) header for the HttpWebRequest.
According to Microsoft, the request for downloading is sent with the call to GetResponse() on the HttpWebRequest, therefore the downloaded file SHOULD be in the stream in the response (returned by the GetResponse() call mentioned above).
I've hit a wall on POST'ing data to a webform :(
Below is the code I've adapted from many places (here most recently: http://p2p.wrox.com/asp-net-1-0-1-1-professional/34517-webrequest-webresponse-form-submit-problem-help.html )
Dim url As String = "https://student.ashford.edu/student/"
Dim data As String = "username=myusername&password=mypassword"
Dim buffer As Byte() = Encoding.UTF8.GetBytes(data)
Dim result As String = ""
Dim req As HttpWebRequest = DirectCast(WebRequest.Create(url), HttpWebRequest)
req.Method = "POST"
req.ContentType = "application/x-www-form-urlencoded"
req.ContentLength = buffer.Length
req.CookieContainer = New CookieContainer()
' enable cookies
Dim reqst As Stream = req.GetRequestStream()
' add form data to request stream
reqst.Write(buffer, 0, buffer.Length)
reqst.Flush()
reqst.Close()
Console.WriteLine(vbLf & "Posting data to " & url)
Dim res As HttpWebResponse = DirectCast(req.GetResponse(), HttpWebResponse)
' send request,get response
Console.WriteLine(vbLf & "Response stream is: " & vbLf)
Dim resst As Stream = res.GetResponseStream()
' display HTTP response
Dim sr2 As New StreamReader(resst)
result = sr2.ReadToEnd()
Using writer As System.IO.StreamWriter = New StreamWriter("C:\Temp\checkcheck.html")
writer.Write(result)
End Using
When I check the checkcheck.html file, it doesn't show the page I normally see when I've successfully logged into my university, but instead shows the login page again prompting for a user name and password. Since it's prompting for a user/pass in the output response, does that mean nothing is posted? Or is the login button not being hit?
As far as the field names in the source, the name of the username field is "username", and the name of the password field is "password". From what I can tell, the name of the login button itself is "submit". Below is the source of the page, am I missing something?
view-source:https://student.ashford.edu/student/
https://student.ashford.edu/student/
What I want to have happen is that in my VB.NET backend, I click a button and my username and password are automatically entered and I am automagically logged in. Once the login is done, I want to retrieve the HTML of the logged in page so I can parse it for items relating to my studies. This may be unrelated, but do I need cookies to keep my application "logged in" if I navigate to other pages?
EDIT:
I tried plugging different URL's, to no avail. I have also tried adding submit=login and login=submit (alternately of course) to the POST data. I'm still getting the original login screen HTML from the result variable at the end, instead of my university homepage that I'm expecting after I normally login by hand as a human would. Maybe there is a follow up step that I am missing?
The actual form action is at https://student.ashford.edu/login/commands/login.php. You might plug that into url instead. Also, try including the submit button - submit=login - in the values.
I found out what I needed to do. First I have to make sure that I have a cookie container. Next, I used Fiddler to find out EXACTLY what is being sent back and forth between my browser and where I want to login. I post all the login info, and I get the response back. This seems to fill up my cookie container variable with the required authentication cookies? Next I make another POST request to the page I want to go to, and voila - it works.