recreate this web request in vb net - vb.net

i?m trying to make automatic image set in pinterest account with WebClient.
I'd like to recreate this http request:
(Request-Line) POST /upload-image/?img=Desert.jpg HTTP/1.1
Host www.pinterest.com
User-Agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0
Accept text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language it-IT,it;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding gzip, deflate
X-Requested-With XMLHttpRequest
X-File-Name Desert.jpg
Cache-Control no-cache
X-CSRFToken RqwJCawJyAGYIZfzob51qxrEGj4GJcSA
Referer https://www.pinterest.com
Content-Length 846128
Content-Type multipart/form-data; boundary=---------------------------5431268530037
Cookie _pinterest_cm=TWc9PSY1YlkwcmtVRGlNRzRQZXpiZXJseVl6TnFHYnEvZlhpNDZPcExCQnhKN3UvdUUveWI0c3p4bWJKUmhoZy9YRG9sS3dNZTZFSFNhN2V3VWhJM1JkbUlxbC92VjhHUGFldlRTVVJTNlA1L1M0SDE5QXhLcHVWS2ZrSUh3NTN2ODA0WSZ5dnpJQkVRUmx5TVJGTEdmQm5EVmRGQXNqbDQ9; csrftoken=RqwJCawJyAGYIZfzob51qxrEGj4GJcSA; _pinterest_sess="TWc9PSYvRm93OFNxbGkxWTJ5bzVoZUFudHJVVDI4bndmL2I5SFVIQjZkVk1KWFJ3WDBmNndOWFR0QnBPazltZFRmcnJpcGU5akhQZS8vcEZWTzM5ODJWNVdKS2syekwwc1p1TVVNeEt3Z3NYa1lsMVFXcFpXYUdkRlE1RElQYTBYeXhyQkFkYVFmSHZnVkRyYWhYcURDYzhhWEpuR2dvekE1SlB6cXp4akNNdzJ6QysrR2MzRGNyRXJKczRuRHZDTm1uQkdLMUJrUnF6UjdZakhDUGNVRnQ4T1ZoQUFIQWJSU1VNUHNjUHV5VjlZbk1INU1FMFNhdGJiVFZRdUNDWFNlMGJvcDBFeXk4a3cxT25ROHpSOXFzcTZ6NFBHekFjNkFNQUtnaktQNGQ1VkhnNDdlSXNQTGhmTzhDWm5UaDNoZzdqbEFHQWQ1RjJXWVo5bjNXVkVUWnVUWXNiL1JLTFdqNDBvMWllT0VyRDRNN1lXN0diQmlWRjdGdWF5UGUzYkNLYlMvamJUSVFwcFZoVVVUL2ROVkFIQUNYODQxR3R5eDFrQ0VpTGhmaGZ1Y2VOdGt6aUdLQmtCTkRYdkpkVGhmLzMvMnVOWHAwQVdZWEs2alE4eTUwd3E1SlJPRWFDc3VKTXByb2tISm8rcldRQT0mejAwN0hvdlRhbU8zYmNJT0lsSm9PSldheGpJPQ=="; sessionFunnelEventLogged=1; __utma=229774877.448600758.1436737610.1436739423.1436745191.3; __utmz=229774877.1436737610.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); c_dpr=1; __utmc=229774877; _b="ARLbRMvYKUdKiaBWDA2Oxko87z7iIN4MuGnJALvZK8vehgzT11AKeoa13PH4l9VjVMU="; _pinterest_pfob=disabled; __utmb=229774877.3.9.1436745219732; __utmt=1
Connection keep-alive
Pragma no-cache
I have try this code, but i can't obtain Content-Length and Content-Type.
Dim wc As New WebClient
wc.UseDefaultCredentials = True
wc.Credentials = New NetworkCredential("pippomio#yahoo.com", "88Y71nR3764")
wc.Headers.Add("Host", "www.pinterest.com")
wc.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.56 Safari/536.5")
wc.Headers.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
wc.Headers.Add("Accept-Language", "it-IT,it;q=0.8,en-US;q=0.5,en;q=0.3")
wc.Headers.Add("Accept-Encoding", "gzip, deflate")
wc.Headers.Add("X-Requested-With", "XMLHttpRequest")
wc.Headers.Add("X-Requested-With", "XMLHttpRequest")
wc.Headers.Add("X-File-Name", "Hydrangeas.jpg")
wc.Headers.Add("Cache-Control", "no-cache")
wc.Headers.Add("X-CSRFToken", token)
wc.Headers.Add("Referer", "https://www.pinterest.com")
wc.Headers.Add("Connection", "keep-alive")
wc.Headers.Add("Pragma", "no - cache")
Dim Response As Byte() = wc.UploadFile("https://www.pinterest.com/upload-image/?img=Hydrangeas.jpg", "POST", "Hydrangeas.jpg")
In wich way can I do this request in vb net?
Thanks

First, I recommend you to check this and this tutorial to learn how to send/receive HTTP requests on correct way.
Second, you should not re-do any web browser actions in your program since it is usually not a good practice as the frontend architecture should be subject an unexpected change any time. Instead of this, you should check Pinterest API, especially the Users API which can help you to achieve your plans. Usually API interfaces are not a subject of random changes, they are more reliable than replaying front-end operations and more stable, has more capabilities to the load.
(Pinterest API seems working only from Firefox, if you get an empty area at right side with a big "None" text, then browse the link from Firefox - it seems can handle the page)

Related

Servicestack UnAuthorized

I am getting "The remote server returned an error: (401) Unauthorized." when using the servicestack utils to call a remote API. It requires auth basic or JWT. Calling the same API from PostMan works fine.
var json = $"http://localhost:5000/API/Proxy/IsValidGuidForSetupDevice?Guid=82870f2ca21148739bff0854b306415c".GetJsonFromUrl(requestFilter: webReq => { webReq.AddBasicAuth("DevAdmin", "test1"); });
if i call below with the same user pass in a browser window. i get a valid connect and a bearer token back. The user\pass is good.
http://localhost:5000/auth/credentials?username=DevAdmin&password=test1&format=json
AM I missing something in request filter of the Util? Maybe I should be calling it differently and using a bearer token?
Update after tracing with fiddler
I moved the code into a unit test in the same project as the servicestack service. Just to remove any variables.
Fiddler shows me that : No Proxy-Authenticate Header is present. WWW-Authenticate Header is present: Basic realm="/auth/apikey" GET http://localhost:5000/auth
Raw View
HTTP/1.1
Host: localhost:5000
Connection: keep-alive
Accept: application/json, text/javascript, */*; q=0.01
DNT: 1
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Referer: http://localhost:5000/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: ss-opt=perm; loginLevel=none; X-UAId=2; ss-id=XPc7ivcCrXuN5tEWwARG; ss-pid=TDeEjUiKck82foJGLGtX
Playing around with it. I can get it to login by calling the URL directly with user and pass. The i get the bearer token back and I am able to pass it.
var json = $"http://localhost:5000//auth/credentials?username=DevAdmin&password=test1".GetJsonFromUrl();
var o = JsonObject.Parse(json);
var token = o.Get<string>("bearerToken");
var jsonr = $"http://localhost:5000/API/Proxy/IsValidGuidForSetupDevice?Guid=bc464658d6a640febbd53ba17c351919".GetJsonFromUrl(
requestFilter: webReq => { webReq.AddBearerToken(token); });
I still can't call this in one call with auth headers and I still don't know why.
Whenever diagnosing different behavior between HTTP Requests of different clients you should inspect the raw HTTP Traffic between the different clients using a HTTP Packet sniffer like Fiddler or WireShark, or if the request is made from a browser you can use their built-in Web Network Inspector tools.
Seeing and comparing raw HTTP Request/Responses are the best way to identify how the requests differ.
The AddBasicAuth() extension method lets you add HTTP Basic Auth to the Web Request, if you want to send the JWT Token instead you would use the AddBearerToken() method instead.

FineUploader - "isBrowserPreviewCapable" variable not thrown out by Chrome 40, though it should

I noticed that in the FineUploader server demos (s3demo-thumbnails-cors.php), there is an area that checks if the browser supports preview. It simply checks this post variable: $_POST["isBrowserPreviewCapable"]
The awkward thing here I'd like to ask is, I'm actually using Chrome 40, and I'm absolutely sure that preview is available for my browser. But in the "upload_success" ajax call sent out by FineUploader, it does NOT contain $_POST["isBrowserPreviewCapable"].
Here is a sample dump of my upload success AJAX call. I was wondering if I had missed some sort of configuration for this:
Remote Address:127.0.0.1:80
Request URL:http://localhost/development/code-base/ci/builds/xyz/en/file/api/notify_successful_upload
Request Method:POST
Status Code:200 OK
Request Headers
Accept:application/json
Accept-Encoding:gzip, deflate
Accept-Language:ja,en;q=0.8,en-US;q=0.6,zh;q=0.4,zh-TW;q=0.2,zh-CN;q=0.2,ko;q=0.2
Cache-Control:no-cache
Connection:keep-alive
Content-Length:614
Content-Type:application/x-www-form-urlencoded
Cookie:PHPSESSID=952ciound37e9dkaf5d1hmmc1; __atuvc=10%7C5; CKFinder_Path=Attachments%3A%2F%3A1; logged_in=9db847c22b2bef66cc06091e355a80e6aff83b7d377; abc_session=a%3A5%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%2244ef7cfb140643bdb09df62f0e9c3561%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A9%3A%22127.0.0.1%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A109%3A%22Mozilla%2F5.0+%28Windows+NT+6.1%3B+WOW64%29+AppleWebKit%2F537.36+%28KHTML%2C+like+Gecko%29+Chrome%2F40.0.2214.115+Safari%2F537.36%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1424977019%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3B%7Ddc95cd1bb88507af0eb260abe18f380bbd80e1fd; user_locale=en
Host:localhost
Origin:http://localhost
Referer:http://localhost/development/code-base/ci/builds/xyz/en/file
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
X-Requested-With:XMLHttpRequest
Form Data
key:clients/abclient/www/gallery/images/w100/_try_90b313b2-9cea-42f3-8332-e6d13071ded3.png
uuid:c9ed8346-43fb-4116-afe6-18867079524e
name:_try (w100).png
bucket:xyz-production
etag:"b2df53409d64b9e1cb9f1e590d2a4bf6"
file_uuid:c9ed8346-43fb-4116-afe6-18867079524e
file_family_uuid:90b313b2-9cea-42f3-8332-e6d13071ded3
file_storage:s3
file_module:gallery
file_type:images
file_variation:w100
file_name:_try_90b313b2-9cea-42f3-8332-e6d13071ded3.png
file_display_name:_try (w100).png
qqparentuuid:90b313b2-9cea-42f3-8332-e6d13071ded3
qqparentsize:44936
qquuid:c9ed8346-43fb-4116-afe6-18867079524e
Response Headers
Access-Control-Allow-Origin:http://localhost
Connection:Keep-Alive
Content-Length:539
Content-Type:text/html
Date:Thu, 26 Feb 2015 18:57:03 GMT
Keep-Alive:timeout=5, max=94
Server:Apache/2.4.9 (Win64) OpenSSL/1.0.1g PHP/5.5.12
Set-Cookie:user_locale=en; expires=Thu, 26-Feb-2015 20:57:03 GMT; Max-Age=7200; path=/
X-Powered-By:PHP/5.5.12
Any help would be greatly appreciated. Thanks!
Cheers,
Thomas
Fine Uploader does not send the POST variable you are speaking of. The S3 demo on http://fineuploader.com/demos includes a line of code that ensures the parameter is sent with the request.
For example, the following option is configured:
uploadSuccess: {
endpoint: "http://s3-demo.fineuploader.com/s3demo-thumbnails-cors.php?success",
params: {
isBrowserPreviewCapable: qq.supportedFeatures.imagePreviews
}
}

POST data not sent in IE 11 using the WebBrowser control

I have the following JavaScript code being executed inside of an onclick event handler on my website.
var newRequest = new window.XMLHttpRequest();
newRequest.open('POST', '/index.cfm', true);
newRequest.send('q');
If I use IE 11 and open up my web page that is located on a testing server I can see that the request gets sent as expected using Fiddler with a content length of 1 and 'q' in the post data. However, if I open up my application that hosts the WebBrowser control and navigate to the same website on my test server and have it execute the above code I can always see that the request is being made but that the Content-Length header is 0 and 'q' is not sent along with the request. Here is the failing request as it appears in Fiddler.
POST http://test.mycompany.com/index.cfm HTTP/1.1
Accept: */*
Referer: http://test.mycompany.com/Curtis/BrowserTests/BrowserEventTests.html
Accept-Language: en-US
Content-Type: text/plain;charset=UTF-8
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Host: test.mycompany.com
Content-Length: 0
Connection: Keep-Alive
Pragma: no-cache
If I then set the HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_BROWSER_EMULATION registry value for my executable to 9000 then it works and does send the post data as expected.
Here is the correct request as reported by Fiddler.
POST http://test.mycompany.com/index.cfm HTTP/1.1
Accept: */*
Referer: http://test.mycompany.com/Curtis/BrowserTests/BrowserEventTests.html
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Host: test.mycompany.com
Content-Length: 1
Connection: Keep-Alive
Pragma: no-cache
q
If I change that value back to 10000 or 11000 then it does not work. Does anyone have any ideas on why the post data would not be sent properly using the WebBrowser control? I have reset my IE settings back to factory default with no change in behavior.
Update: If I change the JavaScript to look like this instead then it works with the emulation mode set to 10000.
var newRequest = new ActiveXObject("Msxml2.XMLHTTP");
newRequest.open('POST', '/index.cfm', true);
newRequest.send('q');
Is this just a bug that needs to be reported?

WIF: LocalSTS not authenticating

I'm totally new to WIF and to start playing with it a bit I tried the simplest "f5 experience" with an MVC4 application; according to the tutorials I found, which sadly for the most part refer to releases before .net 4.5, I should just create an MVC app, configure it with the identity and access tool and hit f5 to get up and running with local STS. Yet, I'm probably missing something obvious because when I try to access a restricted page I always end bumped back to the homepage.
Here is what I did, you can easily repro the issue with these steps (VS2012 in Win8 with WIF SDK; ensure to launch VS with admin rights):
create a new ASP.NET MVC4 Internet application. Set its port to 7777 (just picking the port number used in most code samples for the sake of commodity).
update all the NuGet packages (this is optional).
right click the solution, choose identity and access and set the IP=local STS, then click OK. Then reopen the identity and access popup, and choose generate a controller, then click OK.
add an [Authorize] attribute to the About action of the Home controller.
hit F5 and click the About link. As expected, the login view appears, prompting me to login: the only option is of course localSTS. When I click it, I am returned to the homepage and no authentication occurs. I can repeat the process, but nothing changes, so I can never access the secured About page.
The link underlying the localSTS anchor is:
http://localhost:14743/wsFederationSTS/Issue?wa=wsignin1.0&wtrealm=http%3a%2f%2flocalhost%3a7777%2f&wctx=rm%3d0%26id%3d664ff3c2-95b1-40b3-b538-a8357233ea7e%26ru%3dhttp%253a%252f%252flocalhost%253a7777%252f&wct=2013-03-10T13%3a39%3a32Z
AFAIK, its parameters look OK.
If I examine the network traffic, I cannot see any relevant item (if I understand well, I would expect a response setting some cookies for the current session, representing the IClaimsPrincipal).
(BTW, by looking at the web.config, I can see that in modules WSFederationAuthenticationModule is referenced from System.Identity.Services, which is NOT included in the solution after configuring Identity and access; I suppose this is a bug in the tool. Anyway I added a reference to it, but nothing changed.)
Update
Thank you for the reply! If I examine the traffic, here are the relevant GET/POSTs. I get no cookie. I tried recreating the whole test solution, even skipping step #2 above to keep it minimal, but nothing changed.
As for your suggestion #2, I tried to add in global asax:
FederatedAuthentication.WSFederationAuthenticationModule.SecurityTokenValidated
+= (sender, e) => FederatedAuthentication.SessionAuthenticationModule.IsReferenceMode = true;
but the SessionAuthenticationModule is null at the time this code executes and thus a corresponding exception is thrown. I cannot find up-to-date code samples or articles about this, yet WIF seems a very promising tech; I'd like it to be easy for security newbies like me; my main purpose is applying it to a site providing both MVC controllers and WebApi controllers, to a wide range of consumers (JS code, mobile apps, WinRT apps, the site pages themselves...). Any suggestion?
(1) a GET which gets 307, temporary redirect
GET /wsFederationSTS/Issue?wa=wsignin1.0&wtrealm=http%3a%2f%2flocalhost%3a7777%2f&wctx=rm%3d0%26id%3d664ff3c2-95b1-40b3-b538-a8357233ea7e%26ru%3dhttp%253a%252f%252flocalhost%253a7777%252f&wct=2013-03-10T13%3a39%3a32Z HTTP/1.1
Host: localhost:14743
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.160 Safari/537.22
Referer: http://localhost:7777/HrdAuthentication/Login?ReturnUrl=%2fHome%2fAbout
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie:
(2) GET with signin request:
GET /wsFederationSTS/Issue/?wa=wsignin1.0&wtrealm=http%3a%2f%2flocalhost%3a7777%2f&wctx=rm%3d0%26id%3d664ff3c2-95b1-40b3-b538-a8357233ea7e%26ru%3dhttp%253a%252f%252flocalhost%253a7777%252f&wct=2013-03-10T13%3a39%3a32Z HTTP/1.1
Host: localhost:14743
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.160 Safari/537.22
Referer: http://localhost:7777/HrdAuthentication/Login?ReturnUrl=%2fHome%2fAbout
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie:
(3) a POST to the homepage: the respone is of course the homepage content; no cookies set.
POST / HTTP/1.1
Host: localhost:7777
Connection: keep-alive
Content-Length: 7063
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: http://localhost:14743
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.160 Safari/537.22
Content-Type: application/x-www-form-urlencoded
Referer: http://localhost:14743/wsFederationSTS/Issue/?wa=wsignin1.0&wtrealm=http%3a%2f%2flocalhost%3a7777%2f&wctx=rm%3d0%26id%3d664ff3c2-95b1-40b3-b538-a8357233ea7e%26ru%3dhttp%253a%252f%252flocalhost%253a7777%252f&wct=2013-03-10T13%3a39%3a32Z
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie:
A working variation
I found a way for letting it work, maybe this can be useful to someone else: if you follow the above procedure without changing from the local IIS server to the VS development server, it seems it's working and I'm still redirected to the Home page (I wonder why:), but as an authenticated user; at this point, I can click the About link again to effectively enter the page.
I reproduce your steps and everything worked. Chrome debugger show that you must have two steps:
GET http://localhost:12263/wsFederationSTS/Issue/?wa=wsignin1.0&wtrealm=http%3a%2f%2flocalhost%3a54306%2f&wctx=rm%3d0%26id%3dc6e46b99-417b-49b6-96a0-40efcead898f%26ru%3dhttp%253a%252f%252flocalhost%253a54306%252f&wct=2013-03-10T18%3a14%3a19Z
POST http://localhost:54306/ with wa:wsignin1.0 and wresult:trust:RequestSecurityTokenResponseCollection
Result of POST is Set-Cookie named FedAuth and FedAuth1. Cookie is split because of cookie size limit.
Please check this in debugger.
p.s. Once i saw a same behavior - cookie is not set normally. Problem was in cookie size and solved by switching to ReferenceMode. Don't forget to to register it in Application_Start:
FederatedAuthentication.WSFederationAuthenticationModule.SessionSecurityTokenCreated
+= this.WSFederationAuthenticationModule_SessionSecurityTokenCreated;

ASANA API and Access-Control-* headers

I am writing some code to call the Asana API from the browser. XDR requests emitted from the browsers all begin with a call to OPTIONS to get the Access-Control-* headers, but Asana server does not seem to response to them:
Request URL:https://app.asana.com/api/1.0/users?opt_pretty=true&opt_fields=name,email
Request Method:OPTIONS
Status Code:404 Object Not Found
Request Headers:
Accept:*/*
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4
Access-Control-Request-Headers:origin, authorization, accept
Access-Control-Request-Method:GET
Connection:keep-alive
Host:app.asana.com
Origin:null
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1129.0 Safari/536.11
Query String Parameters:
opt_pretty:true
opt_fields:name,email
Response Headers:
Content-Length:82
Content-Type:application/json
Date:Sat, 12 May 2012 22:23:19 GMT
Server:nginx/0.7.67
...
Response headers should contain something like:
Access-Control-Allow-Headers: Accept, Authorization, Content-Type
Access-Control-Allow-Methods: GET,PUT,POST,DELETE
Access-Control-Allow-Origin: *
Or is there anything that I am missing when requesting the API?
(I work at Asana)
The Asana API does not return these headers right now because it does not support OAuth and cannot securely authenticate requests from clients. It is only secure to return these headers for an authenticated client, otherwise the service would be open to cross-site scripting attacks. OAuth is on the list of features to be supported in the future.