I am trying to login this website https://www.bewickedcostumes.com/login.php by httpwebrequest.
//Here is GET
https://www.bewickedcostumes.com/login.php?open_in_layer=Y&is_ajax_request=Y&keep_https=Y
GET /login.php?open_in_layer=Y&is_ajax_request=Y&keep_https=Y HTTP/1.1
Host: www.bewickedcostumes.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
X-Requested-With: XMLHttpRequest
Referer: https://www.bewickedcostumes.com/login.php
Cookie: store_language=en; __atuvc=22%7C31%2C14%7C32; RefererCookie=http%3A%2F%2Fwww.bewickedcostumes.com%2Fproduct.php%3Fproductid%3D18078%26cat%3D%26bestseller%3DY; GreetingCookie=Sammy+Chan; xid_d9db5C_remember=schan%40dropshipinc.com; xid_d9db5=21e18e3615c5ed2ef0597789bcd74746
If-Modified-Since: Wed, 08 Aug 2012 01:49:39 GMT
HTTP/1.1 200 OK
Date: Wed, 08 Aug 2012 02:10:57 GMT
Server: Apache
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: private, must-revalidate
P3P: CP="NON CURa ADMa DEVa TAIa CONi OUR DELa BUS IND PHY ONL UNI PUR COM NAV DEM STA"
Set-Cookie: xid_d9db5=21e18e3615c5ed2ef0597789bcd74746; path=/; domain=www.bewickedcostumes.com; httponly
Last-Modified: Wed, 08 Aug 2012 02:10:57 GMT
Vary: Accept-Encoding,User-Agent
Content-Encoding: gzip
Content-Length: 1220
Keep-Alive: timeout=5, max=99
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
//Here is POST
https://www.bewickedcostumes.com/login.php?open_in_layer=Y&is_ajax_request=Y&keep_https=Y
POST /login.php?open_in_layer=Y&is_ajax_request=Y&keep_https=Y HTTP/1.1
Host: www.bewickedcostumes.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: https://www.bewickedcostumes.com/login.php
Content-Length: 119
Cookie: store_language=en; __atuvc=22%7C31%2C14%7C32; RefererCookie=http%3A%2F%2Fwww.bewickedcostumes.com%2Fproduct.php%3Fproductid%3D18078%26cat%3D%26bestseller%3DY; GreetingCookie=Sammy+Chan; xid_d9db5C_remember=schan%40dropshipinc.com; xid_d9db5=21e18e3615c5ed2ef0597789bcd74746
Pragma: no-cache
Cache-Control: no-cache
xid_d9db5=21e18e3615c5ed2ef0597789bcd74746&is_remember=&mode=login&username=schan%40dropshipinc.com&password=4700Miller
HTTP/1.1 200 OK
Date: Wed, 08 Aug 2012 02:11:06 GMT
Server: Apache
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: private, must-revalidate
P3P: CP="NON CURa ADMa DEVa TAIa CONi OUR DELa BUS IND PHY ONL UNI PUR COM NAV DEM STA"
Set-Cookie: xid_d9db5=21e7b58ed5b06ad7d5897ad641186f8b; path=/; domain=www.bewickedcostumes.com; httponly
Set-Cookie: GreetingCookie=Sammy+Chan; expires=Mon, 04-Feb-2013 02:11:06 GMT; path=/; domain=www.bewickedcostumes.com; httponly
Last-Modified: Wed, 08 Aug 2012 02:11:06 GMT
Vary: Accept-Encoding,User-Agent
Content-Encoding: gzip
Content-Length: 193
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
My code can login many site but with this site it doesn't work.
This part is a unique code that is generated on the Login page by the looks of it:
xid_d9db5=21e18e3615c5ed2ef0597789bcd74746
Scrape the above field & value & add it to the POST request instead of adding the exact code above to your request.
(You may also want to hide your POST data a bit more carefully incase you use them details elsewhere!)
Related
https://api-staging-weld.freetls.fastly.net/scripts/customdomain_weld.19f3e9ec.js
The error I get in Google Search Console is Temporarily unreachable with no further explanation. I think it's something with my http headers but can't figure out which one. Here they are:
accept-ranges: bytes
access-control-allow-origin: *
age: 4905
cache-control: public, max-age=31536000
content-encoding: gzip
content-length: 29990
content-type: application/javascript; charset=UTF-8
date: Fri, 20 Apr 2018 12:46:37 GMT
etag: W/"1a018-162e269bfe0"
last-modified: Fri, 20 Apr 2018 09:36:44 GMT
server: Cowboy
status: 200
vary: Accept-Encoding
via: 1.1 vegur
via: 1.1 varnish
x-cache: HIT
x-cache-hits: 1
x-powered-by: Express
x-served-by: cache-bma7030-BMA
x-timer: S1524228398.925110,VS0,VE6
What's wrong or what am i Missing?
My browser send to the server the following request:
Host: www.imprimante.be
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
Accept: */*
Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
If-Modified-Since: Fri, 29 May 2015 14:22:44 GMT
If-None-Match: "90-5173935ad3a1a-gzip"
Referer: http://www.imprimante.be/premier-avis-gratuit/
Cookie: <hidden>
Connection: keep-alive
The url used is http://www.imprimante.be/wp-content/themes/mch_imprimante/js/theme.min.js? (note: www.imprimante.be is not accessible trough wlan yet)
And the server send me the file with this (status 200) http header:
Accept-Ranges: bytes
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 137
Content-Type: application/javascript
Date: Wed, 03 Jun 2015 07:18:03 GMT
Etag: "90-5173935ad3a1a-gzip"
Keep-Alive: timeout=5, max=99
Last-Modified: Fri, 29 May 2015 14:22:44 GMT
Server: Apache/2.4.10 (Debian)
Vary: Accept-Encoding
As you might notice (Last-Modified: Fri, 29 May 2015 14:22:44 GMT) the file hasn't been modified since the last request.
So I don't get why the response isn't a 304 status: not modified.
I'd really like to know why the caching of this files (and some others) doesn't work as I expect it.
It is bug in Apache. Turn off mod_deflate.
Sometimes my IIS server starts responding with 304 HTTP status to non-conditional requests. I have yet to determine exactly when this happens, but when it starts all browsers keep getting 304.
This is the request (debugged using fiddler):
GET http://www.example.com/aaa/bbb/GetStaticResult HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0
Accept: */*
Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://localhost:12345/xxx
Origin: http://localhost:12345
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
And the response:
HTTP/1.1 304 Not Modified
Cache-Control: public
Expires: Thu, 30 Apr 2015 09:23:22 GMT
Last-Modified: Wed, 30 Apr 2014 09:23:22 GMT
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Wed, 30 Apr 2014 10:33:58 GMT
The ASP.NET MVC method is the following:
[HttpGet]
[OutputCache(Duration = int.MaxValue, VaryByParam = "")]
public string GetStaticResult()
{
return "static";
}
The request hasn't any If-Modified-Since (or similar) header. Shouldn't the server issue a 200 response?
May be related with Getting a 304 if there is no If-Modified-Since sent?
Why Google webmaster tool doesn't show the verified site on the webmaster tool? I am getting status code 200 when using both OAuth 2 playground and my program. But when check on https://www.google.com/webmasters/tools/ I cannot see the verified site.
This is the response I am getting from Google:
HTTP/1.1 200 OK
Content-length: 220
Via: HTTP/1.1 GWA
X-content-type-options: nosniff
Etag: "00deUEyKiunbfrZpRY_GOwanRBXo/rPzr3x8uGF7cZ_o8tWZ9Z9Yp87RwQU"
X-google-cache-control: remote-fetch
-content-encoding: gzip
Server: GSE
X-xss-protection: 1; mode=block
Pragma: no-cache
Cache-control: no-cache, no-store, max-age=0, must-revalidate
Date: Wed, 10 Jul 2013 09:38:26 GMT
X-frame-options: SAMEORIGIN
Content-type: application/json; charset=UTF-8
Expires: Fri, 01 Jan 1990 00:00:00 GMT
{"id":"http%3A%2F%2Fsample.net%2F","site":{"type":"SITE","identifier":"http://sample.net/"},"owners":["sample#gmail.com"]}
Can anyone suggest the reason behind this?
Does anyone know how to get all the videos from a playlist ?
E.g from this playlist
https://gdata.youtube.com/feeds/api/playlists/454A0274C1223C90?v=2
it caps at 10 results
I tried &max-results=1000 but that 404'ed
try asking for 50 results at a time ( max allowed )
GET /feeds/api/playlists/454A0274C1223C90?v=2&&max-results=100&fields=entry/id,entry/title&key=4{your-key} HTTP/1.1
Host: gdata.youtube.com
Authorization: OAuth ya29.AHES6ZQMwDjrUKh8RLysv-ZzF3DaUmg0XmG0jtVcWdSA0A
HTTP/1.1 400 Bad Request
status: 400
content-length: 246
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
expires: Sat, 28 Apr 2012 16:29:01 GMT
x-google-cache-control: remote-fetch
server: GSE
via: HTTP/1.1 GWA
x-gdata-user-country: US
cache-control: private, max-age=0
date: Sat, 28 Apr 2012 16:29:01 GMT
x-frame-options: SAMEORIGIN
content-type: application/vnd.google.gdata.error+xml
-content-encoding: gzip
<errors xmlns='http://schemas.google.com/g/2005'><error><domain>GData</domain><code>InvalidRequestUriException</code><internalReason>Max-results value is too high. Only up to 50 results can be returned per query.
GET /feeds/api/playlists/454A0274C1223C90?v=2&&max-results=50&fields=entry/id,entry/title&key=${key} HTTP/1.1
Host: gdata.youtube.com
Authorization: OAuth ya29.AHES6ZQMwDjrUKh8RLysv-ZzF3DaUmg0XmG0jtVcWdSA0A
HTTP/1.1 200 OK
status: 200
gdata-version: 2.1
content-length: 6703
via: HTTP/1.1 GWA
content-location: https://gdata.youtube.com/feeds/api/playlists/454A0274C1223C90?v=2&&max-results=50&fields=entry/id,entry/title&key=w1Q
x-content-type-options: nosniff
expires: Sat, 28 Apr 2012 16:32:08 GMT
vary: Accept, X-GData-Authorization, GData-Version
x-google-cache-control: remote-fetch
server: GSE
last-modified: Sat, 28 Apr 2012 10:46:07 GMT
x-xss-protection: 1; mode=block
etag: W/"A04CRn47eCp7I2A9WhVWFk0."
cache-control: private, max-age=300, no-transform
date: Sat, 28 Apr 2012 16:32:08 GMT
x-frame-options: SAMEORIGIN
content-type: application/atom+xml; charset=UTF-8; type=feed
-content-encoding: gzip
x-gdata-user-country: US