I am writing an application that gets the server response code for a set of URLS by using the HttpWebRequest class. I came across a URL today that is causing me problems.
The problematic URL is http://blip.tv/file/5312019
When I load this URL in Internet Explorer, it correctly redirects me to http://blip.tv/sorawut/money-talk-เม่า-นักเขียนการ์ตูนหุ้น-5329374. But when using the HttpWebRequest class, it's having a problem redirecting.
If I set AllowAutoRedirect to false and examine the Location metatag in the response.Headers collection, it is showing the funky URL http://blip.tv/sorawut/money-talk-à¹à¸¡à¹à¸²-à¸à¸±à¸à¹à¸à¸µà¸¢à¸à¸à¸²à¸£à¹à¸à¸¹à¸à¸«à¸¸à¹à¸-5329374. When the request attempts to redirect to this URL, it causes an infinite redirect loop and ultimately ends up throwing a WebException saying "Too many automatic redirections were attempted".
I tried pasting this funky URL into Internet Explorer and it automatically changed it to the correct redirect URL and successfully loaded the page.
So, what do I need to do to have my HttpWebRequest return a status code of 200 for this particular URL? (Since it is a valid and active URL after a successful redirect)
try this in your config file
<configuration>
<uri>
<idn enabled="All" />
<iriParsing enabled="true" />
</uri>
</configuration>
Look at the section International Resource Identifier Support here -> http://msdn.microsoft.com/en-us/library/system.uri.aspx
Related
I am trying to do a performance testing for a project where the main URL performs a redirection and the redirection URL returns the token needed to proceed further.
I recorded the script with Jmeter and i can see that the main URL does the redirection with status code as "302" as expected and has the "Location" header in the response body.
But when the redirection URL is hit after the main URL as the sub sample, it returns "307 temporary redirection" in response header and an error message in the response body as "java.lang.IllegalArgumentException: Missing location header in redirect for GET "
I checked the same scenario in Neoload and i could notice that request sent in Neoload for rediection URL is same as the one sent in JMeter, the response header also matches. But Neoload was able to display the response body with the expected token. In Jmeter, it returned the error message.
Can someone clarify why the response body is not displayed in Jmeter if the same request can work in Neoload. I made sure there is no difference in the request header/body sent for the main URL.
I am pretty sure the issue is related to Jmeter but not sure how to get the proper response body.
Maybe there is more than one redirect and JMeter doesn't follow it.
You can play with Redirect automatically and/or Follow redirects boxes in HTTP Request sampler
also you can use a third-party sniffer tool like Wireshark or Fiddler to inspect which requests are actually being sent, pay attention to literally everything: URL, headers, body, etc.
It should also be possible to disable handling redirects in JMeter completely and extracting the redirect target from the Location header using Regular Expression Extractor or Boundary Extractor and manually add the next request giving the extracted URL in the "Path" field of the HTTP Request sampler
I found a vulnerability for content-spoofing on my webpage.
This URL:
https://www.mygreatsite.com/www.mygreatsite.com%20has%20moved%20to%20www.evilsite.com.%20CHECK%200UT%20H0W%20COOL%20THE%20NEW%20SITE%20IS!%20Sadly,%20the%20file%00
Apache Output:
Not Found
The requested URL /www.mygreatsite.com has moved to www.evilsite.com. CHECK 0UT H0W COOL THE NEW SITE IS! Sadly, the file was not found on this server.
--
The URL Overrides the Apache-Error and outputs the text added to the URL in the Browser, along with a 404-Error.
Actually, all requests to the server should be redirected to the CMS (Typo3) which in turn handles the 404-error and shows a custom page.
When I enter an URL that doesn't exist, this works perfectly. Just the above mentioned URL screws everything up.
Interestingly, when I delete the «%00» from the end of the URL, the request is forwarded to the CMS and the correct error document is displayed.
I tried to add a separate ErrorDocument-Directive to the htaccess-file - with no success.
Any ideas what goes wrong and how I could solve this?
As Anubhava points out, when a NULL-Byte is detected, Apache doesn't load any modules and just throws the 404. That's why ErrorDocument and mod_rewrite in the htaccess don't work.
[Edit for clarification]
%00 or in Hex \x00 is a NULL byte. When web server finds a NULL byte at the end then web server thinks it is a spoofing request and considers it to be dangerous to be processed by any directives.
Due to security reasons web server doesn't load any modules for this request and returns a 404 / Not found status. Browser shows default 404 page with your decoded URL just below Not Found text.
I originally asked the following question, to get some anwsers on how to handle special characters in an URL (GET) request for my Web Api:
Web Api 2 routing issue with special characters in URL
Encoding was obviously the way to go. But in order to get everything working, i had to do a pretty nasty workaround. And now, Im' at the point where i don't really understand why my workaround had to be done in the first place. So, the following is my setup:
A client can call my Web Api 2, Hosted on iis 8.5, by a get request containing an email in the URL. The most extreme example would be the following email:
!$%&'*+-/=?^_`{}|~#test.com
And yes, that sucker is a valid email, which therefore the API has to support. The URL pattern is as follows:
.../api/permissions/{email}/{brand}/
So a get request would be something along the lines of this:
.../api/permissions/#!$%&'*+-/=?^_`{}|~#test.com/economy
As the marked answer to my other question suggests, encoding this url is obviously a necessity. But this left me with a couple of other issues, such as "double escape of characters not allowed", and some specific "404 - not found" (routing could not pass the url). This i could mange to handle with the following settings for iis:
<system.web>
...
<httpRuntime requestPathInvalidCharacters=""/>
</system.web>
<system.webServer>
<security>
<requestFiltering allowDoubleEscaping="true"/>
</security>
...
</system.webServer>
Now i was able to call my method with those pesky special characters, and everything was fine. But i hit another bump. The email specified above, #!$%&'*+-/=?^_`{}|~#test.com, resulted in a 404 - not found. An actual 404 - not found. The routing couldn't handle the request url:
[Route("{email}/{brand}"]
As I understand it, the iis decodes the request url, passes it on to the iis request pipeline, it is then picked up by the web api and run through the http message handlers before hitting the controller. In the controller, i could clearly see that the email part of the url was no longer encoded (provided i used a simple encoded email. the encoded email "#!$%&'*+-/=?^_`{}|~#test.com" still responded 404). I quickly figured, that the routing probably couldn't handle the fragmentation inside the url path, as the iis passes on a decoded url to the web api. So i had to get the url reencoded in the iis before handed on to the web api.
This i was able to make a workaround for, by using Url ReWrite. It reencoded that specific part of the url containing the email, and now the routing was handled properly with the special-character-email. The expected method was hit, and i could just decode the encoded email. To briefly sum up, this was the flow:
Request flow
Now, we have set up a LogMessageHandler which logs incoming requests and outgoing responses. When the logger logs the request.RequestUri, it is clear that the email is double encoded. But when the controller method is hit, it is only encoded once! So.. My question is, why do i have to reencode the URL in the iis for the routing to handle the request properly, when the url is already automatically encoded (and decoded again before hitting the controller)? Is this something i can configure? Can i somehow extend the scope of which the URL is encoded, all the way to the controller??
Regards
Frederik
I have an MVC site which is replacing an old website. I want to be able to handle requests for the old static html files but redirect to the 404 error handler on my MVC site - I'd also like to provide a 301 code instead of a 404 to let the crawlers know that the content has moved
At the moment when I try and navigate to static content I get the IIS 404 error. I can use the custom error handling to handle the missing file but I'm not sure I like the way this is working
<httpErrors errorMode="Custom">
<remove statusCode="404"/>
<error statusCode="404" path="http://localhost/site/Error/NotFound" responseMode="Redirect"/>
</httpErrors>
Is there a better way to do this or is this the way that everyone else is doing it? How do I provide a 301 instead of a 404 since it's the static file handler that's serving the content?
Is there a way to get MVC to handle requests for certain file extensions and catch 404s from these if they don't exist?
I'm try to redirect a illegal access and bring user to a log-in page, if user get permission and continue to access original, I need to keeping original request url. I try to write original url into http header zone, but I cannot retrieve this data from client.
Did apache2 or other module ignore custom http heaer? or I just miss something?
(BTW: I dont like use querystring, think about maybe next page still come as a redirection)
code example:
ap_set_content_type(r, "text/html");
apr_table_add(r->headers_out, "Location", conf->authurl);
apr_table_add(r->headers_out, "RequestUrl", url);
return HTTP_MOVED_TEMPORARILY;
// following code will be work fine.
apr_table_add(r->err_headers_out, "RequestUrl", url);
see as:
https://source.jasig.org/cas-clients/mod_auth_cas/tags/mod_auth_cas-1.0.9.1/src/mod_auth_cas.c