Apparently, I have completely misunderstood its semantics. I thought of something like this:
A client downloads JavaScript code MyCode.js from http://siteA - the origin.
The response header of MyCode.js contains Access-Control-Allow-Origin: http://siteB, which I thought meant that MyCode.js was allowed to make cross-origin references to the site B.
The client triggers some functionality of MyCode.js, which in turn make requests to http://siteB, which should be fine, despite being cross-origin requests.
Well, I am wrong. It does not work like this at all. So, I have read Cross-origin resource sharing and attempted to read Cross-Origin Resource Sharing in w3c recommendation.
One thing is sure - I still do not understand how I am supposed to use this header.
I have full control of both site A and site B. How do I enable the JavaScript code downloaded from the site A to access resources on the site B using this header?
P.S.: I do not want to utilize JSONP.
Access-Control-Allow-Origin is a CORS (cross-origin resource sharing) header.
When Site A tries to fetch content from Site B, Site B can send an Access-Control-Allow-Origin response header to tell the browser that the content of this page is accessible to certain origins. (An origin is a domain, plus a scheme and port number.) By default, Site B's pages are not accessible to any other origin; using the Access-Control-Allow-Origin header opens a door for cross-origin access by specific requesting origins.
For each resource/page that Site B wants to make accessible to Site A, Site B should serve its pages with the response header:
Access-Control-Allow-Origin: http://siteA.com
Modern browsers will not block cross-domain requests outright. If Site A requests a page from Site B, the browser will actually fetch the requested page on the network level and check if the response headers list Site A as a permitted requester domain. If Site B has not indicated that Site A is allowed to access this page, the browser will trigger the XMLHttpRequest's error event and deny the response data to the requesting JavaScript code.
Non-simple requests
What happens on the network level can be slightly more complex than explained above. If the request is a "non-simple" request, the browser first sends a data-less "preflight" OPTIONS request, to verify that the server will accept the request. A request is non-simple when either (or both):
using an HTTP verb other than GET or POST (e.g. PUT, DELETE)
using non-simple request headers; the only simple requests headers are:
Accept
Accept-Language
Content-Language
Content-Type (this is only simple when its value is application/x-www-form-urlencoded, multipart/form-data, or text/plain)
If the server responds to the OPTIONS preflight with appropriate response headers (Access-Control-Allow-Headers for non-simple headers, Access-Control-Allow-Methods for non-simple verbs) that match the non-simple verb and/or non-simple headers, then the browser sends the actual request.
Supposing that Site A wants to send a PUT request for /somePage, with a non-simple Content-Type value of application/json, the browser would first send a preflight request:
OPTIONS /somePage HTTP/1.1
Origin: http://siteA.com
Access-Control-Request-Method: PUT
Access-Control-Request-Headers: Content-Type
Note that Access-Control-Request-Method and Access-Control-Request-Headers are added by the browser automatically; you do not need to add them. This OPTIONS preflight gets the successful response headers:
Access-Control-Allow-Origin: http://siteA.com
Access-Control-Allow-Methods: GET, POST, PUT
Access-Control-Allow-Headers: Content-Type
When sending the actual request (after preflight is done), the behavior is identical to how a simple request is handled. In other words, a non-simple request whose preflight is successful is treated the same as a simple request (i.e., the server must still send Access-Control-Allow-Origin again for the actual response).
The browsers sends the actual request:
PUT /somePage HTTP/1.1
Origin: http://siteA.com
Content-Type: application/json
{ "myRequestContent": "JSON is so great" }
And the server sends back an Access-Control-Allow-Origin, just as it would for a simple request:
Access-Control-Allow-Origin: http://siteA.com
See Understanding XMLHttpRequest over CORS for a little more information about non-simple requests.
Cross-Origin Resource Sharing - CORS (A.K.A. Cross-Domain AJAX request) is an issue that most web developers might encounter, according to Same-Origin-Policy, browsers restrict client JavaScript in a security sandbox, usually JS cannot directly communicate with a remote server from a different domain. In the past developers created many tricky ways to achieve Cross-Domain resource request, most commonly using ways are:
Use Flash/Silverlight or server side as a "proxy" to communicate
with remote.
JSON With Padding (JSONP).
Embeds remote server in an iframe and communicate through fragment or window.name, refer here.
Those tricky ways have more or less some issues, for example JSONP might result in security hole if developers simply "eval" it, and #3 above, although it works, both domains should build strict contract between each other, it neither flexible nor elegant IMHO:)
W3C had introduced Cross-Origin Resource Sharing (CORS) as a standard solution to provide a safe, flexible and a recommended standard way to solve this issue.
The Mechanism
From a high level we can simply deem CORS as a contract between client AJAX call from domain A and a page hosted on domain B, a typical Cross-Origin request/response would be:
DomainA AJAX request headers
Host DomainB.com
User-Agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0) Gecko/20100101 Firefox/4.0
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,application/json
Accept-Language en-us;
Accept-Encoding gzip, deflate
Keep-Alive 115
Origin http://DomainA.com
DomainB response headers
Cache-Control private
Content-Type application/json; charset=utf-8
Access-Control-Allow-Origin DomainA.com
Content-Length 87
Proxy-Connection Keep-Alive
Connection Keep-Alive
The blue parts I marked above were the kernal facts, "Origin" request header "indicates where the cross-origin request or preflight request originates from", the "Access-Control-Allow-Origin" response header indicates this page allows remote request from DomainA (if the value is * indicate allows remote requests from any domain).
As I mentioned above, W3 recommended browser to implement a "preflight request" before submiting the actually Cross-Origin HTTP request, in a nutshell it is an HTTP OPTIONS request:
OPTIONS DomainB.com/foo.aspx HTTP/1.1
If foo.aspx supports OPTIONS HTTP verb, it might return response like below:
HTTP/1.1 200 OK
Date: Wed, 01 Mar 2011 15:38:19 GMT
Access-Control-Allow-Origin: http://DomainA.com
Access-Control-Allow-Methods: POST, GET, OPTIONS, HEAD
Access-Control-Allow-Headers: X-Requested-With
Access-Control-Max-Age: 1728000
Connection: Keep-Alive
Content-Type: application/json
Only if the response contains "Access-Control-Allow-Origin" AND its value is "*" or contain the domain who submitted the CORS request, by satisfying this mandtory condition browser will submit the actual Cross-Domain request, and cache the result in "Preflight-Result-Cache".
I blogged about CORS three years ago: AJAX Cross-Origin HTTP request
According to this Mozilla Developer Network article,
A resource makes a cross-origin HTTP request when it requests a resource from a different domain, or port than the one which the first resource itself serves.
An HTML page served from http://domain-a.com makes an <img> src request for http://domain-b.com/image.jpg.
Many pages on the web today load resources like CSS style sheets, images and scripts from separate domains (thus it should be cool).
Same-Origin Policy
For security reasons, browsers restrict cross-origin HTTP requests initiated from within scripts.
For example, XMLHttpRequest and Fetch follow the same-origin policy.
So, a web application using XMLHttpRequest or Fetch could only make HTTP requests to its own domain.
Cross-Origin Resource Sharing (CORS)
To improve web applications, developers asked browser vendors to allow cross-domain requests.
The Cross-origin resource sharing (CORS) mechanism gives web servers cross-domain access controls, which enable secure cross-domain data transfers.
Modern browsers use CORS in an API container - such as XMLHttpRequest or fetch - to mitigate risks of cross-origin HTTP requests.
How CORS works (Access-Control-Allow-Origin header)
Wikipedia:
The CORS standard describes new HTTP headers which provide browsers and servers a way to request remote URLs only when they have permission.
Although some validation and authorization can be performed by the server, it is generally the browser's responsibility to support these headers and honor the restrictions they impose.
Example
The browser sends the OPTIONS request with an Origin HTTP header.
The value of this header is the domain that served the parent page. When a page from http://www.example.com attempts to access a user's data in service.example.com, the following request header would be sent to service.example.com:
Origin: http://www.example.com
The server at service.example.com may respond with:
An Access-Control-Allow-Origin (ACAO) header in its response indicating which origin sites are allowed.
For example:
Access-Control-Allow-Origin: http://www.example.com
An error page if the server does not allow the cross-origin request
An Access-Control-Allow-Origin (ACAO) header with a wildcard that allows all domains:
Access-Control-Allow-Origin: *
Whenever I start thinking about CORS, my intuition about which site hosts the headers is incorrect, just as you described in your question. For me, it helps to think about the purpose of the same-origin policy.
The purpose of the same-origin policy is to protect you from malicious JavaScript on siteA.com accessing private information you've chosen to share only with siteB.com. Without the same-origin policy, JavaScript written by the authors of siteA.com could have your browser make requests to siteB.com, using your authentication cookies for siteB.com. In this way, siteA.com could steal the secret information you share with siteB.com.
Sometimes you need to work cross domain, which is where CORS comes in. CORS relaxes the same-origin policy for siteB.com, using the Access-Control-Allow-Origin header to list other domains (siteA.com) that are trusted to run JavaScript that can interact with siteB.com.
To understand which domain should serve the CORS headers, consider this. You visit malicious.com, which contains some JavaScript that tries to make a cross domain request to mybank.com. It should be up to mybank.com, not malicious.com, to decide whether or not it sets CORS headers that relax the same-origin policy, allowing the JavaScript from malicious.com to interact with it. If malicous.com could set its own CORS headers allowing its own JavaScript access to mybank.com, this would completely nullify the same-origin policy.
I think the reason for my bad intuition is the point of view I have when developing a site. It's my site, with all my JavaScript. Therefore, it isn't doing anything malicious, and it should be up to me to specify which other sites my JavaScript can interact with. When in fact I should be thinking: Which other sites' JavaScript are trying to interact with my site and should I use CORS to allow them?
From my own experience, it's hard to find a simple explanation why CORS is even a concern.
Once you understand why it's there, the headers and discussion becomes a lot clearer. I'll give it a shot in a few lines.
It's all about cookies. Cookies are stored on a client by their domain.
An example story: On your computer, there's a cookie for yourbank.com. Maybe your session is in there.
Key point: When a client makes a request to the server, it will send the cookies stored under the domain for that request.
You're logged in on your browser to yourbank.com. You request to see all your accounts, and cookies are sent for yourbank.com. yourbank.com receives the pile of cookies and sends back its response (your accounts).
If another client makes a cross origin request to a server, those cookies are sent along, just as before. Ruh roh.
You browse to malicious.com. Malicious makes a bunch of requests to different banks, including yourbank.com.
Since the cookies are validated as expected, the server will authorize the response.
Those cookies get gathered up and sent along - and now, malicious.com has a response from yourbank.
Yikes.
So now, a few questions and answers become apparent:
"Why don't we just block the browser from doing that?" Yep. That's CORS.
"How do we get around it?" Have the server tell the request that CORS is OK.
1. A client downloads javascript code MyCode.js from http://siteA - the origin.
The code that does the downloading - your html script tag or xhr from javascript or whatever - came from, let's say, http://siteZ. And, when the browser requests MyCode.js, it sends an Origin: header saying "Origin: http://siteZ", because it can see that you're requesting to siteA and siteZ != siteA. (You cannot stop or interfere with this.)
2. The response header of MyCode.js contains Access-Control-Allow-Origin: http://siteB, which I thought meant that MyCode.js was allowed to make cross-origin references to the site B.
no. It means, Only siteB is allowed to do this request. So your request for MyCode.js from siteZ gets an error instead, and the browser typically gives you nothing. But if you make your server return A-C-A-O: siteZ instead, you'll get MyCode.js . Or if it sends '*', that'll work, that'll let everybody in. Or if the server always sends the string from the Origin: header... but... for security, if you're afraid of hackers, your server should only allow origins on a shortlist, that are allowed to make those requests.
Then, MyCode.js comes from siteA. When it makes requests to siteB, they are all cross-origin, the browser sends Origin: siteA, and siteB has to take the siteA, recognize it's on the short list of allowed requesters, and send back A-C-A-O: siteA. Only then will the browser let your script get the result of those requests.
Using React and Axios, join a proxy link to the URL and add a header as shown below:
https://cors-anywhere.herokuapp.com/ + Your API URL
Just adding the proxy link will work, but it can also throw an error for No Access again. Hence it is better to add a header as shown below.
axios.get(`https://cors-anywhere.herokuapp.com/[YOUR_API_URL]`,{headers: {'Access-Control-Allow-Origin': '*'}})
.then(response => console.log(response:data);
}
Warning: Not to be used in production
This is just a quick fix. If you're struggling with why you're not able to get a response, you can use this.
But again it's not the best answer for production.
If you are using PHP, try adding the following code at the beginning of the php file:
If you are using localhost, try this:
header("Access-Control-Allow-Origin: *");
If you are using external domains such as server, try this:
header("Access-Control-Allow-Origin: http://www.website.com");
I worked with Express.js 4, Node.js 7.4 and Angular, and I had the same problem. This helped me:
a) server side: in file app.js I add headers to all responses, like:
app.use(function(req, res, next) {
res.header('Access-Control-Allow-Origin', req.headers.origin);
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
This must be before all routes.
I saw a lot of added this headers:
res.header("Access-Control-Allow-Headers","*");
res.header('Access-Control-Allow-Credentials', true);
res.header('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE');
But I don’t need that,
b) client side: in sending by Ajax, you need to add "withCredentials: true," like:
$http({
method: 'POST',
url: 'url',
withCredentials: true,
data : {}
}).then(function(response){
// Code
}, function (response) {
// Code
});
If you want just to test a cross-domain application in which the browser blocks your request, then you can just open your browser in unsafe mode and test your application without changing your code and without making your code unsafe.
From macOS, you can do this from the terminal line:
open -a Google\ Chrome --args --disable-web-security --user-data-dir
In Python, I have been using the Flask-CORS library with great success. It makes dealing with CORS super easy and painless. I added some code from the library's documentation below.
Installing:
pip install -U flask-cors
Simple example that allows CORS for all domains on all routes:
from flask import Flask
from flask_cors import CORS
app = Flask(__name__)
CORS(app)
#app.route("/")
def helloWorld():
return "Hello, cross-origin-world!"
For more specific examples, see the documentation. I have used the simple example above to get around the CORS issue in an Ionic application I am building that has to access a separate flask server.
Simply paste the following code in your web.config file.
Noted that, you have to paste the following code under <system.webServer> tag
<httpProtocol>
<customHeaders>
<add name="Access-Control-Allow-Origin" value="*" />
<add name="Access-Control-Allow-Headers" value="Content-Type" />
<add name="Access-Control-Allow-Methods" value="GET, POST, PUT, DELETE, OPTIONS" />
</customHeaders>
</httpProtocol>
I can't configure it on the back-end server, but with these extensions in the browsers, it works for me:
For Firefox:
CORS Everywhere
For Google Chrome:
Allow CORS: Access-Control-Allow-Origin
Note: CORS works for me with this configuration:
For cross origin sharing, set header: 'Access-Control-Allow-Origin':'*';
Php: header('Access-Control-Allow-Origin':'*');
Node: app.use('Access-Control-Allow-Origin':'*');
This will allow to share content for different domain.
Nginx and Apache
As an addition to apsiller's answer, I would like to add a wiki graph which shows when a request is simple or not (and OPTIONS pre-flight request is send or not)
For a simple request (e.g., hotlinking images), you don't need to change your server configuration files, but you can add headers in the application (hosted on the server, e.g., in PHP) like Melvin Guerrero mentions in his answer - but remember: if you add full CORS headers in your server (configuration) and at same time you allow simple CORS in the application (e.g., PHP), this will not work at all.
And here are configurations for two popular servers:
turn on CORS on Nginx (nginx.conf file)
location ~ ^/index\.php(/|$) {
...
add_header 'Access-Control-Allow-Origin' "$http_origin" always; # if you change "$http_origin" to "*" you shoud get same result - allow all domain to CORS (but better change it to your particular domain)
add_header 'Access-Control-Allow-Credentials' 'true' always;
if ($request_method = OPTIONS) {
add_header 'Access-Control-Allow-Origin' "$http_origin"; # DO NOT remove THIS LINES (doubled with outside 'if' above)
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Max-Age' 1728000; # cache preflight value for 20 days
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS'; # arbitrary methods
add_header 'Access-Control-Allow-Headers' 'My-First-Header,My-Second-Header,Authorization,Content-Type,Accept,Origin'; # arbitrary headers
add_header 'Content-Length' 0;
add_header 'Content-Type' 'text/plain charset=UTF-8';
return 204;
}
}
turn on CORS on Apache (.htaccess file)
# ------------------------------------------------------------------------------
# | Cross-domain Ajax requests |
# ------------------------------------------------------------------------------
# Enable cross-origin Ajax requests.
# http://code.google.com/p/html5security/wiki/CrossOriginRequestSecurity
# http://enable-cors.org/
# change * (allow any domain) below to your domain
Header set Access-Control-Allow-Origin "*"
Header always set Access-Control-Allow-Methods "POST, GET, OPTIONS, DELETE, PUT"
Header always set Access-Control-Allow-Headers "My-First-Header,My-Second-Header,Authorization, content-type, csrf-token"
Header always set Access-Control-Allow-Credentials "true"
The Access-Control-Allow-Origin response header indicates whether the
response can be shared with requesting code from the given origin.
Header type Response header
-------------------------------------------
Forbidden header name no
A response that tells the browser to allow code from any origin to
access a resource will include the following:
Access-Control-Allow-Origin: *
For more information, visit Access-Control-Allow-Origin...
For .NET Core 3.1 API With Angular
Startup.cs : Add CORS
//SERVICES
public void ConfigureServices(IServiceCollection services){
//CORS (Cross Origin Resource Sharing)
//=====================================
services.AddCors();
}
//MIDDLEWARES
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
app.UseRouting();
//ORDER: CORS -> Authentication -> Authorization)
//CORS (Cross Origin Resource Sharing)
//=====================================
app.UseCors(x=>x.AllowAnyHeader().AllowAnyMethod().WithOrigins("http://localhost:4200"));
app.UseHttpsRedirection();
}
}
Controller : Enable CORS For Authorized Controller
//Authorize all methods inside this controller
[Authorize]
[EnableCors()]
public class UsersController : ControllerBase
{
//ActionMethods
}
Note: Only a temporary solution for testing
For those who can't control the backend for Options 405 Method Not Allowed, here is a workaround for theChrome browser.
Execute in the command line:
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --disable-web-security --user-data-dir="path_to_profile"
Example:
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --disable-web-security --user-data-dir="C:\Users\vital\AppData\Local\Google\Chrome\User Data\Profile 2"
Most CORS issues are because you are trying to request via client side ajax from a react, angular, jquery apps that are frontend basic libs.
You must request from a backend application.
You are trying to request from a frontend API, but the API you are trying to consume is expecting this request to be made from a backend application and it will never accept client side requests.
Using Laravel's resource routes, I've set up an API to serve as the back-end of a React JS application. I'm attempting to access the 'update' method currently. I'm using Javascript's fetch() to accomplish this, so its making one OPTIONS request first, then making the POST request (the form has a method spoof in it, setting _method to PATCH instead - this obviously doesn't affect the initial OPTIONS call). This same page is also making a GET request to the same endpoint via the same method, which works fine.
The fetch() call is below. Of course, this being React, it's called through a Redux Saga process, but the actual fetch is there.
function postApi(values, endpoint, token) { // <-- values and endpoint are sent by the component, token is sent by a previous Saga function
return fetch(apiUrl + endpoint, { // <-- apiUrl is defined as a constant earlier in the file
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer ' + token
},
body: JSON.stringify(
values
)
}).then(handleApiErrors)
.then(response => response.json())
.catch((error) => {throw error})
}
And the Laravel routes:
Route::group(['middleware' => 'auth:api'], function() {
Route::resource('users', 'UserController');
}
I was encountering an error where the initial OPTIONS request to the URL was returning a 404 error, which right away is strange, since the endpoint obviously exists, the exact same endpoint having just been queried seconds ago, but I assumed maybe Laravel was returning the wrong error, and I had used the wrong method. I did some digging and debugging trying to get the request to be correct before giving up and making the request in Postman. The thing is: it works fine in Postman.
Here are the response headers from the server (note that any access origin is permitted):
Access-Control-Allow-Origin:*
Cache-Control:no-cache, private
Connection:close
Content-Length:10
Content-Type:text/html; charset=UTF-8
Date:Thu, 21 Sep 2017 13:29:08 GMT
Server:Apache/2.4.27 (Unix) OpenSSL/1.0.2l PHP/7.0.22 mod_perl/2.0.8-dev Perl/v5.16.3
X-Powered-By:PHP/7.0.22
Here's the request headers for the request as made from the React JS app (the one that receives a 404 error):
Accept:*/*
Accept-Encoding:gzip, deflate, br
Accept-Language:en-US,en;q=0.8,fr;q=0.6,ga;q=0.4
Access-Control-Request-Headers:authorization,content-type
Access-Control-Request-Method:POST
Cache-Control:no-cache
Connection:keep-alive
Host:localhost
Origin:http://localhost:3000
Pragma:no-cache
Referer:http://localhost:3000/employees/edit/13
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
In Postman, I set up those exact same request headers and made the exact same OPTIONS request to the server. And it worked fine! I received an empty 200 response.
Just to be sure, I double-checked the Apache access log. And sure enough:
...
::1 - - [20/Sep/2017:15:33:24 -0400] "OPTIONS /the/path/to/api/users/13 HTTP/1.1" 200 -
::1 - - [20/Sep/2017:15:40:26 -0400] "OPTIONS /the/path/to/api/users/13 HTTP/1.1" 404 10
...
Request method the exact same, request URL the exact same, except one returned 200, the other returned 404, for no discernable reason.
Additionally, I should add that another POST request, to the create method, works just fine.
What could be causing this?
ATTEMPTED SOLUTIONS
1. I saw this question (React Native + fetch + API : DELETE request fails in App, works in Postman), and even though I'm running Apache, not Nginx, I thought I'd try adding a trailing slash to the request URL. The OPTIONS request now returns a 301 error (moved permanently).
2. I removed the trailing slash and continued trying to fix. Per comment suggestion, I removed the automatic route generation and created my own:
Route::get('/users', 'UserController#index');
Route::post('/users', 'UserController#create');
Route::put('/users/{user}', 'UserController#update');
Route::patch('/users/{user}', 'UserController#update');
Route::get('/users/{user}', 'UserController#show');
The Postman request still returns 200 OK, and the React request still returns 404 Not Found.
3. Eureka! Kind of. Per another comment suggestion, I exported the request from Chrome as cURL and imported it into Postman directly - maybe I missed something when copying the headers over. It seems I did, because now the Postman request also returns 404!
After playing around with disabling and/or enabling the imported headers, I've determined that the issue is the combination of the Origin and Access-Control-Request-Method headers. If only one is present the request returns 200, but if both are present I receive a 404.
This does still leave me with the question of how to fix the problem, however. At this point I wonder if the question might become more of a Laravel question - IE, why an OPTIONS request to a perfectly valid Resource route would return 404. I assume because those resources routes are listening for PUT or PATCH but not OPTIONS.
Since you have your CORS set up, all you need to do next is handle the 'preflight' OPTIONS request. You can do this using a middleware:
PreflightRequestMiddleware:
if ($request->getMethod() === $request::METHOD_OPTIONS) {
return response()->json([],204);
}
return $next($request);
Add the above code in the handle() method of the newly created middleware. Add the middleware in the global middleware stack.
Also, do not forget to add the OPTIONS method to Access-Control-Allow-Methods in your CORS setup.
For more options, check this out.
Answer:
Read this article. When the browser sends OPTIONS request to your application, the application has no way of handling it since you only defined a GET/POST/DELETE/PUT/PATCH route for the given endpoint.
So, in order for this route to work with preflight requests:
Route::get('/users', 'UserController#index');
it would need a corresponding OPTIONS route:
Route::options('/users', 'UserController#options');
Note: You would use a middleware to handle all OPTIONS requests in one place. If, however, you are using OPTIONS requests for other purposes - check the first link in this answer for more options.
We have an odd, intermittent error that occurs with mod_proxy_ajp, i.e. using apache as a front end to a tomcat server.
The error
User clicks on a link browser prompts
user to "save as...." (e.g. in
Firefox "You have chosen top open
thread.jsp which is a
application/octet-stream"...What
should firefox do with this file)
User says "Huh?" and presses "Cancel"
User clicks again on the same link
Browser displays the page correctly
This error occurs intermittently, but unfortunately rarely on our test server and frequently on production.
In firefox's LiveHttpHeaders I see the following in the above usecase:
first page download (i.e. click on link) is "text/plain"
second download is "text/html"
I thought the problem may stem from ProxyPassReverse (i.e. muddling up whether to use http or ajp), but all these proxypassreverse settings resulted in the same error:
ProxyPassReverse /ajp://localhost:8080/
ProxyPassReverse /pe http://localhost/pe
ProxyPassReverse /pe http://forumstest.company.com/pe
Additionally, I've checked the apache error logs (set to debug) and see no warnings or errors...
** But it works with mod_proxy_http ?? **
It appears that switching to mod_proxy_http 'solves' the problem. Limited testing, I have not been able to recreate the problem in the test environment.
Because the problem is intermittent, I'm not 100% sure that mod_proxy_http "solves" the problem
Environment
Apache 2.2 Windows
Jboss 4.2.2 back end (tomcat 6)
One other data point
For better or worse, a servlet filter in tomcat gzips the html before sending it to apache. (which means extra work as apache must unzip before it performs ProxyPassReverse's "find and replace"). I don't know if "gzip" messes up.
Questions
anyone seen this before?
what tools help analyze the cause?
thanks
Addendum 1: Here is the LiveHttpHeaders output
Browser Incorrectly sees html as "text/plain"
http://forums.customer.com/pe/action/forums/displaythread?rootPostID=10842016&channelID=1&portalPageId=1002
GET http://forums.customer.com/pe/action/forums/displaythread?rootPostID=10842016&channelID=1&portalPageId=1002 HTTP/1.1
Host: forums.customer.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive
Cookie: __utma=156962862.829309431.1260304144.1297956514.1297958674.234; __utmz=156962862.1296760237.232.50.utmcsr=forumstest.customer.com|utmccn=(referral)|utmcmd=referral|utmcct=/pe/action/forums/displaythread; s_vi=[CS]v1|258F5B88051D3FC3-40000105C056085F[CE]; inqVital=xd|0^sesMgr|{"sID":4,"lsts":1292598007}^incMgr|{"id":"755563420055418864","group":"CHAT","ltt":1292598006741,"sid":"755563549194447187","igds":"1290627502757","exempt":false}^inq|{"customerID":"755562378269271622"}^saleMgr|{"state":"UNSOLD","qDat":{},"sDat":{}}; inqState=sLnd|1^Lnd|{"c":4,"flt":1274728016,"lldt":17869990,"pgs":{"201198":{"c":1,"flt":1274728016,"lldt":0},"0":{"c":3,"flt":1274845009,"lldt":17752997}},"pq":["0","0","0","201198"],"fsld":1274728016697}; adv_search_results_page=10; ep_beta=1; visitorID=57307059; JSESSIONID=6jXLNdHRDjR9Th3B5gvTVkw1dZLn1zvhvKLR2r4GTLjylHJgjY3Q!683274050; __utmc=156962862; JSESSIONID=6jXLNdHRDjR9Th3B5gvTVkw1dZLn1zvhvKLR2r4GTLjylHJgjY3Q!683274050; TLTHID=5CCA50304DE99E28DB79A7B3267D4231; TLTSID=9DFCDE8045B374AAB752CC98A30E8311; AreCookiesEnabled=1; s_cc=true; SC_LINKS=%5B%5BB%5D%5D; s_sq=%5B%5BB%5D%5D; __utmb=156962862.64.10.1297958674; memberexists=T; ev1=greywolf%20hdtv%20whmx
Cache-Control: max-age=0
HTTP/1.0 200 OK
Date: Thu, 17 Feb 2011 17:38:42 GMT
Content-Type: text/plain
X-Cache: MISS from samus.company.com
X-Cache-Lookup: MISS from samus.company.com:3128
Via: 1.0 samus.company.com:3128 (squid/2.6.STABLE20)
Proxy-Connection: close
----------------------------------------------------------
Browser Correctly sees html as "text/html"
http://forums.customer.com/pe/action/forums/displaythread?rootPostID=10842016&channelID=1&portalPageId=1002
GET http://forums.customer.com/pe/action/forums/displaythread?rootPostID=10842016&channelID=1&portalPageId=1002 HTTP/1.1
Host: forums.customer.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive
Cookie: __utma=156962862.829309431.1260304144.1297956514.1297958674.234; __utmz=156962862.1296760237.232.50.utmcsr=forumstest.customer.com|utmccn=(referral)|utmcmd=referral|utmcct=/pe/action/forums/displaythread; s_vi=[CS]v1|258F5B88051D3FC3-40000105C056085F[CE]; inqVital=xd|0^sesMgr|{"sID":4,"lsts":1292598007}^incMgr|{"id":"755563420055418864","group":"CHAT","ltt":1292598006741,"sid":"755563549194447187","igds":"1290627502757","exempt":false}^inq|{"customerID":"755562378269271622"}^saleMgr|{"state":"UNSOLD","qDat":{},"sDat":{}}; inqState=sLnd|1^Lnd|{"c":4,"flt":1274728016,"lldt":17869990,"pgs":{"201198":{"c":1,"flt":1274728016,"lldt":0},"0":{"c":3,"flt":1274845009,"lldt":17752997}},"pq":["0","0","0","201198"],"fsld":1274728016697}; adv_search_results_page=10; ep_beta=1; visitorID=57307059; JSESSIONID=6jXLNdHRDjR9Th3B5gvTVkw1dZLn1zvhvKLR2r4GTLjylHJgjY3Q!683274050; __utmc=156962862; JSESSIONID=6jXLNdHRDjR9Th3B5gvTVkw1dZLn1zvhvKLR2r4GTLjylHJgjY3Q!683274050; TLTHID=5CCA50304DE99E28DB79A7B3267D4231; TLTSID=9DFCDE8045B374AAB752CC98A30E8311; AreCookiesEnabled=1; s_cc=true; SC_LINKS=%5B%5BB%5D%5D; s_sq=%5B%5BB%5D%5D; __utmb=156962862.64.10.1297958674; memberexists=T; ev1=greywolf%20hdtv%20whmx
Cache-Control: max-age=0
HTTP/1.0 200 OK
Date: Thu, 17 Feb 2011 17:38:44 GMT
X-Powered-By: Servlet 2.4; JBoss-4.2.1.GA (build: SVNTag=JBoss_4_2_1_GA date=200707131605)/Tomcat-5.5
Content-Encoding: gzip
Content-Type: text/html;charset=UTF-8
Content-Length: 24739
X-Cache: MISS from samus.company.com
X-Cache-Lookup: MISS from samus.company.com:3128
Via: 1.0 samus.company.com:3128 (squid/2.6.STABLE20)
Proxy-Connection: keep-alive
----------------------------------------------------------
Addendum 2: Additional Information
The browser did receive the "gzipped" file. I had earlier clicked "save as..." when a few of these errors occurred. Gunzip successfully processed the files and converted them to html.
Answer is here: How to preserve the Content-Type header of a Tomcat HTTP response sent through an AJP connector to Apache using mod_proxy
set DefaultType to None in apache configuration.
# DefaultType: the default MIME type the server will use for a document
# if it cannot otherwise determine one, such as from filename extensions.
# If your server contains mostly text or HTML documents, "text/plain" is
# a good value. If most of your content is binary, such as applications
# or images, you may want to use "application/octet-stream" instead to
# keep browsers from trying to display binary files as though they are
# text.
#
DefaultType None
The first page download is of mime type "application/octet-stream". May be the zipped stream is being sent back to the browser? (You can confirm by saving the file and looking inside it.)
It would be really helpful to post the Live HTTP Header request / response traces for problematic and normal cases. If the content is same in both cases - the response HTML - may be forcing the mime type to be text/html (using ForceType directive) for that particular context can fix the issue.
If it turns out to be the case that gzipped content is being sent to the browser in the problematic case - then digging deeper would be necessary. Is this browser specific - only happens with Firefox or happens with all browsers?
Ok, based on the new information provided - looks like Squid is caching the problematic response and not sending the right headers back to the client. So the browsers are doing the right thing here - without the Content-Encoding and right Content-Type they cannot do much else.
Missing response headers for the problematic request :
X-Powered-By: Servlet 2.4; JBoss-4.2.1.GA (build: SVNTag=JBoss_4_2_1_GA date=200707131605)/Tomcat-5.5
Content-Encoding: gzip
Content-Type: text/html;charset=UTF-8
X-Powered-By : It sounds like the problematic requests don't actually hit your JBoss server. Can you verify this by checking access logs? Is Squid caching the response and then sending it back as text/plain?
Reconfiguring Squid to not cache that particular URL should help - see http://www.lirmm.fr/doc/Doc/FAQ/FAQ-7.html section 7.8 for example (which is specific to Squid 2 but later versions should have similar capabilities.)
Seems like a content-negotiation problem. Apache is guessing the content type using the "magic" byte and setting the content type incorrectly. That explains why it happens intermittently. Try disabling mod_negotiation and see what happens. See http://httpd.apache.org/docs/2.0/content-negotiation.html for more info.
I saw the following in your settings
ProxyPassReverse /ajp://localhost:8080/
But port 8080 is not ajp port. The default ajp port is 8009. Could this be your problem?
There is most likely something wrong with your web application, not Apache. If your web app sends back the correct Content-Type, Apache will gladly forward it to the client. No content negotiation will be done in that case. If you do not return any Content-Type, Apache will almost surely substitute text/plain, which is not what you want.
Test your web app without Apache in the middle, make sure that it sends back the correct Content-Type.
It uses to be when apache serves secure content in a non secure channel.