Unexpected behavior difference between Nio and Nio2 connectors in Tomcat 8.5 - nio2

I've run across some peculiar behavior with the NIO2 connector in Tomcat 8.5.23
Here is how I am using it:
<Connector port="8443" protocol="org.apache.coyote.http11.Http11Nio2Protocol"
SSLEnabled="true"
sslImplementationName="org.apache.tomcat.util.net.openssl.OpenSSLImplementation"
maxThreads="200"
minSpareThreads="25"
scheme="https"
secure="true"
keystoreFile="/path/to/keystore.jks"
keystorePass="key-pass"
clientAuth="false"
ciphers="long,list,of,ciphers"/>
Now what happens is, if I happen to POST and the request includes the Header: 'Cache-Control: max-age=0' then the form parameters are not there when I try to retrieve them. I've tried putting in a filter to try and grab them and/or the body itself at the very beginning of the filter chain but it is already too late - the body has been consumed, and there are no parameters. The request structure does however show that the body is of the correct size, so it would appear not to be a network issue.
I would normally think I had something configured incorrectly, but everything works fine under Http11NioProtocol.
Any suggestions - aside from not using Nio2?

Related

Tomcat 8.5.29 HTTP/2 is not supporting GZIP compression

I am using Tomcat 8.5.29 and using the respective configuration,I have enabled the HTTP2 support for the site. Below is the configuration in server.xml file.
<Connector port="443" protocol="org.apache.coyote.http11.Http11AprProtocol"
maxThreads="150" SSLEnabled="true" compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json" compression="on" compressionMinSize="1024"
>
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
<SSLHostConfig>
<Certificate certificateKeyFile="conf/localhost-key.pem"
certificateFile="conf/localhost-cert.pem"
certificateChainFile="conf/cacert.pem"
type="RSA" />
</SSLHostConfig>
</Connector>
When i tried to compare the page load time for the site which is supporting HTTPS 1.1 and HTTP2, it is not consistent. Sometime it is taking more time to load and sometime it is taking less time to load compare to HTTPS 1.1.
To measure the page load time i am using Developer tools from Chrome Browser.
Later, I have found that Load time for HTTP/2 is higher because "content-encoding" in response is not gzip. Using HTTP/1.1, a script file size is 4 MB ( gzip) where as with HTTP/2, same file is of 8 MB ( no content-encoding). Compression is not happening. We have also tried with useSendfile and compression as "force" but it did not work.
We have also tried with Tomcat version 8.5.31 but no luck !! Facing same issue.
We have used Tomcat version 9 and using following configuration, got content in gzip (compressed). Now file size got reduced by 2 MB with a quick page load.
<Connector port="443" protocol="org.apache.coyote.http11.Http11AprProtocol"
maxThreads="150" SSLEnabled="true"
>
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json" compression="on" compressionMinSize="1024" />
<SSLHostConfig>
<Certificate certificateKeyFile="conf/localhost-key.pem"
certificateFile="conf/localhost-cert.pem"
certificateChainFile="conf/cacert.pem"
type="RSA" />
</SSLHostConfig>
</Connector>
It looks like Tomcat version 8.5 is not supporting gzip with HTTP2. Any idea on this?
According to the Tomcat documentation:
Note: There is a tradeoff between using compression (saving your bandwidth) and using the sendfile feature (saving your CPU cycles). If the connector supports the sendfile feature, e.g. the NIO2 connector, using sendfile will take precedence over compression. The symptoms will be that static files greater that 48 Kb will be sent uncompressed. You can turn off sendfile by setting useSendfile attribute of the protocol, as documented below, or change the sendfile usage threshold in the configuration of the DefaultServlet in the default conf/web.xml or in the web.xml of your web application.
So try adding useSendfile="false" to your UpgradeProtocol setting
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json" compression="on" compressionMinSize="1024" useSendfile="false" />
Note this option does not appear in the 8.5 documentation so not sure if it exists for that (there is a reference to it "You can turn off sendfile by setting useSendfile attribute of the protocol, as documented below" but then nothing below), though you can also turn off at a server level.
Weirdly I wouldn't have expected this to work in Tomcat 9 either without turning off this option, but you say it does? Are you sure you have not turned this off? Or are you testing with a smaller than 48KB file on that?
Tomcat 8.5.33 fixes this issue; compression works with http/2 then.

Multiple Tomcat instances, starting one kills the other

I have 2 different java applications running in two Tomcat instances (Ubuntu OS, Tomcat 7.0.57). For my Tomcat configuration, I follow instructions provided by this video, but as far as I saw, it is a very standard way to do it.
Running each application separately is working fine, but as soon as I try to run both at same time, the first started one becomes unavailable (HTTP 503 error). Tomcat instance logs do not provide any information about any kind of shutdown, keeping the last "INFO: Server startup in xxx ms". It seems the first tomcat process is simply killed. If I re-start that first application, then the same scenario applies to the second app.
All troubleshooting information I could find talk about port issues. I double checked my port numbers, they are different:
app-1: conf/server.xml:
<Server port="8105" shutdown="SHUTDOWN">
<Connector port="8180" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8143" />
<Connector port="8109" protocol="AJP/1.3" redirectPort="8143" />
...
</Server>
app-2: conf/server.xml:
<Server port="8205" shutdown="SHUTDOWN">
<Connector port="8280" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8243" />
<Connector port="8209" protocol="AJP/1.3" redirectPort="8243" />
...
</Server>
app1.sh
export CATALINA_HOME=/home/tomcat/apache-tomcat-7
export CATALINA_BASE=/home/tomcat/app-1
cd $CATALINA_HOME/bin
./startup.sh
app2.sh
export CATALINA_HOME=/home/tomcat/apache-tomcat-7
export CATALINA_BASE=/home/tomcat/app-2
cd $CATALINA_HOME/bin
./startup.sh
Any idea on what can happen, or how I can get any logs to dig this?
My server was hosted on a EC2 t1.micro instance, with 600MB memory.
I finally decide to update it to an instance with more memory and the problem disappear.

ServletRequest.getAttribute("javax.servlet.request.X509Certificate") returns null

I've got a stop-ship problem that is driving me crazy. I hope that one of you experts out there can help.
I'm running the latest release version of TomEE+ (1.6.0.2) and the latest version of Java 8 (build 1.8.0_05-b13). No matter what I try, the following line of code in my HttpServlet always returns null.
X509Certificate certs[] = (X509Certificate[])request.getAttribute("javax.servlet.request.X509Certificate");
I initially assumed that I had specified the attribute name incorrectly, so to diagnose the problem, I decided to take a look at the full list of attributes using the following:
Enumeration<String> enums = request.getAttributeNames();
However that showed me were only two attributes: one for the cipher suite and the other for the key strength.
I read the other articles and verified that my connector was correct and that it had the clientAuth attribute set properly. Here's the connector:
<Connector port="4449" protocol="org.apache.coyote.http11.Http11AprProtocol"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
clientAuth="true" sslProtocol="TLSv1.2"
SSLCertificateFile="/etc/unipagos/certs/pay.crt"
SSLCertificateKeyFile="/etc/unipagos/certs/pay.key"
SSLVerifyClient="required"
SSLHonorCipherOrder="true"
ciphers="TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"/>
The connection seems to work, however using openssl s_client with -msg shows that the server isn't asking for the client certificate.
Why is the server not asking for a client certificate? What am I doing wrong?
I have a working connector configuration for tomee and server is requesting client cert.
you can try
<Connector port="7443" protocol="org.apache.coyote.http11.Http11AprProtocol"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
SSLProtocol="TLSv1.2"
SSLVerifyClient="require"
SSLCertificateFile="/opt/_cdrom_apache/certs/ec-dev-apr.pem"
SSLCertificateKeyFile="/opt/_cdrom_apache/certs/ec-dev.key"
SSLCACertificateFile="/opt/_cdrom_apache/certs/CA.pem"
/>

WSO2 API Manager - Expose Publisher & Store URLs to public

I'm using WSO2 API Manager for creating and managing a developer community. According to my knowledge API manager is driven on WSO2 Carbon Server which again runs on Apache Tomcat.
Up to now I'm able to run WSO2 API manager without any issues. I could open up store and publisher using below urls.
https://<MyHostName>:9443/publisher
https://<MyHostName>:9443/store
What I want to know is, how can I expose these two URLs to public? I would like something like below as URLs(without ports).
https://<MyHostName>/publisher
https://<MyHostName>/store
or
https://publisher.<MyHostName>
https://store.<MyHostName>
Given that for URLs without https(just http) would be great if possible.
In WSO2 API Manager, How can I expose Publisher & Store URLs to public?
You can do this by editing catalina-server.xml file located in <APIM>/repository/conf/tomcat folder. Change the port and redirectPort values specified in NIO Connectors.
<Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
port="80"
redirectPort="443"
bindOnInit="false"
maxHttpHeaderSize="8192"
acceptorThreadCount="2"
maxThreads="250"
minSpareThreads="50"
disableUploadTimeout="false"
connectionUploadTimeout="120000"
maxKeepAliveRequests="200"
acceptCount="200"
server="WSO2 Carbon Server"
compression="on"
compressionMinSize="2048"
noCompressionUserAgents="gozilla, traviata"
compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
URIEncoding="UTF-8"/>
<Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
port="443"
bindOnInit="false"
sslProtocol="TLS"
maxHttpHeaderSize="8192"
acceptorThreadCount="2"
maxThreads="250"
minSpareThreads="50"
disableUploadTimeout="false"
enableLookups="false"
connectionUploadTimeout="120000"
maxKeepAliveRequests="200"
acceptCount="200"
server="WSO2 Carbon Server"
clientAuth="false"
compression="on"
scheme="https"
secure="true"
SSLEnabled="true"
compressionMinSize="2048"
noCompressionUserAgents="gozilla, traviata"
compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
URIEncoding="UTF-8"/>

How can I get client certificate authentication working in JBoss 5.1.0.GA when I'm using APR, and not all web deployments use CLIENT-CERT auth?

Note: I will be answering my own question... just wanted to add this tidbit to the collective wisdom of The Internets.
I've successfully configured certificate authentication on my JBoss 5.1.0.GA server, largely with the help of the information on this page: http://docs.jboss.org/jbossas/jboss4guide/r1/html/ch8.chapter.html
I have one context (let's call it /openContext) that doesn't require any authentication, and another context (let's call it /securedContext) that requires client certificate authentication (i.e., it's configured to use CLIENT-CERT in web.xml). When using JBoss's default web connector, this works splendidly. I can hit http://myhost/openContext and I'm not prompted for a certificate, but when I hit http://myhost/securedContext, I'm prompted for a client certificate as I'd expect.
However, when I install JBossWeb Native and use APR as my web connector, I'm no longer prompted for a certificate when I hit http://myhost/securedContext.
My APR connector config in server.xml looks like:
<Connector protocol="HTTP/1.1" SSLEnabled="true"
port="8443" address="${jboss.bind.address}"
scheme="https" secure="true" clientAuth="false"
SSLProtocol="SSLv3+TLSv1"
SSLCipherSuite="ALL:!ADH:!SSLv2:!EXPORT40:!EXP:!LOW"
SSLRandomSeed="/dev/urandom"
SSLCertificateFile="/etc/pki/tls/certs/mycert.crt"
SSLCertificateKeyFile="/etc/pki/tls/private/mycert.key"
SSLPassword="mypasswordwhichiassureyouisbetterthanthisone"
SSLCACertificateFile="/etc/pki/tls/certs/clientCAs.crt"
/>
I've also tried adding the SSLVerifyClient parameter to that configuration and setting it to optional, but that prompts for a certificate in both /openContext and /securedContext, which isn't the behavior I want.
How can I get JBoss with APR to require certificate authentication for one web context, but not another web context?
What worked for me was to just add a whole new web connector, and have clients use that alternate port for the secured web context. My connectors config now looks like:
<Connector protocol="HTTP/1.1" SSLEnabled="true"
port="8443" address="${jboss.bind.address}"
scheme="https" secure="true" clientAuth="false"
SSLProtocol="SSLv3+TLSv1"
SSLCipherSuite="ALL:!ADH:!SSLv2:!EXPORT40:!EXP:!LOW"
SSLRandomSeed="/dev/urandom"
SSLCertificateFile="/etc/pki/tls/certs/mycert.crt"
SSLCertificateKeyFile="/etc/pki/tls/private/mycert.key"
SSLPassword="mypasswordwhichiassureyouisbetterthanthisone"
/>
<Connector protocol="HTTP/1.1" SSLEnabled="true"
port="8543" address="${jboss.bind.address}"
scheme="https" secure="true" clientAuth="true"
SSLProtocol="SSLv3+TLSv1"
SSLCipherSuite="ALL:!ADH:!SSLv2:!EXPORT40:!EXP:!LOW"
SSLRandomSeed="/dev/urandom"
SSLCertificateFile="/etc/pki/tls/certs/mycert.crt"
SSLCertificateKeyFile="/etc/pki/tls/private/mycert.key"
SSLPassword="mypasswordwhichiassureyouisbetterthanthisone"
SSLCACertificateFile="/etc/pki/tls/certs/clientCAs.crt"
SSLVerifyClient="require"
/>
Now, if I hit http://myhost:8443/openContext, I'm not prompted for a certificate, but when I hit http://myhost:8543/securedContext, I am prompted for a certificate. Of course, I can still access either web app with the "wrong" port, but the consequences are negligible for my purposes. If a client hits http://myhost:8443/securedContext, they simply get an HTTP authentication error. If a client hits http://myhost:8543/openContext, they're prompted for a client certificate. If they provide one, great (though I don't care who you are), and if they don't provide one or provide an invalid one, they get an HTTP auth error (they should have used the correct port in the first place).
I'm pretty sure there's an alternative way to get this working without requiring a second connector by putting httpd in front of JBoss and doing some clever configuration there, but this worked well enough for my purposes.