Apache to Tomcat Cookie issue when using SSL termination through nginx - apache

Question

I'm testing our app on a kubernetes cluster. So we have Nginx Controller which handles SSL termination and passes HTTP traffic to Apache server. Apache server handles static content and forwards all JSP related to tomcat.
For some reason the webapp doesn't work on the first try (website works fine though) when doing SSL termination but if I reload the page and try to use the app again then during this second attempt everything works fine (means it doesn't load some of the automatic functions on first attempt and can be reproduced by clearing the cache and logging in).
I spoke to dev they mentioned it could be cookie issues.
Current setup which is not working:
Nginx controller (SSL termination) -> Apache (HTTP port 80 ) -> Tomcat (HTTP port 8080).
Setup which works fine:
Nginx controller (SSL passthrough) -> Apache (HTTPS port 443) -> Tomcat (HTTPS port 8080).
I can't get rid of Apache in between and it is really needed for the app temporarily.
What settings are required to make this work? I've tired the following:
Disable port 443 on apache
Disable 8443 ports and all redirects to port 8443 and listen only 8080
Modified web.xml to set http-only to true and secure bit to true on tomcat server.
<session-config>
<session-timeout>60</session-timeout>
<cookie-config>
<http-only>true</http-only>
<secure>true</secure>
</cookie-config>
</session-config>
Anything else that needs to be done? I've spent a day trying to troubleshooting this and couldn't figure it out yet.
Server.xml contains only these enabled lines, rest of them are commented out or defaults:
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000" />
<Connector port="8009" maxThreads="2000"
enableLookups="false" redirectPort="80" protocol="AJP/1.3"/>
<Engine name="Catalina" defaultHost="localhost" jvmRoute="server001">
# Removed cluster config since they're all default
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
<Context path="" docBase="/usr/local/tomcat/webapps/ROOT.war" debug="0" distributable="true">
</Context>
http.conf - has nothing but default since SSL is disabled. So no virtual host. Only thing that's added to http.conf is
JkMount /* ajp13
Worker.properties is the following.
# - An ajp13 worker that connects to localhost:8009
worker.list=ajp13
#------ DEFAULT ajp13 WORKER DEFINITION ------------------------------
# Defining a worker named ajp13 and of type ajp13
worker.ajp13.port=8009
worker.ajp13.host=$(TOMCAT_SERVER)
worker.ajp13.type=ajp13
worker.ajp13.reply_timeout=15000
worker.ajp13.lbfactor=1
#worker.ajp13.cachesize
worker.loadbalancer.balanced_workers=ajp13
The webapp needs to communicate with Java TCP server through websocket so we have a webSocket server written in nodejs. It just forwards traffic from websocket to Java server TCP connection.
But it has its own SSL certs setup. Since by default Nginx controller on GCP doesn't deal with ssl termination for TCP services, I have configured NodeJS backend service to accept SSL traffic directly on port 1234 for example. This service runs on the tomcat server. Don't know if this is creating a conflict since they all connect to same domain name.

Answer 1

Your problem probably arises, because the application does not understand that the request came from a secure channel.
Servlet API applications understand that a request was sent through a secure channel based on the result of ServletRequest#isSecure(). For requests that came through HTTP, this value depends on whether SSL was enabled or not.
When you use the AJP connector, this information and many more are trasmitted by the Apache server. This works perfectly well in the "SSL passthrough" configuration. However when the SSL connection terminates at NGINX you are in the situation described by the Reverse Proxy HOW-TO:
In some situations this is not enough though. Assume there is another less clever reverse proxy in front of your web server, for instance an HTTP load balancer or similar device which also serves as an SSL accelerator.
Then you are sure that all your clients use HTTPS, but your web server doesn't know about that. All it can see is requests coming from the accelerator using plain HTTP.
If you wanted to keep this configuration for a long time, I would suggest following the aforementioned HOW-TO. For short-term usage there is a simpler solution: you need to hardcode in Tomcat's configuration that all AJP requests are secure:
<Connector port="8009"
maxThreads="2000"
enableLookups="false"
protocol="AJP/1.3"
secure="true"
scheme="https"/>
The scheme attribute tells Tomcat which scheme was used by the original client, the connector will still use AJP.