authenticate against sonatype ossindex with maven plugin - sonatype

I am using the mvn org.sonatype.ossindex.maven:ossindex-maven-plugin from command line to check the dependencies of a maven project for CVEs. Locally, everything works fine, but in my build pipeline in Azure Devops, I get the following error:
[DEBUG] Connecting to ossindex.sonatype.org/18.118.116.156:443
[DEBUG] Connecting socket to ossindex.sonatype.org/18.118.116.156:443 with timeout 0
[DEBUG] Enabled protocols: [TLSv1.3, TLSv1.2]
[DEBUG] Enabled cipher suites:[TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
[DEBUG] Starting handshake
[DEBUG] Secure session established
[DEBUG] negotiated protocol: TLSv1.2
[DEBUG] negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
[DEBUG] peer principal: CN=ossindex.sonatype.org
[DEBUG] peer alternative names: [ossindex.sonatype.org]
[DEBUG] issuer principal: CN=Amazon, OU=Server CA 1B, O=Amazon, C=US
[DEBUG] Connection established 172.21.1.242:33030<->18.118.116.156:443
[DEBUG] Executing request POST /api/v3/component-report HTTP/1.1
[DEBUG] Proxy auth state: UNCHALLENGED
[DEBUG] http-outgoing-2 >> POST /api/v3/component-report HTTP/1.1
[DEBUG] http-outgoing-2 >> User-Agent: ossindex-client/1.1.1 (Linux; 5.4.0-124-generic; amd64; 11.0.12) Maven/3.8.1 Maven-Plugin/3.1.0
[DEBUG] http-outgoing-2 >> Authorization: Basic {SOME_BASE_64_THAT_TRANSLATES_TO:AzureDevOps:ey...}
[DEBUG] http-outgoing-2 >> Accept: application/vnd.ossindex.component-report.v1+json
[DEBUG] http-outgoing-2 >> Content-Length: 6737
[DEBUG] http-outgoing-2 >> Content-Type: application/vnd.ossindex.component-report-request.v1+json; charset=UTF-8
[DEBUG] http-outgoing-2 >> Host: ossindex.sonatype.org
[DEBUG] http-outgoing-2 >> Connection: Keep-Alive
[DEBUG] http-outgoing-2 >> Accept-Encoding: gzip,deflate
[DEBUG] http-outgoing-2 >> "POST /api/v3/component-report HTTP/1.1[\r][\n]"
[DEBUG] http-outgoing-2 >> "User-Agent: ossindex-client/1.1.1 (Linux; 5.4.0-124-generic; amd64; 11.0.12) Maven/3.8.1 Maven-Plugin/3.1.0[\r][\n]"
[DEBUG] http-outgoing-2 >> "Authorization: Basic {SOME_OTHER_BASE64_NO_IDEA_WHERE_IT_COMES_FROM=[\r][\n]"
[DEBUG] http-outgoing-2 >> "Accept: application/vnd.ossindex.component-report.v1+json[\r][\n]"
[DEBUG] http-outgoing-2 >> "Content-Length: 6737[\r][\n]"
[DEBUG] http-outgoing-2 >> "Content-Type: application/vnd.ossindex.component-report-request.v1+json; charset=UTF-8[\r][\n]"
[DEBUG] http-outgoing-2 >> "Host: ossindex.sonatype.org[\r][\n]"
[DEBUG] http-outgoing-2 >> "Connection: Keep-Alive[\r][\n]"
[DEBUG] http-outgoing-2 >> "Accept-Encoding: gzip,deflate[\r][\n]"
[DEBUG] http-outgoing-2 >> "[\r][\n]"
[DEBUG] http-outgoing-2 >> "{"coordinates":[ A_LIST_OF_128_COORDINATES]
[DEBUG] http-outgoing-2 << "HTTP/1.1 401 Unauthorized[\r][\n]"
[DEBUG] http-outgoing-2 << "Date: Wed, 07 Dec 2022 13:51:42 GMT[\r][\n]"
[DEBUG] http-outgoing-2 << "Content-Length: 0[\r][\n]"
[DEBUG] http-outgoing-2 << "Connection: keep-alive[\r][\n]"
so I tried setting the clientConfiguration parameter described here.
it has a dead link but I believe it references this class which has this class as a member.
mvn org.sonatype.ossindex.maven:ossindex-maven-plugin:3.1.0:audit -f $(Build.SourcesDirectory)/pom.xml -"Dossindex.authId=MY_SERVER" -"Dossindex.clientConfiguration={\"auth\": {\"username\": \"myemail#myorg.com\", \"password\": \"$(OSS_INDEX_API_TOKEN)\"} }"
but it didn't change anything.
Is this how one passes this parameter? Why am I getting a 401 in my build pipeline but everything works locally? I tried reading the docs on what requests are allowed/authorized but found them rather lacking...

Related

Error 496 when client requests service-worker.js (only) on nginx with M-TLS

I deployed a PWA app with my nginx server and I have no problems connecting with my laptop using chrome or safari. I only get an issue when I re-connect with my iPhone and with a particular file only: sw.js (service worker).
Since it is fundamental for a PWA to get this file downloaded in order to decide whether a new version of the app is available or not, having to clear the cache from safari iOS in order to get it done is really annoying.
ok so, let me explain:
The app is hosted on a nginx server with TLS + Mutual TSL.
Each client I'm connecting from has been configured with both the certs and works fine, so I guess this is not a cert problem (neither from the TLS nor from the M-TLS).
If I connect from my laptop with chrome or safari, I have no problems at all.
2023/01/16 08:54:18 [debug] 19158#19158: *4 http process request line
2023/01/16 08:54:18 [debug] 19158#19158: *4 http request line: "GET /sw.js HTTP/1.1"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http uri: "/sw.js"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http args: ""
2023/01/16 08:54:18 [debug] 19158#19158: *4 http exten: "js"
2023/01/16 08:54:18 [debug] 19158#19158: *4 posix_memalign: 00005566D933EEA0:4096 #16
2023/01/16 08:54:18 [debug] 19158#19158: *4 http process request header line
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Host: xx.xx.net:4765"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Connection: keep-alive"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Cache-Control: max-age=0"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Accept: */*"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Service-Worker: script"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Sec-Fetch-Site: same-origin"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Sec-Fetch-Mode: same-origin"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Sec-Fetch-Dest: serviceworker"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Referer: https://xx.xx.net:4765/sw.js"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Accept-Encoding: gzip, deflate, br"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "Accept-Language: it-IT,it;q=0.9,en-US;q=0.8,en;q=0.7"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "If-None-Match: "63c47f4b-2945""
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header: "If-Modified-Since: Sun, 15 Jan 2023 22:33:47 GMT"
2023/01/16 08:54:18 [debug] 19158#19158: *4 http header done
if I connect from my iPhone with safari mobile:
1) where connecting the first time with cleared cache: no issues
23/01/16 09:31:55 [debug] 19156#19156: *42 http process request line
2023/01/16 09:31:55 [debug] 19156#19156: *42 http request line: "GET /sw.js HTTP/1.1"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http uri: "/sw.js"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http args: ""
2023/01/16 09:31:55 [debug] 19156#19156: *42 http exten: "js"
2023/01/16 09:31:55 [debug] 19156#19156: *42 posix_memalign: 00005566D93FF330:4096 #16
2023/01/16 09:31:55 [debug] 19156#19156: *42 http process request header line
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header: "Host: xx.xx.net:4765"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header: "Cache-Control: max-age=0"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header: "Accept-Encoding: gzip, deflate, br"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header: "Connection: keep-alive"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header: "Accept: */*"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header: "User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 16_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Mobile/15E148 Safari/604.1"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header: "Accept-Language: it-IT,it;q=0.9"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header: "Referer: https://xx.xx.net:4765/index.html"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header: "Service-Worker: script"
2023/01/16 09:31:55 [debug] 19156#19156: *42 http header done
2023/01/16 09:31:55 [debug] 19156#19156: *42 http filename: "/var/www/html/sw.js"
2023/01/16 09:31:55 [debug] 19156#19156: *42 add cleanup: 00005566D93FF728
2023/01/16 09:31:55 [debug] 19156#19156: *42 http static fd: 23
2023/01/16 09:31:55 [debug] 19156#19156: *42 http set discard body
2023/01/16 09:31:55 [debug] 19156#19156: *42 xslt filter header
2023/01/16 09:31:55 [debug] 19156#19156: *42 **HTTP/1.1 200 OK**
2) where closing the app and then resuming it: I get a 496 on a specific file only: the sw.js (service worker) --> client sent no required SSL certificate while reading client request headers
I can't understand why but it definitely prevents my PWA from being updated.
2023/01/16 09:15:39 [debug] 19156#19156: *38 http request line: "GET /sw.js HTTP/1.1"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http uri: "/sw.js"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http args: ""
2023/01/16 09:15:39 [debug] 19156#19156: *38 http exten: "js"
2023/01/16 09:15:39 [debug] 19156#19156: *38 posix_memalign: 00005566D94111B0:4096 #16
2023/01/16 09:15:39 [debug] 19156#19156: *38 http process request header line
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header: "Host: xx.xx.net:4765"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header: "Origin: https://xx.xx.net:4765"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header: "Accept-Encoding: gzip, deflate, br"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header: "Connection: keep-alive"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header: "Accept: */*"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header: "User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 16_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Mobile/15E148 Safari/604.1"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header: "Accept-Language: it-IT,it;q=0.9"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header: "Referer: "
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header: "Service-Worker: script"
2023/01/16 09:15:39 [debug] 19156#19156: *38 http header done
2023/01/16 09:15:39 [info] 19156#19156: *38 **client sent no required SSL certificate while reading client request headers, client: 76.87.343.434, server: xx.xx.net, request: "GET /sw.js HTTP/1.1", host: "xx.xx.net:4765", referrer: ""**
2023/01/16 09:15:39 [debug] 19156#19156: *38 http finalize request: 496, "/sw.js?" a:1, c:1
2023/01/16 09:15:39 [debug] 19156#19156: *38 event timer del: 20: 202080132
2023/01/16 09:15:39 [debug] 19156#19156: *38 http special response: 496, "/sw.js?"
2023/01/16 09:15:39 [debug] 19156#19156: *38 internal redirect: "/custom_404.html?"
Here are my config files:
nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
types_hash_max_size 2048;
server_tokens off;
client_body_buffer_size 1k;
client_header_buffer_size 1k;
client_max_body_size 1k;
large_client_header_buffers 2 1k;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log debug;
##
# Gzip Settings
##
gzip on;
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
site-available file:
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server {
listen 80;
return 301 https://xx.xx.net:port$request_uri;
}
server {
listen 4765 ssl;
server_name xx.xx.net;
# SSL
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
# TLS
ssl_certificate /etc/ssh/TLS/cert.crt;
ssl_certificate_key /etc/ssh/TLS/key.key;
# M-TLS
ssl_client_certificate /etc/ssh/mutual-tls.crt;
ssl_verify_client on;
ssl_verify_depth 2;
# ERRORS
error_page 400 404 495 496 497 /custom_404.html;
location = /custom_404.html {
root /usr/share/nginx/html;
internal;
}
location / {
# Simple requests
if ($request_method ~* "(GET|POST)") {
add_header "Access-Control-Allow-Origin" '*';
add_header "Access-Control-Allow-Headers" "Authorization, Origin, X-Requested-With, Content-Type, Accept";
}
# Preflighted requests
if ($request_method = OPTIONS ) {
add_header "Access-Control-Allow-Origin" '*';
add_header "Access-Control-Allow-Methods" "GET, POST, OPTIONS, HEAD";
add_header "Access-Control-Allow-Headers" "Authorization, Origin, X-Requested-With, Content-Type, Accept";
}
}
location ~* \.(html|css|js)$ {
expires -1;
}
}
Strange thing: if I request the sw.js from my browser (typing https://xx.xx.net:sslport/sw.js), I get the file with no errors.
Do you have an idea on what's going on here? maybe is a header /response-header issue?
Update:
I've successfully tried to send the request via Postman: here is the result.
Again: it only happens with iOs, I really don't know how to solve it.

nginx stream_ssl_preread module unable to read ssl_preread_server_name

I am trying to set up nginx to map TLS connections to different backends based on the SNI server name. From what I can tell, my client is sending the server name, but the preread module is only reading a hyphen.
Here is my nginx congif:
stream {
map_hash_bucket_size 64;
############################################################
### logging
log_format log_stream '$remote_addr [$time_local] $protocol [$ssl_preread_server_name] [$ssl_preread_alpn_protocols] [$instanceport] '
'$status $bytes_sent $bytes_received $session_time';
error_log /usr/home/glance/Logs/pservernginx.error.log info;
access_log /usr/home/glance/Logs/pservernginx.access.log log_stream;
############################################################
### ssl configuration
ssl_certificate /usr/home/glance/GlanceReleases/star.myglance.org.pem;
ssl_certificate_key /usr/home/glance/GlanceReleases/star.myglance.org.pem;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5:!RC4;
limit_conn_zone $binary_remote_addr zone=ip_addr:10m;
########################################################################
### Raw TLS PServer Connections
### Listen for TLS on 5501 and forward to TCP sock 6500 (socket port)
### https://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html
map $ssl_preread_server_name $instanceport {
presence.myglance.org 6500;
presence-1.myglance.org 6501;
presence-2.myglance.org 6502;
default glance-no-upstream-instance-configured;
}
server {
listen 5501 ssl;
ssl_preread on;
proxy_connect_timeout 20s; # max time to connect to pserver
proxy_timeout 30s; # max time between successive reads or writes
proxy_pass 127.0.0.1:$instanceport;
}
}
wireshark shows the Server Name header:
The nginx access log shows only hyphens for the preread variables:
108.49.96.66 [12/Apr/2019:11:50:58 +0000] TCP [-] [-] [glance-no-upstream-instance-configured] 500 0 0 0.066
I'm running nginx 1.14.2 on FreeBSD. How can I debug what is happening in the preread module?
================ UPDATE ===============
Turned on debug logging. Maybe "ssl preread: not a handshake" is a clue.
2019/04/12 14:49:50 [info] 61420#0: *9 client 108.49.96.66:54740 connected to 0.0.0.0:5501
2019/04/12 14:49:50 [debug] 61420#0: *9 posix_memalign: 0000000801C35000:256 #16
2019/04/12 14:49:50 [debug] 61419#0: accept on 0.0.0.0:5501, ready: 1
2019/04/12 14:49:50 [debug] 61419#0: accept() not ready (35: Resource temporarily unavailable)
2019/04/12 14:49:50 [debug] 61420#0: *9 posix_memalign: 0000000801C35600:256 #16
2019/04/12 14:49:50 [debug] 61420#0: *9 generic phase: 0
2019/04/12 14:49:50 [debug] 61420#0: *9 generic phase: 1
2019/04/12 14:49:50 [debug] 61420#0: *9 generic phase: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 tcp_nodelay
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_do_handshake: -1
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_get_error: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 kevent set event: 5: ft:-1 fl:0025
2019/04/12 14:49:50 [debug] 61420#0: *9 event timer add: 5: 60000:29203481224
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL handshake handler: 0
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_do_handshake: 1
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD"
2019/04/12 14:49:50 [debug] 61420#0: *9 event timer del: 5: 29203481224
2019/04/12 14:49:50 [debug] 61420#0: *9 generic phase: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 ssl preread handler
2019/04/12 14:49:50 [debug] 61420#0: *9 malloc: 0000000801CFF000:16384
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_read: -1
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_get_error: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 ssl preread handler
2019/04/12 14:49:50 [debug] 61420#0: *9 posix_memalign: 0000000801C35900:256 #16
2019/04/12 14:49:50 [debug] 61420#0: *9 event timer add: 5: 30000:29203451252
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_read: 81
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_read: -1
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_get_error: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 ssl preread handler
2019/04/12 14:49:50 [debug] 61420#0: *9 ssl preread: not a handshake
2019/04/12 14:49:50 [debug] 61420#0: *9 event timer del: 5: 29203451252
2019/04/12 14:49:50 [debug] 61420#0: *9 proxy connection handler
2019/04/12 14:49:50 [debug] 61420#0: *9 malloc: 0000000801DF7000:400
2019/04/12 14:49:50 [debug] 61420#0: *9 malloc: 0000000801CD9000:16384
2019/04/12 14:49:50 [debug] 61420#0: *9 stream map started
2019/04/12 14:49:50 [debug] 61420#0: *9 stream map: "" "glance-no-upstream-instance-configured"
================= UPDATE 2 ======================
I tested using
openssl s_client -connect ... -servername ...
instead of my client. Now it appears that the preread module is blocked waiting for data for 30 seconds (error code 2 is WANT_READ):
2019/04/23 13:04:30 [debug] 61419#0: *12844 SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD"
2019/04/23 13:04:30 [debug] 61419#0: *12844 event timer del: 3: 30147561850
2019/04/23 13:04:30 [debug] 61419#0: *12844 generic phase: 2
2019/04/23 13:04:30 [debug] 61419#0: *12844 ssl preread handler
2019/04/23 13:04:30 [debug] 61419#0: *12844 malloc: 0000000801CA6140:16384
2019/04/23 13:04:30 [debug] 61419#0: *12844 SSL_read: -1
2019/04/23 13:04:30 [debug] 61419#0: *12844 SSL_get_error: 2
2019/04/23 13:04:30 [debug] 61419#0: *12844 ssl preread handler
2019/04/23 13:04:30 [debug] 61419#0: *12844 posix_memalign: 0000000801DB3400:256 #16
2019/04/23 13:04:30 [debug] 61419#0: *12844 event timer add: 3: 30000:30147531898
2019/04/23 13:05:00 [debug] 61419#0: *12844 event timer del: 3: 30147531898
2019/04/23 13:05:00 [debug] 61419#0: *12844 finalize stream session: 200
2019/04/23 13:05:00 [debug] 61419#0: *12844 stream log handler
2019/04/23 13:05:00 [debug] 61419#0: *12844 stream map started
2019/04/23 13:05:00 [debug] 61419#0: *12844 stream script var: ""
I found the problem:
listen 5501 **ssl**;
ssl_preread on;
ssl in the listen directive caused that nginx server to do the ssl handshake. By the time the preread module was notified, the handshake bytes had already been consumed, which is all consistent with the behavior I was seeing. In my case, I still want nginx to offload the encryption. So I created a set of nginx server directives to terminate the ssl connection before passing to my back end.
This is the relevant portion of my nginx config after fixing it. Note that the last server directive (the one that uses ssl_preread) does not terminate the SSL connection.
########################################################################
### TLS Connections
### Listen for TLS on 5501 and forward to TCP sock 6500 (socket port)
### https://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html
map $ssl_preread_server_name $instanceport {
presence.myglance.org 5502;
presence-1.myglance.org 5503;
presence-2.myglance.org 5504;
default glance-no-upstream-instance-configured;
}
server {
listen 5502 ssl;
ssl_preread off;
proxy_pass 127.0.0.1:6502;
}
server {
listen 5503 ssl;
ssl_preread off;
proxy_pass 127.0.0.1:6503;
}
server {
listen 5504 ssl;
ssl_preread off;
proxy_pass 127.0.0.1:6504;
}
server {
listen 5501;
ssl_preread on;
proxy_connect_timeout 20s; # max time to connect to pserver
proxy_timeout 30s; # max time between successive reads or writes
proxy_pass 127.0.0.1:$instanceport;
}
In case you need to use ssl in listen directive, you can simply use $ssl_server_name in the map block instead of $ssl_preread_server_name

Ingress NGINX client closed connection while SSL handshaking

we have ingress-nginx running for a while and about 10% of requests ending up with some SSL handshake problem.
Here is an example of a failing connection:
2019/02/14 10:15:35 [debug] 237#237: *4612 accept: **.**.**.**:40928 fd:53
2019/02/14 10:15:35 [debug] 237#237: *4612 event timer add: 53: 60000:5527050245
2019/02/14 10:15:35 [debug] 237#237: *4612 reusable connection: 1
2019/02/14 10:15:35 [debug] 237#237: *4612 epoll add event: fd:53 op:1 ev:80002001
2019/02/14 10:15:45 [debug] 237#237: *4612 http check ssl handshake
2019/02/14 10:15:45 [debug] 237#237: *4612 http recv(): 0
2019/02/14 10:15:45 [info] 237#237: *4612 client closed connection while SSL handshaking, client: **.**.**.**, server: 0.0.0.0:443
2019/02/14 10:15:45 [debug] 237#237: *4612 close http connection: 53
2019/02/14 10:15:45 [debug] 237#237: *4612 event timer del: 53: 5527050245
2019/02/14 10:15:45 [debug] 237#237: *4612 reusable connection: 0
2019/02/14 10:15:45 [debug] 237#237: *4612 free: 00007F4CC5858E00, unused: 232
10% of failures seems to be quite a lot to expect.
I really would appreciate any help in this!

getting 400 bad request error when nginx reverse proxy is configured with SSL.

I have configured Nginx reverse proxy server to listen on port 443 and pass the requests to an upstream SAAS client. Below is the configuration.
server {
listen 443;
server_name test.saas.someloggingserver.com;
ssl on;
ssl_certificate C:/nginx-1.13.8/nginx-1.13.8/ssl/server_cert.crt;
ssl_certificate_key C:/nginx-1.13.8/nginx-1.13.8/ssl/server.key;
ssl_session_timeout 5m;
ssl_protocols SSLv2 SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
ssl_prefer_server_ciphers on;
expires 0;
add_header Cache-Control private;
access_log logs/encrypted_access.txt;
error_log logs/encrypted_error.txt debug;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect http:// https://;
proxy_pass https://test.saas.someloggingserver.com;
}
}
The SSL certificate here also includes a intermediate certificate for SSL handshaking with the upstream server. Now when I try to access the https://test.saas.someloggingserver.com from IE (having above proxy configured) I get below 400 bad request error. I am no Nginx debugging expert, appreciate if you can take a look at below logs and direct me towards the possible cause of this issue?
2017/12/28 07:04:45 [debug] 14640#9400: post event 02D76250
2017/12/28 07:04:45 [debug] 14640#9400: delete posted event 02D76250
2017/12/28 07:04:45 [debug] 14640#9400: accept on 0.0.0.0:443, ready: 0
2017/12/28 07:04:45 [debug] 14640#9400: malloc: 02D50808:256
2017/12/28 07:04:45 [debug] 14640#9400: *3695 accept: 10.92.67.192:49268 fd:496
2017/12/28 07:04:45 [debug] 14640#9400: *3695 event timer add: 496: 60000:2616257381
2017/12/28 07:04:45 [debug] 14640#9400: *3695 reusable connection: 1
2017/12/28 07:04:45 [debug] 14640#9400: *3695 select add event fd:496 ev:0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 post event 02D76340
2017/12/28 07:04:45 [debug] 14640#9400: *3695 delete posted event 02D76340
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http check ssl handshake
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http recv(): 1
2017/12/28 07:04:45 [debug] 14640#9400: *3695 plain http
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http wait request handler
2017/12/28 07:04:45 [debug] 14640#9400: *3695 malloc: 02D4CF80:1024
2017/12/28 07:04:45 [debug] 14640#9400: *3695 WSARecv: fd:496 rc:0 266 of 1024
2017/12/28 07:04:45 [debug] 14640#9400: *3695 reusable connection: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 malloc: 02DAF920:4096
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http process request line
2017/12/28 07:04:45 [info] 14640#9400: *3695 client sent invalid request while reading client request line, client: 10.92.67.192, server: test.saas.someloggingserver.com, request: "CONNECT test.saas.someloggingserver.com:443 HTTP/1.0"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http finalize request: 400, "?" a:1, c:1
2017/12/28 07:04:45 [debug] 14640#9400: *3695 event timer del: 496: 2616257381
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http special response: 400, "?"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http set discard body
2017/12/28 07:04:45 [debug] 14640#9400: *3695 HTTP/1.1 400 Bad Request
Server: nginx/1.13.8
Date: Thu, 28 Dec 2017 07:04:45 GMT
Content-Type: text/html
Content-Length: 173
Connection: close
2017/12/28 07:04:45 [debug] 14640#9400: *3695 write new buf t:1 f:0 02DB0018, pos 02DB0018, size: 152 file: 0, size: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http write filter: l:0 f:0 s:152
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http output filter "?"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http copy filter: "?"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http postpone filter "?" 02DB017C
2017/12/28 07:04:45 [debug] 14640#9400: *3695 write old buf t:1 f:0 02DB0018, pos 02DB0018, size: 152 file: 0, size: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 write new buf t:0 f:0 00000000, pos 00F25C68, size: 120 file: 0, size: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 write new buf t:0 f:0 00000000, pos 00F25758, size: 53 file: 0, size: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http write filter: l:1 f:0 s:325
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http write filter limit 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 WSASend: fd:496, s:325
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http write filter 00000000
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http copy filter: 0 "?"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http finalize request: 0, "?" a:1, c:1
2017/12/28 07:04:45 [debug] 14640#9400: *3695 event timer add: 496: 5000:2616202381
2017/12/28 07:04:45 [debug] 14640#9400: *3695 post event 02D76340
2017/12/28 07:04:45 [debug] 14640#9400: *3695 delete posted event 02D76340
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http lingering close handler
2017/12/28 07:04:45 [debug] 14640#9400: *3695 WSARecv: fd:496 rc:0 0 of 4096
2017/12/28 07:04:45 [debug] 14640#9400: *3695 lingering read: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http request count:1 blk:0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http close request
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http log handler
2017/12/28 07:04:45 [debug] 14640#9400: *3695 free: 02DAF920, unused: 1771
2017/12/28 07:04:45 [debug] 14640#9400: *3695 close http connection: 496
2017/12/28 07:04:45 [debug] 14640#9400: *3695 event timer del: 496: 2616202381
2017/12/28 07:04:45 [debug] 14640#9400: *3695 select del event fd:496 ev:0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 reusable connection: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 free: 02D4CF80
2017/12/28 07:04:45 [debug] 14640#9400: *3695 free: 02D50808, unused: 28
You can use Apache as a Forward Proxy, because nginx can only be used as a reverse proxy or as a http proxy.
https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#forwardreverse
An ordinary forward proxy is an intermediate server that sits between the client and the origin server. In order to get content from the origin server, the client sends a request to the proxy naming the origin server as the target. The proxy then requests the content from the origin server and returns it to the client. The client must be specially configured to use the forward proxy to access other sites.
A typical usage of a forward proxy is to provide Internet access to internal clients that are otherwise restricted by a firewall. The forward proxy can also use caching (as provided by mod_cache) to reduce network usage.
Forward Proxy
ProxyRequests On
ProxyVia On
<Proxy "*">
Require host internal.example.com
</Proxy>
As mentioned in one reply, I also faced the same issue with nginx setup so we have done the setup using Apache and wrote a article for the community about steps, which worked for us. Please check it out - https://medium.com/#gaurav.k.sarawgi/use-apache-to-create-forward-proxy-server-on-ubuntu-3299ef91a7cb

deploying war using tomcat7.maven plugin in eclipse but always throwing 401 unauthorized error

Need your help with this issue. I'm trying to deploy war file at localhost:8080 through eclipse using tomcat7.maven plugin version 2.1 scripts but always getting 401 unauthorized error. My PC OS is windows 7 64 bit.
Here is my POM:
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<configuration>
<url>http://localhost:8080/manager/text</url>
<server>localhost</server>
<username>tomcat</username>
<password>tomcat</password>
<path>/${project.build.finalName}</path>
<warDirectory>${project.build.directory}/exploded/${project.build.finalName}.war</warDirectory>
<update>true</update>
</configuration>
<version>2.1</version>
</plugin>
Here is my tomcat-users.xml :
<tomcat-users>
<user name="admin" password="admin" roles="admin-gui,manager-gui"/>
<role rolename="manager-script"/>
<role rolename="manager"/>
<role rolename="tomcat"/>
<user username="tomcat" password="tomcat" roles="manager-script"/>
</tomcat-users>
Here is my settings.xml in /.M2 directory.
<settings>
<servers>
<server>
<id>localhost</id>
<username>tomcat</username>
<password>tomcat</password>
</server>
</servers>
</settings>
Here is my log file output:
[DEBUG] Configuring mojo org.apache.tomcat.maven:tomcat7-maven-plugin:2.1:deploy from plugin realm ClassRealm[plugin>org.apache.tomcat.maven:tomcat7-maven-plugin:2.1, parent: sun.misc.Launcher$AppClassLoader#5d4177f3]
[DEBUG] Configuring mojo 'org.apache.tomcat.maven:tomcat7-maven-plugin:2.1:deploy' with basic configurator -->
[DEBUG] (f) charset = ISO-8859-1
[DEBUG] (f) contextFile = C:\java\Spring3HibernateMaven\target\Spring3HibernateMaven-0.0.1-SNAPSHOT\META-INF\context.xml
[DEBUG] (f) ignorePackaging = false
[DEBUG] (f) mode = war
[DEBUG] (f) packaging = war
[DEBUG] (f) password = tomcat
[DEBUG] (f) path = /Spring3HibernateMaven-0.0.1-SNAPSHOT
[DEBUG] (f) server = localhost
[DEBUG] (f) update = true
[DEBUG] (f) url = http://localhost:8080/manager/text
[DEBUG] (f) username = tomcat
[DEBUG] (f) version = 2.1
[DEBUG] (f) warFile = C:\java\Spring3HibernateMaven\target\Spring3HibernateMaven-0.0.1-SNAPSHOT.war
[DEBUG] -- end configuration --
[INFO] Deploying war to http://localhost:8080/Spring3HibernateMaven-0.0.1-SNAPSHOT
[DEBUG] Connection request: [route: {}->http://localhost:8080][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 5]
[DEBUG] Connection leased: [id: 0][route: {}->http://localhost:8080][total kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 5]
[DEBUG] Connecting to localhost:8080
[DEBUG] CookieSpec selected: best-match
[DEBUG] Re-using cached 'basic' auth scheme for http://localhost:8080
[DEBUG] Target auth state: SUCCESS
[DEBUG] Proxy auth state: UNCHALLENGED
[DEBUG] Attempt 1 to execute request
[DEBUG] Sending request: PUT /manager/text/deploy?path=%2FSpring3HibernateMaven-0.0.1-SNAPSHOT&update=true HTTP/1.1
[DEBUG] >> "PUT /manager/text/deploy?path=%2FSpring3HibernateMaven-0.0.1-SNAPSHOT&update=true HTTP/1.1[\r][\n]"
[DEBUG] >> "User-Agent: Apache Tomcat Maven Plugin/2.1[\r][\n]"
[DEBUG] >> "Content-Length: 11860176[\r][\n]"
[DEBUG] >> "Host: localhost:8080[\r][\n]"
[DEBUG] >> "Connection: Keep-Alive[\r][\n]"
[DEBUG] >> "Authorization: Basic dG9tY2F0OnRvbWNhdA==[\r][\n]"
[DEBUG] >> "[\r][\n]"
[DEBUG] >> PUT /manager/text/deploy?path=%2FSpring3HibernateMaven-0.0.1-SNAPSHOT&update=true HTTP/1.1
[DEBUG] >> User-Agent: Apache Tomcat Maven Plugin/2.1
[DEBUG] >> Content-Length: 11860176
[DEBUG] >> Host: localhost:8080
[DEBUG] >> Connection: Keep-Alive
[DEBUG] >> Authorization: Basic dG9tY2F0OnRvbWNhdA==
Uploading: http://localhost:8080/manager/text/deploy?path=%2FSpring3HibernateMaven-0.0.1-SNAPSHOT&update=true
Uploaded: http://localhost:8080/manager/text/deploy?path=%2FSpring3HibernateMaven-0.0.1-SNAPSHOT&update=true (11583 KB at 1833.5 KB/sec)
[DEBUG] << "HTTP/1.1 401 Unauthorized[\r][\n]"
[DEBUG] << "Server: Apache-Coyote/1.1[\r][\n]"
[DEBUG] << "Cache-Control: private[\r][\n]"
[DEBUG] << "Expires: Wed, 31 Dec 1969 18:00:00 CST[\r][\n]"
[DEBUG] << "WWW-Authenticate: Basic realm="Tomcat Manager Application"[\r][\n]"
[DEBUG] << "Content-Type: text/html;charset=ISO-8859-1[\r][\n]"
[DEBUG] << "Transfer-Encoding: chunked[\r][\n]"
[DEBUG] << "Date: Fri, 20 Sep 2013 05:24:35 GMT[\r][\n]"
[DEBUG] << "[\r][\n]"
[DEBUG] Receiving response: HTTP/1.1 401 Unauthorized
[DEBUG] << HTTP/1.1 401 Unauthorized
[DEBUG] << Server: Apache-Coyote/1.1
[DEBUG] << Cache-Control: private
[DEBUG] << Expires: Wed, 31 Dec 1969 18:00:00 CST
[DEBUG] << WWW-Authenticate: Basic realm="Tomcat Manager Application"
[DEBUG] << Content-Type: text/html;charset=ISO-8859-1
[DEBUG] << Transfer-Encoding: chunked
[DEBUG] << Date: Fri, 20 Sep 2013 05:24:35 GMT
[DEBUG] Connection can be kept alive indefinitely
[DEBUG] localhost:8080 requested authentication
[DEBUG] Authorization challenge processed
[DEBUG] Authentication failed
[DEBUG] Removing from cache 'basic' auth scheme for http://localhost:8080
[DEBUG] << "9ea[\r][\n]"
[DEBUG] << "[\r][\n]"
[DEBUG] << "<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">[\r][\n]"
[DEBUG] << "<html>[\r][\n]"
[DEBUG] << " <head>[\r][\n]"
[DEBUG] << " <title>401 Unauthorized</title>[\r][\n]"
[DEBUG] << " <style type="text/css">[\r][\n]"
[DEBUG] << " <!--[\r][\n]"
[DEBUG] << " BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;font-size:12px;}[\r][\n]"
[DEBUG] << " H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}[\r][\n]"
[DEBUG] << " PRE, TT {border: 1px dotted #525D76}[\r][\n]"
[DEBUG] << " A {color : black;}A.name {color : black;}[\r][\n]"
[DEBUG] << " -->[\r][\n]"
[DEBUG] << " </style>[\r][\n]"
[DEBUG] << " </head>[\r][\n]"
[DEBUG] << " <body>[\r][\n]"
[DEBUG] << " <h1>401 Unauthorized</h1>[\r][\n]"
[DEBUG] << " <p>[\r][\n]"
[DEBUG] << " You are not authorized to view this page. If you have not changed[\r][\n]"
[DEBUG] << " any configuration files, please examine the file[\r][\n]"
[DEBUG] << " <tt>conf/tomcat-users.xml</tt> in your installation. That[\r][\n]"
[DEBUG] << " file must contain the credentials to let you use this webapp.[\r][\n]"
My Run Configuration in Eclipse:
Maven Runtime: 3.1.0 ( External )
Goal: -X -e tomcat7:deploy