we have ingress-nginx running for a while and about 10% of requests ending up with some SSL handshake problem.
Here is an example of a failing connection:
2019/02/14 10:15:35 [debug] 237#237: *4612 accept: **.**.**.**:40928 fd:53
2019/02/14 10:15:35 [debug] 237#237: *4612 event timer add: 53: 60000:5527050245
2019/02/14 10:15:35 [debug] 237#237: *4612 reusable connection: 1
2019/02/14 10:15:35 [debug] 237#237: *4612 epoll add event: fd:53 op:1 ev:80002001
2019/02/14 10:15:45 [debug] 237#237: *4612 http check ssl handshake
2019/02/14 10:15:45 [debug] 237#237: *4612 http recv(): 0
2019/02/14 10:15:45 [info] 237#237: *4612 client closed connection while SSL handshaking, client: **.**.**.**, server: 0.0.0.0:443
2019/02/14 10:15:45 [debug] 237#237: *4612 close http connection: 53
2019/02/14 10:15:45 [debug] 237#237: *4612 event timer del: 53: 5527050245
2019/02/14 10:15:45 [debug] 237#237: *4612 reusable connection: 0
2019/02/14 10:15:45 [debug] 237#237: *4612 free: 00007F4CC5858E00, unused: 232
10% of failures seems to be quite a lot to expect.
I really would appreciate any help in this!
Related
I am using the mvn org.sonatype.ossindex.maven:ossindex-maven-plugin from command line to check the dependencies of a maven project for CVEs. Locally, everything works fine, but in my build pipeline in Azure Devops, I get the following error:
[DEBUG] Connecting to ossindex.sonatype.org/18.118.116.156:443
[DEBUG] Connecting socket to ossindex.sonatype.org/18.118.116.156:443 with timeout 0
[DEBUG] Enabled protocols: [TLSv1.3, TLSv1.2]
[DEBUG] Enabled cipher suites:[TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
[DEBUG] Starting handshake
[DEBUG] Secure session established
[DEBUG] negotiated protocol: TLSv1.2
[DEBUG] negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
[DEBUG] peer principal: CN=ossindex.sonatype.org
[DEBUG] peer alternative names: [ossindex.sonatype.org]
[DEBUG] issuer principal: CN=Amazon, OU=Server CA 1B, O=Amazon, C=US
[DEBUG] Connection established 172.21.1.242:33030<->18.118.116.156:443
[DEBUG] Executing request POST /api/v3/component-report HTTP/1.1
[DEBUG] Proxy auth state: UNCHALLENGED
[DEBUG] http-outgoing-2 >> POST /api/v3/component-report HTTP/1.1
[DEBUG] http-outgoing-2 >> User-Agent: ossindex-client/1.1.1 (Linux; 5.4.0-124-generic; amd64; 11.0.12) Maven/3.8.1 Maven-Plugin/3.1.0
[DEBUG] http-outgoing-2 >> Authorization: Basic {SOME_BASE_64_THAT_TRANSLATES_TO:AzureDevOps:ey...}
[DEBUG] http-outgoing-2 >> Accept: application/vnd.ossindex.component-report.v1+json
[DEBUG] http-outgoing-2 >> Content-Length: 6737
[DEBUG] http-outgoing-2 >> Content-Type: application/vnd.ossindex.component-report-request.v1+json; charset=UTF-8
[DEBUG] http-outgoing-2 >> Host: ossindex.sonatype.org
[DEBUG] http-outgoing-2 >> Connection: Keep-Alive
[DEBUG] http-outgoing-2 >> Accept-Encoding: gzip,deflate
[DEBUG] http-outgoing-2 >> "POST /api/v3/component-report HTTP/1.1[\r][\n]"
[DEBUG] http-outgoing-2 >> "User-Agent: ossindex-client/1.1.1 (Linux; 5.4.0-124-generic; amd64; 11.0.12) Maven/3.8.1 Maven-Plugin/3.1.0[\r][\n]"
[DEBUG] http-outgoing-2 >> "Authorization: Basic {SOME_OTHER_BASE64_NO_IDEA_WHERE_IT_COMES_FROM=[\r][\n]"
[DEBUG] http-outgoing-2 >> "Accept: application/vnd.ossindex.component-report.v1+json[\r][\n]"
[DEBUG] http-outgoing-2 >> "Content-Length: 6737[\r][\n]"
[DEBUG] http-outgoing-2 >> "Content-Type: application/vnd.ossindex.component-report-request.v1+json; charset=UTF-8[\r][\n]"
[DEBUG] http-outgoing-2 >> "Host: ossindex.sonatype.org[\r][\n]"
[DEBUG] http-outgoing-2 >> "Connection: Keep-Alive[\r][\n]"
[DEBUG] http-outgoing-2 >> "Accept-Encoding: gzip,deflate[\r][\n]"
[DEBUG] http-outgoing-2 >> "[\r][\n]"
[DEBUG] http-outgoing-2 >> "{"coordinates":[ A_LIST_OF_128_COORDINATES]
[DEBUG] http-outgoing-2 << "HTTP/1.1 401 Unauthorized[\r][\n]"
[DEBUG] http-outgoing-2 << "Date: Wed, 07 Dec 2022 13:51:42 GMT[\r][\n]"
[DEBUG] http-outgoing-2 << "Content-Length: 0[\r][\n]"
[DEBUG] http-outgoing-2 << "Connection: keep-alive[\r][\n]"
so I tried setting the clientConfiguration parameter described here.
it has a dead link but I believe it references this class which has this class as a member.
mvn org.sonatype.ossindex.maven:ossindex-maven-plugin:3.1.0:audit -f $(Build.SourcesDirectory)/pom.xml -"Dossindex.authId=MY_SERVER" -"Dossindex.clientConfiguration={\"auth\": {\"username\": \"myemail#myorg.com\", \"password\": \"$(OSS_INDEX_API_TOKEN)\"} }"
but it didn't change anything.
Is this how one passes this parameter? Why am I getting a 401 in my build pipeline but everything works locally? I tried reading the docs on what requests are allowed/authorized but found them rather lacking...
I am trying to set up nginx to map TLS connections to different backends based on the SNI server name. From what I can tell, my client is sending the server name, but the preread module is only reading a hyphen.
Here is my nginx congif:
stream {
map_hash_bucket_size 64;
############################################################
### logging
log_format log_stream '$remote_addr [$time_local] $protocol [$ssl_preread_server_name] [$ssl_preread_alpn_protocols] [$instanceport] '
'$status $bytes_sent $bytes_received $session_time';
error_log /usr/home/glance/Logs/pservernginx.error.log info;
access_log /usr/home/glance/Logs/pservernginx.access.log log_stream;
############################################################
### ssl configuration
ssl_certificate /usr/home/glance/GlanceReleases/star.myglance.org.pem;
ssl_certificate_key /usr/home/glance/GlanceReleases/star.myglance.org.pem;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5:!RC4;
limit_conn_zone $binary_remote_addr zone=ip_addr:10m;
########################################################################
### Raw TLS PServer Connections
### Listen for TLS on 5501 and forward to TCP sock 6500 (socket port)
### https://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html
map $ssl_preread_server_name $instanceport {
presence.myglance.org 6500;
presence-1.myglance.org 6501;
presence-2.myglance.org 6502;
default glance-no-upstream-instance-configured;
}
server {
listen 5501 ssl;
ssl_preread on;
proxy_connect_timeout 20s; # max time to connect to pserver
proxy_timeout 30s; # max time between successive reads or writes
proxy_pass 127.0.0.1:$instanceport;
}
}
wireshark shows the Server Name header:
The nginx access log shows only hyphens for the preread variables:
108.49.96.66 [12/Apr/2019:11:50:58 +0000] TCP [-] [-] [glance-no-upstream-instance-configured] 500 0 0 0.066
I'm running nginx 1.14.2 on FreeBSD. How can I debug what is happening in the preread module?
================ UPDATE ===============
Turned on debug logging. Maybe "ssl preread: not a handshake" is a clue.
2019/04/12 14:49:50 [info] 61420#0: *9 client 108.49.96.66:54740 connected to 0.0.0.0:5501
2019/04/12 14:49:50 [debug] 61420#0: *9 posix_memalign: 0000000801C35000:256 #16
2019/04/12 14:49:50 [debug] 61419#0: accept on 0.0.0.0:5501, ready: 1
2019/04/12 14:49:50 [debug] 61419#0: accept() not ready (35: Resource temporarily unavailable)
2019/04/12 14:49:50 [debug] 61420#0: *9 posix_memalign: 0000000801C35600:256 #16
2019/04/12 14:49:50 [debug] 61420#0: *9 generic phase: 0
2019/04/12 14:49:50 [debug] 61420#0: *9 generic phase: 1
2019/04/12 14:49:50 [debug] 61420#0: *9 generic phase: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 tcp_nodelay
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_do_handshake: -1
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_get_error: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 kevent set event: 5: ft:-1 fl:0025
2019/04/12 14:49:50 [debug] 61420#0: *9 event timer add: 5: 60000:29203481224
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL handshake handler: 0
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_do_handshake: 1
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD"
2019/04/12 14:49:50 [debug] 61420#0: *9 event timer del: 5: 29203481224
2019/04/12 14:49:50 [debug] 61420#0: *9 generic phase: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 ssl preread handler
2019/04/12 14:49:50 [debug] 61420#0: *9 malloc: 0000000801CFF000:16384
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_read: -1
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_get_error: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 ssl preread handler
2019/04/12 14:49:50 [debug] 61420#0: *9 posix_memalign: 0000000801C35900:256 #16
2019/04/12 14:49:50 [debug] 61420#0: *9 event timer add: 5: 30000:29203451252
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_read: 81
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_read: -1
2019/04/12 14:49:50 [debug] 61420#0: *9 SSL_get_error: 2
2019/04/12 14:49:50 [debug] 61420#0: *9 ssl preread handler
2019/04/12 14:49:50 [debug] 61420#0: *9 ssl preread: not a handshake
2019/04/12 14:49:50 [debug] 61420#0: *9 event timer del: 5: 29203451252
2019/04/12 14:49:50 [debug] 61420#0: *9 proxy connection handler
2019/04/12 14:49:50 [debug] 61420#0: *9 malloc: 0000000801DF7000:400
2019/04/12 14:49:50 [debug] 61420#0: *9 malloc: 0000000801CD9000:16384
2019/04/12 14:49:50 [debug] 61420#0: *9 stream map started
2019/04/12 14:49:50 [debug] 61420#0: *9 stream map: "" "glance-no-upstream-instance-configured"
================= UPDATE 2 ======================
I tested using
openssl s_client -connect ... -servername ...
instead of my client. Now it appears that the preread module is blocked waiting for data for 30 seconds (error code 2 is WANT_READ):
2019/04/23 13:04:30 [debug] 61419#0: *12844 SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD"
2019/04/23 13:04:30 [debug] 61419#0: *12844 event timer del: 3: 30147561850
2019/04/23 13:04:30 [debug] 61419#0: *12844 generic phase: 2
2019/04/23 13:04:30 [debug] 61419#0: *12844 ssl preread handler
2019/04/23 13:04:30 [debug] 61419#0: *12844 malloc: 0000000801CA6140:16384
2019/04/23 13:04:30 [debug] 61419#0: *12844 SSL_read: -1
2019/04/23 13:04:30 [debug] 61419#0: *12844 SSL_get_error: 2
2019/04/23 13:04:30 [debug] 61419#0: *12844 ssl preread handler
2019/04/23 13:04:30 [debug] 61419#0: *12844 posix_memalign: 0000000801DB3400:256 #16
2019/04/23 13:04:30 [debug] 61419#0: *12844 event timer add: 3: 30000:30147531898
2019/04/23 13:05:00 [debug] 61419#0: *12844 event timer del: 3: 30147531898
2019/04/23 13:05:00 [debug] 61419#0: *12844 finalize stream session: 200
2019/04/23 13:05:00 [debug] 61419#0: *12844 stream log handler
2019/04/23 13:05:00 [debug] 61419#0: *12844 stream map started
2019/04/23 13:05:00 [debug] 61419#0: *12844 stream script var: ""
I found the problem:
listen 5501 **ssl**;
ssl_preread on;
ssl in the listen directive caused that nginx server to do the ssl handshake. By the time the preread module was notified, the handshake bytes had already been consumed, which is all consistent with the behavior I was seeing. In my case, I still want nginx to offload the encryption. So I created a set of nginx server directives to terminate the ssl connection before passing to my back end.
This is the relevant portion of my nginx config after fixing it. Note that the last server directive (the one that uses ssl_preread) does not terminate the SSL connection.
########################################################################
### TLS Connections
### Listen for TLS on 5501 and forward to TCP sock 6500 (socket port)
### https://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html
map $ssl_preread_server_name $instanceport {
presence.myglance.org 5502;
presence-1.myglance.org 5503;
presence-2.myglance.org 5504;
default glance-no-upstream-instance-configured;
}
server {
listen 5502 ssl;
ssl_preread off;
proxy_pass 127.0.0.1:6502;
}
server {
listen 5503 ssl;
ssl_preread off;
proxy_pass 127.0.0.1:6503;
}
server {
listen 5504 ssl;
ssl_preread off;
proxy_pass 127.0.0.1:6504;
}
server {
listen 5501;
ssl_preread on;
proxy_connect_timeout 20s; # max time to connect to pserver
proxy_timeout 30s; # max time between successive reads or writes
proxy_pass 127.0.0.1:$instanceport;
}
In case you need to use ssl in listen directive, you can simply use $ssl_server_name in the map block instead of $ssl_preread_server_name
I have configured Nginx reverse proxy server to listen on port 443 and pass the requests to an upstream SAAS client. Below is the configuration.
server {
listen 443;
server_name test.saas.someloggingserver.com;
ssl on;
ssl_certificate C:/nginx-1.13.8/nginx-1.13.8/ssl/server_cert.crt;
ssl_certificate_key C:/nginx-1.13.8/nginx-1.13.8/ssl/server.key;
ssl_session_timeout 5m;
ssl_protocols SSLv2 SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
ssl_prefer_server_ciphers on;
expires 0;
add_header Cache-Control private;
access_log logs/encrypted_access.txt;
error_log logs/encrypted_error.txt debug;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect http:// https://;
proxy_pass https://test.saas.someloggingserver.com;
}
}
The SSL certificate here also includes a intermediate certificate for SSL handshaking with the upstream server. Now when I try to access the https://test.saas.someloggingserver.com from IE (having above proxy configured) I get below 400 bad request error. I am no Nginx debugging expert, appreciate if you can take a look at below logs and direct me towards the possible cause of this issue?
2017/12/28 07:04:45 [debug] 14640#9400: post event 02D76250
2017/12/28 07:04:45 [debug] 14640#9400: delete posted event 02D76250
2017/12/28 07:04:45 [debug] 14640#9400: accept on 0.0.0.0:443, ready: 0
2017/12/28 07:04:45 [debug] 14640#9400: malloc: 02D50808:256
2017/12/28 07:04:45 [debug] 14640#9400: *3695 accept: 10.92.67.192:49268 fd:496
2017/12/28 07:04:45 [debug] 14640#9400: *3695 event timer add: 496: 60000:2616257381
2017/12/28 07:04:45 [debug] 14640#9400: *3695 reusable connection: 1
2017/12/28 07:04:45 [debug] 14640#9400: *3695 select add event fd:496 ev:0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 post event 02D76340
2017/12/28 07:04:45 [debug] 14640#9400: *3695 delete posted event 02D76340
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http check ssl handshake
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http recv(): 1
2017/12/28 07:04:45 [debug] 14640#9400: *3695 plain http
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http wait request handler
2017/12/28 07:04:45 [debug] 14640#9400: *3695 malloc: 02D4CF80:1024
2017/12/28 07:04:45 [debug] 14640#9400: *3695 WSARecv: fd:496 rc:0 266 of 1024
2017/12/28 07:04:45 [debug] 14640#9400: *3695 reusable connection: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 malloc: 02DAF920:4096
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http process request line
2017/12/28 07:04:45 [info] 14640#9400: *3695 client sent invalid request while reading client request line, client: 10.92.67.192, server: test.saas.someloggingserver.com, request: "CONNECT test.saas.someloggingserver.com:443 HTTP/1.0"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http finalize request: 400, "?" a:1, c:1
2017/12/28 07:04:45 [debug] 14640#9400: *3695 event timer del: 496: 2616257381
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http special response: 400, "?"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http set discard body
2017/12/28 07:04:45 [debug] 14640#9400: *3695 HTTP/1.1 400 Bad Request
Server: nginx/1.13.8
Date: Thu, 28 Dec 2017 07:04:45 GMT
Content-Type: text/html
Content-Length: 173
Connection: close
2017/12/28 07:04:45 [debug] 14640#9400: *3695 write new buf t:1 f:0 02DB0018, pos 02DB0018, size: 152 file: 0, size: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http write filter: l:0 f:0 s:152
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http output filter "?"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http copy filter: "?"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http postpone filter "?" 02DB017C
2017/12/28 07:04:45 [debug] 14640#9400: *3695 write old buf t:1 f:0 02DB0018, pos 02DB0018, size: 152 file: 0, size: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 write new buf t:0 f:0 00000000, pos 00F25C68, size: 120 file: 0, size: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 write new buf t:0 f:0 00000000, pos 00F25758, size: 53 file: 0, size: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http write filter: l:1 f:0 s:325
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http write filter limit 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 WSASend: fd:496, s:325
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http write filter 00000000
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http copy filter: 0 "?"
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http finalize request: 0, "?" a:1, c:1
2017/12/28 07:04:45 [debug] 14640#9400: *3695 event timer add: 496: 5000:2616202381
2017/12/28 07:04:45 [debug] 14640#9400: *3695 post event 02D76340
2017/12/28 07:04:45 [debug] 14640#9400: *3695 delete posted event 02D76340
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http lingering close handler
2017/12/28 07:04:45 [debug] 14640#9400: *3695 WSARecv: fd:496 rc:0 0 of 4096
2017/12/28 07:04:45 [debug] 14640#9400: *3695 lingering read: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http request count:1 blk:0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http close request
2017/12/28 07:04:45 [debug] 14640#9400: *3695 http log handler
2017/12/28 07:04:45 [debug] 14640#9400: *3695 free: 02DAF920, unused: 1771
2017/12/28 07:04:45 [debug] 14640#9400: *3695 close http connection: 496
2017/12/28 07:04:45 [debug] 14640#9400: *3695 event timer del: 496: 2616202381
2017/12/28 07:04:45 [debug] 14640#9400: *3695 select del event fd:496 ev:0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 reusable connection: 0
2017/12/28 07:04:45 [debug] 14640#9400: *3695 free: 02D4CF80
2017/12/28 07:04:45 [debug] 14640#9400: *3695 free: 02D50808, unused: 28
You can use Apache as a Forward Proxy, because nginx can only be used as a reverse proxy or as a http proxy.
https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#forwardreverse
An ordinary forward proxy is an intermediate server that sits between the client and the origin server. In order to get content from the origin server, the client sends a request to the proxy naming the origin server as the target. The proxy then requests the content from the origin server and returns it to the client. The client must be specially configured to use the forward proxy to access other sites.
A typical usage of a forward proxy is to provide Internet access to internal clients that are otherwise restricted by a firewall. The forward proxy can also use caching (as provided by mod_cache) to reduce network usage.
Forward Proxy
ProxyRequests On
ProxyVia On
<Proxy "*">
Require host internal.example.com
</Proxy>
As mentioned in one reply, I also faced the same issue with nginx setup so we have done the setup using Apache and wrote a article for the community about steps, which worked for us. Please check it out - https://medium.com/#gaurav.k.sarawgi/use-apache-to-create-forward-proxy-server-on-ubuntu-3299ef91a7cb
I am using Nginx to forward my ssl req to some server.
I am trying firefox chrome etc but doesn't help
what should be the correct config, also i am trying to implement this along with SSL as well the http_auth
The config i am using :
server {
listen 443 default_server ssl;
#root /var/www;
#index index.html index.htm index.php;
error_log /var/log/nginx/testconfg.log debug;
ssl on;
ssl_certificate /etc/ssl/testconfig/certs/server.crt;
ssl_certificate_key /etc/ssl/testconfig/private/server.key;
ssl_client_certificate /etc/ssl/testconfig/certs/ca.crt;
ssl_crl /etc/ssl/testconfig/private/ca.crl;
ssl_password_file /etc/ssl/testconfig/global.pass;
ssl_verify_client optional;
ssl_session_timeout 10m;
ssl_protocols SSLv3 TLSv1;
ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
ssl_prefer_server_ciphers on;
location / {
proxy_pass http://172.16.8.50:5601;
proxy_ssl_server_name on;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
ERROR:
verify:0, error:18, depth:0, subject:"/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd", issuer:"/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd"
2017/01/04 11:08:23 [debug] 19105#19105: *2 verify:0, error:8, depth:0, subject:"/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd", issuer:"/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd"
2017/01/04 11:08:23 [debug] 19105#19105: *2 verify:1, error:8, depth:0, subject:"/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd", issuer:"/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd"
2017/01/04 11:08:23 [debug] 19105#19105: *2 SSL_do_handshake: 1
2017/01/04 11:08:23 [debug] 19105#19105: *2 SSL: TLSv1, cipher: "ECDHE-RSA-AES256-SHA SSLv3 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA1"
2017/01/04 11:08:23 [debug] 19105#19105: *2 reusable connection: 1
2017/01/04 11:08:23 [debug] 19105#19105: *2 http wait request handler
2017/01/04 11:08:23 [debug] 19105#19105: *2 malloc: 0000556B5D448D50:1024
2017/01/04 11:08:23 [alert] 19105#19105: *2 ignoring stale global SSL error (SSL: error:04091077:rsa routines:INT_RSA_VERIFY:wrong signature length error:0D0C5006:asn1 encoding routines:ASN1_item_verify:EVP lib) while waiting for request, client: 165.225.106.84, server: 0.0.0.0:443
2017/01/04 11:08:23 [debug] 19105#19105: *2 SSL_read: -1
2017/01/04 11:08:23 [debug] 19105#19105: *2 SSL_get_error: 2
2017/01/04 11:08:23 [debug] 19105#19105: *2 free: 0000556B5D448D50
2017/01/04 11:08:24 [debug] 19105#19105: *2 http wait request handler
2017/01/04 11:08:24 [debug] 19105#19105: *2 malloc: 0000556B5D448D50:1024
2017/01/04 11:08:24 [debug] 19105#19105: *2 SSL_read: 0
2017/01/04 11:08:24 [debug] 19105#19105: *2 SSL_get_error: 5
2017/01/04 11:08:24 [debug] 19105#19105: *2 peer shutdown SSL cleanly
2017/01/04 11:08:24 [info] 19105#19105: *2 client closed connection while waiting for request, client: 165.225.106.84, server: 0.0.0.0:443
2017/01/04 11:08:24 [debug] 19105#19105: *2 close http connection: 3
2017/01/04 11:08:24 [debug] 19105#19105: *2 SSL_shutdown: 1
2017/01/04 11:08:24 [debug] 19105#19105: *2 event timer del: 3: 1483528163104
2017/01/04 11:08:24 [debug] 19105#19105: *2 reusable connection: 0
2017/01/04 11:08:24 [debug] 19105#19105: *2 free: 0000556B5D448D50
2017/01/04 11:08:24 [debug] 19105#19105: *2 free: 0000556B5D46F910, unused: 56
can someone help me out whats the issue
WHen using NginX for ssl offloading things work fine, however when trying to do this for serialized stream over a socket it seems to work incorrectly, and the documentation over using streams http://nginx.org/en/docs/stream/ngx_stream_upstream_module.html makes no sense,
can anyone help me with a working foo bar example of how to ssl offload for a streaming input(made of serialized logging event objects in this case) with nginx
Edit:
we are trying to forward and strip SSL off of log4j socketappender output but cannot even get the forwarding working when the SSL is not enabled.
here is the config file without the ssl offloading portion(when retrieving non https the forwarding still does not work correctly)
events {
worker_connections 1024;
}
http {
server {
listen 4560 ;
server_name logstash.corelims.com;
location / {
proxy_pass_request_headers on;
proxy_pass http://localhost:4561/;
proxy_redirect http://localhost:4561/ http://logstash.corelims.com:4560/;
}
error_log C:/ELK/nginx-1.9.4/logs/debug.log debug;
}
}
which leads us to these debug logs:
2015/08/27 00:54:59 [info] 5052#3716: *45 WSARecv() failed (10054: An existing connection was forcibly closed by the remote host) while reading client request line, client: IP.ADDRESS.HIDDENFOR.POST, server: foo.bar.com, request: "’ "
2015/08/27 00:54:59 [debug] 5052#3716: *45 lingering read: -1
2015/08/27 00:54:59 [debug] 5052#3716: *45 http request count:1 blk:0
2015/08/27 00:54:59 [debug] 5052#3716: *45 http close request
2015/08/27 00:54:59 [debug] 5052#3716: *45 http log handler
2015/08/27 00:54:59 [debug] 5052#3716: *45 free: 008DE8E8, unused: 1991
2015/08/27 00:54:59 [debug] 5052#3716: *45 close http connection: 300
2015/08/27 00:54:59 [debug] 5052#3716: *45 event timer del: 300: 1822859279
2015/08/27 00:54:59 [debug] 5052#3716: *45 select del event fd:300 ev:0
2015/08/27 00:54:59 [debug] 5052#3716: *45 reusable connection: 0
2015/08/27 00:54:59 [debug] 5052#3716: *45 free: 023BCED0
2015/08/27 00:54:59 [debug] 5052#3716: *45 free: 023BEEA8, unused: 24
2015/08/27 12:27:45 [debug] 5052#3716: select del event fd:316 ev:0