Spring Cloud Gateway hides server websocket handshake 401 failures to clients - spring-webflux

I'm reverse proxying a websocket backend API with spring-cloud-gateway 2.2.3. When this backend API rejects some websocket handshake request with a 401 Unauthorized status response, then spring-cloud-gateway still returns a 101 handshake status to the client (which gets confused and then misbehaves)
I need spring-cloud-gateway to return the original 401 websocket handshake error to the client so the SCG reverse proxy is transparent to the client (which is conforming to the WebSocket specs handshake)
Here are the full wiretap traces and exception (I have redacted hostnames).
The client-side response in this WSS request is available as a HAR file captured from chrome and which displays in chrome
as this screenpshot.
Here is my spring cloud gateway configuration
spring:
cloud:
gateway:
routes:
- id: route_shield
uri: https://shield-webui-cf-mysql.nd-int-cfapi.was.redacted
predicates:
- Host=**
filters:
- SetRequestHostHeader=shield-webui-cf-mysql.nd-int-cfapi.was.redacted
ssl:
useInsecureTrustManager: true
I'm wondering whether this is a spring-cloud-gateway bug, or a desired behavior which I can override.
To override it, here are alternatives I'm considering:
using circuit breaker filter and fallback to a local handler returning a 401
write a custom post-filter
Override/patch the WebsocketRoutingFilter
However my debugger breakpoint in the handle(WebSocketSession session) method does not trigger, suspecting it is not called
Likely would need to provide a RequestUpgradeStrategy bean as an alternative to the default implementation of org.springframework.web.reactive.socket.server.upgrade.ReactorNettyRequestUpgradeStrategy#getNativeResponse mentionned in the trace
io.netty.handler.codec.http.websocketx.WebSocketHandshakeException: Invalid handshake response getStatus: 401 Unauthorized
at io.netty.handler.codec.http.websocketx.WebSocketClientHandshaker13.verify(WebSocketClientHandshaker13.java:274) ~[netty-codec-http-4.1.51.Final.jar:4.1.51.Final]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
|_ checkpoint ⇢ http://localhost:8080/v2/events [ReactorNettyRequestUpgradeStrategy]

Related

Balancing export to jaeger in openTelemetry collector

I have configuration as documentation says
exporters:
jaeger:
endpoint: "ipv4:firstHost:14250,secondHost:14250"
balancer_name: "round_robin"
Collector produces error.
How I can configure collector to balance exporter for sending requests in different backends?
info exporterhelper/queued_retry.go:276 Exporting failed. Will retry the request after interval. {"component_kind": "exporter", "component_type": "jaeger", "component_name": "jaeger", "error": "failed to push trace data via Jaeger exporter: rpc error: code = Unavailable desc = last connection error: connection error: desc = "transport: Error while dialing dial tcp: address ipv4:firstHost:14250,secondHost:14250: too many colons in address"", "interval": "30.456378855s"}
It doesn't work in golang grpc client. I used openTelemetry load balancing Another option - use kubernetes to balance requests to backends.

The HTTP header line [group Name: XXX-OR-Migration^XXX-OR-Novation] does not conform to RFC 7230 and has been ignored

Our application was working fine on tomcat 8.5.5. Recently, we have upgraded to tomcat 9.0.31 and the login now fails with the below error
Message: The HTTP header line [group Name: XXX-OR-Migration^XXX-OR-Novation] does not conform to RFC 7230 and has been ignored.
Description: The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).
I have added the below attributes in server.xml under the Connector section and the request still fails:
relaxedPathChars="^:{}[]|""
relaxedQueryChars="[]|{}^:\`"<>""
java.net.URLEncoder
URLEncoder.encode(String, StandardCharsets.UTF_8)
Please escape these characters

Apache web server sending 400 response

We have configured NTLM authentication using SSPI on apache due to which the authentication is three steps, where there are two 401 responses followed by 201/200 response.
Now in IE browser, this breaks because of - Why "Content-Length: 0" in POST requests?
Apache web server sends a 400 bad request response due to empty post request due to which POST on the server breaks.
How can I configure Apache to not treat this as 400 BAD request and process it normally?

FIWARE CEP (Proton) REST ouput authentication error

I'm training to send an output event from FIWARE CEP (Proton), using the REST consumer, to an ActiveMQ queue. The credential for access the ActiveMQ queue are included in the URL, as http://user:passwrd#X.X.X.X:xxxx/api/message/myqueue, but I have the following error:
com.ibm.hrl.proton.webapp.resources.EventResource submitNewEvent
INFO: events sent to proton runtime...
org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: basic authentication scheme selected
org.apache.commons.httpclient.HttpMethodDirector processWWWAuthChallenge
INFO: No credentials available for BASIC 'ActiveMQRealm'#X.X.X.X
com.ibm.hrl.proton.server.executorServices.SimpleThreadFactory$ProtonExceptionHandler uncaughtException
SEVERE: Uncaught exception in thread: Thread[4,5,main],exception: com.ibm.hrl.proton.adapters.rest.client.RESTException: com.ibm.hrl.proton.adapters.rest.client.RESTException: Could not perform POST of event instance: ...
with request headers:
Content-Type: text/plain
User-Agent: Jakarta Commons-HttpClient/3.0
Host: X.X.X.X:xxxx
Content-Length: 389
to consumer http://user:passwrd#X.X.X.X:xxx/api/message/myqueue, responce result: 401
Seems like that Proton doesn't extract the credential from the URL.
Anyone else had the same problem?
You can add to the CEP REST consumer definition an AuthToken parameter.
From the CEP user guide (can be found here):
AuthToken – an optional parameter. When set, it is added as an X-Auth-Token
HTTP header of the request.

Gwan report.c statistics

I am testing on G-wan server performance and it's very amazing!!! Here is the output from report.c
Requests
All: 5,725 (6.06% of Cache misses)
HTTP: 66 (1.15% of all requests)
Errors: 70 (1.22% of all requests)
CSP: 5,650 (98.69% of all requests) Exceptions: 1
Connections
Accepted: 4,717 (1.21 requests per connection)
Closed: 4,372
Timeouts: 682 (14.46%) Accept:682 Read:0 Slow:0 Build:0 Send:0 Close:0
Busy: 345 (Waiting: 334 Reading: 9 Replying: 2 Sending: 0 Pushing: 0 Relaying: 0 Closing: 0)
I found that the Errors rate seem to be quite high, and there an exceptions occur on CSP too, could anyone tell me what did "Errors" mean and how to avoid it? Thanks!
the "Errors" rate seem to be quite high
That's HTTP errors (wrong requests coming from a client, not found resources, etc. - look at the error.log file for a trace).
The only way to avoid HTTP errors is to prevent clients from connecting to the server.
If you can't live with this "high rate of HTTP errors" of 1.22% of all requests then use a G-WAN connection handler (with the HTTP_ERROR notification) to make G-WAN ignore HTTP errors and close the connection without sending an HTTP error message (just return 0; in the handler) - but that's probably not what most users want.
there an exceptions occur on CSP too
An exception means a 'graceful crash report' was issued for a servlet bug. As you have only 1 crash on 5,650 dynamic requests, that was probably during the servlet development. Look at your error.log and trace files to check what happened.
Note that the "cache misses" statistics are for static contents only (1.15% of all your HTTP requests).
Apparently, not all your clients are responding in the timely fashion: you have timeouts and pending requests.