Proxy is ignored for localhost in docker selenium version: 3.141.59-zinc - selenium

Description:
I have upgraded docker selenium version to 3.141.59-zinc (from 3.141.59-europium), it started failing the acceptance test due to header info (set through proxy server) not found at server side. If I change image from zinc to europium - all works fine.
Log trace with 3.141.59-europium:
Remote address of request printed at server side: 127.0.0.1
Headers: {accept-language=en-US,en;q=0.9, host=localhost:39868, upgrade-insecure-requests=1, user=123456789, accept-encoding=gzip, deflate, br, user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36,
accept=text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8, via=1.1 browsermobproxy}
Log trace with 3.141.59-zinc :
Remote address of request printed at server side: 0:0:0:0:0:0:0:1
Headers: {sec-fetch-mode=navigate, sec-fetch-site=none, accept-language=en-US,en;q=0.9, host=localhost:42365, upgrade-insecure-requests=1, connection=keep-alive, sec-fetch-user=?1, accept-encoding=gzip, deflate, br, user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36, accept=text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9}
To Reproduce
Create Proxy object with host and port.
Set proxy in webdriver capabilities.
DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(CapabilityType.PROXY, proxy);
Set Proxy header
proxyServer.addHeader("user", "123456789");
Access application
driver.get("http://localhost:/welcome")
Check for proxy header "user", it should be 123456789
Expected behaviour
I am setting header with user=123456789, which is not getting passed if using webdriver 3.141.59-zinc. If I manually call url using URLConnection with proxy - Its working (So no issue in proxy server).
And also If I use ip address instead of localhost, its working fine (proxy header available in request at server). So I guess, its ignoring proxy for localhost in the new version of webdriver 3.141.59-zinc. I also tried with setting noProxy with null/"" but it did not work.
Environment
OS: Oracle Linux Server release 7.5
Docker-Selenium image version: 3.141.59-zinc
Docker version: 17.06.2-ol
Note: Using standalone chrome in headless mode

Related

I set the proxy for selenium chrome driver still traffic is going from my local in java

I set the proxy to the chrome driver
ChromeOptions optionsMap = new ChromeOptions();
optionsMap.addArguments("--headless");
optionsMap.addArguments("--user-agent=Mozilla/5.0 (X11; Linux x86_64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.50 Safari/537.36");
Proxy proxy = new Proxy();
proxy.setProxyType(ProxyType.MANUAL);
proxy.setHttpProxy(proxyValue);
optionsMap.setCapability(CapabilityType.PROXY, proxy);
But the traffic is going from local IP address. I want the traffic to go from the proxy. Could you please resolve the issue.
To test I set the chrome driver proxy to different ip not the local address and run the website https://icanhazip.com/ to know the ip. The website still showing the local IP.
I built the chrome driver using the below code
String driverAbsPath = new File(seleniumChromeDriverPath).getAbsolutePath().replace("/~/", "/");
ChromeDriverService service = new ChromeDriverService.Builder()
.usingDriverExecutable(new File(driverAbsPath))
.usingAnyFreePort()
.build();
optionsMap.merge(chromecapabilities);
driver = new ChromeDriver(service, optionsMap);
Try adding the ip address into the X-Forwarded-For header on the proxy side:
proxy.addHeader("X-Forwarded-For", "1.2.3.4")

NGINX: different logs for two different websocket connections

We have two clients, one javascript and one C++. The C++ is using libwebsockets. Both of them try to connect via secure websockets (wss) to our websocket server via port 7000, that is sitting behind an NGINX server. When the javascript client connects, the connection is successful and in the NGINX 7000 port logs, this shows:
[04/May/2018:12:25:30 +0000] "GET / HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ Safari/537.36"
However, when trying to connect with the C++ client, the connection fails and the logs show
[04/May/2018:10:59:40 +0000] "GET / HTTP/1.1" 400 5 "-" "-"
Why is it throwing a 400 instead of a 101 in the second case? We are not sure how to debug this. This is what we use in the websocket client
ws->init("wss://echo.websocket.org:7000", nullptr, "your CA root file path");
The client just says Connection Failed. We tried inspecting data with wireshark and enabling more logs in the NGINX server but we can't figure it out. What can it be about? Could it be because of miss-matched SSL/TLS versions?
Turns out the the client's library was not actually including the port number in the request, even when it's included in the connection URL. So either configuring NGINX to handle this specific case or making the library include the port in the request, solves the probem.

CORS Fails with Safari 10.1.2 and Google Storage

Platforms:
Tested on iPhone iOS 10, macOS Sierra v10.12.6
Safari v10.1.2 (Safari v10.1.1 and below don't seem to have this problem, and neither do Chrome nor Firefox)
Description:
We're having a problem saving a photo through Google Cloud Storage. From the web inspector, we see that we're making a OPTIONS request to http://storage.googleapis.com/..., but we receive an empty response. Whereas, in other browsers or in other versions of Safari, we don't see a OPTIONS request, only the POST request. We've verified that our CORS configuration on the Google Cloud Storage bucket allows our origin.
Our request headers for the OPTIONS request look like this:
Access-Control-Request-Headers:
Referer: <referrer>
Origin: <origin>
Accept: */
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/603.3.8 (KHTML, like Gecko) Version/10.1.2 Safari/603.3.8
Access-Control-Request-Method: POST
From what we can see in the Safari devtools, there are no response headers, and the response is empty. As for the status code, there's no status code from what we can see in Safari devtools, but when we were using Charles, we saw that we received a 200 status code, but the response and response headers were empty as well.
These are the errors in the console:
http://storage.googleapis.com/... Failed to load resource: Origin
<origin> is not allowed by Access-Control-Allow-Origin
XMLHttpRequest cannot load http://storage.googleapis.com/... Origin
<origin> is not allowed by Access-Control-Allow-Origin.
Is there a issue with the latest version of Safari and CORS?

Websocket Not working with Some ISP's

Not sure why I am getting this issue, But My Websocket Connection works on specific ISP while fails on others. I know it sounds absurd but it is happening. My websocket connection works on Two ISP while fails on one. I am using Wildfly Application Server serving the WS Connection and Apache WebServer for proxy forwarding.
Here is the detail of my Request/Response,
General
Request URL:ws://example.com/chat/3
Request Method:GET
Status Code:101 Switching Protocols
Response Headers
Connection:Upgrade
Content-Length:0
Date:Fri, 13 May 2016 13:09:11 GMT
Origin:http://example.com
Sec-WebSocket-Accept:pPjTLv5Dz+/vyjY/SkeMihaXDd0=
Sec-WebSocket-Location:ws://example.com/chat/3
Server:WildFly/9
Upgrade:WebSocket
X-Powered-By:Undertow/1
Request Headers
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:Upgrade
Cookie:mp_c4f10660603c33a8e9307b70e6767539_mixpanel=%7B%22distinct_id%22%3A%20%2215210855b11180-0ffdda567-1821170c-d37aa-15210855b123f2%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; mf_user=a60cd2cdcfc41836645d949f71ee3127; intercom-id=d1af89ac-9d55-4fef-8a17-3848d8ef0fce; wooTracker=VQf16pMBx4Pu; _ga=GA1.2.544774749.1447732319; JSESSIONID=z4a1hBpQJQz4YCsLivHRRFf8b0dzYzBsT_4PLadB.ip-172-30-0-20; mf_154095de-56ef-4099-9976-f9a298cf0677=8438220eda64d856436d798ca0b9188a|05132367e34aabbf7bcce5b1e8811235b0bd15d4|1463144963483||19|
Host:example.com
Origin:http://example.com
Pragma:no-cache
Sec-WebSocket-Extensions:permessage-deflate; client_max_window_bits
Sec-WebSocket-Key:94OH1SxHvszgJO6Rg31WGA==
Sec-WebSocket-Version:13
Upgrade:websocket
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36
However, When I tried testing some of the Demo Sites like http://websocket.org/echo.html they are working as expected.
The only difference I found between those connections was header on response from the server Upgrade:websocket whereas my server is returning Upgrade:WebSocket as part of the response. However, I believe that the header are case insensitive and it shouldn't be the issue.
Also, Is it possible to rewrite the Header value for the Response using apache header mod?

getting wss working with stunnel & ratchet

Running Apache v2.4, PHP v5.6 w/ php-fpm, on CentOS 7
So I'm trying to get wss working with stunnel & Ratchet. I've got Ratchet's Hello World (http://socketo.me/docs/hello-world) working successfully with the test code on non-ssl at port 8080. I can telnet from the same machine, as well as telnet from a a different machine.
Through reading, I know that ratchet doesn't support ssl connections, but does by wrapping the communications with stunnel, so I setup stunnel with an appropriate cert listening on port 8443, which appears to be working (as evidenced by the log file).
When I try to make a connection using wss from Firefox's scratchpad, I get the following error: Firefox can't establish a connection to the server at wss://testserver.testdomain.com:8443 I can see the connection being created through the various log files, however somewhere along the line communications is failing since I can't create a solid wss connection. Any ideas on what I'm missing or where I'm screwing up? Log/code below
Scratchpad Code
var conn = new WebSocket('wss://testserver.testdomain.com:8443');
conn.onopen = function(e) {
console.log("Connection established!");
};
conn.onmessage = function(e) {
console.log(e.data);
};
Stunnel config:
cert = /etc/httpd/ssl/ov_wildcard.pem
[hublistener]
accept = 8443
connect = 8080
Stunnel log:
2016.05.11 14:43:28 LOG5[38759:140614807435328]: stunnel 4.56 on x86_64-redhat-linux-gnu platform
2016.05.11 14:43:28 LOG5[38759:140614807435328]: Compiled/running with OpenSSL 1.0.1e-fips 11 Feb 2013
2016.05.11 14:43:28 LOG5[38759:140614807435328]: Threading:PTHREAD Sockets:POLL,IPv6 SSL:ENGINE,OCSP,FIPS Auth:LIBWRAP
2016.05.11 14:43:28 LOG5[38759:140614807435328]: Reading configuration from file /etc/stunnel/stunnel.conf
2016.05.11 14:43:28 LOG5[38759:140614807435328]: FIPS mode is enabled
2016.05.11 14:43:28 LOG4[38759:140614807435328]: Insecure file permissions on /etc/httpd/ssl/cert.pem
2016.05.11 14:43:28 LOG5[38759:140614807435328]: Configuration successful
2016.05.11 14:45:59 LOG5[38765:140614807430912]: Service [hublistener] accepted connection from ***.***.***.***:39124
2016.05.11 14:45:59 LOG5[38765:140614807430912]: connect_blocking: connected 127.0.0.1:8080
2016.05.11 14:45:59 LOG5[38765:140614807430912]: Service [hublistener] connected remote server from 127.0.0.1:59795
Ratchet sample output:
[zxurian#ariel dev-hub]$ php module/HubListener/websocket.php
New Connection! (307)
New Connection! (317)
Connection 317 sending message "second window line 1
" to 1 other connection
Connection 317 sending message "second window line 2
" to 1 other connection
New Connection! (318)
Connection 318 sending message "GET / HTTP/1.1
Host: testserver.testdomain.com:8443
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0 FirePHP/0.7.4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Sec-WebSocket-Version: 13
Origin: https://testserver.testdomain.com
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: PrWlBjR2q6a0syT4oplnAQ==
Cookie: PHPSESSID=ci7102qouvqprhdpk483hv6ar3; exclusive_offers_popup=1; __utma=213556497.27501638.1462814210.1462827970.1462831402.3; __utmc=213556497; __utmz=213556497.1462814210.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmv=213556497.|1=l=%2F=1
x-insight: activate
Connection: keep-alive, Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
" to 2 other connections
Telnet Connection 1
[zxurian#ariel ~]$ telnet 127.0.0.1 8080
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
second window line 1
second window line 2
GET / HTTP/1.1
Host: testserver.testdomain.com:8443
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0 FirePHP/0.7.4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Sec-WebSocket-Version: 13
Origin: https://testserver.testdomain.com
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: PrWlBjR2q6a0syT4oplnAQ==
Cookie: PHPSESSID=ci7102qouvqprhdpk483hv6ar3; exclusive_offers_popup=1; __utma=213556497.27501638.1462814210.1462827970.1462831402.3; __utmc=213556497; __utmz=213556497.1462814210.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmv=213556497.|1=l=%2F=1
x-insight: activate
Connection: keep-alive, Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
I have a Ratchet WebSocket running on my own machine and am also using Apache2.4
To make it work over SSL I did:
Proxy pass the socket
inside (/etc/apache2/)apache2.conf I added
//added for the websocket over SSL
ProxyPass /wss/ ws://alpha.example.com:8080/
this passes my websocket running on port 8080 to wss://alpha.example.com/wss/
to make sure your stunnel config is correct, this is what I did
inside of (/etc/stunnel/)stunnel.conf
cert = /etc/letsencrypt/live/www.example.com/cert.pem
key = /etc/letsencrypt/live/www.example.com/privkey.pem
[wss]
accept = 8082
connect = 8080
REMINDER: change the paths to the keys I used to thee paths your keys are using.
inside the PHP file that starts my server I user port 8080 as in the ratchet example
$webSock->listen(8080, '0.0.0.0'); // Binding to 0.0.0.0 means remotes can connect