Machine Details
AME="Amazon Linux AMI"
VERSION="2017.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2017.03"
PRETTY_NAME="Amazon Linux AMI 2017.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2017.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
I am new to selenium stand alone. I have downloaded the latest java version, stand alone using NPM, XVFB and am trying to create sessions after turning on the standalone using the following command:
sudo xvfb-run --server-args="-screen 0, 1366x768x24" selenium-standalone start -- -debug
Please also note that I have attempted this using a Hub and Node also trying to run Selenium tests using C# on my local client.
When I try to create a Chrome session I receive this error (The polling will continue for a few seconds):
DEBUG - Waiting for [http://localhost:29837/status]
DEBUG - Polling http://localhost:29837/status
/usr/lib/node_modules/selenium-standalone/.selenium/chromedriver/2.32-
x64-chromedriver: error while loading shared libraries: libgconf-2.so.4: cannot open shared object file: No such file or directory
DEBUG - Polling http://localhost:29837/status
DEBUG - Polling http://localhost:29837/status
DEBUG - Polling http://localhost:29837/status
ERROR - org.apache.commons.exec.ExecuteException: Process exited with an error: 127 (Exit value: 127)
^C/usr/bin/xvfb-run: line 171: kill: (6252) - No such process
When trying to create a Firefox session:
14:30:26.416 INFO - Binding default provider to: org.openqa.selenium.chrome.ChromeDriverService
14:30:26.418 INFO - Found handler: org.openqa.selenium.remote.server.BeginSession#44e7a749
14:30:26.418 INFO - /session: Executing POST on /session (handler: BeginSession)
14:30:26.442 DEBUG - Memory-based payload for: {desiredCapabilities={browserName=firefox}}
14:30:26.453 INFO - Capabilities are: Capabilities {browserName=firefox}
14:30:26.457 INFO - Capabilities {browserName=firefox} matched class org.openqa.selenium.remote.server.ServicedSession$Factory (provider: org.openqa.selenium.firefox.GeckoDriverService)
14:30:26.458 INFO - Capabilities {browserName=firefox} matched class org.openqa.selenium.remote.server.ServicedSession$Factory (provider: org.openqa.selenium.chrome.ChromeDriverService)
1506349826497 geckodriver INFO geckodriver 0.18.0
1506349826501 geckodriver INFO Listening on 127.0.0.1:5611
DEBUG - CookieSpec selected: default
DEBUG - Auth cache not set in the context
DEBUG - Connection request: [route: {}->http://localhost:5611][total kept alive: 0; route allocated: 0 of 2000; total allocated: 0 of 2000]
DEBUG - Connection leased: [id: 0][route: {}->http://localhost:5611][total kept alive: 0; route allocated: 1 of 2000; total allocated: 1 of 2000]
DEBUG - Opening connection {}->http://localhost:5611
DEBUG - Connecting to localhost/127.0.0.1:5611
DEBUG - Connection established 127.0.0.1:59276<->127.0.0.1:5611
DEBUG - http-outgoing-0: set socket timeout to 10800000
DEBUG - Executing request POST /session HTTP/1.1
DEBUG - Target auth state: UNCHALLENGED
DEBUG - Proxy auth state: UNCHALLENGED
DEBUG - http-outgoing-0 >> POST /session HTTP/1.1
DEBUG - http-outgoing-0 >> Content-Type: application/json; charset=utf-8
DEBUG - http-outgoing-0 >> Content-Length: 49
DEBUG - http-outgoing-0 >> Host: localhost:5611
DEBUG - http-outgoing-0 >> Connection: Keep-Alive
DEBUG - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_131)
DEBUG - http-outgoing-0 >> Accept-Encoding: gzip,deflate
DEBUG - http-outgoing-0 >> "POST /session HTTP/1.1[\r][\n]"
DEBUG - http-outgoing-0 >> "Content-Type: application/json; charset=utf-8[\r][\n]"
DEBUG - http-outgoing-0 >> "Content-Length: 49[\r][\n]"
DEBUG - http-outgoing-0 >> "Host: localhost:5611[\r][\n]"
DEBUG - http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
DEBUG - http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_131)[\r][\n]"
DEBUG - http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
DEBUG - http-outgoing-0 >> "[\r][\n]"
DEBUG - http-outgoing-0 >> "{"desiredCapabilities":{"browserName":"firefox"}}"
DEBUG - http-outgoing-0 << "HTTP/1.1 500 Internal Server Error[\r][\n]"
DEBUG - http-outgoing-0 << "Connection: close[\r][\n]"
DEBUG - http-outgoing-0 << "Content-Type: application/json; charset=utf-8[\r][\n]"
DEBUG - http-outgoing-0 << "Cache-Control: no-cache[\r][\n]"
DEBUG - http-outgoing-0 << "Content-Length: 1337[\r][\n]"
DEBUG - http-outgoing-0 << "Date: Mon, 25 Sep 2017 14:30:27 GMT[\r][\n]"
DEBUG - http-outgoing-0 << "[\r][\n]"
DEBUG - http-outgoing-0 << "{"value":{"error":"session not created","message":"Expected browser binary location, but unable to find binary in default location, no 'moz:firefoxOptions.binary' capability provided, and no binary flag set on the command line","stacktrace":"stack backtrace:\n 0: 0x5787ed - backtrace::backtrace::trace::h59229d13f6a8837d\n 1: 0x578942 - backtrace::capture::Backtrace::new::h23089c033eded8f0\n 2: 0x44da3d - geckodriver::marionette::MarionetteHandler::create_connection::h6f7058fccafe4367\n 3: 0x425c32 - <webdriver::server::Dispatcher<T, U>>::run::h8f5348b8f5f7c053\n 4: 0x40b22c - std::panicking::try::do_call::hb67c6fb6bcd96195\n 5: 0x5dc20a - panic_unwind::__rust_maybe_catch_panic\n at /checkout/src/libpanic_unwind/lib.rs:98\n 6: 0x41b943 - <F as alloc::boxed::FnBox<A>>::call_box::h4100941edc372034\n 7: 0x5d48a4 - alloc::boxed::{{impl}}::call_once<(),()>\n at /checkout/src/liballoc/boxed.rs:650\n - std::sys_common::thread::start_thread\n at /checkout/src/libstd/sys_common/thread.rs:21\n - std::sys::imp::thread::{{impl}}::new::thread_start\n at /checkout/src/libstd/sys/unix/thread.rs:84"}}"
DEBUG - http-outgoing-0 << HTTP/1.1 500 Internal Server Error
DEBUG - http-outgoing-0 << Connection: close
DEBUG - http-outgoing-0 << Content-Type: application/json; charset=utf-8
DEBUG - http-outgoing-0 << Cache-Control: no-cache
DEBUG - http-outgoing-0 << Content-Length: 1337
DEBUG - http-outgoing-0 << Date: Mon, 25 Sep 2017 14:30:27 GMT
DEBUG - http-outgoing-0: Close connection
DEBUG - Connection discarded
DEBUG - Connection released: [id: 0][route: {}->http://localhost:5611][total kept alive: 0; route allocated: 0 of 2000; total allocated: 0 of 2000]
On trying to run this command to install the latest Stable Google Chrome:
sudo yum -y localinstall google-chrome-stable_current_x86_64.rpm
Or
wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm
I get this error:
Error: Package: google-chrome-stable-61.0.3163.100-1.x86_64 (/google-chrome-stable_current_x86_64)
Requires: libgdk-3.so.0()(64bit)
Error: Package: google-chrome-stable-61.0.3163.100-1.x86_64 (/google-chrome-stable_current_x86_64)
Requires: libXss.so.1()(64bit)
Error: Package: google-chrome-stable-61.0.3163.100-1.x86_64 (/google-chrome-stable_current_x86_64)
Requires: libatk-1.0.so.0()(64bit)
Error: Package: google-chrome-stable-61.0.3163.100-1.x86_64 (/google-chrome-stable_current_x86_64)
Requires: libgdk_pixbuf-2.0.so.0()(64bit)
Error: Package: google-chrome-stable-61.0.3163.100-1.x86_64 (/google-chrome-stable_current_x86_64)
Requires: libgconf-2.so.4()(64bit)
Error: Package: google-chrome-stable-61.0.3163.100-1.x86_64 (/google-chrome-stable_current_x86_64)
Requires: libgtk-3.so.0()(64bit)
Error: Package: google-chrome-stable-61.0.3163.100-1.x86_64 (/google-chrome-stable_current_x86_64)
Requires: xdg-utils
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
Has anyone come across this issue and have anyway to solve this? Please let me know if you have any questions.
I was not able to install the libraries on the Linux AMI but what I ended up doing was installing it on a Ubuntu instance. This made for a pretty simple install using the following doc:
https://www.npmjs.com/package/selenium-standalone
HOWEVER, I received a binary error with my Ubuntu linux as well. The issue was that I did not have the browser installed on my Linux Machine and I was able to do so using the instructions here:
https://askubuntu.com/questions/510056/how-to-install-google-chrome
Works well with FireFox too:
#install firefox browser
sudo apt-get install firefox
Related
I am working on automating a API which is hosted in China server and when I send a request it throws me timeout exception ( org.apache.http.conn.HttpHostConnectException).
My feature file:
Background:
* url 'http://myurl'
* configure connectTimeout = 500000
Scenario: Get Client details
Given path '/clients'
And header Authorization = 'sdssSSLwWDSD'
When method get
Then match response.client_id == 'TestId'
Error details:
11:22:30.347 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection request: [route: {s}->https://myurl.com][total kept alive: 0; route allocated: 0 of 5; total allocated: 0 of 10]
11:22:30.365 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {s}->https://myurl.com][total kept alive: 0; route allocated: 1 of 5; total allocated: 1 of 10]
11:22:30.365 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Opening connection {s}->https://myurl.com
11:22:30.384 [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connecting to myurl.com/54.223.191.33:443
11:22:30.384 [main] DEBUG org.apache.http.conn.ssl.SSLConnectionSocketFactory - Connecting socket to myurl.com/54.223.191.33:443 with timeout 500000
11:22:51.407 [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connect to myurl.com/54.223.191.33:443 timed out. Connection will be retried using another IP address
11:22:51.407 [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connecting to myurl.com/52.80.167.86:443
11:22:51.408 [main] DEBUG org.apache.http.conn.ssl.SSLConnectionSocketFactory - Connecting socket to myurl.com/52.80.167.86:443 with timeout 500000
11:23:12.438 [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-0: Shutdown connection
11:23:12.439 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection discarded
11:23:12.440 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 0][route: {s}->https://myurl.com:443][total kept alive: 0; route allocated: 0 of 5; total allocated: 0 of 10]
11:23:12.440 [main] ERROR com.intuit.karate - org.apache.http.conn.HttpHostConnectException: Connect to myurl.com [myurl.com/54.223.191.33, myurl.com/52.80.167.86] failed: Connection timed out: connect, http call failed after 42094 milliseconds for URL: https://myurl.com/tnc/v1/tnc/all
11:23:12.441 [main] ERROR com.intuit.karate - http request failed:
org.apache.http.conn.HttpHostConnectException: Connect to myurl.com:443 [myurl.com/54.223.191.33, myurl.com/52.80.167.86] failed: Connection timed out: connect
11:23:12.601 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection manager is shutting down
11:23:12.601 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection manager shut down
HTML report: (paste into browser to view) | Karate version: 0.9.4
What I have tried yet:
I tried to configure timeout to max as below:
* configure connectTimeout = 500000
It didn't work.
However, the same request when tried from Postman, works fine and I get response within 2000 ms.
Not sure where I am wrong.
Most likely you have a corporate HTTP proxy.
Refer: https://github.com/intuit/karate#configure
* configure proxy = 'http://my.proxy.host:8080'
I am running a mock service using the karate standalone jar version 1.3.0.
Mocking is working fine.
However, when I make a GET request to the /__admin/stop endpoint, the process registers that the endpoint has been called, closes the listen port, but does not stop.
Execution output and scripts...
on startup:
java -jar /opt/karate/karate.jar -m src/test/java/mocks/fs/john.feature -p 80
16:17:16.671 [main] INFO com.intuit.karate - Karate version: 1.3.0
16:17:17.315 [main] INFO com.intuit.karate - mock server initialized: src/test/java/mocks/fs/john.feature
16:17:17.425 [main] DEBUG com.intuit.karate.http.HttpServer - server started: aad-9mpcfg3:65080
netstat on listen socket:
netstat -na | grep 65080
tcp6 0 0 :::65080 :::* LISTEN
submission of curl commands:
curl -v http://localhost:65080/john/transactionservice/ping
* Trying 127.0.0.1:65080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 65080 (#0)
> GET /john/transactionservice/ping HTTP/1.1
> Host: localhost:65080
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-type: application/json
< content-length: 49
< server: Armeria/1.18.0
< date: Mon, 28 Nov 2022 16:19:37 GMT
<
* Connection #0 to host localhost left intact
{"message":"this is the JOHN TransactionService"}
curl -v http://localhost:65080/__admin/stop
* Trying 127.0.0.1:65080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 65080 (#0)
> GET /__admin/stop HTTP/1.1
> Host: localhost:65080
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 202 Accepted
< content-type: text/plain; charset=utf-8
< content-length: 12
< server: Armeria/1.18.0
< date: Mon, 28 Nov 2022 16:19:54 GMT
<
* Connection #0 to host localhost left intact
output after curl commands:
java -jar /opt/karate/karate.jar -m src/test/java/mocks/fs/john.feature -p 80
16:17:16.671 [main] INFO com.intuit.karate - Karate version: 1.3.0
16:17:17.315 [main] INFO com.intuit.karate - mock server initialized: src/test/java/mocks/fs/john.feature
16:17:17.425 [main] DEBUG com.intuit.karate.http.HttpServer - server started: aad-9mpcfg3:65080
16:19:37.312 [armeria-common-worker-epoll-2-1] DEBUG com.intuit.karate - scenario matched at line 3: pathMatches('john/transactionservice/ping') && methodIs('get')
16:19:54.603 [armeria-common-worker-epoll-2-2] DEBUG com.intuit.karate.http.HttpServer - received command to stop server: /__admin/stop
At this point, netstat on listen port shows socket is closed, but the process continues to run:
ps -ef | grep karate
matt 15070 15069 1 16:17 pts/3 00:00:03 java -jar karate-1.3.0.jar -m src/test/java/mocks/fs/john.feature -p 65080
This is the test mock that I am using (src/test/java/mocks/fs/john.feature):
Feature: JOHN TransactionService mock
Scenario: pathMatches('john/transactionservice/ping') && methodIs('get')
* def response = {}
* set response.message = 'this is the JOHN TransactionService'
* def responseStatus = 200
Question:
Is there something else I should be doing to make the mock process stop? I think I'm following the guidance at https://github.com/karatelabs/karate/tree/master/karate-netty#stopping
Thank you in anticipation of responses.
So I've been using karate for a while now, and there has been an issue we were facing since over the last year: org.apache.http.conn.ConnectTimeoutException
Other threads about that mentioned connectionTimeout exception were solvable by specifying proxy, but taht did not help us.
After tons of investigation, it turned out that our Azure SNAT was exhausted, meaning Karate was opening way too many connections.
To verify this I enabled log debugging and used this feature
Background:
* url "https://www.karatelabs.io/"
Scenario:
* method GET
* method GET
the logs then had following lines
13:10:17.868 [main] DEBUG com.intuit.karate - request:
1 > GET https://www.karatelabs.io/
1 > Host: www.karatelabs.io
1 > Connection: Keep-Alive
1 > User-Agent: Apache-HttpClient/4.5.13 (Java/17.0.4.1)
1 > Accept-Encoding: gzip,deflate
13:10:17.868 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection request: [route: {s}->https://www.karatelabs.io:443][total available: 0; route allocated: 0 of 5; total allocated: 0 of 10]
13:10:17.874 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {s}->https://www.karatelabs.io:443][total available: 0; route allocated: 1 of 5; total allocated: 1 of 10]
13:10:17.875 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Opening connection {s}->https://www.karatelabs.io:443
13:10:17.883 [main] DEBUG o.a.h.i.c.DefaultHttpClientConnectionOperator - Connecting to www.karatelabs.io/34.149.87.45:443
13:10:17.883 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Connecting socket to www.karatelabs.io/34.149.87.45:443 with timeout 30000
13:10:17.924 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Enabled protocols: [TLSv1.3, TLSv1.2]
13:10:17.924 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Enabled cipher suites:[...]
13:10:17.924 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Starting handshake
13:10:18.012 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Secure session established
13:10:18.012 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - negotiated protocol: TLSv1.3
13:10:18.012 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - negotiated cipher suite: TLS_AES_256_GCM_SHA384
13:10:18.012 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - peer principal: CN=karatelabs.io
13:10:18.012 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - peer alternative names: [karatelabs.io, www.karatelabs.io]
13:10:18.012 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - issuer principal: CN=Sectigo RSA Domain Validation Secure Server CA, O=Sectigo Limited, L=Salford, ST=Greater Manchester, C=GB
13:10:18.014 [main] DEBUG o.a.h.i.c.DefaultHttpClientConnectionOperator - Connection established localIp<->serverIp
13:10:18.015 [main] DEBUG o.a.h.i.c.DefaultManagedHttpClientConnection - http-outgoing-0: set socket timeout to 120000
13:10:18.015 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Executing request GET / HTTP/1.1
...
13:10:18.066 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
...
...
13:10:18.196 [main] DEBUG com.intuit.karate - request:
2 > GET https://www.karatelabs.io/
13:10:18.196 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection request: [route: {s}->https://www.karatelabs.io:443][total available: 0; route allocated: 0 of 5; total allocated: 0 of 10]
13:10:18.196 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection leased: [id: 1][route: {s}->https://www.karatelabs.io:443][total available: 0; route allocated: 1 of 5; total allocated: 1 of 10]
13:10:18.196 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Opening connection {s}->https://www.karatelabs.io:443
13:10:18.196 [main] DEBUG o.a.h.i.c.DefaultHttpClientConnectionOperator - Connecting to www.karatelabs.io/34.149.87.45:443
13:10:18.196 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Connecting socket to www.karatelabs.io/34.149.87.45:443 with timeout 30000
13:10:18.206 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Enabled protocols: [TLSv1.3, TLSv1.2]
13:10:18.206 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Enabled cipher suites:[...]
13:10:18.206 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Starting handshake
13:10:18.236 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - Secure session established
13:10:18.236 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - negotiated protocol: TLSv1.3
13:10:18.236 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - negotiated cipher suite: TLS_AES_256_GCM_SHA384
13:10:18.236 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - peer principal: CN=karatelabs.io
13:10:18.236 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - peer alternative names: [karatelabs.io, www.karatelabs.io]
13:10:18.236 [main] DEBUG o.a.h.c.s.SSLConnectionSocketFactory - issuer principal: CN=Sectigo RSA Domain Validation Secure Server CA, O=Sectigo Limited, L=Salford, ST=Greater Manchester, C=GB
13:10:18.236 [main] DEBUG o.a.h.i.c.DefaultHttpClientConnectionOperator - Connection established localIp<->serverIp
13:10:18.236 [main] DEBUG o.a.h.i.c.DefaultManagedHttpClientConnection - http-outgoing-1: set socket timeout to 120000
...
13:10:18.279 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
...
...
13:10:18.609 [Finalizer] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection manager is shutting down
13:10:18.610 [Finalizer] DEBUG o.a.h.i.c.DefaultManagedHttpClientConnection - http-outgoing-1: Shutdown connection
13:10:18.611 [Finalizer] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection manager shut down
13:10:18.612 [Finalizer] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection manager is shutting down
13:10:18.612 [Finalizer] DEBUG o.a.h.i.c.DefaultManagedHttpClientConnection - http-outgoing-2: Shutdown connection
13:10:18.612 [Finalizer] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection manager shut down
13:10:18.612 [Finalizer] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection manager is shutting down
"Connecting to socket" and "handshake" indicate that karate is establishing a new connection instead of using an already opened one, even though I am sending a request to the same host.
On the other hand, on longer scenarios, I was seeing "http-outgoing-x: Shutdown connection" after about ~1s from opening it, in the middle of the run, despite having "karate.configure('readTimeout', 120000)" specified.
I don't think that was intentional, especially after seeing the "keep-alive" header and the "Connection can be kept alive indefinitely" in the log"
That being said, is there any way to force karate to use the same connection instead of establishing a new one each request?
As far as we know, we use the Apache HTTP Client API the right way.
But you never know. The best thing is for you to dive into the code and see what we could be missing. Or you could provide a way to replicate following these instructions: https://github.com/karatelabs/karate/wiki/How-to-Submit-an-Issue
I'm having a problem with sending a POST request to Salesforce's web2lead servlet through Microsoft TMG and httpclient 4.3.3. If I try the request using Ctnlm as a proxy, it works fine (that proxies through TMG). Also, if I simply make a GET request to www.google.com through the Proxy, it's working fine. Here is the code that I'm trying to use:
HttpHost httpTargetHost = new HttpHost("www.salesforce.com", 443, "https");
HttpPost httpPost = new HttpPost("/servlet/servlet.WebToLead");
NTCredentials ntCreds = new NTCredentials("proxyUser", "proxyPassword", "workstation", "DOMAIN");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope("proxyHost","proxyPort"), ntCreds );
HttpClientBuilder clientBuilder = HttpClientBuilder.create();
clientBuilder.useSystemProperties();
clientBuilder.setProxy(new HttpHost("proxyHost","proxyPort"));
clientBuilder.setDefaultCredentialsProvider(credsProvider);
clientBuilder.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy());
CloseableHttpClient client = clientBuilder.build();
CloseableHttpResponse response = client.execute(httpTargetHost, httpPost);
In the response, I'm getting this:
2015-01-16 17:33:31,158 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:113) - http-outgoing-0 << HTTP/1.1 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )
2015-01-16 17:33:31,158 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Via: 1.1 PROXYHOST
2015-01-16 17:33:31,159 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Proxy-Authenticate: Negotiate
2015-01-16 17:33:31,159 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Proxy-Authenticate: Kerberos
2015-01-16 17:33:31,159 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Proxy-Authenticate: NTLM
2015-01-16 17:33:31,160 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Connection: close
2015-01-16 17:33:31,160 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Proxy-Connection: close
2015-01-16 17:33:31,160 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Pragma: no-cache
2015-01-16 17:33:31,160 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Cache-Control: no-cache
2015-01-16 17:33:31,161 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Content-Type: text/html
2015-01-16 17:33:31,161 [main] DEBUG org.apache.http.headers(LoggingManagedHttpClientConnection.java:116) - http-outgoing-0 << Content-Length: 2699
2015-01-16 17:33:31,164 [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator(HttpAuthenticator.java:77) - Authentication required
Thanks for any help you can provide.
Regards,
Eric
The error message says quite clearly:
Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied.
You need some kind of authorization which you don't provide. This problem is not specific to Apache HTTPClient. Please check with the administrator of the Forefront TMG installation what kind if authorization (and which credentials) you need. It might be the kind of authorization you've tried and only the credentials are wrong or it might be some other kind of authorization required.
I'm trying to download the vcard list from my zimbra account using iCal4j.
I tried with this code.
It correctly fetches the collections but it doesn't download any vCard.
If i debug the getCollections method, i can see a HTTP/1.1 400 Bad Request error.
Can anyone help ?
Regards
Stefano
CardDavStore kerioStore = new CardDavStore("//MacTI/MKS2Exchange", new URL("https://xxx.xxx.xxx"), PathResolver.ZIMBRA);
kerioStore.connect("xxx.xxx#xxx.xxx", "supersecret".toCharArray());
for (CardDavCollection collection: kerioStore.getCollections()) {
for (VCard card: collection.getComponents()) {
System.out.println(card.toString());
}
}
This is the HTTP request trace:
DEBUG [main] HttpMethodBase.shouldCloseConnection(1024) | Resorting to protocol version default close connection policy
DEBUG [main] HttpMethodBase.shouldCloseConnection(1028) | Should NOT close connection, using HTTP/1.1
DEBUG [main] HttpConnection.releaseConnection(1178) | Releasing connection back to connection manager.
DEBUG [main] header.wire(70) | >> "REPORT /dav/<myemailaccount>/TestCardDav/ HTTP/1.1[\r][\n]"
DEBUG [main] HttpMethodBase.addHostRequestHeader(1352) | Adding Host request header
DEBUG [main] header.wire(70) | >> "Depth: 1[\r][\n]"
DEBUG [main] header.wire(70) | >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
DEBUG [main] header.wire(70) | >> "Host: xxx.xxx.xxx[\r][\n]"
DEBUG [main] header.wire(70) | >> "Content-Length: 199[\r][\n]"
DEBUG [main] header.wire(70) | >> "Content-Type: text/xml; charset=UTF-8[\r][\n]"
DEBUG [main] header.wire(70) | >> "[\r][\n]"
DEBUG [main] content.wire(84) | >> "<?xml version="1.0" encoding="UTF-8" standalone="no"?><C:addressbook-query xmlns:C="urn:ietf:params:xml:ns:carddav"><D:prop xmlns:D="DAV:"><C:address-data/><D:getetag/></D:prop></C:addressbook-query>"
DEBUG [main] EntityEnclosingMethod.writeRequestBody(508) | Request body sent
DEBUG [main] header.wire(70) | << "HTTP/1.1 401 must authenticate[\r][\n]"
DEBUG [main] header.wire(70) | << "HTTP/1.1 401 must authenticate[\r][\n]"
DEBUG [main] header.wire(70) | << "Date: Wed, 04 Dec 2013 22:16:48 GMT[\r][\n]"
DEBUG [main] header.wire(70) | << "WWW-Authenticate: BASIC realm="Zimbra"[\r][\n]"
DEBUG [main] header.wire(70) | << "Content-Length: 0[\r][\n]"
DEBUG [main] header.wire(70) | << "[\r][\n]"
DEBUG [main] HttpMethodDirector.isAuthenticationNeeded(843) | Authorization required
DEBUG [main] AuthChallengeProcessor.selectAuthScheme(90) | Supported authentication schemes in the order of preference: [Digest, Basic]
DEBUG [main] AuthChallengeProcessor.selectAuthScheme(111) | Challenge for Digest authentication scheme not available
INFO [main] AuthChallengeProcessor.selectAuthScheme(101) | Basic authentication scheme selected
DEBUG [main] AuthChallengeProcessor.processChallenge(155) | Using authentication scheme: basic
DEBUG [main] AuthChallengeProcessor.processChallenge(163) | Authorization challenge processed
DEBUG [main] HttpMethodDirector.processWWWAuthChallenge(714) | Authentication scope: BASIC 'Zimbra'#mail.zcsmail.eu:443
DEBUG [main] HttpMethodDirector.executeMethod(194) | Retry authentication
DEBUG [main] HttpMethodBase.shouldCloseConnection(1024) | Resorting to protocol version default close connection policy
DEBUG [main] HttpMethodBase.shouldCloseConnection(1028) | Should NOT close connection, using HTTP/1.1
DEBUG [main] HttpConnection.releaseConnection(1176) | Connection is locked. Call to releaseConnection() ignored.
DEBUG [main] HttpMethodDirector.authenticateHost(278) | Authenticating with BASIC 'Zimbra'#xxx.xxx.xxx:443
DEBUG [main] HttpMethodParams.getCredentialCharset(384) | Credential charset not configured, using HTTP element charset
DEBUG [main] header.wire(70) | >> "REPORT /dav/<myemailaccount>/TestCardDav/ HTTP/1.1[\r][\n]"
DEBUG [main] HttpMethodBase.addHostRequestHeader(1352) | Adding Host request header
DEBUG [main] header.wire(70) | >> "Depth: 1[\r][\n]"
DEBUG [main] header.wire(70) | >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
DEBUG [main] header.wire(70) | >> "Content-Length: 199[\r][\n]"
DEBUG [main] header.wire(70) | >> "Content-Type: text/xml; charset=UTF-8[\r][\n]"
DEBUG [main] header.wire(70) | >> "Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==[\r][\n]"
DEBUG [main] header.wire(70) | >> "Host: xxx.xxx.xxx[\r][\n]"
DEBUG [main] header.wire(70) | >> "[\r][\n]"
DEBUG [main] content.wire(84) | >> "<?xml version="1.0" encoding="UTF-8" standalone="no"?><C:addressbook-query xmlns:C="urn:ietf:params:xml:ns:carddav"><D:prop xmlns:D="DAV:"><C:address-data/><D:getetag/></D:prop></C:addressbook-query>"
DEBUG [main] EntityEnclosingMethod.writeRequestBody(508) | Request body sent
DEBUG [main] header.wire(70) | << "HTTP/1.1 400 Bad Request[\r][\n]"
DEBUG [main] header.wire(70) | << "HTTP/1.1 400 Bad Request[\r][\n]"
DEBUG [main] header.wire(70) | << "Date: Wed, 04 Dec 2013 22:16:49 GMT[\r][\n]"
DEBUG [main] header.wire(70) | << "Content-Length: 0[\r][\n]"
DEBUG [main] header.wire(70) | << "[\r][\n]"