We have a Redis cluster(running in Sentinel mode) with 3 nodes and we are using Redisson (version 3.10.7) to interact with Redis.
We are facing an issue wherein the client application is not able to connect to the cluster. Continuously older connections are being dropped and new connections are being created. The same thing is happening every second. Below is the snapshot for the logger.
12:01:00.832 [redisson-netty-2-6] DEBUG org.redisson.connection.ClientConnectionsEntry - new connection created: RedisConnection#483495011 [redisClient=[addr=redis://10.79.9.163:6379], channel=[id: 0x767f3c87, L:/192.168.7.24:60173 - R:10.79.9.163/10.79.9.163:6379], command=null]
12:01:00.832 [redisson-netty-2-6] DEBUG org.redisson.connection.ClientConnectionsEntry - new connection created: RedisConnection#1510307492 [redisClient=[addr=redis://10.79.9.163:6379], channel=[id: 0xa8c01128, L:/192.168.7.24:60183 - R:10.79.9.163/10.79.9.163:6379], command=null]
12:01:00.832 [redisson-netty-2-6] DEBUG org.redisson.connection.ClientConnectionsEntry - new connection created: RedisConnection#425141862 [redisClient=[addr=redis://10.79.9.164:6379], channel=[id: 0x19afdfb8, L:/192.168.7.24:60199 - R:10.79.9.164/10.79.9.164:6379], command=null]
12:01:00.832 [redisson-netty-2-6] DEBUG org.redisson.connection.ClientConnectionsEntry - new connection created: RedisConnection#457747719 [redisClient=[addr=redis://10.79.9.164:6379], channel=[id: 0xf2431dfb, L:/192.168.7.24:60211 - R:10.79.9.164/10.79.9.164:6379], command=null]
12:01:00.832 [redisson-netty-2-6] DEBUG org.redisson.connection.ClientConnectionsEntry - new connection created: RedisConnection#1595469651 [redisClient=[addr=redis://10.79.9.164:6379], channel=[id: 0x3226faec, L:/192.168.7.24:60204 - R:10.79.9.164/10.79.9.164:6379], command=null]
12:01:00.922 [pool-2-thread-1] DEBUG org.redisson.client.handler.ConnectionWatchdog - reconnecting RedisConnection#541871189 [redisClient=[addr=redis://10.79.9.164:6379], channel=[id: 0x5396b42d, L:/192.168.7.24:59969 ! R:10.79.9.164/10.79.9.164:6379], command=CommandData [promise=RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise#3ecd3a79(failure: java.util.concurrent.CancellationException)], command=(PING), params=[], codec=org.redisson.client.codec.StringCodec]] to 10.79.9.164/10.79.9.164:6379
12:01:00.929 [pool-2-thread-1] DEBUG org.redisson.client.handler.ConnectionWatchdog - reconnecting RedisConnection#495642036 [redisClient=[addr=redis://10.79.9.164:6379], channel=[id: 0xfa2f4f52, L:/192.168.7.24:59973 ! R:10.79.9.164/10.79.9.164:6379], command=CommandData [promise=RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise#28c788eb(failure: java.util.concurrent.CancellationException)], command=(PING), params=[], codec=org.redisson.client.codec.StringCodec]] to 10.79.9.164/10.79.9.164:6379
12:01:00.931 [pool-2-thread-1] DEBUG org.redisson.client.handler.ConnectionWatchdog - reconnecting RedisConnection#169281839 [redisClient=[addr=redis://10.79.9.164:6379], channel=[id: 0xf365bc68, L:/192.168.7.24:59976 ! R:10.79.9.164/10.79.9.164:6379], command=CommandData [promise=RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise#6bd0d406(failure: java.util.concurrent.CancellationException)], command=(PING), params=[], codec=org.redisson.client.codec.StringCodec]] to 10.79.9.164/10.79.9.164:6379
12:01:00.931 [pool-2-thread-1] DEBUG org.redisson.client.handler.ConnectionWatchdog - reconnecting RedisConnection#65640722 [redisClient=[addr=redis://10.79.9.165:6379], channel=[id: 0x03c68dcf, L:/192.168.7.24:59226 ! R:10.79.9.165/10.79.9.165:6379], command=CommandData [promise=RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise#520c88b3(failure: java.util.concurrent.CancellationException)], command=(PING), params=[], codec=org.redisson.client.codec.StringCodec]] to 10.79.9.165/10.79.9.165:6379
12:01:00.932 [pool-2-thread-1] DEBUG org.redisson.client.handler.ConnectionWatchdog - reconnecting RedisConnection#1365249560 [redisClient=[addr=redis://10.79.9.165:6379], channel=[id: 0x0722fcc6, L:/192.168.7.24:59262 ! R:10.79.9.165/10.79.9.165:6379], command=CommandData [promise=RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise#50c8b29f(failure: java.util.concurrent.CancellationException)], command=(PING), params=[], codec=org.redisson.client.codec.StringCodec]] to 10.79.9.165/10.79.9.165:6379
12:01:01.080 [redisson-netty-2-8] DEBUG org.redisson.connection.ClientConnectionsEntry - new connection created: RedisConnection#1230181180 [redisClient=[addr=redis://10.79.9.163:6379], channel=[id: 0x148352d3, L:/192.168.7.24:60224 - R:10.79.9.163/10.79.9.163:6379], command=CommandData [promise=RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise#19cb007f(incomplete)], command=(PING), params=[], codec=org.redisson.client.codec.StringCodec]]
12:01:01.082 [redisson-netty-2-5] DEBUG org.redisson.connection.ClientConnectionsEntry - new connection created: RedisConnection#1221343292 [redisClient=[addr=redis://10.79.9.163:6379], channel=[id: 0xc4ff65cd, L:/192.168.7.24:60229 - R:10.79.9.163/10.79.9.163:6379], command=CommandData [promise=RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise#1891a694(incomplete)], command=(PING), params=[], codec=org.redisson.client.codec.StringCodec]]
12:01:01.082 [redisson-netty-2-7] DEBUG org.redisson.connection.ClientConnectionsEntry - new connection created: RedisConnection#2036916006 [redisClient=[addr=redis://10.79.9.163:6379], channel=[id: 0xff04b8ab, L:/192.168.7.24:60223 - R:10.79.9.163/10.79.9.163:6379], command=CommandData [promise=RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise#1618e7eb(incomplete)], command=(PING), params=[], codec=org.redisson.client.codec.StringCodec]]
12:01:01.082 [redisson-netty-2-9] DEBUG org.redisson.connection.ClientConnectionsEntry - new connection created: RedisConnection#1460667823 [redisClient=[addr=redis://10.79.9.163:6379], channel=[id: 0xbdc0d69e, L:/192.168.7.24:60222 - R:10.79.9.163/10.79.9.163:6379], command=CommandData [promise=RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise#139695d9(incomplete)], command=(PING), params=[], codec=org.redisson.client.codec.StringCodec]]
We tried below points to investigate the issue:
Restart the client application.
Restart Redis cluster nodes.
Post these restarts as well, the same issue persisted.
We were able to resolve the issue by changing the timeout property from 0 to 10 seconds in server.conf of Redis. On reverting the timeout back to 0 also is not causing the issue to reoccur again.
Anyone else faced similar kind of issue?. If yes, do we know the reason behind it?
Related
I am trying to run my protractor scripts on jenkins pipeline and getting the SessionNotCreatedError when it tries to access the Selenium address. When I run the same on local its working fine. Not sure what's going on.
Conf.js script
let domainName = util.domainName;
exports.config = {
seleniumAddress: 'http://selenium-hub:4444/wd/hub', //for testing in the pipeline
directConnect: false, // Set to true for local testing, or provide a link to a running selenium grid.
// specs: ['e2e/**/mailbox-test.js', 'e2e/**/email-dumps-test.js'],
specs: ['e2e/**/*-test.js'],
capabilities: {
browserName: 'chrome',
acceptInsecureCerts: true,
'goog:chromeOptions': {
w3c: false,
args: [
'--no-sandbox',
'--headless',
'--disable-gpu',
'--window-size=1200,1200',
'--disable-dev-shm-usage',
'--allow-insecure-localhost',
'--allow-running-insecure-content',
'--ignore_ssl',
'--user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36',
],
},
},
allScriptsTimeout: 600000,
baseUrl: eaUrl,
framework: 'jasmine',
jasmineNodeOpts: {
isVerbose: true,
showColors: true,
includeStackTrace: true,
defaultTimeoutInterval: 600000,
stackTrace: false
},
Jenkins output error
[15:32:21] D/launcher - Protractor version: 5.4.2 10:31:59 [15:32:21]
D/launcher - Your base url for tests is
https://ea-webapp-int-raven.ocp-nonprod.ice.dhs.gov/ 10:31:59
[15:32:21] I/launcher - Running 1 instances of WebDriver 10:31:59
[15:32:21] I/hosted - Using the selenium server at
http://selenium-hub:4444/wd/hub 10:37:05 [15:37:21] E/launcher -
Could not start a new session. New session request timed out 10:37:05
Build info: version: '4.1.2', revision: '9a5a329c5a' 10:37:05 System
info: host: 'eb3088c85a9d', ip: '172.17.0.2', os.name: 'Linux',
os.arch: 'amd64', os.version: '3.10.0-1160.53.1.el7.x86_64',
java.version: '11.0.13' 10:37:05 Driver info: driver.version: unknown
10:37:05 [15:37:21] E/launcher - SessionNotCreatedError: Could not
start a new session. New session request timed out 10:37:05 Build
info: version: '4.1.2', revision: '9a5a329c5a' 10:37:05 System info:
host: 'eb3088c85a9d', ip: '172.17.0.2', os.name: 'Linux', os.arch:
'amd64', os.version: '3.10.0-1160.53.1.el7.x86_64', java.version:
'11.0.13' 10:37:05 Driver info: driver.version: unknown 10:37:05
at Object.throwDecodedError
(/app/test/node_modules/selenium-webdriver/lib/error.js:514:15)
10:37:05 at parseHttpResponse
(/app/test/node_modules/selenium-webdriver/lib/http.js:519:13)
10:37:05 at doSend.then.response
(/app/test/node_modules/selenium-webdriver/lib/http.js:441:30)
10:37:05 at process._tickCallback
(internal/process/next_tick.js:68:7) 10:37:05 From: Task:
WebDriver.createSession() 10:37:05 at Function.createSession
(/app/test/node_modules/selenium-webdriver/lib/webdriver.js:769:24)
10:37:05 at Function.createSession
(/app/test/node_modules/selenium-webdriver/chrome.js:761:15) 10:37:05
at createDriver
(/app/test/node_modules/selenium-webdriver/index.js:170:33) 10:37:05
at Builder.build
(/app/test/node_modules/selenium-webdriver/index.js:626:16) 10:37:05
at Hosted.getNewDriver
(/app/test/node_modules/protractor/built/driverProviders/driverProvider.js:53:33)
10:37:05 at Runner.createBrowser
(/app/test/node_modules/protractor/built/runner.js:195:43) 10:37:05
at q.then.then
(/app/test/node_modules/protractor/built/runner.js:339:29) 10:37:05
at _fulfilled (/app/test/node_modules/q/q.js:834:54) 10:37:05 at
/app/test/node_modules/q/q.js:863:30 10:37:05 at
Promise.promise.promiseDispatch (/app/test/node_modules/q/q.js:796:13)
10:37:05 [15:37:21] E/launcher - Process exited with error code 199
10:37:05 10:37:05 exitStatus=$? 10:37:05 10:37:05 # Allow the
Jenkin pipeline to access and archive any test output. 10:37:05 chmod
-R 777 ./test_output/
I am adding my observations as others might need if they get similar issues and this might help them too. After a long long investigation of the console output in Jenkins with my docker installation scripts, I noticed that we are trying to install the latest version of the selenium hub and chrome node but the docker command doesn't include the SE_EVENT_BUS_HOST information. After changing my code from
docker run -d --name selenium-node-chrome -e HUB_HOST=selenium-hub --shm-size="2g" nexus2.xxx.com:18443/selenium/node-chrome
to
docker run -d --name selenium-node-chrome -e SE_EVENT_BUS_HOST=selenium-hub --shm-size="2g" -e SE_EVENT_BUS_PUBLISH_PORT=4442 -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 nexus2.xxx.com:18443/selenium/node-chrome
Then it started working for me.
I want to configure a proxy with Nexus for a private repository in cloudsmith.io.
When I configure the proxy with the public npm registry everything works good, but when I configure the proxy with the Cloudsmith repository the command npm install fails.
To authenticate with Cloudsmith (without proxy), the .npmrc file must contains this:
registry=https://npm.cloudsmith.io/<my org>/<my repo>/
//npm.cloudsmith.io/<my org>/<my repo>/:username=<my user>
//npm.cloudsmith.io/<my org>/<my repo>/:_password=<my password in base64>
So I on the proxy configuration I enabled the http authentication, but a 404 is being returned.
have you idea if I missed somenthing on the proxy configuration?
Maybe Nexus can't deal with the Cloudsmith authentication.
Update:
Nexus logs:
2019-06-21 14:10:58,736+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.client.protocol.RequestAddCookies - CookieSpec selected: ignoreCookies
2019-06-21 14:10:58,736+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.client.protocol.RequestAuthCache - Auth cache not set in the context
2019-06-21 14:10:58,737+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.execchain.MainClientExec - Opening connection {s}->https://npm.cloudsmith.io:443
2019-06-21 14:10:58,840+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connecting to npm.cloudsmith.io/13.249.122.86:443
2019-06-21 14:10:58,840+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - Connecting socket to npm.cloudsmith.io/13.249.122.86:443 with timeout 20000
2019-06-21 14:10:58,841+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
2019-06-21 14:10:58,841+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
2019-06-21 14:10:58,841+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - Starting handshake
2019-06-21 14:10:59,270+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - Secure session established
2019-06-21 14:10:59,270+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - negotiated protocol: TLSv1.2
2019-06-21 14:10:59,270+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
2019-06-21 14:10:59,270+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - peer principal: CN=*.cloudsmith.io
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - peer alternative names: [*.cloudsmith.io, cloudsmith.io]
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.conn.ssl.SSLConnectionSocketFactory - issuer principal: CN=Amazon, OU=Server CA 1B, O=Amazon, C=US
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connection established 172.19.0.2:48268<->13.249.122.86:443
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-52: set socket timeout to 20000
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.execchain.MainClientExec - Executing request GET /myorganization/javascript/cordova HTTP/1.1
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.execchain.MainClientExec - Target auth state: UNCHALLENGED
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 >> GET /myorganization/javascript/cordova HTTP/1.1
2019-06-21 14:10:59,272+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 >> Host: npm.cloudsmith.io
2019-06-21 14:10:59,272+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 >> Connection: Keep-Alive
2019-06-21 14:10:59,272+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 >> User-Agent: Nexus/3.16.2-01 (OSS; Linux; 4.4.0-139-generic; amd64; 1.8.0_212)
2019-06-21 14:10:59,272+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 >> Accept-Encoding: gzip,deflate
2019-06-21 14:10:59,804+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << HTTP/1.1 404 Not Found
2019-06-21 14:10:59,804+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Content-Type: application/json
2019-06-21 14:10:59,805+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Content-Length: 71
2019-06-21 14:10:59,805+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Connection: keep-alive
2019-06-21 14:10:59,805+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Date: Fri, 21 Jun 2019 14:10:59 GMT
2019-06-21 14:10:59,805+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Allow: GET, PUT, HEAD, OPTIONS
2019-06-21 14:10:59,805+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << ETag: "2d642f304a79cbf4e5fe6270af0846ca"
2019-06-21 14:10:59,805+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Vary: Cookie
2019-06-21 14:10:59,805+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Server: Cloudsmith MCP
2019-06-21 14:10:59,805+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Content-Security-Policy: default-src 'self'; child-src 'self' https://fast.wistia.net https://giphy.com https://intercom-sheets.com https://js.stripe.com https://player.vimeo.com https://share.intercom.io https://www.google.com/recaptcha/ https://www.intercom-reporting.com https://www.youtube.com; connect-src 'self' https://api-iam.intercom.io https://api.intercom.io https://api-ping.intercom.io https://api.stripe.com https://app.getsentry.com https://bam.nr-data.net https://www.google-analytics.com https://js.intercomcdn.com https://nexus-long-poller-a.intercom.io https://nexus-long-poller-b.intercom.io https://nexus-websocket-a.intercom.io https://nexus-websocket-b.intercom.io https://rs.fullstory.com https://sentry.io/api/ https://stats.g.doubleclick.net https://uploads.intercomcdn.com https://uploads.intercomusercontent.com https://widget.intercom.io https://yt0blqw1vlv7.statuspage.io wss://nexus-websocket-a.intercom.io wss://nexus-websocket-b.intercom.io https://api.cloudsmith.io https://api-prd.cloudsmith.io https://cloudsmith-package-uploads-prd.s3.amazonaws.com https://cloudsmith-package-uploads-prd.s3-accelerate.amazonaws.com; font-src 'self' data: https://js.intercomcdn.com https://fonts.gstatic.com https://assets.cloudsmith.media; frame-src 'self' https://fast.wistia.net https://giphy.com https://intercom-sheets.com https://js.stripe.com https://player.vimeo.com https://share.intercom.io https://www.google.com/recaptcha/ https://www.intercom-reporting.com https://www.youtube.com; img-src 'self' data: https: https://downloads.intercomcdn.com https://gifs.intercomcdn.com https://gravatar.com https://img.shields.io https://js.intercomcdn.com https://static.intercomassets.com https://uploads.intercomusercontent.com https://assets.cloudsmith.media https://prd.cloudsmith.media https://users.cloudsmith.media; media-src 'self' https://giphy.com https://js.intercomcdn.com https://assets.cloudsmith.media https://prd.cloudsmith.media https://users.cloudsmith.media; object-src 'self'; script-src 'self' 'unsafe-inline' data: https://api.stripe.com https://app.intercom.io https://bam.nr-data.net https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/ https://cdn.ravenjs.com https://connect.facebook.net https://fullstory.com https://js.intercomcdn.com https://js.stripe.com https://maps.googleapis.com https://maps.gstatic.com https://nexus-long-poller-a.intercom.io https://nexus-long-poller-b.intercom.io https://rum-static.pingdom.net https://sentry.io/api/ https://sjs.bizographics.com https://tagmanager.google.com https://tpc.googlesyndication.com https://translate.google.com https://widget.intercom.io https://www.googleadservices.com https://www.google-analytics.com https://www.google.com/recaptcha/ https://www.googletagmanager.com https://www.gstatic.com/recaptcha/ https://assets.cloudsmith.media; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://tagmanager.google.com https://assets.cloudsmith.media; worker-src 'self' blob: https://assets.cloudsmith.media; form-action 'self' https://api-iam.intercom.io https://messenger-apps.intercom.io https://intercom.help https://cloudsmith.io https://www.cloudsmith.io https://prd.cloudsmith.io https://web-prd.cloudsmith.io; report-uri https://sentry.io/api/195170/csp-report/?sentry_key=85cf84319e204994878d7afb03753450
2019-06-21 14:10:59,805+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Public-Key-Pins: pin-sha256="++MBgDH5WGvL9Bcn5Be30cRcL0f5O+NyoXuWtQdX1aI="; pin-sha256="klO23nT2ehFDXCfx3eHTDRESMz3asj1muO+4aIdjiuY="; pin-sha256="tFOU95aPSUFNfZLpf6OdBnVvAFKxd8zxp4jCbaL3MJ0="; max-age=86400; report-uri="https://cloudsmith.report-uri.io/r/default/hpkp/enforce"
2019-06-21 14:10:59,806+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Strict-Transport-Security: max-age=5184000
2019-06-21 14:10:59,806+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Referrer-Policy: no-referrer-when-downgrade
2019-06-21 14:10:59,806+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Expect-CT: enforce, max-age=86400, report-uri="https://cloudsmith.report-uri.io/r/default/ct/enforce"
2019-06-21 14:10:59,806+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << X-Cache: Error from cloudfront
2019-06-21 14:10:59,806+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << Via: 1.1 24990d51e53375dffbe8411f5e14f579.cloudfront.net (CloudFront)
2019-06-21 14:10:59,806+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << X-Amz-Cf-Pop: ATL51-C1
2019-06-21 14:10:59,806+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.headers - http-outgoing-52 << X-Amz-Cf-Id: _igVXHch2JQA54gLS1Btz7iTlbtc1W4PQ7S_Jrc4sKYvFwvMsJ5VBg==
2019-06-21 14:10:59,807+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.execchain.MainClientExec - Connection can be kept alive for 30000 MILLISECONDS
2019-06-21 14:10:59,807+0000 INFO [qtp2116157452-1644] admin org.sonatype.nexus.repository.httpclient.internal.HttpClientFacetImpl - Repository status for myorganization-javascript-proxy changed from READY to AVAILABLE - reason n/a for n/a
2019-06-21 14:10:59,807+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-52: set socket timeout to 0
The answer is that Sonatype Nexus expects the upstream request to be challenged with a "401 Unauthorized" response + a valid WWW-Authenticate header, rather than a "404 Not Found" response.
The indication that this is the issue is in the log output at:
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.execchain.MainClientExec - Target auth state: UNCHALLENGED
2019-06-21 14:10:59,271+0000 DEBUG [qtp2116157452-1644] admin org.apache.http.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED
Here, UNCHALLENGED states that Nexus will not be providing credentials, neither pre-emptively nor as a follow-up request. So if the target upstream is private and requires authentication, as in this case, this will cause the proxying to fail.
To fix this, the upstream will need to fix the response to return "401 Unauthorized" for the protected endpoint. This will ensure that the authentication is passed through to the target upstream. Assuming your credentials are correct, you should be authenticated.
It's also worth noting that, as of writing, Nexus does not support Bearer-based (token) authentication [1], so it isn't compatible with the npm login workflow. So make sure authentication is configured directly in .npmrc as username/password. For example:
registry=https://npm.cloudsmith.io/<my org>/<my repo>/
//npm.cloudsmith.io/<my org>/<my repo>/:username=<my user>
//npm.cloudsmith.io/<my org>/<my repo>/:_password=<my password in base64>
[1] https://issues.sonatype.org/browse/NEXUS-12456
macOS: version 10.13.4
Safari: version 11.1
Selenium-standalone: version: 6.14.1
Summary: I'm getting "Cannot establish new session" in the driver logs when running automation on an instance of selenium-standalone that was started through a SSH.
Note: If you start selenium-standalone directly on the mac machine without SSH ing everything works perfectly
Steps to reproduce:
1) SSH into a mac device.
2) start selenium-standalone
command: selenium-standalone start -- -role node -nodeConfig safari.json
safari.json:
{
"capabilities":
[
{
"browserName": "safari",
"platform": "MAC",
"maxInstances": 1,
"seleniumProtocol": "WebDriver"
}
],
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"maxSession": 1,
"port": 5555,
"register": true,
"registerCycle": 5000,
"hub": "http://ReplaceWithHubUrl:4449",
"nodeStatusCheckTimeout": 5000,
"nodePolling": 5000,
"role": "node",
"host":"ReplacewithNodeIP",
"unregisterIfStillDownAfter": 60000,
"downPollingLimit": 2,
"debug": false,
"servlets" : [],
"withoutServlets": [],
"custom": {}
}
3) Run the automation (Of course pointed toward the hub)
selenium server logs:
12:12:41.490 INFO - Selenium build info: version: '3.8.1', revision: '6e95a6684b'
12:12:41.491 INFO - Launching a Selenium Grid node
2018-05-24 12:12:44.846:INFO::main: Logging initialized #3650ms to org.seleniumhq.jetty9.util.log.StdErrLog
12:12:44.871 INFO - Using `new FirefoxOptions()` is preferred to `DesiredCapabilities.firefox()`
12:12:44.877 INFO - Using `new ChromeOptions()` is preferred to `DesiredCapabilities.chrome()`
12:12:44.879 INFO - Using `new EdgeOptions()` is preferred to `DesiredCapabilities.edge()`
12:12:44.879 INFO - Driver class not found: com.opera.core.systems.OperaDriver
12:12:44.879 INFO - Using `new OperaOptions()` is preferred to `DesiredCapabilities.operaBlink()`
12:12:44.880 INFO - Using `new SafariOptions()` is preferred to `DesiredCapabilities.safari()`
12:12:44.880 INFO - Driver class not found: org.openqa.selenium.phantomjs.PhantomJSDriver
12:12:44.890 INFO - Driver provider class org.openqa.selenium.ie.InternetExplorerDriver registration is skipped:
registration capabilities Capabilities {browserName: internet explorer, ensureCleanSession: true, platform: WINDOWS, version: } does not match the current platform MAC
12:12:44.890 INFO - Driver provider class org.openqa.selenium.edge.EdgeDriver registration is skipped:
registration capabilities Capabilities {browserName: MicrosoftEdge, platform: WINDOWS, version: } does not match the current platform MAC
12:12:44.911 INFO - Using `new ChromeOptions()` is preferred to `DesiredCapabilities.chrome()`
12:12:44.911 INFO - Using `new EdgeOptions()` is preferred to `DesiredCapabilities.edge()`
12:12:44.911 INFO - Using `new FirefoxOptions()` is preferred to `DesiredCapabilities.firefox()`
12:12:44.912 INFO - Using `new OperaOptions()` is preferred to `DesiredCapabilities.operaBlink()`
12:12:44.912 INFO - Using `new SafariOptions()` is preferred to `DesiredCapabilities.safari()`
12:12:44.917 INFO - Using the passthrough mode handler
2018-05-24 12:12:44.934:INFO:osjs.Server:main: jetty-9.4.7.v20170914
2018-05-24 12:12:44.950:WARN:osjs.SecurityHandler:main: ServletContext#o.s.j.s.ServletContextHandler#ba54932{/,null,STARTING} has uncovered http methods for path: /
2018-05-24 12:12:44.953:INFO:osjsh.ContextHandler:main: Started o.s.j.s.ServletContextHandler#ba54932{/,null,AVAILABLE}
2018-05-24 12:12:44.971:INFO:osjs.AbstractConnector:main: Started ServerConnector#d6e7bab{HTTP/1.1,[http/1.1]}{0.0.0.0:5555}
2018-05-24 12:12:44.971:INFO:osjs.Server:main: Started #3775ms
12:12:44.971 INFO - Selenium Grid node is up and ready to register to the hub
12:12:44.976 INFO - Starting auto registration thread. Will try to register every 5000 ms.
12:12:44.976 INFO - Registering the node to the hub: http://qa-docker02.avirat.net:4449/grid/register
12:12:44.987 INFO - The node is registered to the hub and ready to use
Selenium started
2018-05-24 12:12:49.235:INFO:osjshC.ROOT:qtp782378927-10: org.openqa.selenium.remote.server.WebDriverServlet-5e5d171f: Initialising WebDriverServlet
12:12:49.249 INFO - Found handler: org.openqa.selenium.remote.server.commandhandler.BeginSession#5452a53c
12:12:49.251 INFO - /session: Executing POST on /session (handler: BeginSession)
12:12:49.295 INFO - Capabilities are: Capabilities {browserName: safari}
12:12:49.297 INFO - Capabilities {browserName: safari} matched class org.openqa.selenium.remote.server.ServicedSession$Factory (provider: org.openqa.selenium.safari.SafariDriverService)
Driver logs:
org.openqa.selenium.SessionNotCreatedException: Cannot establish new session
Build info: version: '3.8.1', revision: '6e95a6684b', time: '2017-12-01T19:05:32.194Z'
System info: host: 'qa01.avirat.net', ip: 'fe80:0:0:0:1854:3678:a42:3622%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.13.4', java.version: '1.8.0_121'
Driver info: driver.version: unknown
Command duration or timeout: 52.01 seconds
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
at org.openqa.selenium.remote.JsonWireProtocolResponse.lambda$new$0(JsonWireProtocolResponse.java:53)
at org.openqa.selenium.remote.JsonWireProtocolResponse.lambda$getResponseFunction$2(JsonWireProtocolResponse.java:91)
at org.openqa.selenium.remote.ProtocolHandshake.lambda$createSession$0(ProtocolHandshake.java:123)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)
at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:464)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:126)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:73)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:136)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:545)
at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:209)
at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:132)
at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:145)
at cucumber.driver.Factory.createDriver(Factory.java:29)
at cucumber.support.Hooks.beforeScenario(Hooks.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at cucumber.runtime.Utils$1.call(Utils.java:31)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.Utils.invoke(Utils.java:25)
at cucumber.runtime.java.JavaHookDefinition.execute(JavaHookDefinition.java:60)
at cucumber.runtime.HookDefinitionMatch.runStep(HookDefinitionMatch.java:17)
at cucumber.runner.UnskipableStep.executeStep(UnskipableStep.java:22)
at cucumber.api.TestStep.run(TestStep.java:83)
at cucumber.api.TestCase.run(TestCase.java:58)
at cucumber.runner.Runner.runPickle(Runner.java:80)
at cucumber.runtime.Runtime.runFeature(Runtime.java:119)
at cucumber.runtime.Runtime.run(Runtime.java:104)
at cucumber.api.cli.Main.run(Main.java:36)
at cucumber.api.cli.Main.main(Main.java:18)
Selenium Hub logs
starting selenium hub with configuration:
{
"host": null,
"port": 4444,
"role": "hub",
"maxSession": 50,
"newSessionWaitTimeout": 120000,
"capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
"throwOnCapabilityNotPresent": true,
"jettyMaxThreads": -1,
"cleanUpCycle": 60000,
"browserTimeout": 60000,
"timeout": 90000,
"debug": false
}
19:33:36.875 INFO - Selenium build info: version: '3.7.1', revision: '8a0099a'
19:33:36.878 INFO - Launching Selenium Grid hub
2018-05-24 19:33:37.883:INFO::main: Logging initialized #2333ms to org.seleniumhq.jetty9.util.log.StdErrLog
19:33:37.914 INFO - Will listen on 4444
2018-05-24 19:33:38.014:INFO:osjs.Server:main: jetty-9.4.5.v20170502
2018-05-24 19:33:38.081:INFO:osjs.session:main: DefaultSessionIdManager workerName=node0
2018-05-24 19:33:38.081:INFO:osjs.session:main: No SessionScavenger set, using defaults
2018-05-24 19:33:38.089:INFO:osjs.session:main: Scavenging every 600000ms
2018-05-24 19:33:38.107:INFO:osjsh.ContextHandler:main: Started o.s.j.s.ServletContextHandler#57d5872c{/,null,AVAILABLE}
2018-05-24 19:33:38.154:INFO:osjs.AbstractConnector:main: Started ServerConnector#1f1c7bf6{HTTP/1.1,[http/1.1]}{0.0.0.0:4444}
2018-05-24 19:33:38.155:INFO:osjs.Server:main: Started #2605ms
19:33:38.155 INFO - Nodes should register to http://10.255.1.238:4444/grid/register/
19:33:38.155 INFO - Selenium Grid hub is up and running
19:33:39.082 INFO - Registered a node http://10.57.85.10:5555
19:34:08.364 INFO - Got a request to create a new session: Capabilities {browserName: safari}
19:34:08.365 INFO - Trying to create a new session on test slot {seleniumProtocol=WebDriver, technologyPreview=false, se:CONFIG_UUID=c82e2553-dfdf-409c-8028-c76da8352368, browserName=safari, maxInstances=1, platform=MAC}
With High Sierra, I had to specify the specific OS version in platform, as well as the safari version as follows (in JS vs Java but same principle):
"safari": {
"browserName": "safari",
"platform": "macOS 10.13",
"version": "11.0",
"port": 4444
},
If this doesn't resolve the problem, the user you end up with on the mac you are SSHing into may not have sufficient permissions to run the driver.
While trying to run my very simple test suite that I've finally started building out -- I'm receiving this amongst the error log:
WebDriverError: Firefox option was set, but is not a FirefoxOption: {}
I'm running the following versions:
Node v7.5.0
Protractor v5.1.1
Selenium Webdriver v3.0.1
Firefox v45.0.1 and now v51.0.1
Full error dump:
[00:44:20] I/testLogger - [firefox #11] PID: 77949
[firefox #11] Specs: /Users/meetbryce/Dropbox (Personal)/Code/Projects/HelloMetrics/utm-link-manager/protractor/auth/login.spec.js
[firefox #11]
[firefox #11] (node:77949) DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
[firefox #11] [00:44:20] I/hosted - Using the selenium server at http://localhost:4444/wd/hub
[firefox #11]
[firefox #11] /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/error.js:505
[firefox #11] throw new ctor(message);
[firefox #11] ^
[firefox #11] WebDriverError: Firefox option was set, but is not a FirefoxOption: {}
[firefox #11] Build info: version: '3.0.1', revision: '1969d75', time: '2016-10-18 09:48:19 -0700'
[firefox #11] System info: host: 'bryceyork.local', ip: '10.40.66.60', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.12.1', java.version: '1.8.0_91'
[firefox #11] Driver info: driver.version: FirefoxDriver
[firefox #11] at Object.checkLegacyResponse (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/error.js:505:15)
[firefox #11] at parseHttpResponse (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/http.js:509:13)
[firefox #11] at doSend.then.response (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/http.js:440:13)
[firefox #11] at process._tickCallback (internal/process/next_tick.js:103:7)
[firefox #11] From: Task: WebDriver.createSession()
[firefox #11] at Function.createSession (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver.js:777:24)
[firefox #11] at Function.createSession (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/firefox/index.js:640:55)
[firefox #11] at createDriver (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/index.js:167:33)
[firefox #11] at Builder.build (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/index.js:629:16)
[firefox #11] at Hosted.getNewDriver (/usr/local/lib/node_modules/protractor/built/driverProviders/driverProvider.js:52:33)
[firefox #11] at Runner.createBrowser (/usr/local/lib/node_modules/protractor/built/runner.js:194:43)
[firefox #11] at q.then.then (/usr/local/lib/node_modules/protractor/built/runner.js:338:29)
[firefox #11] at _fulfilled (/usr/local/lib/node_modules/protractor/node_modules/q/q.js:834:54)
[firefox #11] at self.promiseDispatch.done (/usr/local/lib/node_modules/protractor/node_modules/q/q.js:863:30)
[firefox #11] at Promise.promise.promiseDispatch (/usr/local/lib/node_modules/protractor/node_modules/q/q.js:796:13)
[00:44:20] I/testLogger -
[00:44:20] E/launcher - Runner process exited unexpectedly with error code: 1
[00:44:20] I/launcher - 1 instance(s) of WebDriver still running
..[00:44:27] I/testLogger -
------------------------------------
[00:44:27] I/testLogger - [chrome #01] PID: 77948
[chrome #01] Specs: /Users/meetbryce/Dropbox (Personal)/Code/Projects/HelloMetrics/utm-link-manager/protractor/auth/login.spec.js
[chrome #01]
[chrome #01] (node:77948) DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
[chrome #01] [00:44:20] I/hosted - Using the selenium server at http://localhost:4444/wd/hub
[chrome #01] Started
[chrome #01] log: []
[chrome #01] log: []
[chrome #01]
[chrome #01]
[chrome #01]
[chrome #01] 2 specs, 0 failures
[chrome #01] Finished in 5.816 seconds
[chrome #01]
[00:44:27] I/testLogger -
[00:44:27] I/launcher - 0 instance(s) of WebDriver still running
[00:44:27] I/launcher - firefox #11 failed with exit code: 1
[00:44:27] I/launcher - chrome #01 passed
[00:44:27] I/launcher - overall: 1 process(es) failed to complete
[00:44:27] E/launcher - Process exited with error code 100
Process finished with exit code 100
Here's my protractor config in case that's important:
exports.config = {
framework: 'jasmine',
seleniumAddress: 'http://localhost:4444/wd/hub',
specs: ['**/*.spec.js'],
multiCapabilities: [
{
'browserName': 'chrome',
'loggingPrefs': {
'driver': 'WARNING',
'server': 'WARNING',
'browser': 'SEVERE'
}
},
{
'browserName': 'firefox'
}
]
};
Had the exact same problem.
Launched today a "webdriver-manager update"
And then everything worked fine
EDIT:
update command
Sorry cant comment can only answer - someone in the past seems to have had a similar issue:
https://github.com/seleniumhq/selenium/issues/3115
I can also reproduce this - And I've also recently done a webdriver-manager update
For any Firefox versions below v48 you'll need to run with the legacy FirefoxDriver. Setting marionette: false as #FCin suggested in his comment will tell selenium-webdriver to use the legacy FirefoxDriver, but it will later fail with the error you described because it can't run using the Standalone Selenium Server. To get it to work you'll also need to remove the seleniumAddress from your config and replace it with directConnect: true like so:
exports.config = {
framework: 'jasmine',
directConnect: true,
specs: ['**/*.spec.js'],
multiCapabilities: [
{
'browserName': 'chrome',
'loggingPrefs': {
'driver': 'WARNING',
'server': 'WARNING',
'browser': 'SEVERE'
}
},
{
'browserName': 'firefox'
}
]
};
Unfortunately, a bug was introduced in Protractor v5.1.0 that broke the integration with the legacy FirefoxDriver. You'll need to downgrade it to v5.0.0 if you want to run on Firefox v45.0.1
add the following in your conf file
capabilities: {browserName: 'firefox'}
When I am trying to run oozie Job it's throwing following error
Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
My shell script runs fine when run independently.
I am new to oozie, Any help will be greatly appreciated
Workflow.xml
<workflow-app name="DQ-wf" xmlns="uri:oozie:workflow:0.4">
<start to = 'validate-tables' />
<action name="validate-tables">
<shell xmlns="uri:oozie:shell-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>test.sh</exec>
<file>test.sh</file>
<capture-output/>
</shell>
<ok to="decide"/>
<error to="fail"/>
</action>
<decision name="decide">
<switch>
<case to="shell-file">
${wf:actionData("shell-file")["isMatch"]}
</case>
<default to="end"/>
</switch>
</decision>
<action name="shell-file">
<shell xmlns="uri:oozie:shell-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>copy-output.sh</exec>
<file>copy-output.sh</file>
</shell>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Workflow failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name='end' />
</workflow-app>
Job.Properties
===============
nameNode=hdfs://dev-devwork:8020
jobTracker=isperf.xxx.com:8050
hdfs=hdfs://dev-devwork
queueName=default
workflowRoot=/etc/hive/conf
oozie.use.system.libpath=true
oozieProjectRoot=${nameNode}/user/gdsds/oozie/ing
appPath=${oozieProjectRoot}
oozie.wf.application.path=${appPath}
test.sh
========
DATE=`date +%Y_%m_%d_%H_%M_%S`;
. ./env_settings.ini
destdir=/home/gdsds/oozie/$DATE.txt
Q1=$( hive -S -e "select count(1) from gpc.TABLE_A;")
echo $Q1;
Q1T=$( hive -S -e "select count(1) from gpc.TABLE_B;")
if [ "$Q1" -eq "$Q1T" ]
then
echo "$Q1 TABLE_A VS TABLE_B MATCH" >> "$destdir";
else
echo "$Q1 TABLE_A VS TABLE_B UNMATCH" >> "$destdir";
echo "isMatch=false";
exit 0
fi
Q2=$( hive -S -e "select count(1) from gpc.TABLE_X;")
echo $Q2
Q2T=$( hive -S -e "select count(1) from gpc.TABLE_Y")
if [ "$Q2" -eq "$Q2T" ]
then
echo "$Q2 TABLE_X VS TABLE_Y MATCH" >> "$destdir";
echo "isMatch=true";
else
echo "$Q2 TABLE_X VS TABLE_Y UNMATCH" >> "$destdir";
echo "isMatch=false";
exit 0
fi
Error Logs
2016-05-04 23:04:47,771 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#:start:] Start action [0186090-160418004347476-oozie-oozi-W#:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-05-04 23:04:47,774 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#:start:] [***0186090-160418004347476-oozie-oozi-W#:start:***]Action status=DONE
2016-05-04 23:04:47,774 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#:start:] [***0186090-160418004347476-oozie-oozi-W#:start:***]Action updated in DB!
2016-05-04 23:04:47,799 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#:start:] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W#:start:
2016-05-04 23:04:47,801 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W
2016-05-04 23:04:47,817 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] Start action [0186090-160418004347476-oozie-oozi-W#validate-tables] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-05-04 23:04:49,257 INFO ShellActionExecutor:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] checking action, hadoop job ID [job_1461261058027_467475] status [RUNNING]
2016-05-04 23:04:49,258 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] [***0186090-160418004347476-oozie-oozi-W#validate-tables***]Action status=RUNNING
2016-05-04 23:04:49,258 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] [***0186090-160418004347476-oozie-oozi-W#validate-tables***]Action updated in DB!
2016-05-04 23:04:49,262 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W#validate-tables
2016-05-04 23:05:16,380 INFO CallbackServlet:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] callback for action [0186090-160418004347476-oozie-oozi-W#validate-tables]
2016-05-04 23:05:16,550 INFO ShellActionExecutor:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] action completed, external ID [job_1461261058027_467475]
2016-05-04 23:05:16,554 WARN ShellActionExecutor:523 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
2016-05-04 23:05:16,600 INFO ActionEndXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] ERROR is considered as FAILED for SLA
2016-05-04 23:05:16,627 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#fail] Start action [0186090-160418004347476-oozie-oozi-W#fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-05-04 23:05:16,630 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#fail] [***0186090-160418004347476-oozie-oozi-W#fail***]Action status=DONE
2016-05-04 23:05:16,631 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#fail] [***0186090-160418004347476-oozie-oozi-W#fail***]Action updated in DB!
2016-05-04 23:05:16,678 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#fail] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W#fail
2016-05-04 23:05:16,678 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W
2016-05-04 23:05:16,678 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W#validate-tables] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W#validate-tables