We have near caches configured with main caches (in data node). Looking at documentation it says "Near caches are fully transactional and get updated or invalidated automatically whenever the data changes on the server nodes."
I am trying to understand how this communication of automatic updates work:
Will this communication be driven by "CacheWriteSynchronizationMode" ? So if I choose FULL_ASYNC mode then data node will be blocked till near cache updates are complete?
If above point is true then, will choosing PRIMARY_SYNC mode unblock data nodes?
Whenever cache items are expired in data node or persisted in disk (for persistent cache) then will data node immediately try to replicate this in near cache for client nodes?
We saw an issue where data nodes were continuously trying to connect to client node with this in log lines :
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateRequest(GridDhtAtomicCache.java:3322) [ignite-core-2.9.1.jar:2.9.1]
We suspect that Data nodes were blocked because these continuosly trying to send messages to near caches. This is based on stack trace we saw.
Full Stack trace:
at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:191) ~[ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141) ~[ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:3229) [ignite-core-2.9.1.jar:na]
at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:3013) [ignite-core-2.9.1.jar:na]
at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2960) [ignite-core-2.9.1.jar:na]
at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:2100) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:2365) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1964) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1935) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1917) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendNotification(GridContinuousProcessor.java:1324) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.addNotification(GridContinuousProcessor.java:1261) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:1059) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.access$600(CacheContinuousQueryHandler.java:90) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$2.onEntryUpdated(CacheContinuousQueryHandler.java:459) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:447) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2495) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2657) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2118) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1935) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1734) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateRequest(GridDhtAtomicCache.java:3322) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$400(GridDhtAtomicCache.java:141) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:273) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:268) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1142) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1907) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1528) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.managers.communication.GridIoManager.access$5300(GridIoManager.java:241) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.managers.communication.GridIoManager$9.execute(GridIoManager.java:1421) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:55) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:565) [ignite-core-2.9.1.jar:2.9.1]
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) [ignite-core-2.9.1.jar:2.9.1]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: org.apache.ignite.spi.communication.tcp.NodeForceEvictException: Node evicted forcefully from topology.
at org.apache.ignite.spi.communication.tcp.IBTcpCommunicationSpi.createTcpClient(IBTcpCommunicationSpi.java:60) ~[ib-compute-grid-23.2.10.jar:na]
at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createCommunicationClient(TcpCommunicationSpi.java:3375) [ignite-core-2.9.1.jar:na]
at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:3180) [ignite-core-2.9.1.jar:na]
... 36 common frames omitted
Caused by: org.apache.ignite.spi.communication.tcp.internal.NodeUnreachableException: Failed to connect to all addresses of node 4ae96cc6-d3ba-4bb4-94f8-4c116d5bd9eb: [/10.228.30.249:47000]; inverse connection will be requested.
at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioSession(TcpCommunicationSpi.java:3982) [ignite-core-2.9.1.jar:na]
at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3635) [ignite-core-2.9.1.jar:na]
Related
so my company has implemented Oauth2.0 on two different internal servers. when i try using karate to get the token back on the myldev server. i get it back with any issues. (with configure ssl = True)
But when i do the exact same call against the mylqa server. i get the following error
11:01:46.113 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Opening connection {s}-> private url
11:01:46.113 [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connecting to mylqa.corp.realpage.com/10.34.208.35:443
11:01:46.113 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Connecting socket to mylqa.corp.realpage.com/10.34.208.35:443 with timeout 30000
11:01:46.117 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
11:01:46.120 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
11:01:46.120 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Starting handshake
11:01:46.126 [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-3: Shutdown connection
11:01:46.127 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection discarded
11:01:46.127 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 3][route: {s}-> [private url][total kept alive: 0; route allocated: 0 of 5; total allocated: 0 of 10]
11:01:46.127 [main] ERROR com.intuit.karate - java.net.SocketException: Connection reset, http call failed after 194 milliseconds for URL: private url
11:01:46.127 [main] ERROR com.intuit.karate - http request failed:
java.net.SocketException: Connection reset
I havent faced this issue with other tools in my mac. Jmeter which uses apache client 4.5.5 didnt have an issue getting the response back
Regards,
JK
P.S.
im kinda new to ssl and https. so please go easy on me. Also ive made sure that both dev server and qa server have the exact same configuration.
You are sure that both are HTTPS right ? It sounds very much like the QA server has stronger encryption in place. Have a look at this ticket and I hope that gets you on your way !
https://github.com/intuit/karate/issues/243
EDIT - extra info:
Someone else had a similar question, but sadly no answer yet: Link
Similar issue turned out to be missing Accept header: Link
Can you try the new custom certificate support: https://github.com/intuit/karate#x509-certificate-authentication
Related question on Stack Overflow: SSLHandshakeException for a simple GET request in Karate Framework
so my company has implemented Oauth2.0 on two different internal servers. when i try using karate to get the token back on the myldev server. i get it back with any issues. (with configure ssl = True)
But when i do the exact same call against the mylqa server. i get the following error
11:01:46.113 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Opening connection {s}-> private url
11:01:46.113 [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connecting to mylqa.corp.realpage.com/10.34.208.35:443
11:01:46.113 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Connecting socket to mylqa.corp.realpage.com/10.34.208.35:443 with timeout 30000
11:01:46.117 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
11:01:46.120 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
11:01:46.120 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Starting handshake
11:01:46.126 [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-3: Shutdown connection
11:01:46.127 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection discarded
11:01:46.127 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 3][route: {s}-> [private url][total kept alive: 0; route allocated: 0 of 5; total allocated: 0 of 10]
11:01:46.127 [main] ERROR com.intuit.karate - java.net.SocketException: Connection reset, http call failed after 194 milliseconds for URL: private url
11:01:46.127 [main] ERROR com.intuit.karate - http request failed:
java.net.SocketException: Connection reset
I havent faced this issue with other tools in my mac. Jmeter which uses apache client 4.5.5 didnt have an issue getting the response back
Regards,
JK
P.S.
im kinda new to ssl and https. so please go easy on me. Also ive made sure that both dev server and qa server have the exact same configuration.
You are sure that both are HTTPS right ? It sounds very much like the QA server has stronger encryption in place. Have a look at this ticket and I hope that gets you on your way !
https://github.com/intuit/karate/issues/243
EDIT - extra info:
Someone else had a similar question, but sadly no answer yet: Link
Similar issue turned out to be missing Accept header: Link
Can you try the new custom certificate support: https://github.com/intuit/karate#x509-certificate-authentication
Related question on Stack Overflow: SSLHandshakeException for a simple GET request in Karate Framework
We have a number of wildfly 18 servers running standalone but connected via message queues and infinispan caches.
Recently we had an outage, and started seeing this error on a few of the servers :
12:51:08,437 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p5-t1) ISPN000136: Error executing command RemoveExpiredCommand on Cache 'studiomanager', writing keys [20003$1860529424$187762]: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 7916397 from node4
at org.infinispan.remoting.transport.impl.SingleTargetRequest.onTimeout(SingleTargetRequest.java:65)
at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87)
at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [rt.jar:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [rt.jar:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_181]
Seems fairly clear except that the 'studiomanager' cache is defined as a local cache. It does make use of lifespan/eviction, which explains the first part of the error, but not the second part. Why is a local cache seemingly timing out waiting for another server as the second part of the error seems to show?
We do have some replicated caches too, but this is not one of them.
From the standalone.xml file :
<cache-container name="InfinispanSystem" default-cache="systemSettings">
<transport/>
<local-cache name="studiomanager"/>
<replicated-cache name="systemSettings">
<locking isolation="REPEATABLE_READ"/>
</replicated-cache>
</snip ...>
</cache-container>
I have hunted for anything that vaguely looks like the same problem, but have come up largely blank. Can anyone shed any light on how or why this could happen?
EDIT:
As requested, the infinispan version from the Wildfly log is :
Infinispan 'Infinity Minus ONE +2' 9.4.16.Final
The servers are running on docker, each using the same image so the standalone.xml config is largely the same on all of them. We do have some cli scripts that make changes to individual nodes, but the Infinispan subsystem isn't touched by them.
so my company has implemented Oauth2.0 on two different internal servers. when i try using karate to get the token back on the myldev server. i get it back with any issues. (with configure ssl = True)
But when i do the exact same call against the mylqa server. i get the following error
11:01:46.113 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Opening connection {s}-> private url
11:01:46.113 [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connecting to mylqa.corp.realpage.com/10.34.208.35:443
11:01:46.113 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Connecting socket to mylqa.corp.realpage.com/10.34.208.35:443 with timeout 30000
11:01:46.117 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
11:01:46.120 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
11:01:46.120 [main] DEBUG org.apache.http.conn.ssl.LenientSslConnectionSocketFactory - Starting handshake
11:01:46.126 [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-3: Shutdown connection
11:01:46.127 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection discarded
11:01:46.127 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 3][route: {s}-> [private url][total kept alive: 0; route allocated: 0 of 5; total allocated: 0 of 10]
11:01:46.127 [main] ERROR com.intuit.karate - java.net.SocketException: Connection reset, http call failed after 194 milliseconds for URL: private url
11:01:46.127 [main] ERROR com.intuit.karate - http request failed:
java.net.SocketException: Connection reset
I havent faced this issue with other tools in my mac. Jmeter which uses apache client 4.5.5 didnt have an issue getting the response back
Regards,
JK
P.S.
im kinda new to ssl and https. so please go easy on me. Also ive made sure that both dev server and qa server have the exact same configuration.
You are sure that both are HTTPS right ? It sounds very much like the QA server has stronger encryption in place. Have a look at this ticket and I hope that gets you on your way !
https://github.com/intuit/karate/issues/243
EDIT - extra info:
Someone else had a similar question, but sadly no answer yet: Link
Similar issue turned out to be missing Accept header: Link
Can you try the new custom certificate support: https://github.com/intuit/karate#x509-certificate-authentication
Related question on Stack Overflow: SSLHandshakeException for a simple GET request in Karate Framework
I'm setting up a test lab to learn a bit more about the WSO2 apps. I'm trying to re-create the scenario described here: http://wso2.org/library/tutorials/2012/12/providing-xacml-fine-grained-authorization-webapps
I'm using a single Windows 2008 server and have the WSO apps all listening on different ports. The tryit feature of WSOIS shows the policy is being correctly evaluated. However, when attempting to use the web service I receive a Must Understand Check Failed 500 error.
The full text of the error is below
I've re-created the setup and I get the same error.
TID[-1234] [AS] [2013-04-25 21:36:37,740] ERROR
{org.apache.axis2.engine.AxisEngine} - Must Understand check failed
for header
http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd
: Security
org.apache.axis2.engine.AxisEngine.checkMustUnderstand(AxisEngine.java:104)
org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:170)
org.apache.axis2.description.OutInAxisOperationClient.handleResponse(OutInAxisOperation.java:356)
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:413)
org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:224)
org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
org.wso2.carbon.identity.entitlement.stub.EntitlementServiceStub.getDecision(EntitlementServiceStub.java:1108)
org.wso2.carbon.identity.entitlement.proxy.soap.basicAuth.BasicAuthEntitlementServiceClient.getDecision(BasicAuthEntitlementServiceClient.java:191)
org.wso2.carbon.identity.entitlement.proxy.soap.basicAuth.BasicAuthEntitlementServiceClient.getDecision(BasicAuthEntitlementServiceClient.java:62)
org.wso2.carbon.identity.entitlement.proxy.PEPProxy.getDecision(PEPProxy.java:78)
org.wso2.carbon.identity.entitlement.proxy.PEPProxy.getDecision(PEPProxy.java:122)
org.wso2.carbon.identity.entitlement.proxy.PEPProxy.getDecision(PEPProxy.java:99)
org.wso2.carbon.identity.entitlement.filter.EntitlementFilter.doFilter(EntitlementFilter.java:185)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:581)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:177)
org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:161)
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:722)
The root cause is you are setting mustunderstand header in the soap message, so SOAP processor process it and fails because of a missing requirement. You can go through the following article fro more clarification