As the title says, something goes wrong when calling CuratorCache.start. The problem occurs in my project, so I create a small test project to reproduce it.
Env
jdk17(or jdk11)
spring-cloud-starter-zookeeper-all 3.1.0 (with curator-recipes 5.1.0) or curator-recipes 5.2.0
zookeeper 3.6.X or 3.7.0
MacOS Monterey or CentOS7.4
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>curator-test</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-zookeeper-all</artifactId>
<version>3.1.0</version>
<exclusions>
<exclusion>
<artifactId>curator-recipes</artifactId>
<groupId>org.apache.curator</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-recipes</artifactId>
<version>5.2.0</version>
</dependency>
</dependencies>
</project>
Preparing data
add some String to zookeeper path: /test/1
Test code
var curator = CuratorFrameworkFactory.builder()
.connectString("localhost:2181")
.retryPolicy(new ExponentialBackoffRetry(1000, 3))
.build();
curator.start();
var bytes = curator.getData().forPath("/test/1");
System.out.println("value for path /test/1 is " + new String(bytes));
var curatorCache = CuratorCache.builder(curator, "/test").build();
curatorCache.listenable().addListener(
CuratorCacheListener.builder()
.forCreates(node -> System.out.println(String.format("Node created: [%s]", node)))
.forChanges((oldNode, node) -> System.out.println(String.format("Node changed. Old: [%s] New: [%s]", oldNode, node)))
.forDeletes(oldNode -> System.out.println(String.format("Node deleted. Old value: [%s]", oldNode)))
.forInitialized(() -> System.out.println("Cache initialized"))
.build()
);
curatorCache.start();
The test codes mostly comes from CuratorCache Example from official, as the test code shows, I can read data from the zookeeper path, but when I call CuratorCache.start the exception is thrown:
2022-01-23 16:07:53.578 INFO 55099 --- [ main] org.apache.zookeeper.ClientCnxnSocket : jute.maxbuffer value is 1048575 Bytes
2022-01-23 16:07:53.579 INFO 55099 --- [ main] org.apache.zookeeper.ClientCnxn : zookeeper.request.timeout value is 0. feature enabled=false
2022-01-23 16:07:53.580 INFO 55099 --- [ main] o.a.c.f.imps.CuratorFrameworkImpl : Default schema
2022-01-23 16:07:53.586 INFO 55099 --- [16.153.68:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server 172.16.153.68/172.16.153.68:2181.
2022-01-23 16:07:53.586 INFO 55099 --- [16.153.68:2181)] org.apache.zookeeper.ClientCnxn : SASL config status: Will not attempt to authenticate using SASL (unknown error)
2022-01-23 16:07:53.588 INFO 55099 --- [16.153.68:2181)] org.apache.zookeeper.ClientCnxn : Socket connection established, initiating session, client: /192.168.195.34:49599, server: 172.16.153.68/172.16.153.68:2181
2022-01-23 16:07:53.639 INFO 55099 --- [16.153.68:2181)] org.apache.zookeeper.ClientCnxn : Session establishment complete on server 172.16.153.68/172.16.153.68:2181, session id = 0x1007127ce8c071d, negotiated timeout = 40000
2022-01-23 16:07:53.640 INFO 55099 --- [ain-EventThread] o.a.c.f.state.ConnectionStateManager : State change: CONNECTED
2022-01-23 16:07:53.644 INFO 55099 --- [ain-EventThread] o.a.c.framework.imps.EnsembleTracker : New config event received: {}
2022-01-23 16:07:53.644 INFO 55099 --- [ain-EventThread] o.a.c.framework.imps.EnsembleTracker : New config event received: {}
2022-01-23 16:07:53.829 WARN 55099 --- [ main] iguration$LoadBalancerCaffeineWarnLogger : Spring Cloud LoadBalancer is currently working with the default cache. You can switch to using Caffeine cache, by adding it and org.springframework.cache.caffeine.CaffeineCacheManager to the classpath.
2022-01-23 16:07:53.909 INFO 55099 --- [ main] org.test.Application : Started Application in 1.966 seconds (JVM running for 3.09)
2022-01-23 16:07:53.917 INFO 55099 --- [tor-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl : backgroundOperationsLoop exiting
2022-01-23 16:07:53.929 WARN 55099 --- [16.153.68:2181)] org.apache.zookeeper.ClientCnxn : An exception was thrown while closing send thread for session 0x1007127ce8c071d.
org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x1007127ce8c071d, likely server has closed socket
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.3.jar:3.6.3]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290) ~[zookeeper-3.6.3.jar:3.6.3]
2022-01-23 16:07:54.036 INFO 55099 --- [ain-EventThread] org.apache.zookeeper.ClientCnxn : EventThread shut down for session: 0x1007127ce8c071d
2022-01-23 16:07:54.036 INFO 55099 --- [ionShutdownHook] org.apache.zookeeper.ZooKeeper : Session: 0x1007127ce8c071d closed
So is there anybody has some idea, thanks for your comments.
After starting installing spinnaker on Oracle Kubernates engine(halyard on docker), we are unable to get all spinnaker pods in Ready state as below:
$ kubectl get pod -n spinnaker
NAME READY STATUS RESTARTS AGE
spin-clouddriver-6795648f5c-7695g 0/1 Running 0 10m
spin-deck-6766bfb78-ttqff 1/1 Running 0 10m
spin-echo-76d84bccf4-wqgq4 1/1 Running 0 10m
spin-front50-6fdc874759-jtrfp 1/1 Running 0 6m
spin-gate-6ffdfd7657-xhjh2 0/1 Running 0 10m
spin-igor-6c564fc765-589cz 1/1 Running 0 10m
spin-orca-567879df75-4t2gn 0/1 Running 0 10m
spin-redis-5bd65c56b6-82rxp 1/1 Running 0 10m
spin-rosco-7cdf5858fc-n5vrx 0/1 Running 0 9m
Please find the logs for the pods which are not ready and please help me with any clue to resolve the issue.
LMK in case you need more logs/info.
Also I am able to get the UI of spinnaker but its not able to create anything
Logs:
**************spin-clouddriver logs**************
#################################################
----------------------------------------------------------------
$ kubectl log -n spinnaker spin-clouddriver-6795648f5c-7695g|grep -i error
log is DEPRECATED and will be removed in a future version. Use logs instead.
2018-09-14 07:35:13.842 INFO 1 --- [ main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public java.util.Map com.netflix.spinnaker.kork.web.controllers.GenericErrorController.error(java.lang.Boolean,javax.servlet.http.HttpServletRequest)
----------------------------------------------------------------
$ kubectl log -n spinnaker spin-clouddriver-6795648f5c-7695g|tail -20
log is DEPRECATED and will be removed in a future version. Use logs instead.
2018-09-14 07:35:17.245 INFO 1 --- [ main] o.s.b.a.e.mvc.EndpointHandlerMapping : Mapped "{[/heapdump || /heapdump.json],methods=[GET],produces=[application/octet-stream]}" onto public void org.springframework.boot.actuate.endpoint.mvc.HeapdumpMvcEndpoint.invoke(boolean,javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException,javax.servlet.ServletException
2018-09-14 07:35:17.247 INFO 1 --- [ main] o.s.b.a.e.mvc.EndpointHandlerMapping : Mapped "{[/actuator || /actuator.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public org.springframework.hateoas.ResourceSupport org.springframework.boot.actuate.endpoint.mvc.HalJsonMvcEndpoint.links()
2018-09-14 07:35:17.259 INFO 1 --- [ main] o.s.b.a.e.mvc.EndpointHandlerMapping : Mapped "{[/trace || /trace.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-09-14 07:35:17.276 INFO 1 --- [ main] o.s.b.a.e.mvc.EndpointHandlerMapping : Mapped "{[/dump || /dump.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-09-14 07:35:17.277 INFO 1 --- [ main] o.s.b.a.e.mvc.EndpointHandlerMapping : Mapped "{[/mappings || /mappings.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-09-14 07:35:17.277 INFO 1 --- [ main] o.s.b.a.e.mvc.EndpointHandlerMapping : Mapped "{[/autoconfig || /autoconfig.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-09-14 07:35:17.303 INFO 1 --- [ main] o.s.b.a.e.mvc.EndpointHandlerMapping : Mapped "{[/env/{name:.*}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EnvironmentMvcEndpoint.value(java.lang.String)
2018-09-14 07:35:17.304 INFO 1 --- [ main] o.s.b.a.e.mvc.EndpointHandlerMapping : Mapped "{[/env || /env.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-09-14 07:35:17.305 INFO 1 --- [ main] o.s.b.a.e.mvc.EndpointHandlerMapping : Mapped "{[/auditevents || /auditevents.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public org.springframework.http.ResponseEntity<?> org.springframework.boot.actuate.endpoint.mvc.AuditEventsMvcEndpoint.findByPrincipalAndAfterAndType(java.lang.String,java.util.Date,java.lang.String)
2018-09-14 07:35:19.677 INFO 1 --- [ main] c.n.s.c.security.ProviderUtils : Adding accounts [my-k8s-v2-account] of type KubernetesNamedAccountCredentials...
2018-09-14 07:37:04.151 WARN 1 --- [ main] c.n.s.c.k.v.s.KubernetesV2Credentials : Could not list namespaces for account my-k8s-v2-account: Job took too long to complete
2018-09-14 07:37:04.154 WARN 1 --- [ main] c.n.s.c.k.v.s.KubernetesV2Credentials : There are no namespaces configured (or loadable) -- please check that the list of 'omitNamespaces' for account 'my-k8s-v2-account' doesn't prevent access from all namespaces in this cluster, or that the cluster is reachable.
2018-09-14 07:38:47.351 WARN 1 --- [ main] c.n.s.c.k.v.s.KubernetesV2Credentials : Could not list namespaces for account my-k8s-v2-account: Job took too long to complete
2018-09-14 07:40:30.099 WARN 1 --- [ main] c.n.s.c.k.v.s.KubernetesV2Credentials : Could not list namespaces for account my-k8s-v2-account: Job took too long to complete
2018-09-14 07:42:12.815 WARN 1 --- [ main] c.n.s.c.k.v.s.KubernetesV2Credentials : Could not list namespaces for account my-k8s-v2-account: Job took too long to complete
**************spin-gate logs**************
##########################################
----------------------------------------------------------------
$ kubectl -n spinnaker logs spin-gate-6ffdfd7657-xhjh2|grep -i error
2018-09-14 07:35:35.599 INFO 1 --- [ost-startStop-1] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public java.util.Map com.netflix.spinnaker.kork.web.controllers.GenericErrorController.error(java.lang.Boolean,javax.servlet.http.HttpServletRequest)
2018-09-14 07:35:42.556 WARN 1 --- [ main] ationConfigEmbeddedWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'enableRedisKeyspaceNotificationsInitializer' defined in class path resource [com/netflix/spinnaker/gate/config/GateConfig.class]: Invocation of init method failed; nested exception is org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
2018-09-14 07:35:42.643 ERROR 1 --- [ main] o.s.boot.SpringApplication : Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'enableRedisKeyspaceNotificationsInitializer' defined in class path resource [com/netflix/spinnaker/gate/config/GateConfig.class]: Invocation of init method failed; nested exception is org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
2018-09-14 07:36:38.056 ERROR 1 --- [RxIoScheduler-2] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#661f6038 rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2018-09-14 07:37:38.065 ERROR 1 --- [RxIoScheduler-3] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#1df18e11 rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2018-09-14 07:38:38.066 ERROR 1 --- [RxIoScheduler-2] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#6e311e73 rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2018-09-14 07:39:38.067 ERROR 1 --- [RxIoScheduler-3] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#1709e561 rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2018-09-14 07:40:38.069 ERROR 1 --- [RxIoScheduler-2] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#533017fd rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2018-09-14 07:41:38.070 ERROR 1 --- [RxIoScheduler-3] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#431be4b2 rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2018-09-14 07:42:38.071 ERROR 1 --- [RxIoScheduler-2] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#6fc06d32 rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
----------------------------------------------------------------
$ kubectl -n spinnaker logs spin-gate-6ffdfd7657-xhjh2|tail -40
Caused by: org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:204) ~[spring-data-redis-1.8.10.RELEASE.jar:na]
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:348) ~[spring-data-redis-1.8.10.RELEASE.jar:na]
at org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration$EnableRedisKeyspaceNotificationsInitializer.afterPropertiesSet(RedisHttpSessionConfiguration.java:249) ~[spring-session-1.3.1.RELEASE.jar:na]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687) ~[spring-beans-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624) ~[spring-beans-4.3.14.RELEASE.jar:4.3.14.RELEASE]
... 19 common frames omitted
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:53) ~[jedis-2.9.0.jar:na]
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226) ~[jedis-2.9.0.jar:na]
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:16) ~[jedis-2.9.0.jar:na]
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:194) ~[spring-data-redis-1.8.10.RELEASE.jar:na]
... 23 common frames omitted
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out
at redis.clients.jedis.Connection.connect(Connection.java:207) ~[jedis-2.9.0.jar:na]
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93) ~[jedis-2.9.0.jar:na]
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767) ~[jedis-2.9.0.jar:na]
at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106) ~[jedis-2.9.0.jar:na]
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:888) ~[commons-pool2-2.4.3.jar:2.4.3]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:432) ~[commons-pool2-2.4.3.jar:2.4.3]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:361) ~[commons-pool2-2.4.3.jar:2.4.3]
at redis.clients.util.Pool.getResource(Pool.java:49) ~[jedis-2.9.0.jar:na]
... 26 common frames omitted
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_171]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_171]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_171]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_171]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_171]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_171]
at redis.clients.jedis.Connection.connect(Connection.java:184) ~[jedis-2.9.0.jar:na]
... 33 common frames omitted
2018-09-14 07:36:38.056 ERROR 1 --- [RxIoScheduler-2] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#661f6038 rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2018-09-14 07:37:38.065 ERROR 1 --- [RxIoScheduler-3] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#1df18e11 rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2018-09-14 07:38:38.066 ERROR 1 --- [RxIoScheduler-2] c.n.s.gate.services.ApplicationService : Unable to refresh application list, reason: Task java.util.concurrent.FutureTask#6e311e73 rejected from java.util.concurrent.ThreadPoolExecutor#18483b8b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
**************spin-orca logs**************
##########################################
--------------------------------------------------------------------
$ kubectl -n spinnaker logs spin-orca-567879df75-4t2gn|grep -i error
2018-09-14 07:43:56.069 ERROR 1 --- [ scheduler-9] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:56.080 ERROR 1 --- [ scheduler-9] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:56.090 ERROR 1 --- [ scheduler-9] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:56.101 ERROR 1 --- [ scheduler-9] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:56.111 ERROR 1 --- [ scheduler-9] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
..............
..............
2018-09-14 07:43:59.065 ERROR 1 --- [ scheduler-7] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:59.075 ERROR 1 --- [ scheduler-7] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:59.086 ERROR 1 --- [ scheduler-7] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:59.097 ERROR 1 --- [ scheduler-7] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:59.110 ERROR 1 --- [ scheduler-3] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: orca.task.queue:buffered:pipeline
at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:118)
2018-09-14 07:43:59.110 ERROR 1 --- [ scheduler-4] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:59.120 ERROR 1 --- [ scheduler-4] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
2018-09-14 07:43:59.131 ERROR 1 --- [ scheduler-4] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
----------------------------------------------------------------
$ kubectl -n spinnaker logs spin-orca-567879df75-4t2gn|tail -40
2018-09-14 07:45:04.126 ERROR 1 --- [ scheduler-6] o.s.s.s.TaskUtils$LoggingErrorHandler : [] Unexpected error occurred in scheduled task.
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:53)
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226)
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:16)
at com.netflix.spinnaker.q.redis.RedisQueue.poll(RedisQueue.kt:98)
at com.netflix.spinnaker.q.QueueProcessor.pollOnce(QueueProcessor.kt:76)
at com.netflix.spinnaker.q.QueueProcessor.access$pollOnce(QueueProcessor.kt:37)
at com.netflix.spinnaker.q.QueueProcessor$poll$1.invoke(QueueProcessor.kt:62)
at com.netflix.spinnaker.q.QueueProcessor$poll$1.invoke(QueueProcessor.kt:37)
at com.netflix.spinnaker.q.QueueProcessor.ifEnabled(QueueProcessor.kt:119)
at com.netflix.spinnaker.q.QueueProcessor.poll(QueueProcessor.kt:57)
at sun.reflect.GeneratedMethodAccessor113.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.UnknownHostException: spin-redis.spinnaker
at redis.clients.jedis.Connection.connect(Connection.java:207)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93)
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767)
at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:888)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:432)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:361)
at redis.clients.util.Pool.getResource(Pool.java:49)
... 21 common frames omitted
Caused by: java.net.UnknownHostException: spin-redis.spinnaker
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at redis.clients.jedis.Connection.connect(Connection.java:184)
... 28 common frames omitted
**************spin-rosco logs**************
###########################################
---------------------------------------------------------------------
$ kubectl -n spinnaker logs spin-rosco-7cdf5858fc-n5vrx|grep -i error
2018-09-14 07:36:12.914 INFO 1 --- [ main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public java.util.Map com.netflix.spinnaker.kork.web.controllers.GenericErrorController.error(java.lang.Boolean,javax.servlet.http.HttpServletRequest)
2018-09-14 07:36:24.195 ERROR 1 --- [readScheduler-2] c.n.spinnaker.rosco.executor.BakePoller : Zombie Killer Polling Error:
2018-09-14 07:36:24.195 ERROR 1 --- [readScheduler-1] c.n.spinnaker.rosco.executor.BakePoller : Update Polling Error:
2018-09-14 07:36:34.089 ERROR 1 --- [readScheduler-1] c.n.spinnaker.rosco.executor.BakePoller : Update Polling Error:
2018-09-14 07:36:49.089 ERROR 1 --- [readScheduler-1] c.n.spinnaker.rosco.executor.BakePoller : Update Polling Error:
2018-09-14 07:36:49.099 ERROR 1 --- [readScheduler-2] c.n.spinnaker.rosco.executor.BakePoller : Zombie Killer Polling Error:
----------------------------------------------------------------
$ kubectl -n spinnaker logs spin-rosco-7cdf5858fc-n5vrx|tail -40
at org.codehaus.groovy.runtime.callsite.GetEffectivePogoPropertySite.callGetProperty(GetEffectivePogoPropertySite.java:48) ~[groovy-all-2.4.13.jar:2.4.13]
at com.netflix.spinnaker.rosco.executor.BakePoller$_onApplicationEvent_closure1.doCall(BakePoller.groovy:81) [rosco-core-0.108.0-SNAPSHOT.jar:0.108.0-SNAPSHOT]
at com.netflix.spinnaker.rosco.executor.BakePoller$_onApplicationEvent_closure1.doCall(BakePoller.groovy) [rosco-core-0.108.0-SNAPSHOT.jar:0.108.0-SNAPSHOT]
at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) [groovy-all-2.4.13.jar:2.4.13]
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) [groovy-all-2.4.13.jar:2.4.13]
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) [groovy-all-2.4.13.jar:2.4.13]
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) [groovy-all-2.4.13.jar:2.4.13]
at groovy.lang.Closure.call(Closure.java:414) [groovy-all-2.4.13.jar:2.4.13]
at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54) [groovy-all-2.4.13.jar:2.4.13]
at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124) [groovy-all-2.4.13.jar:2.4.13]
at com.sun.proxy.$Proxy116.call(Unknown Source) [na:na]
at rx.Scheduler$Worker$1.call(Scheduler.java:120) [rxjava-1.0.16.jar:1.0.16]
at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55) [rxjava-1.0.16.jar:1.0.16]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_171]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_171]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_171]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.UnknownHostException: spin-redis.spinnaker
at redis.clients.jedis.Connection.connect(Connection.java:207) ~[jedis-2.9.0.jar:na]
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93) ~[jedis-2.9.0.jar:na]
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767) ~[jedis-2.9.0.jar:na]
at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106) ~[jedis-2.9.0.jar:na]
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:868) ~[commons-pool2-2.4.2.jar:2.4.2]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) ~[commons-pool2-2.4.2.jar:2.4.2]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) ~[commons-pool2-2.4.2.jar:2.4.2]
at redis.clients.util.Pool.getResource(Pool.java:49) ~[jedis-2.9.0.jar:na]
... 32 common frames omitted
Caused by: java.net.UnknownHostException: spin-redis.spinnaker
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) ~[na:1.8.0_171]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_171]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_171]
at redis.clients.jedis.Connection.connect(Connection.java:184) ~[jedis-2.9.0.jar:na]
... 39 common frames omitted
I had the same issue with K8s V2 provider and I figured out that the main cause was an HAL misconfiguration. I just put the providers > kubernetes > [account] > serviceAccount to true. This was the trick.
If it is your case the configuration command is:
hal config provider kubernetes account edit <account-name> --service-account true
...then re-deploy Spinnaker via hal deploy apply
Hope this helps you. Good luck. :)
After change apache Ignite 2.0 to 2.1, I got below warning.
2017-08-17 10:44:21.699 WARN 10884 --- [ main] .i.p.c.GridCachePartitionExchangeManager : Failed to wait for initial partition map exchange. Possible reasons are:
I use third party persistence cache store.
when I remove cacheStore configuration, I didn't got warning. work fine.
Using cacheStore and changing down version 2.1 to 2.0, I didn't got warning. work fine.
Is there significant change in 2.1?
here is my full framework stack.
- spring boot 1.5.6
- spring data jpa
- apache ignite 2.1.0
here is my full configuration in java code.(I use embedded ignite in spring)
I use partitioned cache, write behind cache to rdbms storage using spring data jpa.
IgniteConfiguration igniteConfig = new IgniteConfiguration();
CacheConfiguration<Long, Object> cacheConfig = new CacheConfiguration<>();
cacheConfig.setCopyOnRead(false); //for better performance
cacheConfig
.setWriteThrough(true)
.setWriteBehindEnabled(true)
.setWriteBehindBatchSize(1024)
.setWriteBehindFlushFrequency(10000)
.setWriteBehindCoalescing(true)
.setCacheStoreFactory(new CacheStoreImpl()); //CacheStoreImpl use spring data jpa internally
cacheConfig.setName('myService');
cacheConfig.setCacheMode(CacheMode.PARTITIONED);
cacheConfig.setBackups(2);
cacheConfig.setWriteSynchronizationMode(FULL_ASYNC);
cacheConfig.setNearConfiguration(new NearCacheConfiguration<>());//use default configuration
igniteConfig.setCacheConfiguration(cacheConfig);
igniteConfig.setMemoryConfiguration(new MemoryConfiguration()
.setPageSize(8 * 1024)
.setMemoryPolicies(new MemoryPolicyConfiguration()
.setInitialSize((long) 256L * 1024L * 1024L)
.setMaxSize((long) 1024L * 1024L * 1024L)));
Ignite ignite = IgniteSpring.start(igniteConfig, springApplicationCtx);
ignite.active(true);
here is my full log using -DIGNITE_QUITE=false
2017-08-18 11:54:52.587 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : Config URL: n/a
2017-08-18 11:54:52.587 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : Daemon mode: off
2017-08-18 11:54:52.587 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : OS: Windows 10 10.0 amd64
2017-08-18 11:54:52.587 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : OS user: user
2017-08-18 11:54:52.588 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : PID: 684
2017-08-18 11:54:52.588 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : Language runtime: Java Platform API Specification ver. 1.8
2017-08-18 11:54:52.588 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : VM information: Java(TM) SE Runtime Environment 1.8.0_131-b11 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.131-b11
2017-08-18 11:54:52.588 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : VM total memory: 1.9GB
2017-08-18 11:54:52.589 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : Remote Management [restart: off, REST: on, JMX (remote: on, port: 58771, auth: off, ssl: off)]
2017-08-18 11:54:52.589 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : IGNITE_HOME=null
2017-08-18 11:54:52.589 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : VM arguments: [-Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.port=58771, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Djava.rmi.server.hostname=localhost, -Dspring.liveBeansView.mbeanDomain, -Dspring.application.admin.enabled=true, -Dspring.profiles.active=rdbms,multicastIp, -Dapi.port=10010, -Xmx2g, -Xms2g, -DIGNITE_QUIET=false, -Dfile.encoding=UTF-8, -Xbootclasspath:C:\Program Files\Java\jre1.8.0_131\lib\resources.jar;C:\Program Files\Java\jre1.8.0_131\lib\rt.jar;C:\Program Files\Java\jre1.8.0_131\lib\jsse.jar;C:\Program Files\Java\jre1.8.0_131\lib\jce.jar;C:\Program Files\Java\jre1.8.0_131\lib\charsets.jar;C:\Program Files\Java\jre1.8.0_131\lib\jfr.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\cldrdata.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\dnsns.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\jaccess.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\jfxrt.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\localedata.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\nashorn.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\sunec.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\sunmscapi.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jre1.8.0_131\lib\ext\zipfs.jar]
2017-08-18 11:54:52.589 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : System cache's MemoryPolicy size is configured to 40 MB. Use MemoryConfiguration.systemCacheMemorySize property to change the setting.
2017-08-18 11:54:52.589 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : Configured caches [in 'sysMemPlc' memoryPolicy: ['ignite-sys-cache'], in 'default' memoryPolicy: ['myCache']]
2017-08-18 11:54:52.592 WARN 684 --- [ pub-#11%null%] o.apache.ignite.internal.GridDiagnostic : This operating system has been tested less rigorously: Windows 10 10.0 amd64. Our team will appreciate the feedback if you experience any problems running ignite in this environment.
2017-08-18 11:54:52.657 INFO 684 --- [ main] o.a.i.i.p.plugin.IgnitePluginProcessor : Configured plugins:
2017-08-18 11:54:52.657 INFO 684 --- [ main] o.a.i.i.p.plugin.IgnitePluginProcessor : ^-- None
2017-08-18 11:54:52.657 INFO 684 --- [ main] o.a.i.i.p.plugin.IgnitePluginProcessor :
2017-08-18 11:54:52.724 INFO 684 --- [ main] o.a.i.s.c.tcp.TcpCommunicationSpi : Successfully bound communication NIO server to TCP port [port=47100, locHost=0.0.0.0/0.0.0.0, selectorsCnt=4, selectorSpins=0, pairedConn=false]
2017-08-18 11:54:52.772 WARN 684 --- [ main] o.a.i.s.c.tcp.TcpCommunicationSpi : Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.
2017-08-18 11:54:52.787 WARN 684 --- [ main] o.a.i.s.c.noop.NoopCheckpointSpi : Checkpoints are disabled (to enable configure any GridCheckpointSpi implementation)
2017-08-18 11:54:52.811 WARN 684 --- [ main] o.a.i.i.m.c.GridCollisionManager : Collision resolution is disabled (all jobs will be activated upon arrival).
2017-08-18 11:54:52.812 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : Security status [authentication=off, tls/ssl=off]
2017-08-18 11:54:53.087 INFO 684 --- [ main] o.a.i.i.p.odbc.SqlListenerProcessor : SQL connector processor has started on TCP port 10800
2017-08-18 11:54:53.157 INFO 684 --- [ main] o.a.i.i.p.r.p.tcp.GridTcpRestProtocol : Command protocol successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211]
2017-08-18 11:54:53.373 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : Non-loopback local IPs: 192.168.183.206, 192.168.56.1, 2001:0:9d38:6abd:30a3:1c57:3f57:4831, fe80:0:0:0:159d:5c82:b4ca:7630%eth2, fe80:0:0:0:30a3:1c57:3f57:4831%net0, fe80:0:0:0:3857:b492:48ad:1dc%eth4
2017-08-18 11:54:53.373 INFO 684 --- [ main] org.apache.ignite.internal.IgniteKernal : Enabled local MACs: 00000000000000E0, 0A0027000004, BCEE7B8B7C00
2017-08-18 11:54:53.404 INFO 684 --- [ main] o.a.i.spi.discovery.tcp.TcpDiscoverySpi : Successfully bound to TCP port [port=47500, localHost=0.0.0.0/0.0.0.0, locNodeId=7d90a0ac-b620-436f-b31c-b538a04b0919]
2017-08-18 11:54:53.409 WARN 684 --- [ main] .s.d.t.i.m.TcpDiscoveryMulticastIpFinder : TcpDiscoveryMulticastIpFinder has no pre-configured addresses (it is recommended in production to specify at least one address in TcpDiscoveryMulticastIpFinder.getAddresses() configuration property)
2017-08-18 11:54:55.068 INFO 684 --- [orker-#34%null%] o.apache.ignite.internal.exchange.time : Started exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=0], crd=true, evt=10, node=TcpDiscoveryNode [id=7d90a0ac-b620-436f-b31c-b538a04b0919, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.183.206, 192.168.56.1, 2001:0:9d38:6abd:30a3:1c57:3f57:4831], sockAddrs=[/192.168.183.206:47500, DESKTOP-MDB6VIL/192.168.56.1:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, /2001:0:9d38:6abd:30a3:1c57:3f57:4831:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1503024893396, loc=true, ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false], evtNode=TcpDiscoveryNode [id=7d90a0ac-b620-436f-b31c-b538a04b0919, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.183.206, 192.168.56.1, 2001:0:9d38:6abd:30a3:1c57:3f57:4831], sockAddrs=[/192.168.183.206:47500, DESKTOP-MDB6VIL/192.168.56.1:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, /2001:0:9d38:6abd:30a3:1c57:3f57:4831:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1503024893396, loc=true, ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false], customEvt=null]
2017-08-18 11:54:55.302 INFO 684 --- [orker-#34%null%] o.a.i.i.p.cache.GridCacheProcessor : Started cache [name=ignite-sys-cache, memoryPolicyName=sysMemPlc, mode=REPLICATED, atomicity=TRANSACTIONAL]
2017-08-18 11:55:15.066 WARN 684 --- [ main] .i.p.c.GridCachePartitionExchangeManager : Failed to wait for initial partition map exchange. Possible reasons are:
^-- Transactions in deadlock.
^-- Long running transactions (ignore if this is the case).
^-- Unreleased explicit locks.
2017-08-18 11:55:35.070 WARN 684 --- [ main] .i.p.c.GridCachePartitionExchangeManager : Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=7d90a0ac-b620-436f-b31c-b538a04b0919, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.183.206, 192.168.56.1, 2001:0:9d38:6abd:30a3:1c57:3f57:4831], sockAddrs=[/192.168.183.206:47500, DESKTOP-MDB6VIL/192.168.56.1:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, /2001:0:9d38:6abd:30a3:1c57:3f57:4831:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1503024893396, loc=true, ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false], topVer=1, nodeId8=7d90a0ac, msg=null, type=NODE_JOINED, tstamp=1503024895045], crd=TcpDiscoveryNode [id=7d90a0ac-b620-436f-b31c-b538a04b0919, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.183.206, 192.168.56.1, 2001:0:9d38:6abd:30a3:1c57:3f57:4831], sockAddrs=[/192.168.183.206:47500, DESKTOP-MDB6VIL/192.168.56.1:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, /2001:0:9d38:6abd:30a3:1c57:3f57:4831:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1503024893396, loc=true, ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=0], nodeId=7d90a0ac, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1821989981], init=false, lastVer=null, partReleaseFut=null, exchActions=null, affChangeMsg=null, skipPreload=false, clientOnlyExchange=false, initTs=1503024895057, centralizedAff=false, changeGlobalStateE=null, forcedRebFut=null, done=false, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=733156437]]]
I debug my code, I guess IgniteSpring cannot inject SpringResource
#SpringResource(resourceClass = RdbmsCachePersistenceRepository.class)
private RdbmsCachePersistenceRepository repository;
#SpringResource(resourceClass = RdbmsCachePersistenceRepository.class)
private CacheObjectFactory cacheObjectFactory;
repository, cacheObjectFactory is same instance like below code
public interface RdbmsCachePersistenceRepository extends
JpaRepository<RdbmsCachePersistence, Long>,
CachePersistenceRepository<RdbmsCachePersistence>,
CacheObjectFactory {
#Override
default CachePersistence createCacheObject(long key, Object value, int partition) {
return new RdbmsCachePersistence(key, value, partition);
}
}
And RdbmsCachePersistenceRepository implemented by spring data jpa
when I debug code line by line, IgniteContext cannot bring RdbmsCachePersistenceRepository
I don't know why it is
I resolve this problem, but I don't know why it is resolved.
I added this dummy code before IgniteSpring.start.
springApplicationCtx.getBean(RdbmsCachePersistenceRepository.class);
I think the spring resource bean not initialized when the ignite context get the bean.
I am a new user of RabbitMQ and I really enjoy it but I have an issue (well it doesn't throw any error and it doesn't affect anything except my mind ...).
Each time I run a consumer, it creates 2 connections. I can't find why so I am asking for your help.
I am using Spring-Boot and Spring AMQP (maybe it because of Spring ...)
Here is the code :
receiver-context.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.0.xsd">
<rabbit:connection-factory id="connectionFactory" host="localhost" username="admin" password="admin" />
<bean id="receiver" class="com.test.Receiver" />
<bean id="messageListener" class="org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter" >
<constructor-arg name="delegate" ref="receiver"/>
<constructor-arg name="defaultListenerMethod" value="receiveMessage" />
</bean>
<bean id="container" class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer" >
<property name="connectionFactory" ref="connectionFactory" />
<property name="queueNames" value="AMQP-PoC" />
<property name="messageListener" ref="messageListener" />
<property name="defaultRequeueRejected" value="false" />
</bean>
AMQPPoCReceiverApplication.java
#SpringBootApplication
#ImportResource("classpath:com.test/rabbit-receiver-context.xml")
public class AMQPPoCReceiverApplication implements CommandLineRunner {
private AnnotationConfigApplicationContext context;
#Override
public void run(String... strings) throws Exception {
context = new AnnotationConfigApplicationContext(AMQPPoCReceiverApplication.class);
System.out.println("Waiting for message");
}
#Override
protected void finalize() throws Throwable {
super.finalize();
this.context.close();
}
public static void main(String[] args) {
SpringApplication.run(AMQPPoCReceiverApplication.class, args);
}
}
Receiver.java
public class Receiver {
public void receiveMessage(String message) {
System.out.println("Message received : " + message);
}
}
Here the logs at the start (notice the lines with the double '*'):
2016-02-18 11:32:51.956 INFO 10196 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
2016-02-18 11:32:51.966 INFO 10196 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase -2147482648
2016-02-18 11:32:51.967 INFO 10196 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 2147483647
**2016-02-18 11:32:52.062 INFO 10196 --- [cTaskExecutor-1] o.s.a.r.c.CachingConnectionFactory : Created new connection: SimpleConnection#2069bb0a [delegate=amqp://admin#127.0.0.1:5672/]**
2016-02-18 11:32:52.148 INFO 10196 --- [ main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 52752 (http)
2016-02-18 11:32:52.153 INFO 10196 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext#57bf85b2: startup date [Thu Feb 18 11:32:52 GMT+01:00 2016]; root of contex
t hierarchy
**2016-02-18 11:32:52.320 INFO 10196 --- [ main] o.s.b.f.xml.XmlBeanDefinitionReader : Loading XML bean definitions from class path resource [com.test/receiver-context.xml]**
2016-02-18 11:32:52.357 INFO 10196 --- [ main] f.a.AutowiredAnnotationBeanPostProcessor : JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2016-02-18 11:32:52.362 INFO 10196 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.amqp.rabbit.annotation.RabbitBootstrapConfiguration' of type [class org.springframework.amqp.rabbit.annotation.RabbitBootstrapConfigur
ation$$EnhancerBySpringCGLIB$$eccd4a65] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2016-02-18 11:32:52.487 INFO 10196 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
2016-02-18 11:32:52.489 INFO 10196 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase -2147482648
2016-02-18 11:32:52.489 INFO 10196 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 2147483647
**2016-02-18 11:32:52.498 INFO 10196 --- [cTaskExecutor-1] o.s.a.r.c.CachingConnectionFactory : Created new connection: SimpleConnection#768748cf [delegate=amqp://admin#127.0.0.1:5672/]**
Waiting for message
2016-02-18 11:32:52.505 INFO 10196 --- [ main] com.test.AMQPPoCReceiverApplication : Started AMQPPoCReceiverApplication in 3.509 seconds (JVM running for 6.961)
And here the double connections:
If I stop the client, it closes both (that is why I am sure it's a double connections for the same consumer).
If you need more information, ask here and I will reply as soon as possible.
Thank you all for any kind of help.
The answer is simple : I created 2 contexts in the same application.
new AnnotationConfigApplicationContext(AMQPPoCReceiverApplication.class);
and
SpringApplication.run(AMQPPoCReceiverApplication.class, args);
Only create one and it's done !
I had a Turbine and Hystrix setup working, but decided to change it over to Turbine AMQP so I could aggregate multiple services into one stream/dashboard.
I have set up a Turbine AMQP server running on localhost:8989, but it doesn't appear to be getting Hystrix data from the client service. When I hit the Turbine server's IP in my browser, I see data: {"type":"Ping"} repeatedly, even while I am polling the URL of the Hystrix. If I attempt to show the Turbine AMQP stream in the Hystrix Dashboard, I get: Unable to connect to Command Metric Stream.
I have a default install of RabbitMQ running on port 5672.
My client service using Hystrix-AMQP has a application.yml file that looks like so:
spring:
application:
name: policy-service
eureka:
client:
serviceUrl:
defaultZone: http://localhost:8761/eureka/
spring:
rabbitmq:
addresses: ${vcap.services.${PREFIX:}rabbitmq.credentials.uri:amqp://${RABBITMQ_HOST:localhost}:${RABBITMQ_PORT:5672}}
The tail end of the startup log looks like this:
2015-09-14 16:31:13.030 INFO 52844 --- [ main] com.netflix.discovery.DiscoveryClient : Starting heartbeat executor: renew interval is: 30
2015-09-14 16:31:13.047 INFO 52844 --- [ main] c.n.e.EurekaDiscoveryClientConfiguration : Registering application policy-service with eureka with status UP
2015-09-14 16:31:13.194 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2015-09-14 16:31:13.195 INFO 52844 --- [ main] o.s.i.channel.PublishSubscribeChannel : Channel 'policy-service:8088.errorChannel' has 1 subscriber(s).
2015-09-14 16:31:13.195 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started _org.springframework.integration.errorLogger
2015-09-14 16:31:13.195 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {filter} as a subscriber to the 'cloudBusOutboundFlow.channel#0' channel
2015-09-14 16:31:13.195 INFO 52844 --- [ main] o.s.integration.channel.DirectChannel : Channel 'policy- service:8088.cloudBusOutboundFlow.channel#0' has 1 subscriber(s).
2015-09-14 16:31:13.195 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2015-09-14 16:31:13.195 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {filter} as a subscriber to the 'cloudBusInboundChannel' channel
2015-09-14 16:31:13.195 INFO 52844 --- [ main] o.s.integration.channel.DirectChannel : Channel 'policy-service:8088.cloudBusInboundChannel' has 1 subscriber(s).
2015-09-14 16:31:13.196 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2015-09-14 16:31:13.196 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {message-handler} as a subscriber to the 'cloudBusInboundFlow.channel#0' channel
2015-09-14 16:31:13.196 INFO 52844 --- [ main] o.s.integration.channel.DirectChannel : Channel 'policy-service:8088.cloudBusInboundFlow.channel#0' has 1 subscriber(s).
2015-09-14 16:31:13.196 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.config.ConsumerEndpointFactoryBean#2
2015-09-14 16:31:13.196 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {logging-channel-adapter} as a subscriber to the 'cloudBusWiretapChannel' channel
2015-09-14 16:31:13.196 INFO 52844 --- [ main] o.s.integration.channel.DirectChannel : Channel 'policy-service:8088.cloudBusWiretapChannel' has 1 subscriber(s).
2015-09-14 16:31:13.197 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.config.ConsumerEndpointFactoryBean#3
2015-09-14 16:31:13.197 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {amqp:outbound-channel-adapter} as a subscriber to the 'cloudBusOutboundChannel' channel
2015-09-14 16:31:13.197 INFO 52844 --- [ main] o.s.integration.channel.DirectChannel : Channel 'policy-service:8088.cloudBusOutboundChannel' has 1 subscriber(s).
2015-09-14 16:31:13.197 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.config.ConsumerEndpointFactoryBean#4
2015-09-14 16:31:13.198 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {bridge} as a subscriber to the 'cloudBusAmqpInboundFlow.channel#0' channel
2015-09-14 16:31:13.198 INFO 52844 --- [ main] o.s.integration.channel.DirectChannel : Channel 'policy-service:8088.cloudBusAmqpInboundFlow.channel#0' has 1 subscriber(s).
2015-09-14 16:31:13.198 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.config.ConsumerEndpointFactoryBean#5
2015-09-14 16:31:13.198 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {amqp:outbound-channel-adapter} as a subscriber to the 'hystrixStream' channel
2015-09-14 16:31:13.199 INFO 52844 --- [ main] o.s.integration.channel.DirectChannel : Channel 'policy-service:8088.hystrixStream' has 1 subscriber(s).
2015-09-14 16:31:13.199 INFO 52844 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.config.ConsumerEndpointFactoryBean#6
2015-09-14 16:31:13.219 INFO 52844 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 1073741823
2015-09-14 16:31:13.219 INFO 52844 --- [ main] ApplicationEventListeningMessageProducer : started org.springframework.integration.event.inbound.ApplicationEventListeningMessageProducer#0
2015-09-14 16:31:13.555 INFO 52844 --- [cTaskExecutor-1] o.s.amqp.rabbit.core.RabbitAdmin : Auto-declaring a non-durable, auto-delete, or exclusive Queue (4640c1c8-ff8f-45d7-8426-19d1b7a4cdb0) durable:false, auto-delete:true, exclusive:true. It will be redeclared if the broker stops and is restarted while the connection factory is alive, but all messages will be lost.
2015-09-14 16:31:13.572 INFO 52844 --- [ main] o.s.i.a.i.AmqpInboundChannelAdapter : started org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter#0
2015-09-14 16:31:13.573 INFO 52844 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 2147483647
2015-09-14 16:31:13.576 INFO 52844 --- [ main] c.n.h.c.m.e.HystrixMetricsPoller : Starting HystrixMetricsPoller
2015-09-14 16:31:13.609 INFO 52844 --- [ main] ration$HystrixMetricsPollerConfiguration : Starting poller
2015-09-14 16:31:13.803 INFO 52844 --- [ main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8088 (http)
2015-09-14 16:31:13.805 INFO 52844 --- [ main] com.ml.springboot.PolicyService : Started PolicyService in 22.544 seconds (JVM running for 23.564)
So it looks like PolicyService successfully connects to the message broker.
The Turbine AMQP server's end of log:
2015-09-14 16:58:05.887 INFO 51944 --- [ main] i.reactivex.netty.server.AbstractServer : Rx server started at port: 8989
2015-09-14 16:58:05.991 INFO 51944 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2015-09-14 16:58:05.991 INFO 51944 --- [ main] o.s.i.channel.PublishSubscribeChannel : Channel 'bootstrap:-1.errorChannel' has 1 subscriber(s).
2015-09-14 16:58:05.991 INFO 51944 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started _org.springframework.integration.errorLogger
2015-09-14 16:58:05.991 INFO 51944 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {bridge} as a subscriber to the 'hystrixStreamAggregatorInboundFlow.channel#0' channel
2015-09-14 16:58:05.991 INFO 51944 --- [ main] o.s.integration.channel.DirectChannel : Channel 'bootstrap:-1.hystrixStreamAggregatorInboundFlow.channel#0' has 1 subscriber(s).
2015-09-14 16:58:05.991 INFO 51944 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2015-09-14 16:58:05.991 INFO 51944 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 1073741823
2015-09-14 16:58:06.238 INFO 51944 --- [cTaskExecutor-1] o.s.amqp.rabbit.core.RabbitAdmin : Auto-declaring a non-durable, auto-delete, or exclusive Queue (spring.cloud.hystrix.stream) durable:false, auto-delete:false, exclusive:false. It will be redeclared if the broker stops and is restarted while the connection factory is alive, but all messages will be lost.
2015-09-14 16:58:06.289 INFO 51944 --- [ main] o.s.i.a.i.AmqpInboundChannelAdapter : started org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter#0
2015-09-14 16:58:06.290 INFO 51944 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 2147483647
2015-09-14 16:58:06.434 INFO 51944 --- [ main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): -1 (http)
Any ideas why the Turbine AMQP server is not receiving communication from the Hystrix AMQP client?
EDIT: Turbine-AMQP main looks like:
package com.turbine.amqp;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.cloud.netflix.turbine.amqp.EnableTurbineAmqp;
import org.springframework.context.annotation.Configuration;
#Configuration
#EnableAutoConfiguration
#EnableTurbineAmqp
#EnableDiscoveryClient
public class TurbineAmqpApplication {
public static void main(String[] args) {
SpringApplication.run(TurbineAmqpApplication.class, args);
}
}
Here's its application.yml:
server:
port: 8989
spring:
rabbitmq:
addresses: ${vcap.services.${PREFIX:}rabbitmq.credentials.uri:amqp://${RABBITMQ_HOST:localhost}:${RABBITMQ_PORT:5672}}
Hitting http://localhost:8989/turbine.stream produces a repeating stream of data: {"type":"Ping"}
and shows this in console:
2015-09-15 08:54:37.960 INFO 83480 --- [o-eventloop-3-1] o.s.c.n.t.amqp.TurbineAmqpConfiguration : SSE Request Received
2015-09-15 08:54:38.025 INFO 83480 --- [o-eventloop-3-1] o.s.c.n.t.amqp.TurbineAmqpConfiguration : Starting aggregation
EDIT: The below exception is thrown when I stop listening to the turbine stream, not when I try to listen with the dashboard.
2015-09-15 08:56:47.934 INFO 83480 --- [o-eventloop-3-3] o.s.c.n.t.amqp.TurbineAmqpConfiguration : SSE Request Received
2015-09-15 08:56:47.946 WARN 83480 --- [o-eventloop-3-3] io.netty.channel.DefaultChannelPipeline : An exception was thrown by a user handler's exceptionCaught() method while handling the following exception:
java.lang.NoSuchMethodError: rx.Observable.collect(Lrx/functions/Func0;Lrx/functions/Action2;)Lrx/Observable;
at com.netflix.turbine.aggregator.StreamAggregator.lambda$null$36(StreamAggregator.java:89)
at rx.internal.operators.OnSubscribeMulticastSelector.call(OnSubscribeMulticastSelector.java:60)
at rx.internal.operators.OnSubscribeMulticastSelector.call(OnSubscribeMulticastSelector.java:40)
at rx.Observable.unsafeSubscribe(Observable.java:8591)
at rx.internal.operators.OperatorMerge$MergeSubscriber.handleNewSource(OperatorMerge.java:190)
at rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:160)
at rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:96)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.operators.OperatorGroupBy$GroupBySubscriber.onNext(OperatorGroupBy.java:173)
at rx.subjects.SubjectSubscriptionManager$SubjectObserver.onNext(SubjectSubscriptionManager.java:224)
at rx.subjects.PublishSubject.onNext(PublishSubject.java:101)
at org.springframework.cloud.netflix.turbine.amqp.Aggregator.handle(Aggregator.java:53)
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.expression.spel.support.ReflectiveMethodExecutor.execute(ReflectiveMethodExecutor.java:112)
at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:102)
at org.springframework.expression.spel.ast.MethodReference.access$000(MethodReference.java:49)
at org.springframework.expression.spel.ast.MethodReference$MethodValueRef.getValue(MethodReference.java:342)
at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:88)
at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:131)
at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:330)
at org.springframework.integration.util.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:164)
at org.springframework.integration.util.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:276)
at org.springframework.integration.util.MessagingMethodInvokerHelper.process(MessagingMethodInvokerHelper.java:142)
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:75)
at org.springframework.integration.handler.ServiceActivatingHandler.handleRequestMessage(ServiceActivatingHandler.java:71)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:99)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:277)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:239)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:95)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:248)
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:171)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:119)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:105)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:277)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:239)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:95)
at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:101)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter.access$400(AmqpInboundChannelAdapter.java:45)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$1.onMessage(AmqpInboundChannelAdapter.java:93)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:756)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:679)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$001(SimpleMessageListenerContainer.java:82)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1.invokeListener(SimpleMessageListenerContainer.java:167)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.invokeListener(SimpleMessageListenerContainer.java:1241)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:660)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:1005)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:989)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$700(SimpleMessageListenerContainer.java:82)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1103)
at java.lang.Thread.run(Thread.java:745)
Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: GroupedObservable.class
at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:98)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:56)
... 58 common frames omitted
My dependencies for turbine-amqp are as follows:
dependencies {
compile('org.springframework.cloud:spring-cloud-starter-turbine-amqp:1.0.3.RELEASE')
compile 'org.springframework.boot:spring-boot-starter-web:1.2.5.RELEASE'
compile 'org.springframework.boot:spring-boot-starter-actuator:1.2.5.RELEASE'
testCompile("org.springframework.boot:spring-boot-starter-test")
}
dependencyManagement {
imports {
mavenBom 'org.springframework.cloud:spring-cloud-starter-parent:1.0.2.RELEASE'
}
}
It is so hard to find a solution.
Using Spring cloud 2.1.4.RELEASE I faced with similar problem.
The main cause is the incompatibility [exchanges] name in rabbitMQ between:
spring-cloud-netflix-hystrix-stream and spring-cloud-starter-netflix-turbine-stream.
So solve it:
See the name created exchange name when you start the service componente {the same that declare hystrix-stream}
on the componente that declare {turbine-stream}
update the property
turbine.stream.destination=
in my case
turbine.stream.destination=hystrixStreamOutput
I faced with similar problem and I find a solution.
My Spring Cloud version is 2.1.0.RELEASE
The solution:
add property
spring.cloud.stream.bindings.turbineStreamInput.destination: hystrixStreamOutput
turbine.stream.enabled: false
add auto configuration
#EnableBinding(TurbineStreamClient.class)
public class TurbineStreamAutoConfiguration {
#Autowired
private BindingServiceProperties bindings;
#Autowired
private TurbineStreamProperties properties;
#PostConstruct
public void init() {
BindingProperties inputBinding = this.bindings.getBindings()
.get(TurbineStreamClient.INPUT);
if (inputBinding == null) {
this.bindings.getBindings().put(TurbineStreamClient.INPUT,
new BindingProperties());
}
BindingProperties input = this.bindings.getBindings()
.get(TurbineStreamClient.INPUT);
if (input.getDestination() == null) {
input.setDestination(this.properties.getDestination());
}
if (input.getContentType() == null) {
input.setContentType(this.properties.getContentType());
}
}
#Bean
public HystrixStreamAggregator hystrixStreamAggregator(ObjectMapper mapper,
PublishSubject<Map<String, Object>> publisher) {
return new HystrixStreamAggregator(mapper, publisher);
}
}