Spring integration errorChannel no messages in JUnit - error-handling
I have a simple JUnit test in which I am testing errorChannel by hooking up a service activator. However, nothing comes to this channel
the config is as follows,
<int:channel id="in"/>
<int:service-activator id="inSA" input-channel="in" ref="thrower"
output-channel="nullChannel"/>
<bean id="thrower"
class="com.att.datalake.ifr.loader.exceptions.handler.Thrower" />
<int:service-activator input-channel="errorChannel" ref="errorManager"
id="errorMgr" method="handleMessage" />
<bean id="errorManager"
class="com.att.datalake.ifr.loader.exceptions.handlers.ErrorManager"/>
The ErrorMgr is as follows:
public class ErrorManager {
#ServiceActivator
public void handleMessage(Message<MessageHandlingException> exception) {
System.out.println("In cleaner");
}
}
The Thrower class is as follows:
public class Thrower {
public Message<FileMessage> process(final Message<FileMessage> message) throws PublisherException {
System.out.println("In the Thrower SA at:"+new Date());
// get error type
String errorType = (String) message.getHeaders().get("ERROR_TYPE");
if (errorType.equals("PublisherError")) {
System.out.println("*****About to throw");
throw new PublisherException("Simple Publisher Exception at:"+new Date());
}
return message;
}
}
The Thrower SA simply throws an exception, which is reported by JUnit test, but nothing goes to errorChannel
The log in debug is :
2015-01-23 10:53:20.597 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'integrationHeaderChannelRegistry'
2015-01-23 10:53:20.597 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'globalChannelInterceptorProcessor'
2015-01-23 10:53:20.598 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'inSA'
2015-01-23 10:53:20.598 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'errorMgr'
2015-01-23 10:53:20.598 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean '_org.springframework.integration.errorLogger'
2015-01-23 10:53:20.598 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'lifecycleProcessor'
2015-01-23 10:53:20.599 INFO --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase -2147483648
2015-01-23 10:53:20.600 DEBUG --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting bean 'globalChannelInterceptorProcessor' of type [class org.springframework.integration.config.GlobalChannelInterceptorProcessor]
2015-01-23 10:53:20.602 DEBUG --- [ main] .s.i.c.GlobalChannelInterceptorProcessor : No global channel interceptors.
2015-01-23 10:53:20.603 DEBUG --- [ main] o.s.c.support.DefaultLifecycleProcessor : Successfully started bean 'globalChannelInterceptorProcessor'
2015-01-23 10:53:20.604 INFO --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting beans in phase 0
2015-01-23 10:53:20.605 DEBUG --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting bean 'inSA' of type [class org.springframework.integration.config.ConsumerEndpointFactoryBean]
2015-01-23 10:53:20.605 INFO --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {service-activator:inSA} as a subscriber to the 'in' channel
2015-01-23 10:53:20.606 INFO --- [ main] o.s.integration.channel.DirectChannel : Channel 'org.springframework.context.support.GenericApplicationContext#6f3b5d16.in' has 1 subscriber(s).
2015-01-23 10:53:20.606 INFO --- [ main] o.s.i.endpoint.EventDrivenConsumer : started inSA
2015-01-23 10:53:20.606 DEBUG --- [ main] o.s.c.support.DefaultLifecycleProcessor : Successfully started bean 'inSA'
2015-01-23 10:53:20.606 DEBUG --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting bean 'errorMgr' of type [class org.springframework.integration.config.ConsumerEndpointFactoryBean]
2015-01-23 10:53:20.607 INFO --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {service-activator:errorMgr} as a subscriber to the 'errorChannel' channel
2015-01-23 10:53:20.607 INFO --- [ main] o.s.i.channel.PublishSubscribeChannel : Channel 'org.springframework.context.support.GenericApplicationContext#6f3b5d16.errorChannel' has 1 subscriber(s).
2015-01-23 10:53:20.607 INFO --- [ main] o.s.i.endpoint.EventDrivenConsumer : started errorMgr
2015-01-23 10:53:20.607 DEBUG --- [ main] o.s.c.support.DefaultLifecycleProcessor : Successfully started bean 'errorMgr'
2015-01-23 10:53:20.607 DEBUG --- [ main] o.s.c.support.DefaultLifecycleProcessor : Starting bean '_org.springframework.integration.errorLogger' of type [class org.springframework.integration.config.ConsumerEndpointFactoryBean]
2015-01-23 10:53:20.607 INFO --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2015-01-23 10:53:20.607 INFO --- [ main] o.s.i.channel.PublishSubscribeChannel : Channel 'org.springframework.context.support.GenericApplicationContext#6f3b5d16.errorChannel' has 2 subscriber(s).
2015-01-23 10:53:20.607 INFO --- [ main] o.s.i.endpoint.EventDrivenConsumer : started _org.springframework.integration.errorLogger
2015-01-23 10:53:20.608 DEBUG --- [ main] o.s.c.support.DefaultLifecycleProcessor : Successfully started bean '_org.springframework.integration.errorLogger'
2015-01-23 10:53:20.610 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'org.springframework.integration.config.IdGeneratorConfigurer#0'
2015-01-23 10:53:20.612 DEBUG --- [ main] o.s.c.e.PropertySourcesPropertyResolver : Searching for key 'spring.liveBeansView.mbeanDomain' in [systemProperties]
2015-01-23 10:53:20.612 DEBUG --- [ main] o.s.c.e.PropertySourcesPropertyResolver : Searching for key 'spring.liveBeansView.mbeanDomain' in [systemEnvironment]
2015-01-23 10:53:20.613 DEBUG --- [ main] o.s.c.e.PropertySourcesPropertyResolver : Could not find key 'spring.liveBeansView.mbeanDomain' in any property source. Returning [null]
2015-01-23 10:53:20.614 DEBUG --- [ main] c.DefaultCacheAwareContextLoaderDelegate : Storing ApplicationContext in cache under key [[MergedContextConfiguration#105fece7 testClass = ErrorHandlerTests, locations = '{classpath:com/att/datalake/ifr/loader/flowtests/ErrorHandlerTests-context.xml}', classes = '{}', contextInitializerClasses = '[]', activeProfiles = '{}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextLoader = 'org.springframework.test.context.support.DelegatingSmartContextLoader', parent = [null]]]
2015-01-23 10:53:20.615 DEBUG --- [ main] org.springframework.test.context.cache : Spring test ApplicationContext cache statistics: [ContextCache#7a1a14a4 size = 1, hitCount = 0, missCount = 1, parentContextCount = 0]
2015-01-23 10:53:20.620 DEBUG --- [ main] o.s.b.f.annotation.InjectionMetadata : Processing injected method of bean 'com.att.datalake.ifr.loader.flowtests.ErrorHandlerTests': AutowiredFieldElement for private org.springframework.messaging.MessageChannel com.att.datalake.ifr.loader.flowtests.ErrorHandlerTests.in
2015-01-23 10:53:20.623 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'in'
2015-01-23 10:53:20.623 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'nullChannel'
2015-01-23 10:53:20.624 DEBUG --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'errorChannel'
2015-01-23 10:53:20.625 DEBUG --- [ main] f.a.AutowiredAnnotationBeanPostProcessor : Autowiring by type from bean name 'com.att.datalake.ifr.loader.flowtests.ErrorHandlerTests' to bean named 'in'
2015-01-23 10:53:20.642 DEBUG --- [ main] o.s.i.handler.ServiceActivatingHandler : ServiceActivator for [org.springframework.integration.handler.MethodInvokingMessageProcessor#49ff7d8c] (inSA) received message: GenericMessage [payload=FileMessage [name=2.txt, path=input/2.txt], headers={ERROR_TYPE=PublisherError, correlationId=1, id=dd662ff5-9790-77e2-cceb-f43afc97de81, timestamp=1422028400642}]
In the Thrower SA at:Fri Jan 23 10:53:20 EST 2015
*****About to throw
2015-01-23 10:53:20.648 DEBUG --- [ main] .c.s.DirtiesContextTestExecutionListener : After test method: context [DefaultTestContext#40e6dfe1 testClass = ErrorHandlerTests, testInstance = com.att.datalake.ifr.loader.flowtests.ErrorHandlerTests#1b083826, testMethod = test#ErrorHandlerTests, testException = org.springframework.messaging.MessageHandlingException: ; nested exception is com.att.datalake.ifr.loader.exceptions.PublisherException: Simple Publisher Exception at:Fri Jan 23 10:53:20 EST 2015, mergedContextConfiguration = [MergedContextConfiguration#105fece7 testClass = ErrorHandlerTests, locations = '{classpath:com/att/datalake/ifr/loader/flowtests/ErrorHandlerTests-context.xml}', classes = '{}', contextInitializerClasses = '[]', activeProfiles = '{}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextLoader = 'org.springframework.test.context.support.DelegatingSmartContextLoader', parent = [null]]], class dirties context [false], class mode [null], method dirties context [false].
2015-01-23 10:53:20.657 DEBUG --- [ main] .c.s.DirtiesContextTestExecutionListener : After test class: context [DefaultTestContext#40e6dfe1 testClass = ErrorHandlerTests, testInstance = [null], testMethod = [null], testException = [null], mergedContextConfiguration = [MergedContextConfiguration#105fece7 testClass = ErrorHandlerTests, locations = '{classpath:com/att/datalake/ifr/loader/flowtests/ErrorHandlerTests-context.xml}', classes = '{}', contextInitializerClasses = '[]', activeProfiles = '{}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextLoader = 'org.springframework.test.context.support.DelegatingSmartContextLoader', parent = [null]]], dirtiesContext [false].
2015-01-23 10:53:20.662 INFO --- [ Thread-0] o.s.c.support.GenericApplicationContext : Closing org.springframework.context.support.GenericApplicationContext#6f3b5d16: startup date [Fri Jan 23 10:53:19 EST 2015]; root of context hierarchy
2015-01-23 10:53:20.663 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'org.springframework.integration.config.IdGeneratorConfigurer#0'
2015-01-23 10:53:20.664 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'integrationHeaderChannelRegistry'
2015-01-23 10:53:20.664 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'globalChannelInterceptorProcessor'
2015-01-23 10:53:20.664 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'inSA'
2015-01-23 10:53:20.664 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'errorMgr'
2015-01-23 10:53:20.664 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean '_org.springframework.integration.errorLogger'
2015-01-23 10:53:20.665 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'lifecycleProcessor'
2015-01-23 10:53:20.665 INFO --- [ Thread-0] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 0
2015-01-23 10:53:20.666 DEBUG --- [ Thread-0] o.s.c.support.DefaultLifecycleProcessor : Asking bean 'inSA' of type [class org.springframework.integration.config.ConsumerEndpointFactoryBean] to stop
2015-01-23 10:53:20.667 INFO --- [ Thread-0] o.s.i.endpoint.EventDrivenConsumer : Removing {service-activator:inSA} as a subscriber to the 'in' channel
2015-01-23 10:53:20.667 INFO --- [ Thread-0] o.s.integration.channel.DirectChannel : Channel 'org.springframework.context.support.GenericApplicationContext#6f3b5d16.in' has 0 subscriber(s).
2015-01-23 10:53:20.667 DEBUG --- [ Thread-0] o.s.c.support.DefaultLifecycleProcessor : Bean 'inSA' completed its stop procedure
2015-01-23 10:53:20.667 INFO --- [ Thread-0] o.s.i.endpoint.EventDrivenConsumer : stopped inSA
2015-01-23 10:53:20.667 DEBUG --- [ Thread-0] o.s.c.support.DefaultLifecycleProcessor : Asking bean 'errorMgr' of type [class org.springframework.integration.config.ConsumerEndpointFactoryBean] to stop
2015-01-23 10:53:20.667 INFO --- [ Thread-0] o.s.i.endpoint.EventDrivenConsumer : Removing {service-activator:errorMgr} as a subscriber to the 'errorChannel' channel
2015-01-23 10:53:20.667 INFO --- [ Thread-0] o.s.i.channel.PublishSubscribeChannel : Channel 'org.springframework.context.support.GenericApplicationContext#6f3b5d16.errorChannel' has 1 subscriber(s).
2015-01-23 10:53:20.668 DEBUG --- [ Thread-0] o.s.c.support.DefaultLifecycleProcessor : Bean 'errorMgr' completed its stop procedure
2015-01-23 10:53:20.668 INFO --- [ Thread-0] o.s.i.endpoint.EventDrivenConsumer : stopped errorMgr
2015-01-23 10:53:20.668 DEBUG --- [ Thread-0] o.s.c.support.DefaultLifecycleProcessor : Asking bean '_org.springframework.integration.errorLogger' of type [class org.springframework.integration.config.ConsumerEndpointFactoryBean] to stop
2015-01-23 10:53:20.668 INFO --- [ Thread-0] o.s.i.endpoint.EventDrivenConsumer : Removing {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2015-01-23 10:53:20.668 INFO --- [ Thread-0] o.s.i.channel.PublishSubscribeChannel : Channel 'org.springframework.context.support.GenericApplicationContext#6f3b5d16.errorChannel' has 0 subscriber(s).
2015-01-23 10:53:20.668 DEBUG --- [ Thread-0] o.s.c.support.DefaultLifecycleProcessor : Bean '_org.springframework.integration.errorLogger' completed its stop procedure
2015-01-23 10:53:20.668 INFO --- [ Thread-0] o.s.i.endpoint.EventDrivenConsumer : stopped _org.springframework.integration.errorLogger
2015-01-23 10:53:20.668 INFO --- [ Thread-0] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase -2147483648
2015-01-23 10:53:20.668 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory#25ce9dc4: defining beans [org.springframework.context.support.PropertySourcesPlaceholderConfigurer#0,channelInitializer,$autoCreateChannelCandidates,IntegrationConfigurationBeanFactoryPostProcessor,integrationEvaluationContext,org.springframework.integration.expression.IntegrationEvaluationContextAwareBeanPostProcessor#0,integrationGlobalProperties,integrationHeaderChannelRegistry,globalChannelInterceptorProcessor,toStringFriendlyJsonNodeToStringConverter,converterRegistrar,integrationConversionService,DefaultConfiguringBeanFactoryPostProcessor,datatypeChannelMessageConverter,messageBuilderFactory,in,org.springframework.integration.config.ServiceActivatorFactoryBean#0,inSA,thrower,org.springframework.integration.config.ServiceActivatorFactoryBean#1,fixedSubscriberChannelBeanFactoryPostProcessor,errorMgr,errorManager,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.internalPersistenceAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor.importAwareProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor.enhancedConfigurationProcessor,nullChannel,errorChannel,_org.springframework.integration.errorLogger,taskScheduler,org.springframework.integration.config.IdGeneratorConfigurer#0]; root of factory hierarchy
2015-01-23 10:53:20.694 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Retrieved dependent beans for bean '(inner bean)#74589991': [_org.springframework.integration.errorLogger]
2015-01-23 10:53:20.696 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Retrieved dependent beans for bean 'in': [com.att.datalake.ifr.loader.flowtests.ErrorHandlerTests]
2015-01-23 10:53:20.697 DEBUG --- [ Thread-0] o.s.b.f.support.DisposableBeanAdapter : Invoking destroy() on bean with name 'taskScheduler'
2015-01-23 10:53:20.698 INFO --- [ Thread-0] o.s.s.c.ThreadPoolTaskScheduler : Shutting down ExecutorService 'taskScheduler'
2015-01-23 10:53:20.699 DEBUG --- [ Thread-0] o.s.b.f.s.DefaultListableBeanFactory : Retrieved dependent beans for bean '(inner bean)#4b213651': [taskScheduler]
The ErrorMessage with an Exception is thrown to the error-channel only if the downstream flow is within the separate Thread. Otherwise the Exception is rethrown to the call. It is thrown to the caller anyway, but shifted to the error-channel, if there is no original caller control from Spring Integration.
See more info here: http://docs.spring.io/spring-integration/docs/latest-ga/reference/html/configuration.html#namespace-errorhandler
Related
TestContainer Rabbitmq seems to release connection as soon as it is start
I am using testcontainers in a spring boot project (version : 1.17.2) and I am trying to spin up a rabbitmq container. It seems like rabbitmq container starts up successfully, but it releases connection as soon as it is up. I can see some error in logs but after that I can see that the test container is started. I am kind of flummoxed as to why am I seeing this error and/or if the container is started or not ? Pasting excerpt from logs : 15:00:44.007 [main] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDOUT: 2022-06-29 05:00:24.477316+00:00 [info] <0.703.0> * rabbitmq_management_agent 15:00:44.007 [main] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDOUT: 2022-06-29 05:00:24.477316+00:00 [info] <0.703.0> * rabbitmq_web_dispatch 15:00:44.007 [main] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDOUT: 2022-06-29 05:00:24.477316+00:00 [info] <0.703.0> * rabbitmq_management 15:00:44.007 [main] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDOUT: 2022-06-29 05:00:24.477316+00:00 [info] <0.703.0> * rabbitmq_prometheus 15:00:44.007 [main] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDOUT: 2022-06-29 05:00:24.477316+00:00 [info] <0.703.0> Server startup complete; 4 plugins started. 15:00:44.007 [main] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.classic.InternalHttpClient - ep-0000000C: cancel 15:00:44.007 [main] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection - http-outgoing-1: close connection IMMEDIATE 15:00:44.008 [main] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.classic.InternalHttpClient - ep-0000000C: endpoint closed 15:00:44.008 [main] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.classic.InternalHttpClient - ep-0000000C: discarding endpoint 15:00:44.008 [main] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager - ep-0000000C: releasing endpoint 15:00:44.008 [main] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager - ep-0000000C: connection is not kept alive 15:00:44.008 [docker-java-stream--540868428] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-1 << "end of stream" 15:00:44.008 [main] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager - ep-0000000C: connection released [route: {}->npipe://localhost:2375][total available: 0; route allocated: 0 of 2147483647; total allocated: 0 of 2147483647] 15:00:44.008 [main] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-1 << "[read] I/O error: java.nio.channels.ClosedChannelException" 15:00:44.008 [docker-java-stream--540868428] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-1 << "[read] I/O error: java.nio.channels.ClosedChannelException" 15:00:44.008 [docker-java-stream--540868428] DEBUG com.github.dockerjava.zerodep.ApacheDockerHttpClientImpl$ApacheResponse - Failed to close the response java.io.IOException: java.nio.channels.ClosedChannelException at java.base/java.nio.channels.Channels$2.read(Channels.java:240) at com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.io.LoggingInputStream.read(LoggingInputStream.java:81) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:261) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:147) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:314) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.io.Closer.close(Closer.java:48) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.http.impl.io.IncomingHttpEntity.close(IncomingHttpEntity.java:111) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.http.io.entity.HttpEntityWrapper.close(HttpEntityWrapper.java:120) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.io.Closer.close(Closer.java:48) at com.github.dockerjava.zerodep.shaded.org.apache.hc.core5.http.message.BasicClassicHttpResponse.close(BasicClassicHttpResponse.java:93) at com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.classic.CloseableHttpResponse.close(CloseableHttpResponse.java:200) at com.github.dockerjava.zerodep.ApacheDockerHttpClientImpl$ApacheResponse.close(ApacheDockerHttpClientImpl.java:256) at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:277) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: java.nio.channels.ClosedChannelException: null .................................... 15:00:44.009 [main] INFO 🐳 [rabbitmq:3.9.13-management-alpine] - Container rabbitmq:3.9.13-management-alpine started in PT7.8752359S Config Java : public abstract class RabbitMqTestContainerConfiguration { private static final int RABBITMQ_DEFAULT_PORT = 5672; #Container public static RabbitMQContainer rabbitMQContainer = new RabbitMQContainer("rabbitmq:3.9.13-management-alpine") .withExposedPorts(RABBITMQ_DEFAULT_PORT).withStartupTimeout(Duration.ofMinutes(3)); public static class Initializer implements ApplicationContextInitializer<ConfigurableApplicationContext> { #Override public void initialize(ConfigurableApplicationContext configurableApplicationContext) { TestPropertySourceUtils.addInlinedPropertiesToEnvironment(configurableApplicationContext, "spring.rabbitmq.host=" + rabbitMQContainer.getHost(), "spring.rabbitmq.port=" + rabbitMQContainer.getMappedPort(RABBITMQ_DEFAULT_PORT), "spring.rabbitmq.username=" + rabbitMQContainer.getAdminUsername(), "spring.rabbitmq.password=" + rabbitMQContainer.getAdminPassword()); } } } ```
Why Apache Camel result on Graceful shutdown
Hello I have an app written in Kotlin and using Spring boot Framework and Apache Camel: https://camel.apache.org/components/latest/smpp-component.html#_spring_boot_auto_configuration So I am new in Smpp protocol I am suppuse trying to connecto to SMC simulator so I guess my "client" should be wait for a message from SMC in this case when I run the application it appears just stop and anymore. As it just stops I would think something is wrong. #Bean fun camelContextConfiguration(): CamelContextConfiguration{ return object : CamelContextConfiguration { override fun afterApplicationStart(camelContext: CamelContext?) { logger.info("*********************************************************************************") //var query : Query? //val oldRoute : List<Route> = camelContext!!.routes camelContext!!.routes //logger.info("Old Routes") //logger.info(oldRoute.size.toString()) // for(route:Route in oldRoute) { // logger.info("Route with routeId {} is deleted",route.id) // camelContext.stopRoute(route.id) // camelContext.removeRoute(route.id) // } camelContext.isUseMDCLogging=true logger.info("ROUTES") val routeDefinition = RouteDefinition() routeDefinition.id="1" routeDefinition.onException(Exception::class.java).handled(true) .process(exceptionProcessor()) .end() routeDefinition.from("smpp://smppclient#smscsim.melroselabs.com:2775?password=password&enquireLinkTimer=3000&transactionTimer=5000&systemId=145244&systemType='Receiver'") routeDefinition.to("log:com.app.smppdispenser.infraestructure.smpp?showBody=true&showHeaders=true") routeDefinition.process(messageProcessor()) camelContext.addRouteDefinition(routeDefinition) //routeDefinition.setHeader("CamelSmppCommandId", constant("query_sm")) //routeDefinition.setHeader("CamelSmppSequenceNumber", constant("1")) logger.info("Route is added with routeId {} ",routeDefinition.id) //} logger.info("CamelConfiguration afterApplicationStart OK") } override fun beforeApplicationStart(camelContext: CamelContext?) { logger.info("CamelConfiguration beforeApplicationStart") } } } This is the log> 2021-06-06 10:37:06.824 INFO 5604 --- [ main] o.a.c.s.boot.CamelContextConfiguration : ROUTES 2021-06-06 10:37:07.538 INFO 5604 --- [ main] org.jsmpp.session.SMPPSession : Connected to smscsim.melroselabs.com/18.200.51.80 2021-06-06 10:37:07.541 INFO 5604 --- [ession#31859960] org.jsmpp.session.SMPPSession : Starting PDUReaderWorker 2021-06-06 10:37:07.817 INFO 5604 --- [ main] org.jsmpp.session.SMPPSession : Other side reports SMPP interface version 34 2021-06-06 10:37:07.817 INFO 5604 --- [ main] org.jsmpp.session.SMPPSession : Changing processor degree to 3 2021-06-06 10:37:07.819 INFO 5604 --- [ main] o.a.camel.component.smpp.SmppConsumer : Connected to: smpp://145244#smscsim.melroselabs.com:2775 2021-06-06 10:37:07.820 INFO 5604 --- [ main] o.a.camel.spring.SpringCamelContext : Route: 1 started and consuming from: smpp://smppclient#smscsim.melroselabs.com:2775?enquireLinkTimer=3000&password=xxxxxx&systemId=145244&systemType=%27Receiver%27&transactionTimer=5000 2021-06-06 10:37:07.821 INFO 5604 --- [ main] o.a.c.s.boot.CamelContextConfiguration : Route is added with routeId 1 2021-06-06 10:37:07.821 INFO 5604 --- [ main] o.a.c.s.boot.CamelContextConfiguration : CamelConfiguration afterApplicationStart OK 2021-06-06 10:37:07.936 INFO 5604 --- [ Thread-2] o.s.w.c.s.GenericWebApplicationContext : Closing org.springframework.web.context.support.GenericWebApplicationContext#2cf3d63b: startup date [Sun Jun 06 10:36:55 COT 2021]; root of context hierarchy 2021-06-06 10:37:07.944 INFO 5604 --- [ Thread-2] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 2147483647 2021-06-06 10:37:07.945 INFO 5604 --- [ Thread-2] o.a.camel.spring.SpringCamelContext : Apache Camel 2.20.1 (CamelContext: camel-1) is shutting down 2021-06-06 10:37:07.946 INFO 5604 --- [ Thread-2] o.a.camel.impl.DefaultShutdownStrategy : Starting to graceful shutdown 1 routes (timeout 300 seconds) 2021-06-06 10:37:08.163 WARN 5604 --- [ession#31859960] org.jsmpp.session.SMPPSession : IOException while reading: Socket Closed 2021-06-06 10:37:08.164 INFO 5604 --- [ - ShutdownTask] o.a.camel.component.smpp.SmppConsumer : Disconnected from: smpp://145244#smscsim.melroselabs.com:2775 2021-06-06 10:37:08.164 INFO 5604 --- [ - ShutdownTask] o.a.camel.impl.DefaultShutdownStrategy : Route: 1 shutdown complete, was consuming from: smpp://smppclient#smscsim.melroselabs.com:2775?enquireLinkTimer=3000&password=xxxxxx&systemId=145244&systemType=%27Receiver%27&transactionTimer=5000 2021-06-06 10:37:08.165 INFO 5604 --- [ession#31859960] org.jsmpp.session.SMPPSession : PDUReaderWorker stop 2021-06-06 10:37:08.166 INFO 5604 --- [ Thread-2] o.a.camel.impl.DefaultShutdownStrategy : Graceful shutdown of 1 routes completed in 0 seconds 2021-06-06 10:37:08.202 INFO 5604 --- [ Thread-2] o.a.camel.spring.SpringCamelContext : Apache Camel 2.20.1 (CamelContext: camel-1) uptime 1.629 seconds 2021-06-06 10:37:08.203 INFO 5604 --- [ Thread-2] o.a.camel.spring.SpringCamelContext : Apache Camel 2.20.1 (CamelContext: camel-1) is shutdown in 0.257 seconds Process finished with exit code 0 Edit I have added thse option: camel.springboot.main-run-controller = true then I see this new line in my log but it finish of the same way.
there are some options: add a spring boot starter web which starts a web server and keep the application running use a property as explained here: https://camel.apache.org/camel-spring-boot/3.7.x/spring-boot.html#SpringBoot-Keepingapplicationalive
Field 'id' doesn't have a default value. AbstractPersistable problem
I'm crating webservice with connection to db. I have met one problem. In my User class i used AbstractPersistable to generate primary keys but it seems to have some problem with that task. import javax.persistence.Entity; import org.springframework.data.jpa.domain.AbstractPersistable; #Entity public class User extends AbstractPersistable<Long>{ private String userId; private String userName; private String password; public String getUserId() { return userId; } public void setUserId(String userId) { this.userId = userId; } public String getUserName() { return userName; } public void setUserName(String userName) { this.userName = userName; } public String getPassword() { return password; } public void setPassword(String password) { this.password = password; } } It throws me something like that: . ____ _ __ _ _ /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \ ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \ \\/ ___)| |_)| | | | | || (_| | ) ) ) ) ' |____| .__|_| |_|_| |_\__, | / / / / =========|_|==============|___/=/_/_/_/ :: Spring Boot :: (v2.3.0.RELEASE) 2020-05-27 14:22:09.345 INFO 15080 --- [ restartedMain] com.webservice.Application : Starting Application on DESKTOP-7IMDU22 with PID 15080 (C:\Users\Daniel\eclipse-workspace\MyWebService\target\classes started by Daniel in C:\Users\Daniel\eclipse-workspace\MyWebService) 2020-05-27 14:22:09.348 INFO 15080 --- [ restartedMain] com.webservice.Application : No active profile set, falling back to default profiles: default 2020-05-27 14:22:09.405 INFO 15080 --- [ restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools property defaults active! Set 'spring.devtools.add-properties' to 'false' to disable 2020-05-27 14:22:09.405 INFO 15080 --- [ restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : For additional web related logging consider setting the 'logging.level.web' property to 'DEBUG' 2020-05-27 14:22:10.477 INFO 15080 --- [ restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFERRED mode. 2020-05-27 14:22:10.502 INFO 15080 --- [ restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 16ms. Found 0 JPA repository interfaces. 2020-05-27 14:22:11.549 INFO 15080 --- [ restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 9090 (http) 2020-05-27 14:22:11.560 INFO 15080 --- [ restartedMain] o.apache.catalina.core.StandardService : Starting service [Tomcat] 2020-05-27 14:22:11.560 INFO 15080 --- [ restartedMain] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.35] 2020-05-27 14:22:11.684 INFO 15080 --- [ restartedMain] o.a.c.c.C.[.[localhost].[/springboot] : Initializing Spring embedded WebApplicationContext 2020-05-27 14:22:11.684 INFO 15080 --- [ restartedMain] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 2279 ms 2020-05-27 14:22:12.050 INFO 15080 --- [ restartedMain] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor' 2020-05-27 14:22:12.102 INFO 15080 --- [ task-1] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default] 2020-05-27 14:22:12.137 WARN 15080 --- [ restartedMain] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning 2020-05-27 14:22:12.170 INFO 15080 --- [ task-1] org.hibernate.Version : HHH000412: Hibernate ORM core version 5.4.15.Final 2020-05-27 14:22:12.358 INFO 15080 --- [ task-1] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.1.0.Final} 2020-05-27 14:22:12.490 INFO 15080 --- [ task-1] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting... 2020-05-27 14:22:12.741 INFO 15080 --- [ task-1] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed. 2020-05-27 14:22:12.758 INFO 15080 --- [ task-1] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MySQL5Dialect 2020-05-27 14:22:13.435 INFO 15080 --- [ task-1] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform] 2020-05-27 14:22:13.442 INFO 15080 --- [ task-1] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default' 2020-05-27 14:22:13.462 INFO 15080 --- [ task-2] o.s.b.d.a.OptionalLiveReloadServer : LiveReload server is running on port 35729 Exception in thread "task-2" org.springframework.jdbc.datasource.init.ScriptStatementFailedException: Failed to execute SQL script statement #2 of URL [file:/C:/Users/Daniel/eclipse-workspace/MyWebService/target/classes/data.sql]: insert into `User`(`user_id`,`user_name`,`password`) values ('user id 1', 'name 1' , 'pass1'), ('user id 2', 'name 2' , 'pass2'); nested exception is java.sql.SQLException: Field 'id' doesn't have a default value at org.springframework.jdbc.datasource.init.ScriptUtils.executeSqlScript(ScriptUtils.java:622) at org.springframework.jdbc.datasource.init.ResourceDatabasePopulator.populate(ResourceDatabasePopulator.java:254) at org.springframework.jdbc.datasource.init.DatabasePopulatorUtils.execute(DatabasePopulatorUtils.java:49) at org.springframework.boot.autoconfigure.jdbc.DataSourceInitializer.runScripts(DataSourceInitializer.java:203) at org.springframework.boot.autoconfigure.jdbc.DataSourceInitializer.initSchema(DataSourceInitializer.java:120) at org.springframework.boot.autoconfigure.jdbc.DataSourceInitializerInvoker.onApplicationEvent(DataSourceInitializerInvoker.java:91) at org.springframework.boot.autoconfigure.jdbc.DataSourceInitializerInvoker.onApplicationEvent(DataSourceInitializerInvoker.java:38) at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172) at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165) at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:403) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:360) at org.springframework.boot.autoconfigure.orm.jpa.DataSourceInitializedPublisher.publishEventIfRequired(DataSourceInitializedPublisher.java:99) at org.springframework.boot.autoconfigure.orm.jpa.DataSourceInitializedPublisher.access$100(DataSourceInitializedPublisher.java:50) at org.springframework.boot.autoconfigure.orm.jpa.DataSourceInitializedPublisher$DataSourceSchemaCreatedPublisher.lambda$postProcessEntityManagerFactory$0(DataSourceInitializedPublisher.java:200) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base/java.lang.Thread.run(Thread.java:844) Caused by: java.sql.SQLException: Field 'id' doesn't have a default value at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129) at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:764) at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:648) at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:95) at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java) at org.springframework.jdbc.datasource.init.ScriptUtils.executeSqlScript(ScriptUtils.java:601) ... 17 more 2020-05-27 14:22:13.531 INFO 15080 --- [ restartedMain] o.s.b.a.e.web.EndpointLinksResolver : Exposing 2 endpoint(s) beneath base path '/actuator' 2020-05-27 14:22:13.582 INFO 15080 --- [ restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 9090 (http) with context path '/springboot' 2020-05-27 14:22:13.583 INFO 15080 --- [ restartedMain] DeferredRepositoryInitializationListener : Triggering deferred initialization of Spring Data repositories… 2020-05-27 14:22:13.584 INFO 15080 --- [ restartedMain] DeferredRepositoryInitializationListener : Spring Data repositories initialized! 2020-05-27 14:22:13.597 INFO 15080 --- [ restartedMain] com.webservice.Application : Started Application in 4.64 seconds (JVM running for 5.221) It must generate primary keys but they dont have default value. Here is my data.sql file: use springboot; insert into `User`(`user_id`,`user_name`,`password`) values ('user id 1', 'name 1' , 'pass1'), ('user id 2', 'name 2' , 'pass2'); And aplication properties server.servlet.contextPath=/springboot server.port=9090 menagement.security.enabled=false #Hibernate spring.datasource.data=classpath*:data.sql spring.datasource.url=jdbc:mysql://localhost:3306/springboot spring.datasource.username=admin spring.datasource.password=admin spring.datasource.initialize=true spring.datasource.initialization-mode=always #Jpa spring.jpa.hibernate.ddl-auto=update spring.jpa.show-sql=true spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
The following changes worked for me: Application.properties file: spring.jpa.hibernate.ddl-auto=create Entity class: #Entity public class User { // extends AbstractPersistable<Long> #Id #GeneratedValue(strategy = GenerationType.IDENTITY) private Integer id; .. }
H2 db cannot be found during test execution
I want to use my profile 'dev' during the test execution. But when I run tests H2 db cannot be found. Abstract test class: #RunWith(SpringRunner.class) #SpringBootTest(classes = MyServiceStarter.class) #ActiveProfiles("dev") public abstract class AbstractModulIntegrationTest { ... } Properties: flyway.locations=filesystem:./database/h2 flyway.enabled=true # hsqldb spring.datasource.platform=h2 spring.jpa.hibernate.ddl-auto=validate spring.h2.console.enabled=true spring.datasource.url=jdbc:h2:./db/gcrs;AUTO_SERVER=TRUE spring.datasource.driver-class-name=org.h2.Driver spring.datasource.password= spring.datasource.username=sa spring.jpa.database=h2 spring.jpa.database-platform=org.hibernate.dialect.H2Dialect When I run test in the output it prints Unable to resolve location: filesystem:./database/h2 2017-03-14 18:05:24.500 INFO 7460 --- [ main] o.f.core.internal.util.VersionPrinter : Flyway 4.0.3 by Boxfuse 2017-03-14 18:05:24.500 INFO 7460 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Started. 2017-03-14 18:05:25.002 INFO 7460 --- [ main] o.f.c.i.dbsupport.DbSupportFactory : Database: jdbc:h2:./db/gcrs (H2 1.4) 2017-03-14 18:05:25.018 WARN 7460 --- [ main] o.f.c.i.u.s.f.FileSystemScanner : Unable to resolve location filesystem:./database/h2 2017-03-14 18:05:25.049 WARN 7460 --- [ main] o.f.c.i.u.s.f.FileSystemScanner : Unable to resolve location filesystem:./database/h2 2017-03-14 18:05:25.049 WARN 7460 --- [ main] o.f.c.i.u.s.f.FileSystemScanner : Unable to resolve location filesystem:./database/h2 2017-03-14 18:05:25.049 INFO 7460 --- [ main] o.f.core.internal.command.DbValidate : Successfully validated 0 migrations (execution time 00:00.006s) 2017-03-14 18:05:25.064 INFO 7460 --- [ main] o.f.core.internal.command.DbMigrate : Current version of schema "PUBLIC": << Empty Schema >> 2017-03-14 18:05:25.064 INFO 7460 --- [ main] o.f.core.internal.command.DbMigrate : Schema "PUBLIC" is up to date. No migration necessary. 2017-03-14 18:05:25.205 INFO 7460 --- [ main] j.LocalContainerEntityManagerFactoryBean : Building JPA container EntityManagerFactory for persistence unit 'default' 2017-03-14 18:05:25.220 INFO 7460 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [ name: default ...] Spring boot version: 1.5.1.RELEASE
My colleague found the solution. The problem was in the wrong path. This solved my issue: #SpringBootTest(classes = MyServiceStarter.class, properties = {"flyway.locations=filesystem:../database/h2"})
Error from Json Loader in Pig
I have got below error while writing json scripts.. Please let me know how to write json loader script in pig. script: x = LOAD 'hdfs://user/spanda20/pig/phone.dat' USING JsonLoader('id:chararray, phone:(home:{(num:chararray, city:chararray)})'); Data set: { "id": "12345", "phone": { "home": [ { "zip": "23060", "city": "henrico" }, { "zip": "08902", "city": "northbrunswick" } ] } } 2015-03-18 14:24:10,917 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 2015-03-18 14:24:10,918 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1426618756946_0028 has failed! Stop running all dependent jobs 2015-03-18 14:24:10,918 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete 2015-03-18 14:24:10,977 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate exception from backed error: AttemptID:attempt_1426618756946_0028_m_000000_3 Info:Error: org.codehaus.jackson.JsonParseException: Unexpected end-of-input: expected close marker for OBJECT (from [Source: java.io.ByteArrayInputStream#43c59008; line: 1, column: 0]) at [Source: java.io.ByteArrayInputStream#43c59008; line: 1, column: 3] at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291) at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:385) at org.codehaus.jackson.impl.JsonParserMinimalBase._reportInvalidEOF(JsonParserMinimalBase.java:318) at org.codehaus.jackson.impl.JsonParserBase._handleEOF(JsonParserBase.java:354) at org.codehaus.jackson.impl.Utf8StreamParser._skipWSOrEnd(Utf8StreamParser.java:1841) at org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:275) at org.apache.pig.builtin.JsonLoader.readField(JsonLoader.java:180) at org.apache.pig.builtin.JsonLoader.getNext(JsonLoader.java:164) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553) at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) 2015-03-18 14:24:10,977 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! 2015-03-18 14:24:10,978 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: HadoopVersion PigVersion UserId StartedAt FinishedAt Features 2.5.0-cdh5.2.0 0.12.0-cdh5.2.0 spanda20 2015-03-18 14:23:02 2015-03-18 14:24:10 UNKNOWN Regards Sanjeeb
Sanjeeb - Use this json: {"id":"12345","phone":{"home":[{"zip":"23060","city":"henrico"},{"zip":"08902","city":"northbrunswick"}]}} output shall be: (12345,({(23060,henrico),(08902,northbrunswick)})) PS: Pig doesn't usually like "human readable" json. Get rid of the spaces and/or indentations, and you're good.