Spring LDAP search for subgroups results in CommunicationException [root cause: UnknownHostException] - ldap

We have a plain java command line software which performs a recursive LDAP search with Spring LDAP, starting from a specified group and searching all the users from the specified groups and subgroups.
The Ldap search is configured as follows:
LdapContextSource ctxSrc = buildLdapContextSource();
PoolingContextSource poolingContextSource = buildPoolingContextSource(ctxSrc);
LdapTemplate ldapTemplate = new LdapTemplate(poolingContextSource);
private LdapContextSource buildLdapContextSource() {
LdapContextSource ctxSrc = new LdapContextSource();
ctxSrc.setUrl(config.getProperty(Config.LDAP_URL));
ctxSrc.setUserDn(config.getProperty(Config.LDAP_LOGIN_USERNAME));
ctxSrc.setPassword(config.getProperty(Config.LDAP_LOGIN_PASSWORD));
ctxSrc.setReferral("follow");
Map<String, Object> baseEnvironmentProperties = new HashMap<>();
baseEnvironmentProperties.put("java.naming.ldap.attributes.binary", "objectGUID");
ctxSrc.setBaseEnvironmentProperties(baseEnvironmentProperties);
ctxSrc.afterPropertiesSet();
return ctxSrc;
}
private PoolingContextSource buildPoolingContextSource(LdapContextSource ctxSrc) {
PoolingContextSource poolingContextSource = new MutablePoolingContextSource();
poolingContextSource.setContextSource(ctxSrc);
poolingContextSource.setDirContextValidator(new DefaultDirContextValidator());
poolingContextSource.setTestOnBorrow(true);
poolingContextSource.setTestWhileIdle(true);
return poolingContextSource;
}
The software performs an empty search (in case of refused connection or connection timeout), and then starts recursive to two different groups. The software searchs for possible subgroups with lines:
AndFilter filter = new AndFilter().and(new EqualsFilter("objectclass", "group")).and(new EqualsFilter("memberof", groupName));
List<String> subgroups = ldapTemplate.search(base, filter.encode(), SearchControls.SUBTREE_SCOPE, new GroupNameMapper());
GroupNameMapper maps group's distinguished name to String. Thus far the tool has worked with handful of customers just fine, but now we encountered a new problem.
The tool makes two different searches. Admins and normal users. The customer has provided the group DNs like:
CN=SyncAdminsGroup,OU=Services,DC=example,DC=org
CN=SyncNormalGroup,OU=Services,DC=example,DC=org
Base and Organisational Unit is same for both groups. The first search for admin group works fine. No recursive search were made, so the admins did not contain any subgroups.
The second results in an error:
ERROR - Encountered error with LDAP:
org.springframework.ldap.CommunicationException: example.org :389; nested exception is javax.naming.CommunicationException: example.org :389 [Root exception is java.net.UnknownHostException: example.org ]
at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:108)
at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:397)
at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:328)
at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:604)
at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:594)
at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:482)
at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:498)
at LDAP.LdapImporter.searchLdapGroup(LdapImporter.java:128)
...(omitted lines from the software)
Caused by: javax.naming.CommunicationException: example.org :389 [Root exception is java.net.UnknownHostException: example.org ]
at com.sun.jndi.ldap.LdapReferralContext.<init>(Unknown Source)
at com.sun.jndi.ldap.LdapReferralException.getReferralContext(Unknown Source)
at com.sun.jndi.ldap.LdapCtx.searchAux(Unknown Source)
at com.sun.jndi.ldap.LdapCtx.c_search(Unknown Source)
at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(Unknown Source)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(Unknown Source)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(Unknown Source)
at javax.naming.directory.InitialDirContext.search(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.ldap.pool.factory.DirContextPoolableObjectFactory$FailureAwareContextProxy.invoke(DirContextPoolableObjectFactory.java:271)
at com.sun.proxy.$Proxy17.search(Unknown Source)
at org.springframework.ldap.pool.DelegatingDirContext.search(DelegatingDirContext.java:347)
at org.springframework.ldap.core.LdapTemplate$4.executeSearch(LdapTemplate.java:322)
at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:363)
... 11 more
Caused by: java.net.UnknownHostException: example.org
In the exception, the "Unknown Host" is same as the search base, eg. CN=example,CN=org --> org.springframework.ldap.CommunicationException: example.org:389;
What could be the cause for such error? This error seems to indicate some kind of DNS problem, but since the first search with the same base works, it leaves me baffled.
Searching for the error, all others seem to encounter either "Connection refused" errors or PartialResultExceptions, which is not the case here.

Related

Payara DataSource not being injected

I'm using Payara 5.184 and Java 1.8.241 on Linux (Ubuntu 19.10). I'm also using Liquibase as my database schema change management. A field named myDataSource is being injected as follows:
#Resource(mappedName = "java:global/ds")
private DataSource myDataSource;
When the method createDataSource() from Liquibase is invoked I noticed that the variable myDataSource is null, what makes me understand that the resource is not being properly injected. This error seems to happen only on Linux, as my other colleagues that run Windows did not have any problems so far. We're using exactly the same versions of Payara/Java. Is there any specific step that needs to be done on Linux? Here it is the stacktrace found on Payara logs.
Exception during lifecycle processing
org.glassfish.deployment.common.DeploymentException: CDI deployment failure:Exception List with 2 exceptions:
Exception 0 :
org.jboss.weld.exceptions.WeldException: WELD-000049: Unable to invoke public void liquibase.integration.cdi.CDILiquibase.onStartup() on liquibase.integration.cdi.CDILiquibase#911a89a
at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:85)
at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.postConstruct(DefaultLifecycleCallbackInvoker.java:66)
at org.jboss.weld.injection.producer.BasicInjectionTarget.postConstruct(BasicInjectionTarget.java:122)
at liquibase.integration.cdi.CDIBootstrap$1.create(CDIBootstrap.java:95)
at liquibase.integration.cdi.CDIBootstrap$1.create(CDIBootstrap.java:38)
at org.jboss.weld.contexts.AbstractContext.get(AbstractContext.java:96)
at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:100)
at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:102)
at org.jboss.weld.bean.proxy.ProxyMethodHandler.getInstance(ProxyMethodHandler.java:131)
at liquibase.integration.cdi.CDILiquibase$Proxy$_$$_WeldClientProxy.toString(Unknown Source)
at liquibase.integration.cdi.CDIBootstrap.afterDeploymentValidation(CDIBootstrap.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:95)
at org.jboss.weld.injection.MethodInvocationStrategy$SpecialParamPlusBeanManagerStrategy.invoke(MethodInvocationStrategy.java:144)
at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:330)
at org.jboss.weld.event.ExtensionObserverMethodImpl.sendEvent(ExtensionObserverMethodImpl.java:123)
at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:308)
at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:286)
at javax.enterprise.inject.spi.ObserverMethod.notify(ObserverMethod.java:124)
at org.jboss.weld.util.Observers.notify(Observers.java:166)
at org.jboss.weld.event.ObserverNotifier.notifySyncObservers(ObserverNotifier.java:285)
at org.jboss.weld.event.ObserverNotifier.notify(ObserverNotifier.java:273)
at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:177)
at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:171)
at org.jboss.weld.bootstrap.events.AbstractContainerEvent.fire(AbstractContainerEvent.java:53)
at org.jboss.weld.bootstrap.events.AbstractDeploymentContainerEvent.fire(AbstractDeploymentContainerEvent.java:35)
at org.jboss.weld.bootstrap.events.AfterDeploymentValidationImpl.fire(AfterDeploymentValidationImpl.java:28)
at org.jboss.weld.bootstrap.WeldStartup.validateBeans(WeldStartup.java:499)
at org.jboss.weld.bootstrap.WeldBootstrap.validateBeans(WeldBootstrap.java:93)
at org.glassfish.weld.WeldDeployer.processApplicationLoaded(WeldDeployer.java:517)
at org.glassfish.weld.WeldDeployer.event(WeldDeployer.java:428)
at org.glassfish.kernel.event.EventsImpl.send(EventsImpl.java:131)
at org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:333)
at com.sun.enterprise.v3.server.ApplicationLifecycle.prepare(ApplicationLifecycle.java:497)
at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:540)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:549)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:545)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:544)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:575)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:566)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1475)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1300(CommandRunnerImpl.java:111)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1857)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1733)
at com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:564)
at com.sun.enterprise.v3.admin.AdminAdapter.onMissingResource(AdminAdapter.java:251)
at org.glassfish.grizzly.http.server.StaticHttpHandlerBase.service(StaticHttpHandlerBase.java:166)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:83)
... 74 more
Caused by: java.lang.NullPointerException
at liquibase.integration.cdi.CDILiquibase.performUpdate(CDILiquibase.java:126)
at liquibase.integration.cdi.CDILiquibase.onStartup(CDILiquibase.java:116)
... 79 more
EDIT: Added stacktrace
EDIT 2: Problem solved! It ended up being something not related with Liquibase at all. The queue size of the PayaraExecutorService was too small, which leads me to think that some tasks submissions were rejected. Apparently this is a known issue of Payara 5.184, and a related issue had already been opened on Github (https://github.com/payara/Payara/issues/3495). Once I increased the size of the queue to a large number everything started working as expected.
I think you need to use lookup to inject the resource.
Compare this bug report: https://github.com/payara/Payara/issues/4413

WSO identity server , service provider creation with permissions does not work

I got following exception while creating new service provider with permissions , following is some portion of its code.
iManagementServiceStub = new IdentityApplicationManagementServiceStub();
iManagementServiceStub.createApplication(createApplication);
Following is exception i am getting on client side.
identity.IdentityApplicationManagementServiceIdentityApplicationManagementException: Error while storing permissions for application sp3
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at identity.IdentityApplicationManagementServiceStub.createApplication(IdentityApplicationManagementServiceStub.java:1007)
at identity.AddServiceProvider.main(AddServiceProvider.java:92)
Following are exception on server side.
Caused by: org.wso2.carbon.registry.core.exceptions.RegistryException: The path '/_system/governance/permission/applications/sp3/org.wso2.carbon.identity.application.common.model.ApplicationPermission#12809798' contains one or more illegal characters (~!##;%^*()+={}|\<>"',)
at org.wso2.carbon.registry.core.jdbc.Repository.put(Repository.java:262)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.put(EmbeddedRegistry.java:717)
at org.wso2.carbon.registry.core.caching.CacheBackedRegistry.put(CacheBackedRegistry.java:591)
at org.wso2.carbon.registry.core.session.UserRegistry.putInternal(UserRegistry.java:828)
at org.wso2.carbon.registry.core.session.UserRegistry.putInternal(UserRegistry.java:796)
at org.wso2.carbon.registry.core.session.UserRegistry.access$900(UserRegistry.java:61)
at org.wso2.carbon.registry.core.session.UserRegistry$10.run(UserRegistry.java:786)
at org.wso2.carbon.registry.core.session.UserRegistry$10.run(UserRegistry.java:783)
at java.security.AccessController.doPrivileged(Native Method)
at org.wso2.carbon.registry.core.session.UserRegistry.put(UserRegistry.java:783)
at org.wso2.carbon.identity.application.mgt.ApplicationMgtUtil.storePermissions(ApplicationMgtUtil.java:299)
... 64 more
Please suggest.
When analyzing the error log, you can see that there are illegal charactors in your permission.
Caused by: org.wso2.carbon.registry.core.exceptions.RegistryException: The path '/_system/governance/permission/applications/sp3/org.wso2.carbon.identity.application.common.model.ApplicationPermission#12809798' contains one or more illegal characters (~!##;%^*()+={}|\<>"',)
Please check the permission name. Basically those charactors are reserved and have an specific use. So you cannot use them elsewhere whithout encoding them[1].
[1] https://en.wikipedia.org/wiki/Percent-encoding

JDBC | SQL Anywhere Error -188: Not enough values for host variables

JDBC or Hibernate is showing strange behavior while calling a stored procedure.
Here is the sample code:
return session.createSQLQuery("select * from pr_ss_coo_10x('31748','NEWRX',null,'5053')")
..
....tolist();
It is throwing error
Caused by: org.hibernate.exception.SQLGrammarException: could not execute query
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:106)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109)
at org.hibernate.loader.Loader.doList(Loader.java:2620)
at org.hibernate.loader.Loader.doList(Loader.java:2600)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2429)
at org.hibernate.loader.Loader.list(Loader.java:2424)
at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:336)
at org.hibernate.internal.SessionImpl.listCustomQuery(SessionImpl.java:1967)
at org.hibernate.internal.AbstractSessionImpl.list(AbstractSessionImpl.java:322)
at org.hibernate.internal.SQLQueryImpl.list(SQLQueryImpl.java:125)
at com.mps.surescript.repository.impl.SSGetInfoRepoImpl.getBenefitCoordInformation(SSGetInfoRepoImpl.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
... 76 common frames omitted
Caused by: com.sybase.jdbc3.jdbc.SybSQLException: SQL Anywhere Error -188: Not enough values for host variables
at com.sybase.jdbc3.tds.Tds.a(Unknown Source)
at com.sybase.jdbc3.tds.Tds.nextResult(Unknown Source)
However, it is working fine when I fire the same query through SQL Anywhere client. Sybase error code document says
"You have not provided enough host variables for either the number of
bind variables, or the statement, or the number of SELECT list items."
I have checked it twice and the result set is correctly mapped with the Pojo. What can be the possible root cause for this problem?
Bypassing the Hibernate transaction Management did the trick and fetching result set using PreparedStatement.
Connection con = DriverManager.getConnection(url, user, password);
PreparedStatement ps = con.prepareStatement(
"select * from pr_ss_coo_10x(#param1=?,#param2=?, #param3=?, #param4=? )");
ResultSet rs = ps.executeQuery();
This is not a solution, it is just a way around.

Hive error : java.io.IOException: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider

I have hive query which run successful sometimes but maximum time gives an error "java.io.IOException: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider"
Below is my error log
java.lang.RuntimeException: java.io.IOException: Couldn't create proxy
provider class org.apache.hadoop.hdfs.server.namenode.ha.Con\
figuredFailoverProxyProvider at
org.apache.hadoop.mapred.lib.CombineFileInputFormat.isSplitable(CombineFileInputFormat.java:154)
at
org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat.getMoreSplits(CombineFileInputFormat.java:283)
at
org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:239)
at
org.apache.hadoop.mapred.lib.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:75)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:336)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:302)
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:435)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:525)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:517)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:399)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295) at
org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:415) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292) at
org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:564) at
org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:559) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:415) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:559)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:550)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
at
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516) at
org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283) at
org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:924) at
org.apache.hadoop.hive.ql.Driver.run(Driver.java:914) at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
at
org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
at
org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694) at
org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by:
java.io.IOException: Couldn't create proxy provider class
org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverPr\
oxyProvider at
org.apache.hadoop.hdfs.NameNodeProxies.createFailoverProxyProvider(NameNodeProxies.java:475)
at
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:148)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:632) at
org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:570) at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:147)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169) at
org.apache.hadoop.mapred.lib.CombineFileInputFormat.isSplitable(CombineFileInputFormat.java:151)
... 45 more Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedConstructorAccessor32.newInstance(Unknown
Source) at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hdfs.NameNodeProxies.createFailoverProxyProvider(NameNodeProxies.java:458)
... 53 more Caused by: java.lang.OutOfMemoryError: GC overhead limit
exceeded at java.util.Arrays.copyOf(Arrays.java:2219) at
java.util.ArrayList.grow(ArrayList.java:242) at
java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:216) at
java.util.ArrayList.ensureCapacityInternal(ArrayList.java:208) at
java.util.ArrayList.add(ArrayList.java:440) at
java.lang.String.split(String.java:2288) at
sun.net.util.IPAddressUtil.textToNumericFormatV4(IPAddressUtil.java:47)
at java.net.InetAddress.getAllByName(InetAddress.java:1129) at
java.net.InetAddress.getAllByName(InetAddress.java:1098) at
java.net.InetAddress.getByName(InetAddress.java:1048) at
org.apache.hadoop.security.SecurityUtil$StandardHostResolver.getByName(SecurityUtil.java:474)
at
org.apache.hadoop.security.SecurityUtil.getByName(SecurityUtil.java:461)
at
org.apache.hadoop.net.NetUtils.createSocketAddrForHost(NetUtils.java:235)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:215)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:163)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:152)
at
org.apache.hadoop.hdfs.DFSUtil.getAddressesForNameserviceId(DFSUtil.java:677)
at
org.apache.hadoop.hdfs.DFSUtil.getAddressesForNsIds(DFSUtil.java:645)
at org.apache.hadoop.hdfs.DFSUtil.getAddresses(DFSUtil.java:628) at
org.apache.hadoop.hdfs.DFSUtil.getHaNnRpcAddresses(DFSUtil.java:727)
at
org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider.(ConfiguredFailoverProxyProvider.java:88)
at sun.reflect.GeneratedConstructorAccessor32.newInstance(Unknown
Source) at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hdfs.NameNodeProxies.createFailoverProxyProvider(NameNodeProxies.java:458)
at
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:148)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:632) at
org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:570) at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:147)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169) Job
Submission failed with exception
'java.lang.RuntimeException(java.io.IOException: Couldn't create proxy
provider class org.apac\
he.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider)'
Could anyone tell me why this happen?
I just stumbled across a similar exception myself, and increasing the hive client heap didn't help. I found I was able to clear up the OutOfMemory GC Overhead exception by adding a partition column to the where clause of the query, so I've concluded that having a very large number of splits is causing this exception. I haven't dug into the code, but I believe I've seen this happen with string concatenation in a loop triggering gc thrashing, and something similar might be happening in the CombineHiveInputFormat.getSplits method.

IWAB0014E Unexpected exception occurred

I am trying to create a simple soap web service application using eclipse and I encountered this error.
I am using Tomcat v7.0 server and eclipse Kepler. Can anyone please help me out with this exception ?
IWAB0014E Unexpected exception occurred.
java.lang.NullPointerException
at org.eclipse.jst.ws.internal.creation.ui.extension.PreServiceDevelopCommand.execute(PreServiceDevelopCommand.java:143)
at org.eclipse.wst.command.internal.env.core.fragment.CommandFragmentEngine.runCommand(CommandFragmentEngine.java:419)
at org.eclipse.wst.command.internal.env.core.fragment.CommandFragmentEngine.visitTop(CommandFragmentEngine.java:359)
at org.eclipse.wst.command.internal.env.core.fragment.CommandFragmentEngine.moveForwardToNextStop(CommandFragmentEngine.java:254)
at org.eclipse.wst.command.internal.env.ui.widgets.SimpleCommandEngineManager$6.run(SimpleCommandEngineManager.java:294)
at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:464)
at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:372)
at org.eclipse.jface.wizard.WizardDialog.run(WizardDialog.java:1028)
at org.eclipse.wst.command.internal.env.ui.widgets.SimpleCommandEngineManager.runForwardToNextStop(SimpleCommandEngineManager.java:264)
at org.eclipse.wst.command.internal.env.ui.widgets.WizardPageManager.runForwardToNextStop(WizardPageManager.java:91)
at org.eclipse.wst.command.internal.env.ui.widgets.WizardPageManager.getNextPage(WizardPageManager.java:154)
at org.eclipse.wst.command.internal.env.ui.widgets.SimpleWizardPage.getNextPage(SimpleWizardPage.java:136)
at org.eclipse.jface.wizard.WizardDialog.nextPressed(WizardDialog.java:908)
at org.eclipse.jface.wizard.WizardDialog.buttonPressed(WizardDialog.java:428)
at org.eclipse.jface.dialogs.Dialog$2.widgetSelected(Dialog.java:628)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1057)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4170)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3759)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:826)
at org.eclipse.jface.window.Window.open(Window.java:802)
at org.eclipse.ui.internal.handlers.WizardHandler$New.executeHandler(WizardHandler.java:259)
at org.eclipse.ui.internal.handlers.WizardHandler.execute(WizardHandler.java:279)
at org.eclipse.ui.internal.handlers.HandlerProxy.execute(HandlerProxy.java:290)
at org.eclipse.ui.internal.handlers.E4HandlerProxy.execute(E4HandlerProxy.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:56)
at org.eclipse.e4.core.internal.di.InjectorImpl.invokeUsingClass(InjectorImpl.java:243)
at org.eclipse.e4.core.internal.di.InjectorImpl.invoke(InjectorImpl.java:224)
at org.eclipse.e4.core.contexts.ContextInjectionFactory.invoke(ContextInjectionFactory.java:132)
at org.eclipse.e4.core.commands.internal.HandlerServiceHandler.execute(HandlerServiceHandler.java:167)
at org.eclipse.core.commands.Command.executeWithChecks(Command.java:499)
at org.eclipse.core.commands.ParameterizedCommand.executeWithChecks(ParameterizedCommand.java:508)
at org.eclipse.e4.core.commands.internal.HandlerServiceImpl.executeHandler(HandlerServiceImpl.java:213)
at org.eclipse.ui.internal.handlers.LegacyHandlerService.executeCommand(LegacyHandlerService.java:420)
at org.eclipse.ui.internal.actions.CommandAction.runWithEvent(CommandAction.java:157)
at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:584)
at org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:501)
at org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:411)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1057)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4170)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3759)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1113)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:997)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:140)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:611)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:567)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:124)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:354)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:181)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:636)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:591)
at org.eclipse.equinox.launcher.Main.run(Main.java:1450)
This is happening because of improper setup of Axis2.
Steps:
Download Axis2 : Link--> http://ws.apache.org/axis2/download.cgi
Point the Axis2 runtime dir in eclipse : Menu--> Windows--> Preference --> Web Services--> Axis2 Preferences
Your code throws a NullPointerException at
PreServiceDevelopCommand.java:143
NullPointerException means that you wanted to use a reference which was not initialized. Most common scenario is that you want to use a member or call a method of an object which was not initialized using the new operator or a factory method. Check the file where the error occurs at line 143 and see what could be null there.
Official description from the docs:
Thrown when an application attempts to use null in a case where an
object is required. These include:
- Calling the instance method of a null object.
- Accessing or modifying the field of a null object.
- Taking the length of null as if it were an array.
- Accessing or modifying the slots of null as if it were an array.
- Throwing null as if it were a Throwable value.
Just try to clean the pojo class for which you are trying to generate web service.
Clean the class and compile again.
It worked for me.
Deleting the client which you have already created will solve the problem.