I am unable to test Hive sink with Flume on a kerberized cluster, see below exception. Also, I noticed there is no option in Flume user guide to provide a keytab/principal similar to HDFS and HBase sinks. How do we use Hive sink on a kerberized cluster?
2022-07-21T07:39:29,035 ERROR [hive-sink1-call-runner-0] org.apache.thrift.transport.TSaslTransport - SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) ~[?:1.8.0_331]
at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:96) ~[libthrift-0.14.1.jar:0.14.1]
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:236) [libthrift-0.14.1.jar:0.14.1]
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:39) [libthrift-0.14.1.jar:0.14.1]
at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:51) [hive-exec-3.1.2.jar:3.1.2]
at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:48) [hive-exec-3.1.2.jar:3.1.2]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_331]
at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_331]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) [hadoop-common-3.1.2.odh.1.0.dfc6dd56066.jar:?]
at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport.open(TUGIAssumingTransport.java:48) [hive-exec-3.1.2.jar:3.1.2]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:531) [hive-exec-3.1.2.jar:3.1.2]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) [hive-exec-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:409) [hive-hcatalog-core-3.1.2.jar:3.1.2]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_331]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [?:1.8.0_331]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [?:1.8.0_331]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [?:1.8.0_331]
at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) [hive-exec-3.1.2.jar:3.1.2]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:95) [hive-exec-3.1.2.jar:3.1.2]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148) [hive-exec-3.1.2.jar:3.1.2]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) [hive-exec-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:297) [hive-hcatalog-core-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:292) [hive-hcatalog-core-3.1.2.jar:3.1.2]
at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767) [guava-11.0.2.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) [guava-11.0.2.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) [guava-11.0.2.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) [guava-11.0.2.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) [guava-11.0.2.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3965) [guava-11.0.2.jar:?]
at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764) [guava-11.0.2.jar:?]
at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:292) [hive-hcatalog-core-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:267) [hive-hcatalog-core-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:569) [hive-hcatalog-core-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.getMetaStoreClient(HiveEndPoint.java:529) [hive-hcatalog-streaming-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:327) [hive-hcatalog-streaming-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:294) [hive-hcatalog-streaming-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnectionImpl(HiveEndPoint.java:229) [hive-hcatalog-streaming-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.java:206) [hive-hcatalog-streaming-3.1.2.jar:3.1.2]
at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.java:117) [hive-hcatalog-streaming-3.1.2.jar:3.1.2]
at org.apache.flume.sink.hive.HiveWriter$8.call(HiveWriter.java:379) [flume-hive-sink-1.10.0.jar:1.10.0]
at org.apache.flume.sink.hive.HiveWriter$8.call(HiveWriter.java:376) [flume-hive-sink-1.10.0.jar:1.10.0]
at org.apache.flume.sink.hive.HiveWriter$11.call(HiveWriter.java:428) [flume-hive-sink-1.10.0.jar:1.10.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_331]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_331]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_331]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_331]
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:772) ~[?:1.8.0_331]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) ~[?:1.8.0_331]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.8.0_331]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ~[?:1.8.0_331]
... 45 more
Related
As the title, I cannot add the Glassfish server to NetBeans IDE 8.2. I pressed the "Add Server..." menu entry, then on the "Choose Server" screen I selected "GlassFish Server". Then, "Not a valid GlassFish Server installation" is shown. I can click the "next" button but the screen does not move on. Anyone know how to fix it? Thank you.
JDK: jdk1.8.0_321
Netbeans IDE 8.2
Here is a screen shot:
About in Netbeans
Path in System Variable
Start GlassFish 5.0 in Command Prompt
WARNING [org.netbeans.modules.options.keymap.LayersBridge]: Invalid shortcut: org.openide.loaders.BrokenDataShadow#3c62f87e[MultiFileObject#d4180b7[Keymaps/NetBeans/D-BACK_QUOTE.shadow]]
WARNING [null]: Last record repeated again.
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Resolving dependencies took: 143 ms
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 0 binary roots took: 1 ms
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: C:\Program Files\NetBeans 8.2\webcommon\jsstubs\domstubs.zip took: 468 ms (New or modified files: 0, Deleted files: 0) [Adding listeners took: 0 ms]
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: C:\Program Files\NetBeans 8.2\webcommon\jsstubs\corestubs.zip took: 71 ms (New or modified files: 0, Deleted files: 0) [Adding listeners took: 0 ms]
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: C:\Program Files\NetBeans 8.2\webcommon\jsstubs\reststubs.zip took: 101 ms (New or modified files: 0, Deleted files: 0) [Adding listeners took: 0 ms]
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 3 source roots took: 640 ms (New or modified files: 0, Deleted files: 0) [Adding listeners took: 0 ms]
SEVERE [global]
java.lang.UnsupportedClassVersionError: com/sun/appserv/server/util/Version has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at org.netbeans.modules.glassfish.tooling.utils.ServerUtils.getServerVersion(ServerUtils.java:589)
at org.netbeans.modules.glassfish.common.ServerDetails.getVersionFromInstallDirectory(ServerDetails.java:222)
at org.netbeans.modules.glassfish.common.ServerDetails.isInstalledInDirectory(ServerDetails.java:305)
at org.netbeans.modules.glassfish.common.wizards.ServerWizardIterator.isValidInstall(ServerWizardIterator.java:373)
at org.netbeans.modules.glassfish.common.wizards.AddServerLocationPanel.isValid(AddServerLocationPanel.java:189)
at org.openide.WizardDescriptor.updateStateOpen(WizardDescriptor.java:874)
at org.openide.WizardDescriptor.updateState(WizardDescriptor.java:822)
at org.netbeans.modules.server.ui.wizard.AddServerInstanceWizard.updateState(AddServerInstanceWizard.java:238)
at org.openide.WizardDescriptor._updateState(WizardDescriptor.java:800)
at org.openide.WizardDescriptor.goToNextStep(WizardDescriptor.java:1072)
at org.openide.WizardDescriptor.access$1900(WizardDescriptor.java:92)
at org.openide.WizardDescriptor$Listener$1.run(WizardDescriptor.java:2208)
at org.openide.WizardDescriptor.lazyValidate(WizardDescriptor.java:1516)
at org.openide.WizardDescriptor.access$2000(WizardDescriptor.java:92)
at org.openide.WizardDescriptor$Listener.actionPerformed(WizardDescriptor.java:2227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.openide.util.WeakListenerImpl$ProxyListener.invoke(WeakListenerImpl.java:487)
at com.sun.proxy.$Proxy24.actionPerformed(Unknown Source)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:262)
at java.awt.Component.processMouseEvent(Component.java:6539)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6304)
at java.awt.Container.processEvent(Container.java:2239)
at java.awt.Component.dispatchEventImpl(Component.java:4889)
at java.awt.Container.dispatchEventImpl(Container.java:2297)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
at java.awt.Container.dispatchEventImpl(Container.java:2283)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159)
[catch] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:109)
at java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:190)
at java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:235)
at java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:233)
at java.security.AccessController.doPrivileged(Native Method)
at java.awt.WaitDispatchSupport.enter(WaitDispatchSupport.java:233)
at java.awt.Dialog.show(Dialog.java:1084)
at org.netbeans.core.windows.services.NbPresenter.superShow(NbPresenter.java:1086)
at org.netbeans.core.windows.services.NbPresenter.doShow(NbPresenter.java:1136)
at org.netbeans.core.windows.services.NbPresenter.run(NbPresenter.java:1108)
at org.netbeans.core.windows.services.NbPresenter.run(NbPresenter.java:131)
at org.netbeans.modules.openide.util.NbMutexEventProvider$Event.doEventAccess(NbMutexEventProvider.java:138)
at org.netbeans.modules.openide.util.NbMutexEventProvider$Event.readAccess(NbMutexEventProvider.java:98)
at org.netbeans.modules.openide.util.LazyMutexImplementation.readAccess(LazyMutexImplementation.java:94)
at org.openide.util.Mutex.readAccess(Mutex.java:218)
at org.netbeans.core.windows.services.NbPresenter.show(NbPresenter.java:1093)
at java.awt.Component.show(Component.java:1671)
at java.awt.Component.setVisible(Component.java:1623)
at java.awt.Window.setVisible(Window.java:1014)
at java.awt.Dialog.setVisible(Dialog.java:1005)
at org.netbeans.modules.server.ui.wizard.AddServerInstanceWizard.showAddServerInstanceWizard(AddServerInstanceWizard.java:210)
at org.netbeans.modules.server.ui.wizard.AddServerInstanceWizard.showAddServerInstanceWizard(AddServerInstanceWizard.java:130)
at org.netbeans.modules.server.ui.node.AddServerInstanceAction.actionPerformed(AddServerInstanceAction.java:59)
at org.openide.awt.AlwaysEnabledAction$1.run(AlwaysEnabledAction.java:199)
at org.openide.util.actions.ActionInvoker$1.run(ActionInvoker.java:95)
at org.openide.util.actions.ActionInvoker.doPerformAction(ActionInvoker.java:116)
at org.openide.util.actions.ActionInvoker.invokeAction(ActionInvoker.java:99)
at org.openide.awt.AlwaysEnabledAction.actionPerformed(AlwaysEnabledAction.java:202)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.AbstractButton.doClick(AbstractButton.java:376)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:882)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:926)
at java.awt.Component.processMouseEvent(Component.java:6539)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6304)
at java.awt.Container.processEvent(Container.java:2239)
at java.awt.Component.dispatchEventImpl(Component.java:4889)
at java.awt.Container.dispatchEventImpl(Container.java:2297)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
at java.awt.Container.dispatchEventImpl(Container.java:2283)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
SEVERE [global]
java.lang.NullPointerException
at java.io.File.<init>(File.java:277)
at org.netbeans.modules.glassfish.common.wizards.AddDomainLocationVisualPanel.initModels(AddDomainLocationVisualPanel.java:193)
at org.netbeans.modules.glassfish.common.wizards.AddDomainLocationPanel.readSettings(AddDomainLocationPanel.java:185)
at org.openide.WizardDescriptor.updateStateOpen(WizardDescriptor.java:867)
at org.openide.WizardDescriptor.updateState(WizardDescriptor.java:822)
at org.netbeans.modules.server.ui.wizard.AddServerInstanceWizard.updateState(AddServerInstanceWizard.java:238)
at org.openide.WizardDescriptor._updateState(WizardDescriptor.java:800)
at org.openide.WizardDescriptor.goToNextStep(WizardDescriptor.java:1072)
at org.openide.WizardDescriptor.access$1900(WizardDescriptor.java:92)
at org.openide.WizardDescriptor$Listener$1.run(WizardDescriptor.java:2208)
at org.openide.WizardDescriptor.lazyValidate(WizardDescriptor.java:1516)
at org.openide.WizardDescriptor.access$2000(WizardDescriptor.java:92)
at org.openide.WizardDescriptor$Listener.actionPerformed(WizardDescriptor.java:2227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.openide.util.WeakListenerImpl$ProxyListener.invoke(WeakListenerImpl.java:487)
at com.sun.proxy.$Proxy24.actionPerformed(Unknown Source)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:262)
at java.awt.Component.processMouseEvent(Component.java:6539)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6304)
at java.awt.Container.processEvent(Container.java:2239)
at java.awt.Component.dispatchEventImpl(Component.java:4889)
at java.awt.Container.dispatchEventImpl(Container.java:2297)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
at java.awt.Container.dispatchEventImpl(Container.java:2283)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159)
[catch] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:109)
at java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:190)
at java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:235)
at java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:233)
at java.security.AccessController.doPrivileged(Native Method)
at java.awt.WaitDispatchSupport.enter(WaitDispatchSupport.java:233)
at java.awt.Dialog.show(Dialog.java:1084)
at org.netbeans.core.windows.services.NbPresenter.superShow(NbPresenter.java:1086)
at org.netbeans.core.windows.services.NbPresenter.doShow(NbPresenter.java:1136)
at org.netbeans.core.windows.services.NbPresenter.run(NbPresenter.java:1108)
at org.netbeans.core.windows.services.NbPresenter.run(NbPresenter.java:131)
at org.netbeans.modules.openide.util.NbMutexEventProvider$Event.doEventAccess(NbMutexEventProvider.java:138)
at org.netbeans.modules.openide.util.NbMutexEventProvider$Event.readAccess(NbMutexEventProvider.java:98)
at org.netbeans.modules.openide.util.LazyMutexImplementation.readAccess(LazyMutexImplementation.java:94)
at org.openide.util.Mutex.readAccess(Mutex.java:218)
at org.netbeans.core.windows.services.NbPresenter.show(NbPresenter.java:1093)
at java.awt.Component.show(Component.java:1671)
at java.awt.Component.setVisible(Component.java:1623)
at java.awt.Window.setVisible(Window.java:1014)
at java.awt.Dialog.setVisible(Dialog.java:1005)
at org.netbeans.modules.server.ui.wizard.AddServerInstanceWizard.showAddServerInstanceWizard(AddServerInstanceWizard.java:210)
at org.netbeans.modules.server.ui.wizard.AddServerInstanceWizard.showAddServerInstanceWizard(AddServerInstanceWizard.java:130)
at org.netbeans.modules.server.ui.node.AddServerInstanceAction.actionPerformed(AddServerInstanceAction.java:59)
at org.openide.awt.AlwaysEnabledAction$1.run(AlwaysEnabledAction.java:199)
at org.openide.util.actions.ActionInvoker$1.run(ActionInvoker.java:95)
at org.openide.util.actions.ActionInvoker.doPerformAction(ActionInvoker.java:116)
at org.openide.util.actions.ActionInvoker.invokeAction(ActionInvoker.java:99)
at org.openide.awt.AlwaysEnabledAction.actionPerformed(AlwaysEnabledAction.java:202)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.AbstractButton.doClick(AbstractButton.java:376)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:882)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:926)
at java.awt.Component.processMouseEvent(Component.java:6539)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6304)
at java.awt.Container.processEvent(Container.java:2239)
at java.awt.Component.dispatchEventImpl(Component.java:4889)
at java.awt.Container.dispatchEventImpl(Container.java:2297)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
at java.awt.Container.dispatchEventImpl(Container.java:2283)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
NetBeans 8.2 is outdated and not supposed to work with current Glassfish versions.
Please get a recent NetBeans version: https://netbeans.apache.org/download/index.html
I am trying to connect to the OIM server using oimclient.jar. When I try to login using below code. I'm getting error: An error occurred while trying to notify server that a new login session has been created.
code:
configEnv.put(OIMClient.JAVA_NAMING_FACTORY_INITIAL,
OIMClientConnectorConstants.JAVA_NAMING_FACTORY_INITIAL);
configEnv.put(OIMClient.JAVA_NAMING_PROVIDER_URL,
OIMClientConnectorConstants.JAVA_NAMING_PROVIDER_URL);
oimClient = new OIMClient(configEnv);
List response = oimClient.login(
OIMClientConnectorConstants.OIM_USERNAME,
OIMClientConnectorConstants.OIM_PASSWORD);
connector.setOimClient(oimClient);
Error:
Apr 18, 2014 10:03:30 AM oracle.iam.platform.OIMClient loginSessionCreated
SEVERE: An error occurred while trying to notify server that a new login session has been created.
oracle.iam.platform.utils.NoSuchServiceException: java.lang.reflect.InvocationTargetException
at oracle.iam.platform.OIMClient.getServiceDelegate(OIMClient.java:278)
at oracle.iam.platform.OIMClient.getService(OIMClient.java:255)
at oracle.iam.platform.OIMClient.loginSessionCreated(OIMClient.java:290)
at oracle.iam.platform.OIMClient.login(OIMClient.java:214)
at oracle.iam.platform.OIMClient.login(OIMClient.java:183)
at com.cred.service.oim.OIMClientConnector.getInstance(OIMClientConnector.java:92)
at com.cred.data.UserProfileDataService.fecthCompleteUserProfile(UserProfileDataService.java:28)
at com.cred.rest.UserProfileRestService.fecthCompleteUserProfile(UserProfileRestService.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:184)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3732)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3696)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2273)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2179)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1490)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at oracle.iam.platform.OIMClient.getServiceDelegate(OIMClient.java:274)
... 40 more
Caused by: oracle.iam.platform.utils.NoSuchServiceException: javax.naming.AuthenticationException [Root exception is java.lang.SecurityException: User: weblogic, failed to be authenticated.]
at oracle.iam.platformservice.api.ClientLoginSessionServiceDelegate.<init>(Unknown Source)
... 45 more
Caused by: javax.naming.AuthenticationException [Root exception is java.lang.SecurityException: User: weblogic, failed to be authenticated.]
at weblogic.jndi.internal.ExceptionTranslator.toNamingException(ExceptionTranslator.java:42)
at weblogic.jndi.WLInitialContextFactoryDelegate.toNamingException(WLInitialContextFactoryDelegate.java:792)
at weblogic.jndi.WLInitialContextFactoryDelegate.pushSubject(WLInitialContextFactoryDelegate.java:686)
at weblogic.jndi.WLInitialContextFactoryDelegate.newContext(WLInitialContextFactoryDelegate.java:471)
at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:376)
at weblogic.jndi.Environment.getContext(Environment.java:315)
at weblogic.jndi.Environment.getContext(Environment.java:285)
at weblogic.jndi.WLInitialContextFactory.getInitialContext(WLInitialContextFactory.java:117)
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
at javax.naming.InitialContext.init(InitialContext.java:242)
at javax.naming.InitialContext.<init>(InitialContext.java:216)
at org.springframework.jndi.JndiTemplate.createInitialContext(JndiTemplate.java:137)
at org.springframework.jndi.JndiTemplate.getContext(JndiTemplate.java:104)
at org.springframework.jndi.JndiTemplate.execute(JndiTemplate.java:86)
at org.springframework.jndi.JndiTemplate.lookup(JndiTemplate.java:153)
at org.springframework.jndi.JndiTemplate.lookup(JndiTemplate.java:178)
... 46 more
Caused by: java.lang.SecurityException: User: weblogic, failed to be authenticated.
at weblogic.common.internal.RMIBootServiceImpl.authenticate(RMIBootServiceImpl.java:116)
at weblogic.common.internal.RMIBootServiceImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:667)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:522)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:518)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
... 2 more
When I execute a spark process in oozie I have the following error. The database not found.
2018-09-26 15:27:23,576 INFO [main] org.apache.spark.deploy.yarn.Client:
client token: Token { kind: YARN_CLIENT_TOKEN, service: }
diagnostics: User class threw exception: org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'zdm_datos_externos' not found;
ApplicationMaster host: 10.74.234.6
ApplicationMaster RPC port: 0
queue: default
queue user: administrador
start time: 1537986410475
final status: FAILED
tracking URL: https://BR-PC-CentOS-02:26001/proxy/application_1537467570666_4127/
user: administrador
This is my spark config
String warehouseLocation = new File("spark-warehouse").getAbsolutePath();
SparkSession spark = SparkSession
.builder()
.appName("Java Spark Hive Example")
.master("yarn")
.config("spark.sql.warehouse.dir", warehouseLocation)
.config("spark.driver.maxResultSize", "3g")
.config("spark.debug.maxToStringFields", "10000")
.config("spark.sql.crossJoin.enabled", "true")
.enableHiveSupport()
.getOrCreate();
spark.conf().set("spark.driver.maxResultSize", "3g");
metastore, current connections: 1 2018-09-26 17:31:42,598 WARN [main]
hive.metastore: set_ugi() not successful, Likely cause: new client
talking to old server. Continuing without it.
org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3748)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3734)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:557)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:249)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1533)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3157)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3176)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3409)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:178)
at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.security.HiveCredentialProvider$$anonfun$obtainCredentials$1.apply$mcV$sp(HiveCredentialProvider.scala:91)
at org.apache.spark.deploy.yarn.security.HiveCredentialProvider$$anonfun$obtainCredentials$1.apply(HiveCredentialProvider.scala:90)
at org.apache.spark.deploy.yarn.security.HiveCredentialProvider$$anonfun$obtainCredentials$1.apply(HiveCredentialProvider.scala:90)
at org.apache.spark.deploy.yarn.security.HiveCredentialProvider$$anon$1.run(HiveCredentialProvider.scala:124)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1778)
at org.apache.spark.deploy.yarn.security.HiveCredentialProvider.doAsRealUser(HiveCredentialProvider.scala:123)
at org.apache.spark.deploy.yarn.security.HiveCredentialProvider.obtainCredentials(HiveCredentialProvider.scala:90)
at org.apache.spark.deploy.yarn.security.ConfigurableCredentialManager$$anonfun$obtainCredentials$2.apply(ConfigurableCredentialManager.scala:82)
at org.apache.spark.deploy.yarn.security.ConfigurableCredentialManager$$anonfun$obtainCredentials$2.apply(ConfigurableCredentialManager.scala:80)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.MapLike$DefaultValuesIterable.foreach(MapLike.scala:206)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.spark.deploy.yarn.security.ConfigurableCredentialManager.obtainCredentials(ConfigurableCredentialManager.scala:80)
at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:430)
at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:915)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:195)
at org.apache.spark.deploy.yarn.Client.run(Client.scala:1205)
at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1261)
at org.apache.spark.deploy.yarn.Client.main(Client.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:761)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:190)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:215)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:129)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
at org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:113)
at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:104)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:238)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:187)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
What version of Spark are you using? Did you enable Hive support on your sparkSession?
sparkBuilder.enableHiveSupport().appName(appName).getOrCreate()
We have IBM MFP 8.0.0.00-20171220-1341 version available on google cloud.
We are using authentication adapter to do login.
While doing login we are getting java.net.SocketTimeoutException: timeout exception.
Please check below logs.
com.mfpauth.impl.Authentication E java.net.SocketTimeoutException: timeout
at okio.Okio$4.newTimeoutException(Okio.java:230)
at okio.AsyncTimeout.exit(AsyncTimeout.java:285)
at okio.AsyncTimeout$2.read(AsyncTimeout.java:241)
at okio.RealBufferedSource.indexOf(RealBufferedSource.java:345)
at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:217)
at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:211)
at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:189)
at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:75)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
at okhttp3.RealCall.execute(RealCall.java:69)
at com.mfpauth.impl.Authentication.callAdapterWebService(Authentication.java:118)
at com.mfpauth.impl.Authentication.getDealerProfileAttributes(Authentication.java:551)
at com.mfpauth.impl.Authentication.validateCredentials(Authentication.java:229)
at com.ibm.mfp.security.checks.base.CredentialsValidationSecurityCheck.authorize(CredentialsValidationSecurityCheck.java:59)
at com.ibm.mfp.security.checks.base.UserAuthenticationSecurityCheck.authorize(UserAuthenticationSecurityCheck.java:68)
at com.mfpauth.impl.Authentication.authorize(Authentication.java:99)
at com.ibm.mfp.server.security.internal.context.ClientSecurityContextImpl$1.visit(ClientSecurityContextImpl.java:380)
at com.ibm.mfp.server.security.internal.context.ClientSecurityContextImpl.iterateThroughSecurityChecks(ClientSecurityContextImpl.java:461)
at com.ibm.mfp.server.security.internal.context.ClientSecurityContextImpl.processAuthorizationRequest(ClientSecurityContextImpl.java:370)
at sun.reflect.GeneratedMethodAccessor360.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy246.processAuthorizationRequest(Unknown Source)
at com.ibm.mfp.server.security.internal.services.PreAuthorizationServiceImpl.authorize(PreAuthorizationServiceImpl.java:36)
at com.ibm.mfp.server.security.internal.rest.PreAuthorizationEndpoint$1.call(PreAuthorizationEndpoint.java:82)
at com.ibm.mfp.server.security.internal.rest.PreAuthorizationEndpoint$1.call(PreAuthorizationEndpoint.java:79)
at com.ibm.mfp.server.persistency.internal.transaction.StorageManagerImpl.doWithStorage(StorageManagerImpl.java:49)
at com.ibm.mfp.server.security.internal.rest.PreAuthorizationEndpoint.authorize(PreAuthorizationEndpoint.java:78)
at sun.reflect.GeneratedMethodAccessor359.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:776)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:705)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1290)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:778)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:475)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1161)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:82)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:956)
at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:280)
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:967)
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:359)
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:318)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:471)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:405)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:285)
at com.ibm.ws.http.channel.internal.inbound.HttpICLReadCallback.complete(HttpICLReadCallback.java:66)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:504)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:574)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:929)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1018)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:811)
Caused by: java.net.SocketException: Socket closed
at java.net.SocketInputStream.read(SocketInputStream.java:215)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at okio.Okio$2.read(Okio.java:139)
at okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
... 81 more
What is blocking as we are getting this intermittent issue. Is this issue happening due to google cloud.
This may not have anything to do with Google Cloud. Within your securitycheck, you seem to be invoking a backend at:
at com.mfpauth.impl.Authentication.callAdapterWebService(Authentication.java:118)
For this you are using okhttp library.
The default timeout of okhttp3 library is 10 seconds. You may want to increase the timeout value and check if that resolves the issue.
For more details, refer to:
Timeouts
okhttp3 timeout
We have MFP on Development(VM) and Production (Google Cloud) environment.
We are calling adapter which contains parameter which is large json object more than 1 MB.
While calling adapter mashup javascript to javascript on DEV env its working fine while on Google Could MFP its giving following error.
java.lang.RuntimeException: java.net.SocketException: Broken pipe (Write failed)
[6/15/18 5:52:51:878 UTC] 00003978 com.ibm.mfp.server.js.adapter.internal.JavascriptManagerImpl E FWLST0904E: Exception was thrown while invoking procedure: setPlansData in adapter: RESTAdapter
java.lang.RuntimeException: java.net.SocketException: Broken pipe (Write failed)
at com.ibm.mfp.server.js.adapter.internal.invocation.JavaScriptIntegrationLibraryImplementation.invokeProcedure(JavaScriptIntegrationLibraryImplementation.java:255)
at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:225)
at org.mozilla.javascript.optimizer.OptRuntime.callN(OptRuntime.java:52)
at org.mozilla.javascript.gen.integration_js_1._c_anonymous_13(integration.js:590)
at org.mozilla.javascript.gen.integration_js_1.call(integration.js)
at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
at org.mozilla.javascript.gen.RESTAdapter_impl_js_20._c_setPlansData_162(RESTAdapter-impl.js:9497)
at org.mozilla.javascript.gen.RESTAdapter_impl_js_20.call(RESTAdapter-impl.js)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:393)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3282)
at org.mozilla.javascript.gen.RESTAdapter_impl_js_20.call(RESTAdapter-impl.js)
at com.ibm.mfp.server.js.adapter.internal.JavascriptManagerImpl.invokeFunction(JavascriptManagerImpl.java:280)
at com.ibm.mfp.server.js.adapter.internal.JavascriptManagerImpl.invokeProcedure(JavascriptManagerImpl.java:187)
at com.ibm.mfp.server.js.adapter.internal.invocation.JavascriptInvoker.invoke(JavascriptInvoker.java:57)
at com.ibm.mfp.gen.JavascriptAdapterRestWrapperResource.setPlansDataPost(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:181)
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:97)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:200)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:99)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:251)
at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)
at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:171)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:293)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:212)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:268)
at com.ibm.mfp.server.java.adapter.shared.JAXRSSandbox$3.doFilter(JAXRSSandbox.java:579)
at com.ibm.mfp.server.java.adapter.shared.FilterChainImpl.doFilter(FilterChainImpl.java:86)
at com.ibm.mfp.server.java.adapter.shared.JAXRSSandbox.handleRequest(JAXRSSandbox.java:584)
at com.ibm.mfp.server.java.adapter.internal.rest.AdaptersEndpoint.adapterServing(AdaptersEndpoint.java:115)
at sun.reflect.GeneratedMethodAccessor327.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:776)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:705)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1290)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:778)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:475)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1161)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:82)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:956)
at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:280)
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:967)
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:359)
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:318)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:471)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:405)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:285)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:256)
at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:174)
at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:83)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:504)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:574)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:929)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1018)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:811)
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:122)
at java.net.SocketOutputStream.write(SocketOutputStream.java:166)
at org.apache.http.impl.conn.LoggingOutputStream.write(LoggingOutputStream.java:77)
at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:123)
at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:135)
at org.apache.http.impl.io.SessionOutputBufferImpl.writeLine(SessionOutputBufferImpl.java:239)
at org.apache.http.impl.io.DefaultHttpRequestWriter.writeHeadLine(DefaultHttpRequestWriter.java:66)
at org.apache.http.impl.io.DefaultHttpRequestWriter.writeHeadLine(DefaultHttpRequestWriter.java:42)
at org.apache.http.impl.io.AbstractMessageWriter.write(AbstractMessageWriter.java:109)
at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestHeader(DefaultBHttpClientConnection.java:141)
at org.apache.http.impl.conn.CPoolProxy.sendRequestHeader(CPoolProxy.java:145)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:203)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:121)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at com.ibm.mfp.server.java.adapter.internal.beans.api.AdaptersApiImpl.internalExecuteAdapterRequest(AdaptersApiImpl.java:114)
at com.ibm.mfp.server.java.adapter.internal.beans.api.AdaptersApiImpl.executeAdapterRequest(AdaptersApiImpl.java:95)
at com.ibm.mfp.server.java.adapter.internal.beans.api.AdaptersApiImpl.executeAdapterRequest(AdaptersApiImpl.java:46)
at com.ibm.mfp.server.js.adapter.internal.invocation.JavaScriptIntegrationLibraryImplementation.invokeProcedure(JavaScriptIntegrationLibraryImplementation.java:250)
... 82 more
Not getting why its not working on GCP.
We are calling adapter mashup (Javascript to Javascript)
var inputData = {
"custId": "dataPlans",
"PLAN_DATA": FetchBcdpPlansResponse,
"searchKey": "custId",
"expire": REDIS_BCD_PLAN_EXP_TIME_SEC
}
var serviceResponse = MFP.Server.invokeProcedure({
adapter: 'ServiceWrapperAdapter',
procedure: 'storeInRedis',
parameters: [inputData]
});