Write onto RTMP stream using XUGGLER - red5

I am working on a project, I want to write packets on to an RTMP container.
What I have achieved is:
To start an app which listens for RTMP requests using RED5. Lets say rtmp://localhost/myapp
I could open IContainer with IContainerFormat's output format give as ("flv", "rtmp://localhost/myapp/test", null)
Whats going wrong is :
I want to write packets on to this RTMP channel that is opened for me.
Issue is FFMPEG starts giving error 32. A bit of search related to this error showed that this is happens usually because there is a lag between client - server data consumption. But I really was not able to solve this issue. I have searched a lot but everyone is crying about the same issue. The error i get is :
12:57:20.242 [Thread-0] ERROR org.ffmpeg - WriteN, RTMP send error 32 (129 bytes)
12:57:20.246 [Thread-0] ERROR org.ffmpeg - WriteN, RTMP send error 32 (45 bytes)
12:57:20.246 [Thread-0] ERROR org.ffmpeg - WriteN, RTMP send error 9 (42 bytes)
And this is the error i receive on Red5 Server
[INFO] [NioProcessor-1] org.red5.server.stream.ClientBroadcastStream - Provider connect
[INFO] [NioProcessor-1] org.red5.server.stream.ClientBroadcastStream - Stream start
[INFO] [NioProcessor-1] org.red5.server.stream.ClientBroadcastStream - Provider connect
[ERROR] [NioProcessor-1] org.red5.server.net.rtmp.codec.RTMPProtocolDecoder - Error decoding buffer
org.red5.server.net.protocol.ProtocolException: Error during decoding
at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decode(RTMPProtocolDecoder.java:203) [red5.jar:na]
at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decodeBuffer(RTMPProtocolDecoder.java:122) [red5.jar:na]
at org.red5.server.net.rtmp.codec.RTMPMinaProtocolDecoder.decode(RTMPMinaProtocolDecoder.java:61) [red5.jar:na]
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:225) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:793) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:119) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:638) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:598) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:587) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:61) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:969) [mina-core-2.0.0-RC1.jar:na]
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) [mina-core-2.0.0-RC1.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_07]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_07]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_07]
Caused by: java.lang.IllegalArgumentException: null
at java.nio.Buffer.limit(Unknown Source) [na:1.7.0_07]
at org.red5.io.amf.Input.bufferToString(Input.java:275) [red5.jar:na]
at org.red5.io.amf.Input.getString(Input.java:258) [red5.jar:na]
at org.red5.io.amf.Input.readPropertyName(Input.java:565) [red5.jar:na]
at org.red5.io.amf.Input.readKeyValues(Input.java:351) [red5.jar:na]
at org.red5.io.amf.Input.readSimpleObject(Input.java:501) [red5.jar:na]
at org.red5.io.amf.Input.readObject(Input.java:537) [red5.jar:na]
at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decodeStreamMetadata(RTMPProtocolDecoder.java:978) [red5.jar:na]
at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decodeMessage(RTMPProtocolDecoder.java:564) [red5.jar:na]
at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decodePacket(RTMPProtocolDecoder.java:385) [red5.jar:na]
at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decode(RTMPProtocolDecoder.java:190) [red5.jar:na]
... 18 common frames omitted
[WARN] [NioProcessor-1] org.red5.server.net.rtmp.codec.RTMPProtocolDecoder - Closing connection because decoding failed: RTMPMinaConnection from 127.0.0.1 : 44814 to localhost (in: 3569 out 3521 )
[INFO] [NioProcessor-1] org.red5.server.stream.ClientBroadcastStream - Close
[INFO] [NioProcessor-1] org.red5.server.stream.ClientBroadcastStream - Provider disconnect
[INFO] [NioProcessor-1] org.red5.server.stream.ClientBroadcastStream - Provider disconnect
[INFO] [NioProcessor-1] org.red5.server.net.rtmp.RTMPHandler - Connecting to: [Scope#5f524524 Depth = 2, Path = '/default/myapp', Name = 'test']
[INFO] [NioProcessor-1] org.red5.server.net.rtmp.RTMPHandler - Remembering client buffer on stream: 300
[ERROR] [NioProcessor-1] org.red5.server.service.ServiceInvoker - Method FCSubscribe with parameters [teststream] not found in org.red5.core.Application#b9e19da
[WARN] [NioProcessor-1] org.red5.server.stream.PlayEngine - Subscribe to teststream provider failed
I am new to media streaming, so not sure if I am missing something.
Thanks in advance.

I don't think that there is a different chance then debugging the Java Source code of RTMPProtocolDecoder.
But I don't know what Red5 version you are using.
RTMPProtocolDecoder.java:203
=> In the latest version of Red5/Sources your exception is in line 147:
http://code.google.com/p/red5/source/browse/java/server/trunk/src/org/red5/server/net/rtmp/codec/RTMPProtocolDecoder.java#147
And also added with some comments. I guess first thing you would need to do is to update to latest red5. Based on that you can then debug and ask for more help.

Just to complete the answer. Do not rely on Red5 eclipse plugin, it generates code that is not compatible with Red 5 1.0RC2.
Red5 1.0 RC 2 works fine with a configuration change in red5-web.xml. The app starts working and I could start a stream on the container using xuggler.
Thank you #seba.wagner.

Related

Intellij Pull Requests Tool Windows has errors

Intellij version: 2018.3 (183.4284.148)
I am trying to use the feature https://blog.jetbrains.com/idea/2018/10/intellij-idea-2018-3-eap-github-pull-requests-and-more/ in my Enterprise Github.
When I open the Pull requests tool window it loads pull requests however when I click on any one of them I see an error such as below
There is not much in the idea logs except may be for this one (not sure if this is actually relevant)
2018-12-14 15:56:18,582 [1655214] INFO - lij.diagnostic.DebugLogManager - Set DEBUG for the following categories: #org.jetbrains.plugins.github.api.GithubApiRequestExecutor
2018-12-14 15:56:28,710 [1665342] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/api/v3/repos/MY_ORG/MY_REPO get information for repository MY_ORG/MY_REPO : Connecting
2018-12-14 15:56:29,507 [1666139] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/api/v3/repos/MY_ORG/MY_REPO GET : Connected
2018-12-14 15:56:29,509 [1666141] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/api/v3/repos/MY_ORG/MY_REPO GET : Result extracted
2018-12-14 15:56:29,519 [1666151] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/api/v3/search/issues?q=type%3Apr%20repo%3AMY_ORG%2FMY_REPO%20state%3Aopen search issues in repository : Connecting
2018-12-14 15:56:30,218 [1666850] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/api/v3/search/issues?q=type%3Apr%20repo%3AMY_ORG%2FMY_REPO%20state%3Aopen GET : Connected
2018-12-14 15:56:30,458 [1667090] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/api/v3/search/issues?q=type%3Apr%20repo%3AMY_ORG%2FMY_REPO%20state%3Aopen GET : Result extracted
2018-12-14 15:56:30,460 [1667092] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/avatars/u/9264? get profile avatar : Connecting
2018-12-14 15:56:30,461 [1667093] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/avatars/u/7541? get profile avatar : Connecting
2018-12-14 15:56:31,659 [1668291] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/login?return_to=https%3A%2F%2FMY_COMPANY_URL%2Favatars%2Fu%2F7541%3F GET : Connected
2018-12-14 15:56:31,660 [1668292] INFO - .CachingGithubUserAvatarLoader - Error loading image from https://MY_COMPANY_URL/avatars/u/7541?
java.lang.IllegalStateException: ImageIO.read(stream) must not be null
at org.jetbrains.plugins.github.api.GithubApiContentHelper.loadImage(GithubApiContentHelper.kt:64)
at org.jetbrains.plugins.github.api.GithubApiRequests$CurrentUser$getAvatar$1$extractResult$1.convert(GithubApiRequests.kt:29)
at org.jetbrains.plugins.github.api.GithubApiRequests$CurrentUser$getAvatar$1$extractResult$1.convert(GithubApiRequests.kt:26)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$Base$createResponse$1.handleBody(GithubApiRequestExecutor.kt:214)
at org.jetbrains.plugins.github.api.GithubApiRequests$CurrentUser$getAvatar$1.extractResult(GithubApiRequests.kt:28)
at org.jetbrains.plugins.github.api.GithubApiRequests$CurrentUser$getAvatar$1.extractResult(GithubApiRequests.kt:26)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$Base$execute$1.process(GithubApiRequestExecutor.kt:122)
at com.intellij.util.io.HttpRequests.doProcess(HttpRequests.java:523)
at com.intellij.util.io.HttpRequests.process(HttpRequests.java:499)
at com.intellij.util.io.HttpRequests.access$100(HttpRequests.java:59)
at com.intellij.util.io.HttpRequests$RequestBuilderImpl.connect(HttpRequests.java:352)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$Base.execute(GithubApiRequestExecutor.kt:111)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$WithTokenAuth.execute(GithubApiRequestExecutor.kt:72)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader.loadAndDownscale(CachingGithubUserAvatarLoader.kt:58)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader.access$loadAndDownscale(CachingGithubUserAvatarLoader.kt:22)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader$requestAvatar$1$1$1.compute(CachingGithubUserAvatarLoader.kt:46)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader$requestAvatar$1$1$1.compute(CachingGithubUserAvatarLoader.kt:22)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$3(CoreProgressManager.java:180)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:164)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:582)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:532)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:87)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:151)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:180)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader$requestAvatar$1$1.get(CachingGithubUserAvatarLoader.kt:46)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader$requestAvatar$1$1.get(CachingGithubUserAvatarLoader.kt:22)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
2018-12-14 15:56:31,676 [1668308] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/login?return_to=https%3A%2F%2FMY_COMPANY_URL%2Favatars%2Fu%2F9264%3F GET : Connected
2018-12-14 15:56:31,677 [1668309] INFO - .CachingGithubUserAvatarLoader - Error loading image from https://MY_COMPANY_URL/avatars/u/9264?
java.lang.IllegalStateException: ImageIO.read(stream) must not be null
at org.jetbrains.plugins.github.api.GithubApiContentHelper.loadImage(GithubApiContentHelper.kt:64)
at org.jetbrains.plugins.github.api.GithubApiRequests$CurrentUser$getAvatar$1$extractResult$1.convert(GithubApiRequests.kt:29)
at org.jetbrains.plugins.github.api.GithubApiRequests$CurrentUser$getAvatar$1$extractResult$1.convert(GithubApiRequests.kt:26)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$Base$createResponse$1.handleBody(GithubApiRequestExecutor.kt:214)
at org.jetbrains.plugins.github.api.GithubApiRequests$CurrentUser$getAvatar$1.extractResult(GithubApiRequests.kt:28)
at org.jetbrains.plugins.github.api.GithubApiRequests$CurrentUser$getAvatar$1.extractResult(GithubApiRequests.kt:26)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$Base$execute$1.process(GithubApiRequestExecutor.kt:122)
at com.intellij.util.io.HttpRequests.doProcess(HttpRequests.java:523)
at com.intellij.util.io.HttpRequests.process(HttpRequests.java:499)
at com.intellij.util.io.HttpRequests.access$100(HttpRequests.java:59)
at com.intellij.util.io.HttpRequests$RequestBuilderImpl.connect(HttpRequests.java:352)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$Base.execute(GithubApiRequestExecutor.kt:111)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$WithTokenAuth.execute(GithubApiRequestExecutor.kt:72)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader.loadAndDownscale(CachingGithubUserAvatarLoader.kt:58)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader.access$loadAndDownscale(CachingGithubUserAvatarLoader.kt:22)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader$requestAvatar$1$1$1.compute(CachingGithubUserAvatarLoader.kt:46)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader$requestAvatar$1$1$1.compute(CachingGithubUserAvatarLoader.kt:22)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$3(CoreProgressManager.java:180)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:164)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:582)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:532)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:87)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:151)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:180)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader$requestAvatar$1$1.get(CachingGithubUserAvatarLoader.kt:46)
at org.jetbrains.plugins.github.util.CachingGithubUserAvatarLoader$requestAvatar$1$1.get(CachingGithubUserAvatarLoader.kt:22)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
2018-12-14 15:56:32,848 [1669480] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/api/v3/repos/MY_ORG/MY_REPO/pulls/3863 get pull request : Connecting
2018-12-14 15:56:33,488 [1670120] DEBUG - b.api.GithubApiRequestExecutor - Request: https://MY_COMPANY_URL/api/v3/repos/MY_ORG/MY_REPO/pulls/3863 GET : Connected
2018-12-14 15:56:33,571 [1670203] DEBUG - b.api.GithubApiRequestExecutor - Can't get pull request
org.jetbrains.plugins.github.exceptions.GithubJsonException: Can't get pull request
Couldn't parse GitHub response
at org.jetbrains.plugins.github.api.GithubApiContentHelper.readJson(GithubApiContentHelper.kt:46)
at org.jetbrains.plugins.github.api.GithubApiRequest$Companion$parseJsonResponse$1.convert(GithubApiRequest.kt:165)
at org.jetbrains.plugins.github.api.GithubApiRequest$Companion$parseJsonResponse$1.convert(GithubApiRequest.kt:163)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$Base$createResponse$1.readBody(GithubApiRequestExecutor.kt:210)
at org.jetbrains.plugins.github.api.GithubApiRequest$Companion.parseJsonResponse(GithubApiRequest.kt:165)
at org.jetbrains.plugins.github.api.GithubApiRequest$Companion.access$parseJsonResponse(GithubApiRequest.kt:163)
at org.jetbrains.plugins.github.api.GithubApiRequest$Get$Json.extractResult(GithubApiRequest.kt:61)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$Base$execute$1.process(GithubApiRequestExecutor.kt:122)
at com.intellij.util.io.HttpRequests.doProcess(HttpRequests.java:523)
at com.intellij.util.io.HttpRequests.process(HttpRequests.java:499)
at com.intellij.util.io.HttpRequests.access$100(HttpRequests.java:59)
at com.intellij.util.io.HttpRequests$RequestBuilderImpl.connect(HttpRequests.java:352)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$Base.execute(GithubApiRequestExecutor.kt:111)
at org.jetbrains.plugins.github.api.GithubApiRequestExecutor$WithTokenAuth.execute(GithubApiRequestExecutor.kt:72)
at org.jetbrains.plugins.github.pullrequest.data.GithubPullRequestsDataLoader$DataTask.run(GithubPullRequestsDataLoader.kt:90)
at com.intellij.openapi.progress.impl.CoreProgressManager$TaskRunnable.run(CoreProgressManager.java:727)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:164)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:582)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:532)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:87)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:151)
at com.intellij.openapi.progress.impl.CoreProgressManager$4.run(CoreProgressManager.java:403)
at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:314)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.jetbrains.io.mandatory.JsonMandatoryException: Field 'labels' is mandatory, but missing in response
at org.jetbrains.io.mandatory.NullCheckingFactory$1.read(NullCheckingFactory.java:57)
at com.google.gson.Gson.fromJson(Gson.java:927)
at com.google.gson.Gson.fromJson(Gson.java:892)
at org.jetbrains.plugins.github.api.GithubApiContentHelper.readJson(GithubApiContentHelper.kt:43)
... 27 more
Can someone please help?
I can't comment on yole's answer (not enough karma) and I couldn't pinpoint your profile on youtrack but I need some more logs to determine the root-cause of the issue.
Please enable debug logs (Help - Debug Log Settings...) for the following categories:
#org.jetbrains.plugins.github.api.GithubApiRequestExecutor
Then please reproduce the issue and share the entire logs folder zipped in the youtrack issue.
See https://intellij-support.jetbrains.com/hc/en-us/articles/207241085 to locate the logs.
This is a bug in IntelliJ IDEA. I've reported it to our issue tracker.
The actual issue appears to be Field 'labels' is mandatory, but missing in response. The field was implemented in GitHub Enterprise 2.13 only, and earlier versions do not return it in response.
The issue has been fixed in dev version and the will be backported to 2018.3 soon - https://youtrack.jetbrains.com/issue/IDEA-200881
As a side note, GitHub Enterprise 2.12 has been discontinued https://enterprise.github.com/releases/series/2.12 and most likely it will be updated soon in your organization, and PR will work even with the current IDE version

Unable to connect to Redis Server when using apache beam sdk

So I have a dataflow job doing
p.apply(RedisIO.read()
.withEndpoint(<public endpoint>, 6379)
.withAuth(<password>)
.withTimeout(60000)
.withKeyPattern("UID*"))
.apply(ParDo.of(new Format()))
.apply(TextIO.write().to(options.getOutput()));
The redis endpoint is public authenticated with a password with no firewall settings. When I run the above, I get the following error.
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-
plugin:1.6.0:java (default-cli) on project word-count-beam: An
exception occured while executing the Java class.
org.apache.beam.sdk.util.UserCodeException:
redis.clients.jedis.exceptions.JedisConnectionException:
java.net.ConnectException: Connection refused -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-
cli) on project word-count-beam: An exception occured while executing
the Java class. org.apache.beam.sdk.util.UserCodeException:
redis.clients.jedis.exceptions.JedisConnectionException:
java.net.ConnectException: Connection refused
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception
occured while executing the Java class.
org.apache.beam.sdk.util.UserCodeException: redis.clients.jedis.exceptions.JedisConnectionException: java.net.ConnectException: Connection refused
at org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:339)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: org.apache.beam.runners.direct.repackaged.com.google.common.util.concurrent
.UncheckedExecutionException:
org.apache.beam.sdk.util.UserCodeException:
redis.clients.jedis.exceptions.JedisConnectionException:
java.net.ConnectException: Connection refused
So connection is not getting established with public redis endpoint. I am getting the same error when I am running through DirectRunner. Am i missing something here?
There is a known error in the Apache Beam source code for RedisIO where withEndpoint ignores the input host and will attempt to use localhost instead. Attempting to connect to a redis server on localhost when there is none will give the error you are seeing.
You can read more about the issue here, and see a pull request with a fix here.
Until that pull request gets merged you should be able to resolve the problem by implementing the change yourself by copying RedisIO.java into your project and changing
.setConnectionConfiguration(connectionConfiguration().withHost(host))
.setConnectionConfiguration(connectionConfiguration().withPort(port))
to
.setConnectionConfiguration(connectionConfiguration().withHost(host).withPort(port))
Note this same error occurs 3 times in RedisIO, once each for Read (line 168), ReadAll (line 233), and Write (line 365).

wso2 iot analytics geofencing error

I've downloaded WSO2 IOT Server 3.0.0 and installed the product by following steps in wso2 documentation. I use Ubuntu 14.04 LTS.
https://docs.wso2.com/display/IoTS300/Installing+on+Linux+or+OS+X
Then, I started broker, core and analytics servers in given order. I did not make any additional configuration to follow link below:
https://docs.wso2.com/display/IoTS300/Running+the+Product#RunningtheProduct-StartingtheServer
I created sample users and user roles by running mobile-qsg.sh script.
I successfully enrolled to wso2 iot server with my Android( 6.0.1) device( Samsung SM-G900FQ). I used account for alex who has iotMobileUser role. I can see my device info on devicemgt console. In order to add a geofencing rule, I opened geo dashboard running on https//localhost:9445/portal/dashboards/geo-dashboard/, authenticated with admin account and able to login.
When I send a location request to my device via devicemgt api, it throws EventConversionException.
[2017-05-28 13:03:46,391] [IoT-Analytics] INFO {org.wso2.carbon.dashboard.deployment.DashboardDeployer} - Gadget directory [geo-dashboard] has been copied to path /home/dozc/wso2iot-3.0.0/analytics/repository/deployment/server/jaggeryapps/portal/store/carbon.super/fs/gadget/geo-dashboard
[2017-05-28 13:03:46,427] [IoT-Analytics] INFO {org.wso2.carbon.dashboard.deployment.DashboardDeployer} - Dashboard definition [geo-dashboard] has been created.
[2017-05-28 13:03:46,428] [IoT-Analytics] ERROR {org.wso2.carbon.dashboard.deployment.DashboardDeployer} - A theme already exists with the name geo-dashboard
[2017-05-28 13:03:46,442] [IoT-Analytics] INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} - Successfully Deployed Carbon Application : org.wso2.carbon.iot.geo.dashboard_1.0.0 {super-tenant}
[2017-05-28 13:03:46,751] [IoT-Analytics] INFO {org.wso2.carbon.ntask.core.impl.AbstractQuartzTaskManager} - Task scheduled: [-1234][ANALYTICS_SPARK_EVENTING][STORE_EVENT_ROUTER_TASK]
[2017-05-28 13:03:46,877] [IoT-Analytics] INFO {org.wso2.carbon.core.init.JMXServerManager} - JMX Service URL : service:jmx:rmi://localhost:11113/jndi/rmi://localhost:10001/jmxrmi
[2017-05-28 13:03:46,880] [IoT-Analytics] INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} - Deploying analytics event store: org_wso2_geo_FusedSpatialEvent.xml
[2017-05-28 13:03:46,918] [IoT-Analytics] INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} - Server : WSO2 IoT - Analytics-3.0.0
[2017-05-28 13:03:46,920] [IoT-Analytics] INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} - WSO2 Carbon started in 95 sec
[2017-05-28 13:03:46,990] [IoT-Analytics] INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} - WSO2EventConsumer added to the junction. Stream:org.wso2.geo.FusedSpatialEvent:1.0.0
[2017-05-28 13:03:46,992] [IoT-Analytics] INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} - Deployed successfully analytics event store: org_wso2_geo_FusedSpatialEvent.xml
[2017-05-28 13:03:47,606] [IoT-Analytics] INFO {org.wso2.carbon.ui.internal.CarbonUIServiceComponent} - Mgt Console URL : https://192.x.x.x:9445/carbon/
[2017-05-28 13:03:47,686] [IoT-Analytics] INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} - Event Receiver undeployed successfully: Geo-Receiver-HTTP-LocationStream.xml
[2017-05-28 13:03:47,721] [IoT-Analytics] INFO {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime} - Connecting receiver Geo-Receiver-HTTP-LocationStream
[2017-05-28 13:03:47,725] [IoT-Analytics] INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} - Producer added to the junction. Stream:org.wso2.geo.LocationStream:1.0.0
[2017-05-28 13:03:47,725] [IoT-Analytics] INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} - Event Receiver configuration successfully deployed and in active state: Geo-Receiver-HTTP-LocationStream
[2017-05-28 13:03:58,761] [IoT-Analytics] INFO {org.wso2.carbon.event.processor.manager.core.internal.CarbonEventManagementService} - Starting polling event receivers
[2017-05-28 13:26:31,353] [IoT-Analytics] INFO {org.wso2.carbon.databridge.core.DataBridge} - user admin connected
[2017-05-28 13:26:31,468] [IoT-Analytics] ERROR {org.wso2.carbon.databridge.core.internal.queue.QueueWorker} - Dropping wrongly formatted event sent for -1234
org.wso2.carbon.databridge.core.exception.EventConversionException: Error when converting org.wso2.android.agent.Stream:1.0.0 of event bundle with events 1
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:181)
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.toEventList(ThriftEventConverter.java:90)
at org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:73)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.wso2.carbon.databridge.core.exception.EventConversionException: No StreamDefinition for streamId org.wso2.android.agent.Stream:1.0.0 present in cache
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:166)
... 7 more
[2017-05-28 13:32:32,042] [IoT-Analytics] ERROR {org.wso2.carbon.databridge.core.internal.queue.QueueWorker} - Dropping wrongly formatted event sent for -1234
org.wso2.carbon.databridge.core.exception.EventConversionException: Error when converting org.wso2.android.agent.Stream:1.0.0 of event bundle with events 1
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:181)
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.toEventList(ThriftEventConverter.java:90)
at org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:73)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.wso2.carbon.databridge.core.exception.EventConversionException: No StreamDefinition for streamId org.wso2.android.agent.Stream:1.0.0 present in cache
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:166)
... 7 more
When I try to define a geofencing query and shape on geo dashboard, after pressing save, it throws follwing error:
[2017-05-28 13:47:35,301] [IoT-Analytics] ERROR {org.wso2.carbon.databridge.core.internal.queue.QueueWorker} - Dropping wrongly formatted event sent for -1234
org.wso2.carbon.databridge.core.exception.EventConversionException: Error when converting org.wso2.android.agent.Stream:1.0.0 of event bundle with events 1
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:181)
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.toEventList(ThriftEventConverter.java:90)
at org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:73)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.wso2.carbon.databridge.core.exception.EventConversionException: No StreamDefinition for streamId org.wso2.android.agent.Stream:1.0.0 present in cache
at org.wso2.carbon.databridge.receiver.thrift.converter.ThriftEventConverter.createEventList(ThriftEventConverter.java:166)
... 7 more
[2017-05-28 13:48:14,089] [IoT-Analytics] INFO {JAGGERY.store.carbon.super.fs.gadget.geo-dashboard.controllers.set_alerts:jag} - /* Enter a unique ExecutionPlan */
#Plan:name('Geo-ExecutionPlan-Within_home---_4e42e50120311aea_alert')
/* Enter a unique description for ExecutionPlan */
-- #Plan:description('ExecutionPlan')
/* define streams/tables and write queries here ... */
#Import('org.wso2.geo.StandardSpatialEvents:1.0.0')
define stream dataIn (id string, latitude double, longitude double, timeStamp long, type string ,speed float, heading float, eventId string);
#Export('org.wso2.geo.ProcessedSpatialEvents:1.0.0')
define stream dataOut (id string, latitude double, longitude double, timeStamp long, type string ,speed float, heading float, eventId string, state string, information string);
from dataIn[geo:within(longitude,latitude,"{'type':'Polygon','coordinates':[[[32.79895305633545,39.98135815980835],[32.799296379089355,39.98124306559073],[32.79925346374511,39.98077857624252],[32.799028158187866,39.98035107883081],[32.79843807220459,39.97984547747823],[32.79794454574585,39.98008800217112],[32.79849708080292,39.980803239473126],[32.79895305633545,39.98135815980835]]]}")==true and id == "4e42e50120311aea"]#geodashboard:subscribe()
select id , latitude, longitude,timeStamp, type, speed, heading ,eventId , "ALERTED" as state, "This device is in home restricted area!!!" as information
insert into dataOut;
from dataIn[geo:within(longitude,latitude,"{'type':'Polygon','coordinates':[[[32.79895305633545,39.98135815980835],[32.799296379089355,39.98124306559073],[32.79925346374511,39.98077857624252],[32.799028158187866,39.98035107883081],[32.79843807220459,39.97984547747823],[32.79794454574585,39.98008800217112],[32.79849708080292,39.980803239473126],[32.79895305633545,39.98135815980835]]]}")!=true and id == "4e42e50120311aea"]
select id , latitude, longitude,timeStamp, type, speed, heading ,eventId , "NORMAL" as state, "" as information
insert into dataOut;
[2017-05-28 13:48:14,139] [IoT-Analytics] WARN {org.wso2.carbon.server.admin.module.handler.AuthenticationHandler} - Illegal access attempt at [2017-05-28 13:48:14,0139] from IP address 127.0.0.1 while trying to authenticate access to service EventProcessorAdminService
[2017-05-28 13:48:14,144] [IoT-Analytics] ERROR {org.jaggeryjs.jaggery.core.manager.WebAppManager} - org.mozilla.javascript.WrappedException: Wrapped org.apache.axis2.AxisFault: Transport error: 401 Error: Unauthorized (/portal/store/carbon.super/fs/gadget/geo-dashboard/controllers/set_alerts.jag#117)
org.jaggeryjs.scriptengine.exceptions.ScriptException: org.mozilla.javascript.WrappedException: Wrapped org.apache.axis2.AxisFault: Transport error: 401 Error: Unauthorized (/portal/store/carbon.super/fs/gadget/geo-dashboard/controllers/set_alerts.jag#117)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:571)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:273)
at org.jaggeryjs.jaggery.core.manager.WebAppManager.exec(WebAppManager.java:588)
at org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:508)
at org.jaggeryjs.jaggery.core.JaggeryServlet.doPost(JaggeryServlet.java:29)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:747)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:485)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:377)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:337)
at org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:21)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.dashboard.store.filter.TenantStoreFilter.doFilter(TenantStoreFilter.java:51)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:120)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.event.receiver.core.internal.tenantmgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:48)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:958)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1756)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1715)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.mozilla.javascript.WrappedException: Wrapped org.apache.axis2.AxisFault: Transport error: 401 Error: Unauthorized (/portal/store/carbon.super/fs/gadget/geo-dashboard/controllers/set_alerts.jag#117)
at org.mozilla.javascript.Context.throwAsScriptRuntimeEx(Context.java:1754)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:148)
at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:225)
at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
at org.jaggeryjs.rhino.portal.store.carbon.super.fs.gadget.geo_dashboard.controllers.c6._c_script_0(/portal/store/carbon.super/fs/gadget/geo-dashboard/controllers/set_alerts.jag:117)
at org.jaggeryjs.rhino.portal.store.carbon.super.fs.gadget.geo_dashboard.controllers.c6.call(/portal/store/carbon.super/fs/gadget/geo-dashboard/controllers/set_alerts.jag)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
at org.jaggeryjs.rhino.portal.store.carbon.super.fs.gadget.geo_dashboard.controllers.c6.call(/portal/store/carbon.super/fs/gadget/geo-dashboard/controllers/set_alerts.jag)
at org.jaggeryjs.rhino.portal.store.carbon.super.fs.gadget.geo_dashboard.controllers.c6.exec(/portal/store/carbon.super/fs/gadget/geo-dashboard/controllers/set_alerts.jag)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:567)
... 45 more
Caused by: org.apache.axis2.AxisFault: Transport error: 401 Error: Unauthorized
at org.apache.axis2.transport.http.HTTPSender.handleResponse(HTTPSender.java:326)
at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:196)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:451)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:278)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:430)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.wso2.carbon.event.processor.stub.EventProcessorAdminServiceStub.validateExecutionPlan(EventProcessorAdminServiceStub.java:2207)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
... 54 more
I searched all issues related to these exceptions and made additional configurations mentioned in older product documentations, but couldn't find a solution. Is there any additional configuration that I could miss?
Thanks in advance
First issue is because of that org.wso2.android.agent.Stream definition is not available with vanila distribution pack.
You can download it from the this link or build this. Extract the archive and copy the android_agent_analytics-1.0.0.car into $IOT_HOME/analytics/repository/deployments/server/carbonapps path (this issue is fixed in the latest).
For the second issue; There's an authentication issue. Did you try login into the /portal with a private browser window? It might fix the issue.
WSO2 IoT 3.1.0 is released, and we have fixed those issues. And now it includes the geo-fencing alert for exit and entry too.

JDBC client to Hive - No data or no sasl data in the stream Exception

We have a Kerberised cluster and I'm trying to run a Java action in Oozie where I make a JDBC connection to Hive. This JDBC connections works fine on the Sandbox without Kerberos.
The connection string is as simple as the following, where I'm providing username and password in it:
Connection con = DriverManager.getConnection("jdbc:hive2://W12345:10000/control;principal=hive/W12345.companynet.net#COMPANYNET.NET","user123","passw123");
The Oozie action (strangely) completes succesfully, and the Java action log does not present any error:
1742 [main] INFO org.apache.hive.jdbc.Utils - Supplied authorities: W12345:10000
1742 [main] INFO org.apache.hive.jdbc.Utils - Resolved authority: W12345:10000
1766 [main] INFO org.apache.hive.jdbc.HiveConnection - Will try to open client transport with JDBC Uri: jdbc:hive2://W12345:10000/control;principal=hive/W12345.companynet.net#COMPANYNET.NET
<<< Invocation of Main class completed <<<
Oozie Launcher ends
1785 [main] INFO org.apache.hadoop.mapred.Task - Task:attempt_1464245290012_0129_m_000000_0 is done. And is in the process of committing
1847 [main] INFO org.apache.hadoop.mapred.Task - Task attempt_1464245290012_0129_m_000000_0 is allowed to commit now
1854 [main] INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output of task 'attempt_1464245290012_0129_m_000000_0' to hdfs://danskehadoop/user/user123/oozie-oozi/0000013-160527101253015-oozie-oozi-W/JavaAction--java/output/_temporary/1/task_1464245290012_0129_m_000000
1909 [main] INFO org.apache.hadoop.mapred.Task - Task 'attempt_1464245290012_0129_m_000000_0' done.
But in reality the Java main does not complete correctly the execution (and does not execute the needed queries) because the JDBC connection fails with an exception that I can see only in the Hive log:
ERROR [HiveServer2-Handler-Pool: Thread-78363]: server.TThreadPoolServer (TThreadPoolServer.java:run(296)) - Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:739)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:736)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1637)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:736)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:328)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
... 10 more
I'm actually connected to the cluster, and already done further kinit on my username.
Does anybody know what could the cause of this exception be?
Thanks in advance for the help!
Antonio
This happened to me on MapR hadoop distribution platform.
In my case it was Keepalived checking Hive port every 5 seconds and producing such error. I simply used "nc" command to check if Hive port is in use and did not use any authentication method. Later I switched to "maprcli" command which uses SASL authentication and the error was gone.

WSO2 api manager is not sending data to WSO2 DAS (Data analytics server)

I configured API manager to send data to WSO2 data analytics server.
My configuration on API server is:
Event Receiver Configurations: {tcp://wso2-dac-svc.libre-dev.com:7611}
Data Analyzer Configurations: https://wso2-dac-svc.libre-dev.com:8443
On DAC server I installed API_Manager_Analytics.car which has event receiver definitions.
On API server I have imported certificate from DAC server using keytool. I restarted both servers.
I am seeing following error in the log:
TID: [-1] [] [2016-05-16 16:06:11,417] ERROR {org.wso2.carbon.databridge.agent.thrift.AsyncDataPublisher} - Error while connection to event receiver {org.wso2.carbon.databridge.agent.thrift.AsyncDataPublisher}
org.wso2.carbon.databridge.commons.exception.AuthenticationException: Access denied for user admin to login TCP,wso2-dac-svc.libre-dev.com:7611,TCP,wso2-dac-svc.libre-dev.com:7711
at org.wso2.carbon.databridge.agent.thrift.internal.publisher.authenticator.AgentAuthenticator.connect(AgentAuthenticator.java:54)
at org.wso2.carbon.databridge.agent.thrift.DataPublisher.start(DataPublisher.java:273)
at org.wso2.carbon.databridge.agent.thrift.DataPublisher.<init>(DataPublisher.java:161)
at org.wso2.carbon.databridge.agent.thrift.AsyncDataPublisher$ReceiverConnectionWorker.run(AsyncDataPublisher.java:843)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.wso2.carbon.databridge.agent.thrift.exception.AgentAuthenticatorException: Thrift exception
at org.wso2.carbon.databridge.agent.thrift.internal.publisher.authenticator.ThriftAgentAuthenticator.connect(ThriftAgentAuthenticator.java:51)
at org.wso2.carbon.databridge.agent.thrift.internal.publisher.authenticator.AgentAuthenticator.connect(AgentAuthenticator.java:51)
... 8 more
Caused by: org.apache.thrift.transport.TTransportException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: No trusted certificate found
at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147)
at org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163)
at org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:91)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
at org.wso2.carbon.databridge.commons.thrift.service.secure.ThriftSecureEventTransmissionService$Client.send_connect(ThriftSecureEventTransmissionService.java:82)
at org.wso2.carbon.databridge.commons.thrift.service.secure.ThriftSecureEventTransmissionService$Client.connect(ThriftSecureEventTransmissionService.java:73)
at org.wso2.carbon.databridge.agent.thrift.internal.publisher.authenticator.ThriftAgentAuthenticator.connect(ThriftAgentAuthenticator.java:47)
... 9 more
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: No trusted certificate found
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:914)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:747)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145)
... 15 more
Caused by: sun.security.validator.ValidatorException: No trusted certificate found
at sun.security.validator.SimpleValidator.buildTrustedChain(SimpleValidator.java:394)
at sun.security.validator.SimpleValidator.engineValidate(SimpleValidator.java:133)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491)
... 23 more
TID: [-1] [] [2016-05-16 16:06:41,363] ERROR {org.wso2.carbon.databridge.agent.thrift.AsyncDataPublisher} - Reconnection failed fortcp://wso2-dac-svc.libre-dev.com:7611 {org.wso2.carbon.databridge.agent.thrift.AsyncDataPublisher}
TID: [-1] [] [2016-05-16 16:07:11,004] WARN {org.apache.synapse.core.axis2.TimeoutHandler} - Expiring message ID : urn:uuid:6ed9fae5-d1fb-4cdf-885b-e101e79faf40; dropping message after timeout of : 30 seconds {org.apache.synapse.core.axis2.TimeoutHandler}
TID: [-1] [] [2016-05-16 16:07:11,371] ERROR {org.wso2.carbon.databridge.agent.thrift.AsyncDataPublisher} - Reconnection failed fortcp://wso2-dac-svc.libre-dev.com:7611 {org.wso2.carbon.databridge.agent.thrift.AsyncDataPublisher}
TID: [-1] [] [2016-05-16 16:07:34,514] WARN {org.apache.synapse.transport.passthru.TargetHandler} - http-outgoing-9: Connection time out while in state: REQUEST_DONE {org.apache.synapse.transport.passthru.TargetHandler}
Basically I cannot get API stats sent to DAS server. Any help is appreciated.
Based on the error log the issue seems to be wit
The Data Analytic Server Thrift runs on port 7711 uses the 'wso2carbon.jks' located in /repository/resources/security by deault. As a result when we need to configure this on /repository/conf/carbon.xml as shown below
<Security> <br>
<!--<br>
KeyStore which will be used for encrypting/decrypting passwords<br>
and other sensitive information.
--><br>
<KeyStore><br>
<!-- Keystore file location--><br>
<Location>${carbon.home}/repository/resources/security/wso2carbon.jks</Location><br>
<!-- Keystore type (JKS/PKCS12 etc.)--><br>
<Type>JKS</Type><br>
<!-- Keystore password--><br>
<Password>wso2carbon</Password>
<!-- Private Key alias--><br>
<KeyAlias>wso2carbon</KeyAlias>
<!-- Private Key password-->
<KeyPassword>wso2carbon</KeyPassword><br>
</KeyStore><br>
For adding a new key store please use the below steps
1. Place the key store in '/repository/resources/security/' folder
2. Update the section 'Security/KeyStore/' of /repository/conf/carbon.xml accordingly
3. Update the keystore references of the data-agent-config.xml accordingly
4. Import the certificate of the new keystore to ESB's trust store located at /repository/resources/security/client-truststore.jks.
Finally, once the private key is changed, its relevent certificate should be imported to the API-Manager trust store located at /repository/resources/security/client-truststore.jks.
Hope this steps will sort the issues on the given error log.