i would like to install datastax 4.7.2 or the latest version on ec2 and take advantage of spark and mllib. Is there a recommended image that I can use to ssh into?
You can use the DataStax AMI. DSE is free in Dev or in Prod for qualifying startups.
Just get your credentials by filling out the download form.
Related
We use Apache Ignite 2.11.0 and plan to migrate to GridGain.
Which version is compatible and how to migrate to GridGain Community?
We use .NET Core. Do we change the program code or migrate without change?
GridGain is based on Apache Ignite, but they are not 100% compatible, and there is no 1-to-1 version mapping.
Having said that:
In most cases the migration is smooth - just change the package reference.
Latest available GridGain version is recommended.
How do I change the table version via the Hudi CLI?
Steps:
ssh into EMR
kick off the hudi cli /usr/lib/hudi/cli/bin/hudi-cli.sh. Version of the Hudi CLI is 1.
connect to my table connect --path s3://bucket/db/table
In the desc of the table I see that it is version=3, but I want to use Hudi 0.9.0 to write to the table so I would like to set the table to version=2.
org.apache.hudi.exception.HoodieException: Unknown versionCode:3
at org.apache.hudi.common.table.HoodieTableVersion.lambda$versionFromCode$1(HoodieTableVersion.java:54)
at java.util.Optional.orElseThrow(Optional.java:290)
at org.apache.hudi.common.table.HoodieTableVersion.versionFromCode(HoodieTableVersion.java:54)
at org.apache.hudi.common.table.HoodieTableConfig.getTableVersion(HoodieTableConfig.java:246)
Sadly, I'm not aware of any way to use version 0.9.0 to downgrade 3 to 2, due to the error you are getting. There is no way for version 0.9.0 to know how 0.10.0 was writing things differently.
Recently, AWS has 6.6 available for use, but it isn't well documented. I'd recommend switching over to that, because it has hudi version 0.10.0 and can then do that downgrade.
This link should get updated whenever 6.6 gets updated in the docs.
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-app-versions-6.x.html
Side note, if you are using the bootstrap action script provided by AWS to repair the log4j vulnerability, I'd recommend taking the version 6.5 version provided and editing it to be 6.6. There is not a 6.6 script available at this time, but I did that and was not able to detect any vulnerabilities.
This link provides an explanation on the bootstrap action:
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-log4j-vulnerability.html
I am trying to install impala in a docker container(using MAPR documentstion).In this docker I am running only Impala service and remaining hive,maprfs services will be running on physical node.When starting impala-server(impala daemon) I am getting wearied errors.I just wanted to know whether this kind of installation is possible or not.
Thanks for Help!!
It is possible, but it depends on your Impala and MapR version. Impala 2.2.0 is supported on MapR 5.x. Impala 2.5.0 is supported on MapR 5.1 and later. Check enter link description here before proceeding.
From Zeppelin-0.7, Zeppelin started supporting Helium plugins/packages using Helium Framework. However, I am not able to view any of the plugin on Helium page (localhost:8080/#/helium). As per this JIRA, I placed sample Helium.json (available on s3) under /local-repo/helium-registry-cache. However, after that I got NPE while restarting Apache Zeppelin service.
I have tried Zeppelin 0.7 as well as Zeppelin 0.8.0 snaptshot versions. In particular, I want to use map Helium package - Helium-Map in Zeppelin note.
Can some one point me to any guide or documentation having detailed steps of using Helium package in Zeppelin? Any help would be greatly appreciated!
Zeppelin 0.7.x
Zeppelin 0.7.x doesn't support the online registry. In other words,
Zeppelin doesn't use helium.json. So you need to install each package by yourself.
clone the helium package what you want to install
modify the artifact value to the absolute path considering your local machine in helium-xxx.json
copy zeppelin-xxx.json into the $ZEPPELIN_HOME/helium directory (create if it doesn't exist yet)
restart Zeppelin and go to the localhost:8080/#/helium page, then install the package.
Zeppelin 0.8.0-SNAPSHOT
Zeppelin 0.8.0-SNAPSHOT supports the online registry. So you can install without any preparation.
But the NPE problem you've faced was fixed after https://github.com/apache/zeppelin/pull/2380.
So please git pull origin master and rebuild it :)
FYI, Now Zeppelin provides proxy functionality for helium. Refer https://github.com/apache/zeppelin/pull/2363
Is there a single version of phoenix that is compatible with HBase provided in both Cloudera 5.5 and Hortonworks 2.4?
Hortonworks provides custom fixes and "backports" to their version of Phoenix in their HDP distribution. Cloudera may do the same as well.
I am assuming that you are asking about a client version that is compatible with both server versions.
Are you using the "thin" client jars? Do you find that your application does not work for one distribution or the other (dependent on which version jars you have)? Your application may work for both distributions if you use the non-thin jars.
If you would like to continue using the thin client, you may have to set phoenix.queryserver.serialization to JSON. HDP 2.3.4+ use PROTOBUF by default whereas CDH does not currently support PROTOBUF.
If you are asking about manually installing a version of the Phoenix server that can be installed on both distributions, both use HBase 1.1.x. Any Phoenix version 4.4+ can be used on either distribution. But I recommend using the version that is distributed with the platform.
A Phoenix 4.5.2 package for CDH 5.5.x is available via Cloudera Labs:
http://blog.cloudera.com/blog/2015/11/new-apache-phoenix-4-5-2-package-from-cloudera-labs/
Note however that Cloudera Labs packages are for dev/test only (not supported by Cloudera).