does Flyway can support Hive or Impala JDBC Drivers?
I Googled around and found almost nothing (examples, problems) about this.
If i start a migration specifying the Hive Driver (previously downloaded in the /drivers path) I'm getting
"ERROR: Unable to autodetect JDBC driver for url: jdbc:hive2://HOSTNAME:10000/DATABASE;principal=hive/HOSTNAME#PRINCIPAL"
Thanks all in advance for the support.
Please refer the below link, there is an Open issue for Hive support for flyway
https://github.com/flyway/flyway/issues/142
Related
I am trying install hive in ubuntu. But I am getting this error when i try to create derby metastore
schematool -dbType derby -initSchema
I have configured these things in .bashrc
HIVE_HOME
I configured these things in bin/hive-config.sh
hive-config.sh
What am I doing wrong here? Please help me with this
Thanks in advance
I have tried different versions of hive. Also tried pasting there HIVE_HOME variables in different lines. Hadoop was running while configuring these things.
I am unable to connect to an AWS RDS instance using liquibase from the terminal using a Mac. I am able to connect to the same database and URL using MySQL workbench and pymysql in python. I have mysql-connector-java.jar in Liquibase's lib folder. I have also done brew install mysql and brew install mysql-client. I get the same error when using an invalid URL, but I have checked (many times) to make sure the URL string is correct. The exact error I get is:
Unexpected error running Liquibase: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: Connection could not be created to jdbc:mysql://-URL IS HERE-?createDatabaseIfNotExists=true with driver com.mysql.cj.jdbc.Driver. Communications link failure
From the command:
liquibase --url=jdbc:mysql://-URL IS HERE-:3306/cdp_sms?createDatabaseIfNotExists=true --username="cdpadmin" --password=$CDP_DB_PW --changeLogFile=db.changelog-master.xml update
Any help would be greatly appreciated.
I had encounter a similar issue a while ago with Liquibase and AWS RDS. I would suggest you to check your security groups (inbound/outbound rules) in the aws settings.
we have usecase of presto hive accessing s3 file present in avro format.
When we try to use standalone hive-metastore and read this avro data using external table ,we are getting issue SerDeStorageSchemaReader class not found issue
MetaException(message:org.apache.hadoop.hive.metastore.SerDeStorageSchemaReader class not found)
at org.apache.hadoop.hive.metastore.utils.JavaUtils.getClass(JavaUtils.java:54)
We understand this error is coming because SerDeStorageSchemaReader class is not available in standalone-metastore.
i want to understand can be run hive-metastore without using hive/hadoop or there is any other option too?
standalone hive doesnt support avro. we need to install full hadoop plus hive version and start only hive metastore to fix it
In beeline, I could not see the job execution info (like job progress), I have already set the following properties in hive-site.xml. Could anyone help to figure out how to diagnose such issues ? How can I check whether hive server2 take the correct configuration ?
hive.server2.logging.operation.level VERBOSE
I only see the following log in beeline
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
did you try
set hive.root.logger=INFO,console;
I forget to set hive.async.log.enabled to be false, after this setting it works
I am trying to configure Hive on Spark but even after trying for 5 days i am not getting any solution..
Steps followed:
1.After spark installation,going in hive console and setting below proeprties
set hive.execution.engine=spark;
set spark.master=spark://INBBRDSSVM294:7077;
set spark.executor.memory=2g;
set spark.serializer=org.apache.spark.serializer.KryoSerializer;
2.Added spark -asembly jar in hive lib.
3.When running select count(*) from table_name I am getting below error:
2016-08-08 15:17:30,207 ERROR [main]: spark.SparkTask (SparkTask.java:execute(131))
- Failed to execute spark task, with exception
'org.apache.hadoop.hive.ql.metadata.HiveException (Failed to create spark client.)'
Hive version: 1.2.1
Spark version: tried with 1.6.1,1.3.1 and 2.0.0
Would appreciate if any one can suggest something.
You can download spark-1.3.1 src from spark download website and try to build spark-1.3.1 without hive version using:
./make-distribution.sh --name "hadoop2-without-hive" --tgz "-Pyarn,hadoop-provided,hadoop-2.4" -Dhadoop.version=2.7.1 -Dyarn.version=2.7.1 –DskipTests
Then copy spark-assembly-1.3.1-hadoop2.7.1.jar to hive/lib folder.
And follow https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started#HiveonSpark:GettingStarted-SparkInstallation to set necessary properties.
First of all, you need to pay attention to which versions are compatible. If you choose Hive 1.2.1, I advise you to use Spark 1.3.1. You can see the version compatibility list here.
The mistake you have is a general mistake. You need to start Spark and see what errors the Spark Workers says. However, have you already copied the hive-site.xml to spark/conf?