Currently in our dev environment we have hardcoded the beeline connect string to something like
beeline -u 'jdbc:hive2://zk0-hi-clu.3qy32mhqlj1ubaea5iyw5joamf.ax.internal.cloudapp.net:2181,zk1-hi-clu.3qy32mhqlj1ubaea5iyw5joamf.ax.internal.cloudapp.net:2181,zk6-hi-clu.3qy32mhqlj1ubaea5iyw5joamf.ax.internal.cloudapp.net:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2' --hivevar date=20190101 -f test.hql
I am trying to see if there are ways we can make the connect dynamic like it will look up a config file like odbc.ini. so when we promote the code to other environment, it will automatically connect to the correct target. Is this possible?
Not exactly your case: I need some defaults in my shell and use the alias functionality of the bash.
export BEELINE_CONNECTION_STRING='jdbc:hive2://myzookeeper1:2181,myzookeeper2:2181,myzookeeper3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;'
alias beeline='beeline -u ${BEELINE_CONNECTION_STRING}'
After this typing beeline causes this:
beeline
Connecting to jdbc:hive2://myzookeeper1:2181,myzookeeper2:2181,myzookeeper3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;
Connected to: Apache Hive (version 1.2.1000.2.6.5.0-292)
Driver: Hive JDBC (version 1.2.1000.2.6.5.0-292)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1000.2.6.5.0-292 by Apache Hive
0: jdbc:hive2://myhive>
Related
I have an airflow dag that uses beeline to execute queries. Everything was working well,then system has to reboot due to blackout. After that airflow couldn't connect to beeline. The error says
[Errno 2] No such file or directory: 'beeline': 'beeline'.
But beeline is installed in the same server where airflow runs. What could be the reason. Can anybody help me for this.
I can execute same beeline -u command outside airflow in command line. And it could connect to beeline.
When i run command hive it is only able to start from bin folder beacause metastore is created in bin only if i run it from home its not able to get start and shows error.
I have added these lines in my .bashrc file for hive
HIVE env variables
export HIVE_HOME=/opt/hadoop/hive/apache-hive-2.3.4-bin
export PATH=$HIVE_HOME/bin:$PATH
Can you try to setup path as mentioned below and retry,
user#ubuntu:~$ sudo gedit ~/.bashrc
Copy and paste the following lines at end of the file
# Set HIVE_HOME
export HIVE_HOME="/opt/hadoop/hive/apache-hive-2.3.4-bin"
PATH=$PATH:$HIVE_HOME/bin
export PATH
But here my suggestion is, instead of using hive command prompt try to use recommended way that is beeline client. If you have hiveserver2 configured you can connect using beeline client and query to hive.
I have a script that needs to connect to oracle db hosted on a different server .I am able to connect to this oracle db using sqldeveloper.But i am not able to configured it in my bash script .
SQLDEVELOPER 4.0 is the tool that i use to connect through gui .How can i use this in my script .Is there any other way to do it ?Do i need any other software (sqlplus)
try using following once:
sqlplus db_user_name/password_for_user#DB_schema < Input_file.sql > Output
You need sqlplus to achieve what you're trying to do. The syntax of the command you need to put in you shell script should be like:
sqlplus 'USER/PASSWORD#(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=DB_HOST)(PORT=1521)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=SERVICE_NAME_YOU_USE_IN_SQLDEVELOPER)))'
On *nix systems, this will create a csv stream of results to standard out:
java -Djava.security.egd=file///dev/urandom -jar jdbcsql.jar -d oracledb_SID -h $host -p 1521 -U some_username -m oracle -P "$PW" -f excel -s "," "$1"
Note that adding the -Djava.security.egd=file///dev/urandom increases performance greatly
Windows commands are similar: see http://jdbcsql.sourceforge.net/
I am using Postgres via a vagrant instance running ubuntu-xenial-16.04-cloudimg box and I have an sql dump from another developer.
By the way, I tried using PGAdmin IV from my Win 10 host machine after I had connected to the Postgres server on the virtualbox (ubuntu) but it takes forever and not running.
How can I import this to the Postgres running on virtualbox instance?
So given an sql dump file as dump.sql.
Run vagrant ssh on an ssh client like git bash(for windows)
Put the dump file in the directory containing the vagrantfile on the host machine. As it syncs by default with the guest machine or run vagrant rsync, just to make sure.
Navigate to the vagrant directory on the host machine (eg cd ../../ for an ubuntu guest on a window host)
Run psql -h hostname -U test -d databasename -f dump.sql.
Depending on the format of the dump (normal or custom), you can use psql or pg_restore.
Check the --format option in the documentation for pg_dump
Following the simple steps below solves my problem:
After vagrant up, vagrant ssh to log into the os
Type psql command
Then create database your_db_name to create the empty db
Make sure the dump sql file is in the folder containing the vagrantfile ( cd vagrant) or a sub folder within
Write this command to import the dump file into the newly create db
your_db_name -f /path/to/the/dump.sql
I hope the steps will help you too.
We have couple of hql files for compiling ddls.
in hive we used the following command from bash :
hive -v -f abc.hql
but, in beeline this doesn't work from bash. Any idea what can be the equivalent command for beeline.
Make sure your hiveserver2 is up & running on some port
In beeline
beeline -u "jdbc:hive2://localhost:port/database_name/" -f abc.hql
Refer this doc for more commands
https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients
Refer this doc if you have not yet configured hiveserver2
https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2