I am new to Hive, MapReduce and Hadoop.
I am using Putty to connect to hive table and access records in the tables. So what I did is- I opened Putty and in the host name I typed- vip.name.com and then I click Open. And then I entered my username and password and then few commands to get to Hive sql. Below is the list what I did
$ bash
bash-3.00$ hive
Hive history file=/tmp/rkost/hive_job_log_rkost_201207010451_1212680168.txt
hive> set mapred.job.queue.name=mdhi-technology;
hive> select * from table LIMIT 1;
So my question is-
Is there any other way I can do the same thing in any Sql client like Sql Developer or Squirel SQL Client instead of doing it from the command prompt. And if it is there then what is the step by step process to do this considering my example as I am logging to vip.name.com from Putty .
And same thing if I need to do through JDBC Program in my windows machine then how I can do it. Means with the JDBC Program, how I can access Hive tables and get the result back. As I know how I can do this with the oracle tables. But the only confusion I have is, as I am using this hostname vip.name.com to log into Putty. I am hoping the question is clear. Any suggestion will be appreciated.
In short my question is- Can I do the same thing in any SQLClient instead of logging from the Putty?
Update-
I tried doing the way Mark has suggested me. But I am always getting- Hive: Could not establish connection to vip.host.com:10000/default: java.net.ConnectionException: Connection timed out: connect
What are you doing with Putty is SSH'ing into a machine with Hive installed and set up. Then you are issuing Hive queries from the Hive command line. That is one way of issuing Hive queries. There are other ways that don't require SSH'ing, one you probably need is connection via JDBC.
Here is an article which describes how to connect to a Hive installation on Amazon's EMR cluster using SQuirreL via JDBC. The article might appear to be Amazon specific but it's not. As long you have Hive server running on one of the nodes of the cluster and no firewall impeding connection between the client machine and one running Hive, you should be able to connect.
A couple things you might want to keep in mind related to the above link:
You can ignore step 3 where it asks you to create a SSH tunnel unless you are using EMR.
The port that you enter in your connection URI might be different in your case. Replace localhost with the fully qualified domain name of the machine that Hive is running on. To find out which port Hive server is listening on, you can look into your Hive server nanny log file present in the log directory (whose location depends on your installation) or run a simple netstat -a command. I believe 10000 is the default port number, so it might make sense to try out 10000 directly.
Related
I have a flyway script that I run to migrate my db to Azure SQL db. Looks something like this:
flyway.url=jdbc:sqlserver://myazuresqlserver.database.windows.net/myazuresqldatabase;ssl=request;integratedSecurity=false;
flyway.user=myazuresqluser
flyway.password=myazuresqlpassword
...
And when I run it I get the following error:
Unable to obtain connection from database (jdbc:sqlserver://myazuresqlserver.database.windows.net/myazuresqldatabase;ssl=request;integratedSecurity=false;) for user 'myazuresqluser': The TCP/IP connection to the host dbserver-appinfra-test-000.database.windows.net/sqldb-appinfra-smartsuitemainoperational, port 1433 has failed. Error: "myazuresqlserver.database.windows.net/myazuresqldatabase. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".
So the thing is I already have one db that I migrated to another Azure SQL db. The script file for that one was almost idetical to the one I am using to migrate this database. I saw posts similar to mine, and, as you can see, I tried adding stuff like "ssl=request" to the connection string, which didn't work, and also setting setting the minimum required TLS version of the azure sql db to the lowest one.
So, because I followed the exact same steps for the other database and it the migration to Azure worked there, I looked closely into the connection strings of my azure databases. Both of the databases are on the same azure SQL server that I created, and both of them are in the same elastic pool that I created.
From my understanding the problem is that, looking at the connection strings, both of the database have this in their connection strings:
myazuresqlserver:1433
Also, important to mention - I did try to connect to this database from Azure Data Studio and SSMS. And it connects just fine
So I think they are both on the same port of the same server, that is the problem.
Could someone let me know how I could change the port of my Azure SQL database to something else in Azure? Or if it is unrelated please let me know what I can do. Help much appreciated!
The problem was in the line flyway.url=jdbc:sqlserver://myazuresqlserver.database.windows.net/myazuresqldatabase
This is what I had in my other flyway migration file (which worked perfectly) so I thought there shouldn't be any problem with this line. But I had to change it to
flyway.url=jdbc:sqlserver://myazuresqlserver.database.windows.net;database=myazuresqldatabase;
Hope this will help someone!
I am trying to connect to hive databases with a client, I have tried using DBeaver and downloaded the hive driver, but after that I have noticed that there is a kerbero's instance in the middle, and it seems that the dbeaver driver doesn't supoort kerberos.
¿There is some windows client suitable to query hive databases easy to plug in, considering the kerbero's instance?
Thanks in advance.
I'm getting started with Data Grip and I'm stuck before I started doing anything. First thing I've done was creating postgre Data Source with default parameters.
Now I opened a console, and tried to run a script:
drop table table1;
The console then prints:
Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
What can I do about that, so that I could create my project in e.g.: D:\Database\ instead of connecting to other hosts?
Sounds like you're not actually running PostgreSQL locally. Is it installed and running? PostgreSQL is a client/server-based database system, so it requires the server process to be running. This is not like SQLite or HSQLDB that run self-contained within your application (or DataGrip in this case). Please see here for a tutorial on how to get started with Postgres on a Mac. There are similar tutorials for other operating systems.
I installed oracle sqlplus and instant client on mac os using this tutorial: https://tomeuwork.wordpress.com/2014/05/12/how-to-install-oracle-sqlplus-and-oracle-client-in-mac-os/comment-page-1/
I can now open sqlplus in the terminal by typing sqlplus but it asks me for username and password.
I researched on the net that these should be created during installation however when I done it, it didn't prompt me and in the tutorial it doesn't mention it. It only says you should now be able to connect using $ sqlplus username/password#database but I don't have a username, password, nor a database ready.
How do I do this?
I want to set everything up so I can create a database and create tables etc using sqlplus.
SQLplus is a command line client for the Oracle database system and the Instant Client is a slim driver from accessing it. So you haven't installed the Oracle database (server) yet, just the client.
The Oracle server does not run on Mac OS X. You will need to install it on a machine with a different operation system.
Update
Using SQLplus you can connect to the Oracle server at your university (unless they have specifically guarded against it). You will need some information to connect:
Host name
Port number
SID or service name
username
password
Then you can connect using the following command:
sqlplus username#host:port/service
Or:
sqlplus username#'(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname)(PORT=port))(CONNECT_DATA=(SID=sid)(SERVER=dedicated)))'
SQL Developer from Oracle is a free alternative client with a UI that will make it easier to connect to the database and do your assignment. It requires the same information though.
I just heard about amazon rds and MySQL Workbench today from my partner for my group project, so I'm not very familiar with them. My partner has created a host account on amazon rds for our database, and I am suppose to help create table and make sql queries. However, I'm having no luck connecting to the database through MySQL workbench. The error message I received is "Unknown MySQL server host '*host_address*'. I have tried adding port 3306 to my firewall as suggested by a blogger, but that still doesn't work. There are some answers that say in order to connect to amazon rds, the user would need to add his/her IP address to the "DB Security Group" in amazon rds. So, does that mean my partner would need to add MY IP address to "DB Security Group" in the amazon rds account that my partner had created for the database? I been working on getting the connection for the last 6 hours, so I'm very greatful if someone could answer my question or point me in the right direction. Thank you for all your help.
The RDS Security Group for your RDS instance needs to be configured to allow access from your IP address. Once it is done, then you can verify that you have GOOD connectivity by running telnet <you_RDS_Instance_Name> 3306. Once you are able to connect fine, then I should be able to connect using MySQLWorkbecnh.
I have personally never used MySQL Workbench but I connect using MySQL cli to my RDS instance.
Here is the sample telnet command:
# telnet myrds.us-west-2.rds.amazonaws.com 3306
Trying 10.249.88.207...
Connected to myrds.us-west-2.rds.amazonaws.com.
Escape character is '^]'.
N
5.6.13-log▒)BM#5)n▒\eeM/B?Vr[P_mysql_native_password^]
telnet> q
Connection closed.
Read More about RDS Security Groups here.
This blog posting (http://thoughtsandideas.wordpress.com/2012/05/17/monitoring-and-managing-amazon-rds-databases-using-mysql-workbench/) describes how to use MySQL Workbench with Amazon RDS. It's for an older version of MySQL Workbench but still valid.