query to both hive and metastore - hive

I want to get column names and min/max of each columns with query.
Assume that i only know the name of table.
i know below queries.
table_name=people
select min(some_col_name_which_don't_know) from people
SELECT t.TBL_ID, d.NAME as `schema`, t.TBL_NAME name, t.TBL_TYPE, tp.PARAM_VALUE as description,
p.PKEY_NAME as col_name, p.INTEGER_IDX as col_sort_order,
p.PKEY_TYPE as col_type, p.PKEY_COMMENT as col_description, 1 as "is_partition_col",
IF(t.TBL_TYPE = 'VIRTUAL_VIEW', 1, 0) "is_view"
FROM TBLS t
JOIN DBS d ON t.DB_ID = d.DB_ID
JOIN PARTITION_KEYS p ON t.TBL_ID = p.TBL_ID
WHRER TBL_NAME=people
Can i merge these two queries to one query?
All is there any table like information_schema in hive?

Possible Duplicate: Hive, how do I retrieve all the database's tables columns
You could list the total number of columns in a table using the below command:
hive -e "show columns in <table name>" > table_list.txt
Next step would be to iterate over the table_list.txt file and build a query string with all the field names and its max/min queries.
for column in table_list:
hive -e "select min("+column+") from <table name>" >> min_max_table.txt
Hope this helps.

Related

SQL query : name columns by "columnname.field"

Hello I've written the following query :
SELECT *
FROM [woJob]
LEFT JOIN [woJobTask]
ON [woJob].jobID=[woJobTask].jobID
The query it returns has duplication columns but they are named the same. Is it possible to name column by table.Field. For example, name woJob.jobID and woJobTask.jobID?
My work flow is to use SQL to get the data out of the database and then im using pandas (a python library) to explore the data. Having duplicate column names makes things a little more complicated analyzing the data in python. I want to get all the data out labeled up with column names so I know each column belongs to which table and then analyze the data in Pandas, I can drop any columns in pandas I don't want.
You need to enumerate the columns, and assign alias as needed.
You did not tell what the columns of the tables are, so here is a contrived example, assuming colums jobid, name and value in both tables:
SELECT j.jobid, j.name, j.value, jt.name as jt_name, jt.value as jt_value
FROM [woJob] j
LEFT JOIN [woJobTask] jt ON j.jobid = jt.jobid
Or more simply:
SELECT j.*, jt.name as jt_name, jt.value as jt_value
FROM [woJob] j
LEFT JOIN [woJobTask] jt ON j.jobid = jt.jobid

How to programmatically find all of the file locations of an external table in Hive?

I have some external tables in Hive and want to write a script that will tweak the data in them. Since Hive is the source of truth for what files it reads, I want to get the locations for the partitions from it.
Is there a convenient syntax to get a list of the locations for all partitions for an external table?
The describe command will do it:
DESCRIBE FORMATTED Db_name.table_name
If you have access to hive metastore, you can get details via query under script/program:
E.g. query on hive metastore:
SELECT d.NAME, a.tbl_name, b.location, c.PART_NAME
FROM dbs d
inner join tbls a ON d.DB_ID = a.db_id
inner JOIN sds b ON a.SD_ID = b.sd_id
LEFT outer JOIN partitions c ON a.TBL_ID = c.tbl_id
WHERE a.tbl_name = 'your_table_name' AND d.name = 'your_db_name'

oracle sql column alias

I have 3 tables A, B, c and I want to join those tables. These tables have common columns like, id_no, order_no
and i want to write a query that returns all columns from all 3 tables with column name extension like tabA., tabB., tabC....I don't want to manually specify all column names. In that way i can differentiate the common columns among tables.
select tabA.id_no, tabA.order_no, tabA....., tabB.id_no, tabB.order_no,tabB..., tabC.id_no, tabC.order_no,tabC..
from A tabA, B tabB, C tabC
where tabA.id_no = tabB.id_no
and tabB.id_no = tabC.id_no
could u pls let me know how to achieve this in oracle sql.
Oracle SQL Developer can do that.
Write your * query, put your mouse over the '*'
SQL Developer offers to explode that to the fully qualified column list, click the blue text.
Ta-da.
Don't forget your WHERE clause or ANSI join in the FROM, or your DBA will explain to you what a Cartesian product is.
If your table has foreign keys, SQLDev can generate that as well.
You can do the following:
SELECT tabA.*, tabB.*, tabC.*
FROM a tabA INNER JOIN b tabB
ON tabA.id_no = tabB.id_no
INNER JOIN c tabC
ON tabB.id_no = tabC.id_no;
EDIT
If you want only to get a list of the columns associated with the three tables, and to see which column names are common among the three, then you can try something like the following:
SELECT column_name, COUNT(*), LISTAGG(table_name, ',') WITHIN GROUP ( ORDER BY table_name
FROM all_tab_columns
WHERE owner = '<table_owner>'
AND table_name IN ('A','B','C')
GROUP BY column_name;
N.B. LISTAGG() assumes you're using Oracle 11g or greater; prior to that you can use the undocumented function WM_CONCAT().
Hope this helps.

Merging tables in SQL Server

I have two tables in SQL Server that I want to merge.
The first table is dbo.bac and has a column counter, the second table is dbo.data and also has a column counter.
I want to load the first table and the second table with all their columns that have the same counter value. Thanks for the help...
You need to use JOIN to join both tables.
SELECT *
FROM dbo.bac A
INNER JOIN dbo.data B ON B.counter = A.counter
this should do it, but you need to filter your records as needed.

Hive : join 2 tables and select different columns in single query

I have two tables in hive say Table A and Table B. Basically i want to join both of them and want to select the different column based on some condition in single query .
Table A:
empid;name;sal;dept
1;'X';100;IT
2;'Y';100;IT
3;'Z';100;ADMIN
Table B:
empid;name;address
1;'X';A
2;'Y';B
3;'Z';C
Desired output:
When Dept='IT'
select empid,name,address from Table A join Table B on (A.empid=B.empid)
When Dept='ADMIN'
select empid,address from Table A join Table B on (A.empid=B.empid)
Can someone please help me with the approach?
If you are looking for a single query, the output has to be in the same structure.
I assume you don't want to show the name of the ADMIN due to some security reasons.
If so, you could do the below instead. Admin's names will be shown as 'Name not available'. Hope this helps. If not, please tell us the reason behind your question.
SELECT ta.empid,
CASE WHEN ta.dept = 'IT' THEN ta.name
WHEN ta.dept = 'ADMIN' THEN 'Name not available'
END AS name,
tb.address
FROM tableA ta,
tableB tb
WHERE ta.empid = tb.empid;