condition on hql statements - hive

I have som hql statements running every day like
drop table if exists table1;
create table table1 as
select ....
from A;
But some times create table fails and then my original table is¨deleted without a new one is created
I would like to do this a smarter way
like this pseudo code
if ( create table table1_tmp as
select ....
from A; )
then ( drop table table1 ;
rename table1_tmp to table1 ;
)
Is it possible to do that in hql or do i have to at
pig, spark, python etc.

INSERT OVERWRITE is the best fit for what you are trying to do.
You might want to keep an eye on your HDFS Trash folder because I think the underlying files always move to /user/<username>/.Trash when you perform an INSERT OVERWRITE

Related

Select All From Table Except Specific Column/s Without Temporary Table

I've been searching for a way to get all data from a certain table except for a certain column.
Is there a way to do this without creating a temporary table? I find this although creative, inefficient.
I found this but again it's creating a temporary table. I guess temporary table is okay as long as it not an actual table that I can access.
SELECT * INTO #TempTable
FROM TABLE_NAME
ALTER TABLE #TempTable
DROP COLUMN COLUMN_NAME
SELECT * FROM #TempTable
DROP TABLE TempTable;
Again, my goal is to avoid creating a temporary table where I would later delete it to make it seem 'data-like'. Sorry I'm not quite sure how to put it into words.
Just select the columns you do want . . .
select . . .
from t;
You can create a view with the columns you want.

Create external table with select from other table

I am using HDInsight and need to delete my clusters when I am finished running queries. However, I need the data I gather to survive for another day. I am working on queries that would create calculated columns from table1 and insert them into table2. First I wanted a simple test to copy the rows. Can you create an external table from a select statement?
drop table if exists table2;
create external table table2 as
select *
from table1
STORED AS TEXTFILE LOCATION 'wasb://{container name}#{storage name}.blob.core.windows.net/';
yes but you have to seperate it into two commands. First create the external table then fill it.
create external table table2(attribute STRING)
STORED AS TEXTFILE
LOCATION 'table2';
INSERT OVERWRITE TABLE table2 Select * from table1;
The schema of table2 has to be the same as the select query, in this example it consists only of one string attribute.
I know this is too stale question but here is the solution.
CREATE EXTERNAL TABLE table2
STORED AS textfile
LOCATION wasb://....
AS SELECT * FROM table1
Since create external table with "as select" clause is not supported in Hive, first we need to create external table with complete DDL command and then load the data into the table. Please go through this for different data format supports.
create external table table_ext(col1 typ1,...)
STORED AS ORC
LOCATION 'table2'; // optional if not provided then default location is used
INSERT OVERWRITE TABLE table_ext Select * from table1;
make sure table_ext has same DDL as table1.

Bulk insert between tables in same database

Fastest way to do insert in a table A with same structure of a table B:
INSERT INTO A
SELECT * FROM B
I can't rename table because in table A there are indexes that when query is executed it's impossible to know without read catalog and create a dynamic script to modify table B to be equal to table A. Also, on table A there are on it some views than it's difficult to delete.
I can't use COPY command to make a COPY TO and next a COPY FROM because database user haven't permission on file and directory.
Is there a way to do a COPY A FROM (SELECT * FROM B) ?
PostgreSQL version 9.2.
Your question states the correct answer:
INSERT INTO A
SELECT * FROM B;
That's all you need.
Now you can also do this if A doesn't exist yet:
CREATE TABLE A (LIKE B INCLUDING ALL);
INSERT INTO A SELECT * FROM B;

"select * into table" Will it work for inserting data into existing table

I am trying to insert data from one of my existing table into another existing table.
Is it possible to insert data into any existing table using select * into query.
I think it can be done using union but in that case i need to record all data of my existing table into temporary table, then drop that table and finally than apply union to insert all records into same table
eg.
select * into #tblExisting from tblExisting
drop table tblExisting
select * into tblExisting from #tblExisting union tblActualData
Here tblExisting is the table where I actually want to store all data
tblActualData is the table from where data is to be appended to tblExisting.
Is it right method.
Do we have some other alternative ?
You should try
INSERT INTO ExistingTable (Columns,..)
SELECT Columns,...
FROM OtherTable
Have a look at INSERT
and SQL SERVER – Insert Data From One Table to Another Table – INSERT INTO SELECT – SELECT INTO TABLE
No, you cannot use SELECT INTO to insert data into an existing table.
The documentation makes this very clear:
SELECT…INTO creates a new table in the default filegroup and inserts the resulting rows from the query into it.
You generally want to avoid using SELECT INTO in production because it gives you very little control over how the table is created, and can lead to all sorts of nasty locking and other performance problems. You should create schemas explicitly and use INSERT - even for temporary tables.
#Ryan Chase
Can you do this by selecting all columns using *?
Yes!
INSERT INTO yourtable2
SELECT * FROM yourtable1
Update from CTE? http://www.sqlservercentral.com/Forums/Topic629743-338-1.aspx

How can I create a copy of an Oracle table without copying the data?

I know the statement:
create table xyz_new as select * from xyz;
Which copies the structure and the data, but what if I just want the structure?
Just use a where clause that won't select any rows:
create table xyz_new as select * from xyz where 1=0;
Limitations
The following things will not be copied to the new table:
sequences
triggers
indexes
some constraints may not be copied
materialized view logs
This also does not handle partitions
I used the method that you accepted a lot, but as someone pointed out it doesn't duplicate constraints (except for NOT NULL, I think).
A more advanced method if you want to duplicate the full structure is:
SET LONG 5000
SELECT dbms_metadata.get_ddl( 'TABLE', 'MY_TABLE_NAME' ) FROM DUAL;
This will give you the full create statement text which you can modify as you wish for creating the new table. You would have to change the names of the table and all constraints of course.
(You could also do this in older versions using EXP/IMP, but it's much easier now.)
Edited to add
If the table you are after is in a different schema:
SELECT dbms_metadata.get_ddl( 'TABLE', 'MY_TABLE_NAME', 'OTHER_SCHEMA_NAME' ) FROM DUAL;
create table xyz_new as select * from xyz where rownum = -1;
To avoid iterate again and again and insert nothing based on the condition where 1=2
Using sql developer select the table and click on the DDL tab
You can use that code to create a new table with no data when you run it in a sql worksheet
sqldeveloper is a free to use app from oracle.
If the table has sequences or triggers the ddl will sometimes generate those for you too. You just have to be careful what order you make them in and know when to turn the triggers on or off.
You can do this
Create table New_table as select * from Old_table where 1=2 ;
but be careful
The table you create does not have any Index, PK and so on like the old_table.
DECLARE
l_ddl VARCHAR2 (32767);
BEGIN
l_ddl := REPLACE (
REPLACE (
DBMS_LOB.SUBSTR (DBMS_METADATA.get_ddl ('TABLE', 'ACTIVITY_LOG', 'OLDSCHEMA'))
, q'["OLDSCHEMA"]'
, q'["NEWSCHEMA"]'
)
, q'["OLDTABLSPACE"]'
, q'["NEWTABLESPACE"]'
);
EXECUTE IMMEDIATE l_ddl;
END;
Simply write a query like:
create table new_table as select * from old_table where 1=2;
where new_table is the name of the new table that you want to create and old_table is the name of the existing table whose structure you want to copy, this will copy only structure.
SELECT * INTO newtable
FROM oldtable
WHERE 1 = 0;
Create a new, empty table using the schema of another. Just add a WHERE clause that causes the query to return no data:
WHERE 1 = 0 or similar false conditions work, but I dislike how they look. Marginally cleaner code for Oracle 12c+ IMHO is
CREATE TABLE bar AS
SELECT *
FROM foo
FETCH FIRST 0 ROWS ONLY;
Same limitations apply: only column definitions and their nullability are copied into a new table.
If one needs to create a table (with an empty structure) just to EXCHANGE PARTITION, it is best to use the "..FOR EXCHANGE.." clause. It's available only from Oracle version 12.2 onwards though.
CREATE TABLE t1_temp FOR EXCHANGE WITH TABLE t1;
This addresses 'ORA-14097' during the 'exchange partition' seamlessly if table structures are not exactly copied by normal CTAS operation. I have seen Oracle missing some of the "DEFAULT" column and "HIDDEN" columns definitions from the original table.
ORA-14097: column type or size mismatch in ALTER TABLE EXCHANGE
PARTITION
See this for further read...
you can also do a
create table abc_new as select * from abc;
then truncate the table abc_new. Hope this will suffice your requirement.
Using pl/sql developer you can right click on the table_name either in the sql workspace or in the object explorer, than click on "view" and than click "view sql" which generates the sql script to create the table along with all the constraints, indexes, partitions etc..
Next you run the script using the new_table_name
copy without table data
create table <target_table> as select * from <source_table> where 1=2;
copy with table data
create table <target_table> as select * from <source_table>;
In other way you can get ddl of table creation from command listed below, and execute the creation.
SELECT DBMS_METADATA.GET_DDL('TYPE','OBJECT_NAME','DATA_BASE_USER') TEXT FROM DUAL
TYPE is TABLE,PROCEDURE etc.
With this command you can get majority of ddl from database objects.
Create table target_table
As
Select *
from source_table
where 1=2;
Source_table is the table u wanna copy the structure of.
create table xyz_new as select * from xyz;
-- This will create table and copy all data.
delete from xyz_new;
-- This will have same table structure but all data copied will be deleted.
If you want to overcome the limitations specified by answer:
How can I create a copy of an Oracle table without copying the data?
The task above can be completed in two simple steps.
STEP 1:
CREATE table new_table_name AS(Select * from old_table_name);
The query above creates a duplicate of a table (with contents as well).
To get the structure, delete the contents of the table using.
STEP 2:
DELETE * FROM new_table_name.
Hope this solves your problem. And thanks to the earlier posts. Gave me a lot of insight.