During practice i have created tmp table using following query inside hive prompt.
$create temporary table tmp (id int);
Now table is getting successfully created and if i close the hive session table will be deleted automatically by hive which is true according to documentation.
Now, there are other ways to run same query by using following command.
$hive -e 'create temporary table tmp (id int);'
table is getting created successfully but my doubt is this time, why tmp table will not get auto deleted this time. i can still see the tmp table after executing next following command.
$hive -e 'show tables;'
OK
customers
order
product
tmp
Time taken: 0.856 seconds, Fetch: 4 row(s)
Most probably you have already the table with the same name but not temporary.
Steps for reproduce:
I have checked that there is no such table (not temporary) already exists.
hive -e 'use myschema; show tables "tmp";'
--no rows returned
Then I ran your example:
$hive -e 'use myschema; create temporary table tmp (id int);'
OK
Check there is no table:
hive -e 'use myschema; show tables "tmp";'
--no rows returned - it works correctly
Create not temporary table
hive -e 'use myschema; create table tmp (id int);'
--ok
Now there is persistent table:
hive -e 'use myschema; show tables "tmp";'
OK
tmp
Time taken: 0.081 seconds, Fetched: 1 row(s)
Try to create the same temporary table:
hive -e 'use myschema; create temporary table tmp (id int);'
OK
Persistent table remains in the schema. Temporary table was dropped successfully and temporary table was isolated inside session, not visible to other sessions.
Related
I make a temporary table in SQL Server:
create table #stun (name varchar(40),id int,gender varchar(40))
How can I view its definition afterwards?
you can check this way-
SELECT *
INTO #TempTable
FROM table_name -- any table from database
EXEC tempdb..sp_help '#TempTable'
DROP TABLE #TempTable
Solution 1 :
You can query data against it within the current session :
SELECT
*
FROM
#yourtemporarytable;
Sometimes, you may want to create a temporary table that is accessible across connections. In this case, you can use global temporary tables.
Unlike a temporary table, the name of a global temporary table starts with a double hash symbol (##).
CREATE TABLE ##global_temp (
...
);
SELECT
*
FROM
##global_temp
Solution 2 :
Using SSMS, you can find the table in the left pane >> Design >> get the table structure.
I want to delete a partition in Hive with its value being in another table or being created by a function on-the-fly. For example:
ALTER TABLE
table_1
DROP IF EXISTS
PARTITION (dt = FROM_UNIXTIME(UNIX_TIMESTAMP(DATE_SUB(CURRENT_DATE, 63),'yyyy-MM-dd'), 'yyyyMMdd'))
Or something like this:
ALTER TABLE
table_1
DROP IF EXISTS
PARTITION (dt = SELECT date FROM table_2 LIMIT 1))
However, this returns the following error:
cannot recognize input near 'FROM_UNIXTIME' '(' 'UNIX_TIMESTAMP' in constant
If I replace the whole call to FROM_UNIXTIME() with a fixed number, it works fine. Is there a way to do this witouth hard-coding the value of the partition?
In Hive-cli doesn't support that, We need to use Shell script for that
Sample Script:
#!/bin/bash
my_value=$(hive -S -e "select FROM_UNIXTIME(UNIX_TIMESTAMP(DATE_SUB(CURRENT_DATE, 63),'yyyy-MM-dd'), 'yyyyMMdd')")
echo $my_value
hive -S -e "alter table table_1 drop partition (dt = $my_value)"
For more details refer to this and this links about hive variables.
I'm planning to truncate the hive external table which has one partition. So, I have used the following command to truncate the table :
hive> truncate table abc;
But, it is throwing me an error stating : Cannot truncate non-managed table abc.
Can anyone please suggest me out regarding the same ...
Make your table MANAGED first:
ALTER TABLE abc SET TBLPROPERTIES('EXTERNAL'='FALSE');
Then truncate:
truncate table abc;
And finally you can make it external again:
ALTER TABLE abc SET TBLPROPERTIES('EXTERNAL'='TRUE');
By default, TRUNCATE TABLE is supported only on managed tables. Attempting to truncate an external table results in the following error:
Error: org.apache.spark.sql.AnalysisException: Operation not allowed: TRUNCATE TABLE on external tables
Action Required
Change applications. Do not attempt to run TRUNCATE TABLE on an external table.
Alternatively, change applications to alter a table property to set external.table.purge to true to allow truncation of an external table:
ALTER TABLE mytable SET TBLPROPERTIES ('external.table.purge'='true');
There is an even better solution to this, which is basically a one liner.
insert overwrite table table_xyz select * from table_xyz where 1=2;
This code will delete all the files and create a blank file in the external folder location with absolute zero records.
Look at https://issues.apache.org/jira/browse/HIVE-4367 : use
truncate table my_ext_table force;
I have a hive table in XYZ db named ABC.
When I run describe formatted XYZ.ABC; from hue, I get the following..
that is
Table Type: MANAGED_TABLE
Table Parameters: EXTERNAL True
So is this actually an external or a managed/internal hive table?
This is treated as an EXTERNAL table. Dropping table will keep the underlying HDFS data. The table type is being shown as MANAGED_TABLE since the parameter EXTERNAL is set to True, instead of TRUE.
To fix this metadata, you can run this query:
hive> ALTER TABLE XYZ.ABC SET TBLPROPERTIES('EXTERNAL'='TRUE');
Some details:
The table XYZ.ABC must have been created via this kind of query:
hive> CREATE TABLE XYZ.ABC
<additional table definition details>
TBLPROPERTIES (
'EXTERNAL'='True');
Describing this table will give:
hive> desc formatted XYZ.ABC;
:
Location: hdfs://<location_of_data>
Table Type: MANAGED_TABLE
:
Table Parameters:
EXTERNAL True
Dropping this table will keep the data referenced in Location in describe output.
hive> drop table XYZ.ABC;
# does not drop table data in HDFS
The Table Type still shows as MANAGED_TABLE which is confusing.
Making the value for EXTERNAL as TRUE will fix this.
hive> ALTER TABLE XYZ.ABC SET TBLPROPERTIES('EXTERNAL'='TRUE');
Now, doing a describe will show it as expected:
hive> desc formatted XYZ.ABC;
:
Location: hdfs://<location_of_data>
Table Type: EXTERNAL_TABLE
:
Table Parameters:
EXTERNAL TRUE
Example -
Lets create a sample MANAGED table,
CREATE TABLE TEST_TBL(abc int, xyz string);
INSERT INTO TABLE test_tbl values(1, 'abc'),(2, 'xyz');
DESCRIBE FORMATTED test_tbl;
Changing type to EXTERNAL (in the wrong way using True, instead of TRUE):
ALTER TABLE test_tbl SET TBLPROPERTIES('EXTERNAL'='True');
This gives,
Now lets DROP the table,
DROP TABLE test_tbl;
The result:
Table is dropped but data on HDFS isn't. Showing correct external table behavior!
If we re-create the table we can see data exists:
CREATE TABLE test_tbl(abc int, xyz string);
SELECT * FROM test_tbl;
Result:
The describe shows it wrongly as MANAGED TABLE along with EXTERNAL True because of:
.equals check in the meta
Hive Issue JIRA: HIVE-20057
Proposed fix: Use case insensitive equals
Is there an easy way to copy an existing table structure to a new one?
(dont need the data, only the structure -> like id INTEGER, name varchar(20) ...)
Thx
You could use a command like this:
CREATE TABLE copied AS SELECT * FROM mytable WHERE 0
but due to SQLite's dynamic typing, most type information would be lost.
If you need just a table that behaves like the original, i.e., has the same number and names of columns, and can store the same values, this is enough.
If you really need the type information exactly like the original, you can read the original SQL CREATE TABLE statement from the sqlite_master table, like this:
SELECT sql FROM sqlite_master WHERE type='table' AND name='mytable'
SQLite cannot clone table with PK, defaults and indices.
Hacking by another tool is necessary.
In shell, replace the table name by sed.
sqlite3 dbfile '.schema oldtable' | sed '1s/oldtable/newtable/' | sqlite3 dbfile
And you can check new table.
sqlite3 dbfile '.schema newtable'
Primary key, defaults and indices will be reserved.
I hope this command can help you.
sqlite> .schema
CREATE TABLE [About](
[id],
[name],
[value]);
.schema command will give you structure of About-table how it could be made by programming SQLite interpreter by hand, typing in commands.
Paste in and execute, the CREATE block giving the table new name:
sqlite> CREATE TABLE [AboutToo](
[id],
[name],
[value]);
.tables command now will show you have two tables, old and new, "copied".
sqlite> .tables
About AboutToo
p.s. sqlite> is command prompt you get in console after launching SQLite.exe interpreter. To get it go to www.sqlite.org
Just for the record - This worked for me:
CREATE TABLE mytable (
contact_id INTEGER PRIMARY KEY,
first_name TEXT NOT NULL,
last_name TEXT NOT NULL,
email TEXT NOT NULL UNIQUE,
phone TEXT NOT NULL UNIQUE
);
-- Two variations
INSERT INTO mytable VALUES ( 1, "Donald", "Duck", "noone#nowhere.com", "1234");
INSERT INTO mytable ( contact_id,first_name,last_name,email,phone ) VALUES ( 2, "Daisy", "Duck", "daisy#nowhere.com", "45678");
.output copied.sql
-- Add new table name
.print CREATE TABLE copied (
-- Comment out first line from SQL
SELECT "-- " || sql FROM sqlite_master WHERE type='table';
.output
.read copied.sql
.schema
select * from copied;
Beware that this only works if schema is wrapped after CREATE TABLE mytable (.
Otherwise you'll need some string replacement using .system
Yes by using the SQLiteStudio you can use the last icon in the structure table called create similar table from any existing table.
I would prefer :
> sqlite3 <db_file>
sqlite3 > .output <output_file>
sqlite3 > .dump <table_name>
The line above generates the dump of table that includes DDL and DML statement.
Make changes in this file, i.e. find and replace the table name with new table name
Also, replace "CREATE TRIGGER " with "CREATE TRIGGER <NEWTABLE>_" , this will replace existing triggers with trigger names with a new table name on it. That will make it unique and will not cause conflicts with existing triggers. Once all schema changes are implemented, read it back into database using .read
sqlite3 > .read output_file
This can be scripted in shell file using shell commands like :
echo ".dump <table>" | sqlite3 <db_file> > <table_file>
sed -i.bak "s/\b<table_name>\b/<new_table_name>/g" <table_file>
sed -i.bak "s/\bCREATE TRIGGER \b/CREATE TRIGGER <new_table_name_>/g" <table_file>
echo ".read <table_file>" | sqlite3 <db_file>
rm <table_name>.bak
For example :
If you have table T and new table is TClone in db file D with file F to be created : then
echo ".dump T" | sqlite3 D.sqlite > F
sed -i.bak "s/\bT\b/TClone/g" F
sed -i.bak "s/\bCREATE TRIGGER \b/CREATE TRIGGER TClone_>/g" F
echo ".read F" | sqlite3 D.sqlite
rm T.bak
Finally, you can generalize this script by creating a parameterized version where you can pass source_table, destination_table , db_file as parameters that can be used to clone any table.
I tested this and it works.
Testing :
sqlite3 <database_file>
sqlite3 > select * from <new_table>;
should give you same results as original table. and
sqlite3 > .schema <new_table>
should have same schema as that of original table with a new name.