I need to prepare the script to increase the partition range if the partition is going to get finished in next 2-3 months. How to find the existing table partition and we can edit to existing table or we need to create a new script.
Appreciate response
How to find the existing table partition
You could either generate the table DDL using DBMS_METADATA package to get the complete table DDL.
Or, query the user_tab_partitions view to get the table partition information.
To add new partitions, you need to use ADD PARTITION clause:
ALTER TABLE <table_name>
ADD PARTITION <new_partition>
VALUES (<new_value>)
TABLESPACE <tablespace_name>;
Related
I have around 600 tables. How to drop list partition from oracle table without dropping table structure.
ALTER TABLE table DROP PARTITION partition_name;
or
DELETE FROM table PARTITION(partition_name);
I understand that you want to drop them not truncate, otherwise, you must replace DROP by TRUNCATE.
If you want to remove all partitions automatically.
Create a CURSOR, sharing information of ALL_TAB_PARTITIONS.
After the CURSOR create a LOOP to move into the table partitions.
If that can helps you, ask me and I give you more details.
I am trying to change the already existing partition column to another column.
The current workflow I'm using:
Backup the existing data
Create a new table with new partition column
Reload the data into new partitions
My problem:
Since there is huge data in our existing partition tables, this way will be costly
Is there a way we can do Alter table and change partition column name to another?
You can not avoid 1-time cost of scanning the table as you can see from the error message generated from this CREATE OR REPLACE DML command
#standardSQL
CREATE OR REPLACE TABLE `project.dataset.table`
PARTITION BY DATE(ts)
AS
SELECT * FROM `project.dataset.table`
Cannot replace a table with a different partitioning spec. Instead, DROP the table, and then recreate it. New partitioning spec is interval(type:day,field:ts) and existing spec is none
What you can do to save cost is use the WHERE command to limit the number of the partition you move from existing table to the new table
CREATE TABLE project.mydataset.newPartitionTable
PARTITION BY date
OPTIONS (
partition_expiration_days=365,
description="Table with a new partition"
) AS
SELECT * from `project.dataset.table` WHERE
PARTITIONTIME >= '2019-01-23 00:00:00'
AND _PARTITIONTIME <= '2019-01-23 00:00:00'
You can consider for example not to move your Long-term storage which is data you haven't access for the last 90 days (see this link for more details)
If you want to keep your original table name you can drop/create it with the new partition field, after the copy, and use the copy option from webUI which will be free of charge
I want to use the hive dynamic partitioning to overwrite a partitioned table "page_view":
INSERT OVERWRITE TABLE page_view PARTITION(date)
SELECT pvs.viewTime FROM page_view_stg pvs
My question is : If the table "page_view_stg" only has the data of "date=2017-01-01", but the dest table "page_view" has a partition "date=2017-01-02". So after running this query, will the partition "date=2017-01-02" get dropped or not? If not, how should I handle this case using dynamic partitioning?
Thanks
Query with dynamic partitioning will overwrite only partitions existing in the source dataset. In your case partition "date=2017-01-02" will remain unchanged if the the source table does not contain such date. If you want to drop it, the fastest method is to execute alter table drop partition statement because this is the metadata operation. You select partitions from target table which do not exist in the source and generate drop statements using shell. Or insert into new table, drop old target, then rename.
I am not sure about altering a table to create a new partition as I am afraid I will lose data. If a table in an Oracle SQL DB is already partitioned but I am adding a new partition, will the existing data in the table be deleted?
No you dont lose the data
You can create list partition and expand it on default partition
for example if your partion is date:
alter table your_table split partition PDEFAULT values(TO_DATE('20161206','yyyymmdd')) into ( partition P20161206,partition PDEFAULT)
The only ALTER TABLE partitioning commands that can destroy data are DROP and TRUNCATE.
The EXCHANGE partition command can move data from a table partition to a different table, and vice-versa.
ADD, MOVE, COALESCE, RENAME, SPLIT, and MERGE do not change the table's data, although COALESCE, SPLIT, and MERGE can change the partition or subpartition in which data is stored.
How to apply Partition on hive table which is already partitioned. I am not able to fetch the partitioned data into the folder after the data is loaded.
1st rule of partitioning in hive is that the partitionioning column should be the last column in the data. since the data is already partitioned lets say we are partitioning data on gender M/F there will be two directories gender=M and gender=F be created inside each of the directories respective gender data will be available and last column again in this data will be gender.
If you want to partiton data again on partitioned table use insert into select and make sure last column you use is the partition column you want to on the partitioned data.
Did you add a partition manually with the Hdfs command ? In that case metastore will not keep track of partitions being added unless you specify " alter table add partition "...
try this
MSCK REPAIR TABLE table_name;
If that is not the case , then try to drop partitions and create the partitions again . Use alter table command to do this. but you will lose the data . and your partitioning column value should be mentioned as last column in case if you are doing a dynamic partition insert.