I'm having an odd problem with impala when I'm trying to create a table via "create table abc as (select)". Even if it creates the table the query returns 'table not found'
Anyone knows why this can happen?
I0126 04:01:36.553565 25748 coordinator.cc:584] Finalizing query: 86456b134c56d5e6:9f58d67400000000
I0126 04:01:36.643385 25748 coordinator.cc:606] Removing staging directory: hdfs://nameserviceHDFS/user/hive/warehouse/abc/_impala_insert_staging/86456b134c56d5e6_9f58d67400000000/
I0126 04:01:36.658812 25748 coordinator.cc:488] ExecState: query id=86456b134c56d5e6:9f58d67400000000 execution completed
I0126 04:01:36.658973 25748 coordinator.cc:863] Release admission control resources for query_id=86456b134c56d5e6:9f58d67400000000
I0126 04:01:36.673594 25748 client-request-state.cc:1100] Updating metastore with 1 altered partitions ()
I0126 04:01:36.673655 25748 client-request-state.cc:1115] Executing FinalizeDml() using CatalogService
E0126 04:01:36.677054 25748 client-request-state.cc:1121] ERROR Finalizing DML: TableNotFoundException: Table not found: abc
Try adding the schema name before table name and also share the full query here.
Related
I have an existing External Table called for example YYYYYY that contains n number of columns and this table is loaded daily with partitioned column as extract_date.
We got a request from business to add few more columns in the existing table. To implement this we have done following things.
DROP existing partitions from Hive
alter table xxxx.yyyyyy add columns (
`c10` string COMMENT '',
`b` string COMMENT '',
`c11` string COMMENT '',
`c12` string COMMENT '',
`c13` string COMMENT '',
`c14` string COMMENT '',
`c15` string COMMENT '') CASCADE;
alter table xxxx.yyyyyyy change `c8` `c8` string COMMENT '' after `c7` CASCADE;
After I did the above 2 steps, I went to Hive and did MSCK REPAIR TABLE xxxx.yyyyyy;
Partitions added (there are partitions from 2018) along with my new fields.
Before the changes I was able to query the data both from Impala and Hive, but after executing ALTER commands, I am getting the error as below.
> select * from xxxx.yyyyyyy where extract_date like '2019%';
Query: select * from XXXXX.YYYYYYY where extract_date like '2019%'
Query submitted at: 2020-05-09 11:57:10 (Coordinator: ' xxxx.yyyyyyy .c9'. Column type: STRING, Parquet schema:
optional fixed_len_byte_array a_auth [i:12 d:1 r:0]
Whereas in Hive I am able to browse the data with no issues. So I have an issue only in Impala.
Troubleshooting steps:
Created new table without additional columns and pointed the external path as new and copied the previously created partitions to new path.
MSCK REPAIR TABLE TABLE NAME;
Both in Impala and Hive a select query is working.
Added additional fields to the newly created table with ALTER commands then did the following things
MSCK REPAIR TABLE TABLE NAME;
In Impala :
REFRESH TABLE TABLE NAME;
INVALIDATE METADATA TABLE NAME;
This time in Hive select query worked but in Impala got the above mentioned error.
Can some one guide me why this is happening and how to fix this issue.
Impala Shell v2.12.0-cdh5.16.2
I am trying to build a managed table (which orc formatted ,bucketed and table properties is set to true for transnational )on which i can run the update/Insert Statement In hive .
I am running this whole setup on AWS EMR and the Hive version is 2.4.3 the default directory store the data is S3.
I am able to populate the table from another external table .
However am getting select count(*) as zero and no output for select *
i dropped the table and recreated the table and repopulated the data .
The ANALYZE TABLE TABLE-NAME COMPUTE STATISTICS gives proper output .
I'm trying to create the table in the redshift DB using the SQL workbench using the following command.
CREATE TABLE test_table (columnone varchar, columntwo varchar,
columnthree varchar,columnfour varchar,columnfive varchar,columnsix
varchar,columnseven varchar,columneight varchar);
It is created successfully.
I verified the table is created from the following line. and saw the table is present and table_type is BASE TABLE
SELECT * FROM information_schema.tables WHERE table_schema = 'public';
I was able to insert the data into the table.
INSERT INTO test_table VALUES ('123456', '123456',
'123456','123456', '123456','123456','123456','123456');
Row insertion is done successfully
After that I performed the query
Select * from test_table
and I'm able to see the results as well for this
But after 5 to 10 mins table is deleted.
I checked that table is not available in the DB by executing the following line
SELECT * FROM information_schema.tables WHERE table_schema = 'public';
I googled it and was not able to find any. I'm new to redshift and SQL was there any mistake I did while creating the table. The tables that are created in this DB by some other people earlier are available still. I also checked with the people who created tables earlier are not able to figure out what is the issue
As per Jon Scott comment, I tried using commit command and then I able to see the table after a long time as well. The table is not deleted now.
commit;
This is the command I executed after the creation of the table immediately.
Also, have gone through this we have to execute this commit after performing any operation on the table else the data will not change.
Also, I have seen another option after exploring the SQL workbench connect window. There is a checkbox for Autocommit. if we select the checkbox then we need not execute the commit command everytime
i have an external partitioned table named employee with partition(year,month,day), everyday a new file come and seat at the particular day location call for today's date it will be at 2016/10/13.
TABLE SCHEMA:
create External table employee(EMPID Int,FirstName String,.....)
partitioned by (year string,month string,day string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/.../emp';
so everyday we need to run command which is working fine as
ALTER TABLE employee ADD IF NOT EXISTS PARTITION (year=2016,month=10,day=14) LOCATION '/.../emp/2016/10/14';
but once we are trying with below command because we don't want to execute the above alter table command manually, it throws below Error
hive> MSCK REPAIR TABLE employee;
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Note:
hive> MSCK TABLE employee; //this show me that a partition has not added in the table
OK
Partitions not in metastore: employee:2016/10/14
Time taken: 1.066 seconds, Fetched: 1 row(s)
please help me as i stuck with this. do we have any workaround for this type of situation?
I got a workaround solution for my problem which is, if the table static partition name is like 'year=2016/month=10/day=13' then we can use below command and it is working...
set hive.msck.path.validation=ignore;
MSCK REPAIR TABLE table_name;
I'm having this odd problem since yesterday. I've tried several options and I actually reinstalled ORACLE and the DB itself.
Here's the problem: I have this table that is somekind of zombie. Here are the symptoms:
SELECT TABLE_NAME FROM USER_TABLES WHERE TABLE_NAME='MYTABLE'
Returns a record, meaning that the table exists.
SELECT COLUMN_NAME FROM USER_TAB_COLUMNS WHERE TABLE_NAME = 'MYTABLE'
Returns all the columns of MYTABLE. So far so good, the table exists.
SELECT * FROM MYTABLE
Returns ORA-00942: table or view does not exist.
At this point I'm quite confused: the table seems to exist on the USERTABLES but I cannot SELECT over it?
CREATE TABLE MYTABLE (Foo NUMBER) TABLESPACE MYTABLESPACE
Returns:
ORA-00604: error occurred at recursive SQL level 1
ORA-00001: unique constraint (SYS.I_OBJ2) violated
I do not understand this error. But the best is yet to come.
SELECT * FROM MYTABLE
Surprisingly, the above query (an exact copy of the 3rd query) returns several records now!
Moreover, I noticed that the column Foo is not present: the table I now see is my initial table that had other columns.
DROP TABLE MYTABLE
I now try to drop the table and I get the following errors:
ORA-00604: error occurred at recursive SQL level 1
ORA-00942: table or view does not exist
ORA-06512: at line 19
SELECT * FROM MYTABLE
More confused than ever, I try the above query and, surprise surprise, the table no longer exists.
I don't undestand this: the table is on USERTABLES but I cannot SELECT over it, however, if I create a new table with the same name, I get an error but now I can SELECT over the previous version of that table with several records.
Any thoughts ? I really need your help :(
EDIT - I checked now: I'm unable to drop ANY table. This might just be a new symptom.
Solution
The problem was that MDSYS.SDO_GEOR_SYSDATA_TABLE table was missing and a drop event trigger was trying to access it, generating the error. The solution was restoring that table.
If have privileges, try this query:
SELECT *
FROM dba_objects
WHERE object_name = 'MYTABLE';
And see what objects exist with that name. It might point you in the right direction.
You didn't qualify the schema names when trying to select and drop. The CURRENT_SCHEMA of your session may be different form the log-on user. Check by trying
select SYS_CONTEXT('USERENV', 'CURRENT_SCHEMA') from dual;
Instead of describing what the output was, could you please copy/paste the complete output for us?
Lastly, can you exclude that someone messed up the dictionary? You know, SYSDBA can do anything....