I would like to create an external table in hive from a view and change the default location:
CREATE external TABLE market.resultats like v_ca_mag
LOCATION '/user/training/market/db/resultats';
The table is created and is external but the location is the default one /user/hive/warehouse/market.db/resultats.
Why is the location not taken into account?
I am using cdh 5.4.
Probably it's a bug please open a jira to account for this issue.
As a work around once you are done with creating external table then execute alter table statement to change the location of your newly created table to the desired location.
hive> CREATE external TABLE market.resultats like v_ca_mag;
hive> alter table market.resultats set location 'hdfs://nnaddress/user/training/market/db/resultats';
Related
Based on: https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_SCHEMA.html
I have my schema declared in the following way:
create external schema spectrum_schema
from data catalog
database 'spectrum_db'
iam_role 'arn:aws:iam::123456789012:role/myRedshiftRole,arn:aws:iam::123456789012:role/myS3Role'
catalog_role 'arn:aws:iam::123456789012:role/myAthenaRole'
create external database if not exists;
I decided I want to change my IAM Role for this schema and having something like arn:aws:iam::123456789012:role/moreBeautifulRole instead.
Is it possible to change it? Ideally I'd like to change it with something like
ALTER SCHEMA spectrum_schema IAM_ROLE 'arn:aws:iam::123456789012:role/moreBeautifulRole'
without destroying the schema. Please share the available options.
After confirming with AWS Support, at the date posting this answer it is not supported to edit the IAM Role in an existing External Schema.
Available options I can think are:
Create a new External Schema with the moreBeautifulRole
Add Policy to myS3Role with the required permission - or edit the already assigned one
Drop and recreate external schema and depending objects (including external tables). Using DBT is not too bad as approach
I've created an external schema in my redshift database using the script below:
create external schema exampleschema
from data catalog database 'examplesource'
iam_role 'arn:aws:iam::627xxxxx:role/dxxxx'
region 'us-west-2'
CREATE EXTERNAL DATABASE IF NOT EXISTS;
I'm now trying to create a view in that exampleschema schema using the script below, but I seem to only be able to create views in the "public" schema. How do I create a view in the exampleschema schema?
create view vw_ticket as select * from exampleschema.ticket
with no schema binding;
You need to specify the schema in the CREATE statement i.e. create view exampleschema.vw_ticket ...
We want to migrate tables to Spectrum, which requires defining an external schema
create external schema spectrum
from data catalog
database 'spectrumdb'
iam_role 'my_iam_role'
create external database if not exists;
I created an external table in Redshift like this:
create external table spectrum.my_table(
id bigint,
accountId bigint,
state varchar(65535),
) stored as parquet
location 's3://some_bucket/my_table_files';
Is it possible to alias the table such that when querying it, I can call it my_table_alias instead of spectrum.my_table? Basically, we want to make the change to external tables opaque to clients of our Redshift instance (this means we can't change the table names). Thanks so much for your help!
Redshift does not have aliases, your best option is to create a view.
You need to use WITH NO SCHEMA BINDING option while creating the view since the view is on an external table.
If you like to not specify schema names or you have a requirement like this create the view(s) in public schema or set the users default schema to the schema where the views are
alter user .. set search_path to ..
Additional benefits of using a view to access an external table are, you can
rename columns to be more user friendly
add or remove columns with view definition
change data types and/or date/time formats
you will have the ability to change name/structure of the external table without effecting user access
Let me know if this answers your question.
I have created a managed table.now i want to change the default location of the table to non hdfs location
i tried the below command but not working
ALTER TABLE my_table SET LOCATION "/path/to/file"
A piculiarity I noticed.
When I try
create table dbname.table_name as select
I get Error creating temporary folder on: hdfs://nameservice1/apps/hive/warehouse. Error encountered near token 'TOK_TMP_FILE'
But If I first do
use dbname;
and then
create table table_name as select
It works. Why is that?
To create table in any database user need to have write permission on current database and database in which table is being created.
I.e. while running create table dbname.table_name as select statement , you need to have write permission on current database as well.
This is known issue reported in jira HIVE-11427.