So I got the command for adding a comment/field definition for a column of a Hive table. I believe the following statement (maybe with slight modifications) does the purpose:
alter table abc CHANGE x x bigint [COMMENT "hello"];
However, I wish to add a "hyperlink" comment instead of just text so that I can show the field definitions over a web interface
Is there a way to do this?
Related
I cannot imagine there is such issue in BigQuery:
le's say if I drop a column using below command in BQ console for User table:
Alter table User drop column name -> successful
I am aware this column is preserved for 7 day(for time travel duration purpose).
But I cannot add any column anymore by running below command in BQ console:
ALTER TABLE User add column first_name STRING
Cause it will give an error like below even though the two columns have totally different naming:
Column name was recently deleted in the table User. Deleted column name is reserved for up to the time travel duration, use a different column name instead.
The above error is same as when I try to drop the same column again even with IF EXISTS:
Alter table User drop IF EXISTS column name
My question:
Why is this issue will happen? After 7 days, Can I add new columns as usual?
I have recreated your issue wherein I dropped a column named employee_like_2 and then tried to add a new column named new_column.
There is already a created bug for this issue. You may click on +1 to bring more attention on the issue and STAR the issue so that you can be notified for updates.
For the meantime, a possible workaround is to manually add columns through BigQuery UI.
Apart from the solution using UI suggested #Scott B, we can also do it using bq command:
Basically bq query --use_legacy_sql=false 'ALTER TABLE User add column first_name STRING' will fail to add a column. But I found a workaround
I can run bq update command instead like below:
bq show --schema --format=prettyjson DATASET.User > user_schema.json
Add a new column I want into file user_schema.json
bp update DATASET.User user_schema.json
So this basically means it is a 100% bug in BigQuery SQL command
I got a table with more than 2000 columns, but i need to drop only one column
There is any efficient way to make it in impala?
I'm trying this way:
alter table proceso.prueba drop subsegm
select * from proceso.prueba
but, i got this error in the "select":
'hdfs://nameservice1/user/hive/warehouse/proceso.db/prueba/914a7dd4a8462ff1-
860a4c1d00000011_978927331_data.1.parq' has an incompatible Parquet schema
for column 'proceso.prueba.nfi_meses_antiguedad_bco'. Column type: INT,
Parquet schema: optional byte_array subsegm [i:4 d:1 r:0]
What am i doing wrong?
Thanks for help
This error occurs when the schema defined for the table (datatype of columns in this case) is conflicting with the schema present in the corresponding Parquet files of the table.
To fix this, you can check the below,
Perform a SHOW CREATE TABLE proceso.prueba and list the columns.
Run the command parquet-tools meta hdfs://nameservice1/user/hive/warehouse/proceso.db/prueba/914a7dd4a8462ff1-
860a4c1d00000011_978927331_data.1.parq to see the metadata that includes the column details.
Compare the results from #1, #2 to see if the number of columns are correct and see if the datatype for the column subsegm (in #2 result) is equivalent to what it is supposed to have (in #1 result).
Modifying the table datatype to the correct value, structure (if needed) should help you fix this issue.
Hope that helps!
So for my previous homework, we were asked to import a csv file with no columns names to impala, where we explicitly give the name and type of each column while creating the table. However, now we have a csv file but with column names given, in this case, do we still need to write down the name and type of it even it is provided in the data?
Yes, you still have to create an external table and define the column names and types. But you have to pass the following option right at the end of the create table statement
tblproperties ("skip.header.line.count"="1");
-- Once the table property is set, queries skip the specified number of lines
-- at the beginning of each text data file. Therefore, all the files in the table
-- should follow the same convention for header lines.
I have Access as my back end. Student is the database name and its path is c:\Project\student.mdb
And am having few tables in it. When i click a command button in vb.net a new table must be created in the path mentioned. And it should have the fields I declare.
What s the query I need to use ? Is it possible to do like that ?
Yes, it's possible, you can check out the syntax for the create table command.
However, I have to say that creating tables dynamically suggests a bad database design. The database layout should normally remain the same whatever data you put in the database.
If you create tables dynamically, that means that you have data in the table names, but data should go inside the tables, not in table names (or even field names).
Yes it's possible . Just pass the create table command as query and the table name as variable , which you want to add dynamically. for reference of create table command
Create table command
I have a script that parses xml file to generate SQLite table automatically. And, the simplified command is as follows.
Table string CREATE TABLE IF NOT EXISTS benchmark (id integer primary key autoincrement,Version float, CompilationParameters_Family text, CompilationParameters_XilinxVersion text, CompilationParameters_Device text, CompilationParameters_XilinxParameterList_Parameter_OptimizationGoal text, CompilationParameters_XilinxParameterList_Parameter_PlacerEffortLevel text)
It works well, but I wonder if I can attach some aliases for the long name in a database.
Is this possible? I mean, can I have a command something like
Table string ... CompilationParameters_XilinxVersion tex >>as version<< ...
so that I can use CompilationParameters_XilinxVersion or version when retrieve the data.
What you're trying to do is not possible in SQL. However, you may want to create a VIEW that simply substitutes the long column names with your short column aliases. Note that VIEWs in sqlite are read-only and therefore cannot be written to.