I'm creating an archive table using SELECT INTO. This table however has a time stamp column, and when I try to insert values I get the following error:
Cannot insert an explicit value into a timestamp column
I do not wish to insert using a column list, and then insert null to the TimeStamp column
What I want to do is on the archive table change dynamically all colums of type Timestamp to varbinary(8).
What I have tried so far is to get the column:
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = 'AppServices_Notification' AND
DATA_TYPE = 'TimeStamp'
ALTER TABLE AppServices_Notification
ALTER COLUMN COLUMN_NAME varbinary(8)
Is there a way to this in one action?
Related
After removing a column from a table by:
ALTER TABLE MyTable
DROP COLUMN IF EXISTS MyColumn
In BigQuery UI I Can see that the column was deleted successfully & I can't query the specific column but when I query DDL I can see that the column still exists in the scheme:
SELECT DDL FROM MyDataSet.INFORMATION_SCHEMA.TABLES
WHERE DDL LIKE '%MyTable%'
What am I doing wrong?
This is a nasty, undocumented side effect of Bigquery's Time Travel. Time Travel makes it unsafe to use ALTER TABLE statements in bigquery.
Demonstration of problem:
create table apu.time_travel_problem
( id int64
, name string
);
select column_name, data_type
FROM apu.INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'time_travel_problem';
column_name
data_type
id
INT64
name
STRING
This is all normal so far, but after an ALTER TABLE everything goes odd:
alter table apu.time_travel_problem drop column name;
select column_name, data_type
FROM apu.INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'time_travel_problem';
column_name
data_type
id
INT64
name
STRING
The column we just dropped is still there!
Now try this:
alter table apu.time_travel_problem add column name string;
Column `name` was recently deleted in the table `time_travel_problem`. Deleted column name is reserved for up to the time travel duration, use a different column name instead.
Solution:
Do not use ALTER TABLE in bigquery. Instead DROP and reCREATE using a temporary table.
This is a jinja template which I use:
/* {{TABLE}} */
CREATE TABLE IF NOT EXISTS {{DATASET}}.{{TABLE}}_migration
OPTIONS (expiration_timestamp = timestamp_add(CURRENT_TIMESTAMP(), INTERVAL 8 HOUR))
AS SELECT * FROM {{DATASET}}.{{TABLE}};
DROP TABLE {{DATASET}}.{{TABLE}};
CREATE TABLE {{DATASET}}.{{TABLE}}
(
{{COLUMN_DDL}}
);
INSERT INTO {{DATASET}}.{{TABLE}}
(
{{COLUMN_LIST}}
)
SELECT
{{COLUMN_LIST}}
FROM {{DATASET}}.{{TABLE}}_migration;
We're changing a column run_dates (per request) from daterange to tstzrange, a column with a few months of test data. Essentially we want to migrate!
I'm running the script below, inspired by postgreSQL alter column data type to timestamp without time zone and I have to admit, my SQL background is basically nothing
-- Create a temporary TIMESTAMP column
ALTER TABLE table_name ADD COLUMN run_dates_holder TSTZRANGE NULL;
-- Copy casted value over to the temporary column
UPDATE table_name SET run_dates_holder = run_dates::TSTZRANGE;
-- Modify original column using the temporary column
ALTER TABLE table_name ALTER COLUMN run_dates TYPE
TSTZRANGE USING run_dates_holder;
-- Drop the temporary column (after examining altered column values)
ALTER TABLE table_name DROP COLUMN run_dates_holder;
Unfortunately..... These types don't naturally translate.
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) cannot cast type daterange to tstzrange
and
psycopg2.ProgrammingError: cannot cast type daterange to tstzrange
LINE 5: ...ble_name SET run_dates_holder = run_dates::TSTZRANG...
Has anyone ever successfully migrated a daterange to a tstzrange?
As a backup, we can def just drop the daterange column and recreate as tstzrange, since that would only affect test data. Just less optimal for the team. It's worth a shot to look into this, and I think the resolution here is at least worthwhile to future 'daterange-to-tstzrange' migrators, as I found no other docs/resources on the matter
You can't just cast a daterange to a tstzrange. Use lower() and upper() to extract the bounds of the daterange and upper_inc() and lower_inc() to extract their inclusivity, and construct a new tstzrange.
UPDATE table_name
SET run_dates_holder=tstzrange(
lower(run_dates), upper(run_dates),
concat(
CASE WHEN lower_inc(run_dates) THEN '[' else '(' END,
CASE WHEN upper_inc(run_dates) THEN ']' ELSE ')' END)
);
Using the line from AdamKG's answer (selected as the BEST!) we arrive at this
-- -- Create a placeholder tstzrange column
ALTER TABLE SB ADD COLUMN run_dates_holder tstzrange NULL;
-- -- Update placeholder tstzrange column with existing run_dates
UPDATE SB SET run_dates_holder=tstzrange(lower(run_dates), upper(run_dates),'[)');
-- -- Remove original run_dates
ALTER TABLE SB DROP COLUMN run_dates;
-- -- Rename 'run_dates_holder' to 'run_dates'
ALTER TABLE SB RENAME COLUMN run_dates_holder to run_dates;
Test data's remained intact
I added a column using following commands (I used quotation marks so that N is in upper case in my column name)
ALTER TABLE new_table
ADD “Name” VARCHAR(50);
However, I see "nam" column in my table now after running that command.
How can I drop that column?
ALTER TABLE new_table
DROP COLUMN "Name";
I get following error:
ERROR: column "name" of relation "new_table" does not exist
The following statement causes this error:
ALTER TABLE new_table
DROP COLUMN "Name";
seems you are not using double quote " in first query bus some others quotes try suing the same chars “”
ALTER TABLE new_table DROP COLUMN “Name”;
Check how Postgres is storing the column name when you are using a double quote in column name:
select table_name, column_name from information_schema.columns where
table_name='tab1';
Also, you may view the same when you execute a select statement:
select * from table;
Copy the same column name text in your alter statement. Here is the sample column name with both kind of double quotes in PostgreSQL:
create table tab1(data varchar(30));
alter table tab1 ADD “Name1” varchar(50);
alter table tab1 ADD "Name2" varchar(50);
select *from tab1;
select table_name, column_name from information_schema.columns where table_name='tab1';
alter table tab1 drop column “Name1”;
alter table tab1 drop column "Name2";
table_name column_name
tab1 data
tab1 “name1”
tab1 Name2
Here is the link to the fiddle
Note: Avoid using a double quote in the table name, column, etc. In case you use you have to ensure that the same names are specified in all queries.
Edit:
It's simple. If you use "Name" (not same as “Name”, notice quote angle) in the column name then you have to refer the column as "Name". In case “Name” is used then you have to refer by “Name”. The quotes need to match.
Another observation is when "Name" used as column name it makes the column name as Name (N uppercase) as opposed to all lowercase column names by default in the database but needs to be referred as "Name".
Iam trying to change the datatype of the column from integer(9) to Numeric(14,3).
but not able to change
i tried below query
alter table TableName alter ColumnName NUMERIC (14,3);
hope below one will help.
ALTER TABLE Table_Name ALTER COLUMN Column_Name SET DATA TYPE NUMERIC(14,3);
I have loaded the date field with dates and type is varchar.
How to convert date field(varchar) to date field(date) in oracle express/sql loader while displaying the fields?
You can't change the data type of a column in a permanent table from VARCHAR2 to DATE when it has data.
You can, however, add a new column
ALTER TABLE table_name
ADD( new_date_column DATE );
move the data over
UPDATE table_name
SET new_date_column = to_date( old_varchar2_column, format_mask );
drop the old column
ALTER TABLE table_name
DROP COLUMN old_varchar2_column;
and then rename the new column to the old column name
ALTER TABLE table_name
RENAME COLUMN new_date_column TO old_column_name
Of course, once you do this, you'll need to change your SQL*Loader script to convert the data to a DATE if you ever want to load into this table again.