I need to change the type of a column from varchar(255) to uuid. I am using:
ALTER TABLE table_name
ALTER COLUMN col TYPE UUID USING col::UUID;
But if there is some value in that column, which is not in uuid format, I will get this error:
SQL Error [22P02]: ERROR: invalid input syntax for type uuid: "some_text"
Is it possible to delete those rows which have values that cannot be converted to uuid?
Using uuid_or_null function that #JuliusTuskenis suggests you can simply do this:
ALTER TABLE table_name ALTER COLUMN col TYPE UUID USING uuid_or_null(col);
delete from table_name where col is null;
You have to define the function before that.
create function uuid_or_null(s text) returns uuid immutable AS
$$
begin
return s::uuid;
exception when others then
return null;
end;
$$ language plpgsql;
The way uuid_or_null is built is quite generic and more or less a pattern for safe casting - try to cast and if it bangs then react accordingly. There are several SO threads using it.
You can also sanitize the table upfront and then alter column type like this:
delete from table_name
where col !~* '^[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12}$';
ALTER TABLE table_name ALTER COLUMN col TYPE UUID USING col::UUID;
I need to add a required field for newly added rows. However, it is undesirable to set the default value for old rows due to the large size of the table. I need to provide an automated script that will do this.
I tried this, but it does not work:
do $$
declare
max_id int8;
begin
select max(id) into max_id from transactions;
alter table transactions add constraint check_process_is_assigned check (id <= max_id or process_id is not null);
end $$;
Utility commands like ALTER TABLE do not accept parameters. Only the basic DML commands SELECT, INSERT, UPDATE, DELETE do.
See:
set "VALID UNTIL" value with a calculated timestamp
“ERROR: there is no parameter $1” in “EXECUTE .. USING ..;” statement in plpgsql
Creating user with password from variables in anonymous block
You need dynamic SQL like:
DO
$do$
BEGIN
EXECUTE format(
'ALTER TABLE transactions
ADD CONSTRAINT check_process_is_assigned CHECK (id <= %s OR process_id IS NOT NULL)'
, (SELECT max(id) FROM transactions)
);
END
$do$;
db<>fiddle here
This creates a CHECK constraint based on the current maximum id.
Maybe a NOT VALID constraint would serve better? That is not checked against existing rows:
ALTER TABLE transactions
ADD CONSTRAINT check_process_is_assigned CHECK (process_id IS NOT NULL) NOT VALID;
But you do have to "fix" old rows that get updated in this case. (I.e. assign a value to process_id if it was NULL so far.) See:
Enforce NOT NULL for set of columns with a CHECK constraint only for new rows
Best way to populate a new column in a large table?
Say I have a unique column of VarChar(32).
ex. 13bfa574e23848b68f1b7b5ff6d794e1.
I want to preserve the uniqueness of this while converting the column to int. I figure I can convert all of the letters to their ascii equivalent, while retaining the numbers and character position. To do this, I will use the translate function.
psuedo code: select translate(uid, '[^0-9]', ascii('[^0-9]'))
My issue is finding all of the letters in the VarChar column originally.
I've tried
select uid, substring(uid from '[^0-9]') from test_table;
But it only returns the first letter it encounters. Using the above example, I would be looking for bfaebfbbffde
Any help is appreciated!
First off, I agree with the two commenters who said you should use a UID datatype.
That aside...
Your UID looks like a traditional one, in that it's not alphanumeric, it's hex. If this is the case, you can convert the hex to the numeric value using this solution:
PostgreSQL: convert hex string of a very large number to a NUMERIC
Notice the accepted solution (mine, shame) is not as good as the other solution listed, as mine will not work for hex values this large.
That said, yikes, what a huge number. Holy smokes.
Depending on how many records are in your table and the frequency of insert/update, I would consider a radically different approach. In a nutshell, I would create another column to store your numeric ID whose value would be determined by a sequence.
If you really want to make it bulletproof, you can also create a cross-reference table to store the relationships that would
Reuse an ID if it ever repeated (I know UIDs don't, but this would cover cases where a record is deleted by mistake, re-appears, and you want to retain the original id)
If UIDs repeat (like this is a child table with multiple records per UID), it would cover that case as well
If neither of these apply, you could dumb it down quite a bit.
The solution would look something like this:
Add an ID column that will be your numeric equivalent to the UID:
alter table test_table
add column id bigint
Create a sequence:
CREATE SEQUENCE test_id
create a cross-reference table (again, not necessary for the dumbed down version):
create table test_id_xref (
uid varchar(32) not null,
id bigint not null,
constraint test_id_xref_pk primary key (uid)
)
Then do a one-time update to assign a surrogate ID to each UID for both the cross-reference and actual tables:
insert into test_id_xref
with uids as (
select distinct uid
from test_table
)
select uid, nextval ('test_id')
from uids;
update test_table tt
set id = x.id
from test_id_xref x
where tt.uid = x.uid;
And finally, for all future inserts, create a trigger to assign the next value:
CREATE OR REPLACE FUNCTION test_table_insert_trigger()
RETURNS trigger AS
$BODY$
BEGIN
select t.id
from test_id_xref t
into NEW.id
where t.uid = NEW.uid;
if NEW.id is null then
NEW.id := nextval('test_id');
insert into test_id_xref values (NEW.uid, NEW.id);
end if;
return NEW;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
CREATE TRIGGER insert_test_table_trigger
BEFORE INSERT
ON test_table
FOR EACH ROW
EXECUTE PROCEDURE test_table_insert_trigger();
create one function which replace charter with blank which you not need in string,
CREATE FUNCTION replace_char(v_string VARCHAR(32) CHARSET utf8) RETURNS VARCHAR(32)
DETERMINISTIC
BEGIN
DECLARE v_return_string VARCHAR(32) DEFAULT '';
DECLARE v_remove_char VARCHAR(200) DEFAULT '1,2,3,4,5,6,7,8,9,0';
DECLARE v_length, j INT(3) DEFAULT 0;
SET v_length = LENGTH(v_string);
WHILE(j < v_length) DO
IF ( FIND_IN_SET( SUBSTR(v_string, (j+1), 1), v_remove_char ) = 0) THEN
SET v_return_string = CONCAT(v_return_string, SUBSTR(v_string, (j+1), 1) );
END IF;
SET j = j+1;
END WHILE;
RETURN v_return_string;
END$$
DELIMITER ;
Now you just nee to call this function in query
select uid, replace_char(uid) from test_table;
It will give you string what you need (bfaebfbbffde)
If you want to int number only i.e 13574238486817567941 then change value of variable, and also column datatype in decimal(50,0), decimal can stored large number and there is 0 decimal point so it will store int value as decimal.
v_remove_char = 'a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z';
I have a postgres table with millions of record in it. Now I want to add new column to that table called "time_modified" with the value in another column "last_event_time". Running a migration script is taking long time , so need a simple solution to run in production.
Assuming that the columns are timestamps you can try:
alter table my_table add time_modified text;
alter table my_table alter time_modified type timestamp using last_event_time;
I suggest use function with pg_sleep, which wait between iteriation in loop
This way don't invoke exclusive lock and others locks on your_table.
SELECT pg_sleep(seconds);
But time of execute is long
alter table my_table add time_modified timestamp;
CREATE OR REPLACE FUNCTION update_mew_column()
RETURNS void AS
$BODY$
DECLARE
rec record;
BEGIN
for rec in (select id,last_event_time from your_table) loop
update your_table set time_modified = rec.last_event_time where id = rec.id;
PERFORM pg_sleep(0.01);
end loop;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
and execute function:
select update_mew_column();
I've got a table that people have been inserting into getting the primary key by doing a
SELECT max(id)+1 from table_a;
I want to add some records to that table using a INSERT INTO table_a SELECT ... FROM table_b, table_c ... simple SQL script, and I'm wondering how to generate the primary keys. My first thought was to create a temporary sequence, but Oracle evidently doesn't have a select setval to set the first value. So how do I get the current value of max(id)+1 to set the "start with" parameter to my sequence?
I found something on-line that I thought would work:
COLUMN S new_value st select max(id)+1 S from table_a;
CREATE SEQUENCE cra_seq start with &st;
But it doesn't actually use st in the CREATE SEQUENCE but instead prompts me to enter it, which isn't what I need.
Is this something like what you want?
1 declare
2 id integer;
3 begin
4 select max(rownum)+1 into id from dual;
5 execute immediate 'create sequence myseq start with '||TO_CHAR(id);
6* end;
7 /
Couldn't you use the row_number function like so:
Insert Destination( Id, ...)
Select row_number() over( order by TableA.Col1... ) + MaxDestination.MaxId + 1 Num
, ....
From TableA, TableB,...
Cross Join ( Select Max(Id) MaxId From Destination ) MaxDestination
You can use the row_number analytical function to generate row numbers (1 through N).
Before you do the insert, get the max id that is in the table, and then add the row number to that max and it will populate your table correctly.
In PostgreSQL:
CREATE SEQUENCE new_seq;
ALTER TABLE existing_table ADD COLUMN new_serial_column bigint DEFAULT 0;
UPDATE existing_table SET new_serial_column = nextval('new_seq');
ALTER TABLE existing_table ALTER COLUMN new_serial_column SET NOT NULL;
ALTER TABLE existing_table ALTER COLUMN new_serial_column SET DEFAULT nextval('new_seq');
Although, that code is not idempotent, so check you havn't already created the new sequence, something like:
CREATE FUNCTION fixup_existing_table() RETURNS void AS $$
DECLARE
new_id bigint;
seq_column_exists integer;
BEGIN
SELECT Count(column_name) INTO seq_column_exists FROM information_schema.columns WHERE table_name='existing_table' and column_name='new_serial_column';
IF seq_column_exists != 0 THEN RETURN; END IF;
CREATE SEQUENCE new_seq;
ALTER TABLE existing_table ADD COLUMN new_serial_column bigint DEFAULT 0;
UPDATE existing_table SET new_serial_column = nextval('new_seq');
ALTER TABLE existing_table ALTER COLUMN new_serial_column SET NOT NULL;
ALTER TABLE existing_table ALTER COLUMN new_serial_column SET DEFAULT nextval('new_seq');
END;
$$ LANGUAGE plpgsql;
Then, you can safely call: SELECT fixup_existing_table() to alter the schema as many times as you like, i.e. call from some dumb update script.