We are dealing with legacy code that doesn't auto-increment the primary key (see serial) so I have to manually do it. What is the correct way to manually update the primary key field on insert. I am getting an error when I do the below
Table:
CREATE TABLE pizza (
id bigint not null,
price int
)
Insert statement:
INSERT INTO pizza
(id, price)
VALUES
(
(SELECT max(id) from pizza)+1,
1.75
)
Don't use max()+1 to generate a primary key. It's not safe for concurrent inserts and it doesn't really scale well.
Just create a sequence and use that:
create sequence pizza_id_seq;
Then synchronize it with the current values in the table:
select setval('pizza_id_seq', coalesce(max(id),1))
from pizza;
Then, instead of changing your INSERT statements to use the dreaded max() + 1, just use the sequence:
INSERT INTO pizza
(id, price)
VALUES
(nextval('pizza_id_seq'), 1.75)
Related
So I've been going through SQL migrations to insert data in a SEQUENTIAL manner specifically from parent to child.
I've inserted data in the parent table. Now I've to store the primary key value of that
specific row (WHERE condition is defined in query for reference " where description = '1234'") in a variable.
And while inserting data to the child table I've to use that primary key value stored in a variable in place of a foreign key column("country_code_id") of the child table.
I'm using Postgresql
CREATE TABLE Countries
(
id SERIAL,
description VARCHAR(100),
CONSTRAINT coutry_pkey PRIMARY KEY (id)
);
CREATE TABLE Cities
(
country_code_id int ,
city_id int,
description VARCHAR(100),
CONSTRAINT cities_pkey PRIMARY KEY (city_id),
CONSTRAINT fk_cities_countries FOREIGN KEY (country_code_id) REFERENCES Countries (id)
);
INSERT INTO COUNTRIES (description) VALUES('asdf');
#countrid = SELECT id FROM COUNTRIES WHERE description = 'asdf';
INSERT INTO cities VALUES (countrid, 1 , 'abc');
SQL does not have variables. The normal way to do this is to use INSERT ... RETURNING:
INSERT INTO countries (description) VALUES ('1234')
RETURNING id;
This will return the automatically generated primary key. You store that in a variable on the client side and run a second statement:
INSERT INTO cities (country_code_id, city_id, description)
VALUES (4711, 1, 'abc');
where 4711 is the value returned from the first statement. To avoid hard-coding the value, you can use a prepared statement, which also will boost performance.
An alternative, more complicated, solution is to run both statements in a single statement using a common table expression:
WITH country_ids AS (
INSERT INTO countries (description) VALUES ('1234')
RETURNING id
INSERT INTO (country_code_id, city_id, description)
SELECT id, 1, 'abc'
FROM country_ids;
I have two table with one to one relation and I want to insert two rows to the tables with the same auto increment id. Is it possible?
create table first
(
id bigint primary key,
value varchar(100) not null
);
create table second
(
id bigint references first (id),
sign boolean
);
insert into first(id, value)
values (-- autoincremented, 'some_value');
insert into second(id, sign)
values (-- the same autoincremented, true);
Your id column must be defined as an "auto increment" one before you can use that:
create table first
(
id bigint generated always as identity primary key,
value varchar(100) not null
);
Then you can use lastval() to get the last generated id:
insert into first(id, value)
values (default, 'some_value');
insert into second(id, sign)
values (lastval(), true);
Or if you want to be explicit:
insert into first(id, value)
values (default, 'some_value');
insert into second(id, sign)
values (currval(pg_get_serial_sequence('first','id')), true);
One option uses a cte with the returning clause:
with i as (
insert into first(value) values('some_value')
returning id
)
insert into second(id, sign)
select i.id, true from i
This performs the two inserts at once; the id of the first insert is auto-generated, and then used in the second insert.
For this to work, you need the id of the first table to be defined as serial.
I've recently been caught out by being unaware of the issue where SERIAL fields increment whether data is inserted or not.
Most of the answers I've read on this matter discuss preventing holes from appearing in the column, which I'm fairly certain in most cases isn't what the question posed is concerned with, and it certainly wasn't in my case.
My situation was that a specific user of my software was using a feature in a way that caused millions of upserts to be performed on a single record. That record was used as status information, and in my naivety I was blissfully unaware of the impending failure when the INTEGER id fields nextval() reached its limit, that being the following error:
ERROR: integer out of range
SQL state: 22003
So my question is and was, how can I prevent id fields from incrementing the next sequence value in the case of a conflict rollback.
I look foward to others adding their knowledge to my solution.
My immediate solution to this issue which alleviated the out of range situation was to alter the column to BIGINT, as follows:
ALTER TABLE MyTable ALTER COLUMN idMyTable TYPE BIGINT;
The number of records in my case was extremely small (<1000) so this was a trivial alteration to perform.
Once that was out of the way, it was time to look for a solution to the underlying issue. My solution is unlikely to be as performant as using a SERIAL field, so keep that in mind based on your use case if you are going to implement something similar to what I have done - there's always a trade off somewhere.
Consider the following table and resulting data insert/query:
CREATE TABLE TestTable ( id SERIAL PRIMARY KEY NOT NULL, Key TEXT UNIQUE NOT NULL, Val TEXT );
INSERT INTO TestTable (Key,Val) VALUES ('Fruit', 'banana') ON CONFLICT( Key ) DO UPDATE SET Val=EXCLUDED.Val;
INSERT INTO TestTable (Key,Val) VALUES ('Fruit', 'apple') ON CONFLICT( Key ) DO UPDATE SET Val=EXCLUDED.Val;
INSERT INTO TestTable (Key,Val) VALUES ('Fruit', 'peach') ON CONFLICT( Key ) DO UPDATE SET Val=EXCLUDED.Val;
INSERT INTO TestTable (Key,Val) VALUES ('Animal', 'horse') ON CONFLICT( Key ) DO UPDATE SET Val=EXCLUDED.Val;
SELECT * FROM TestTable;
id Key Val
1 Fruit peach
4 Animal horse
In this case, each conflict during the Fruit update has bumped the SERIAL value, even though it wasn't creating any new record in TestTable.
Now this is the workaround I'm currently working with. If anyone knows how to concatenate the table name onto 'NEW.id', I'd love to hear that, as I like to name my id columns idTablename for consistency.
CREATE OR REPLACE FUNCTION IncrementSerial()
RETURNS trigger AS $fn$
BEGIN
EXECUTE format('SELECT COALESCE( MAX( id ), 0 ) + 1 FROM %I.%I;',TG_TABLE_SCHEMA,TG_TABLE_NAME) INTO NEW.id;
RETURN NEW;
END
$fn$ LANGUAGE 'plpgsql'
CREATE TABLE TestTable ( id INTEGER PRIMARY KEY NOT NULL, Key TEXT UNIQUE NOT NULL, Val TEXT );
CREATE TRIGGER trgIncrementSerial
BEFORE INSERT ON TestTable
FOR EACH ROW
EXECUTE PROCEDURE IncrementSerial()
INSERT INTO TestTable (Key,Val) VALUES ('Fruit', 'banana') ON CONFLICT( Key ) DO UPDATE SET Val=EXCLUDED.Val;
INSERT INTO TestTable (Key,Val) VALUES ('Fruit', 'apple') ON CONFLICT( Key ) DO UPDATE SET Val=EXCLUDED.Val;
INSERT INTO TestTable (Key,Val) VALUES ('Fruit', 'peach') ON CONFLICT( Key ) DO UPDATE SET Val=EXCLUDED.Val;
INSERT INTO TestTable (Key,Val) VALUES ('Animal', 'horse') ON CONFLICT( Key ) DO UPDATE SET Val=EXCLUDED.Val;
SELECT * FROM TestTable;
id Key Val
1 Fruit peach
2 Animal horse
As you can see, the SERIAL id is now just using the next highest id number, which is ideal for most of my use cases.
Obviously this is going to be a problem if the id key must always be unique, as removal of the last record will free up that id. If that's not a problem (ie. where the id is just used for references and cascades) then this might be a good solution for you.
I'm just starting to wade into backend development after my first few months on the job as a front end dev. I'm working with postgreSQL and can't seem to wrap my head around the nextval() function. I read this, but it's not clear to me.
http://www.postgresql.org/docs/current/interactive/functions-sequence.html
what are the benefits/use cases for nexval()?
NEXTVAL is a function to get the next value from a sequence.
Sequence is an object which returns ever-increasing numbers, different for each call, regardless of transactions etc.
Each time you call NEXTVAL, you get a different number.
This is mainly used to generate surrogate primary keys for you tables.
You can create a table like this:
CREATE SEQUENCE mysequence;
CREATE TABLE mytable (id BIGINT NOT NULL PRIMARY KEY, value INT);
and insert values like this:
INSERT
INTO mytable (id, value)
VALUES
(NEXTVAL('mysequence'), 1),
(NEXTVAL('mysequence'), 2);
and see what you get:
SELECT * FROM mytable;
id | value
----+-------
1 | 1
2 | 2
PostgreSQL offers a nice syntax sugar for this:
CREATE TABLE mytable (id BIGSERIAL PRIMARY KEY, value INT);
which is equivalent to
CREATE SEQUENCE mytable_id_seq; -- table_column_'seq'
CREATE TABLE mytable (id BIGINT NOT NULL PRIMARY KEY DEFAULT NEXTVAL('mytable_id_seq'), value INT); -- it's not null and has a default value automatically
and can be used like this:
INSERT
INTO mytable (value)
VALUES (1),
(2); -- you can omit id, it will get filled for you.
Note that even if you rollback your insert statement or run concurrent statements from two different sessions, the returned sequence values will never be the same and never get reused (read the fine print in the docs though under CYCLE).
So you can be sure all the values of your primary keys will be generated unique within the table.
Facing some issues with the auto-increment property in postgresql
I created a table say emp
create table emp
( empid serial,
empname varcha(50),
primary key (empid)
);
I inserted one value with empid as blank:
insert into emp (empname) values ('test1');
Next insert by specifying the empid value:
insert into emp (empid,empname) values (2,'test2');
Now, the next time if I insert a value without specifying the empid value, it will give an error because it will try to insert the empid as 2:
insert into emp (empname) values ('test3');
ERROR: duplicate key value violates unique constraint "emp_pkey"
DETAIL: Key (empid)=(2) already exists.
Can someone help me with a workaround for this issue so that with or without specifying a value, the autoincrement should pick up the max(value) +1 ??
Thanks
You can't cleanly mix use of sequences and fixed IDs. Inserting values without using the sequence won't update the sequence, so you'll get collisions.
If you're doing your manual insertions in a bulk load phase or something you can:
BEGIN
LOCK TABLE the_table IN ACCESS EXCLUSIVE MODE
Do your INSERTs
SELECT setval('seq_name', 14), replacing 14 with the new sequence value. This can be a subquery against the table.
COMMIT
... but in general, it's better to just avoid mixing sequence ID generation and manually assigned IDs.