Custom SERIAL / autoincrement per group of values - sql

I'm trying to make a blog system of sort and I ran into a slight problem.
Simply put, there's 3 columns in my article table:
id SERIAL,
category VARCHAR FK,
category_id INT
id column is obviously the PK and it is used as a global identifier for all articles.
category column is well .. category.
category_id is used as a UNIQUE ID within a category so currently there is a UNIQUE(category, category_id) constraint in place.
However, I also want for category_id to auto-increment.
I want it so that every time I execute a query like
INSERT INTO article(category) VALUES ('stackoverflow');
I want the category_id column to be automatically be filled according to the latest category_id of the 'stackoverflow' category.
Achieving this in my logic code is quite easy. I just select latest num and insert +1 of that but that involves two separate queries.
I am looking for a SQL solution that can do all this in one query.

This has been asked many times and the general idea is bound to fail in a multi-user environment - and a blog system sounds like exactly such a case.
So the best answer is: Don't. Consider a different approach.
Drop the column category_id completely from your table - it does not store any information the other two columns (id, category) wouldn't store already.
Your id is a serial column and already auto-increments in a reliable fashion.
Auto increment SQL function
If you need some kind of category_id without gaps per category, generate it on the fly with row_number():
Serial numbers per group of rows for compound key

Concept
There are at least several ways to approach this. First one that comes to my mind:
Assign a value for category_id column inside a trigger executed for each row, by overwriting the input value from INSERT statement.
Action
Here's the SQL Fiddle to see the code in action
For a simple test, I'm creating article table holding categories and their id's that should be unique for each category. I have omitted constraint creation - that's not relevant to present the point.
create table article ( id serial, category varchar, category_id int )
Inserting some values for two distinct categories using generate_series() function to have an auto-increment already in place.
insert into article(category, category_id)
select 'stackoverflow', i from generate_series(1,1) i
union all
select 'stackexchange', i from generate_series(1,3) i
Creating a trigger function, that would select MAX(category_id) and increment its value by 1 for a category we're inserting a row with and then overwrite the value right before moving on with the actual INSERT to table (BEFORE INSERT trigger takes care of that).
CREATE OR REPLACE FUNCTION category_increment()
RETURNS trigger
LANGUAGE plpgsql
AS
$$
DECLARE
v_category_inc int := 0;
BEGIN
SELECT MAX(category_id) + 1 INTO v_category_inc FROM article WHERE category = NEW.category;
IF v_category_inc is null THEN
NEW.category_id := 1;
ELSE
NEW.category_id := v_category_inc;
END IF;
RETURN NEW;
END;
$$
Using the function as a trigger.
CREATE TRIGGER trg_category_increment
BEFORE INSERT ON article
FOR EACH ROW EXECUTE PROCEDURE category_increment()
Inserting some more values (post trigger appliance) for already existing categories and non-existing ones.
INSERT INTO article(category) VALUES
('stackoverflow'),
('stackexchange'),
('nonexisting');
Query used to select data:
select category, category_id From article order by 1,2
Result for initial inserts:
category category_id
stackexchange 1
stackexchange 2
stackexchange 3
stackoverflow 1
Result after final inserts:
category category_id
nonexisting 1
stackexchange 1
stackexchange 2
stackexchange 3
stackexchange 4
stackoverflow 1
stackoverflow 2

Postgresql uses sequences to achieve this; it's a different approach from what you are used to in MySQL. Take a look at http://www.postgresql.org/docs/current/static/sql-createsequence.html for complete reference.
Basically you create a sequence (a database object) by:
CREATE SEQUENCE serials;
And then when you want to add to your table you will have:
INSERT INTO mytable (name, id) VALUES ('The Name', NEXTVAL('serials')

Related

How to insert a row if not exists otherwise select and return its ID in both cases in MariaDB?

I have a table with ID primary key (autoincrement) and a unique column Name. Is there an efficient way in MariaDB to insert a row into this table if the same Name doesn't exist, otherwise select the existing row and, in both cases, return the ID of the row with this Name?
Here's a solution for Postgres. However, it seems MariaDB doesn't have the RETURNING id clause.
What I have tried so far is brute-force:
INSERT IGNORE INTO services (Name) VALUES ('JohnDoe');
SELECT ID FROM services WHERE Name='JohnDoe';
UPDATE: MariaDB 10.5 has RETURNING clause, however, the queries I have tried so far throw a syntax error:
WITH i AS (INSERT IGNORE INTO services (`Name`) VALUES ('John') RETURNING ID)
SELECT ID FROM i
UNION
SELECT ID FROM services WHERE `Name`='John'
For a single row, assuming id is AUTO_INCREMENT.
INSERT INTO t (name)
VALUES ('JohnDoe')
ON DUPLICATE KEY id = LAST_INSERT_ID(id);
SELECT LAST_INSERT_ID();
That looks kludgy, but it is an example in the documentation.
Caution: Most forms of INSERT will "burn" auto_inc ids. That is, they grab the next id(s) before realizing that the id won't be used. This could lead to overflowing the max auto_inc size.
It is also wise not to put the normalization inside the transaction that does the "meat" of the code. It ties up the table unnecessarily long and runs extra risk of burning ids in the case of rollback.
For batch updating of a 'normalization' table like that, see my notes here: http://mysql.rjweb.org/doc.php/staging_table#normalization (It avoids burning ids.)

PostgreSQL: Inserting tuples in multiple tables using a view and a trigger

I am trying to build an order system that is able to insert a compound order that consists of multiple items and amounts. My database layout is as follows: I have an order table, containing an autoincrement id, item_id, amount and order_group_id columns. I also have an order_group table containing an autoincrement id and a person_id column. The idea is that when a person orders, one new order_group entry is created, and its id is used as the fk in the orders that the person has done.
I presume that this would normally be done in the code of the application. However, I am using postgrest to provide an API for me, which suggests creating a custom view to insert compound entries via that route. This is described here.
This is what I have so far:
CREATE FUNCTION kzc.new_order()
RETURNS TRIGGER
LANGUAGE plpgsql
AS $$
DECLARE
group_id int;
BEGIN
INSERT INTO kzc.order_group (person) VALUES (new.person) RETURNING id AS group_id;
INSERT INTO kzc."order" (item, amount, order_group) VALUES (new.item_id, new.amount, group_id);
RETURN new;
END;
$$;
CREATE TRIGGER new_order
INSTEAD OF INSERT ON kzc.new_order
FOR EACH ROW
EXECUTE FUNCTION kzc.new_order()
However, this code makes a new ordergroup for every order that is in the compound insert. How can I make it so that my code only makes one new ordergroup entry and assigns its id to all orders?
Thanks in advance!
I suggest that you add an order_group_id column to the new_order view and create a sequence for it. Then create a DEFAULT value for the column:
ALTER VIEW kzc.new_order
ALTER order_group_id SET DEFAULT currval('order_group_id_seq');
Add a BEFORE INSERT trigger FOR EACH STATEMENT that just calls nextval for the sequence. The currval calls will all pick up the same generated value.
Then you have that number in your trigger and can use it as a primary key for order_group.
To avoid adding the row multiple times, use
INSERT INTO kzc.order_group (id, person)
VALUES (NEW.order_group_id, NEW.person)
ON CONFLICT (id) DO NOTHING;

SQL Server Unique Composite Key of Two Field With Second Field Auto-Increment

I have the following problem, I want to have Composite Primary Key like:
PRIMARY KEY (`base`, `id`);
for which when I insert a base the id to be auto-incremented based on the previous id for the same base
Example:
base id
A 1
A 2
B 1
C 1
Is there a way when I say:
INSERT INTO table(base) VALUES ('A')
to insert a new record with id 3 because that is the next id for base 'A'?
The resulting table should be:
base id
A 1
A 2
B 1
C 1
A 3
Is it possible to do it on the DB exactly since if done programmatically it could cause racing conditions.
EDIT
The base currently represents a company, the id represents invoice number. There should be auto-incrementing invoice numbers for each company but there could be cases where two companies have invoices with the same number. Users logged with a company should be able to sort, filter and search by those invoice numbers.
Ever since someone posted a similar question, I've been pondering this. The first problem is that DBs don't provide "partitionable" sequences (that would restart/remember based on different keys). The second is that the SEQUENCE objects that are provided are geared around fast access, and can't be rolled back (ie, you will get gaps). This essentially this rules out using a built-in utility... meaning we have to roll our own.
The first thing we're going to need is a table to store our sequence numbers. This can be fairly simple:
CREATE TABLE Invoice_Sequence (base CHAR(1) PRIMARY KEY CLUSTERED,
invoiceNumber INTEGER);
In reality the base column should be a foreign-key reference to whatever table/id defines the business(es)/entities you're issuing invoices for. In this table, you want entries to be unique per issued-entity.
Next, you want a stored proc that will take a key (base) and spit out the next number in the sequence (invoiceNumber). The set of keys necessary will vary (ie, some invoice numbers must contain the year or full date of issue), but the base form for this situation is as follows:
CREATE PROCEDURE Next_Invoice_Number #baseKey CHAR(1),
#invoiceNumber INTEGER OUTPUT
AS MERGE INTO Invoice_Sequence Stored
USING (VALUES (#baseKey)) Incoming(base)
ON Incoming.base = Stored.base
WHEN MATCHED THEN UPDATE SET Stored.invoiceNumber = Stored.invoiceNumber + 1
WHEN NOT MATCHED BY TARGET THEN INSERT (base) VALUES(#baseKey)
OUTPUT INSERTED.invoiceNumber ;;
Note that:
You must run this in a serialized transaction
The transaction must be the same one that's inserting into the destination (invoice) table.
That's right, you'll still get blocking per-business when issuing invoice numbers. You can't avoid this if invoice numbers must be sequential, with no gaps - until the row is actually committed, it might be rolled back, meaning that the invoice number wouldn't have been issued.
Now, since you don't want to have to remember to call the procedure for the entry, wrap it up in a trigger:
CREATE TRIGGER Populate_Invoice_Number ON Invoice INSTEAD OF INSERT
AS
DECLARE #invoiceNumber INTEGER
BEGIN
EXEC Next_Invoice_Number Inserted.base, #invoiceNumber OUTPUT
INSERT INTO Invoice (base, invoiceNumber)
VALUES (Inserted.base, #invoiceNumber)
END
(obviously, you have more columns, including others that should be auto-populated - you'll need to fill them in)
...which you can then use by simply saying:
INSERT INTO Invoice (base) VALUES('A');
So what have we done? Mostly, all this work was about shrinking the number of rows locked by a transaction. Until this INSERT is committed, there are only two rows locked:
The row in Invoice_Sequence maintaining the sequence number
The row in Invoice for the new invoice.
All other rows for a particular base are free - they can be updated or queried at will (deleting information out of this kind of system tends to make accountants nervous). You probably need to decide what should happen when queries would normally include the pending invoice...
you can use the trigger for before insert and assign the next value by taking the max(id) with "base" filter which is "A" in this case.
That will give you the max(id) value as 2 and than increment it by max(id)+1. now push the new value to the "id" field. before insert.
I think this may help you
MSSQL Triggers: http://msdn.microsoft.com/en-in/library/ms189799.aspx
Test Table
CREATE TABLE MyTable
( base CHAR(1),
id INT
)
GO
Trigger Definition
CREATE TRIGGER dbo.tr_Populate_ID
ON dbo.MyTable
INSTEAD OF INSERT
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO MyTable (base,id)
SELECT i.base, ISNULL(MAX(mt.id),0) +1 AS NextValue
FROM inserted i left join MyTable mt
on i.base = mt.base
GROUP BY i.base
END
Test
Execute the following statement multiple times and you will see the next values available in that group will be assigned to ID.
INSERT INTO MyTable VALUES
('A'),
('B'),
('C')
GO
SELECT * FROM MyTable
GO

INSERT, and get the auto-incremented value

Consider the following table:
create table language (
id integer generated always as identity (START WITH 1, INCREMENT BY 1),
name long varchar,
constraint language_pk primary key (id)
);
To which I'd insert an entry this way.
insert into language(name) values ('value');
How does one know what value for id was created? Just doing a SELECT using the name field is not valid, because there can be duplicate entries.
Through plain SQL:
insert into language(name) values ('value');
SELECT IDENTITY_VAL_LOCAL();
See the manual for details: http://db.apache.org/derby/docs/10.7/ref/rrefidentityvallocal.html
When doing this from a Java class (through JDBC) you can use getGeneratedKeys() after "requesting" them with the approriate executeUpdate() method.
You use the JDBC method
st.execute(sql, Statement.RETURN_GENERATED_KEYS);
ResultSet keys = st.getGeneratedKeys();
as documented in the Derby manual.
See also Javadocs: DatabaseMetaData#supportsGetGeneratedKeys()
and Statement#getGeneratedKeys()
You could execute this statement (NB, not 100% sure this syntax is correct for Derby:
SELECT TOP 1 id FROM language ORDER BY id DESC
To find the last inserted ID.
Alternative for Derby:
SELECT MAX(id) from language
Obviously this will only be accurate if no other inserts (including inserts by other users) have happened between your insert and select.
See also this discussion:

Retrieve inserted row ID in SQL

How do I retrieve the ID of an inserted row in SQL?
Users Table:
Column | Type
--------|--------------------------------
ID | * Auto-incrementing primary key
Name |
Age |
Query Sample:
insert into users (Name, Age) values ('charuka',12)
In MySQL:
SELECT LAST_INSERT_ID();
In SQL Server:
SELECT SCOPE_IDENTITY();
In Oracle:
SELECT SEQNAME.CURRVAL FROM DUAL;
In PostgreSQL:
SELECT lastval();
(edited: lastval is any, currval requires a named sequence)
Note: lastval() returns the latest sequence value assigned by your session, independently of what is happening in other sessions.
In SQL Server, you can do (in addition to the other solutions already present):
INSERT INTO dbo.Users(Name, Age)
OUTPUT INSERTED.ID AS 'New User ID'
VALUES('charuka', 12)
The OUTPUT clause is very handy when doing inserts, updates, deletes, and you can return any of the columns - not just the auto-incremented ID column.
Read more about the OUTPUT clause in the SQL Server Books Online.
In Oracle and PostgreSQL you can do this:
INSERT INTO some_table (name, age)
VALUES
('charuka', 12)
RETURNING ID
When doing this through JDBC you can also do that in a cross-DBMS manner (without the need for RETURNING) by calling getGeneratedKeys() after running the INSERT
I had the same need and found this answer ..
This creates a record in the company table (comp), it the grabs the auto ID created on the company table and drops that into a Staff table (staff) so the 2 tables can be linked, MANY staff to ONE company. It works on my SQL 2008 DB, should work on SQL 2005 and above.
===========================
CREATE PROCEDURE [dbo].[InsertNewCompanyAndStaffDetails]
#comp_name varchar(55) = 'Big Company',
#comp_regno nchar(8) = '12345678',
#comp_email nvarchar(50) = 'no1#home.com',
#recID INT OUTPUT
-- The '#recID' is used to hold the Company auto generated ID number that we are about to grab
AS
Begin
SET NOCOUNT ON
DECLARE #tableVar TABLE (tempID INT)
-- The line above is used to create a tempory table to hold the auto generated ID number for later use. It has only one field 'tempID' and its type INT is the same as the '#recID'.
INSERT INTO comp(comp_name, comp_regno, comp_email)
OUTPUT inserted.comp_id INTO #tableVar
-- The 'OUTPUT inserted.' line above is used to grab data out of any field in the record it is creating right now. This data we want is the ID autonumber. So make sure it says the correct field name for your table, mine is 'comp_id'. This is then dropped into the tempory table we created earlier.
VALUES (#comp_name, #comp_regno, #comp_email)
SET #recID = (SELECT tempID FROM #tableVar)
-- The line above is used to search the tempory table we created earlier where the ID we need is saved. Since there is only one record in this tempory table, and only one field, it will only select the ID number you need and drop it into '#recID'. '#recID' now has the ID number you want and you can use it how you want like i have used it below.
INSERT INTO staff(Staff_comp_id)
VALUES (#recID)
End
-- So there you go. I was looking for something like this for ages, with this detailed break down, I hope this helps.