Postgres insert on conflict update using other table - sql

I having syntax error for the following sql which spent an hour but cant find any answer form postgres documentation.
CREATE TABLE transaction (userid SMALLINT, point INT, timestamp TIMESTAMPTZ);
CREATE TABLE point (userid SMALLINT PRIMARY KEY, balance1 INT, balance2 INT);
CREATE TABLE settings (userid SMALLINT, bonus INT);
INSERT INTO settings VALUES (1, 10); -- sample data
WITH trans AS (
INSERT INTO transaction (userid, point, timestamp)
VALUES ($1, $2, NOW())
RETURNING *
)
INSERT INTO point (userid, balance1)
SELECT userid, point
FROM trans -- first time
ON CONFLICT (userid) DO UPDATE
SET balance1=point.balance1 + excluded.balance1,
balance2=point.balance2 + excluded.balance1 + settings.bonus
FROM settings -- tried USING, not work
WHERE settings.userid=excluded.userid;
My question is how can I include extra table without using FROM during update? My target is to keep all into 1 query.

It's not totally clear to me what you are trying to do. The way I understand it, is that you are trying to use the value from settings.bonus when updating the row in case on conflict kicks in.
This can be done using a co-related sub-query in the UPDATE part:
WITH trans (userid, point, timestamp) AS (
INSERT INTO transaction (userid, point, timestamp)
VALUES (1, 1, NOW())
RETURNING *
)
INSERT INTO point (userid, balance1)
SELECT trans.userid, trans.point
FROM trans
ON CONFLICT (userid) DO UPDATE
SET balance1 = point.balance1 + excluded.balance1,
balance2 = point.balance2 + excluded.balance1 + (select s.bonus
from settings s
where s.userid = excluded.userid)
Note however, that balance2 will never be incremented this way because on the first insert it will be null subsequent updates will try to add something to null but any expression involving null yields null (null + 5 is null).
So you either need to insert 0 when doing the insert the first time, or use coalesce() when doing the update, e.g.
balance2 = coalesce(point.balance2, 0) + excluded.balance1 + (select ...);
If it's possible that there is no row in settings for that user, then you need to apply coalesce() on the result of the sub-query as well:
... excluded.balance1 + coalesce( (select ... ), 0)
Also: you used excluded.point in your question which is incorrect. excluded refers to the target table point but that does not have a column point - you probably meant: excluded.balance1 (as that is the column into which the inserted trans.point will go)
Unrelated, but: timestamp is a horrible name for a column. For one because it's a keyword, but more importantly because it does not document what the column is for. Does it mean "created at"? "valid until"? "start at"? "obsolete from"? Something entirely different?

Related

Leveraging CHECKSUM in MERGE but unable to get all rows to merge

I am having trouble getting MERGE statements to work properly, and I have recently started to try to use checksums.
In the toy example below, I cannot get this row to insert (1, 'ANDREW', 334.3) that is sitting in the staging table.
DROP TABLE TEMP1
DROP TABLE TEMP1_STAGE
-- create table
CREATE TABLE TEMP1
(
[ID] INT,
[NAME] VARCHAR(55),
[SALARY] FLOAT,
[SCD] INT
)
-- create stage
CREATE TABLE TEMP1_STAGE
(
[ID] INT,
[NAME] VARCHAR(55),
[SALARY] FLOAT,
[SCD] INT
)
-- insert vals into stage
INSERT INTO TEMP1_STAGE (ID, NAME, SALARY)
VALUES
(1, 'ANDREW', 333.3),
(2, 'JOHN', 555.3),
(3, 'SARAH', 444.3)
-- insert stage table into main table
INSERT INTO TEMP1
SELECT *
FROM TEMP1_STAGE;
-- clean up stage table
TRUNCATE TABLE TEMP1_STAGE;
-- put some new values in the stage table
INSERT INTO TEMP1_STAGE (ID, NAME, SALARY)
VALUES
(1, 'ANDREW', 334.3),
(4, 'CARL', NULL)
-- CHECKSUMS
update TEMP1_STAGE
set SCD = binary_checksum(ID, NAME, SALARY);
update TEMP1
set SCD = binary_checksum(ID, NAME, SALARY);
-- run merge
MERGE TEMP1 AS TARGET
USING TEMP1_STAGE AS SOURCE
-- match
ON (SOURCE.[ID] = TARGET.[ID])
WHEN NOT MATCHED BY TARGET
THEN INSERT (
[ID], [NAME], [SALARY], [SCD]) VALUES (
SOURCE.[ID], SOURCE.[NAME], SOURCE.[SALARY], SOURCE.[SCD]);
-- the value: (1, 'ANDREW', 334.3) is not merged in
SELECT * FROM TEMP1;
How can I use the checksum to my advantage in the MERGE?
Your issue is that the NOT MATCHED condition is only considering the ID values specified in the ON condition.
If you want duplicate, but distinct records, include SCD to the ON condition.
If (more likely) your intent is that record ID = 1 be updated with the new SALARY, you will need to add a WHEN MATCHED AND SOURCE.SCD <> TARGET.SCD THEN UPDATE ... clause.
That said, the 32-bit int value returned by the `binary_checksum()' function is not sufficiently distinct to avoid collisions and unwanted missed updates. Take a look at HASHBYTES instead. See Binary_Checksum Vs HashBytes function.
Even that may not yield your intended performance gain. Assuming that you have to calculate the hash for all records in the staging table for each update cycle, you may find that it is simpler to just compare each potentially different field before the update. Something like:
WHEN MATCHED AND (SOURCE.NAME <> TARGET.NAME OR SOURCE.SALARY <> TARGET.SALARY)
THEN UPDATE ...
Even then, you need to be careful of potential NULL values and COLLATION. Both NULL <> 50000.00 and 'Andrew' <> 'ANDREW' may not give you the results you expect. It might be easiest and most reliable to just code WHEN MATCHED THEN UPDATE ....
Lastly, I suggest using DECIMAL instead of FLOAT for Salary.

How to get the MAX Id value from a table in MS SQL Server

I'm trying to update the table data with some row that should have defined a MAX Id + 1 value for the Movement Id field which is the PK in my table.
The Movement Id field is the INT PK, the Added At field is a date and the IncOutc is the INT field that can store either 1 or 2(Income or Outcome).
query.sql
DECLARE #max_id INT;
SET #max_id = SELECT MAX([Movement Id]) FROM Movement
INSERT INTO Movement([Movement Id], [Added At], IncOutc)
VALUES (max_id, GETDATE(), 1)
I tried the query above but got an error: Incorrect syntax near the keyword 'SELECT'. (on line 1).
I already tried inserting the values like this:
VALUES (SELECT MAX([Movement Id]) FROM Movement, max_id, GETDATE(), 1)
But got an error saying: Incorrect syntax near '1'. (on line 4)
You don't. You use an IDENTITY column or SEQUENCE when you create the table. So:
CREATE TABLE Movements (
Movement_Id INT IDENTITY(1, 1) PRIMARY KEY,
Added_At DATETIME DEFAULT GETDATE(),
IncOutc Int
);
Then you insert to it as:
INSERT INTO Movements (IncOutc)
VALUES (1);
Movement_Id and Added_At are given appropriate default values on the insert.
Attempting to do this outside the database is very problematic. In particular, two inserts at the same time might generate the same id in the table -- presumably not what you want. Preventing that requires locking the table, which is very expensive. Especially considering that SQL Server has built-in functionality to do this.
To answer your actual question, you use:
SET #max_id = (SELECT MAX([Movement Id]) FROM Movement);
Notice the statement terminator. That is a good habit to develop. You might wonder what happens when your table has no rows. I suggest you try it and see for yourself.

Set Identity ON with a merge statement

I am inserting and deleting elements in a table, as a result, when I want to insert a new element, it takes a new id number, but this id is not taking the last id+1. For example: the last id is 5 and I inserted a 5 elements and deleted after that, the new id will take the value of 11, and I need 6. Here is my code
CREATE TABLE #FC
(
Code varchar(25),
Description varchar(50),
Category varchar(10),
CreatedDate datetime,
LastModifiedDate datetime
);
--Adding just one record
INSERT INTO #FC (Code, Description, Category, CreatedDate, LastModifiedDate)
VALUES ('DELETE_MEMBER', 'Delete Member', 'POLICY', #Now, #Now);
;
SET IDENTITY_INSERT [dbo].[Function_Code] ON;
MERGE
INTO [dbo].[Function_Code] AS T
USING #FC AS S
ON (T.Code = S.Code) AND (T.Description = S.Description) AND(T.Category = S.Category)
WHEN MATCHED THEN
UPDATE SET
[Code] = S.[Code]
, [Description] = S.Description
, [Category] = S.Category
, [CreatedDate] = S.CreatedDate
, [LastModifiedDate] = S.LastModifiedDate
WHEN NOT MATCHED THEN
INSERT (Code, Description, Category, CreatedDate, LastModifiedDate)
VALUES(S.Code, S.Description, S.Category, S.CreatedDate, S.LastModifiedDate)
;
SET IDENTITY_INSERT [dbo].[Function_Code] OFF;
An identity is a technical field that you should not handle yourself. If you want to manage the sequence yourself, then don't use an identity field.
Nevertheless, if you really want to do it, you'll have to reseed the table to the desired value :
DELETE YourTable
DECLARE #n INT;
SELECT #n = MAX(YourId) FROM YourTable
DBCC CHECKIDENT ('YourTable', RESEED, #n)
INSERT YourTable
What you are asking is dangerous. If you make a column an identity column, don't touch it, let sql server do its job. Otherwise you can start getting primary key errors. The identity column is ready to insert 11. You insert six through eleven in your code by running it multiple time and you can get a primary key error next time the identity tries to insert a row into the table.
As Thomas Haratyk said you can reseed your table. Or you can use:
select MAX(YourId) + 1 FROM YourTable
and insert that into your identity column if you are sure you will always insert an id that has already been used by the identity column.
However, if you are commonly overwriting the default identity behavior, it may be better to manage this column yourself because deleting from an identity column results in gaps by default.

INSERT SELECT in Firebird

I'm new to firebird and I have verious issues. I want to insert various lines into a table selected from another table.
Here's the code:
/*CREATE GENERATOR POS; */
SET GENERATOR POS TO 1;
SET TERM ^;
create trigger BAS_pkassign
for MATERIAL
active before insert position 66
EXECUTE BLOCK
AS
declare posid bigint;
select gen_id(POS, 1)
from RDB$DATABASE
into :posid;
BEGIN
END
SET TERM ; ^
INSERT INTO MATERIAL ( /*ID */ LOCATION, POSID, ARTID, ARTIDCONT, QUANTITY )
SELECT 1000, ':posid', 309, BAS_ART.ID, 1
FROM BAS_ART
WHERE BAS_ART.ARTCATEGORY LIKE '%MyWord%'
The ID should autoincrement from 66 on. The posid should autoincrement from 1 on.
Actually it is not inserting anything.
I'm using Firebird Maestro and have just opened the SQL Script Editor (which doesnt throw any error message on executing the script).
Can anybody help me?
Thanks!
Additional information:
The trigger should autoincrement the column "ID" - but I dont know how exactly I can change it so it works.. The ':posid' throws an error using it :posid but like this theres no error (I guess its interpretated as a string). But how do I use it right?
I dont get errors when I execute it. The table structure is easy. I have 2 tables:
1.
Material (
ID (INTEGER),
Location (INTEGER),
POSID (INTEGER),
ARTID (INTEGER),
ARTIDCONT (INTEGER),
QUANTITY (INTEGER),
OTHERCOLUMN (INTEGER))
and the 2. other table
BAS_ART (ID (INTEGER), ARTCATEGORY (VARCHAR255))
-> I want to insert all entries from the table BAS_ART which contain "MyWord" in the column ARTCATEGORY into the MATERIAL table.
I don't understand why you need the trigger at all.
This problem:
I want to insert all entries from the table BAS_ART which contain "MyWord" into the MATERIAL table
Can be solved with a single insert ... select statement.
insert into material (id, location, posid, artid, quantity)
select next value for seq_mat_id, 1000, next value for seq_pos, id, 1
from bas_art
where artcategory = 'My Word';
This assumes that there is a second sequence (aka "generator") that is named seq_mat_id that provides the new id for the column material.id
For most of my answer I will assume a very simple table:
CREATE TABLE MyTable (
ID BIGINT PRIMARY KEY,
SomeValue VARCHAR(255),
posid INTEGER
)
Auto-increment identifier
Firebird (up to version 2.5) does not have an identity column type (this will be added in Firebird 3), instead you need to use a sequence (aka generator) and a trigger to get this.
Sequence
First you need to create a sequence using CREATE SEQUENCE:
CREATE SEQUENCE seqMyTable
A sequence is atomic which means interleaving transactions/connections will not get duplicate values, it is also outside transaction control, which means that a ROLLBACK will not revert to the previous value. In most uses a sequences should always increase, so the value reset you do at the start of your question is wrong for almost all purposes; for example another connection could reset the sequence as well midway in your execution leaving you with unintended duplicates of POSID.
Trigger
To generate a value for an auto-increment identifier, you need to use a BEFORE INSERT TRIGGER that assigns a generated value to the - in this example - ID column.
CREATE TRIGGER trgMyTableAutoIncrement FOR MyTable
ACTIVE BEFORE INSERT POSITION 0
AS
BEGIN
NEW.ID = NEXT VALUE FOR seqMyTable;
END
In this example I always assign a generated value, other examples assign a generated value only when the ID is NULL.
Getting the value
To get the generated value you can use the RETURNING-clause of the INSERT-statement:
INSERT INTO MyTable (SomeValue) VALUES ('abc') RETURNING ID
INSERT INTO ... SELECT
Using INSERT INTO ... SELECT you can select rows from one table and insert them into others. The reason it doesn't work for you is because you are trying to assign the string value ':pos' to a column of type INTEGER, and that is not allowed.
Assuming I have another table MyOtherTable with a similar structure as MyTable I can transfer values using:
INSERT INTO MyTable (SomeValue)
SELECT SomeOtherValue
FROM MyOtherTable
Using INSERT INTO ... SELECT it is not possible to obtain the generated values unless only a single row was inserted.
Guesswork with regard to POSID
It is not clear to me what POSID is supposed to be, and what values it should have. It looks like you want to have an increasing value starting at 1 for a single INSERT INTO ... SELECT. In versions of Firebird up to 2.5 that is not possible in this way (in Firebird 3 you would be able to use ROW_NUMBER() for this).
If my guess is right, then you will need to use an EXECUTE BLOCK (or a stored procedure) to assign and increase the value for every row to be inserted.
The execute block would be something like:
EXECUTE BLOCK
AS
DECLARE posid INTEGER = 1;
DECLARE someothervalue VARCHAR(255);
BEGIN
FOR SELECT SomeOtherValue FROM MyOtherTable INTO :someothervalue DO
BEGIN
INSERT INTO MyTable (SomeValue, posid) VALUES (:someothervalue, :posid);
posid = posid + 1;
END
END
Without an ORDER BY with the SELECT the value of posid is essentially meaningless, because there is no guaranteed order.

Adding max(value)+1 in new row, can this be a problem?

In a SQL Server table I have the following 2 columns:
RowId: primary key, numaric, identity column and auto insert.
MailId: Non key, numaric, non identity and non auto insert.
Mail Id can be duplicate. In case of new MailId I will check max(MailId)+1 and insert it in new row and in case of duplication value will be coming as parameter.
Logic looks fine but here is an issue, I was just considering (yet chacnes of accurance are ver low) In the same time there can be two different new MailId requests. Can this casue logical error ? For example when code checked max(MailId)+1 was 101 and I stored it in a variable but may be before next insert statment executs a new record inserted in table. Now max(MailId)+1 in table will be 102 but value in variable will be 101 ?
Any suggestion please I want to control this error chances as well.
EDIT
(I am not using identity(1,1) because I also have to pass custom values in it)
Why would you use a custom-rolled Identity field when there is such a great one already in SQL Server?
Just use INT Identity (1,1) for your ID field and it will automatically increment each time a row is inserted. It also handles concurrency much better than pretty much anything you could implement manually.
EDIT:
Sample of a manual ID value:
SET IDENTITY_INSERT MyTable ON
INSERT INTO MyTable (IdField, Col1, Col2, Col3,...)
VALUES
(1234, 'Col1', 'Col2', 'Col3',...)
SET IDENTITY_INSERT MyTable OFF
You need to include an explicit field list for the INSERT.
Use OUTPUT on your insert to be sure that you have the right value. If you insert and then select MAX, it is possible that someone could "sneak" in and end up with duplication. That is, you insert MAX + 1, at the same time someone else inserts MAX + 1 then you select MAX and they select MAX, you both have the same value. Whereas if you INSERT and use OUTPUT, you'll be sure that you're unique. This is rarely a problem, but if you have a lot of activity, it can happen (speaking from experience).
EDIT
USE AdventureWorks2008R2;
GO
DECLARE #MyTableVar table(
EmpID int NOT NULL,
OldVacationHours int,
NewVacationHours int,
ModifiedDate datetime);
UPDATE TOP (10) HumanResources.Employee
SET VacationHours = VacationHours * 1.25,
ModifiedDate = GETDATE()
OUTPUT inserted.BusinessEntityID,
deleted.VacationHours,
inserted.VacationHours,
inserted.ModifiedDate
INTO #MyTableVar;
--Display the result set of the table variable.
SELECT EmpID, OldVacationHours, NewVacationHours, ModifiedDate
FROM #MyTableVar;
GO
--Display the result set of the table.
SELECT TOP (10) BusinessEntityID, VacationHours, ModifiedDate
FROM HumanResources.Employee;
GO