I have the following tables:
CREATE TABLE [dbo].[articles](
[ID] [int] IDENTITY(1,1) NOT NULL,
[name] [nvarchar](200) NOT NULL,
CONSTRAINT [PK_articles] PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
✓
CREATE TABLE [dbo].[test_users](
[ID] [int] IDENTITY(1,1) NOT NULL,
[GUID] [nvarchar](100) NOT NULL,
CONSTRAINT [PK_test_users] PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
CONSTRAINT [unique_guid] UNIQUE NONCLUSTERED
(
[GUID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
✓
CREATE TABLE [dbo].[relational](
[ID] [int] IDENTITY(1,1) NOT NULL,
[articleid] [int] NOT NULL,
[userguid] [nvarchar](100) NOT NULL,
CONSTRAINT [PK_relational] PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
ALTER TABLE [dbo].[relational] WITH CHECK ADD CONSTRAINT [FK_relational_articles] FOREIGN KEY([articleid])
REFERENCES [dbo].[articles] ([ID])
ALTER TABLE [dbo].[relational] CHECK CONSTRAINT [FK_relational_articles]
ALTER TABLE [dbo].[relational] WITH CHECK ADD CONSTRAINT [FK_relational_relational] FOREIGN KEY([userguid])
REFERENCES [dbo].[test_users] ([GUID])
ALTER TABLE [dbo].[relational] CHECK CONSTRAINT [FK_relational_relational]
GO
✓
CREATE TABLE [dbo].[seperated](
[ID] [int] IDENTITY(1,1) NOT NULL,
[userguid] [nvarchar](100) NOT NULL,
[articles] [nvarchar](max) NOT NULL,
CONSTRAINT [PK_seperated] PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
ALTER TABLE [dbo].[seperated] WITH CHECK ADD CONSTRAINT [FK_seperated_test_users] FOREIGN KEY([userguid])
REFERENCES [dbo].[test_users] ([GUID])
ALTER TABLE [dbo].[seperated] CHECK CONSTRAINT [FK_seperated_test_users]
GO
✓
SET IDENTITY_INSERT [dbo].[test_users] ON
INSERT [dbo].[test_users] ([ID], [GUID]) VALUES (1, N'guid1')
INSERT [dbo].[test_users] ([ID], [GUID]) VALUES (2, N'guid2')
INSERT [dbo].[test_users] ([ID], [GUID]) VALUES (3, N'guid3')
INSERT [dbo].[test_users] ([ID], [GUID]) VALUES (4, N'guid4')
SET IDENTITY_INSERT [dbo].[test_users] OFF
GO
4 rows affected
SELECT * FROM [dbo].[test_users]
GO
ID | GUID
-: | :----
1 | guid1
2 | guid2
3 | guid3
4 | guid4
SET IDENTITY_INSERT [dbo].[articles] ON
INSERT [dbo].[articles] ([ID], [name]) VALUES (1, N'article1')
INSERT [dbo].[articles] ([ID], [name]) VALUES (2, N'article2')
INSERT [dbo].[articles] ([ID], [name]) VALUES (3, N'article3')
SET IDENTITY_INSERT [dbo].[articles] OFF
GO
3 rows affected
SELECT * FROM [dbo].[articles]
GO
ID | name
-: | :-------
1 | article1
2 | article2
3 | article3
SET IDENTITY_INSERT [dbo].[relational] ON
INSERT [dbo].[relational] ([ID], [articleid], [userguid]) VALUES (1, 1, N'guid1')
INSERT [dbo].[relational] ([ID], [articleid], [userguid]) VALUES (2, 1, N'guid2')
INSERT [dbo].[relational] ([ID], [articleid], [userguid]) VALUES (3, 2, N'guid2')
INSERT [dbo].[relational] ([ID], [articleid], [userguid]) VALUES (4, 1, N'guid3')
INSERT [dbo].[relational] ([ID], [articleid], [userguid]) VALUES (5, 3, N'guid3')
SET IDENTITY_INSERT [dbo].[relational] OFF
GO
5 rows affected
SELECT * FROM [dbo].[relational]
GO
ID | articleid | userguid
-: | --------: | :-------
1 | 1 | guid1
2 | 1 | guid2
3 | 2 | guid2
4 | 1 | guid3
5 | 3 | guid3
SET IDENTITY_INSERT [dbo].[seperated] ON
INSERT [dbo].[seperated] ([ID], [userguid], [articles]) VALUES (1, N'guid1', N'1')
INSERT [dbo].[seperated] ([ID], [userguid], [articles]) VALUES (3, N'guid2', N'1,2')
INSERT [dbo].[seperated] ([ID], [userguid], [articles]) VALUES (4, N'guid3', N'1,3')
INSERT [dbo].[seperated] ([ID], [userguid], [articles]) VALUES (5, N'guid4', N'')
SET IDENTITY_INSERT [dbo].[seperated] OFF
GO
4 rows affected
SELECT * FROM [dbo].[seperated]
GO
ID | userguid | articles
-: | :------- | :-------
1 | guid1 | 1
3 | guid2 | 1,2
4 | guid3 | 1,3
5 | guid4 |
db<>fiddle here
On the article table there are triggers for insert, delete and update.
After an insert, records are created in the relational table (only for the new article)
After an delete, records are removed in the relational table (only for the delete article)
After an update, records are created/removed (only for the updated article)
Afterwards the seperated table needs to be updated.
For each user there must be exactly 1 record, with comma seperated articles (basically user.guid, SELECT * FROM relational WHERE userguid = user.guid)
At the moment the seperated table gets completely updated using XML FOR PATH, after each insert/delete/update.
However this is no longer a viable option, since this operation is not fast enough (~5 - 10 million records in the relational table)
To fix this, I would like to update the seperated table only for the new/updated/deleted article:
Inside the trigger I have declared the articleid with #articleId.
On insert: add ',articleId' or add '#articleId' if the row is empty (cannot be null)
On delete: remove the articleId from all rows
On update: remove when needed and add when needed (depends on what is found in the relational table)
As I basically never work with comma seperated stuff, I have no idea how to do this.
Any help would be greatly appreciated :)
Notes:
No I cannot do this inside the application as the application that creates the article records cannot connect to the database (it uses some obscure API), and it is also not possible due to the network.
The amount of inserts, updates and deletes is quite low (thus we can get away with doing this in the trigger)
I HAVE to use a comma seperated structure, I cannot change the model, as the application that runs on top of this database is a 3rd party application, which we cannot change and neither cannot ask to be modified...
Yes, these tables are minified, not all columns are here as they do not matter for this question
Good day,
At the moment the seperated table gets completely updated using XML FOR PATH, after each insert/delete/update.
This make no sense in first glance...
if you edit users x,y,z then you should only update these rows in the [seperated] table. I see no explanation why you update all rows.
SQL Server is a tabular database and even so it supports unstructured data like XML and JSON, it usually do not perform best when we need to parse the values.
You have all the data that you need for the [seperated] table in the [relational] table. So there is no sense in parsing each value.
What do you care what is the current value? if you change the the value (add or remove ID) then simply update the value (string) in the [seperated] table using the information from the [relational] table filter by the userguid which were changed (using the inserted/deleted logical table).
1+2 can be done using "UPDATE FROM SELECT". You can get the list of the IDs using simple query with STRING_AGG function. Do not parse the content, but simply update (replace if you want) and SET the value of the column [articles] for all the rows that were changed (and only these who changed) which where inserted/deleted
On insert: add ',articleId' or add '#articleId' if the row is empty (cannot be null)
On delete: remove the articleId from all rows
On update: remove when needed and add when needed (depends on what is found in the relational table)
Same issue. Do not parse the values for each case. Adding IDs is simple but removing can be more complex. There is no reason to "add" or "remove" anything from the string. Simply update the value of the string to a new value which include all the relevant IDs each time.
Note! If you will provide your code instead of stories then we can provide a working example. You should provide all the queries to reproduce the entire scenario including the triggers and your old and new solutions/attempts
Approach 1 (simple solution, best for remove IDs and in some cases for all tasks): UPDATE the rows in [seperated], which were changed in [relational] using the [inserted] and [deleted] tables
In general the query should be something like bellow (remember that this is done AFTER the [relational] table was updated - I read the data from that table and re-create the value of the rows that were changed)
-------------------------------------------------- Replace
--> assuming that each user has a row in [seperated] even if no id (as described)
;With NyCTE as (
SELECT [userguid], SA = STRING_AGG([articleid], ',')
WITHIN GROUP (ORDER BY [articleid] ASC)
FROM dbo.[relational]
where userguid in (select userguid from inserted)
GROUP BY [userguid]
)
UPDATE [seperated] SET articles = NyCTE.SA
FROM[seperated]
INNER JOIN NyCTE ON NyCTE.userguid = [seperated].userguid
GO
--------------------------------------------------
Approach 2 (Fits only for adding IDs and in most cases it is best approach since we do need to aggregate the [relational] table but only the inserted table): ADDing Ids to existing values of the rows in [seperated], which were changed in [relational] using the [inserted] and [deleted] tables
If you want to use the approach of adding IDs to existing string (remember that SQL Server do not add or edit the string but replace it anyway when we use UPDATE.
---------------------------------------------------- Add
;With NyCTE as (
SELECT [userguid], SA = STRING_AGG([articleid], ',')
WITHIN GROUP (ORDER BY [articleid] ASC)
FROM dbo.inserted
where userguid in (select userguid from inserted)
GROUP BY [userguid]
)
UPDATE [seperated] SET articles = REPLACE(REPLACE('#' + CONCAT (articles,',', SA), '#,', ''), '#', '')
FROM[seperated]
INNER JOIN NyCTE ON NyCTE.userguid = [seperated].userguid
GO
--------------------------------------------------
---------- More Information --------------
Missing information and clarification
On the article table there are triggers for insert, delete and update.
Please provide the full code to reproduce the issue (missing the code for the triggers). Always prefer to use code over stories in technical forums.
At the moment the seperated table gets completely updated using XML FOR PATH
Where is the code?!?
Again, you give stories instead of code.
Inside the trigger I have declared the articleid with #articleId...
AGAIN! You provide stories instead of sample code. Show us what you do. Provide the code which you tested :-(
It sound like you planing to work with loop which is probably really bad idea, or even worse you plan a solution only to solve one row change. What oi you get 100k rows to insert?!?
The trigger should probably work with the "INSERTED" logical table and use simple JOIN, EXCEPT, or INTERSECT for the task. You should plan the trigger to work on SET of data and not single row.
On insert: add ',articleId' or add '#articleId' if the row is empty (cannot be null). On delete: remove the articleId from all rows. On update: remove when needed and add when needed (depends on what is found in the relational table)
SQL Server do not "ADD" string to a value in the table. It can only UPDATE the entire value. It can parse the value in advance and then it uses UPDATE and replace the value in the table. As mentioned SQL Server designed for best performance on relational model and not for parsing simple string in a loop.
Note! You should confirm that this is really a new id each time if you want to be safe. If you do this then you got more complex task.
Related
I have trouble in copying table data and structure to another table since I want to keep the Id column of Identity column and keep its oridinal value instead of starting from 1
I use below sql to insert all data except the ID column from MY_TABLE to MY_TABLE_NEW since it has error saying that
Only when the column list is used and IDENTITY_INSERT is ON, an explicit value can be specified for the identity column in the table'My_TABLE_NEW'.
But I have set it like below SQL:
IF NOT EXISTS (select * from sys.objects where name = 'My_TABLE_NEW')
BEGIN
CREATE TABLE [dbo].[My_TABLE_NEW]
(
[ID] [int] IDENTITY(1,1) NOT NULL,
[OBJECT_ID] [int] NOT NULL,
[YEAR_MONTH] [int] NOT NULL,
CONSTRAINT [PK_My_TABLE_NEW]
PRIMARY KEY CLUSTERED ([ID] ASC)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF,
IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON,
ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
END
GO
IF EXISTS (SELECT * FROM sys.objects WHERE name = 'My_TABLE_NEW')
BEGIN
SET IDENTITY_INSERT My_TABLE_NEW ON
INSERT INTO My_TABLE_NEW
SELECT [ID]
,[OBJECT_ID]
,[YEAR_MONTH]
FROM My_TABLE
SET IDENTITY_INSERT My_TABLE_NEW OFF
END
GO
What is the problem?
Try your insert with the column names:
INSERT INTO My_TABLE_NEW ([ID], [OBJECT_ID], [YEAR_MONTH])
SELECT [ID]
,[OBJECT_ID]
,[YEAR_MONTH]
FROM My_TABLE
From the documentation:
When an existing identity column is selected into a new table, the new column inherits the IDENTITY property, unless one of the following conditions is true:
The SELECT statement contains a join, GROUP BY clause, or aggregate function.
Multiple SELECT statements are joined by using UNION.
The identity column is listed more than one time in the select list.
The identity column is part of an expression.
The identity column is from a remote data source.
That means you can copy the table with SELECT INTO while retaining the identity column and can just add the PK after.
SELECT *
INTO My_TABLE_NEW
FROM My_TABLE
Here is a demo with fiddle.
You can use the built-in tool sp_rename for this, as long as you are just renaming the table not trying to create a copy of it.
EXEC sp_rename 'My_TABLE', 'My_TABLE_NEW'
GO;
https://learn.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sp-rename-transact-sql?view=sql-server-ver15
If you want to create a copy, then you just have to do the following:
SELECT *
INTO My_TABLE_NEW
FROM My_TABLE
Note: If you do this, you will have to re-add any key constraints, computed value columns, etc.
i have this sql table
CREATE TABLE Notes(
NoteID [int] IDENTITY(1,1) NOT NULL,
NoteTitle [nvarchar](255) NULL,
NoteDescription [nvarchar](4000) NULL
) CONSTRAINT [PK_Notes] PRIMARY KEY CLUSTERED
(
NoteID ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
And i want to copy records from a temporary table INCLUDING the NoteID(using sql query)..
this is my script:
SET IDENTITY_INSERT Notes OFF
INSERT INTO Notes (NoteID, NoteTitle,NoteDescription)
SELECT NoteID, NoteTitle,NoteDescription from Notes_Temp
SET IDENTITY_INSERT Notes ON
with this script, i'm getting an error:
Cannot insert explicit value for identity column in table 'Notes' when IDENTITY_INSERT is set to OFF.
is there other way of insert records to a table with identity column using sql query?
Change the OFF and ON around
SET IDENTITY_INSERT Notes ON
INSERT INTO Notes (NoteID, NoteTitle,NoteDescription)
SELECT NoteID, NoteTitle,NoteDescription from Notes_Temp
SET IDENTITY_INSERT Notes OFF
SET IDENTITY_INSERT Notes ON
INSERT INTO Notes
/*Note the column list is REQUIRED here, not optional*/
(NoteID, NoteTitle,NoteDescription)
SELECT NoteID, NoteTitle,NoteDescription from Notes_Temp
SET IDENTITY_INSERT Notes OFF
You're inserting values for NoteId that is an identity column.
You can turn on identity insert on the table like this so that you can specify your own identity values.
Presumably you are using SQL Server (you don't say) and have misunderstood the meaning and purpose of IDENTITY_INSERT. In general, you are not allowed to explicitly set the value of an IDENTITY column, but by setting IDENTITY_INSERT to ON for a table you can temporarily permit such inserts.
I have the following SQL command:
ALTER TABLE dbo.UserProfiles
ADD ChatId UniqueIdentifier NOT NULL,
UNIQUE(ChatId),
CONSTRAINT "ChatId_default" SET DEFAULT newid()
I want to be able to make this column unique, and I want it to be able to generate a new guid every time a row is added to the table. This column is not an IDENTITY column because I already have one. This is something separate. How would I go about adding this column to a table with users already in it.
see this sample:
create table test (mycol UniqueIdentifier NOT NULL default newid(), name varchar(100))
insert into test (name) values ('Roger Medeiros')
select * from test
for add a not null field on a populated table you need this.
alter table test add mycol2 UniqueIdentifier NOT NULL default newid() with values
CREATE UNIQUE NONCLUSTERED INDEX IX_test ON dbo.test
(
mycol
) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
Don't use newid() as default, instead use newsequentialid(). newid() creates a lot of fragmentation and that's bad for indexes.
As far as adding the new column to a table with existing data, simply do this:
ALTER TABLE your_table
ADD your_column UNIQUEIDENTIFIER DEFAULT newsequentialid() NOT null
I have a simple table with tax rates
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[TaxRates](
[Id] [bigint] IDENTITY(1,1) NOT NULL,
[Name] [nvarchar](50) NULL,
CONSTRAINT [PK_TaxRates] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
If user deleted record I want to not change autoincrementer while next insert.
To have more clearance.
Now I have 3 records with id 0,1 and 2. When I delete row with Id 2 and some time later I add next tax rate I want to have records in this table like before 0,1,2.
There shouldn't be chance to have a gap like 0,1,2,4,6. It must be trigger.
Could you help with that?
You need to accept gaps or don't use IDENTITY
id should have no external meaning
You can't update IDENTITY values
IDENTITY columns will always have gaps
In this case you'd update the clustered Pk which will be expensive
What about foreign keys? you'd need a CASCADE
Contiguous numbers can be generated with ROW_NUMBER() at read time
Without IDENTITY (whether you load this table or another) won't be concurrency-safe under load
Trying to INSERT into a gap (by an INSTEAD OF trigger) won't be concurrency-safe under load
(Edit) History tables may have the deleted values anyway
An option, if the identity column has become something passed around in your organization is to duplicate that column into a non-identity column on the same table and you can modify those new id values at will while retaining the actual identity field.
turning identity_insert on and off can allow you to insert identity values.
I have a following table
CREATE TABLE [dbo].[test_table]
(
[ShoppingCartID] [int] IDENTITY(1,1) NOT NULL,
[CartTimeoutInMinutes] [int] NOT NULL,
[MaximumOrderLimitPerUser] [int] NOT NULL,
[MaximumOrderLimitPerSession] [int] NOT NULL,
CONSTRAINT [PK_test_table] PRIMARY KEY CLUSTERED
(
[ShoppingCartID] ASC
)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
)
ON [PRIMARY]
GO
Sometimes Identity isn't working, it's start with 0 and sometimes its start with 1.
Thank you in advance.
How are you putting the data in there? If you are using regular INSERT it should start at 1. You can, however, bulk-insert into the table, or otherwise use identity-insert; in which case all bets are off:
create table test (
id int not null identity(1,1),
name varchar(20) not null)
set identity_insert test on
insert test (id, name) values (0, 'abc')
insert test (id, name) values (27, 'def')
set identity_insert test off
select * from test
with output:
id name
----------- --------------------
0 abc
27 def
Or is the problem relating to ##IDENTITY (in which case: use SCOPE_IDENTITY() instead).
Possible
Are you using DBCC CHECKIDENT? This is invoked by some data compare tools (eg Red Gate) and has the following behaviour:
DBCC CHECKIDENT ( table_name, RESEED, new_reseed_value )
Current identity value is set to the new_reseed_value.
If no rows have been inserted into the table since the table was created, or if all rows have been removed by using the TRUNCATE TABLE statement, the first row inserted after you run DBCC CHECKIDENT uses new_reseed_value as the identity. Otherwise, the next row inserted uses new_reseed_value + the current increment value.
Or: are you using SET IDENTITY_INSERT?
These assume you are looking at the table, rather then using ##IDENTITY (as Mark suggested)