I try to find a way to let the SGBD perform a population of merge fields within a long text.
Create the structure :
CREATE TABLE [dbo].[store]
(
[id] [int] NOT NULL,
[text] [nvarchar](MAX) NOT NULL
)
CREATE TABLE [dbo].[statement]
(
[id] [int] NOT NULL,
[store_id] [int] NOT NULL
)
CREATE TABLE [dbo].[statement_merges]
(
[statement_id] [int] NOT NULL,
[merge_field] [nvarchar](30) NOT NULL,
[user_data] [nvarchar](MAX) NOT NULL
)
Now, create test values
INSERT INTO [store] (id, text)
VALUES (1, 'Waw, stackoverflow is an amazing library of lost people in the IT hell, and i have the feeling that $$PERC_SAT$$ of the users found a solution, personally I asked $$ASKED$$ questions.')
INSERT INTO [statement] (id, store_id)
VALUES (1, 1)
INSERT INTO [statement_merges] (statement_id, merge_field, user_data)
VALUES (1, '$$PERC_SAT$$', '85%')
INSERT INTO [statement_merges] (statement_id, merge_field, user_data)
VALUES (1, '$$ASKED$$', '12')
At the time being my app is delivering the final statement, looping through merges, replacing in the stored text and output
Waw, stackoverflow is an amazing library of lost people in the IT
hell, and i have the feeling that 85% of the users found a solution,
personally I asked 12 questions.
I try to find a way to be code-independent and serve the output in a single query, as u understood, select a statement in which the stored text have been populated with user data. I hope I'm clear.
I looked on TRANSLATE function but it looks like a char replacement, so I have two choices :
I try a recursive function, replacing one by one until no merge_fields is found in the calculated text; but I have doubts about the performance of this approach;
There is a magic to do that but I need your knowledge...
Consider that I want this because the real texts are very long, and I don't want to store it more than once in my database. You can imagine a 3 pages contract with only 12 parameters, like start date, invoiced amount, etc... Everything else cant be changed for compliance.
Thank you for your time!
EDIT :
Thanks to Randy's help, this looks to do the trick :
WITH cte_replace_tokens AS (
SELECT replace(r.text, m.merge_field, m.user_data) as [final], m.merge_field, s.id, 1 AS i
FROM store r
INNER JOIN statement s ON s.store_id = r.id
INNER JOIN statement_merges m ON m.statement_id = s.id
WHERE m.statement_id = 1
UNION ALL
SELECT replace(r.final, m.merge_field, m.user_data) as [final], m.merge_field, r.id, r.i + 1 AS i
FROM cte_replace_tokens r
INNER JOIN statement_merges m ON m.statement_id = r.id
WHERE m.merge_field > r.merge_field
)
select TOP 1 final from cte_replace_tokens ORDER BY i DESC
I will check with a bigger database if the performance is good...
At least, I can "populate" one statement, I need to figure out to be able to extract a list as well.
Thanks again !
If a record is updated more than once by the same update, the last wins. None of the updates are affected by the others - no cumulative effect. It is possible to trick SQL using a local variable to get cumulative effects in some cases, but it's tricky and not recommended. (Order becomes important and is not reliable in an update.)
One alternate is recursion in a CTE. Generate a new record from the prior as each token is replaced until there are no tokens. Here is a working example that replaces 1 with A, 2 with B, etc. (I wonder if there is some tricky xml that can do this as well.)
if not object_id('tempdb..#Raw') is null drop table #Raw
CREATE TABLE #Raw(
[test] [varchar](100) NOT NULL PRIMARY KEY CLUSTERED,
)
if not object_id('tempdb..#Token') is null drop table #Token
CREATE TABLE #Token(
[id] [int] NOT NULL PRIMARY KEY CLUSTERED,
[token] [char](1) NOT NULL,
[value] [char](1) NOT NULL,
)
insert into #Raw values('123456'), ('1122334456')
insert into #Token values(1, '1', 'A'), (2, '2', 'B'), (3, '3', 'C'), (4, '4', 'D'), (5, '5', 'E'), (6, '6', 'F');
WITH cte_replace_tokens AS (
SELECT r.test, replace(r.test, l.token, l.value) as [final], l.id
FROM [Raw] r
CROSS JOIN #Token l
WHERE l.id = 1
UNION ALL
SELECT r.test, replace(r.final, l.token, l.value) as [final], l.id
FROM cte_replace_tokens r
CROSS JOIN #Token l
WHERE l.id = r.id + 1
)
select * from cte_replace_tokens where id = 6
It's not recommended to do such tasks inside sql engine but if you want to do that, you need to do it in a loop using cursor in a function or stored procedure like so :
DECLARE #merge_field nvarchar(30)
, #user_data nvarchar(MAX)
, #statementid INT = 1
, #text varchar(MAX) = 'Waw, stackoverflow is an amazing library of lost people in the IT hell, and i have the feeling that $$PERC_SAT$$ of the users found a solution, personally I asked $$ASKED$$ questions.'
DECLARE merge_statements CURSOR FAST_FORWARD
FOR SELECT
sm.merge_field
, sm.user_data
FROM dbo.statement_merges AS sm
WHERE sm.statement_id = #statementid
OPEN merge_statements
FETCH NEXT FROM merge_statements
INTO #merge_field , #user_data
WHILE ##FETCH_STATUS = 0
BEGIN
set #text = REPLACE(#text , #merge_field, #user_data )
FETCH NEXT FROM merge_statements
INTO #merge_field , #user_data
END
CLOSE merge_statements
DEALLOCATE merge_statements
SELECT #text
Here is a recursive solution.
SQL Fiddle
MS SQL Server 2017 Schema Setup:
CREATE TABLE [dbo].[store]
(
[id] [int] NOT NULL,
[text] [nvarchar](MAX) NOT NULL
)
CREATE TABLE [dbo].[statement]
(
[id] [int] NOT NULL,
[store_id] [int] NOT NULL
)
CREATE TABLE [dbo].[statement_merges]
(
[statement_id] [int] NOT NULL,
[merge_field] [nvarchar](30) NOT NULL,
[user_data] [nvarchar](MAX) NOT NULL
)
INSERT INTO store (id, text)
VALUES (1, '$$(*)$$, stackoverflow...$$PERC_SAT$$...$$ASKED$$ questions.')
INSERT INTO store (id, text)
VALUES (2, 'Use The #_#')
INSERT INTO statement (id, store_id) VALUES (1, 1)
INSERT INTO statement (id, store_id) VALUES (2, 2)
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$PERC_SAT$$', '85%')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$ASKED$$', '12')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$(*)$$', 'Wow')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (2, ' #_#', 'Flux!')
Query 1:
;WITH Normalized AS
(
SELECT
store_id=store.id,
store.text,
sm.merge_field,
sm.user_data,
RowNumber = ROW_NUMBER() OVER(PARTITION BY store.id,sm.statement_id ORDER BY merge_field),
statement_id = st.id
FROM
store store
INNER JOIN statement st ON st.store_id = store.id
INNER JOIN statement_merges sm ON sm.statement_id = st.id
)
, Recurse AS
(
SELECT
store_id, statement_id, old_text = text, merge_field,user_data, RowNumber,
Iteration=1,
new_text = REPLACE(text, merge_field, user_data)
FROM
Normalized
WHERE
RowNumber=1
UNION ALL
SELECT
n.store_id, n.statement_id, r.old_text, n.merge_field, n.user_data,
RowNumber=r.RowNumber+1,
Iteration=Iteration+1,
new_text = REPLACE(r.new_text, n.merge_field, n.user_data)
FROM
Normalized n
INNER JOIN Recurse r ON r.RowNumber = n.RowNumber AND r.statement_id = n.statement_id
)
,ReverseOnIteration AS
(
SELECT *,
ReverseIteration = ROW_NUMBER() OVER(PARTITION BY statement_id ORDER BY Iteration DESC)
FROM
Recurse
)
SELECT
store_id, statement_id, new_text, old_text
FROM
ReverseOnIteration
WHERE
ReverseIteration=1
Results:
| store_id | statement_id | new_text | old_text |
|----------|--------------|------------------------------------------|--------------------------------------------------------------|
| 1 | 1 | Wow, stackoverflow...85%...12 questions. | $$(*)$$, stackoverflow...$$PERC_SAT$$...$$ASKED$$ questions. |
| 2 | 2 | Use TheFlux! | Use The #_# |
With the help of Randy, I think I've achieved what I wanted to do !
Known the fact that my real case is a contract, in which there are several statements that may be :
free text
stored text without any merges
stored text with one or
several merges
this CTE does the job !
WITH cte_replace_tokens AS (
-- The initial query dont join on merges neither on store because can be a free text
SELECT COALESCE(r.text, s.part_text) AS [final], CAST('' AS NVARCHAR) AS merge_field, s.id, 1 AS i, s.contract_id
FROM statement s
LEFT JOIN store r ON s.store_id = r.id
UNION ALL
-- We loop till the last merge field, output contains iteration to be able to keep the last record ( all fields updated )
SELECT replace(r.final, m.merge_field, m.user_data) as [final], m.merge_field, r.id, r.i + 1 AS i, r.contract_id
FROM cte_replace_tokens r
INNER JOIN statement_merges m ON m.statement_id = r.id
WHERE m.merge_field > r.merge_field AND r.final LIKE '%' + m.merge_field + '%'
-- spare lost replacements by forcing only one merge_field per loop
AND NOT EXISTS( SELECT mm.statement_id FROM statement_merges mm WHERE mm.statement_id = m.statement_id AND mm.merge_field > r.merge_field AND mm.merge_field < m.merge_field)
)
select s.id,
(select top 1 final from cte_replace_tokens t WHERE t.contract_id = s.contract_id AND t.id = s.id ORDER BY i DESC) as res
FROM statement s
where contract_id = 1
If the CTE solution with a cross join is too slow, an alternate solution would be to build a scalar fn dynamically that has every REPLACE required from the token table. One scalar fn call per record then is order(N). I get the same result as before.
The function is simple and likely not to be too long, depending upon how big the token table becomes...256 MB batch limit. I've seen attempts to dynamically create queries to improve performance backfire - moved the problem to compile time. Should not be a problem here.
if not object_id('tempdb..#Raw') is null drop table #Raw
CREATE TABLE #Raw(
[test] [varchar](100) NOT NULL PRIMARY KEY CLUSTERED,
)
if not object_id('tempdb..#Token') is null drop table #Token
CREATE TABLE #Token(
[id] [int] NOT NULL PRIMARY KEY CLUSTERED,
[token] [char](1) NOT NULL,
[value] [char](1) NOT NULL,
)
insert into #Raw values('123456'), ('1122334456')
insert into #Token values(1, '1', 'A'), (2, '2', 'B'), (3, '3', 'C'), (4, '4', 'D'), (5, '5', 'E'), (6, '6', 'F');
DECLARE #sql varchar(max) = 'CREATE FUNCTION dbo.fn_ReplaceTokens(#raw varchar(8000)) RETURNS varchar(8000) AS BEGIN RETURN ';
WITH cte_replace_statement AS (
SELECT a.id, CAST('replace(#raw,''' + a.token + ''',''' + a.value + ''')' as varchar(max)) as [statement]
FROM #Token a
WHERE a.id = 1
UNION ALL
SELECT n.id, CAST(replace(l.[statement], '#raw', 'replace(#raw,''' + n.token + ''',''' + n.value + ''')') as varchar(max)) as [statement]
FROM #Token n
INNER JOIN cte_replace_statement l
ON n.id = l.id + 1
)
select #sql += [statement] + ' END' from cte_replace_statement where id = 6
print #sql
if not object_id('dbo.fn_ReplaceTokens') is null drop function dbo.fn_ReplaceTokens
execute (#sql)
SELECT r.test, dbo.fn_ReplaceTokens(r.test) as [final] FROM [Raw] r
I'm trying to create my own logic for tables synchronization in SQL Server Express 2019. I was hoping that such simple task would work:
Have a Customers table
Have a Synchronization table
CREATE TABLE [dbo].[Synchronization]
(
[PK] [uniqueidentifier] NOT NULL,
[TableName] [nchar](50) NOT NULL,
[RecordPK] [uniqueidentifier] NOT NULL,
[RecordChecksum] [int] NOT NULL,
[RecordDate] [datetime] NOT NULL,
[RecordIsDeleted] [bit] NOT NULL
)
Have a trigger on Customers:
CREATE TRIGGER trgCustomers_INSERT
ON Customers
AFTER INSERT
AS
INSERT INTO Synchronization(PK, TableName, RecordPK, RecordChecksum,
RecordDate, RecordIsDeleted)
VALUES (NEWID(), 'Customers',
(SELECT PK FROM inserted),
(SELECT CHECKSUM(*) FROM inserted),
GETDATE(), 0)
... but I got an error about the SELECT CHECKSUM(*) FROM inserted part:
Cannot use CHECKSUM(*) in a computed column, constraint, default definition, or INSERT statement.
Is there any other way to add new Customer's CHECKSUM or some hash to the Synchronization table?
Don't use the VALUES syntax when inserting and you won't get an error using CHECKSUM while inserting.
Example:
declare #t table (val int)
-- works
insert into #t select checksum(*) from ( select ID from (select 1 as ID union select 2) b ) a
-- reproduce error
insert into #t
values
((select top 1 checksum(*) C from ( select ID from (select 1 as ID union select 2) b ) a))
Implementing the concept in your trigger:
CREATE TRIGGER trgCustomers_INSERT
ON Customers
AFTER INSERT
AS
begin
INSERT INTO Synchronization(PK, TableName, RecordPK, RecordChecksum,
RecordDate, RecordIsDeleted)
select NEWID() as PK,
'Customers' as TableName,
PK as RecordPK,
checksum(*) as RecordChecksum,
GETDATE() as RecordDate,
0 as RecordIsDeleted
from inserted
end
I need to update existing MS SQL indexed view by including additional columns values from newly created table.
Indexed view:
CREATE OR ALTER VIEW [dbo].[MySelectionInfo]
WITH schemabinding
AS
SELECT C.Id id0,
C.Code Code1,
C.Name Name2,
C.ProgramLevel Level3,
C.Department Department4,
C.City City10,
C.STATE State11,
C.StartDate StartDate12,
C.Deadline Deadline13,
B.ID Table_B_ID,
A.Id Table_A_ID
FROM dbo.Table_A A
INNER JOIN dbo.Table_B B ON A.id = B.Table_A_Id
INNER JOIN dbo.Table_C C ON C.Table_B_Id = B.Id
New table:
CREATE TABLE [dbo].[Table_D] (
[Id] [int] IDENTITY (1, 1) PRIMARY KEY NOT NULL,
[ModelName] [varchar](max) NOT NULL,
[Table_C_Id] [int] NOT NULL,
[AttributeValue] [varchar](max) NOT NULL,
[CreatedDate] [datetime] NOT NULL,
[UpdatedDate] [datetime] NOT NULL,
CONSTRAINT FK_Table_C_Id FOREIGN KEY (Table_C_Id) REFERENCES some_schema.dbo.[Table_C] (Id)
ON DELETE CASCADE
ON UPDATE CASCADE
)
Data in the new table:
I want to include only some of the ModelName column values as a column names and AttributeValue as values in the select * from [dbo].[MySelectionInfo] result set:
I can achieve the desired result using the PIVOT function:
CREATE OR ALTER VIEW [dbo].[MySelectionInfo]
WITH schemabinding
AS
SELECT C.Id id0,
C.Code Code1,
C.Name Name2,
C.StartDate StartDate12,
C.Deadline Deadline13,
B.ID Table_B_ID,
A.Id Table_A_ID
FROM dbo.Table_A A
INNER JOIN dbo.Table_B B ON A.id = B.Table_A_Id
INNER JOIN dbo.Table_C C ON C.Table_B_Id = B.Id
LEFT JOIN (SELECT PivotTable.Table_C_Id,
PivotTable.attribute1,
PivotTable.attribute2,
PivotTable.attribute3
FROM (SELECT Table_D.Table_C_Id,
Table_D.ModelName,
Table_D.AttributeValue
FROM dbo.Table_C
INNER JOIN dbo.Table_D
ON Table_C.Id = Table_D.Table_C_Id) AS sourceTable
PIVOT (
Max(AttributeValue) FOR ModelName IN (attribute1, attribute2, attribute3)
) AS PivotTable) dbo.Table_D D ON D.Table_C_Id = C.Id
But, after running the SQL statement above, I am not be able to create the clustered index for the view, because LEFT JOIN, PIVOT, MAX are prohibited to be used in the indexed views.
Question: Is there any other solutions to achieve the desired result and still have an existing view as an Indexed view?
if your data allow it, you could cross tab with case statements instead of PIVOTing:
/*
drop view dbo.mytestinfoview;
go
drop table dbo.Table1;
go
--*/
create table dbo.Table1
(
id int,
ModelName varchar(20),
AttributeValue int
);
insert into dbo.Table1(id, ModelName, AttributeValue)
select distinct o.object_id, concat('attribute', v.id), o.attributevalue
from
(
select
object_id ,
abs(checksum(newid())) % 2 as attributevalue
from sys.objects
) as o
cross apply (values(1), (2), (3), (4)) as v(id)
go
create or alter view dbo.mytestinfoview
with schemabinding
as
select id, count_big(*) as thecounter,
sum(isnull(case ModelName when 'attribute1' then AttributeValue end, 0)) as attribute1,
sum(isnull(case ModelName when 'attribute2' then AttributeValue else 0 end, 0)) as attribute2,
sum(isnull(case ModelName when 'attribute3' then AttributeValue else 0 end, 0)) as attribute3
from dbo.Table1
group by id
go
create unique clustered index idx_v1 on dbo.mytestinfoview(id);
go
select * from dbo.mytestinfoview;
go
I'm trying to declare a table variable and then join it to a table I created in the database. Every time I try to insert my "NAME" field into my table, I get the error 'Invalid Column Name "NAME"', even though the GNAME field works fine. What am I doing wrong, and how can I join me NAME column?
DECLARE #Names TABLE
(
ID INT,
NAME VARCHAR(100),
GNAME VARCHAR(100)
)
INSERT INTO #Names
(
ID,
NAME,
GNAME
)
SELECT
CName.ID,
Ref.NAME,
Ref.GNAME
FROM
#CurrentPositions AS CName
LEFT OUTER JOIN
dbo.NameField AS Ref
ON
CName.ID = Ref.ID
IF ( OBJECT_ID('dbo.ReportTable', 'U') IS NOT NULL)
DROP TABLE dbo.ReportTable
CREATE TABLE [dbo].[ReportTable]
(
[ID_NUMBER] [INT],
[NAME] [VARCHAR](150)
[GNAME] [VARCHAR](150)
)
INSERT INTO [dbo].[ReportTable]
(
ID_NUMBER,
NAME,
GNAME
)
SELECT
C.ID_NUMBER,
N.NAME,
N.GNAME
FROM
#Names AS N
INNER JOIN
#CurrentPositions AS C
ON N.ID_NUMBER = C.ID_NUMBER
Try using a Temp table :
CREATE TABLE #Names
(
ID INT,
NAME VARCHAR(100),
GNAME VARCHAR(100)
)
INSERT INTO #Names
(
ID,
NAME,
GNAME
)
SELECT
CName.ID,
Ref.NAME,
Ref.GNAME
FROM
#CurrentPositions AS CName
LEFT OUTER JOIN
dbo.NameField AS Ref
ON
CName.ID = Ref.ID
IF ( OBJECT_ID('dbo.ReportTable', 'U') IS NOT NULL)
DROP TABLE dbo.ReportTable
CREATE TABLE [dbo].[ReportTable]
(
[ID_NUMBER] [INT],
[NAME] [VARCHAR](150)
[GNAME] [VARCHAR](150)
)
INSERT INTO [dbo].[ReportTable]
(
ID_NUMBER,
NAME,
GNAME
)
SELECT
C.ID_NUMBER,
N.NAME,
N.GNAME
FROM
#Names AS N
INNER JOIN
#CurrentPositions AS C
ON N.ID_NUMBER = C.ID_NUMBER
I've assumed that you will also change the table variable #CurrentPositions to a temp table
Just remember to drop the tables after you use them.
It is quite possible that all you need to do is wrap your field names in square brackets, e.g.
INSERT INTO #Names
(
[ID],
[NAME],
[GNAME]
)
SELECT
CName.[ID],
Ref.[NAME],
Ref.[GNAME]
FROM
#CurrentPositions AS CName
LEFT OUTER JOIN
dbo.NameField AS Ref
ON
CName.[ID] = Ref.[ID]
If that doesn't fix it, please post the schema of your #CurrentPositions and dbo.NameField tables.
I am trying to insert multiple records (~250) in a table (say MyTable) and would like to insert a new row only if it does not exist already.
I am using SQL Server 2008 R2 and got help from other threads like SQL conditional insert if row doesn't already exist.
While I am able to achieve that with following stripped script, I would like to know if there is a better (short) way to do this as I
have to repeat this checking for every row inserted. Since we need to execute this script only once during DB deployment, I am not too much
worried about performance.
INSERT INTO MyTable([Description], [CreatedDate], [CreatedBy], [ModifiedDate], [ModifiedBy], [IsActive], [IsDeleted])
SELECT N'ababab', GETDATE(), 1, NULL, NULL, 1, 0
WHERE NOT EXISTS(SELECT * FROM MyTable WITH (ROWLOCK, HOLDLOCK, UPDLOCK)
WHERE
([InstanceId] IS NULL OR [InstanceId] = 1)
AND [ChannelPartnerId] IS NULL
AND [CreatedBy] = 1)
UNION ALL
SELECT N'xyz', 1, GETDATE(), 1, NULL, NULL, 1, 0
WHERE NOT EXISTS(SELECT * FROM [dbo].[TemplateQualifierCategoryMyTest] WITH (ROWLOCK, HOLDLOCK, UPDLOCK)
WHERE
([InstanceId] IS NULL OR [InstanceId] = 1)
AND [ChannelPartnerId] IS NULL
AND [CreatedBy] = 1)
-- More SELECT statements goes here
You could create a temporary table with your descriptions, then insert them all into the MyTable with a select that will check for rows in the temporary table that is not yet present in your destination, (this trick in implemented by the LEFT OUTER JOIN in conjunction with the IS NULL for the MyTable.Description part in the WHERE-Clause):
DECLARE #Descriptions TABLE ([Description] VARCHAR(200) NOT NULL )
INSERT INTO #Descriptions ( Description )VALUES ( 'ababab' )
INSERT INTO #Descriptions ( Description )VALUES ( 'xyz' )
INSERT INTO dbo.MyTable
( Description ,
CreatedDate ,
CreatedBy ,
ModifiedDate ,
ModifiedBy ,
IsActive ,
IsDeleted
)
SELECT d.Description, GETDATE(), 1, NULL, NULL, 1, 0
FROM #Descriptions d
LEFT OUTER JOIN dbo.MyTable mt ON d.Description = mt.Description
WHERE mt.Description IS NULL