Related
I try to find a way to let the SGBD perform a population of merge fields within a long text.
Create the structure :
CREATE TABLE [dbo].[store]
(
[id] [int] NOT NULL,
[text] [nvarchar](MAX) NOT NULL
)
CREATE TABLE [dbo].[statement]
(
[id] [int] NOT NULL,
[store_id] [int] NOT NULL
)
CREATE TABLE [dbo].[statement_merges]
(
[statement_id] [int] NOT NULL,
[merge_field] [nvarchar](30) NOT NULL,
[user_data] [nvarchar](MAX) NOT NULL
)
Now, create test values
INSERT INTO [store] (id, text)
VALUES (1, 'Waw, stackoverflow is an amazing library of lost people in the IT hell, and i have the feeling that $$PERC_SAT$$ of the users found a solution, personally I asked $$ASKED$$ questions.')
INSERT INTO [statement] (id, store_id)
VALUES (1, 1)
INSERT INTO [statement_merges] (statement_id, merge_field, user_data)
VALUES (1, '$$PERC_SAT$$', '85%')
INSERT INTO [statement_merges] (statement_id, merge_field, user_data)
VALUES (1, '$$ASKED$$', '12')
At the time being my app is delivering the final statement, looping through merges, replacing in the stored text and output
Waw, stackoverflow is an amazing library of lost people in the IT
hell, and i have the feeling that 85% of the users found a solution,
personally I asked 12 questions.
I try to find a way to be code-independent and serve the output in a single query, as u understood, select a statement in which the stored text have been populated with user data. I hope I'm clear.
I looked on TRANSLATE function but it looks like a char replacement, so I have two choices :
I try a recursive function, replacing one by one until no merge_fields is found in the calculated text; but I have doubts about the performance of this approach;
There is a magic to do that but I need your knowledge...
Consider that I want this because the real texts are very long, and I don't want to store it more than once in my database. You can imagine a 3 pages contract with only 12 parameters, like start date, invoiced amount, etc... Everything else cant be changed for compliance.
Thank you for your time!
EDIT :
Thanks to Randy's help, this looks to do the trick :
WITH cte_replace_tokens AS (
SELECT replace(r.text, m.merge_field, m.user_data) as [final], m.merge_field, s.id, 1 AS i
FROM store r
INNER JOIN statement s ON s.store_id = r.id
INNER JOIN statement_merges m ON m.statement_id = s.id
WHERE m.statement_id = 1
UNION ALL
SELECT replace(r.final, m.merge_field, m.user_data) as [final], m.merge_field, r.id, r.i + 1 AS i
FROM cte_replace_tokens r
INNER JOIN statement_merges m ON m.statement_id = r.id
WHERE m.merge_field > r.merge_field
)
select TOP 1 final from cte_replace_tokens ORDER BY i DESC
I will check with a bigger database if the performance is good...
At least, I can "populate" one statement, I need to figure out to be able to extract a list as well.
Thanks again !
If a record is updated more than once by the same update, the last wins. None of the updates are affected by the others - no cumulative effect. It is possible to trick SQL using a local variable to get cumulative effects in some cases, but it's tricky and not recommended. (Order becomes important and is not reliable in an update.)
One alternate is recursion in a CTE. Generate a new record from the prior as each token is replaced until there are no tokens. Here is a working example that replaces 1 with A, 2 with B, etc. (I wonder if there is some tricky xml that can do this as well.)
if not object_id('tempdb..#Raw') is null drop table #Raw
CREATE TABLE #Raw(
[test] [varchar](100) NOT NULL PRIMARY KEY CLUSTERED,
)
if not object_id('tempdb..#Token') is null drop table #Token
CREATE TABLE #Token(
[id] [int] NOT NULL PRIMARY KEY CLUSTERED,
[token] [char](1) NOT NULL,
[value] [char](1) NOT NULL,
)
insert into #Raw values('123456'), ('1122334456')
insert into #Token values(1, '1', 'A'), (2, '2', 'B'), (3, '3', 'C'), (4, '4', 'D'), (5, '5', 'E'), (6, '6', 'F');
WITH cte_replace_tokens AS (
SELECT r.test, replace(r.test, l.token, l.value) as [final], l.id
FROM [Raw] r
CROSS JOIN #Token l
WHERE l.id = 1
UNION ALL
SELECT r.test, replace(r.final, l.token, l.value) as [final], l.id
FROM cte_replace_tokens r
CROSS JOIN #Token l
WHERE l.id = r.id + 1
)
select * from cte_replace_tokens where id = 6
It's not recommended to do such tasks inside sql engine but if you want to do that, you need to do it in a loop using cursor in a function or stored procedure like so :
DECLARE #merge_field nvarchar(30)
, #user_data nvarchar(MAX)
, #statementid INT = 1
, #text varchar(MAX) = 'Waw, stackoverflow is an amazing library of lost people in the IT hell, and i have the feeling that $$PERC_SAT$$ of the users found a solution, personally I asked $$ASKED$$ questions.'
DECLARE merge_statements CURSOR FAST_FORWARD
FOR SELECT
sm.merge_field
, sm.user_data
FROM dbo.statement_merges AS sm
WHERE sm.statement_id = #statementid
OPEN merge_statements
FETCH NEXT FROM merge_statements
INTO #merge_field , #user_data
WHILE ##FETCH_STATUS = 0
BEGIN
set #text = REPLACE(#text , #merge_field, #user_data )
FETCH NEXT FROM merge_statements
INTO #merge_field , #user_data
END
CLOSE merge_statements
DEALLOCATE merge_statements
SELECT #text
Here is a recursive solution.
SQL Fiddle
MS SQL Server 2017 Schema Setup:
CREATE TABLE [dbo].[store]
(
[id] [int] NOT NULL,
[text] [nvarchar](MAX) NOT NULL
)
CREATE TABLE [dbo].[statement]
(
[id] [int] NOT NULL,
[store_id] [int] NOT NULL
)
CREATE TABLE [dbo].[statement_merges]
(
[statement_id] [int] NOT NULL,
[merge_field] [nvarchar](30) NOT NULL,
[user_data] [nvarchar](MAX) NOT NULL
)
INSERT INTO store (id, text)
VALUES (1, '$$(*)$$, stackoverflow...$$PERC_SAT$$...$$ASKED$$ questions.')
INSERT INTO store (id, text)
VALUES (2, 'Use The #_#')
INSERT INTO statement (id, store_id) VALUES (1, 1)
INSERT INTO statement (id, store_id) VALUES (2, 2)
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$PERC_SAT$$', '85%')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$ASKED$$', '12')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$(*)$$', 'Wow')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (2, ' #_#', 'Flux!')
Query 1:
;WITH Normalized AS
(
SELECT
store_id=store.id,
store.text,
sm.merge_field,
sm.user_data,
RowNumber = ROW_NUMBER() OVER(PARTITION BY store.id,sm.statement_id ORDER BY merge_field),
statement_id = st.id
FROM
store store
INNER JOIN statement st ON st.store_id = store.id
INNER JOIN statement_merges sm ON sm.statement_id = st.id
)
, Recurse AS
(
SELECT
store_id, statement_id, old_text = text, merge_field,user_data, RowNumber,
Iteration=1,
new_text = REPLACE(text, merge_field, user_data)
FROM
Normalized
WHERE
RowNumber=1
UNION ALL
SELECT
n.store_id, n.statement_id, r.old_text, n.merge_field, n.user_data,
RowNumber=r.RowNumber+1,
Iteration=Iteration+1,
new_text = REPLACE(r.new_text, n.merge_field, n.user_data)
FROM
Normalized n
INNER JOIN Recurse r ON r.RowNumber = n.RowNumber AND r.statement_id = n.statement_id
)
,ReverseOnIteration AS
(
SELECT *,
ReverseIteration = ROW_NUMBER() OVER(PARTITION BY statement_id ORDER BY Iteration DESC)
FROM
Recurse
)
SELECT
store_id, statement_id, new_text, old_text
FROM
ReverseOnIteration
WHERE
ReverseIteration=1
Results:
| store_id | statement_id | new_text | old_text |
|----------|--------------|------------------------------------------|--------------------------------------------------------------|
| 1 | 1 | Wow, stackoverflow...85%...12 questions. | $$(*)$$, stackoverflow...$$PERC_SAT$$...$$ASKED$$ questions. |
| 2 | 2 | Use TheFlux! | Use The #_# |
With the help of Randy, I think I've achieved what I wanted to do !
Known the fact that my real case is a contract, in which there are several statements that may be :
free text
stored text without any merges
stored text with one or
several merges
this CTE does the job !
WITH cte_replace_tokens AS (
-- The initial query dont join on merges neither on store because can be a free text
SELECT COALESCE(r.text, s.part_text) AS [final], CAST('' AS NVARCHAR) AS merge_field, s.id, 1 AS i, s.contract_id
FROM statement s
LEFT JOIN store r ON s.store_id = r.id
UNION ALL
-- We loop till the last merge field, output contains iteration to be able to keep the last record ( all fields updated )
SELECT replace(r.final, m.merge_field, m.user_data) as [final], m.merge_field, r.id, r.i + 1 AS i, r.contract_id
FROM cte_replace_tokens r
INNER JOIN statement_merges m ON m.statement_id = r.id
WHERE m.merge_field > r.merge_field AND r.final LIKE '%' + m.merge_field + '%'
-- spare lost replacements by forcing only one merge_field per loop
AND NOT EXISTS( SELECT mm.statement_id FROM statement_merges mm WHERE mm.statement_id = m.statement_id AND mm.merge_field > r.merge_field AND mm.merge_field < m.merge_field)
)
select s.id,
(select top 1 final from cte_replace_tokens t WHERE t.contract_id = s.contract_id AND t.id = s.id ORDER BY i DESC) as res
FROM statement s
where contract_id = 1
If the CTE solution with a cross join is too slow, an alternate solution would be to build a scalar fn dynamically that has every REPLACE required from the token table. One scalar fn call per record then is order(N). I get the same result as before.
The function is simple and likely not to be too long, depending upon how big the token table becomes...256 MB batch limit. I've seen attempts to dynamically create queries to improve performance backfire - moved the problem to compile time. Should not be a problem here.
if not object_id('tempdb..#Raw') is null drop table #Raw
CREATE TABLE #Raw(
[test] [varchar](100) NOT NULL PRIMARY KEY CLUSTERED,
)
if not object_id('tempdb..#Token') is null drop table #Token
CREATE TABLE #Token(
[id] [int] NOT NULL PRIMARY KEY CLUSTERED,
[token] [char](1) NOT NULL,
[value] [char](1) NOT NULL,
)
insert into #Raw values('123456'), ('1122334456')
insert into #Token values(1, '1', 'A'), (2, '2', 'B'), (3, '3', 'C'), (4, '4', 'D'), (5, '5', 'E'), (6, '6', 'F');
DECLARE #sql varchar(max) = 'CREATE FUNCTION dbo.fn_ReplaceTokens(#raw varchar(8000)) RETURNS varchar(8000) AS BEGIN RETURN ';
WITH cte_replace_statement AS (
SELECT a.id, CAST('replace(#raw,''' + a.token + ''',''' + a.value + ''')' as varchar(max)) as [statement]
FROM #Token a
WHERE a.id = 1
UNION ALL
SELECT n.id, CAST(replace(l.[statement], '#raw', 'replace(#raw,''' + n.token + ''',''' + n.value + ''')') as varchar(max)) as [statement]
FROM #Token n
INNER JOIN cte_replace_statement l
ON n.id = l.id + 1
)
select #sql += [statement] + ' END' from cte_replace_statement where id = 6
print #sql
if not object_id('dbo.fn_ReplaceTokens') is null drop function dbo.fn_ReplaceTokens
execute (#sql)
SELECT r.test, dbo.fn_ReplaceTokens(r.test) as [final] FROM [Raw] r
I'm trying to create my own logic for tables synchronization in SQL Server Express 2019. I was hoping that such simple task would work:
Have a Customers table
Have a Synchronization table
CREATE TABLE [dbo].[Synchronization]
(
[PK] [uniqueidentifier] NOT NULL,
[TableName] [nchar](50) NOT NULL,
[RecordPK] [uniqueidentifier] NOT NULL,
[RecordChecksum] [int] NOT NULL,
[RecordDate] [datetime] NOT NULL,
[RecordIsDeleted] [bit] NOT NULL
)
Have a trigger on Customers:
CREATE TRIGGER trgCustomers_INSERT
ON Customers
AFTER INSERT
AS
INSERT INTO Synchronization(PK, TableName, RecordPK, RecordChecksum,
RecordDate, RecordIsDeleted)
VALUES (NEWID(), 'Customers',
(SELECT PK FROM inserted),
(SELECT CHECKSUM(*) FROM inserted),
GETDATE(), 0)
... but I got an error about the SELECT CHECKSUM(*) FROM inserted part:
Cannot use CHECKSUM(*) in a computed column, constraint, default definition, or INSERT statement.
Is there any other way to add new Customer's CHECKSUM or some hash to the Synchronization table?
Don't use the VALUES syntax when inserting and you won't get an error using CHECKSUM while inserting.
Example:
declare #t table (val int)
-- works
insert into #t select checksum(*) from ( select ID from (select 1 as ID union select 2) b ) a
-- reproduce error
insert into #t
values
((select top 1 checksum(*) C from ( select ID from (select 1 as ID union select 2) b ) a))
Implementing the concept in your trigger:
CREATE TRIGGER trgCustomers_INSERT
ON Customers
AFTER INSERT
AS
begin
INSERT INTO Synchronization(PK, TableName, RecordPK, RecordChecksum,
RecordDate, RecordIsDeleted)
select NEWID() as PK,
'Customers' as TableName,
PK as RecordPK,
checksum(*) as RecordChecksum,
GETDATE() as RecordDate,
0 as RecordIsDeleted
from inserted
end
I have a table that looks like below:
The table lists countries and regions (states, provinces, counties, etc) within those countries. I need to generate a count of all the regions within all countries. As you can see, each region has a ParentID which is the ID of the country in which you can find the region.
As an example, California is in USA, so its parent ID is 1 (which is the ID of USA).
So, the results from the simple table above should be:
USA: 2 and
Canada: 1
I have tried the following:
Select all values into a table which have ID a 1 (for USA)
Select all values into a table which have ID a 3 (for Canada)
Select all values into the USA table with Parent ID as 1
Select all values into the Canada table with Parent ID as 3
Do counts on both tables
The problem with the above approach is that if a new country is added, a count will not be automatically generated.
Any ideas on making this more dynamic?
You have to join the table with itself:
select t1.ID, t1.segment, count(distinct t2.ID)
from yourTable t1
join yourTable t2
on t1.ID = t2.parentID
where t1.parentID is null
group by t1.ID, t1.segment
The where clause ensures you that only "top level" rows will be displayed.
Perhaps it makes sense to re-format the data, incase there are other sorts of queries that you want to make in addition to a count of countries and regions.
CREATE TABLE #CountriesRegions
(
[ID] [int] NOT NULL,
parentid [int] NULL,
segment [nvarchar](50) NULL)
insert into #CountriesRegions values (1,null,'usa'), (2,1, 'california'), (3, null, 'canada'), (4, 3, 'quebec'), (5, 1, 'NY')
select * from #CountriesRegions
Create table #Country
([ID] [int] NOT NULL
,[country_name] [nvarchar](50) NOT NULL)
Insert into #Country select ID, segment AS country_name from #CountriesRegions where parentid IS NULL
select * from #Country
Create table #Region
([ID] [int] NOT NULL
,[country_id] [int] NOT NULL
,[region_name] [nvarchar](50) NOT NULL)
Insert into #Region select ID, parentid AS country_ID, segment AS region_name from #CountriesRegions where parentid IS NOT NULL
select * from #Region
Select COUNT(*) As 'Num of Countries' from #Country
Select COUNT(*) As 'Num of Regions' from #Region
CREATE TABLE CountriesRegions
(
[ID] [int] NOT NULL,
parentid [int] NULL,
segment [nvarchar](50) NULL)
insert into CountriesRegions values (1,null,'usa'), (2,1, 'california'), (3, null, 'canada'), (4, 3, 'quebec'), (5, 1, 'NY')
select a.id, a.segment, count(*) as [Region Count]
from CountriesRegions a
left join CountriesRegions b
on a.id=b.parentid
where b.id is not null
group by a.id, a.segment
Below is my table.
CREATE TABLE [dbo].[CCMaster](
[CCId] [int] IDENTITY(1,1) NOT NULL,
[GId] [int] NULL,
[BKId] [int] NULL,
[Class] [varchar](100) NULL,
[ParentId] [int] NULL,)
SET IDENTITY_INSERT CCMaster ON
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(1,33,162,'CORPORATE',NULL)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(10,33,162,'Call center related',4)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(11,33,162,'Channel related',2)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(12,33,162,'Advertisement',6)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(13,33,162,'Brand Ambassador',6)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(14,33,162,'OTAF/TVP (New Activation)',11)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(15,33,162,'Service Barred',7)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(16,33,162,'Call center behaviour',10)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(17,33,162,'Store Personnel behavior',8)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(2,33,162,'DTH',NULL)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(3,33,162,'2G',NULL)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(4,33,162,'3GS',NULL)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(5,33,162,'Broadband',NULL)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(6,33,162,'Brand',1)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(7,33,162,'Barring-related',3)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(8,33,162,'Behaviour related',4)
insert into CCMaster([CCId],[GID],[BKID],[Class],[ParentId]) values(9,33,162,'Call center related',3)
SET IDENTITY_INSERT CCMaster OFF
There are thousands of records in this table. I want to clone only the records where GID = 33 and BKID = 162. The parent of the child must match accordingly to the auto generated id of the respective parent. I tried this query but it din't work for me.
I tried using a cursor by first inserting the parents and then trying to query for the child but that too din't work out. Any help would be appreciated.
Use a recursive CTE to get the rows you want to insert.
To figure out what new value is to be used as ParentId you can use merge with output to get a mapping between the old CCId and the new CCId. Capture the output from the merge to table variable and after the merge you can update ParentId from the table variable.
declare #T table
(
OldCCId int,
OldParentId int,
NewCCId int
);
with C as
(
select M.*
from dbo.CCMaster as M
where ParentId is null and
GId = 33 and
BKId = 162
union all
select M.*
from dbo.CCMaster as M
inner join C
on C.CCId = M.ParentId
)
merge into dbo.CCMaster as M
using C
on 1 = 0
when not matched then
insert(GID, BKID, Class)
values (C.GID, C.BKID, C.Class)
output C.CCId,
C.ParentId,
inserted.CCId
into #T;
update M
set ParentId = T2.NewCCId
from CCMaster as M
inner join #T as T1
on M.CCId = T1.NewCCId
inner join #T as T2
on T1.OldParentId = T2.OldCCId;
SQL Fiddle
Let me describe the whole situation: I have below two tables. The values of MaxTestCasesExecutedBySingleUser and TotalTestCasesExecutedByAllUser in Module table needs to be updated using some Aggregate clause
CREATE TABLE [dbo].[Module](
[ProjectID] [int] NOT NULL,
[ModuleID] [int] NOT NULL,
[ModuleName] [nvarchar](100) NOT NULL,
[MaxTestCasesExecutedBySingleUser] [int] NULL,
[TotalTestCasesExecutedByAllUser] [int] NULL,
PRIMARY KEY ([ProjectID],[ModuleID]))
CREATE TABLE [dbo].[ModuleMember](
[ProjectID] [int] NOT NULL,
[ModuleID] [int] NOT NULL,
[SerialNo] [int] NOT NULL,
[Name] [nvarchar](100) NOT NULL,
PRIMARY KEY ([ProjectID],[ModuleID],[SerialNo]))
Insert Into Module Values (1,1,'Installation_Test',null,null)
Insert Into Module Values (1,2,'Server_Test',null,null)
Insert Into Module Values (1,3,'Client_Test',null,null)
Insert Into Module Values (1,4,'Security_Test',null,null)
Insert Into ModuleMember Values(1,1,0,'Jim')
Insert Into ModuleMember Values(1,1,1,'Bob')
Insert Into ModuleMember Values(1,2,0,'Jack')
Insert Into ModuleMember Values(1,2,1,'Steve')
Insert Into ModuleMember Values(1,2,2,'Roy')
Insert Into ModuleMember Values(1,2,3,'Jerry')
Insert Into ModuleMember Values(1,3,0,'Root')
Insert Into ModuleMember Values(1,3,1,'Tom')
Insert Into ModuleMember Values(1,4,0,'Evil')
There is another Table Valued Function dbo.GetValue which takes Name (from ModuleMember table) as parameter and returns the values of Name and TestCasesExecutedByTheUser in the form of a table. I need to update these two values in Module table.
Definition:
MaxTestCasesExecutedBySingleUser = maximum number if test case executed by a member in a module
TotalTestCasesExecutedByAllUser = total number of test cases executed by all member in a module.
I tried below query but it is throwing me error : An aggregate may not appear in the set list of an UPDATE statement.
UPDATE M
SET
M.MaxTestCasesExecutedBySingleUser = MAX(N.TestCasesExecutedByTheUser),
M.TotalTestCasesExecutedByAllUser = SUM(N.TestCasesExecutedByTheUser)
FROM Module M
JOIN ModuleMember Mem ON (M.ProjectID = Mem.ProjectID AND M.ModuleID = Mem.ModuleID)
CROSS APPLY dbo.GetValue(Mem.Name) N
The workaround is a little more verbose - you have to get the aggregates first (e.g. in a CTE), and then update the target table:
;WITH src AS
(
SELECT
Mem.ProjectID,
Mem.ModuleID,
mtc = MAX(N.TestCasesExecutedByTheUser),
stc = SUM(N.TestCasesExecutedByTheUser)
FROM dbo.ModuleMember AS Mem
CROSS APPLY dbo.GetValue(Mem.Name) AS N
GROUP BY Mem.ProjectID, Mem.ModuleID
)
UPDATE M
SET M.MaxTestCasesExecutedBySingleUser = src.mtc,
M.TotalTestCasesExecutedBySingleUser = src.stc
FROM dbo.Module AS M
INNER JOIN src
ON M.ProjectID = src.ProjectID
AND M.ModuleID = src.ModuleID;
I think it's perfect case to use apply
update Module set
MaxTestCasesExecutedBySingleUser = C.maxtc,
TotalTestCasesExecutedByAllUser = C.sumtc
from Module as M
cross apply (
select
max(N.TestCasesExecutedByTheUser) as maxtc,
sum(N.TestCasesExecutedByTheUser) as sumtc
from ModuleMember as MM
cross apply dbo.GetValue(MM.Name) as N
where MM.ProjectID = M.ProjectID and MM.ModuleID = M.ModuleID
) as C