Related
Sample Data:
DECLARE #Parent TABLE
(
[Id] INT
, [Misc_Val] VARCHAR(5)
) ;
DECLARE #Children TABLE
(
[Id] INT
, [P_ID] INT
) ;
INSERT INTO #Parent
VALUES
( 1, 'One' )
, ( 2, 'Two' )
, ( 3, 'Three' )
, ( 5, 'Four' ) ;
INSERT INTO #Children
VALUES
( 10, 1 )
, ( 11, 1 )
, ( 21, 2 )
, ( 23, 2 )
, ( 30, 3 )
, ( 40, 4 ) ;
Goal:
To efficiently output three fields ( [Id] and [IsChild], [Misc_Val] ). Output all records from #Parent table with [IsChild] = 0 and output all MATCHING records from #Child table (#Parent.Id = #Children.P_Id) with [IsChild] = 1.
Expected Output
Id IsChild Misc_Val
1 0 One
2 0 Two
3 0 Three
5 0 Four
10 1 One
11 1 One
21 1 Two
23 1 Two
30 1 Three
My try:
SELECT [P].[Id]
, 0 AS [IsChild]
, [P].[Misc_Val]
FROM #Parent AS [P]
UNION ALL
SELECT [C].[Id]
, 1
, [P].[Misc_Val]
FROM #Parent AS [P]
JOIN #Children AS [C]
ON [C].[P_ID] = [P].[Id] ;
Is there a better way to do this than using UNION ALL? #Parent and #Children tables are quite big and so am trying to avoid querying the #Parent table twice.
UPDATE: The below answer made me realized something I missed out when creating the post with mocked data. We do need some additional data from #Parent table regardless in the final output.
You can use CROSS APPLY to add the child table to the parent table.
This may or may not be faster, it can depend on indexing and so forth. You need to check the query plan.
SELECT v.Id
, v.IsChild
, P.Misc_Val
FROM #Parent AS P
CROSS APPLY (
SELECT
P.Id,
0 AS IsChild
UNION ALL
SELECT
C.Id,
1
FROM #Children AS C
WHERE C.P_ID = P.Id
) v;
Note that the first SELECT in the apply has no FROM and therefore does not do any table access.
I try to find a way to let the SGBD perform a population of merge fields within a long text.
Create the structure :
CREATE TABLE [dbo].[store]
(
[id] [int] NOT NULL,
[text] [nvarchar](MAX) NOT NULL
)
CREATE TABLE [dbo].[statement]
(
[id] [int] NOT NULL,
[store_id] [int] NOT NULL
)
CREATE TABLE [dbo].[statement_merges]
(
[statement_id] [int] NOT NULL,
[merge_field] [nvarchar](30) NOT NULL,
[user_data] [nvarchar](MAX) NOT NULL
)
Now, create test values
INSERT INTO [store] (id, text)
VALUES (1, 'Waw, stackoverflow is an amazing library of lost people in the IT hell, and i have the feeling that $$PERC_SAT$$ of the users found a solution, personally I asked $$ASKED$$ questions.')
INSERT INTO [statement] (id, store_id)
VALUES (1, 1)
INSERT INTO [statement_merges] (statement_id, merge_field, user_data)
VALUES (1, '$$PERC_SAT$$', '85%')
INSERT INTO [statement_merges] (statement_id, merge_field, user_data)
VALUES (1, '$$ASKED$$', '12')
At the time being my app is delivering the final statement, looping through merges, replacing in the stored text and output
Waw, stackoverflow is an amazing library of lost people in the IT
hell, and i have the feeling that 85% of the users found a solution,
personally I asked 12 questions.
I try to find a way to be code-independent and serve the output in a single query, as u understood, select a statement in which the stored text have been populated with user data. I hope I'm clear.
I looked on TRANSLATE function but it looks like a char replacement, so I have two choices :
I try a recursive function, replacing one by one until no merge_fields is found in the calculated text; but I have doubts about the performance of this approach;
There is a magic to do that but I need your knowledge...
Consider that I want this because the real texts are very long, and I don't want to store it more than once in my database. You can imagine a 3 pages contract with only 12 parameters, like start date, invoiced amount, etc... Everything else cant be changed for compliance.
Thank you for your time!
EDIT :
Thanks to Randy's help, this looks to do the trick :
WITH cte_replace_tokens AS (
SELECT replace(r.text, m.merge_field, m.user_data) as [final], m.merge_field, s.id, 1 AS i
FROM store r
INNER JOIN statement s ON s.store_id = r.id
INNER JOIN statement_merges m ON m.statement_id = s.id
WHERE m.statement_id = 1
UNION ALL
SELECT replace(r.final, m.merge_field, m.user_data) as [final], m.merge_field, r.id, r.i + 1 AS i
FROM cte_replace_tokens r
INNER JOIN statement_merges m ON m.statement_id = r.id
WHERE m.merge_field > r.merge_field
)
select TOP 1 final from cte_replace_tokens ORDER BY i DESC
I will check with a bigger database if the performance is good...
At least, I can "populate" one statement, I need to figure out to be able to extract a list as well.
Thanks again !
If a record is updated more than once by the same update, the last wins. None of the updates are affected by the others - no cumulative effect. It is possible to trick SQL using a local variable to get cumulative effects in some cases, but it's tricky and not recommended. (Order becomes important and is not reliable in an update.)
One alternate is recursion in a CTE. Generate a new record from the prior as each token is replaced until there are no tokens. Here is a working example that replaces 1 with A, 2 with B, etc. (I wonder if there is some tricky xml that can do this as well.)
if not object_id('tempdb..#Raw') is null drop table #Raw
CREATE TABLE #Raw(
[test] [varchar](100) NOT NULL PRIMARY KEY CLUSTERED,
)
if not object_id('tempdb..#Token') is null drop table #Token
CREATE TABLE #Token(
[id] [int] NOT NULL PRIMARY KEY CLUSTERED,
[token] [char](1) NOT NULL,
[value] [char](1) NOT NULL,
)
insert into #Raw values('123456'), ('1122334456')
insert into #Token values(1, '1', 'A'), (2, '2', 'B'), (3, '3', 'C'), (4, '4', 'D'), (5, '5', 'E'), (6, '6', 'F');
WITH cte_replace_tokens AS (
SELECT r.test, replace(r.test, l.token, l.value) as [final], l.id
FROM [Raw] r
CROSS JOIN #Token l
WHERE l.id = 1
UNION ALL
SELECT r.test, replace(r.final, l.token, l.value) as [final], l.id
FROM cte_replace_tokens r
CROSS JOIN #Token l
WHERE l.id = r.id + 1
)
select * from cte_replace_tokens where id = 6
It's not recommended to do such tasks inside sql engine but if you want to do that, you need to do it in a loop using cursor in a function or stored procedure like so :
DECLARE #merge_field nvarchar(30)
, #user_data nvarchar(MAX)
, #statementid INT = 1
, #text varchar(MAX) = 'Waw, stackoverflow is an amazing library of lost people in the IT hell, and i have the feeling that $$PERC_SAT$$ of the users found a solution, personally I asked $$ASKED$$ questions.'
DECLARE merge_statements CURSOR FAST_FORWARD
FOR SELECT
sm.merge_field
, sm.user_data
FROM dbo.statement_merges AS sm
WHERE sm.statement_id = #statementid
OPEN merge_statements
FETCH NEXT FROM merge_statements
INTO #merge_field , #user_data
WHILE ##FETCH_STATUS = 0
BEGIN
set #text = REPLACE(#text , #merge_field, #user_data )
FETCH NEXT FROM merge_statements
INTO #merge_field , #user_data
END
CLOSE merge_statements
DEALLOCATE merge_statements
SELECT #text
Here is a recursive solution.
SQL Fiddle
MS SQL Server 2017 Schema Setup:
CREATE TABLE [dbo].[store]
(
[id] [int] NOT NULL,
[text] [nvarchar](MAX) NOT NULL
)
CREATE TABLE [dbo].[statement]
(
[id] [int] NOT NULL,
[store_id] [int] NOT NULL
)
CREATE TABLE [dbo].[statement_merges]
(
[statement_id] [int] NOT NULL,
[merge_field] [nvarchar](30) NOT NULL,
[user_data] [nvarchar](MAX) NOT NULL
)
INSERT INTO store (id, text)
VALUES (1, '$$(*)$$, stackoverflow...$$PERC_SAT$$...$$ASKED$$ questions.')
INSERT INTO store (id, text)
VALUES (2, 'Use The #_#')
INSERT INTO statement (id, store_id) VALUES (1, 1)
INSERT INTO statement (id, store_id) VALUES (2, 2)
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$PERC_SAT$$', '85%')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$ASKED$$', '12')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$(*)$$', 'Wow')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (2, ' #_#', 'Flux!')
Query 1:
;WITH Normalized AS
(
SELECT
store_id=store.id,
store.text,
sm.merge_field,
sm.user_data,
RowNumber = ROW_NUMBER() OVER(PARTITION BY store.id,sm.statement_id ORDER BY merge_field),
statement_id = st.id
FROM
store store
INNER JOIN statement st ON st.store_id = store.id
INNER JOIN statement_merges sm ON sm.statement_id = st.id
)
, Recurse AS
(
SELECT
store_id, statement_id, old_text = text, merge_field,user_data, RowNumber,
Iteration=1,
new_text = REPLACE(text, merge_field, user_data)
FROM
Normalized
WHERE
RowNumber=1
UNION ALL
SELECT
n.store_id, n.statement_id, r.old_text, n.merge_field, n.user_data,
RowNumber=r.RowNumber+1,
Iteration=Iteration+1,
new_text = REPLACE(r.new_text, n.merge_field, n.user_data)
FROM
Normalized n
INNER JOIN Recurse r ON r.RowNumber = n.RowNumber AND r.statement_id = n.statement_id
)
,ReverseOnIteration AS
(
SELECT *,
ReverseIteration = ROW_NUMBER() OVER(PARTITION BY statement_id ORDER BY Iteration DESC)
FROM
Recurse
)
SELECT
store_id, statement_id, new_text, old_text
FROM
ReverseOnIteration
WHERE
ReverseIteration=1
Results:
| store_id | statement_id | new_text | old_text |
|----------|--------------|------------------------------------------|--------------------------------------------------------------|
| 1 | 1 | Wow, stackoverflow...85%...12 questions. | $$(*)$$, stackoverflow...$$PERC_SAT$$...$$ASKED$$ questions. |
| 2 | 2 | Use TheFlux! | Use The #_# |
With the help of Randy, I think I've achieved what I wanted to do !
Known the fact that my real case is a contract, in which there are several statements that may be :
free text
stored text without any merges
stored text with one or
several merges
this CTE does the job !
WITH cte_replace_tokens AS (
-- The initial query dont join on merges neither on store because can be a free text
SELECT COALESCE(r.text, s.part_text) AS [final], CAST('' AS NVARCHAR) AS merge_field, s.id, 1 AS i, s.contract_id
FROM statement s
LEFT JOIN store r ON s.store_id = r.id
UNION ALL
-- We loop till the last merge field, output contains iteration to be able to keep the last record ( all fields updated )
SELECT replace(r.final, m.merge_field, m.user_data) as [final], m.merge_field, r.id, r.i + 1 AS i, r.contract_id
FROM cte_replace_tokens r
INNER JOIN statement_merges m ON m.statement_id = r.id
WHERE m.merge_field > r.merge_field AND r.final LIKE '%' + m.merge_field + '%'
-- spare lost replacements by forcing only one merge_field per loop
AND NOT EXISTS( SELECT mm.statement_id FROM statement_merges mm WHERE mm.statement_id = m.statement_id AND mm.merge_field > r.merge_field AND mm.merge_field < m.merge_field)
)
select s.id,
(select top 1 final from cte_replace_tokens t WHERE t.contract_id = s.contract_id AND t.id = s.id ORDER BY i DESC) as res
FROM statement s
where contract_id = 1
If the CTE solution with a cross join is too slow, an alternate solution would be to build a scalar fn dynamically that has every REPLACE required from the token table. One scalar fn call per record then is order(N). I get the same result as before.
The function is simple and likely not to be too long, depending upon how big the token table becomes...256 MB batch limit. I've seen attempts to dynamically create queries to improve performance backfire - moved the problem to compile time. Should not be a problem here.
if not object_id('tempdb..#Raw') is null drop table #Raw
CREATE TABLE #Raw(
[test] [varchar](100) NOT NULL PRIMARY KEY CLUSTERED,
)
if not object_id('tempdb..#Token') is null drop table #Token
CREATE TABLE #Token(
[id] [int] NOT NULL PRIMARY KEY CLUSTERED,
[token] [char](1) NOT NULL,
[value] [char](1) NOT NULL,
)
insert into #Raw values('123456'), ('1122334456')
insert into #Token values(1, '1', 'A'), (2, '2', 'B'), (3, '3', 'C'), (4, '4', 'D'), (5, '5', 'E'), (6, '6', 'F');
DECLARE #sql varchar(max) = 'CREATE FUNCTION dbo.fn_ReplaceTokens(#raw varchar(8000)) RETURNS varchar(8000) AS BEGIN RETURN ';
WITH cte_replace_statement AS (
SELECT a.id, CAST('replace(#raw,''' + a.token + ''',''' + a.value + ''')' as varchar(max)) as [statement]
FROM #Token a
WHERE a.id = 1
UNION ALL
SELECT n.id, CAST(replace(l.[statement], '#raw', 'replace(#raw,''' + n.token + ''',''' + n.value + ''')') as varchar(max)) as [statement]
FROM #Token n
INNER JOIN cte_replace_statement l
ON n.id = l.id + 1
)
select #sql += [statement] + ' END' from cte_replace_statement where id = 6
print #sql
if not object_id('dbo.fn_ReplaceTokens') is null drop function dbo.fn_ReplaceTokens
execute (#sql)
SELECT r.test, dbo.fn_ReplaceTokens(r.test) as [final] FROM [Raw] r
When executing a CTE expression to query for an ordered child parent relation by using a shift, it fails with
Arithmetic overflow error converting expression to data type bigint
The problem is that the shift value becomes big very easily. I know I could increase the datatype to support 38 numeric values but I would still hit this number when having deep parent child relations. I'm wondering if there are any other method to order the results, so I would not hit this limit.
Here is a sample script that shows the increase of the shift parameter.
CREATE TABLE [dbo].[ParentChild] (
[Id] [int] IDENTITY(1,1) NOT NULL,
[ParentId] [int] NULL,
[Name] [nvarchar](150) NOT NULL
CONSTRAINT [PK_Dialog] PRIMARY KEY CLUSTERED
(
[Id] ASC
))
GO
ALTER TABLE [dbo].[ParentChild] WITH CHECK ADD CONSTRAINT [FK_ParentChild_ParentId] FOREIGN KEY([ParentId])
REFERENCES [dbo].[ParentChild] ([Id])
GO
ALTER TABLE [dbo].[ParentChild] CHECK CONSTRAINT [FK_ParentChild_ParentId]
GO
set identity_insert [dbo].[ParentChild] on
insert into [dbo].[ParentChild] ([Id], [ParentId],[Name])
values
(1, NULL, '1'),
(2, NULL, '2'),
(3, 1, '1.1'),
(4, 1, '1.2'),
(5, 2, '2.1'),
(6, 5, '2.1.1')
set identity_insert [dbo].[ParentChild] off
-- without shift
with Parent as (
select d1.[Id], d1.[ParentId], d1.[Name], 0 AS [Level]
FROM [dbo].[ParentChild] as d1
WHERE d1.[ParentId] IS NULL
UNION ALL
SELECT d2.Id, d2.ParentId, d2.[Name], [Level] + 1
FROM [dbo].[ParentChild] as d2
INNER JOIN Parent d1 ON d1.[Id] = d2.ParentId
)
select p.Id, p.ParentId, p.[Name], [Level]
from Parent p
group by p.Id, p.ParentId, p.[Name], [Level];
-- desired
with Parent as (
select d1.[Id], d1.[ParentId], d1.[Name], 0 AS [Level],
CAST(row_number() over(order by id) as DECIMAL(38,0)) as [shift]
FROM [dbo].[ParentChild] as d1
WHERE d1.[ParentId] IS NULL
UNION ALL
SELECT d2.Id, d2.ParentId, d2.[Name], [Level] + 1,
CAST([shift] * 100 + row_number() over(order by d2.id) as DECIMAL(38,0))
FROM [dbo].[ParentChild] as d2
INNER JOIN Parent d1 ON d1.[Id] = d2.ParentId
)
select p.Id, p.ParentId, p.[Name], [Level], [shift]
from Parent p
group by p.Id, p.ParentId, p.[Name], [Level], [shift]
order by cast([shift] as varchar(50))
Output without the shift parameter
Id ParentId Name Level
1 NULL 1 0
2 NULL 2 0
3 1 1.1 1
4 1 1.2 1
5 2 2.1 1
6 5 2.1.1 2
Output with the shift parameter (desired)
Id ParentId Name Level shift
1 NULL 1 0 1
3 1 1.1 1 101
4 1 1.2 1 102
2 NULL 2 0 2
5 2 2.1 1 201
6 5 2.1.1 2 20101
Assuming we can make shift a string rather than a maths-supporting data type, we can just do this:
with Parent as (
select d1.[Id], d1.[ParentId], d1.[Name], 0 AS [Level],
CONVERT(varchar(max),row_number() over(order by id)) as [shift]
FROM [dbo].[ParentChild] as d1
WHERE d1.[ParentId] IS NULL
UNION ALL
SELECT d2.Id, d2.ParentId, d2.[Name], [Level] + 1,
shift + RIGHT('0' + CONVERT(varchar(2),row_number() over(order by d2.id)),2)
FROM [dbo].[ParentChild] as d2
INNER JOIN Parent d1 ON d1.[Id] = d2.ParentId
)
select p.Id, p.ParentId, p.[Name], [Level], [shift]
from Parent p
group by p.Id, p.ParentId, p.[Name], [Level], [shift]
order by shift
It produces different results if the row numbers can ever exceed 100 but that seems to lead to problems with this representation anyway (ambiguous encodings).
I have a table that looks like below:
The table lists countries and regions (states, provinces, counties, etc) within those countries. I need to generate a count of all the regions within all countries. As you can see, each region has a ParentID which is the ID of the country in which you can find the region.
As an example, California is in USA, so its parent ID is 1 (which is the ID of USA).
So, the results from the simple table above should be:
USA: 2 and
Canada: 1
I have tried the following:
Select all values into a table which have ID a 1 (for USA)
Select all values into a table which have ID a 3 (for Canada)
Select all values into the USA table with Parent ID as 1
Select all values into the Canada table with Parent ID as 3
Do counts on both tables
The problem with the above approach is that if a new country is added, a count will not be automatically generated.
Any ideas on making this more dynamic?
You have to join the table with itself:
select t1.ID, t1.segment, count(distinct t2.ID)
from yourTable t1
join yourTable t2
on t1.ID = t2.parentID
where t1.parentID is null
group by t1.ID, t1.segment
The where clause ensures you that only "top level" rows will be displayed.
Perhaps it makes sense to re-format the data, incase there are other sorts of queries that you want to make in addition to a count of countries and regions.
CREATE TABLE #CountriesRegions
(
[ID] [int] NOT NULL,
parentid [int] NULL,
segment [nvarchar](50) NULL)
insert into #CountriesRegions values (1,null,'usa'), (2,1, 'california'), (3, null, 'canada'), (4, 3, 'quebec'), (5, 1, 'NY')
select * from #CountriesRegions
Create table #Country
([ID] [int] NOT NULL
,[country_name] [nvarchar](50) NOT NULL)
Insert into #Country select ID, segment AS country_name from #CountriesRegions where parentid IS NULL
select * from #Country
Create table #Region
([ID] [int] NOT NULL
,[country_id] [int] NOT NULL
,[region_name] [nvarchar](50) NOT NULL)
Insert into #Region select ID, parentid AS country_ID, segment AS region_name from #CountriesRegions where parentid IS NOT NULL
select * from #Region
Select COUNT(*) As 'Num of Countries' from #Country
Select COUNT(*) As 'Num of Regions' from #Region
CREATE TABLE CountriesRegions
(
[ID] [int] NOT NULL,
parentid [int] NULL,
segment [nvarchar](50) NULL)
insert into CountriesRegions values (1,null,'usa'), (2,1, 'california'), (3, null, 'canada'), (4, 3, 'quebec'), (5, 1, 'NY')
select a.id, a.segment, count(*) as [Region Count]
from CountriesRegions a
left join CountriesRegions b
on a.id=b.parentid
where b.id is not null
group by a.id, a.segment
I have a data as below:
I need to update Matching_id and Matching_Type by using column id, region, company, dept, subdept and amountsepend. The logic is:
Sum AmountSepend by Region, Company, Dept and SubDept. If the sum amount is 0 then Matching_Type is 'Match' and Matching_id is the combination of the id for the matched record else 'Not Match' and Matching_id is the id. **SUM means the total sum of all records for same criteria regardless the AmountSepend is positive or negative.
Another important criteria is if the transaction is single record, meaning the total count by grouping by Region, Company, Dept and SubDept is 1 then Matching type is Not Match and Matching_UID is id regardless the AmountSepend is 0 or positive/negative value. Example id 8.
Below is the output:
Here the table and data script
CREATE TABLE [dbo].[StackoverflowQuest](
[id] [int] NOT NULL,
[Region] [varchar](50) NULL,
[Company] [varchar](50) NULL,
[Dept] [varchar](50) NULL,
[SubDept] [varchar](50) NULL,
[AmountSepend] [float] NULL,
[Matching_id] [varchar](100) NULL,
[Matching_Type] [varchar](100) NULL
) ON [PRIMARY]
How could I achieved such result ? Any help/hint would be appreciate
CREATE TABLE #Table(Id INT,Region VARCHAR(100),Company INT,Dept INT,SubDept
INT,AmtSpend INT,MatchingId VARCHAR(100),MatchingType VARCHAR(100))
INSERT INTO #Table(Id ,Region ,Company , Dept ,SubDept ,AmtSpend )
SELECT 1,'NAM',12378,1,NULL,900 UNION ALL
SELECT 2,'NAM',12378,1,NULL,-900 UNION ALL
SELECT 3,'NAM',12370,1,23,1000 UNION ALL
SELECT 4,'ASA',1234,9,12,5000 UNION ALL
SELECT 5,'NAM',12370,1,23,-1000 UNION ALL
SELECT 6,'ASA',1234,9,12,800 UNION ALL
SELECT 7,'ASA',1234,9,12,-600 UNION ALL
SELECT 8,'ASA',12311,6,NULL,200
UPDATE #Table SET MatchingId = MatchIds,MatchingType = 'Match'
FROM
(
SELECT T2.Company,STUFF( ( SELECT ',' + CAST(T3.Id AS VARCHAR) FROM #Table
T3 WHERE T2.Company = T3.Company FOR XML PATH('')),1,1,'') MatchIds
FROM #Table T2
JOIN
(
SELECT T1.Company Company,SUM(T1.AmtSpend) Total
FROM #Table T1
GROUP BY T1.Company
HAVING SUM(T1.AmtSpend) = 0
)A ON A.Company = T2.Company
GROUP BY T2.Company
) A
WHERE A.Company = #Table.Company
UPDATE #Table SET MatchingId = CAST(Id AS VARCHAR),MatchingType = 'Not
Match' WHERE ISNULL(MatchingId,'') = ''
SELECT * FROM #Table