Recursive query to speed up system - sql

My application reads a table with parent child relation. The application queries every level of the tree on his own and it really slow (must do multiple levels deep). I has searching for another solution and came to the recursive queries. With the examples that I have found I cannot map it to my data structure.
My structure looks like:
CREATE TABLE [products].[BillOfMaterial](
[id] [bigint] IDENTITY(1,1) NOT NULL,
[parentNumber] [nvarchar](50) NOT NULL,
[warehouse] [nvarchar](50) NOT NULL,
[sequenceNumber] [int] NOT NULL,
[childNumber] [nvarchar](50) NOT NULL,
[childDescription] [nvarchar](50) NULL,
[qtyRequired] [numeric](18, 3) NOT NULL,
[childItemClass] [nvarchar](50) NULL,
[childItemType] [nvarchar](50) NULL,
[scrapFactor] [numeric](18, 3) NULL,
[bubbleNumber] [int] NOT NULL,
[operationNumber] [int] NOT NULL,
[effectivityDate] [date] NULL,
[discontinuityDate] [date] NULL,
[companyID] [bigint] NOT NULL,
CONSTRAINT [PK_BillOfMaterial] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
Some example data:
When I query for parentNumber 1 it should give me all those rows:
And for parentNumber 3 the output must be:
Now I need all the children of a parent in a recursive way. How can I achieve this using only sql? (I'm using SQL server)
I tried already using the with statement in sql, but it will now give a lot of results.
WITH bom ( [id]
,[parentNumber]
,[warehouse]
,[sequenceNumber]
,[childNumber]
,[childDescription]
,[qtyRequired]
,[childItemClass]
,[childItemType]
,[scrapFactor]
,[bubbleNumber]
,[operationNumber]
,[effectivityDate]
,[discontinuityDate]
,[companyID] )
AS
(
select * from [CR_ApplicationSuite].[products].[BillOfMaterial] where parentNumber IN ('F611882261', '2912435206')
UNION ALL
select b.* from [CR_ApplicationSuite].[products].[BillOfMaterial] b
INNER JOIN [CR_ApplicationSuite].[products].[BillOfMaterial] c on c.childNumber = b.parentNumber
)
SELECT *
FROM bom

Here's an example that uses a table variable for demonstration purposes.
For a recursive query you need to use the CTE within itself.
I included a rootParentNumber, so it's more obvious what the base parent was.
The WHERE clauses are commented in the example.
Because you can either put a WHERE clause within the Recursive Query, or at the outer query. The former should be faster.
declare #BillOfMaterial TABLE (
[id] [bigint] IDENTITY(1,1) NOT NULL,
[parentNumber] [nvarchar](50) NOT NULL,
[warehouse] [nvarchar](50) NOT NULL,
[sequenceNumber] [int] NOT NULL,
[childNumber] [nvarchar](50) NOT NULL,
[childDescription] [nvarchar](50) NULL,
[qtyRequired] [numeric](18, 3) NOT NULL,
[childItemClass] [nvarchar](50) NULL,
[childItemType] [nvarchar](50) NULL,
[scrapFactor] [numeric](18, 3) NULL,
[bubbleNumber] [int] NOT NULL,
[operationNumber] [int] NOT NULL,
[effectivityDate] [date] NULL,
[discontinuityDate] [date] NULL,
[companyID] [bigint] NOT NULL
);
insert into #BillOfMaterial (parentNumber, childNumber, warehouse, sequenceNumber, qtyRequired, bubbleNumber, operationNumber, companyID) values
('1','2','WH1',1,0,0,0,1),
('2','4','WH1',2,0,0,0,1),
('3','4','WH1',3,0,0,0,1),
('4','5','WH1',4,0,0,0,1),
('5','0','WH1',5,0,0,0,1);
WITH BOM
AS
(
select parentNumber as rootParentNumber, *
from #BillOfMaterial
--where parentNumber IN ('1','3')
union all
select bom.rootParentNumber, b.*
from BOM
INNER JOIN #BillOfMaterial b
on (BOM.childNumber = b.parentNumber and b.childNumber <> '0')
)
SELECT
[rootParentNumber]
,[parentNumber]
,[childNumber]
,[id]
,[warehouse]
,[sequenceNumber]
,[childDescription]
,[qtyRequired]
,[childItemClass]
,[childItemType]
,[scrapFactor]
,[bubbleNumber]
,[operationNumber]
,[effectivityDate]
,[discontinuityDate]
,[companyID]
FROM BOM
--WHERE rootParentNumber IN ('1','3')
ORDER BY [rootParentNumber], [parentNumber], [childNumber]
;

Related

Speed up Group By and count SQL query

I am counting/grouping about 1.3 million records with the statement below. The query works but is taking about a minute and a half, which is way too long for my application.
The goal is to get a one each (no duplicates) of the listed fields. The current query returns around 846 rows. I don't have any indexes so far, nor do I know much about adding them.
SELECT
[OfferId]
,[Name]
,COUNT([Name]) AS 'Count'
,[Offer]
,[Title]
,[Text]
,[Amount]
,[Start]
,[End]
,[Image]
,[ImageText]
,[Type]
,[Disclaimer]
,[Link]
,[Status]
FROM
ClientDB.[dbo].[Offers]
GROUP BY
[OfferId]
,[Name]
,[Offer]
,[Title]
,[Text]
,[Amount]
,[Start]
,[End]
,[Image]
,[ImageText]
,[Type]
,[Disclaimer]
,[Link]
,[Status]
Table structure (not sure how to index this to make it faster):
CREATE TABLE [dbo].[Offers]
(
[ID] [int] IDENTITY(1,1) NOT NULL,
[Company] [nvarchar](max) NULL,
[Property] [nvarchar](max) NULL,
[Account] [int] NULL,
[OfferID] [nvarchar](50) NULL,
[Offer] [nvarchar](50) NULL,
[Title] [nvarchar](30) NULL,
[Text] [nvarchar](max) NULL,
[AwardCode] [nvarchar](150) NULL,
[Amount] [decimal](18, 2) NULL,
[Start] [datetime] NULL,
[End] [datetime] NULL,
[Image] [nvarchar](max) NULL,
[ImageText] [nvarchar](250) NULL,
[Type] [nvarchar](50) NULL,
[CampaignTier] [nvarchar](50) NULL,
[Name] [nvarchar](max) NULL,
[Disclaimer] [nvarchar](max) NULL,
[Status] [nvarchar](100) NULL,
CONSTRAINT [PK_Offers]
PRIMARY KEY CLUSTERED ([ID] ASC)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF,
IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON,
ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
I added indexes to the fields I was grouping together and this cut the time from a minute and a half to 1 second.

SQL Query to update a colums from a table base on a datetime field

I have 2 table called tblSetting and tblPaquets.
I need to update 3 fields of tblPaquets from tblSetting base on a where clause that use a datetime field of tblPaquest and tblSetting.
The sql below is to represent what I am trying to do and I know it make no sense right now.
My Goal is to have One query to achieve this goal.
I need to extract the data from tblSettings like this
SELECT TOP(1) [SupplierID],[MillID],[GradeFamilyID] FROM [tblSettings]
WHERE [DateHeure] <= [tblPaquets].[DateHeure]
ORDER BY [DateHeure] DESC
And Update tblPaquets with this data
UPDATE [tblPaquets]
SET( [SupplierID] = PREVIOUS_SELECT.[SupplierID]
[MillID] = PREVIOUS_SELECT.[MillID]
[GradeFamilly] = PREVIOUS_SELECT.[GradeFamilyID] )
Here the table design
CREATE TABLE [tblSettings](
[ID] [int] NOT NULL,
[SupplierID] [int] NOT NULL,
[MillID] [int] NOT NULL,
[GradeID] [int] NOT NULL,
[TypeID] [int] NOT NULL,
[GradeFamilyID] [int] NOT NULL,
[DateHeure] [datetime] NOT NULL,
[PeakWetEnable] [tinyint] NULL)
CREATE TABLE [tblPaquets](
[ID] [int] IDENTITY(1,1) NOT NULL,
[PaquetID] [int] NOT NULL,
[DateHeure] [datetime] NULL,
[BarreCode] [int] NULL,
[Grade] [tinyint] NULL,
[SupplierID] [int] NULL,
[MillID] [int] NULL,
[AutologSort] [tinyint] NULL,
[GradeFamilly] [int] NULL)
You can do this using CROSS APPLY:
UPDATE p
SET SupplierID = s.SupplierID,
MillID = s.MillID
GradeFamilly = s.GradeFamilyID
FROM tblPaquets p CROSS APPLY
(SELECT TOP (1) s.*
FROM tblSettings s
WHERE s.DateHeure <= p.DateHeure
ORDER BY p.DateHeure DESC
) s;
Notes:
There are no parentheses before SET.
I don't recommend using [ and ] to escape identifiers, unless they need to be escaped.
I presume the query on tblSettings should have an ORDER BY to get the most recent rows.

SQL get duplicates created within n seconds, update f key row in other table, delete dups

I have a table InspectionNotes - I am able to get duplicate notes that are created within n seconds - this works well.
Problem is when I try to delete where rn > 1 I hit a foreign key constraint - in
InspectionHoldDurations column PutOnHoldNoteID.
table Inspection Notes:
CREATE TABLE [dbo].[InspectionNotes](
[InspectionNoteID] [uniqueidentifier] NOT NULL,
[InspectionID] [uniqueidentifier] NOT NULL,
[UserRoleID] [uniqueidentifier] NOT NULL,
[Notes] [nvarchar](max) NOT NULL,
[CurrentInspectionStatusID] [int] NOT NULL,
[CreatedOn] [datetime] NOT NULL,
[InspectionNoteTypeID] [int] NOT NULL,
[QuickAnswerID] [uniqueidentifier] NULL,
[Reviewed] [bit] NOT NULL CONSTRAINT [DF_InspectionNotes_Reviewed] DEFAULT ((0)),
[DateReviewed] [datetime] NULL,
[ReviewedByUserRoleID] [uniqueidentifier] NULL,
CONSTRAINT [PK_InspectionNotes] PRIMARY KEY CLUSTERED
Table InspectionHoldDurations
CREATE TABLE [dbo].[InspectionHoldDurations](
[InspectionHoldDurationID] [uniqueidentifier] NOT NULL,
[InspectionID] [uniqueidentifier] NOT NULL,
[PutOnHoldByRoleID] [uniqueidentifier] NOT NULL,
[TookOffHoldByRoleID] [uniqueidentifier] NULL,
[StartDate] [datetime] NOT NULL,
[EndDate] [datetime] NULL,
[TotalHoldDurationMinutes] [int] NULL,
[ExpirationDate] [datetime] NULL,
[PutOnHoldNoteID] [uniqueidentifier] NULL,
[TakeOffHoldNoteID] [uniqueidentifier] NULL,
PRIMARY KEY CLUSTERED
(
[InspectionHoldDurationID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
My current query to delete duplicates:
;WITH CTE1
AS (
SELECT
[InspectionNoteID],
ROW_NUMBER() OVER (partition by InspectionID, Notes,UserRoleID,CurrentInspectionStatusId,InspectionNoteTypeID,QuickAnswerID,Reviewed,DateReviewed,ReviewedByUserRoleID,DATEDIFF(ss,'1/1/1970',CreatedOn) / 10000 ORDER BY createdon ASC) AS rn
FROM [DBO].[InspectionNotes]
)
DELETE inote
FROM
[DBO].[InspectionNotes] inote
INNER JOIN
CTE1
ON inote.InspectionNoteID = CTE1.[InspectionNoteID]
where CTE1.rn > 1
AND NOT EXISTS (SELECT * FROM [DBO].[InspectionHoldDurations] WHERE PutOnHoldNoteID = CTE1.InspectionNoteID or TakeOffHoldNoteID = CTE1.InspectionNoteID)
I want to get rid of the AND NOT EXISTS (SELECT * FROM [DBO].[InspectionHoldDurations] and instead update this table with corresponding row from CTE1 where rn = 1 - then delete from InspectionNotes where rn > 1. I have tried this and keep getting fkey error.

SQL Query to Count the Number of People who qualify required criteria

I am working on a project that has multiple agents and members. Now I have to do the following (through a SQL Query):
Select agents and members under a Parent Agent who have made payments, in full (amounting to total of 430, i.e. 130+150+150) for 3 months.
Each Agent has n-number of members, so for each agent I need to check if all the members have paid their 3month dues.
If dues are paid, return the total member count agent wise for the parent agent.
I have tried the following query, but no luck:
CREATE TABLE [dbo].[SlipDetails](
[SlipDetailsID] [int] IDENTITY(1,1) NOT NULL,
[SlipID] [int] NULL,
[SlipNumber] [nvarchar](50) NULL,
[AgentID] [int] NULL,
[AgentName] [nvarchar](50) NULL,
[MemberID] [int] NULL,
[MemberName] [nvarchar](50) NULL,
[MonthID] [int] NULL,
[MonthAmount] [int] NULL,
[LateFine] [int] NULL,
[SubmittedDateByAgent] [datetime] NULL,
[ApprovedByAdmin] [nvarchar](1) NULL,
[ApprovedDate] [datetime] NULL,
[MonthName] [nvarchar](50) NULL,
[Blocked] [nvarchar](1) NULL,
CONSTRAINT [PK_SlipDetails] PRIMARY KEY CLUSTERED
(
[SlipDetailsID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[MemberMaster](
[MemberID] [int] IDENTITY(1,1) NOT NULL,
[MemberName] [nvarchar](50) NULL,
[MemberMobile] [varchar](50) NULL,
[MemberEmail] [nvarchar](50) NULL,
[MemberDOB] [nvarchar](50) NULL,
[MemberDOJ] [nvarchar](50) NULL,
[MemberGender] [nvarchar](50) NULL,
[MemberGenderID] [int] NULL,
[MemberAddress] [nvarchar](50) NULL,
[MemberPhoto] [nvarchar](50) NULL,
[IsFreeGift] [int] NULL,
[GiftID] [int] NULL,
[GiftName] [nvarchar](50) NULL,
[AgentID] [int] NULL,
[AgentName] [nvarchar](50) NULL,
[CardID] [int] NULL,
[CardNumber] [nvarchar](50) NULL,
[SID] [int] NULL,
[SName] [nvarchar](50) NULL,
[Custom1] [nvarchar](50) NULL,
[Custom2] [nvarchar](50) NULL,
[IsActive] [nvarchar](1) NULL,
[IsBlocked] [nvarchar](1) NULL,
CONSTRAINT [PK_MemberMaster] PRIMARY KEY CLUSTERED
(
[MemberID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[AgentMaster](
[AgentID] [int] IDENTITY(1,1) NOT NULL,
[AgentName] [nvarchar](50) NULL,
[Mobile] [nvarchar](50) NULL,
[AgentUserName] [nvarchar](50) NULL,
[AgentPassword] [nvarchar](50) NULL,
[IsActive] [nvarchar](1) NULL,
[ParentAgentID] [int] NULL,
[ParentAgentName] [nvarchar](50) NULL,
[BankName] [nvarchar](50) NULL,
[AccountHolderName] [nvarchar](50) NULL,
[IFSC] [nvarchar](50) NULL,
[BranchName] [nvarchar](50) NULL,
[AccountNo] [nvarchar](50) NULL,
[AgentPhoto] [nvarchar](50) NULL,
[DOJ] [datetime] NULL,
[SelectedInDraw] [nvarchar](1) NULL,
CONSTRAINT [PK_AgentMaster] PRIMARY KEY CLUSTERED
(
[AgentID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
Query:
SELECT COUNT(dbo.MemberMaster.MemberID) AS CMemberID
FROM dbo.SlipDetails
INNER JOIN dbo.MemberMaster ON dbo.SlipDetails.MemberID = dbo.MemberMaster.MemberID
WHERE dbo.MemberMaster.AgentID = 203
GROUP BY dbo.MemberMaster.AgentID,
dbo.SlipDetails.MonthAmount
HAVING(SUM(dbo.SlipDetails.MonthAmount) >= 430);
How do I get the Exact Count for the members under each agent? Any help would be great.
Select mm.AgentName,Count(mm.MemberID) as MemberCount from MemberMaster mm
INNER JOIN
(Select AgentID,MemberId from SlipDetails
group by AgentID,MemberID
having Sum(MonthAmount)>=430)sd
on sd.AgentID=mm.AgentID and sd.MemberID=mm.MemberID
Group by mm.AgentID, mm.AgentName
or even:
Select am.AgentName,MemberCount from AgentMaster am
inner join
(Select mm.AgentID,Count(mm.MemberID) as MemberCount from MemberMaster mm
INNER JOIN
(Select AgentID,MemberId from SlipDetails
group by AgentID,MemberID
having Sum(MonthAmount)>=430)sd
on sd.AgentID=mm.AgentID and sd.MemberID=mm.MemberID
Group by mm.AgentID)mm1
on am.AgentID=mm1.AgentID
Try this
WITH SlipDetails_ttl as (
Select st.MemberID, sum(MonthAmount) as MonthAmount_ttl
from SlipDetails as st
group by st.MemberID
having sum(MonthAmount)>=430
)
Select am.AgentID, count(stt.MemberID)
From SlipDetails_ttl as stt
join MemberMaster as mm on mm.MemberID = stt.MemberID
join AgentMaster as am on mm.AgentID = am.AgentID
group by am.AgentID
Give this a try:
SELECT agt.AgentID, COUNT(*)
FROM dbo.AgentMaster agt -- Get all agents
INNER JOIN dbo.MemberMaster mbr ON agt.AgentID = mbr.AgentID -- Get each agents' members
INNER JOIN dbo.SlipDetails slp ON mbr.MemberID = slp.MemberID -- Get payment details for members
GROUP BY agt.AgentID
HAVING(SUM(slp.MonthAmount)) = 430 -- Only return members that have paid 430
A few assumptions/notes:
only looks at each agent's immediate members (not recursive, one level only)
does not factor in 3-month constraint, only checks that 430 has been paid per member

Select all rows from table having two given values anywhere in rows

I have a table of bus route. this table has fields like bus no. , route code , starting point , end point, and upto 10 halts from halt1 , halt2...halt10 . i have filled data in this table. now i want to select all rows having two values,for example jaipur and vasai. in my table, there are two rows that have jaipur and vasai. In one row, jaipur is in column halt2 and vasai in halt9. Similarly another row has jaipur in halt4 column and vasai in halt10 column.
please help me to find out sql query. I am using MS SQL server.
scrip
USE [JaipuBus]
GO
/****** Object: Table [dbo].[MyRoutes] Script Date: 02/24/2014 13:28:54 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[MyRoutes](
[id] [int] IDENTITY(1,1) NOT NULL,
[Route_No] [nvarchar](50) NULL,
[Route_Code] [nvarchar](50) NULL,
[Color] [nvarchar](50) NULL,
[Start_Point] [nvarchar](200) NULL,
[End_Point] [nvarchar](200) NULL,
[halt1] [nvarchar](50) NULL,
[halt2] [nvarchar](50) NULL,
[halt3] [nvarchar](50) NULL,
[halt4] [nvarchar](50) NULL,
[halt5] [nvarchar](50) NULL,
[halt6] [nvarchar](50) NULL,
[halt7] [nvarchar](50) NULL,
[halt8] [nvarchar](50) NULL,
[halt9] [nvarchar](50) NULL,
[halt10] [nvarchar](50) NULL,
CONSTRAINT [PK_MyRoutes] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
use CONTAINS
SELECT *
WHERE CONTAINS((startingpoint,endpoint,halt1,halt2,halt3,halt4,halt5,halt6,halt7,halt8,halt9,halt10), 'jaipur')
AND CONTAINS((startingpoint,endpoint,halt1,halt2,halt3,halt4,halt5,halt6,halt7,halt8,halt9,halt10), 'vasai');
SELECT * FROM bus_rout WHERE (halt1='aaa' OR halt2='aaa' OR .... halt10='aaa') AND (halt1='bbb' OR halt2='bbb' OR ....... halt10='bbb')
The where clause could be generated by code.
Based on your input, it seems to be a necessity for you to have Normalized table structure.
CREATE TABLE [dbo].[MyRoutes](
[id] [int] IDENTITY(1,1) NOT NULL,
[Route_No] [nvarchar](50) NULL,
[Route_Code] [nvarchar](50) NULL,
[Color] [nvarchar](50) NULL,
[Start_Point] [nvarchar](200) NULL,
[End_Point] [nvarchar](200) NULL,
[HaltNum] INT,
[Halt] [nvarchar](50) NULL
)
Then a query to solve the routes problem can be written as below:
SELECT a.Route_No, a.Route_Code, a.Color, a.Start_Point, a.End_Point,
a.HaltNum StartNum, b.HaltNum StopNum
FROM MyRoutes a
INNER JOIN MyRoutes b
ON a.id = b.id
WHERE a.Halt = 'jaipur' AND b.Halt = 'vasai'
AND a.HaltNum < b.HaltNum
Even better design of the table structure would be to have a separate master table for all Stops where you can maintain only StopId and StopName. In the MyRoutes table then you can have HaltId as foreign key referencing StopId column of the all stops master table. The above query would then need to inner join with this table twice to have conditions on StopName