Multi Parent-Child Insertion - sql

I'm trying to write a stored procedure to pull information from an XML string and use it to create multiple parent-child relationships. I am trying to push this XML into actual database tables. Basically, the local client will send an XML file to the database and store it as a string. I then need to pull the information out of that string and update the appropriate tables. If this was just a Table-A to Table-B, this wouldn't be so difficult. The problem I'm running into is it need to go from Table-A to Table-B to Table-C to Table-D where applicable. Below is a sample XML:
<RunRecordFile>
<Competition>
<Name>Daily</Name>
<StartDate>11/9/2015 12:40:07 AM</StartDate>
<Runs>
<Id>123</Id>
<Name>Daily Run</Name>
<RunDate>11/9/2015 12:40:07 AM</RunDate>
<CompetitionId>1</CompetitionId>
<RunRecords>
<Id>001</Id>
<Number>007</Number>
<ElapsedTime>23.007</ElapsedTime>
<RunId>123</RunId>
</RunRecords>
</Runs>
<Runs>
<Id>456</Id>
<Name>Daily Run</Name>
<RunDate>11/9/2015 12:47:07 AM</RunDate>
<CompetitionId>1</CompetitionId>
<RunRecords>
<Id>002</Id>
<Number>700</Number>
<ElapsedTime>23.707</ElapsedTime>
<RunId>456</RunId>
<RunRecordSpecialty>
<Id>1</Id>
<Handicap>17</Handicap>
<TeamPoints>50000</TeamPoints>
<RunRecordId>002</RunRecordId>
</RunRecordSpecialty>
</RunRecords>
</Runs>
</Competition>
</RunRecordFile>
I've attempted to use a DECLARED table to hold each of the created Primary Keys and to use SQL OUTPUT in order to gather those. When I run my SQL I'm getting (0) Rows Updated. Here's what I've tried in SQL:
CREATE PROC [dbo].[RaceFilePush]
AS
DECLARE #CompetitionIdMapping TABLE ( CompetitionId bigint )
DECLARE #RunIdMapping TABLE ( RunId bigint )
DECLARE #RunRecordIdMapping TABLE ( RunRecordId bigint )
BEGIN
DECLARE #rrXML AS XML
DECLARE #rrfId AS BIGINT
SET #rrfId = (SELECT TOP 1 Id FROM RunRecordFile WHERE Submitted IS NULL)
SET #rrXML = (SELECT TOP 1 RaceFile FROM RunRecordFile WHERE Id = #rrfId)
BEGIN TRAN Competitions
BEGIN TRY
INSERT INTO Competition (
Name
,StartDate
)
OUTPUT INSERTED.Id INTO #CompetitionIdMapping(CompetitionId)
SELECT
xCompetition.value('(Name)[1]', 'varchar(225)') AS Name
,xCompetition.value('(StartDate)[1]', 'datetime') AS StartDate
,#rrfId AS RunRecordFileId
FROM
#rrXML.nodes('/RunRecordFile/Competition') AS E(xCompetition)
INSERT INTO Run (
Name
,RunDate
,CompetitionId
)
OUTPUT INSERTED.Id INTO #RunIdMapping(RunId)
SELECT
xRuns.value('(Name)[1]','varchar(80)') AS Name
,xRuns.value('(RunDate)[1]','datetime') AS RunDate
,(SELECT CompetitionId FROM #CompetitionIdMapping)
FROM
#rrXML.nodes('/RunRecordFile/Competition/Runs') AS E(xRuns)
INSERT INTO RunRecord (
Number
,ElapsedTime
,RunId
)
OUTPUT INSERTED.Id INTO #RunRecordIdMapping(RunRecordId)
SELECT
xRunRecords.value('(Number)[1]','varchar(10)') AS Number
,xRunRecords.value('(ElapsedTime)[1]','numeric(10,5)') AS ElapsedTime
,(SELECT RunId FROM #RunIdMapping)
FROM
#rrXML.nodes('/RunRecordFile/Competition/Runs/RunRecords') AS E(xRunRecords)
INSERT INTO RunRecordSpecialty (
Handicap
,TeamPoints
,RunRecordId
)
SELECT
xRunRecordSpecialty.value('(Handicap)[1]','numeric(10,5)') AS Handicap
,xRunRecordSpecialty.value('(TeamPoints)[1]','numeric(10,5)') AS TeamPoints
,(SELECT RunRecordId FROM #RunRecordIdMapping)
FROM
#rrXML.nodes('/RunRecordFile/Competition/Runs/RunRecordSpecialty') AS E(xRunRecordSpecialty)
UPDATE RunRecordFile SET Submitted = GETDATE() WHERE Id = #rrfId
COMMIT TRAN Competitions
END TRY
BEGIN CATCH
ROLLBACK TRAN Competitions
END CATCH
END

With this SQL you get the whole thing into a flat declared table #tbl:
Remark: I placed the XML from your question into a variable called #xml. Adapt this to your needs...
DECLARE #tbl TABLE (
[Competition_Name] [varchar](max) NULL,
[Competition_StartDate] [datetime] NULL,
[Run_Id] [int] NULL,
[Run_Name] [varchar](max) NULL,
[Run_RunDate] [datetime] NULL,
[Run_CompetitionId] [int] NULL,
[RunRecords_Id] [int] NULL,
[RunRecords_Number] [int] NULL,
[RunRecords_ElapsedTime] [float] NULL,
[RunRecords_RunId] [int] NULL,
[RunRecordSpecialty_Id] [int] NULL,
[RunRecordSpecialty_Handicap] [int] NULL,
[RunRecordSpecialty_TeamPoints] [int] NULL,
[RunRecordSpecialty_RunRecordId] [int] NULL
);
INSERT INTO #tbl
SELECT Competition.value('Name[1]','varchar(max)') AS Competition_Name
,Competition.value('StartDate[1]','datetime') AS Competition_StartDate
,Run.value('Id[1]','int') AS Run_Id
,Run.value('Name[1]','varchar(max)') AS Run_Name
,Run.value('RunDate[1]','datetime') AS Run_RunDate
,Run.value('CompetitionId[1]','int') AS Run_CompetitionId
,RunRecords.value('Id[1]','int') AS RunRecords_Id
,RunRecords.value('Number[1]','int') AS RunRecords_Number
,RunRecords.value('ElapsedTime[1]','float') AS RunRecords_ElapsedTime
,RunRecords.value('RunId[1]','int') AS RunRecords_RunId
,RunRecordSpecialty.value('Id[1]','int') AS RunRecordSpecialty_Id
,RunRecordSpecialty.value('Handicap[1]','int') AS RunRecordSpecialty_Handicap
,RunRecordSpecialty.value('TeamPoints[1]','int') AS RunRecordSpecialty_TeamPoints
,RunRecordSpecialty.value('RunRecordId[1]','int') AS RunRecordSpecialty_RunRecordId
FROM #xml.nodes('/RunRecordFile/Competition') AS A(Competition)
OUTER APPLY Competition.nodes('Runs') AS B(Run)
OUTER APPLY Run.nodes('RunRecords') AS C(RunRecords)
OUTER APPLY RunRecords.nodes('RunRecordSpecialty') AS D(RunRecordSpecialty)
;
SELECT * FROM #tbl
If you need generated IDs, you just add the columns to #tbl and write them there, either "on the flow" or afterwards wiht UPDATE statement.
It should be easy to work through this flat table, select just the needed data level with DISTINCT and insert the rows, then the next level and so on...
Good luck!

Related

How do I pass values from a table to a stored procedure as parameters?

I'm struggling with something that looks very simple, and yet I'm not finding any solution:
I have Table called Hotels, which the first column in it, is = [hotels_id] [int]
I have stored procedure that depends on hotels_id value
Now in the Stored Procedure, I've typed just one value (1) as example, let say I have something like 50 rows.
There is any way to pass all the rows from the table to the SP as parameter each one at the time? I mean one by one.
ALTER PROCEDURE [dbo].[spAvailableRooms]
AS
BEGIN
-- Derived tables
;with
DatesTable as
(SELECT top (100) PERCENT Dates.dates_id as Id, Dates.dates_date as Date FROM Dates order by Dates.dates_id),
AvaliablePerHotel as
(SELECT top (100)percent Available_since AS Since, Available_value AS Value, Available_hotel as Hotel
FROM Available_rooms
where Available_hotel =1 --(HERE I NEED THE VALUES FROM TABLE)
ORDER BY Available_hotel, Available_since),
AllDays as
(Select top (100)percent Id, Date, Value as Rooms, iif(value is null, '0' ,'1') as New, Hotel
From DatesTable left JOIN AvaliablePerHotel ON Id = Since
order by id),
AvailableGroups as
(Select top (100)percent Hotel, Id, Date, Rooms, (sum(isnull(cast(new as float),0))over(order by id)) as RowGroup
From AllDays
order by id)
--
-- Query itself
Select Id, Date, iif(Rooms is null,(first_value(rooms) over (partition by RowGroup order by Id)) , Rooms) as AvailableRooms,
iif(Hotel is null,(first_value(Hotel) over (partition by RowGroup order by Id)) , Hotel) as Hotel
From AvailableGroups
order by id
END
If you want to pass values as you said one by one then cursor is your best option
declare #id int
DECLARE cursor CURSOR FOR
SELECT hotels_id
FROM Hotels
OPEN cursor
FETCH NEXT FROM cursor
INTO #id
WHILE ##FETCH_STATUS = 0
BEGIN
exec [dbo].[spAvailableRooms] #id
FETCH NEXT FROM cursor
INTO #id
END
CLOSE cursor;
DEALLOCATE cursor;
Then change in your procedure pass a paramter
ALTER PROCEDURE [dbo].[spAvailableRooms] #id int
And replace where you had hardcoded "1" in your code with #id
Note that cursors are single threaded as your table grows the execution time will grow pretty rapidly.
You can use User-Defined Table Types (more info here)
First you define a type that represents the structure you want to pass to the stored procedure:
CREATE TYPE [schemaName].[typeName] AS TABLE(
[Column0] [nvarchar](255) NULL,
[Column1] [nvarchar](255) NULL,
[Column2] [nvarchar](255) NULL,
[Column3] [nvarchar](255) NULL,
[Column4] [nvarchar](255) NULL,
[Column5] [nvarchar](255) NULL,
[Column6] [nvarchar](255) NULL,
...
)
Now you can create a stored procedure that takes as input a variable of the type defined with the previous script:
CREATE PROCEDURE [schemaName].[SpLoad]
(
#myRows [schemaName].[typeName] READONLY
)
AS
BEGIN
INSERT INTO schemaName.DestinationTable
SELECT * FROM #myRows
END

How do I create a dataset in SQL Server and process each record?

I have the following table:
CREATE TABLE [dbo].[proposalReviewAction]
(
[proposalReviewActionID] [INT] IDENTITY(1,1) NOT NULL,
[proposalPackageID] [INT] NULL,
[approvalTypeID] [INT] NULL,
[comments] [VARCHAR](2000) NULL,
[reviewedByID] [INT] NULL,
[reviewedDate] [DATETIME] NULL
) ON [PRIMARY]
I want to return each record that has an approvalTypeID = 100 and use the proposalPackageID in another process that moves the data from the proposal table to the project table.
I considered using a WHILE loop, but everyone seems to favor processing a dataset but I cannot find any way to do this in SQL/T-SQL.
I found an example of a WHILE loops and modified it ...
WHILE (SELECT[approvalTypeID] FROM [proposalReviewAction]) = 100
BEGIN
DECLARE #ppID AS INT ;
SELECT [proposalPackageID] AS ppID, [approvalTypeID] AS atID
FROM [proposalReviewAction]
IF (SELECT [approvalTypeID] AS atID FROM [proposalReviewAction]) = 100
PRINT #ppID
ELSE
BREAK
END
PRINT 'I got to the print statement';
... but I get the following error:
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
Then, it prints the "PRINT" statement.
Any help is appreciated...
Bob
You can simply insert these records directly to your ProjectTable! like:
INSERT INTO [ProjectTable] (...)
SELECT proposalPackageID
FROM [proposalReviewAction]
WHERE approvalTypeID = 100
If you need a loop anyway for other reasons you can use a cursor instead of a WHILE. Using a cursor helps you have results of a query in variables record by record and process each record separately :
DECLARE #proposalPackageID INT
DECLARE product_cursor CURSOR FOR
SELECT proposalPackageID
FROM dbo.proposalReviewAction
WHERE approvalTypeID = 100
OPEN product_cursor
FETCH NEXT FROM product_cursor INTO #proposalPackageID
WHILE ##FETCH_STATUS = 0
BEGIN
-- Insert directly into projectTable or call external process
INSERT INTO [projectTable] (proposalPackageID ...)
SELECT #proposalPackageID
FETCH NEXT FROM product_cursor INTO #proposalPackageID
END
CLOSE product_cursor
DEALLOCATE product_cursor
Hope this will help
Given this table:
CREATE TABLE [dbo].[proposalReviewAction]
(
[proposalReviewActionID] [INT] IDENTITY(1,1) NOT NULL,
[proposalPackageID] [INT] NULL,
[approvalTypeID] [INT] NULL,
[comments] [VARCHAR](2000) NULL,
[reviewedByID] [INT] NULL,
[reviewedDate] [DATETIME] NULL
) ON [PRIMARY]
You can select all records with approvalTypeID = 100 by using a simple SELECT statement
SELECT * FROM dbo.proposalReviewAction WHERE approvalTypeID = 100;
If you only need proposalPackageID then you can SELECT just it using the QUERY
SELECT proposalPackageID FROM dbo.proposalReviewAction WHERE approvalTypeID = 100;
Depending on the processing you could insert directly from here to the 'project table' as well
INSERT INTO [projectTable] (proposalPackageID ...)
SELECT proposalPackageID, ...
FROM dbo.proposalReviewAction WHERE approvalTypeID = 100;
select column
from table
where columns_name
in ( select column from table where columns_name = 'value');
NOTE: If we use sub query and it return one value in this case we can use (=,!=,<>,<,>....)
Or more than one values, in this case we should use (in,any,all,some )

SQL Query Optimization with million of records

I am currently using below query to get record based on senderid. I have 2 million record in messagein table and also entries comes parallel in this table as well.
But it take morethen 5 sec to return result. This table having only one non-clustered index created on Providerid (include column priorityid, senderid,maskid)
Can any one sql expert help me out on this.
ALTER PROCEDURE [dbo].[GetNextSmsQueue] #NoOfRow int,
#GatewayId int
AS
BEGIN TRY
BEGIN TRAN;
CREATE TABLE #SmsIn ([Id] [bigint] NOT NULL,
[UserId] [bigint] NOT NULL,
[MaskId] [varchar](50) NOT NULL,
[Number] [varchar](20) NOT NULL,
[Message] [nvarchar](1300) NOT NULL,
[SenderId] [varchar](20) NOT NULL,
[UDH] [nvarchar](50) NULL,
[Credit] [int] NOT NULL,
[CurrentStatus] [int] NOT NULL,
[CheckDND] [bit] NULL,
[CheckFail] [bit] NULL,
[CheckBlackList] [bit] NULL,
[ProviderId] [int] NULL,
[PriorityId] [int] NULL,
[ScheduleDate] [datetime] NOT NULL,
[CreatedDate] [datetime] NOT NULL,
[EsmClass] [nvarchar](10) NOT NULL,
[DataCoding] [int] NOT NULL,
[Priority] [int] NOT NULL,
[Interface] [int] NOT NULL);
DECLARE #PriorityIn table ([PriorityId] [int] NOT NULL);
DECLARE #COUNT bigint;
INSERT INTO #PriorityIn
SELECT PriorityId
FROM PriorityProviders
WHERE ProviderId = #GatewayId
AND Type = 0;
SELECT #COUNT = COUNT(*)
FROM MessageIn m
LEFT JOIN #PriorityIn o ON m.PriorityId = o.PriorityId
WHERE ((ProviderId IS NULL
AND o.PriorityId IS NOT NULL)
OR ProviderId = #GatewayId);
IF #COUNT > 0
BEGIN
INSERT INTO #SmsIn ([Id],
[UserId],
[MaskId],
[Number],
[Message],
[SenderId],
[UDH],
[Credit],
[CurrentStatus],
[CheckDND],
[CheckFail],
[CheckBlackList],
[ProviderId],
[PriorityId],
[ScheduleDate],
[CreatedDate],
[EsmClass],
[DataCoding],
[Priority],
[Interface])
(SELECT [Id],
[UserId],
[MaskId],
[Number],
[Message],
[SenderId],
[UDH],
[Credit],
[CurrentStatus],
[CheckDND],
[CheckFail],
[CheckBlackList],
[ProviderId],
[PriorityId],
[ScheduleDate],
[CreatedDate],
[EsmClass],
[DataCoding],
[Priority],
[Interface]
FROM MessageIn
WHERE MaskId IN (SELECT MaskId
FROM (SELECT ROW_NUMBER() OVER (PARTITION BY SenderId ORDER BY ScheduleDate) AS RowNo,
MaskId
FROM MessageIn msg
LEFT JOIN #PriorityIn o ON msg.PriorityId = o.PriorityId
WHERE ((msg.ProviderId IS NULL
AND o.PriorityId IS NOT NULL)
OR msg.ProviderId = #GatewayId)) res
WHERE res.RowNo <= #NoOfRow));
DELETE msgin
FROM MessageIn msgin
INNER JOIN #SmsIn temp ON msgin.MaskId = temp.MaskId;
END;
SELECT *
FROM #SmsIn;
DROP TABLE #SmsIn;
COMMIT;
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0
BEGIN
ROLLBACK TRANSACTION;
END;
END CATCH;
Execution plan is available in Here:
Execution Plan
Updated Query:
BEGIN TRY
begin tran;
CREATE TABLE #tmpMaskId (MaskId varchar(25) PRIMARY KEY)
INSERT INTO #tmpMaskId(MaskId)
SELECT DISTINCT MaskId From
(SELECT ROW_NUMBER() OVER ( PARTITION BY SenderId ORDER BY scheduledate ) AS RowNo, MaskId FROM MessageIn msg
LEFT JOIN PriorityProviders o on o.ProviderId = #GatewayId AND o.Type = 0 and msg.PriorityId = o.PriorityId
WHERE
((msg.ProviderId is null AND o.PriorityId is not null) OR msg.ProviderId = #GatewayId)
)as res WHERE res.RowNo <= #NoOfRow
Select [Id],[UserId],m.[MaskId],[Number],[Message],[SenderId],[UDH],[Credit],[CurrentStatus],[CheckDND],[CheckFail],[CheckBlackList],[ProviderId]
,[PriorityId],[ScheduleDate],[CreatedDate],[EsmClass],[DataCoding],[Priority],[Interface]
From MessageIn m inner join #tmpMaskId msk on m.MaskId = msk.MaskId
DELETE msgin
FROM MessageIn msgin
INNER JOIN #tmpMaskId temp ON msgin.MaskId=temp.MaskId
DROP TABLE #tmpMaskId
Commit;
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0
BEGIN
ROLLBACK TRANSACTION;
END
END CATCH;
ALTER PROCEDURE [dbo].[GetNextSmsQueue]
#NoOfRow INT
,#GatewayId INT
AS
BEGIN TRY
BEGIN TRAN;
CREATE TABLE #tmpMaskId (MaskId INT PRIMARY KEY)
DECLARE #PriorityIn TABLE ([PriorityId] [INT] NOT NULL)
INSERT INTO #PriorityIn
SELECT PriorityId
FROM PriorityProviders
WHERE ProviderId=#GatewayId AND Type=0
INSERT INTO #tmpMaskId (MaskId)
SELECT DISTINCT MaskId
FROM (
SELECT ROW_NUMBER() OVER (PARTITION BY SenderId ORDER BY ScheduleDate) AS RowNo
,MaskId
FROM MessageIn msg
WHERE ((msg.ProviderId IS NULL AND o.PriorityId IS NOT NULL) OR msg.ProviderId=#GatewayId)
) res
WHERE res.RowNo<=#NoOfRow
SELECT [Id]
,[UserId]
,[MaskId]
,[Number]
,[Message]
,[SenderId]
,[UDH]
,[Credit]
,[CurrentStatus]
,[CheckDND]
,[CheckFail]
,[CheckBlackList]
,[ProviderId]
,[PriorityId]
,[ScheduleDate]
,[CreatedDate]
,[EsmClass]
,[DataCoding]
,[Priority]
,[Interface]
FROM MessageIn mi
WHERE EXISTS (SELECT 1 FROM #tmpMaskId AS tmi WHERE tmi.MaskId=mi.MaskId)
DELETE msgin
FROM MessageIn msgin
INNER JOIN #tmpMaskId temp ON msgin.MaskId=temp.MaskId
COMMIT;
END TRY
BEGIN CATCH
IF ##TRANCOUNT>0
BEGIN
ROLLBACK TRANSACTION;
END;
END CATCH;
DROP TABLE #tmpMaskId
IMO, As per your requirement, I will only return record from this proc to send sms.After sms is successfully I send only require id from Message table to another proc to delete those records.
Technically it sound good.your existing proc is not slow because of delete.but its not ok to delete before sending sms and again trying to insert.
In my previous scipt,I pointed that you do not need Join on PriorityProviders.
I have revise my script(INNER if possible),
SET NOCOUNT ON
BEGIN TRY
begin tran;
CREATE TABLE #tmpMaskId (MaskId varchar(25) not null)
INSERT INTO #tmpMaskId(MaskId)
SELECT MaskId From
(SELECT ROW_NUMBER() OVER ( PARTITION BY SenderId ORDER BY scheduledate ) AS RowNo, MaskId FROM MessageIn msg with(nolock)
LEFT JOIN PriorityProviders with(nolock)
o on o.ProviderId = msg.ProviderId and o.ProviderId= #GatewayId AND o.Type = 0 and msg.PriorityId = o.PriorityId
WHERE
((msg.ProviderId is null AND o.PriorityId is not null) OR msg.ProviderId = #GatewayId)
)as res WHERE res.RowNo <= #NoOfRow
CREATE TABLE #tmpMaskId (MaskId INT not null)
create clusetered index ix_mask on #tmpMaskId
Select [Id],[UserId],m.[MaskId],[Number],[Message],[SenderId],[UDH],[Credit],[CurrentStatus]
,[CheckDND],[CheckFail],[CheckBlackList],[ProviderId]
,[PriorityId],[ScheduleDate],[CreatedDate],[EsmClass],[DataCoding],[Priority],[Interface]
From MessageIn m
inner join
#tmpMaskId msk
on m.MaskId = msk.MaskId
DELETE msgin
FROM MessageIn msgin
where exists(select 1 from #tmpMaskId temp where msgin.MaskId=temp.MaskId)
DROP TABLE #tmpMaskId
Commit;
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0
BEGIN
ROLLBACK TRANSACTION;
END
END CATCH;
notice how I hv remove PK from Temp table and made it clustered Index.
How I hv remove distinct ?
Now main culprit is this statement,
ROW_NUMBER() OVER ( PARTITION BY SenderId ORDER BY scheduledate ) AS RowNo
I think once if you comment it then proc will perform better.
Now you need only index.
Whichever column is most Selective make that column as Clustered Index.
Since I do not know selectivity of each column I can't say whether you should make composite clustered or composite Non clustered index.
If you go for Composite Non Clustered index then make ID as clustered index(PK) and keep the most selective column on left side and so on
Composite Non Clustered index can be (maskid,ProviderId,SenderId,PriorityId)Include(other columns of message table which are require in Resultset)
I am not telling you to remove Row_number().Create composite non clustered index and rebuild index as I have describe above.
With (nolock) :It has nothing to do with data duplicity.
If there is no chance of getting uncommitted data.
If there is not much concurrency issue in message table and is not very frequently insert/updated.
Then you can safely use it.you can Google this "Advantage and disadvantage of with (Nolock)".
In one or two places you can use it if it improve your important query.
Like you said if you create index on maskid then it create deadlock.That is because of faulty script in Insert.

Stored procedure to Insert data between tables

I want to insert data from a table called temp_menu into another called menu.
They have the same structure, they store the same data, I want to create a stored procedure to check the differences between the tables. If there are any different rows and the rows don't exist in table menu, I want to insert them into menu; if the rows exists, I want to update the rows in menu if the DateReg column is higher that the DateReg column in the temp_menu table.
The tables have this structure:
CREATE TABLE [dbo].[Menu_Temp]
(
[Date] [datetime] NOT NULL,
[Ref] [int] NOT NULL,
[Art] [char](60) NOT NULL,
[Dish] [char](60) NOT NULL,
[DateReg] [datetime] NOT NULL,
[Zone] [char](60) NOT NULL,
);
I have this code to check for differences between the tables:
SELECT *
INTO #diffs
FROM [Regi].dbo.menu
EXCEPT
SELECT * FROM [Regi].dbo.menu_Temp
IF ##ROWCOUNT = 0
RETURN
SELECT * FROM #diffs
Full details are here : https://learn.microsoft.com/en-us/sql/t-sql/statements/merge-transact-sql
An example for your situation could be...
MERGE
[Regi].dbo.menu
USING
[Regi].dbo.menu_Temp
ON (menu_Temp.[Ref] = menu.[Ref]) -- Assumes [Ref] is the identifying column?
WHEN
MATCHED AND (menu_Temp.[DateReg] > menu.[DateReg])
THEN
UPDATE SET [Art] = menu_Temp.[Art],
[Dish] = menu_Temp.[Dish],
[Zone] = menu_Temp.[Zone],
[Date] = menu_Temp.[Date],
[DateReg] = menu_Temp.[DateReg]
WHEN
NOT MATCHED
THEN
INSERT (
[Date],
[Ref],
[Art],
[Dish],
[DateReg],
[Zone]
)
VALUES (
menu_Temp.[Date],
menu_Temp.[Ref],
menu_Temp.[Art],
menu_Temp.[Dish],
menu_Temp.[DateReg],
menu_Temp.[Zone]
)
http://dbfiddle.uk/?rdbms=sqlserver_2017&fiddle=b47d7e879856ffe6210589f6bb64829f

optimal solution for lookup on large sql server table

I have a large table of user ids and another table of user records which contains a user post with user ids. The process is whenever a new feed post is retrieved,I do a request to the user id table for an id that is marked inactive ( I have that field ACTIVE because I have another process that creates these ids and inserts it continuously into table 1) and when an id is requested it is marked as inactive.
Then I check if the user exists in the user table(table 2) and if so return the user id associated with that user.
I was told that I can speed up this process but creating a hash table to do the lookup on table 2. I am not sure how to even start this and any links or samples will be appreciated.
Also I need to run a separate process that cleans table 1 and removes all inactive user ids.
When I call the procedure to insert into table 2, I pass the user id retrieved from table 1.
CREATE TABLE [dbo].[userforums]
(
[userid] [VARCHAR](16) NOT NULL CONSTRAINT [PK_forumssiteid] PRIMARY KEY CLUSTERED ,
[forumname] [VARCHAR](500) NOT NULL,
[exported] [INT] NULL,
[lastcrawled] [DATETIME] NULL,
[priority] [INT] NULL,
[origin] [VARCHAR](50) NULL,
[queryid] [VARCHAR](25) NULL,
[dateinserted] [DATETIME] NULL DEFAULT (getdate())
)
second table
CREATE TABLE [dbo].[userids]
(
[userid] [NVARCHAR](20) NOT NULL CONSTRAINT [PK_userids] PRIMARY KEY CLUSTERED,
[active] [NVARCHAR](20) NULL CONSTRAINT [IX_userids] UNIQUE NONCLUSTERED
)
get user id stored procedure
BEGIN TRANSACTION
SELECT TOP 1 #id = userid
FROM userids WITH (UPDLOCK, HOLDLOCK)
WHERE active = 'Y'
OR active IS NULL
UPDATE userids
SET active = 'N'
WHERE userid = #id
COMMIT TRANSACTION
check if userid exists
CREATE PROC Foo #forumname VARCHAR(500),
#userid VARCHAR(16),
#origin VARCHAR(50),
#queryid VARCHAR(25)
AS
SET NOCOUNT ON;
DECLARE #cnt INT
DECLARE #serverip VARCHAR(16)
DECLARE #mincnt INT
DECLARE #siteservercnt INT
SELECT #cnt = COUNT(*)
FROM userforums
WHERE forumname = #forumname
IF #cnt = 0
BEGIN
INSERT INTO userforums
(forumname,
userid,
exported,
origin,
queryid)
VALUES (#forumname,
#userid,
1,
#origin,
#queryid)
SELECT #siteservercnt = COUNT(*)
FROM siteserverip
WHERE userid = #userid
IF #siteservercnt = 0
BEGIN
SELECT TOP 1 #mincnt = COUNT(*),
#serverip = serverip
FROM siteserverip
GROUP BY serverip
ORDER BY COUNT(*)
SELECT TOP 1 #mincnt = sitecount,
#serverip = serverip
FROM serveripcounts
ORDER BY sitecount
INSERT INTO siteserverip
VALUES (#siteid,
#serverip)
UPDATE serveripcounts
SET sitecount = sitecount + 1
WHERE serverip = #serverip
END
END
SELECT userid
FROM userforums
WHERE forumname = #forumname
RETURN
Your existing dequeue query can be improved. Instead of
DECLARE #id INT
SELECT TOP 1 #id = userid
FROM userids WITH (UPDLOCK, HOLDLOCK)
WHERE active = 'Y'
OR active IS NULL
UPDATE userids
SET active = 'N'
WHERE userid = #id
Which is two operations (a clustered index scan followed by an index seek) you can do
UPDATE TOP (1) userids
WITH (ROWLOCK, READPAST)
SET active = 'N'
OUTPUT INSERTED.userid
WHERE active <> 'N'
Which is one operation and gives a plan with two range seeks.
A Hash table #TableName is a temporary object in tempdb that functions as a table. They are generally called 'temp tables'. I would NOT be using them as a first solution for retrieving data on the fly if this is a common occurrence. Instead I would create an index and see if that justifies your needs. Generally hash tables are used for intense operations where you want to get a set of things that may or may not be indexed and then relate it to something else and you want to keep it in memory.
I would create an index and that should improve speed. Also if you find is slow, a hash table won't speed that part up, it will just be putting a collection of that into a source to reuse seperated from the main table.
create index IX_[yourtableName]_[TableColumn(s)] on [Tablename]([Column(s)]
I would not create more objects unless necessary. Generally if your UserId's are valid ints you can search on them quite fast.