I'm running the following query:
select
max(count) Max
from
(select
count(iStockID) Count
from
_etblInvJrBatchLines
group by
iStockID) X
As you know, the result of this is an integer; in this case, the result was 5.
Based on the above result, I need to insert generic values into another table, that looks like this:
INSERT INTO _etblInvJrBatches
(
cInvJrNumber -- IJ0001 Plus 1
, cInvJrDescription -- Inventory Journal Batch
, cInvJrReference -- IJR10001 Plus 1
, iCreateAgentID -- 1
, bClearAfterPost -- 1
, bAllowDupRef -- 1
, bAllowEditGLContra -- 0
, iNewLineDateOpt -- 0
, iNewLineRefOpt -- 0
, cNewLineRefDef -- ''
, bNewLineRefInc -- 0
, iNewLineDescOpt -- 0
, cNewLineDescDef -- ''
, bNewLineDescInc -- 0
, iNewLineProjectOpt -- 0
, iNewLineProjectDefID -- 0
, iNewLineWarehouseOpt -- 0
, iNewLineWarehouseDefID -- 0
, bJustCleared -- 0
, iTransactionCode -- 31 (select TrCodeID where TrCode = 'ADJ')
)
SELECT
'IJ000' -- Plus 1
, 'Inventory Journal Batch'
, 'IJR1000' -- Plus 1
, 1
, 1
, 1
, 0
, 0
, 0
, ''
, 0
, 0
, ''
, 0
, 0
, 0
, 0
, 0
, 0
, (select idTrCodes from TrCodes where Code = 'ADJ')
The only problem is, it only inserts this once.
How do I insert this 5 times based on the results I get from my first select statement?
In other words, if the integer result is 24, it needs to import / insert the above 24 times.
Thank you for assisting.
Attie.
Cant you just use a tally table?
declare #count int
set #count = (select
max(count) Max
from
(select
count(iStockID) Count
from
_etblInvJrBatchLines
group by
iStockID) X) --Hardcoded 10
IF OBJECT_ID('tempdb..#tally') IS NOT NULL
/*Then it exists*/
DROP TABLE #tally
SELECT TOP (#count) --equates to more than 30 years of dates
IDENTITY(INT,1,1) AS N
INTO #tally
FROM Master.dbo.SysColumns sc1,
Master.dbo.SysColumns sc2
--PRINT #count
INSERT INTO _etblInvJrBatches
(
cInvJrNumber -- IJ0001 Plus 1
, cInvJrDescription -- Inventory Journal Batch
, cInvJrReference -- IJR10001 Plus 1
, iCreateAgentID -- 1
, bClearAfterPost -- 1
, bAllowDupRef -- 1
, bAllowEditGLContra -- 0
, iNewLineDateOpt -- 0
, iNewLineRefOpt -- 0
, cNewLineRefDef -- ''
, bNewLineRefInc -- 0
, iNewLineDescOpt -- 0
, cNewLineDescDef -- ''
, bNewLineDescInc -- 0
, iNewLineProjectOpt -- 0
, iNewLineProjectDefID -- 0
, iNewLineWarehouseOpt -- 0
, iNewLineWarehouseDefID -- 0
, bJustCleared -- 0
, iTransactionCode -- 31 (select TrCodeID where TrCode = 'ADJ')
)
select N, 'IJ000' -- Plus 1
, 'Inventory Journal Batch'
, 'IJR1000' -- Plus 1
, 1
, 1
, 1
, 0
, 0
, 0
, ''
, 0
, 0
, ''
, 0
, 0
, 0
, 0
, 0
, 0 from #tally where n >0 and n <= #count
Result
There are many ways to accomplish this.
INSERT only inserts one row into the target table. Ultimately you are going to have to INSERT in a loop.
Possible solutions...
Do the looping in the database...
declare #count int = (select
max(count) Max
from
(select
count(iStockID) Count
from
_etblInvJrBatchLines
group by
iStockID) X)
declare #increment int =1
while #increment <= #count
{
-- do your insert here...
#increment = #increment +1
}
Do the looping in your client code.
Retrieve the count value from your initial query
For increment = 1 to #count
'execute SQL to do insert here...
Next increment
Or better (as #scsimon hinted at)...
For increment = 1 to #count
'build the VALUES () clauses for your insert statement...
Next
'execute your insert statement
I didn't realize that T-SQL allowed multiple VALUES clauses in an INSERT. Thank you #scsimon!
If it's me, I'm doing all of this in a client with code, not in the database. I'm old school, and I don't think that solutions like this scale well when built in the database as in my first example.
Thank you everyone for your contribution!
Special thanks to #Thomas as his advice worked the best in my environment.
I altered his script a little bit to accommodate the reference numbering:
declare #count int
set #count = (select
max(count) Max
from
(select
count(iStockID) Count
from
_etblInvJrBatchLines
group by
iStockID) X) --Hardcoded 10
IF OBJECT_ID('tempdb..#tally') IS NOT NULL
/*Then it exists*/
DROP TABLE #tally
SELECT TOP (#count) --equates to more than 30 years of dates
IDENTITY(INT,1,1) AS N
INTO #tally
FROM Master.dbo.SysColumns sc1,
Master.dbo.SysColumns sc2
--PRINT #count
INSERT INTO _etblInvJrBatches
(
cInvJrNumber -- IJ0001
, cInvJrDescription -- Inventory Journal Batch
, cInvJrReference -- IJR10001
, iCreateAgentID -- 1
, bClearAfterPost -- 1
, bAllowDupRef -- 1
, bAllowEditGLContra -- 0
, iNewLineDateOpt -- 0
, iNewLineRefOpt -- 0
, cNewLineRefDef -- ''
, bNewLineRefInc -- 0
, iNewLineDescOpt -- 0
, cNewLineDescDef -- ''
, bNewLineDescInc -- 0
, iNewLineProjectOpt -- 0
, iNewLineProjectDefID -- 0
, iNewLineWarehouseOpt -- 0
, iNewLineWarehouseDefID -- 0
, bJustCleared -- 0
, iTransactionCode -- 31 (select idTrCodes from TrCodes where Code = 'ADJ')
)
select
'IJ000' + cast(N as varchar)
, 'Inventory Journal Batch'
, 'IJR1000' + cast(N as varchar)
, 1
, 1
, 1
, 0
, 0
, 0
, ''
, 0
, 0
, ''
, 0
, 0
, 0
, 0
, 0
, 0
, (select idTrCodes from TrCodes where Code = 'ADJ')
from #tally where n >0 and n <= #count
This worked like a charm!
See results:Results
Related
I already asked a similar question and Ian Peters and Nik Shenoy provided awesome solutions, but I forgot to mention that performance is a very interesting aspect. So I decided to ask a new question providing some more insights.
I'm maintaining an application which allows visual analyzation of a certain kind log files. The processing of log entries is mainly driven by stored procedures in a SQL Server database. One step of this processing is to relate the log entries to their corresponding workers and worker steps and to know when this worker instance was executed the last time.
Here is an example of the table containing the log entries:
Id , Time, LogCategory , LogMessage
1 , 100 , WORKER_START , Started worker
2 , 110 , WORKER_STEP_START, Entered Step 1
3 , 120 , SOMETHING_ELSE , Do cool stuff
4 , 130 , WORKER_STEP_END , Exited Step 1
5 , 140 , SOMETHING_ELSE , Some event here
6 , 150 , WORKER_START , Started worker
7 , 160 , SOMETHING_ELSE , Do more cool stuff
8 , 170 , WORKER_STEP_START, Entered Step 1
9 , 180 , SOMETHING_ELSE , Do cool stuff
10 , 190 , WORKER_STEP_END , Exited Step 1
11 , 200 , SOMETHING_ELSE , Bad event, exit worker
12 , 210 , WORKER_END , Exited worker
13 , 220 , WORKER_STEP_START, Entered Step 2
14 , 230 , SOMETHING_ELSE , Do further cool stuff
15 , 240 , WORKER_STEP_END , Exited Step 2
16 , 250 , WORKER_END , Exited worker
I want to
add the entry id of the worker this entry corresponds to
add the entry id of the worker step this entry corresponds to
update the WORKER_START entry and set the maximum timestamp of entries belonging to this worker.
This would result in:
Id , Time, LogCategory , LogMessage , Worker, Step , LastUpdate
1 , 100 , WORKER_START , Started worker , 1 , null , 250
2 , 110 , WORKER_STEP_START, Entered Step 1 , 1 , 2 , null
3 , 120 , SOMETHING_ELSE , Do cool stuff , 1 , 2 , null
4 , 130 , WORKER_STEP_END , Exited Step 1 , 1 , 2 , null
5 , 140 , SOMETHING_ELSE , Some event here , 1 , null , null
6 , 150 , WORKER_START , Started worker , 6 , null , 210
7 , 160 , SOMETHING_ELSE , Do more cool stuff , 6 , null , null
8 , 170 , WORKER_STEP_START, Entered Step 1 , 6 , 8 , null
9 , 180 , SOMETHING_ELSE , Do cool stuff , 6 , 8 , null
10 , 190 , WORKER_STEP_END , Exited Step 1 , 6 , 8 , null
11 , 200 , SOMETHING_ELSE , Bad event, exit worker , 6 , null , null
12 , 210 , WORKER_END , Exited worker , 6 , null , null
13 , 220 , WORKER_STEP_START, Entered Step 2 , 1 , 13 , null
14 , 230 , SOMETHING_ELSE , Do further cool stuff , 1 , 13 , null
15 , 240 , WORKER_STEP_END , Exited Step 2 , 1 , 13 , null
16 , 250 , WORKER_END , Exited worker , 1 , null , null
Currently I'm doing this by executing this stored procedure:
CREATE PROCEDURE CalculateRelations
AS
BEGIN
DECLARE entries_cur CURSOR FOR
SELECT Id, LogCategory
FROM LogEntries
ORDER BY Id;
DECLARE #Id BIGINT;
DECLARE #LogCategory VARCHAR(128);
DECLARE #Worker BIGINT;
DECLARE #WorkerStack VARCHAR(MAX) = '';
DECLARE #Step BIGINT;
DECLARE #StepStack VARCHAR(MAX) = '';
OPEN entries_cur;
FETCH NEXT FROM entries_cur INTO #Id, #LogCategory;
WHILE ##FETCH_STATUS = 0
BEGIN
EXEC dbo.GetParentWorker #WorkerStack OUT, #Id, #LogCategory, #Worker OUT;
EXEC dbo.GetParentStep #StepStack OUT, #Id, #LogCategory, #Step OUT;
UPDATE LogEntries
SET
Worker = #Worker,
Step = #Step
WHERE Id = #Id;
UPDATE LogEntries
SET LastUpdate = #Time
WHERE Id = #Worker;
FETCH NEXT FROM entries_cur INTO #Id, #LogCategory;
END;
CLOSE entries_cur;
DEALLOCATE entries_cur;
END;
GetParentWorker and GetParentStep are stored procedures using the given VARCHAR variables WorkerStack and StepStack as a stack. This means
"WORKER_START"/"WORKER_STEP_START" entry leads to adding (push) the Id to that VARCHAR
"WORKER_END"/"WORKER_STEP_END" entry leads to removing and returning (pop) the last Id from that VARCHAR
all other entries lead to just returning (read) the last Id from that VARCHAR without modifying it.
This procedure runs on ~850k entries in 24 minutes and 33 seconds. I also tried a slight modification of that stored procedure. There I switched from a normal cursor to an update cursor and I introduced a temporary table to calculate the LastUpdate of a worker:
CREATE PROCEDURE CalculateRelations
AS
BEGIN
DECLARE entries_cur CURSOR FOR
SELECT Id, LogCategory
FROM LogEntries
ORDER BY Id
FOR UPDATE OF Worker, Step;
DECLARE #Id BIGINT;
DECLARE #LogCategory VARCHAR(128);
DECLARE #Worker BIGINT;
DECLARE #WorkerStack VARCHAR(MAX) = '';
DECLARE #Step BIGINT;
DECLARE #StepStack VARCHAR(MAX) = '';
TRUNCATE TABLE LogEntries_LastUpdate;
OPEN entries_cur;
FETCH NEXT FROM entries_cur INTO #Id, #LogCategory;
WHILE ##FETCH_STATUS = 0
BEGIN
EXEC dbo.GetParentWorker #WorkerStack OUT, #Id, #LogCategory, #Worker OUT;
EXEC dbo.GetParentStep #StepStack OUT, #Id, #LogCategory, #Step OUT;
UPDATE LogEntries
SET
Worker = #Worker,
Step = #Step
WHERE CURRENT OF entries_cur;
INSERT INTO LogEntries_LastUpdate(Id, Time)
VALUES (#Worker, #Time);
FETCH NEXT FROM entries_cur INTO #Id, #LogCategory;
END;
CLOSE entries_cur;
DEALLOCATE entries_cur;
UPDATE le
SET le.LastUpdate = CAST(0.5 * (CAST(le.LastUpdate AS numeric(12,1)) + CAST(lelu.Time AS numeric(12,1)) + ABS(le.LastUpdate - lelu.Time)) AS int)
FROM LogEntries le
JOIN (
SELECT Id=ilelu.Id, Time=MAX(ilelu.Time)
FROM LogEntries_LastUpdate ilelu
GROUP BY ilelu.Id) lelu
ON lelu.Id = le.Id;
END;
This modification takes longer than the first version I used which was quite surprising. So I thought a non-cursor solution had to be found and Ian Peters and Nik Shenoy came up with something like this:
WITH LogWithWorkerLevel AS (
SELECT LogEntries.*
, SUM(CASE LogCategory WHEN 'WORKER_START' THEN 1
WHEN 'WORKER_END' THEN -1
ELSE 0 END)
OVER (ORDER BY Id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)
+ CASE LogCategory WHEN 'WORKER_END' THEN 1 ELSE 0 END AS [WorkerLevel]
, SUM(CASE LogCategory WHEN 'WORKER_STEP_START' THEN 1
WHEN 'WORKER_STEP_END' THEN -1
ELSE 0 END)
OVER (ORDER BY Id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)
+ CASE LogCategory WHEN 'WORKER_STEP_END' THEN 1 ELSE 0 END AS [StepLevel]
FROM LogEntries
)
UPDATE LogWithWorkerLevel
SET
Worker = (
SELECT TOP 1 Id
FROM LogWithWorkerLevel AS Lookup
WHERE
Lookup.Id <= LogWithWorkerLevel.Id
AND Lookup.WorkerLevel = LogWithWorkerLevel.WorkerLevel
AND Lookup.LogCategory = 'WORKER_START'
ORDER BY Id DESC)
, Step = (
SELECT TOP 1 Id
FROM LogWithWorkerLevel AS Lookup
WHERE
Lookup.Id <= LogWithWorkerLevel.Id
AND Lookup.StepLevel = LogWithWorkerLevel.StepLevel
AND Lookup.LogCategory = 'WORKER_STEP_START'
ORDER BY Id DESC);
Here we don't care about LastUpdate since it's just an idea by now. While the SELECT statement of the temporary table LogWithWorkerLevel runs blazing fast (9 seconds), the UPDATE statement takes pretty long, longer than the cursor I started with.
Sorry for this long explanation to come to this short question: Do you see a way to boost the performance of this processing using a stored procedure?
Thanks in advance!
UPDATE
Here is the table DDL:
CREATE TABLE LogEntries (
Id int,
Time int,
LogCategory varchar (128),
LogType varchar (128),
Worker int NULL,
Step int NULL,
LastUpdate int NULL,
CONSTRAINT PK_le_Id PRIMARY KEY(Id)
);
CREATE INDEX IDX_le_LogDomain ON LogEntries(LogDomain);
CREATE INDEX IDX_le_Worker ON LogEntries(Worker);
CREATE INDEX IDX_le_Step ON LogEntries(Step);
According the execution plan: it takes quite long (4+ hours) to get the actually executed execution plan. So, I take a look into the estimated execution plan (if it's called so in English, my SSMS speaks German):
In total 55% costs go on the cursor creation. Seems to be a "dynamic" cursor and the most expensive operations are a "Clustered Index Insert" (29%) and a "Clustered Index Search" (25%)
Another 23% go on the "Clustered Index Update" where I set the Worker and Step of a log entry.
In total 17% go on the UPDATE of the LastUpdate.
Here is the execution plan:
UPDATE 2
After four days and eight hours, the query (including the creation of the exact execution plan) was successfully executed:
(If you're interested in the execution plan itself, just let me know.)
So, there are two main reasons for this long duration:
The filter nodes in the two lower branches get a 12 digit count of rows.
Some sort nodes seem to cause a data overflow in tempdb.
I'm not sure if it's possible to fix that.
I also modified the cursor to conditionally execute the UPDATE statements. So I inserted an IF statement. And that boosted the query from 24 minutes to 3 minutes.
I got a raw data file with its content looking like this:
MSN_Check,Text,25,MSN check
0,Text,1,(Result)
HWIMPL,Text,10,HWIMPL version reading
007F,Text,6,(Measure)
1,Text,1,(Result)
VHW,Text,10,FMT hardware version
494131383346,Text,10,(Measure)
0,Text,1,(Result)
TOTAL_VER,Text,25,Total version reading
313031303130,Text,6,(Measure)
1,Text,1,(Result)
CAL_MCU,Text,25,Total version reading
05,Text,6,(Measure)
Error,Text,25,Error
9.8499985089315E-07,Numeric,Float 3.3,(Measure)
CAL_EEPROM,Text,25,Total version reading
05,Numeric,Float 3.3,(Measure)
1,Text,1,(Result)
And I needed to extract and store in variables the name, example MSN_Check ,the description, example MSN check its result for example 0 and its measure , for example 007F but in some places I have results only or measures only so just spliting them wouldn't have helped.So my idea was:
First of all I created a template table named dbo.template that looks like this:
Name TestDescription Measure Result ID
----------------------------------------------
MSN_Check MSN check 0 1 1
HWIMPL HWIMPL version reading 1 1 2
VHW FMT hardware version 1 1 3
TOTAL_VER Total version reading 1 1 4
CAL_MCU Total version reading 1 0 5
Error Error 1 0 6
CAL_EEPROM Total version reading 1 1 7
In this table we have the name,description,if_measure(meaning 1 if we have a measure or 0 if we dont) and the if_result.And I made a query looking like this:
DECLARE #crlf AS CHAR(2) = CHAR(13) + CHAR(10)
declare #testname varchar(max),#testDescription varchar(max), #if_measure char(1), #if_result char(1), #row int = '1', #id int
set #LogEntry = (SELECT REPLACE(#LogEntry,#crlf,','))
declare #name varchar(max),#description varchar(MAX), #measure varchar(20), #result char(1)
declare #Output table(OutTestName varchar(max),OUTTestDescription varchar(max), OutMeasure varchar(50), OutResult varchar(50))
declare #maximum int = (select MAX(ID) from dbo.template_FMT)
declare #LogEntry1 as nvarchar(max)
declare #LogEntry2 as nvarchar(max)
while #row <= #maximum
BEGIN
set #name = null
set #description = null
set #measure = null
set #result = null
set #testname = (select Name from dbo.template_FMT where ID = #row)
set #testDescription = (select TestDescription from dbo.template_FMT where ID = #row)
set #if_measure = (select Measure from dbo.template_FMT where ID = #row)
set #if_result = (select Result from dbo.template_FMT where ID = #row)
set #id = (select ID from dbo.Split(#LogEntry, ',') where Data = #testname)
SELECT #LogEntry1 = Name FROM dbo.template_FMT where id = #row
set #name = #LogEntry1
SELECT #LogEntry2 = TestDescription FROM dbo.template_FMT where id = #row
set #description = #LogEntry2
if #if_measure > 0 and #if_result > 0
begin
set #measure = (select Data from dbo.Split(#LogEntry, ',') where ID = #id+4)
set #result = (select Data from dbo.Split(#LogEntry, ',') where ID = #id+8)
insert into #Output (OutTestName, OUTTestDescription, OutMeasure, OutResult) Values(#name,#description, #measure, #result)
end
if #if_measure > 0 and #if_result = 0
begin
set #measure = (select Data from dbo.Split(#LogEntry, ',') where ID = #id+4)
set #result = null
insert into #Output (OutTestName, OUTTestDescription, OutMeasure, OutResult) Values(#name,#description, #measure, #result)
end
if #if_measure = 0 and #if_result > 0
begin
set #measure = null
set #result = (select Data from dbo.Split(#LogEntry, ',') where ID = #id+4)
insert into #Output (OutTestName, OUTTestDescription, OutMeasure, OutResult) Values(#name,#description, #measure, #result)
end
set #row = #row + 1
END
select * from #Output
And it worked! but the only problem I have is where I have the row with the name Error with the description Error,it would return the last remembered value so instead of having
CAL_MCU Total version reading 05 NULL
Error Error 9.8499985089315E-07 NULL
CAL_EEPROM Total version reading 05 1
I get:
CAL_MCU Total version reading 05 NULL
Error Error 05 NULL
CAL_EEPROM Total version reading 05 1
And I would like to store the Error cant find Result with ID into variables if any of you have any suggestions :)
P.S. I think it has something to do with the fact that the name and description have the same name (Error)
I believe that your problem can be solved without the need for while loops and string splitting functions. I recommend using the OPENROWSET function to read your raw data file as a standard table. You can then use standard T-SQL query to format the result into the desired output.
The first step is to ensure that ad-hoc queries is enable on your server this can be accomplished by executing the following command.
sp_configure 'show advanced options', 1;
RECONFIGURE;
GO
sp_configure 'Ad Hoc Distributed Queries', 1;
RECONFIGURE;
The next step is to define a format file for your text file. This will help SQL Server understand the text file structure when loading the raw data. Based on the supplied sample data your format file should look as follow:
10.0
4
1 SQLCHAR 0 100 "," 1 Col1 SQL_Latin1_General_CP1_CI_AS
2 SQLCHAR 0 100 "," 2 Col2 SQL_Latin1_General_CP1_CI_AS
3 SQLCHAR 0 100 "," 3 Col3 SQL_Latin1_General_CP1_CI_AS
4 SQLCHAR 0 100 "\r\n" 4 Col4 SQL_Latin1_General_CP1_CI_AS
I have uploaded the format file and example raw data file I have used to test the example at the following links:
http://www.filedropper.com/format
http://www.filedropper.com/rawdatafile
The final step is to run the OPENROWSET query to load the file data and transform the data to the desired output. If you are using SQL Server 2008 r2 then the following query should work:
-- 2008 R2 Version
WITH CTE_VariableRawData
AS
(
SELECT ROW_NUMBER() OVER(ORDER BY (SELECT 0)) AS ID
,[RawData].Col1 AS [VariableOrMeasure]
,(
CASE [RawData].Col4
WHEN '(Result)' THEN 0
WHEN '(Measure)' THEN 0
ELSE 1
END
) AS IsVariable
,(
CASE [RawData].Col4
WHEN '(Result)' THEN 1
ELSE 0
END
) AS IsResult
,(
CASE [RawData].Col4
WHEN '(Measure)' THEN 1
ELSE 0
END
) AS IsMeasure
,[RawData].Col4 AS [Description]
FROM OPENROWSET(BULK N'C:\temp\raw_data_file.txt', FORMATFILE = 'c:\temp\format.txt') AS [RawData]
)
,
CTE_RawDataByVariableID
AS
(
SELECT ID
,(
SELECT SUM([IsVariable])
FROM CTE_VariableRawData RunningTotal
WHERE RunningTotal.ID <= CTE_VariableRawData.ID
) AS VariableID
,[VariableOrMeasure]
,[IsVariable]
,[IsResult]
,[IsMeasure]
,[Description]
FROM CTE_VariableRawData
)
SELECT VariableID
,MAX(
CASE [IsVariable]
WHEN 1 THEN [VariableOrMeasure]
ELSE NULL
END
) AS [Variable]
,MAX(
CASE [IsVariable]
WHEN 1 THEN [Description]
ELSE NULL
END
) AS [Description]
,MAX(
CASE [IsMeasure]
WHEN 1 THEN [VariableOrMeasure]
ELSE NULL
END
) AS [Measure]
,MAX(
CASE [IsResult]
WHEN 1 THEN [VariableOrMeasure]
ELSE NULL
END
) AS [Result]
FROM CTE_RawDataByVariableID
GROUP BY VariableID
ORDER BY VariableID
If you are using SQL Server 2012 or later then the following query will be a bit more optimal:
WITH CTE_VariableRawData
AS
(
SELECT ROW_NUMBER() OVER(ORDER BY (SELECT 0)) AS ID
,[RawData].Col1 AS [VariableOrMeasure]
,(
CASE [RawData].Col4
WHEN '(Result)' THEN 0
WHEN '(Measure)' THEN 0
ELSE 1
END
) AS IsVariable
,(
CASE [RawData].Col4
WHEN '(Result)' THEN 1
ELSE 0
END
) AS IsResult
,(
CASE [RawData].Col4
WHEN '(Measure)' THEN 1
ELSE 0
END
) AS IsMeasure
,[RawData].Col4 AS [Description]
FROM OPENROWSET(BULK N'C:\temp\raw_data_file.txt', FORMATFILE = 'c:\temp\format.txt') AS [RawData]
)
,
CTE_RawDataByVariableID
AS
(
SELECT ID
,SUM([IsVariable]) OVER (ORDER BY ID) AS VariableID
,[VariableOrMeasure]
,[IsVariable]
,[IsResult]
,[IsMeasure]
,[Description]
FROM CTE_VariableRawData
)
SELECT VariableID
,MAX(
CASE [IsVariable]
WHEN 1 THEN [VariableOrMeasure]
ELSE NULL
END
) AS [Variable]
,MAX(
CASE [IsVariable]
WHEN 1 THEN [Description]
ELSE NULL
END
) AS [Description]
,MAX(
CASE [IsMeasure]
WHEN 1 THEN [VariableOrMeasure]
ELSE NULL
END
) AS [Measure]
,MAX(
CASE [IsResult]
WHEN 1 THEN [VariableOrMeasure]
ELSE NULL
END
) AS [Result]
FROM CTE_RawDataByVariableID
GROUP BY VariableID
ORDER BY VariableID;
Note that in both queries you will have to change the location of your raw data file and format file to the desired location within the OPENROWSET(BULK N'C:\temp\raw_data_file.txt', FORMATFILE = 'c:\temp\format.txt') call.
I have the below sql. I am wanting my results to be grouped if the boolean is checked 0, but not grouped if the boolean is checked 1. Is the code below a good way of doing this? I feel like it should work but it is throwing the error "InvoicePeriodID is invalid in the select list because it is not contained in an aggraagte function or group by" Any advice?
----insert invoices for rebates grouped by date, address, contract, rebate note, and order type
declare #invoices table(InvoicePeriodID int, InvoiceStartDate datetime, InvoiceEndDate datetime, JDEAddressNo float, ContractID int, RebateNoteID int, JDEOrderType char(2), RebateInvoiceID int)
insert #invoices (InvoicePeriodID , InvoiceStartDate , InvoiceEndDate , JDEAddressNo , ContractID , RebateNoteID, JDEOrderType)
select
i.InvoicePeriodID
, i.InvoiceStartDate
, i.InvoiceEndDate
, i.JDEAddressNo
, i.ContractID
, i.RebateNoteID
, i.JDEOrderType
from
#inv i
group by
case i.InvoiceSeparately
when 1 then null
when 0 then i.InvoicePeriodID
end
, case i.InvoiceSeparately
when 1 then null
when 0 then i.InvoiceStartDate
end
, case i.InvoiceSeparately
when 1 then null
when 0 then i.InvoiceEndDate
end
, case i.InvoiceSeparately
when 1 then null
when 0 then i.JDEAddressNo
end
, case i.InvoiceSeparately
when 1 then null
when 0 then i.ContractID
end
, case i.InvoiceSeparately
when 1 then null
when 0 then i.RebateNoteId
end
, case i.InvoiceSeparately
when 1 then null
when 0 then i.JDEOrderType
end
I think it is more readable and easier to do what you want this way
select
i.InvoicePeriodID
, i.InvoiceStartDate
, i.InvoiceEndDate
, i.JDEAddressNo
, i.ContractID
, i.RebateNoteID
, i.JDEOrderType
from
#inv i
WHERE i.InvoiceSeparately = 0
group BY i.InvoicePeriodID
,i.InvoiceStartDate
,i.InvoiceEndDate
, i.JDEAddressNo
, i.ContractID
,i.RebateNoteId
, i.JDEOrderType
UNION ALL
select
i.InvoicePeriodID
, i.InvoiceStartDate
, i.InvoiceEndDate
, i.JDEAddressNo
, i.ContractID
, i.RebateNoteID
, i.JDEOrderType
from
#inv i
WHERE i.InvoiceSeparately = 1
I have a query in which I am getting different information about encounters from two different queries, a third query will be added.
For each one of those queries I create a table for the records of that query to be inserted into, and at the end I create a sort of Master Answer Table, where I want to insert all records from the different tables into it.
It is possible that a person may not have a result being returned for all queries.
I think I am messing up my join in the final part of the query as I am getting TWO results, which I know is impossible. I am using SQL-Server 2008.
Here is what I have written so far:
-- VARIABLE DECLARATION AND INITIALIZATION
SET ANSI_NULLS OFF
GO
DECLARE #SD DATETIME
DECLARE #ED DATETIME
-- THESE ARE PATIENT ADMIT DATES
SET #SD = '2013-01-01'
SET #ED = '2013-01-02'
-- #T1 ###############################################################
-- TABLE DECLARATION WHERE ALL RESULTS WILL GET DEPOSITED OF THE FIRST
-- QUERY WILL GET DEPOSITED. THIS TALBE WILL GET USED IN CONJUNCTION
-- WITH TWO OTHER TABLES IN ORDER TO COMPUTE THE FINAL
DECLARE #T1 TABLE (
ENCOUNTER_ID VARCHAR(200)
, MRN VARCHAR(200)
, [PT AGE] VARCHAR(200)
, [PT NAME] VARCHAR(500)
, [DAYS STAY] VARCHAR(200)
, [LACE DAYS SCORE] VARCHAR(100)
, [ACUTE ADMIT SCORE] VARCHAR(100)
)
--#####################################################################
-- #T1 RECORD INSERTIONS ##############################################
INSERT INTO #T1
SELECT
A.PT_NO
, A.MED_REC_NO
, A.PT_AGE
, A.PT_NAME
, A.DAYS_STAY
, A.LACE_DAYS_SCORE
, A.ACUTE_ADMIT_LACE_SCORE
--#####################################################################
-- DAYS STAY, ACUTE ADMIT AND RELATED SCORING -------------------------
FROM
(SELECT PT_NO
, Med_Rec_No
, Pt_Age
, Pt_Name
, Days_Stay
, CASE
WHEN Days_Stay < 1 THEN 0
WHEN Days_Stay = 1 THEN 1
WHEN Days_Stay = 2 THEN 2
WHEN Days_Stay = 3 THEN 3
WHEN Days_Stay BETWEEN 4 AND 6 THEN 4
WHEN Days_Stay BETWEEN 7 AND 13 THEN 5
WHEN Days_Stay >= 14 THEN 6
END AS LACE_DAYS_SCORE
, CASE
WHEN PLM_PT_ACCT_TYPE = 'I' THEN 3
END AS ACUTE_ADMIT_LACE_SCORE
FROM SMSDSS.BMH_PLM_PTACCT_V
WHERE DSCH_DATE BETWEEN #SD AND #ED
)A
--/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/--
-- #T2 ###############################################################
-- TABLE DECLARATION WHERE THE ER PORTION OF LACE SCORE FOR PATIENT
-- WILL GO.
DECLARE #T2 TABLE (
[MRN T2] VARCHAR(100)
, [PT NO] VARCHAR(20)
, [ER VISITS] VARCHAR (200)
, [ER VISITS LACE SCORE] VARCHAR(100)
)
--#####################################################################
-- #T2 RECORD INSERTIONS ##############################################
INSERT INTO #T2
SELECT B.MED_REC_NO
, B.Pt_No
, B.COUNT_MRN
, B.ER_LACE_SCORE
--#####################################################################
FROM
(
SELECT DISTINCT Med_Rec_No
, Pt_No
, COUNT(Pt_No) AS COUNT_MRN
, CASE
WHEN COUNT(MED_REC_NO) = 0 THEN 0
WHEN COUNT(MED_REC_NO) = 1 THEN 1
WHEN COUNT(MED_REC_NO) = 2 THEN 2
WHEN COUNT(MED_REC_NO) = 3 THEN 3
WHEN COUNT(MED_REC_NO) >= 4 THEN 4
END AS ER_LACE_SCORE
FROM SMSDSS.BMH_PLM_PTACCT_V
WHERE ADM_DATE >= '2013-01-01'
AND (
PLM_PT_ACCT_TYPE = 'I'
AND ADM_SOURCE NOT IN
('RA',
'RP'
)
)
OR PLM_PT_ACCT_TYPE = 'E'
GROUP BY MED_REC_NO, Pt_No
)B
--/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/--
-- FINAL TABLE DECLARATION WHERE ALL INTERMEDIATE RESULTS GET DEPOSITED
DECLARE #LT TABLE (
ENCOUNTER_ID VARCHAR(200)
, MRN VARCHAR(200)
, AGE VARCHAR(30)
, NAME VARCHAR(500)
, LACE_DAYS_SCORE VARCHAR(100)
, LACE_ACUTE_ADM VARCHAR(100)
-- , LACE_COMORBID_SCORE VARCHAR(100)
, LACE_ER_VISITS VARCHAR(100)
)
-- ####################################################################
INSERT INTO #LT
SELECT
Q1.ENCOUNTER_ID
, Q1.MRN
, Q1.[PT AGE]
, Q1.[PT NAME]
, Q1.[LACE DAYS SCORE]
, Q1.[ACUTE ADMIT SCORE]
, Q1.[ER VISITS LACE SCORE]
FROM
(
SELECT
DISTINCT T1.ENCOUNTER_ID
, T1.MRN
, T1.[PT AGE]
, T1.[PT NAME]
, T1.[LACE DAYS SCORE]
, T1.[ACUTE ADMIT SCORE]
, T2.[ER VISITS LACE SCORE]
FROM #T1 T1
JOIN #T2 T2
ON T1.ENCOUNTER_ID = T2.[PT NO]
)Q1
--#####################################################################
SELECT *
FROM #LT
I have updated the code at #T2 to join on the encounter number since any Med_rec_no can have a theoretically infinite amount of encounter numbers. The Med_rec_no therefore can show up multiple times in the original dbo table and this is not a problem. The #T2 table is trying to count how many times a Med_rec_no has been to a certain place in the last 6 months, so it is dynamic, meaning if I come in today, then how many times have I been in the last six months including today.
I have made the code adjustments based on commetns that lead me in a better direction from the two who have commented on this post.
I'm pretty sure when you do
SELECT PT_NO
, Med_Rec_No
, Pt_Age
, Pt_Name
, Days_Stay
, CASE
WHEN Days_Stay < 1 THEN 0
WHEN Days_Stay = 1 THEN 1
WHEN Days_Stay = 2 THEN 2
WHEN Days_Stay = 3 THEN 3
WHEN Days_Stay BETWEEN 4 AND 6 THEN 4
WHEN Days_Stay BETWEEN 7 AND 13 THEN 5
WHEN Days_Stay >= 14 THEN 6
END AS LACE_DAYS_SCORE
, CASE
WHEN PLM_PT_ACCT_TYPE = 'I' THEN 3
END AS ACUTE_ADMIT_LACE_SCORE
FROM SMSDSS.BMH_PLM_PTACCT_V
WHERE DSCH_DATE BETWEEN #SD AND #ED
There can be several rows with the same Med_Rec_No
I'm trying to do an insert into an established table which has a primary key fields and another field (call it field1) that is unique (this other unique field has a unique constraint preventing my inserts). Field1 is not an identity field, so it does NOT autonumber. Unfortunately I can't change the table. Existing inserts are made using code to increment and all involve looping/cursors. Something like SELECT MAX(field1) + 1
So, is there anyway to do this insert without looping/cursor? This field means nothing to me, but there are already 500,000+ records using their silly numbering scheme, so I must respect that.
This is simplified (ReceiptNumber is the field I want to insert unique), but:
SET XACT_ABORT ON
Begin Transaction TransMain
Declare #nvErrMsg nvarchar(4000)
--Insert inventory receipts
Insert Into Avanti_InventoryReceipts (
ReceiptNumber , ItemNumber , ReceiptDate , OrderNumber , JobNumber , Supplier ,
LineNumber , MultiLineNumber , [Status] , QtyOrdered , QtyReceived , QtyToReceive ,
QtyBackOrdered , Cost , Wholesale , LastCost , QtyToInvoice , QtyUsed ,
ReferenceNumber , [Description] , SupplierType , Processed , DateExpected , DateReceived ,
AccountNumber , Reference2 , EmployeeCode , ExtraCode , Location , RollNumber ,
QtyIssues , Notes , NumPackages , BundleSize , ConsignmentUnitPrice , RecFromProduction ,
QtyCommitted )
SELECT ( SELECT MAX(ReceiptNumber) + 1 FROM Avanti_inventoryReceipts ) , CR.ItemNumber , Convert(char(8), GETDATE(), 112) , PONum , 'FL-INV' , PH.POVendor ,
0 , 0 , 'O' , CR.QtyOrdered , QtyReceivedToday , QtyReceivedToday ,
Case #closePO
When 'N' Then Case When ( QtyOrdered - QtyReceivedToday ) < 0 Then 0 Else ( QtyOrdered - QtyReceivedToday) End
When 'Y' Then 0
Else 0 End
, PD.TransCost * QtyReceivedToday , IH.PriceWholeSale , IH.CostLast , QtyReceivedToday , 0 ,
'' , PODetailDescription , '' , '' , '' , Convert(char(8), GETDATE(), 112) ,
'' , '' , #employeeCode , '' , 'F L E X O' , '' ,
0 , 'Flexo Materials' , 0 , 0 , 0 , '' , 0
FROM FI_CurrentReceiptData CR
LEFT JOIN Avanti_PODetails PD ON CR.PONum = PD.PONumber
LEFT JOIN Avanti_POHeader PH ON CR.PONum = PH.PONumber
LEFT JOIN Avanti_InventoryHeader IH ON CR.ItemNumber = IH.ItemNumber
IF ##ERROR <> 0
Begin
Select #nvErrMsg = 'Error entering into [InventoryReceipts] -' + [description]
From master..sysmessages
Where [error] = ##ERROR
RAISERROR ( #nvErrMsg , 16, 1 )
Goto Err_
End
Commit Transaction TransMain
Goto Exit_
Err_:
Rollback Transaction TransMain
Exit_:
SET XACT_ABORT OFF
You could do this:
insert into mytable (field1, field2, ...)
values (( SELECT MAX(field1) + 1 from mytable), 'value2', ...);
Why not looping? It should be quite efficient.
Since you already have a UNIQUE constraint on the field, you can:
Simply try to insert MAX(field1) + 1. Since there is index on UNIQUE field, MAX is fast.
If its passes, great you are done.
If it fails (which will typically be manifested as an exception in your client code), just try again until you succeed.
Most of the time, the INSERT will succeed right away. In rare instances where a concurrent user tries to insert the same value, you'll handle that gracefully by trying the "next" value.
I added an autonumber starting from 0 in client code and passed that in. Now I'm adding that value to the max receiptnumber to get a unique one. Also, I realized I already had an identity column in FI_CurrentReceiptData, but I didn't want to use that one because it won't start at 0 for each receipt set, and reseeding the identity each time seems like a waste of processor time.