I am running into a very strange problem in a SQL Server stored procedure.
I have two databases. One contains the database for my billing system. The other is a reporting system with summarized data. Within this database there is a table with summarized job information. When this data is created, one of the fields, BilledToDate, is null. I wrote a stored procedure that creates a cursor that goes through this table and gets all of the job numbers. I then go through each job number and run a query against the billing database to get the total amount of billing that has been charged against job. Once I have this total, I update the BilledToDate column with this value.
The problem is that after running the stored procedure, some of the results are correct and some aren't. There doesn't appear to be any logical explanation as to why one is right and the next one is isn't. I put some print statements in the stored procedure and all of the values were correct. As an example, for one record the correct sum was 99,218.25 but the update put a value of 14,700.70 into the BilledToDate field. I added a varchar column to the table and populated that field. They are all correct. This leads me to believe that it is a casting problem but I checked and double checked my datatypes and they all look correct. I am pulling my hair out on this one (what little that is left).
My stored procedure is below. The InvoiceAmt field is a decimal(16,2) in the invchead table and I have kept it consistent throughout the process so I don't undertand why this is happening.
ALTER PROCEDURE [dbo].[sp_CalculateBilledToDate]
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE #JobID varchar(10)
DECLARE #RecordID int
DECLARE #BilledToDate decimal(16,2)
DECLARE c1 CURSOR FOR
SELECT JobID, RecordID
FROM StructuralOpenBilling
OPEN c1
FETCH NEXT FROM c1
INTO #JobID, #RecordID
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #BilledToDate = CONVERT(money, CASE WHEN SUM(invoiceamt) > 0 THEN SUM(InvoiceAmt) ELSE 0 END)
FROM mfgsys803.dbo.invchead
WHERE shortchar01 = RTRIM(#JobID)
PRINT 'Record ID: ' + CONVERT(varchar(10), #RecordID) + ' JobID: ' + RTRIM(CONVERT(varchar(10), #JobID)) + ' Billed: ' + CONVERT(varchar(10), #BilledToDate)
UPDATE StructuralOpenBilling
SET BilledToDate = #BilledToDate, BilledCheck = CONVERT(varchar(50), #BilledToDate)
WHERE RecordID = #RecordID
PRINT 'Record ID: ' + CONVERT(varchar(10), #RecordID) + ' JobID: ' + RTRIM(CONVERT(varchar(10), #JobID)) + ' Billed: ' + CONVERT(varchar(10), #BilledToDate)
FETCH NEXT FROM c1
INTO #JobID, #RecordID
END
CLOSE c1
DEALLOCATE c1
END
Any ideas would be appreciated.
Thanks.
John
I notice a few things you might look at. BTW -- you're really over-thinking this -- a few ideas about that here as well.
SELECT #BilledToDate = CONVERT(money, CASE WHEN SUM(ISNULL(invoiceamt,0)) > 0 THEN SUM(ISNULL(InvoiceAmt,0)) ELSE 0 END)
Is the same as
SELECT #BilledToDate = CONVERT(money, SUM(ISNULL(invoiceamt,0)))
*NOTE the use of ISNULL() in both -- this would be important, as you can't do math on nulls.
Not necessary to use a cursor. Just join your two tables together in a single update statement and work on it as a batch.
UPDATE StructuralOpenBilling
SET S.BilledToDate = I.BilledToDate
FROM
StructuralOpenBilling S
INNER JOIN
(SELECT shortchar01, CONVERT(money, SUM(ISNULL(invoiceamt,0))) as BilledToDate
FROM mfgsys803.dbo.invchead) I
ON
S.JobID = I.shortchar01
Does this do what you are trying to do?
WITH inv AS
(
SELECT shortchar01,
CONVERT(MONEY, CASE WHEN SUM(invoiceamt) > 0 THEN
SUM(InvoiceAmt)
ELSE 0 END) AS BilledToDate
FROM mfgsys803.dbo.invchead
GROUP BY shortchar01
)
UPDATE StructuralOpenBilling
SET BilledToDate = inv.BilledToDate,
BilledCheck = CONVERT(VARCHAR(50), inv.BilledToDate)
FROM StructuralOpenBilling sob
JOIN inv ON inv.shortchar01 = RTRIM(sob.JobID)
Related
In the following script, I want to set the WHERE statement only if the CASE is set.
DECLARE #cmdINSERT varchar(8000);
declare #dowhere binary;
declare #date varchar(10);
set #dowhere = 1;
set #date = '14.03.2020';
set #cmdINSERT =
'SELECT *
FROM [TABLE]
WHERE [Changed At] >=
CASE WHEN ' + #dowhere + '
when 1 THEN ''' + #date + '''
ELSE [Changed At]
end;'
EXEC(#cmdINSERT)
But I am getting the error:
The data types varchar and binary are incompatible in the add operator
It seems like you're overly complicating the matter. You don't need a dynamic statement here, nor do you need 2 parameters. Use a single parameter, and pass the value NULL if you don't want to consider it. This is what is known as a "catch up" or "kitchen sink" query:
DECLARE #Date date = '20200314'; --Notice it is NOT a varchar
SELECT {Your Columns} --Replace this with a list of your columns
FROM dbo.YourTable --Replace this with your actual schema and table
WHERE [Changed At] >= #Date --Ideally, don't use names that must be delimit Identified
OR #Date IS NULL
OPTION (RECOMPILE);
The OPTION (RECOMPILE) is there to stop caching of the wrong plan, as if #Date has a value of NULL a full table scan will be required, however, if not then (depending on your indexes) one may not be required, which could be far faster. As the query is incredibly simple, the expense of regenerating query plan will be negligible, and likely <=1ms in cost.
It seems that you have complicated the solution. Why you do not try this?
SELECT *
FROM [Table]
WHERE #dowhere = 0
OR [Changed At] >= #date;
However it would be better if you change the data type of #dowhere to BIT instead of BINARY
Your approach is silly. Get rid of the case in the where altogether if no filtering is desired:
set #cmdINSERT = '
SELECT *
FROM [TABLE]
[where]';
SET #cmdInsert = REPLACE(#cmdInsert,
'[where]',
(CASE WHEN #dowwhere = 1
THEN 'WHERE [Changed At] = #date'
ELSE ''
END)
);
exec sp_executesql #cmdInsert, N'#date date', #date=#date;
Note that this also properly passes in the date as a parameter rather than munging the query string. If the date parameter is not used, that is fine: SQL Server just ignores parameters that are not used.
I have a stored procedure which has a parameter #FilterToUse.
I have a query inside the stored procedure as shown below:
INSERT INTO #StatusCalculation
SELECT COUNT(*)
FROM Data
WHERE DataType = 'Bid'
--This part can change based on SP param
AND CreatedDate BETWEEN CONVERT(DATETIME, ('1/1' + '/' + #RequestYear))
AND CONVERT(DATETIME, ('12/31' + '/' + #RequestYear))
--end dynamic part
AND Services LIKE '%Streamline Payroll%'
AND Services LIKE '%GlobalView Payroll%'
So if I have a different filter I ll have to write
INSERT INTO #StatusCalculation
SELECT COUNT(*)
FROM Data
WHERE DataType = 'Bid'
--This part can change based on param
AND BidId BETWEEN 1 AND 10
--end dynamic part
AND Services LIKE '%Streamline Payroll%'
AND Services LIKE '%GlobalView Payroll%'
Now my issue is that based on the #FilterToUse parameter I might have to use a different query expression. The same query based on different options is used about 10 times. Is there a way that I can dynamically add the filter conditions to the query based on parameter without duplicating the whole query.
I was thinking if somehow I can do the below
declare #data nvarchar(max)
Set #data = 'and CreatedDate between convert(datetime,("1/1"+"/"+#RequestYear))
and convert(datetime,("12/31"+"/"+#RequestYear))'
and then just add that variable to the query
Insert into #BidStatusCalculation
select 'SL Payroll',Count(*) from DashboardData
where DataType = 'Bid'
#data
You need to use dinamic SQL for this purpose.
First define and assign the #DinamicPart but you may include this in the call to the SP and consider as a parameter.
DECLARE #DinamicPart AS NVARCHAR(MAX), -- This is the part that will be passed as parameter to the SP
#SQL AS NVARCHAR(MAX);
SET #DinamicPart = ' AND CreatedDate BETWEEN CONVERT(DATETIME, (''1/1/' + #RequestYear+'))
AND CONVERT(DATETIME, (''12/31/' + #RequestYear))';
SET #SQL = 'INSERT INTO #StatusCalculation
SELECT COUNT(*)
FROM Data
WHERE DataType = 'Bid'
--This part can change based on SP param ' +
#DinamicPart + '
--end dynamic part
AND Services LIKE ''%Streamline Payroll%''
AND Services LIKE ''%GlobalView Payroll%'';
-- This part if for debugging purposes as Dinamic SQL can be very tricky
PRINT #SQL;
-- And last part, you run the Dinamic SQL
EXEC SP_ExecuteSQL #SQL;
Proposed solution:
insert into #StatusCalculation
select COUNT(*)
from Data
where DataType = 'Bid'
--This part can change based on param
and 1 = (
case #FilterToUse
when 1 then
case when year(CreateDate) = #RequestYear then 1 else 0 end
when 2 then
case when BidId between 1 and 10 then 1 else 0 end
end
)
--end dynamic part
and Services like '%Streamline Payroll%'
and Services like '%GlobalView Payroll%'
I am trying to create some queries passing the column name dynamically, but for some reason is returning the Column name and not the value.
I am not very familiar with this technique, for now #cmd is empty because before I write the dynamic query I wanted to make sure I will pass the correct parameters. In other words, I want to print the value that is in the column A1.
Can anyone please tell or guide me to get the value instead? I will appreciate any help.
HubFinal
id Cart PO A1 A1E
----------------------------------------------------------
01 Cart1 24432 upc1,1/25/2016,1 Available
-----------------------------------------------------------
02 Cart2 24888 upc10,1/25/2030,1 No Available
Query
WHILE (#i <= 1)
BEGIN
-- get Column Name Example A1
SET #Compartment = (SELECT compartment FROM #Compartment_table WHERE idx = #i);
-- get data from HUBFINAL to insert into HUBTEMP
SET #PO = (Select PO FROM HubFinal Where CartPlate =#CartPlate);
-- pass dynamically the comlumn name, in this case A1
SET #CompValue = (Select #Compartment From HubFinal Where CartPlate =#CartPlate);
Print #Compartment
Print #PO
Print #CompValue
--insert to final table
Declare #cmd nvarchar(4000) =
-- do something with values gotten above
EXEC(#cmd)
-- increment counter for next compartment
SET #i = #i + 1
END
Output
-- this is what is printed
A1
24432
A1
as #Sean Lange told you ... It's not recommended to loop in sql server as it will hit the performance (you should find another way to solve you problem), but if you want to get the value of a dynamic column name you can do it like that
as I don't know the data type you are working with I assuming that it's NVARCHAR
DECLARE #value NVARCHAR(MAX);
SET #CompValue = CONVERT(NVARCHAR(MAX), 'Select #val='+ #Compartment + ' From HubFinal Where CartPlate = #CartPlate')
EXECUTE sp_executesql #CompValue, N'#CartPlate NVARCHAR(MAX),#val NVARCHAR OUTPUT', #CartPlate = #CartPlate, #val= #value OUTPUT
PRINT(#value)
Using SQL Server I am trying to inject a string into a SQL statement based on an if statement, note that am trying to accomplish this inside a stored procedure.
I am currently getting an error for this code:
Declare #topString varchar(240)
IF #topRecords > 0
SET #topString = 'top 500'
ELSE
SET #topString = ''
SELECT #topString * FROM( //incorrect syntax near FROM
SELECT top 500 c.Id as [Customer Id],....
UNION
SELECT top 500 c.Id as [Customer Id],....
)as table1
Order by 1 desc
Edit
if somethingTrue
#whereCondition = '1 = 1 '
else
#whereCondition = branch = #branch
select * from table
where #whereCondition AND etc...
Correct
for injection inside an if statement go with Jodrell
but if you need a dynamic top then go with what was suggested by Kaf.
thanks both for the help!
If you want to decide number of top records depending on #topRecords, you can do it using an INT or BIGINT depending on the number of records needed.
DECLARE #top INT --This is declared as an int here
IF #topRecords > 0
SET #top = 500
ELSE
SET #top = 5000000 --Make it more than records if you need all
--or make it 0 if no records needed.
--#top has to be >=0
--How to use it
SELECT TOP (#top) * FROM YourTable
EDIT: Your question had only a top injection initially. However, If you need more injections (as per your recent question edit) then I would suggest to use a dynamic query as per #Jordell's answer.
You can't inject statement parts as variables like that, however you can change most values for parameters.
Having a stored procedure perform operations that may require different query plans, based on a parameter is a bad idea, the results of this SP could vary wildly based on the value of the #topRecords parameter. You would need to use the RECOMPILE option to warn the query engine, mitigating much of the benefit of SPs. Have you considered just having two stored procedures?
If you want to do it dynamically, you could build the whole statement dynamically, making one big string, then execute that.
You should investigate using sp_executesql to execute the string/VarChar. Then similar queries will benefit from query plan reuse.
As ever Sommarskog is a good reference.
Something like this
DECLARE #topString varchar(240);
DECLARE #statement varchar(max);
IF #topRecords > 0
SET #topString = 'TOP 500';
ELSE
SET #topString = '';
SET #statement = 'SELECT ' + #topString + ' * FROM
(
SELECT TOP 500 c.Id as [Customer Id], ....
UNION
SELECT TOP 500 c.Id as [Customer Id], ....
) table1
ORDER BY 1 DESC'
/* Then execute #statement */
EXEC sp_executesql #statement
I want to copy large amount of datas from one table to another table.I used cursors in Stored Procedure to do the same.But it is working only for tables with less records.If the tables contain more records it is executing for long time and hanged.Please give some suggestion as how can i copy the datas in faster way,My SP is as below:
--exec uds_shop
--select * from CMA_UDS.dbo.Dim_Shop
--select * from UDS.dbo.Dim_Shop
--delete from CMA_UDS.dbo.Dim_Shop
alter procedure uds_shop
as
begin
declare #dwkeyshop int
declare #shopdb int
declare #shopid int
declare #shopname nvarchar(60)
declare #shoptrade int
declare #dwkeytradecat int
declare #recordowner nvarchar(20)
declare #LogMessage varchar(600)
Exec CreateLog 'Starting Process', 1
DECLARE cur_shop CURSOR FOR
select
DW_Key_Shop,Shop_ID,Shop_Name,Trade_Sub_Category_Code,DW_Key_Source_DB,DW_Key_Trade_Category,Record_Owner
from
UDS.dbo.Dim_Shop
OPEN cur_shop
FETCH NEXT FROM cur_shop INTO #dwkeyshop,#shopid,#shopname,#shoptrade, #shopdb ,#dwkeytradecat,#recordowner
WHILE ##FETCH_STATUS = 0
BEGIN
Set #LogMessage = ''
Set #LogMessage = 'Records insertion/updation start date and time : ''' + Convert(varchar(19), GetDate()) + ''''
if (isnull(#dwkeyshop, '') <> '')
begin
if not exists (select crmshop.DW_Key_Shop from CMA_UDS.dbo.Dim_Shop as crmshop where (convert(varchar,crmshop.DW_Key_Shop)+CONVERT(varchar,crmshop.DW_Key_Source_DB)) = convert(varchar,(CONVERT(varchar, #dwkeyshop) + CONVERT(varchar, #shopdb))) )
begin
Set #LogMessage = Ltrim(Rtrim(#LogMessage)) + ' ' + 'Record for shop table is inserting...'
insert into
CMA_UDS.dbo.Dim_Shop
(DW_Key_Shop,DW_Key_Source_DB,DW_Key_Trade_Category,Record_Owner,Shop_ID,Shop_Name,Trade_Sub_Category_Code)
values
(#dwkeyshop,#shopdb,#dwkeytradecat,#recordowner,#shopid,#shopname,#shoptrade)
Set #LogMessage = Ltrim(Rtrim(#LogMessage)) + ' ' + 'Record successfully inserted in shop table for shop Id : ' + Convert(varchar, #shopid)
end
else
begin
Set #LogMessage = Ltrim(Rtrim(#LogMessage)) + ' ' + 'Record for Shop table is updating...'
update
CMA_UDS.dbo.Dim_Shop
set DW_Key_Trade_Category=#dwkeytradecat,
Record_Owner=#recordowner,
Shop_ID=#shopid,Shop_Name=#shopname,Trade_Sub_Category_Code=#shoptrade
where
DW_Key_Shop=#dwkeyshop and DW_Key_Source_DB=#shopdb
Set #LogMessage = Ltrim(Rtrim(#LogMessage)) + ' ' + 'Record successfully updated for shop Id : ' + Convert(varchar, #shopid)
end
end
Exec CreateLog #LogMessage, 0
FETCH NEXT FROM cur_shop INTO #dwkeyshop,#shopid,#shopname,#shoptrade, #shopdb ,#dwkeytradecat,#recordowner
end
CLOSE cur_shop
DEALLOCATE cur_shop
End
Assuming targetTable and destinationTable have the same schema...
INSERT INTO targetTable t
SELECT * FROM destinationTable d
WHERE someCriteria
Avoid the use of cursors unless there is no other way (rare).
You can use the WHERE clause to filter out any duplicate records.
If you have an identity column, use an explicit column list that doesn't contain the identity column.
You can also try disabling constraints and removing indexes provided you replace them (and make sure the constraints are checked) afterwards.
If you are on SQL Server 2008 (onwards) you can use the MERGE statement.
From my personal experience, when you copy the huge data from one table to another (with similar constraints), drop the constraints on the table where the data is getting copied. Once the copy is done, reinstate all the constraints again.
I could reduce the copy time from 7 hours to 30 mins in my case (100 million records with 6 constraints)
INSERT INTO targetTable
SELECT * FROM destinationTable
WHERE someCriteria (based on Criteria you can copy/move the records)
Cursors are notoriously slow and ram can begin to become a problem for very large datasets.
It does look like you are doing a good bit of logging in each iteration, so you may be stuck with the cursor, but I would instead look for a way to break the job up into multiple invocations so that you can keep your footprint small.
If you have an autonumber column, I would add a '#startIdx bigint' to the procedure, and redefine your cursor statement to take the 'TOP 1000' 'WHERE [autonumberFeild] <= #startIdx Order by [autonumberFeild]'. Then create a new stored procedure with something like:
DECLARE #startIdx bigint = 0
WHILE select COUNT(*) FROM <sourceTable> > #startIdx
BEGIN
EXEC <your stored procedure> #startIdx
END
SET #startIdx = #startIdx + 1000
Also, make sure your database files are set to auto-grow, and that it does so in large increments, so you are not spending all your time growing your datafiles.