We have an C# application which posts to a database which is replicated to another database (using merge-replication) and has one custom resolver which is a stored procedure.
This was working fine under SQL Server 2000 , but when testing under SQL Server 2005 the custom resolver is attempting to change any empty varchar columns to be nulls (and failing cos this particular column does not allow nulls).
Note that these varchar fields are not the ones which cause the conflict as they are current empty on both databases and are not being changed and the stored procedure does not change them (all it is doing is attempting to set the value of another money column).
Has anyone come across this problem, or has example of a stored procedure which will leave empty strings as they are?
The actual stored procedure is fairly simply and and re-calculates the customer balance in the event of a conflict.
ALTER procedure [dbo].[ReCalculateCustomerBalance]
#tableowner sysname,
#tablename sysname,
#rowguid varchar(36),
#subscriber sysname,
#subscriber_db sysname,
#log_conflict INT OUTPUT,
#conflict_message nvarchar(512) OUTPUT
AS
set nocount on
DECLARE
#CustomerID bigint,
#SysBalance money,
#CurBalance money,
#SQL_TEXT nvarchar(2000)
Select #CustomerID = customer.id from customer where rowguid= #rowguid
Select #SysBalance = Sum(SystemTotal), #CurBalance = Sum(CurrencyTotal) From CustomerTransaction Where CustomerTransaction.CustomerID = #CustomerID
Update Customer Set SystemBalance = IsNull(#SysBalance, 0), CurrencyBalance = IsNull(#CurBalance, 0) Where id = #CustomerID
Select * From Customer Where rowguid= #rowguid
Select #log_conflict =0
Select #conflict_message ='successful'
Return(0)
You have a few options here, each are a bit of a workaround from what my research seems to show is an issue with SQL Server.
1- Alter this statement: Select * From Customer Where rowguid= #rowguid to explicitly mention each of the columns, and use an "isNull" for the offending fields
2- Alter the column in the table to add a default constraint for ''. What this will do, is if you attempt to insert a 'null', it will replace it with the empty string
3- Add a 'before insert' trigger which will alter the data before the insert, to not contain a 'null' anymore
PS: Are you positive that the replication system has that column marked as "required"? I think if it is not required, it will insert 'null' if no data exists.
Related
I have a a number of sp's that create a temporary table #TempData with various fields. Within these sp's I call some processing sp that operates on #TempData. Temp data processing depends on sp input parameters. SP code is:
CREATE PROCEDURE [dbo].[tempdata_proc]
#ID int,
#NeedAvg tinyint = 0
AS
BEGIN
SET NOCOUNT ON;
if #NeedAvg = 1
Update #TempData set AvgValue = 1
Update #TempData set Value = -1;
END
Then, this sp is called in outer sp with the following code:
USE [BN]
--GO
--DBCC FREEPROCCACHE;
GO
Create table #TempData
(
tele_time datetime
, Value float
--, AvgValue float
)
Create clustered index IXTemp on #TempData(tele_time);
insert into #TempData(tele_time, Value ) values( GETDATE(), 50 ); --sample data
declare
#ID int,
#UpdAvg int;
select
#ID = 1000,
#UpdAvg = 1
;
Exec dbo.tempdata_proc #ID, #UpdAvg ;
select * from #TempData;
drop table #TempData
This code throws an error: Msg 207, Level 16, State 1, Procedure tempdata_proc, Line 8: Invalid column name "AvgValue".
But if only I uncomment declaration AvgValue float - everything works OK.
The question: is there any workaround letting the stored proc code remain the same and providing a tip to the optimizer - skip this because AvgValue column will not be used by the sp due to params passed.
Dynamic SQL is not a welcomed solution BTW. Using alternative to #TempData tablename is undesireable solution according to existing tsql code (huge modifications necessary for that).
Tried SET FMTONLY, tempdb.tempdb.sys.columns, try-catch wrapping without any success.
The way that stored procedures are processed is split into two parts - one part, checking for syntactical correctness, is performed at the time that the stored procedure is created or altered. The remaining part of compilation is deferred until the point in time at which the store procedure is executed. This is referred to as Deferred Name Resolution and allows a stored procedure to include references to tables (not just limited to temp tables) that do not exist at the point in time that the procedure is created.
Unfortunately, when it comes to the point in time that the procedure is executed, it needs to be able to compile all of the individual statements, and it's at this time that it will discover that the table exists but that the column doesn't - and so at this time, it will generate an error and refuse to run the procedure.
The T-SQL language is unfortunately a very simplistic compiler, and doesn't take runtime control flow into account when attempting to perform the compilation. It doesn't analyse the control flow or attempt to defer the compilation in conditional paths - it just fails the compilation because the column doesn't (at this time) exist.
Unfortunately, there aren't any mechanisms built in to SQL Server to control this behaviour - this is the behaviour you get, and anything that addresses it is going to be perceived as a workaround - as evidenced already by the (valid) suggestions in the comments - the two main ways to deal with it are to use dynamic SQL or to ensure that the temp table always contains all columns required.
One way to workaround your concerns about maintenance if you go down the "all uses of the temp table should have all columns" is to move the column definitions into a separate stored procedure, that can then augment the temporary table with all of the required columns - something like:
create procedure S_TT_Init
as
alter table #TT add Column1 int not null
alter table #TT add Column2 varchar(9) null
go
create procedure S_TT_Consumer
as
insert into #TT(Column1,Column2) values (9,'abc')
go
create procedure S_TT_User
as
create table #TT (tmp int null)
exec S_TT_Init
insert into #TT(Column1) values (8)
exec S_TT_Consumer
select Column1 from #TT
go
exec S_TT_User
Which produces the output 8 and 9. You'd put your temp table definition in S_TT_Init, S_TT_Consumer is the inner query that multiple stored procedures call, and S_TT_User is an example of one such stored procedure.
Create the table with the column initially. If you're populating the TEMP table with SPROC output just make it an IDENTITY INT (1,1) so the columns line up with your output.
Then drop the column and re-add it as the appropriate data type later on in the SPROC.
The only (or maybe best) way i can thing off beyond dynamic SQL is using checks for database structure.
if exists (Select 1 From tempdb.sys.columns Where object_id=OBJECT_ID('tempdb.dbo.#TTT') and name = 'AvgValue')
begin
--do something AvgValue related
end
maybe create a simple function that takes table name and column or only column if its always #TempTable and retursn 1/0 if the column exists, would be useful in the long run i think
if dbo.TempTableHasField('AvgValue')=1
begin
-- do something AvgValue related
end
EDIT1: Dang, you are right, sorry about that, i was sure i had ... this.... :( let me thing a bit more
I have a stored procedure as follows:
CREATE PROCEDURE [ODataTaskResult_Create]
#ODataTaskId BIGINT,
#ODataTaskResultTypeId INTEGER,
#Details CHARACTER VARYING(MAX)
AS
BEGIN TRANSACTION
INSERT INTO [ODataTaskResult] WITH (ROWLOCK, XLOCK) ([ODataTaskId], [ODataTaskResultTypeId], [Details], [CreatedOn])
VALUES (#ODataTaskId, #ODataTaskResultTypeId, #Details, SYSDATETIMEOFFSET())
DECLARE #ODataTaskResultTypeName CHARACTER VARYING(255)
SET #ODataTaskResultTypeName = (
SELECT TOP 1 [ODataTaskType].[Name] FROM [ODataTaskType]
WHERE [ODataTaskType].[Id] = #ODataTaskResultTypeId)
IF (#ODataTaskResultTypeName = 'Finish')
BEGIN
UPDATE [ODataTask]
SET [ODataTask].[FinishedOn] = SYSDATETIMEOFFSET()
WHERE [ODataTask].[Id] = #ODataTaskId
END ELSE IF (#ODataTaskResultTypeName = 'Delete')
BEGIN
UPDATE [ODataTask]
SET [ODataTask].[DeletedOn] = SYSDATETIMEOFFSET()
WHERE [ODataTask].[Id] = #ODataTaskId
END ELSE
RAISERROR('Invalid result type', 16, 1)
COMMIT TRANSACTION
GO
This procedure is supposed to look at the incoming #ODataTaskResultTypeId parameter, pull the result type down from another table, and do something based on the Name column in that record.
Basically when a result is entered against a task, it defines how it completed. If a task is finished, I need to modify the FinishedOn column on the parent task record and not alter the DeletedOn column. We have a constraint that indicates FinishedOn and DeletedOn may not both be NOT NULL.
Feeling at this point that since I have hard coded the different case logic into the stored procedure, it makes maintainability difficult and prevents this from working properly unless the ODataTaskResult table has the correct initial entries.
Should I make the ODataTaskResult_Create procedure only create the result and then have another procedure called ODataTask_Finish as well as another procedure called ODataTask_Delete?
Is there a different approach to this problem that is generally easier to maintain?
We never hard delete entries, only soft delete.
If you want a flexible solution, you can add a column to your ODataTaskType table to hold the stored procedure to run afterwards. You can then use some dynamic sql to dispatch. If the column is called PostComplete_Proc, say:
create proc dbo.[ODataTaskResult_Create]
#ODataTaskId bigint,
#ODataTaskResultTypeId int,
#Details varchar(max)
as
declare
#proc sysname,
#params nvarchar(max) = '#ODataTaskId bigint'
begin transaction
insert into dbo.[ODataTaskResult] with (rowlock, xlock) (
[ODataTaskId], [ODataTaskResultTypeId], [Details], [CreatedOn]
) values
#ODataTaskId,
#ODataTaskResultTypeId,
#Details,
sysdatetimeoffset()
);
select top 1 -- Would there really be more than 1? Why hide potential errors?
#proc = PostComplete_Proc
from
dbo.[ODataTaskType]
where
Id = #ODataTaskResultTypeId;
if #proc is null
raiserror('Invalid result type', 16, 1);
else
exec #proc, #params, #ODataTaskId;
commit transaction;
then create the relevant stored procedures. If you have many result types and few procedures, you can even add another level, where the procedures are stored on a separate table and referenced via foreign keys.
I find it hard to convince myself that rowlock, xlock is doing anything here.
trying to get this stored procedure to work.
ALTER PROCEDURE [team1].[add_testimonial]
-- Add the parameters for the stored procedure here
#currentTestimonialDate char(10),#currentTestimonialContent varchar(512),#currentTestimonialOriginator varchar(20)
AS
BEGIN
DECLARE
#keyValue int
SET NOCOUNT ON;
--Get the Highest Key Value
SELECT #keyValue=max(TestimonialKey)
FROM Testimonial
--Update the Key by 1
SET #keyValue=#keyValue+1
--Store into table
INSERT INTO Testimonial VALUES (#keyValue, #currentTestimonialDate, #currentTestimonialContent, #currentTestimonialOriginator)
END
yet it just returns
Running [team1].[add_testimonial] ( #currentTestimonialDate = 11/11/10, #currentTestimonialContent = this is a test, #currentTestimonialOriginator = theman ).
No rows affected.
(0 row(s) returned)
#RETURN_VALUE = 0
Finished running [team1].[add_testimonial].
and nothing is added to the database, what might be the problem?
There may have problems in two place:
a. There is no data in the table so, max(TestimonialKey) returns null, below is the appropriate way to handle it.
--Get the Highest Key Value
SELECT #keyValue= ISNULL(MAX(TestimonialKey), 0)
FROM Testimonial
--Update the Key by 1
SET #keyValue=#keyValue+1
b. Check your data type of the column currentTestimonialDate whether it is char or DateTime type, if this field is datetime type in the table then convert #currentTestimonialDate to DateTime before inserting to the table.
Also, check number of columns that are not null allowed and you're passing data to them.
If you're not passing data for all columns then try by specifying columns name as below:
--Store into table
INSERT INTO Testimonial(keyValue, currentTestimonialDate,
currentTestimonialContent, currentTestimonialOriginator)
VALUES (#keyValue, #currentTestimonialDate,
#currentTestimonialContent, #currentTestimonialOriginator)
EDIT:
After getting the comment from marc_s:
Make keyValue as INT IDENTITY, If multiple user call it concurrently that wont be problem, DBMS will handle it, so the ultimate query in procedure might be as below:
ALTER PROCEDURE [team1].[add_testimonial]
-- Add the parameters for the stored procedure here
#currentTestimonialDate char(10),
#currentTestimonialContent varchar(512),#currentTestimonialOriginator varchar(20)
AS
BEGIN
SET NOCOUNT ON;
--Store into table
INSERT INTO Testimonial VALUES (#currentTestimonialDate,
#currentTestimonialContent, #currentTestimonialOriginator)
END
Two issues that I can spot:
SELECT #keyValue=max(TestimonialKey)
should be
SELECT #keyValue=ISNULL(max(TestimonialKey), 0)
To account for the case when there are no records in the database
Second, I believe that with NOCOUNT ON, you will not return the count of inserted rows to the caller. So, before your INSERT statement, add
SET NOCOUNT OFF
I have an SQL Server locking question regarding an application we have in house. The application takes submissions of data and persists them into an SQL Server table. Each submission is also assigned a special catalog number (unrelated to the identity field in the table) which is a sequential alpha numeric number. These numbers are pulled from another table and are not generated at run time. So the steps are
Insert Data into Submission Table
Grab next Unassigned Catalog
Number from Catalog Table
Assign the Catalog Number to the
Submission in the Submission table
All these steps happen sequentially in the same stored procedure.
Its, rate but sometimes we manage to get two submission at the same second and they both get assigned the same Catalog Number which causes a localized version of the Apocalypse in our company for a small while.
What can we do to limit the over assignment of the catalog numbers?
When getting your next catalog number, use row locking to protect the time between you finding it and marking it as in use, e.g.:
set transaction isolation level REPEATABLE READ
begin transaction
select top 1 #catalog_number = catalog_number
from catalog_numbers with (updlock,rowlock)
where assigned = 0
update catalog_numbers set assigned = 1 where catalog_number = :catalog_number
commit transaction
You could use an identity field to produce the catalog numbers, that way you can safely create and get the number:
insert into Catalog () values ()
set #CatalogNumber = scope_identity()
The scope_identity function will return the id of the last record created in the same session, so separate sessions can create records at the same time and still end up with the correct id.
If you can't use an identity field to create the catalog numbers, you have to use a transaction to make sure that you can determine the next number and create it without another session accessing the table.
I like araqnid's response. You could also use an insert trigger on the submission table to accomplish this. The trigger would be in the scope of the insert, and you would effectively embed the logic to assign the catalog_number in the trigger. Just wanted to put your options up here.
Here's the easy solution. No race condition. No blocking from a restrictive transaction isolation level. Probably won't work in SQL dialects other than T-SQL, though.
I assume their is some outside force at work to keep your catalog number table populated with unassigned catalog numbers.
This technique should work for you: just do the same sort of "interlocked update" that retrieves a value, something like:
update top 1 CatalogNumber
set in_use = 1 ,
#newCatalogNumber = catalog_number
from CatalogNumber
where in_use = 0
Anyway, the following stored procedure just just ticks up a number on each execution and hands back the previous one. If you want fancier value, add a computed column that applies the transform of choice to the incrementing value to get the desired value.
drop table dbo.PrimaryKeyGenerator
go
create table dbo.PrimaryKeyGenerator
(
id varchar(100) not null ,
current_value int not null default(1) ,
constraint PrimaryKeyGenerator_PK primary key clustered ( id ) ,
)
go
drop procedure dbo.GetNewPrimaryKey
go
create procedure dbo.GetNewPrimaryKey
#name varchar(100)
as
set nocount on
set ansi_nulls on
set concat_null_yields_null on
set xact_abort on
declare
#uniqueValue int
--
-- put the supplied key in canonical form
--
set #name = ltrim(rtrim(lower(#name)))
--
-- if the name isn't already defined in the table, define it.
--
insert dbo.PrimaryKeyGenerator ( id )
select id = #name
where not exists ( select *
from dbo.PrimaryKeyGenerator pkg
where pkg.id = #name
)
--
-- now, an interlocked update to get the current value and increment the table
--
update PrimaryKeyGenerator
set #uniqueValue = current_value ,
current_value = current_value + 1
where id = #name
--
-- return the new unique value to the caller
--
return #uniqueValue
go
To use it:
declare #pk int
exec #pk = dbo.GetNewPrimaryKey 'foobar'
select #pk
Trivial to mod it to return a result set or return the value via an OUTPUT parameter.
Brief history:
I'm writing a stored procedure to support a legacy reporting system (using SQL Server Reporting Services 2000) on a legacy web application.
In keeping with the original implementation style, each report has a dedicated stored procedure in the database that performs all the querying necessary to return a "final" dataset that can be rendered simply by the report server.
Due to the business requirements of this report, the returned dataset has an unknown number of columns (it depends on the user who executes the report, but may have 4-30 columns).
Throughout the stored procedure, I keep a column UserID to track the user's ID to perform additional querying. At the end, however, I do something like this:
UPDATE #result
SET Name = ppl.LastName + ', ' + ppl.FirstName
FROM #result r
LEFT JOIN Users u ON u.id = r.userID
LEFT JOIN People ppl ON ppl.id = u.PersonID
ALTER TABLE #result
DROP COLUMN [UserID]
SELECT * FROM #result r ORDER BY Name
Effectively I set the Name varchar column (that was previously left NULL while I was performing some pivot logic) to the desired name format in plain text.
When finished, I want to drop the UserID column as the report user shouldn't see this.
Finally, the data set returned has one column for the username, and an arbitrary number of INT columns with performance totals. For this reason, I can't simply exclude the UserID column since SQL doesn't support "SELECT * EXCEPT [UserID]" or the like.
With this known (any style pointers are appreciated but not central to this problem), here's the problem:
When I execute this stored procedure, I get an execution error:
Invalid column name 'userID'.
However, if I comment out my DROP COLUMN statement and retain the UserID, the stored procedure performs correctly.
What's going on? It certainly looks like the statements are executing out of order and it's dropping the column before I can use it to set the name strings!
[Edit 1]
I defined UserID previously (the whole stored procedure is about 200 lies of mostly irrelevant logic, so I'll paste snippets:
CREATE TABLE #result ([Name] NVARCHAR(256), [UserID] INT);
Case sensitivity isn't the problem but did point me to the right line - there was one place in which I had userID instead of UserID. Now that I fixed the case, the error message complains about UserID.
My "broken" stored procedure also works properly in SQL Server 2008 - this is either a 2000 bug or I'm severely misunderstanding how SQL Server used to work.
Thanks everyone for chiming in!
For anyone searching this in the future, I've added an extremely crude workaround to be 2000-compatible until we update our production version:
DECLARE #workaroundTableName NVARCHAR(256), #workaroundQuery NVARCHAR(2000)
SET #workaroundQuery = 'SELECT [Name]';
DECLARE cur_workaround CURSOR FOR
SELECT COLUMN_NAME FROM [tempdb].INFORMATION_SCHEMA.Columns WHERE TABLE_NAME LIKE '#result%' AND COLUMN_NAME <> 'UserID'
OPEN cur_workaround;
FETCH NEXT FROM cur_workaround INTO #workaroundTableName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #workaroundQuery = #workaroundQuery + ',[' + #workaroundTableName + ']'
FETCH NEXT FROM cur_workaround INTO #workaroundTableName
END
CLOSE cur_workaround;
DEALLOCATE cur_workaround;
SET #workaroundQuery = #workaroundQuery + ' FROM #result ORDER BY Name ASC'
EXEC(#workaroundQuery);
Thanks everyone!
A much easier solution would be to not drop the column, but don't return it in the final select.
There are all sorts of reasons why you shouldn't be returning select * from your procedure anyway.
EDIT: I see now that you have to do it this way because of an unknown number of columns.
Based on the error message, is the database case sensitive, and so there's a difference between userID and UserID?
This works for me:
CREATE TABLE #temp_t
(
myInt int,
myUser varchar(100)
)
INSERT INTO #temp_t(myInt, myUser) VALUES(1, 'Jon1')
INSERT INTO #temp_t(myInt, myUser) VALUES(2, 'Jon2')
INSERT INTO #temp_t(myInt, myUser) VALUES(3, 'Jon3')
INSERT INTO #temp_t(myInt, myUser) VALUES(4, 'Jon4')
ALTER TABLE #temp_t
DROP Column myUser
SELECT * FROM #temp_t
DROP TABLE #temp_t
It says invalid column for you. Did you check the spelling and ensure there even exists that column in your temp table.
You might try wrapping everything preceding the DROP COLUMN in a BEGIN...COMMIT transaction.
At compile time, SQL Server is probably expanding the * into the full list of columns. Thus, at run time, SQL Server executes "SELECT UserID, Name, LastName, FirstName, ..." instead of "SELECT *". Dynamically assembling the final SELECT into a string and then EXECing it at the end of the stored procedure may be the way to go.