Scalar Variable not Populating - sql

Here's the story. I'm trying to pull metadata from a master table and from a series of tables whose names are based on values in the master table. There is no foreign key.
If there was a key, it is that the primary key from from the master table is appended to the end of the child table. The master table is hsi.keytypetable. The child tables are hsi.keyitemxxx where the xxx is a value (keytypenum) pulled from the master table.
All I'm trying to pull from the child table right now is a count of values. In the current form, the query, #sql1, is failing to populate #keytypenum, although when I look at query itself, and run it in a separate window, it works like a champ. The problem continues in the second query, #sql2, where I am getting the error,
Must declare the scalar variable "#keytypenum"
As far as I can tell, I've declared the thing. I'm guessing I have a similar problem with syntax in each query.
SET CONCAT_NULL_YIELDS_NULL OFF
declare #keytypedata table
(
Keyword varchar(50),
DateType varchar(50),
"Length" int,
"Count" int
)
declare #keywordcount int
declare #x int = 1
declare #keytypenum int
declare #sql1 varchar(max)
declare #sql2 varchar(max)
/* Determine how many records are in the master table so that I can cycle thru each one, getting the count of the child tables. */
Select #keywordcount = (Select count(*) from hsi.keytypetable)
/* #x is the counter. I'll cycle through each row in the master using a WHILE loop */
WHILE #x < #keywordcount+1
BEGIN
/* One row at a time, I'll pull the KEYTYPENUM and store it in #keytypenum. (I don't really need the order by, but I like having things in order!)
** I take the rows in order b using my counter, #x, as the offset value and fetch only 1 row at a time. When I run this query in a separate screen,
** it works well, obviously with providing a fixed offset value. */
set #sql1 =
'Set #keytypenum =
(Select
KEYTYPENUM
from hsi.keytypetable
order by KEYTYPENUM
OFFSET ' + cast(#x as varchar(4)) + ' ROWS
FETCH NEXT 1 ROWS ONLY)'
EXEC(#sql1)
/* For debugging purposes, I wanted to see that #keytypenum got assigned. This is working. */
print 'KeyTypeNum: '+cast(#keytypenum as varchar(4))
/* I don't know why I had to be my table variable, #keytypedata, in single quotes at the beginning, but it wouldn't work if
** I didn't. The problem comes later on with restricting the query by the aforementioned #keytypenum. Remember this variable is an INT. All values
** for this field are indeed integers, and there are presently 955 rows in the table. The maximum value is 1012, so we're safe with varchar(4).
*/
SET #sql2 =
'Insert into ' + '#keytypedata' + '
Select
keytype,
CASE
WHEN k.datatype = 1 THEN ''Numeric 20''
WHEN k.datatype = 2 THEN ''Dual Table Alpha''
WHEN k.datatype = 3 THEN ''Currency''
WHEN k.datatype = 4 THEN ''Date''
WHEN k.datatype = 5 THEN ''Float''
WHEN k.datatype = 6 THEN ''Numeric 9''
WHEN k.datatype = 9 THEN ''DateTime''
WHEN k.datatype = 10 THEN ''Single Table Alpha''
WHEN k.datatype = 11 THEN ''Specific Currency''
WHEN k.datatype = 12 THEN ''Mixed Case Dual Table Alpha''
WHEN k.datatype = 13 THEN ''Mixed Case Single Table Alpha''
END,
keytypelen,
(Select count(*) from hsi.keyitem' + cast(#keytypenum as varchar(4)) + ')
FROM
hsi.keytypetable k
where
k.keytypenum = ' + cast(#keytypenum as varchar(4))+''
/* Printing out where I am with cycling thru the master table, just for troubleshooting*/
print #x
/* Increment the counter*/
set #x = #x + 1
END
/* This query is simply to display the final results. */
select *
from #keytypedata
order by 1
/* Print statements below are for troubleshooting. They should show what the 2 queries currently look like. */
Print #sql1
Print #sql2

Yeah, you can't do that. Your variables are going out of scope.
Each variable has it's scope restricted to it's own session, and exec() basically creates a new session.
This will throw and error that #x is undefined:
declare #x int = 1
exec ('select #x')
As will this:
exec ('declare #x int = 2')
exec ('select #x')
And this:
exec ('declare #x int = 2')
select #x
You'd have to do it like this:
exec ('declare #x int = 2; select #x')
Or otherwise pass the results back somehow. The sp_executesql suggestion in #TT.'s answer is a good idea.

When you declare variables, the visibility is restricted to the scope in which they are declared. When you EXEC a statement, a new session and thereby a new scope is created.
What you need to do is output the scalar variable using an OUTPUT parameter in a call to sp_executesql:
SET #sql='Set #keytypenum = ...';
EXEC sp_executesql #sql,N'#keytypenum INT OUT', #keytypenum OUTPUT;
Note that the #sql variable needs to be an NVARCHAR.
You can find more examples here.

Related

Store the value of a query into a SQL Server variable

The objective of the code is to run a query dynamically and return 0 if there are no rows with data present in the columns and to return 1 if there are rows with data in the columns. This is my code for the stored procedure:
ALTER proc [dbo].[usp_ColumnFieldValidator]
(
#TblName nvarchar(30),
#ColumnName nvarchar(30),
#RetVal bit output
)
as
begin
declare #CountOfRowsQuery as nvarchar(300)
set #CountOfRowsQuery = 'select count('+quotename(#ColumnName)+') from '+quotename(#TblName)+' having count(' +quotename(#ColumnName)+') = nullif(count('+quotename(#ColumnName)+'),0)'
execute sp_executesql #CountOfRowsQuery
select #RetVal = dbo.fn_ColumnValidator(#CountOfRowsQuery)
end
As you can see, a user-defined function is being called to set the value of #RetVal. This is my code for the user-defined function.
ALTER function [dbo].[fn_ColumnValidator]
(
#NullChecker as nvarchar(max)
)
returns bit
as
begin
declare #returnVar as bit
if #NullChecker is null
set #returnVar = 0
else
set #returnVar = 1
return #returnVar
end
The output of #RetVal is always 1 and I have attributed this error to #CountOfRowsQuery storing the entire string rather than the value of the query ie: #CountOfRowsQuery = null if the count of rows is zero else, #CountOfRowsQuery = the number of rows present in the column. To make things clearer I am attaching screenshots of the output when I run the program.
Output of a table that contains rows with data
Output of a table that contains no rows with no data
As you can see in list item.2, the sp returns null but the function_returned_value is being set to 1 instead of 0.
The objective of the code is to run a query dynamically and return 0 if there are no rows with data present in the columns and to return 1 if there are rows with data in the columns.
Man, if this is not an over-complication I don't know what is.
Here's a much simpler (and more efficient) query that does the work:
SELECT CAST(IIF(EXISTS(
SELECT 1
FROM TableName
WHERE ColumnName IS NOT NULL
), 1, 0) As Bit)
Now, to change that to a procedure using dynamic SQL in a way that will not expose you to SQL Injection threats you can do this:
ALTER PROCEDURE [dbo].[usp_ColumnFieldValidator]
(
#TblName sysname,
#ColumnName sysname,
#RetVal bit output
)
AS
BEGIN
IF NOT EXISTS(
SELECT 1
FROM Information_Schema.Columns
WHERE Table_Name = #TblName
AND Column_Name = #ColumnName
)
RETURN;
DECLARE #Sql nvarchar(1000) =
N'SELECT #RetVal = CAST(IIF(EXISTS(
SELECT 1
FROM '+ QUOTENAME(#TblName) + N'
WHERE '+ QUOTENAME(#ColumnName) + N' IS NOT NULL
), 1, 0) As Bit)'
EXEC sp_executesql #Sql, N'#RetVal bit output', #RetVal OUTPUT;
END
Key notes:
I've changed the #TblName and #ColumnName variables to data type sysname instead of your original nvarchar(30) - since that is the data type SQL Server use internally to store identifiers.
Since identifiers can't be parameterized, I've white-listed them.
I'm using sp_executeSql to get back the value of the dynamic query directly into my output parameter.
For more tips and tricks on dynamic SQL, you can read my blog post entitled The do’s and don’ts of dynamic SQL for SQL Server

Generating filed name with concat

My table has column names m1,m2,m3...,m12.
I'm using iterator to select them and insert them one by one in another table.
In this iterator I'm trying to generate filed names with:
'['+concat('m',cast(#P_MONTH as nvarchar))+']'
where #P_MONTH is incrementing in each loop.
so for #P_MONTH = 1 this suppose to give [m1] which works fine.
But when I run query I get:
Conversion failed when converting the nvarchar value '[m1]' to data
type int.
And if I put simply [m1] in that select it works ok.
How to concat filed name so it can be actually interpreted as filed name from certain table?
EDIT
Here is full query:
DECLARE #SQLString nvarchar(500),
#P_YEAR int,
#P_MONTH int = 1
set #P_YEAR = 2018
WHILE #P_MONTH < 13
BEGIN
SET #SQLString =
'INSERT INTO [dbo].[MASTER_TABLE]
(sector,serial,
date, number, source)'+
'SELECT ' + '[SECTOR],[DEPARTMENT]' +
QUOTENAME(cast(CONVERT(datetime,CONVERT(VARCHAR(4),#P_YEAR)+RIGHT('0'+CONVERT(VARCHAR(2),#P_MONTH),2)+'01',5) as nvarchar))+
QUOTENAME ('M',cast(#P_MONTH as nvarchar)) +
'EMPLOYED' +
'FROM [dbo].[STATS]'+
'where YEAR= #P_YEAR'
EXECUTE sp_executesql #SQLString
SET #P_MONTH = #P_MONTH + 1
END
It's still not working. It executes successfully but it does nothing.
Good day,
Let's create a simple table for the sake of the explanation
DROP TABLE IF EXISTS T
GO
CREATE TABLE T(a1 INT)
GO
INSERT T(a1) VALUES (1),(2)
GO
SELECT a1 FROM T
GO
When we are using a query like bellow, the server parse the text as a value and not as a column name
DECLARE #String NVARCHAR(10)
SELECT #String = '1'
--
SELECT '['+concat('a',cast(#String as nvarchar))+']'
FROM T
GO
This mean that the result will be 2 rows with no name for the column and the value will be "[a1]"
Moreover, the above query uses the brackets as part of the string.
One simple solution is to use the function QUOTENAME in order to add brackets around a name.
Another issue in this approach is the optional risk of SQL Injection. QUOTENAME might not be perfect solution but can help in this as well.
If we need to use entities name dynamically like in this case the column name then for most cases using dynamic query is the best solution. This mean to use the Stored Procedure sp_executesql as bellow
DECLARE #String INT
SELECT #String = 1
DECLARE #SQLString nvarchar(500);
SET #SQLString =
'SELECT ' + QUOTENAME(concat('a',cast(#String as nvarchar))) + ' FROM T'
EXECUTE sp_executesql #SQLString
GO

SQL Server update statement performance

I have problem in optimizing an SQL query to do some data cleansing.
In fact, I have a table which is a sort of referential of a multiple special characters and word. Let's call it ABNORMAL(ID,PATTERN)
I have also another table INDIVIDUALS containing a column (NAME) which I want to clean by removing from it all characters that exist in the table ABNORMAL.
Currently, I have tried to use update statements, but I'm not sure if there is a better way to do this.
Approach one
Use a while loop to build a replace containing all characters from ABNORMALS by a blank '' and do one update using the built-in REPLACE
DECLARE #REPLACE_EXPRESSION nvarchar(max) ='REPLACE(NAME,'''','''')'
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- NEW REPLACE EXPRESSION TO IMBRICATE INTO THE REPLACE EXPRESSION VARIABLE
DECLARE #CURR_REP NVARCHAR(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
SET #REPLACE_EXPRESSION = REPLACE(#REPLACE_EXPRESSION ,'NAME','REPLACE(NAME,'+''''+#CURRENT_CHARAC+''''+','''')')
set #i=#i+1
END
SET #UPDATE_QUERY = 'UPDATE INDIVIDUAL SET NAME ='+ #REPLACE_EXPRESSION
EXEC sp_executesql #UPDATE_QUERY
Approach two
Use a while loop to select every character in abnormal and do an update using replace containing the characters to remove:
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
UPDATE INDIVIDUAL
SET NAME = REPLACE(NAME,#CURRENT_CHARAC,'')
SET #i=#i+1
END
I already tested both approaches for 2 millions records, and I found that the first approach is faster than the second. I would know if you have already done something similar and new (better) ideas to try.
If you are using SQL Server 2017 you could use TRANSLATE and avoid dynamic SQL:
SELECT i.*
, REPLACE(TRANSLATE(i.NAME, f, REPLICATE('!', s.l)), '!', '') AS cleansed
FROM INDIVIDUALS i
OUTER APPLY (SELECT STRING_AGG(PATTERN, '') AS f
,LEN(STRING_AGG(PATTERN,'')) AS l
FROM ABNORMAL) AS s
DBFiddle Demo
Anyway 1st approach is better becasue you do one UPDATE, with second approach you remove characters one character at time (so you will have multiple UPDATE).
I would also track transaction log growth with both approaches.
If there's not too many characters that to be cleaned, then this trick might work.
Basically, you build 1 big update statement with a replace for each value in the table with the characters to be removed.
Example code:
Test data (using temp tables)
create table #ABNORMAL_CHARACTERS (id int identity(1,1), chr varchar(30));
insert into #ABNORMAL_CHARACTERS (chr) values ('!'),('&'),('#');
create table #INDIVIDUAL (id int identity(1,1), name varchar(30));
insert into #INDIVIDUAL (name) values ('test 1 &'),('test !&#2'),('test 3');
Code:
declare #FieldName varchar(30) = 'name';
declare #Replaces varchar(max) = #FieldName;
declare #UpdateSQL varchar(max);
select #Replaces = concat('replace('+#Replaces+', ', ''''+chr+''','''')') from #ABNORMAL_CHARACTERS order by id;
set #UpdateSQL = 'update #INDIVIDUAL
set name = '+#Replaces + '
where exists (select 1 from #ABNORMAL_CHARACTERS where charindex(chr,name)>0)';
exec (#UpdateSQL);
select * from #INDIVIDUAL;
A test here on rextester
And if you would have a UDF that can do a regex replace.
For example here
Then the #Replaces variable could be simplified with only 1 RegexReplace function and a pattern.

Dynamic SQL Query returns column name instead of value

I am trying to create some queries passing the column name dynamically, but for some reason is returning the Column name and not the value.
I am not very familiar with this technique, for now #cmd is empty because before I write the dynamic query I wanted to make sure I will pass the correct parameters. In other words, I want to print the value that is in the column A1.
Can anyone please tell or guide me to get the value instead? I will appreciate any help.
HubFinal
id Cart PO A1 A1E
----------------------------------------------------------
01 Cart1 24432 upc1,1/25/2016,1 Available
-----------------------------------------------------------
02 Cart2 24888 upc10,1/25/2030,1 No Available
Query
WHILE (#i <= 1)
BEGIN
-- get Column Name Example A1
SET #Compartment = (SELECT compartment FROM #Compartment_table WHERE idx = #i);
-- get data from HUBFINAL to insert into HUBTEMP
SET #PO = (Select PO FROM HubFinal Where CartPlate =#CartPlate);
-- pass dynamically the comlumn name, in this case A1
SET #CompValue = (Select #Compartment From HubFinal Where CartPlate =#CartPlate);
Print #Compartment
Print #PO
Print #CompValue
--insert to final table
Declare #cmd nvarchar(4000) =
-- do something with values gotten above
EXEC(#cmd)
-- increment counter for next compartment
SET #i = #i + 1
END
Output
-- this is what is printed
A1
24432
A1
as #Sean Lange told you ... It's not recommended to loop in sql server as it will hit the performance (you should find another way to solve you problem), but if you want to get the value of a dynamic column name you can do it like that
as I don't know the data type you are working with I assuming that it's NVARCHAR
DECLARE #value NVARCHAR(MAX);
SET #CompValue = CONVERT(NVARCHAR(MAX), 'Select #val='+ #Compartment + ' From HubFinal Where CartPlate = #CartPlate')
EXECUTE sp_executesql #CompValue, N'#CartPlate NVARCHAR(MAX),#val NVARCHAR OUTPUT', #CartPlate = #CartPlate, #val= #value OUTPUT
PRINT(#value)

"Subsequent parameters" error in SSRS?

I'm getting a strange error in SSRS, in a report (which gets put into a sproc) with many drop-down parameters:
Query execution failed for dataset 'DataSet1'.
Must pass parameter number 3 and subsequent parameters as '#name = value'. After the form '#name = value' has been used, all subsequent parameters must be passed in the form '#name = value'
I'm lost on what's going on here - what is meant by #name = value .
I searched online, someone mentioned that you should alter the stored-procedure?
Here is how the top half of my stored-proc looks:
USE [FederatedSample]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[prc_RPT_Select_BI_Completes_Data_View_2]
#FromDate DATETIME,
#ToDate DATETIME,
#AccountIDs VARCHAR(max) = null,
#ClientIDs VARCHAR(max) = null,
#SupplierIDs VARCHAR(max) = null,
#CompleteType INT = NULL,
/*
* 0 - Routed
* 1 - Targeted
* 2 - Offerwall
*/
#SourceType BIT = NULL,
/*
* Works if #AccountID is not null
* (should only be used if #AccountID has a single value)
*
* 0 - Owned by #AccountID
* 1 - External (not owned by #AccountID)
*/
#SurveyStatus INT = NULL,
/*
* NULL - All Surveys
* 0 - Completes Approved Surveys
* 1 - Invoiced Surveys
*/
#IsSupplierUser BIT = 0
/*
* used to decide whether to display FEDSurveyName or SupplierSurveyName
*/
AS
BEGIN
SET NOCOUNT ON
DECLARE #SQL NVARCHAR(MAX) = N'',
#Params NVARCHAR(MAX)
IF #AccountIDs is not null
BEGIN
SET #SQL += N'DECLARE #AccountIDs VARCHAR(MAX) = #pAccountIDs; '
END
IF #ClientIDs is not null
BEGIN
SET #SQL += N'DECLARE #ClientIDs VARCHAR(MAX) = #pClientIDs; '
END
IF #SupplierIDs is not null
BEGIN
SET #SQL += N'DECLARE #SupplierIDs VARCHAR(MAX) = #pSupplierIDs; '
END
SET #SQL += N'
SELECT bi.SupplierID as ''Supplier ID''
,bi.SupplierName as ''Supplier Name''
,bi.PID as ''PID''
,bi.RespondentID as ''Respondent ID''
,lk_slt.Name as ''Entry Link Type''
,ts.SurveyNumber as ''Initial Survey ID'''
And later in the stored proc. it does stuff like this to split strings:
IF #AccountIDs is not null
BEGIN
SET #SQL += CHAR(13) + CHAR(9)
SET #SQL += N' and bi.AccountID in (SELECT CAST(val as INT) FROM dbo.Split(#AccountIDs, '','
When invoking a stored procedure, you either can pass the parameters by position (not a good idea) or by Name (a better approach IMHO).
EXEC dbo.MyStoredProcedure '12/31/2012', 1; -- Not a great way to pass parameters
EXEC dbo.MyStoredProcedure #AsOfDate = '12/31/2012', #AccountID = 1; -- A better way
From the error message you are receiving, I suspect that SSRS is using the second approach and is running into an issue with the third parameter being provided to the stored procedure.
Without more information to go off of it is difficult to provide you with an exact explanation for the error (the stored procedure would perhaps help), an educated guess is that the way the parameters are being provided for Account IDs, Client IDs and Supplier IDs isn't quite correct. Specifically, I think the problem might be that you are providing multiple identifiers delimited by a comma.
You might try passing a single Account ID, Client ID and Supplier ID to see if you still receive the error. I would also try to look at the stored procedure (or talk to the DBA \ Developer who wrote it) to ascertain the intended usage of the stored procedure.
I got a similar message when passing a comma instead of a full stop for a decimal value in one of the parameters of a stored procedure.
Here is a simplified example of what happened.
The following command string was erroneously generated with the value of #param2 with a comma instead of a full stop, causing a misunderstanding of the number of parameters.
EXEC myStoredProc #param1 = 1, #param2 = 0,5 ,#param3 = 'something'