Simplest way to insert variables into SQL Stored Procedure using Python? - sql

I have a stored SQL procedure I would like to generalize. This involves passing the Table name, a start time, and an end time through as variables, so one doesn't have to edit the procedure every time. I'm trying to use basic python variables to prompt the user. (Start = int(input("Enter Start Time: "))
I just haven't found a simple way to do this. I've used 'execute sp_execute_external_scripts' and have been seeing if Pyodbc is the right tool for this, but nothing so far has worked, and I didn't fully understand the MS documentation/tutorial for creating a wrapper. So what is the easiest way to prompt the user for input that can be injected into a procedure as a variable? I feel like I'm missing something very simple here.
Upon reading comments here, my original method is overbroad and would require dynamicsql, which I'm not automatically opposed to, but it seems safer to create procedures for individual tables. So I should only need to push Start, End and ChunkSize as integer variables into the procedure.
SOLVED: I got the desired results by writing a small python program using pyodbc. Thank you for the help and gentle nudging away from my original, naive idea.
import pyodbc
connection = pyodbc.connect('Driver={SQL Server};'
'Server=CROWN;'
'Database=ControlInformation;'
'Trusted_Connection=yes;')
cursor = connection.cursor()
Start = int(input("Enter Start Time: "))
End = int(input("Enter End Time: "))
ChunkSize = int(input("Enter # of files to delete at once: "))
cursor.execute('exec PurgeCurrencyExchangeRates #start = %d, #end = %d,
#ChunkSize = %d' %(Start, End, ChunkSize))
cursor.commit(),
connection.close()
SQL stored procedure
CREATE PROCEDURE UserInput
#r int = 1,
#Start bigint = 0,
#End bigint = 0,
#TableName varchar(40) = 'CurrencyExchangeRates',
#ChunkSize int = 10000,
#ColumnName varchar(40) = 'Timestamp'
AS
BEGIN
SET NOCOUNT ON;
while #r > 0
delete top(#ChunkSize)
ControlInformation.dbo.#TableName
where #ColumnName > #Start and #ColumnName < #End
set #r = ##ROWCOUNT

The stored procedure should be something like:
use CurrencyExchangeRates
go
CREATE PROCEDURE PurgeCurrencyExchangeRates
#Start bigint = 0,
#End bigint = 0,
#ChunkSize int = 10000
AS
BEGIN
SET NOCOUNT ON;
declare #r int = 1;
while #r > 0
begin
delete top(#ChunkSize)
from dbo.CurrencyExchangeRates
where [Timestamp] >= #Start and [Timestamp] < #End
set #r = ##ROWCOUNT
end
end

Maybe you need in python:
Start = int(input("Enter Start Time: "))
sql = 'exec UserInput #start =?'
cursor.execute(sql, Start)
cursor.commit()

If you really want to make the table name dynamic then you'll need to use dynamic SQL, like so:
ALTER PROCEDURE DeleteFrom
#tableName sysname ='Table0'
AS
BEGIN
SET NOCOUNT ON;
DECLARE #sql nvarchar(4000);
SET #sql = N'DELETE TOP (1) FROM ' + QUOTENAME(#tableName);
EXEC(#sql);
END

Related

Facing Jump of around 80-100 in Id column in SQL Server 2016 which did not have identity but generate number using sequence in code

Please find the SQL followed to generate new seq number
CREAT PROCEDURE [dbo].[axsp_get_next_no_internal_64]
#table_name nvarchar(30),
#increment int = 1,
#log_id BIGINT = 0,
#is_use_sequence bit = 1,
#new_next_no bigint OUTPUT
as
SET #new_next_no = 0;
IF #is_use_sequence = 0
BEGIN
DECLARE #next_no_tab table(next_no bigint);
UPDATE [dbo].[next_no_64] set next_no = next_no + #increment output inserted.next_no into #next_no_tab where table_name = #table_name
SELECT #new_next_no = next_no - #increment from #next_no_tab
END
ELSE
BEGIN
DECLARE
#first_seq_num sql_variant,
#seq_name nvarchar(max) = 'dbo.n_' + #table_name,
#sql nvarchar(max) = N'SELECT #retval_out = (NEXT VALUE FOR n_' + #table_name + N')';
IF #increment = 1
EXECUTE sp_executesql #sql, N'#retval_out bigint OUTPUT', #retval_out=#new_next_no OUTPUT;
ELSE
BEGIN
EXEC sys.sp_sequence_get_range
#sequence_name = #seq_name
, #range_size = #increment
, #range_first_value = #first_seq_num OUTPUT;
SELECT
#new_next_no = convert(bigint, #first_seq_num);
END
END
create procedure sys.sp_sequence_get_range
#sequence_name nvarchar(776),
#range_size bigint,
#range_first_value sql_variant output,
#range_last_value sql_variant = null output,
#range_cycle_count int = null output,
#sequence_increment sql_variant = null output,
#sequence_min_value sql_variant = null output,
#sequence_max_value sql_variant = null output
as
declare #ret int
exec #ret = sys.sp_sequence_get_range_internal #sequence_name, #range_size, #range_first_value output, #range_last_value output,
#range_cycle_count output, #sequence_increment output, #sequence_min_value output, #sequence_max_value output
return #ret
Parameter always goes as #increment = 1, #log_id =1,
#is_use_sequence = 1 with table name
I have already read article about sequence chasm (https://sqlity.net/en/792/the-gap-in-the-identity-value-sequence/) but can it be possible with above sequencing as i can see sequencing is solution to this by switching off catching.
But let me know and who can i replicate so i can document for my project.
With a lot of replication, i found when i do the SQL server shutdown directly from active window and then again start MSSQLSERVER Service. In sys.sequences contained table new value is updated with 100+ number and totally replaced what previous value is there.
Due to this whole sequencing logic return number by incrementing the new value. like 104+1 = 105, 106,107 ..etc.
Below are screenshot 1st one is before the server shutdown and 2nd is after server restart.
So i need to do workaround this sys.sequences sys.sequences contained table or may be i can provide them script so if anytime if someone take server shutdown or restart they can execute the script to reset the start value to what is max value available in table.

SELECT statement in while loop isn't working

I have created the following Select statement which is working fine:
SELECT #Block = [Blok], #Year = [Aar]
FROM [PT99991_Result].[dbo].[Testheader]
WHERE N = #TestHeaderID
The problem is that this Select statement is used in a While loop where the database can change to another one during the loop. I have tried to modify the statement to the following but it's not working. I have also tried to use EXEC which takes care of my problem but then I'm facing a problem with the local variables #Block and #Year instead.
SET #DataBase = 'PT99991_RESULT' --This is only for test!
SELECT #Block = [Blok], #Year = [Aar]
FROM '[' + #DataBase + '].[dbo].[Testheader]'
WHERE N = #TestHeaderID
I'm not sure what I'm doing wrong?
First, generate a T-SQL template like this:
DECLARE #DynamicTSQLStatement NVARCHAR(MAX) = N'SELECT #Block = [Blok], #Year = [Aar]
FROM [#DataBase].[dbo].[Testheader]
WHERE N = #TestHeaderID';
Then, let's say that the variale #Datbase holds your current database name. If it is not extracted from sys.database you can perform additional validation to ensure nobody is doing something wrong.
IF NOT EXISTS(SELEFT 1 FROM sys.database WHERE [name] = #Datbase
BEGIN
....
END;
After the validation, you just replace the database name in the template:
SET #DynamicTSQLStatement = REPLACE(#DynamicTSQLStatement, '#Database', #Database);
Then, execute the code passing the parameters:
EXEC sp_executesql #DynamicTSQLStatement
,N'#TestHeaderID INT'
,N'#Block INT OUTPUT'
,N'#Year INT OUTPUT'
,#TestHeaderID
,#Block OUTPUT
,#Year OUTPUT;
of course on every loop iteration, reset the template.
Instead of while loop, You can go for undocumented stored procedure: ms_foreachdb and execute against the databases and finally apply filter for the specific database.
Caveat: Don't use this in production code, because, it uses undocumented stored procedure.
CREATE TABLE #test(dbname sysname, blok int, aar int)
DECLARE #db_list NVARCHAR(max) = 'DB1,DB2'
EXECUTE master.sys.sp_MSforeachdb 'USE [?]; declare #blok int, #aar int; INSERT INTO #test SELECT db_name(), blok, aar from [dbo].[Testheader] WHERE N = TestHeaderId;'
SELECT * FROM #test where dbname in
(
SELECT value FROM string_split(#db_list,',')
)

T-SQL: can't seem to set variables immediately after declaring them

I have looked through countless posts on declaring/setting variables in a sql script. But most seem to involve syntax errors or using exec commands. The script I am trying to execute is quite simple - so I am having a hard time understanding why I cant set values.
Here is the SQL:
declare #counter int = 1,
#batchSize int = 100000,
#tableRows int,
#totalBatches int;
set #tableRows= (select count(distinct chFileVersionID) from IRISDocuments)
set #totalBatches = (ceiling((#tableRows / #batchSize)));
--print some stuff
while #counter <= #totalBatches
begin
. . . loop logic that only uses #counter variable for incrementing
end
The error I get is
Must declare the scalar variable "#tableRows"
which is clearly declared directly above. I have tried setting values with a select statement, as well as declaring each variable individually, and declaring and setting value in the same statement with no avail.
The above actually works (I mean testing without the WHILE loop of course).
2 possible source of the error if the variable is not exactly used as above:
Using a variable to define another variable in the same declaration
E.g.:
declare #counter int = 1,
#batchSize int = 100000,
#tableRows int = 10000000000000,
#totalBatches int = (ceiling((#tableRows / #batchSize)))
This won't work and produces your error - within a statement the order of execution is not row-by-row, so you can not give a value to #totalBatches as at that point #tableRows is not implemented yet.
Trying to use a variable that is out of scope
If this is a query that contains the GO batch separator then after that all variables before GO are out of scope.
E.g.:
declare #counter int = 1,
#batchSize int = 100000,
#tableRows int,
#totalBatches int;
SELECT #counter, #batchSize, #tableRows, #totalBatches int
GO
SELECT #counter, #batchSize, #tableRows, #totalBatches int
The second SELECT will throw your error.

"Subsequent parameters" error in SSRS?

I'm getting a strange error in SSRS, in a report (which gets put into a sproc) with many drop-down parameters:
Query execution failed for dataset 'DataSet1'.
Must pass parameter number 3 and subsequent parameters as '#name = value'. After the form '#name = value' has been used, all subsequent parameters must be passed in the form '#name = value'
I'm lost on what's going on here - what is meant by #name = value .
I searched online, someone mentioned that you should alter the stored-procedure?
Here is how the top half of my stored-proc looks:
USE [FederatedSample]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[prc_RPT_Select_BI_Completes_Data_View_2]
#FromDate DATETIME,
#ToDate DATETIME,
#AccountIDs VARCHAR(max) = null,
#ClientIDs VARCHAR(max) = null,
#SupplierIDs VARCHAR(max) = null,
#CompleteType INT = NULL,
/*
* 0 - Routed
* 1 - Targeted
* 2 - Offerwall
*/
#SourceType BIT = NULL,
/*
* Works if #AccountID is not null
* (should only be used if #AccountID has a single value)
*
* 0 - Owned by #AccountID
* 1 - External (not owned by #AccountID)
*/
#SurveyStatus INT = NULL,
/*
* NULL - All Surveys
* 0 - Completes Approved Surveys
* 1 - Invoiced Surveys
*/
#IsSupplierUser BIT = 0
/*
* used to decide whether to display FEDSurveyName or SupplierSurveyName
*/
AS
BEGIN
SET NOCOUNT ON
DECLARE #SQL NVARCHAR(MAX) = N'',
#Params NVARCHAR(MAX)
IF #AccountIDs is not null
BEGIN
SET #SQL += N'DECLARE #AccountIDs VARCHAR(MAX) = #pAccountIDs; '
END
IF #ClientIDs is not null
BEGIN
SET #SQL += N'DECLARE #ClientIDs VARCHAR(MAX) = #pClientIDs; '
END
IF #SupplierIDs is not null
BEGIN
SET #SQL += N'DECLARE #SupplierIDs VARCHAR(MAX) = #pSupplierIDs; '
END
SET #SQL += N'
SELECT bi.SupplierID as ''Supplier ID''
,bi.SupplierName as ''Supplier Name''
,bi.PID as ''PID''
,bi.RespondentID as ''Respondent ID''
,lk_slt.Name as ''Entry Link Type''
,ts.SurveyNumber as ''Initial Survey ID'''
And later in the stored proc. it does stuff like this to split strings:
IF #AccountIDs is not null
BEGIN
SET #SQL += CHAR(13) + CHAR(9)
SET #SQL += N' and bi.AccountID in (SELECT CAST(val as INT) FROM dbo.Split(#AccountIDs, '','
When invoking a stored procedure, you either can pass the parameters by position (not a good idea) or by Name (a better approach IMHO).
EXEC dbo.MyStoredProcedure '12/31/2012', 1; -- Not a great way to pass parameters
EXEC dbo.MyStoredProcedure #AsOfDate = '12/31/2012', #AccountID = 1; -- A better way
From the error message you are receiving, I suspect that SSRS is using the second approach and is running into an issue with the third parameter being provided to the stored procedure.
Without more information to go off of it is difficult to provide you with an exact explanation for the error (the stored procedure would perhaps help), an educated guess is that the way the parameters are being provided for Account IDs, Client IDs and Supplier IDs isn't quite correct. Specifically, I think the problem might be that you are providing multiple identifiers delimited by a comma.
You might try passing a single Account ID, Client ID and Supplier ID to see if you still receive the error. I would also try to look at the stored procedure (or talk to the DBA \ Developer who wrote it) to ascertain the intended usage of the stored procedure.
I got a similar message when passing a comma instead of a full stop for a decimal value in one of the parameters of a stored procedure.
Here is a simplified example of what happened.
The following command string was erroneously generated with the value of #param2 with a comma instead of a full stop, causing a misunderstanding of the number of parameters.
EXEC myStoredProc #param1 = 1, #param2 = 0,5 ,#param3 = 'something'

Pass string variable into procedure and add it to a query

I have an application written in C#, which connects to database and analyze its data, database stores information about execution of automated tests, what I would like to do is to retrieve those tests that fulfill the above given conditions. But we are having different projects and will be supporting more and more so I do not want to construct different procedure for each one, but pass the name - 2nd parameter deploy as the parameter so the query will depend on the project and return the data to the application, then I will send it in a report.
For the time being it looks like this:
CREATE PROCEDURE [dbo].[SuspectsForFalsePositive](#build_id INT, #deploy VARCHAR(25))
AS
BEGIN
SET NOCOUNT ON;
DECLARE #i int, #build int, #deployname varchar(25), #SQL varchar(max)
DECLARE #result table (tc int, fp float)
SET #i = 0
SET #build = #build_id
SET #deployname = #deploy
SET #SQL = 'insert '+#result+'select testcase_id, fail_percentage FROM [BuildTestResults].[dbo].['+#deployname+'TestCaseExecution]
where build_id = #build and fail_percentage >= 70'
--INSERT #result select testcase_id, fail_percentage FROM [BuildTestResults]
--.[dbo].[ABCTestCaseExecution]
--where build_id = #build and fail_percentage >= 70
--commented works
EXEC(#SQL)
WHILE (##rowcount = 0)
BEGIN
SET #build = #build - 1
EXEC(#SQL)
--INSERT #result select testcase_id, fail_percentage FROM [BuildTestResults].[dbo]. --[ABCTestCaseExecution]
--where build_id = #build and fail_percentage >= 70
--commented works
END
select * from #result order by fp DESC
END
GO
Thanks for any advice !
In your string you have #build - this is interpreted as a string. At the time you execute the #SQL it doesn't contain such a variable, so you get a failure.
You need to concatenate the value directly:
SET #SQL = 'insert '+#result+'select testcase_id, fail_percentage FROM [BuildTestResults].[dbo].['+#deployname+'TestCaseExecution]
where build_id = '+#build+' and fail_percentage >= 70'
You will need to do that between executions too.
There are a few issues with your example. This is, however, one over-arching consideration.
Variables (table and/or scalar) are only visible in the StoredProcedure they are defined in. And calling EXEC(#SQL) is calling a stored. This means that neither your #result table, not your other parameters are visible to the dynamic SQL you are executing.
In terms of the table, you can get around that by creating a temp table instead. And for the scalar variables, you can pass them around when using SP_EXECUTESQL instead of EXEC.
I don't have access to sql server at present, but maybe somethign like this can start you on your way...
CREATE PROCEDURE [dbo].[SuspectsForFalsePositive](#build_id INT, #deploy VARCHAR(25))
AS
BEGIN
SET NOCOUNT ON;
DECLARE
#i int,
#build int,
#deployname varchar(25),
#SQL varchar(max)
CREATE TABLE #result (
tc int,
fp float
)
SELECT
#i = 0,
#build = #build_id,
#deployname = #deploy
SET #sql = ''
SET #sql = #sql + ' INSERT INTO #result'
SET #sql = #sql + ' SELECT testcase_id, fail_percentage'
SET #sql = #sql + ' FROM [BuildTestResults].[dbo].['+#deployname+'TestCaseExecution]'
SET #sql = #sql + ' WHERE build_id = #build and fail_percentage >= 70'
SP_EXECUTESQL
#SQL,
'#build INT',
#build
WHILE (##rowcount = 0)
BEGIN
SET #build = #build - 1
SP_EXECUTESQL
#SQL,
'#build INT',
#build
END
SELECT * FROM #result ORDER BY fp DESC
END
GO
I also occures to me that ##rowcount may now see the rows being processes within SP_EXECUTESQL. In which case you may need to re-arrange things a little (using an output parameter, or embedding the loop in the #SQL, etc).
Overall it feels a bit clunky. With more information about your schema, etc, it may be possible ot avoid the dynamic SQL. This will have several benefits, but one in particular:
- Right now you're open to SQL Injection Attacks on the #deploy parameter
Anyone that can execute this SP, and/or control the value in the #deploy parameter could wreak havok in your database.
For example... Could you store all the TestCaseExecutions in the same table? But with an extra field: TestCaseID *(Or even TestCaseName)?
Then you wouldn't need to build dynamic SQL to control which data set you are processing. Instead you just add WHERE TestCaseID = #TestCaseID to your query...