How to retrieve multiple rows from a SQL Server stored procedure using Pyodbc in Jupyter Notebook - sql

I have a stored procedure in SQL Server which takes 3 input parameters and can produce multiple rows as output. In fact, it is returning 20 rows in my current case. For instance, if I manually execute the stored procedure from SSMS, I get the following code and partial output, respectively:
Code:
DECLARE #return_value int
EXEC #return_value = [Coverage-Source].[ReadCoverageMapping]
#client = N'Capital BlueCross',
#lineOfBusiness = N'Commercial',
#distributionChannel = N'Retail'
SELECT 'Return Value' = #return_value
GO
Output:
ID Attribute Value CoverageName
-----------------------------------------
1 Copay Yes Retail Base
2 Copay No Retail
. . . .
. . . .
Return Value
20
Now, while I try to read the stored procedure from jupyter notebook using pyodbc, I get an error
Procedure or function ReadCoverageMapping has too many arguments specified
I want output something like this:
ID Attribute Value CoverageName
------------------------------------------
1 Copay Yes Retail Base
2 Copay No Retail
. . . .
. . . .
I tried this code:
client = 'Capital BlueCross'
lineOfBusiness = 'Commercial'
distributionChannel = 'Retail'
cnxn = pyodbc.connect(r'Driver={SQL
Server};Server=MyServer;Database=COV_SRCE_TEST;Trusted_Connection=yes;')
sql = """\
DECLARE #out nvarchar(max);
EXEC [cov_srce_test].[coverage-source].ReadCoverageMapping #client = ?, #lineOfBusiness = ?,
#distributionChannel = ?, #param_out = #out OUTPUT;
SELECT #out AS the_output;
"""
values = (client, lineOfBusiness, distributionChannel)
cnxn.execute(sql, values)
rows = cnxn.fetchall()
while rows:
print(rows)
if cnxn.nextset():
rows = cnxn.fetchall()
else:
rows = None
Is there any way to achieve this? I tried using multiple ways, but couldn't find a solution.

Those batches are different. Looks like it should be:
sql = """\
DECLARE #return_value int
EXEC #return_value = [Coverage-Source].[ReadCoverageMapping]
#client = ?,
#lineOfBusiness = ?,
#distributionChannel = ?
SELECT 'Return Value' = #return_value
"""
which will output two resultsets, one from the stored procedure, and a second one for the Return value which you can probably omit.

Related

How to use parameters in SQL Server?

Beginner here! I just started working with SQL Server in Azure functions (in Typescript). I am trying to update a row in the database using declared variables but I am failing to do so with VARCHAR types. Interesting is that in the database tool it is working fine but once I try to run the query inside the Azure function, I get an http me 500 error
RequestError: Incorrect syntax near 'kg'
(working with weight as a string).
At the moment, my code looks like this:
const trainingId: number = parseInt(req.params.training_id);
const exerciseId: number = parseInt(req.params.exercise_id);
const weight: string = req.body.weight;
await mssql.query(
`DECLARE #sql NVARCHAR(4000);
DECLARE #training_id INT;
DECLARE #exercise_id INT;
DECLARE #weight VARCHAR(255);
set #sql = N'
UPDATE Exercises SET weight = #weight WHERE training_id = #training_id AND exercise_id = #exercise_id'
SET #training_id = ${trainingId};
SET #exercise_id = ${exerciseId};
SET #weight = ${weight};
exec sp_executesql #sql, N'#training_id INT, #exercise_id INT, #weight VARCHAR(255)',
#training_id, #exercise_id, #weight`
I have also tried the syntax where I insert the weight variable into the query like this:
SET weight ' + #weight + '
and did not work either.
I have also tried to use this approach here: How to pass parameter to mssql query in node js
But it did not work for me either. So how can I actually pass the parameters correctly? I am trying to avoid SQL injection.
Thank you so much for any advice!
With SET #weight = ${weight}; you are injecting values into a query that you use to execute another query. Why not use that UPDATE query directly after you use mssql.input(...) to set the parameter values (as per your link)?
As I understand it, it should be something like:
const trainingId: number = parseInt(req.params.training_id);
const exerciseId: number = parseInt(req.params.exercise_id);
const weight: string = req.body.weight;
// set up parameters
mssql.input('training_id', Sql.Int, trainingId);
mssql.input('exercise_id', Sql.Int, exerciseId);
mssql.input('weight ', Sql.VarChar, weight);
// execute just the UPDATE query, using the parameters
await mssql.query('UPDATE Exercises SET weight = #weight WHERE training_id = #training_id AND exercise_id = #exercise_id');
Perfect! I was doing it wrong the whole time as I was not linking the new Request to the mssql. What works for me now is:
const request: mssql.Request = new mssql.Request();
request.input('training_id', mssql.Int, trainingId);
request.input('exercise_id', mssql.Int, exerciseId);
request.input('weight ', mssql.VarChar, weight);
const result = await request.query('UPDATE Exercises SET weight = #weight WHERE training_id = #training_id AND exercise_id = #exercise_id');
Thanks a lot for the help!

Is it possible to invoke BigQuery procedures in python client?

Scripting/procedures for BigQuery just came out in beta - is it possible to invoke procedures using the BigQuery python client?
I tried:
query = """CALL `myproject.dataset.procedure`()...."""
job = client.query(query, location="US",)
print(job.results())
print(job.ddl_operation_performed)
print(job._properties) but that didn't give me the result set from the procedure. Is it possible to get the results?
Thank you!
Edited - stored procedure I am calling
CREATE OR REPLACE PROCEDURE `Project.Dataset.Table`(IN country STRING, IN accessDate DATE, IN accessId, OUT saleExists INT64)
BEGIN
IF EXISTS (SELECT 1 FROM dataset.table where purchaseCountry = country and purchaseDate=accessDate and customerId = accessId)
THEN
SET saleExists = (SELECT 1);
ELSE
INSERT Dataset.MissingSalesTable (purchaseCountry, purchaseDate, customerId) VALUES (country, accessDate, accessId);
SET saleExists = (SELECT 0);
END IF;
END;
If you follow the CALL command with a SELECT statement, you can get the return value of the function as a result set. For example, I created the following stored procedure:
BEGIN
-- Build an array of the top 100 names from the year 2017.
DECLARE
top_names ARRAY<STRING>;
SET
top_names = (
SELECT
ARRAY_AGG(name
ORDER BY
number DESC
LIMIT
100)
FROM
`bigquery-public-data.usa_names.usa_1910_current`
WHERE
year = 2017 );
-- Which names appear as words in Shakespeare's plays?
SET
top_shakespeare_names = (
SELECT
ARRAY_AGG(name)
FROM
UNNEST(top_names) AS name
WHERE
name IN (
SELECT
word
FROM
`bigquery-public-data.samples.shakespeare` ));
END
Running the following query will return the procedure's return as the top-level results set.
DECLARE top_shakespeare_names ARRAY<STRING> DEFAULT NULL;
CALL `my-project.test_dataset.top_names`(top_shakespeare_names);
SELECT top_shakespeare_names;
In Python:
from google.cloud import bigquery
client = bigquery.Client()
query_string = """
DECLARE top_shakespeare_names ARRAY<STRING> DEFAULT NULL;
CALL `swast-scratch.test_dataset.top_names`(top_shakespeare_names);
SELECT top_shakespeare_names;
"""
query_job = client.query(query_string)
rows = list(query_job.result())
print(rows)
Related: If you have SELECT statements within a stored procedure, you can walk the job to fetch the results, even if the SELECT statement isn't the last statement in the procedure.
# TODO(developer): Import the client library.
# from google.cloud import bigquery
# TODO(developer): Construct a BigQuery client object.
# client = bigquery.Client()
# Run a SQL script.
sql_script = """
-- Declare a variable to hold names as an array.
DECLARE top_names ARRAY<STRING>;
-- Build an array of the top 100 names from the year 2017.
SET top_names = (
SELECT ARRAY_AGG(name ORDER BY number DESC LIMIT 100)
FROM `bigquery-public-data.usa_names.usa_1910_2013`
WHERE year = 2000
);
-- Which names appear as words in Shakespeare's plays?
SELECT
name AS shakespeare_name
FROM UNNEST(top_names) AS name
WHERE name IN (
SELECT word
FROM `bigquery-public-data.samples.shakespeare`
);
"""
parent_job = client.query(sql_script)
# Wait for the whole script to finish.
rows_iterable = parent_job.result()
print("Script created {} child jobs.".format(parent_job.num_child_jobs))
# Fetch result rows for the final sub-job in the script.
rows = list(rows_iterable)
print("{} of the top 100 names from year 2000 also appear in Shakespeare's works.".format(len(rows)))
# Fetch jobs created by the SQL script.
child_jobs_iterable = client.list_jobs(parent_job=parent_job)
for child_job in child_jobs_iterable:
child_rows = list(child_job.result())
print("Child job with ID {} produced {} rows.".format(child_job.job_id, len(child_rows)))
It works if you have SELECT inside your procedure, given the procedure being:
create or replace procedure dataset.proc_output() BEGIN
SELECT t FROM UNNEST(['1','2','3']) t;
END;
Code:
from google.cloud import bigquery
client = bigquery.Client()
query = """CALL dataset.proc_output()"""
job = client.query(query, location="US")
for result in job.result():
print result
will output:
Row((u'1',), {u't': 0})
Row((u'2',), {u't': 0})
Row((u'3',), {u't': 0})
However, if there are multiple SELECT inside a procedure, only the last result set can be fetched this way.
Update
See below example:
CREATE OR REPLACE PROCEDURE zyun.exists(IN country STRING, IN accessDate DATE, OUT saleExists INT64)
BEGIN
SET saleExists = (WITH data AS (SELECT "US" purchaseCountry, DATE "2019-1-1" purchaseDate)
SELECT Count(*) FROM data where purchaseCountry = country and purchaseDate=accessDate);
IF saleExists = 0 THEN
INSERT Dataset.MissingSalesTable (purchaseCountry, purchaseDate, customerId) VALUES (country, accessDate, accessId);
END IF;
END;
BEGIN
DECLARE saleExists INT64;
CALL zyun.exists("US", DATE "2019-2-1", saleExists);
SELECT saleExists;
END
BTW, your example is much better served with a single MERGE statement instead of a script.

Combining current date code with stored procedure ti use in PowerBi

To get the required table I have to input the value "201801" into the stored procedure query. I want to place the following code:
SELECT CONVERT(nvarchar(6), GETDATE(), 112)
In the following Sp:
USE [MDAmanager]
GO
DECLARE #return_value int
EXEC #return_value = [dbo].[sqSupplierBalances]
#Period = 201801,
#SupplierString = 'select SupplierID from Suppliers ',
#SelectionString = 'select * from vSupplierBalances order by
ControlOfficeName, PortfolioName, OwnerEntityName, RegionName,
PropertyName, PropertyManagerName, Custom1Name, Custom2Name,
ServiceTypeDescription, AnalysisCode, SupplierName',
#WithCommitments = NULL,
#UserID = NULL,
#ExcludeInDispute = NULL,
#IncludeSupplierPropertyReference = NULL
SELECT 'Return Value' = #return_value
GO
Is it possible to assign a value to the first code example and replace the "201801" in the second code example with that variable. I have been trying this but not getting it right.
Update: So I realize M query functions and SQL server functions are different. I don't how to go about answering my own question but I figured I'd give the answer here anyway.
I replaced the initial date code with:
Perdiod0 = (Date.Year(DateTime.LocalNow()) * 100) +
Date.Month(DateTime.LocalNow())
And then just replaced the 201801 with:
'" & Text.From(Period0) & "'
Seems to work now

While loop as a calculated Field in Select Statement

I'm trying to implement a calculated field that utilizes Newtons Method. I have f(x) and f'(x) and the calculation works fine when I run it alone for one calculation as a 'while begin' with static parameters. However, I need the Newton's method to run in the SELECT statement for each row in a query in it's own column.
'Perfect world' Pseudo code below:
DECLARE #x AS Float
DECLARE #count AS INT = 1
SELECT
Col1
,Col2
,Col3
, Set #x = col1
Then DO (
FOR #count = 1 to 100
SET #x = #x - f(#x)/f'(#x)
SET #count = #count + 1
NEXT #count )
OUTPUT #x AS 'Newtons Method'
ex: f(#x) = and f'(#x) can be defined arbitrary equations. i.e f(#x) = POWER(#x, 1/2) and f'(#x) = POWER(#x/2, -1/2).
Output would be of the form:
col1 col2 col3 Newtons Method
float float float float
. . . .
. . . .
. . . .
for all records in query .
Restriction 1) I do NOT have permissions to create stored procs or functions in my DB.
Restriction 2) I must complete this task only using TSQL.
Is this even possible?

SQL - Trying to add variable into string

I have a store procedure where I pass a path to the file like:
EXEC spMyPathFile
#PFile = 'C:\TFiles\Paths\Test_1.1_Version.txt'
What I'd like to do it loop through and be able to pass a number of versions of the file like 1.1 and 1.2 etc using:
DECLARE #intLp INT
DECLARE #a varchar(2)
SET #intLp = 1 WHILE (#intLp <2)
BEGIN IF #intLp = 1 BEGIN
SET #a = '1.1'
END
ELSE IF #intLp = 2
BEGIN
SET #a = '1.2'
END
EXEC spMyPathFile
#PFile = 'C:\TFiles\Paths\Test_'+#a+'_Version.txt'
SET #intLp = #intLp + 1
END
For some reason I get "Incorrect syntax near '+'." which is just before the #a. I'm obviously not joining my variable to my string properly.
Could someone give me an example of how this should look?
Change
EXEC spMyPathFile
#PFile = 'C:\TFiles\Paths\Test_'+#a+'_Version.txt'
to
declare #FileName varchar(100) = 'C:\TFiles\Paths\Test_' + #a + '_Version.txt'
EXEC spMyPathFile
#PFile = #FileName
Edit:
From MSDN - Specify Parameters
The parameter values supplied with a procedure call must be constants or a variable; a function name cannot be used as a parameter value. Variables can be user-defined or system variables such as ##spid.