This probably has a simple answer but I cannot figure it out as I'm still getting a hang of working with R in SQL Server. I have a piece of code that reads in data from a SQL Server table, executes in R and returns a data frame.
execute sp_execute_external_script
#language=N'R',
#script=N'inp_dat=InputDataSet
inp_dat$NewCol=max(inp_dat$col1,inp_dat$col2)
new_dat=inp_dat
OutputDataSet=new_dat'
#input_data_1=N'select * from IM_COMP_TEST_SQL2016.dbo.temp_table';
I want to insert new_dat into a SQL Server table (select * into new_table from new_dat). How do I go about this?
As shown in this tutorial, you can use INSERT INTO ... EXEC in a previously created table with columns aligning to script's dataframe return:
INSERT INTO Table1
execute sp_execute_external_script
#language=N'R',
#script=N'inp_dat <- InputDataSet
inp_dat$NewCol <- max(inp_dat$col1,inp_dat$col2)
new_dat <- inp_dat',
#input_data_1=N'SELECT * FROM IM_COMP_TEST_SQL2016.dbo.temp_table',
#output_data_1=N'newdat';
However, to use the make-table query may require OPENQUERY() or OPENROWSET() using an ad-hoc distributed query as described in this SO Post to return the output of stored procedure:
Stored Procedure
CREATE PROCEDURE dbo.R_DataFrame
AS
BEGIN
execute sp_execute_external_script
#language=N'R',
#script=N'inp_dat <- InputDataSet
inp_dat$NewCol <- max(inp_dat$col1,inp_dat$col2)
new_dat <- inp_dat',
#input_data_1=N'SELECT * FROM IM_COMP_TEST_SQL2016.dbo.temp_table',
#output_data_1=N'newdat';
-- ADD ALL COLUMN TYPES;
WITH RESULT SETS (("newdat" [col1] varchar(20), [col2] double, [col3] int ...));
END
GO
Action Query
SELECT * INTO Table1
FROM OPENROWSET('SQLNCLI', 'Server=(local);Trusted_Connection=yes;',
'EXEC dbo.R_DataFrame')
Related
The query that I've been working on for awhile now was filled with 7 Temp Tables until I had to replace them with CTE's (7 CTE's) due to OPENQUERY giving the following error when using TempTables:
Metadata discovery only supports temp tables when analyzing a single- statement batch.
When I run the Query with Temp Tables, the run duration is:
7:50
When I run the Query with CTE's, the run duration is:
15:00
Almost double the time! Is there any other alternative to OPENQUERY that might make it run faster while perhaps keeping my temp tables?
Current execution Query:
SET #XSql = 'SELECT * FROM OPENQUERY([server], ''' + REPLACE(#QSql, '''', '''''') + ''')'
EXEC(#XSql)
I used this for reference: Stored Procedure and populating a Temp table from a linked Stored Procedure with parameters
And need a optimal solution.
Open to suggestions!
Can you use EXEC ... AT SERVER? This worked fine for me:
EXEC ('CREATE TABLE #TestTable1 (ID int); CREATE TABLE #TestTable2 (ID int); SELECT * FROM #TestTable1, #TestTable2;') AT LinkedServer;
I have been trying to call stored procedure from another stored procedure. Now issue is that under lying nested stored procedure contains CROSS APPLY with temp table and it runs fine when i execute it directly.
But when i try to call this SP from other SP, it gives error that one of the column is invalid. "Invalid column name 'levels'" in this case.
Plus, when i execute this SP from calling SP SQL window with passing parameters, it runs fine and whole main procedure starts running smoothly.
I am not able to get why this issue happens. Below is kind of implementation for reference.
1.) Main SP
CREATE STORED PROCEDURE ....
INSERT INTO #TempTable
EXEC [Child_SP] #Param1 = 1, #Param2 = 1
...
Gives error.
2.) Once i execute below given as single statement once from main PS. It starts working fine.
EXEC [Child_SP] #Param1 = 1, #Param2 = 1
3.) Child SP has CROSS APLLY with one of the temp table. something like below.
SELECT ID, '1,2,3,4,5' AS levels
INTO #Temp1
FROM ABC
SELECT ID
FROM #Temp1 x0
CROSS APPLY (SELECT * FROM dbo.iter_charlist_to_table(x0.levels, ',') AS x) x1
WHERE x1.listPos > 1
"iter_charlist_to_table" is table value function which get values as table from comma seperated list.
Is it related to SQL Thread anyhow or whats the issue? Thanks.
I recommend to use this code to drop your tmp table on the beginning of the SP because your insert INTO will ALWAYS tried to create the table doesn't matter if already exists.
IF OBJECT_ID('tempdb..#Temp1') IS NOT NULL DROP TABLE #Temp1
If you share more code will be more helpful to understand.
And just in case don't forget to put the alias on the table maybe this correction sometimes is not needed but is a good practice for avoid problems on querying the data on joined tables
SELECT x0.ID
FROM #Temp1 x0
CROSS APPLY (SELECT fnAlias.* FROM dbo.iter_charlist_to_table(x0.levels, ',') fnAlias) x1
WHERE x1.listPos > 1
I've got a very basic SQL query that I'd like to be able to view in R.
The trouble is, I need to be able to reference a #table:
select
RAND(1) as random
into #test
select * from #test
Is this possible, or will I need to create permanent tables, or find some other work around?
I currently do this via a RODBC script which allows me to choose which SQL file to run:
require(RODBC)
sql.filename <- choose.files('T:\\*.*')
sqlconn <- odbcDriverConnect("driver={SQL Server};Server=SERVER_NAME;Trusted_Connection=True;")
file.content <- readLines(sql.filename)
output <- sqlQuery(sqlconn, paste(file.content[file.content!='--'],collapse=' '))
closeAllConnections()
Do you have any advice on how I can utilise #tables in my SQL scrips in R?
Thanks in advance!
When you use temp tables SQL outputs a message with the number of rows in the table. R doesn't know what to do with this message. If you begin your SQL query with SET NOCOUNT ON SQL will not output the count message.
I use #tables by separating my query into two parts, it returns character(0) if I do like:
sqlQuery(test_conn, paste("
drop table #test;
select
RAND(1) as random
into #test
select * from #test
"))
So instead I would use:
sqlQuery(test_conn, paste("
drop table #test;
select
RAND(1) as random
into #test
"))
sqlQuery(test_conn,"select * from #test")
It seems to work fine if you send one Query to make the #table, and a second to retrieve the contents. I also added in drop table #test; to my query, this makes sure there is not already a #test. If you try to write to a #table name that is already there you will get an error
select top 0 * INTO #temp from stored procedure
Need to create temp table based on the structure of data type returned from stored procedure.
Using sql server 2000,2005, 0r 2008
You can't do this. To get the results from a stored procedure, you have to first define the structure of the results:
create table #temp ( . . . );
insert into #temp
exec(stored procedure)
If you examine the syntax for the SELECT statement (here), you'll see no reference to running a stored procedure.
Perhaps you should post another question describing what you are trying to do. Why would a stored procedure be returning different result formats?
I have a Stored Procedure in MSSQL 2008, inside of this i've created a Temporary Table, and then i executed several inserts into the temporary Table.
How can i select all the columns of the Temporary Table outside the stored procedure? I Mean, i have this:
CREATE PROCEDURE [dbo].[LIST_CLIENTS]
CREATE TABLE #CLIENT(
--Varchar And Numeric Values goes here
)
/*Several Select's and Insert's against the Temporary Table*/
SELECT * FROM #CLIENT
END
In another Query i'm doing this:
sp_configure 'Show Advanced Options', 1
GO
RECONFIGURE
GO
sp_configure 'Ad Hoc Distributed Queries', 1
GO
RECONFIGURE
GO
SELECT *
INTO #CLIENT
FROM OPENROWSET
('SQLOLEDB','Server=(local);Uid=Cnx;pwd=Cnx;database=r8;Trusted_Connection=yes;
Integrated Security=SSPI',
'EXEC dbo.LIST_CLIENTS ''20110602'', NULL, NULL, NULL, NULL, NULL')
But i get this error:
Msg 208, Level 16, State 1, Procedure LIST_CLIENTS, Line 43
Invalid object name '#CLIENT'.
I've tried with Global Temporary Tables and It doesn't work.
I know that is the scope of the temporary table, but, how can i get the table outside the scope of the SP?
Thanks in advance
I think there is something deeper going on here.
One idea is to use a table variable inside the stored procedure instead of a #temp table (I have to assume you're using SQL Server 2005+ but it's always nice to state this up front). And use OPENQUERY instead of OPENROWSET. This works fine for me:
USE tempdb;
GO
CREATE PROCEDURE dbo.proc_x
AS
BEGIN
SET NOCOUNT ON;
DECLARE #x TABLE(id INT);
INSERT #x VALUES(1),(2);
SELECT * FROM #x;
END
GO
SELECT *
INTO #client
FROM OPENQUERY
(
[loopback linked server name],
'EXEC tempdb.dbo.proc_x'
) AS y;
SELECT * FROM #client;
DROP TABLE #client;
DROP PROCEDURE dbo.proc_x;
Another idea is that perhaps the error is occurring even without using SELECT INTO. Does the stored procedure reference the #CLIENT table in any dynamic SQL, for example? Does it work when you call it on its own or when you just say SELECT * FROM OPENROWSET instead of SELECT INTO? Obviously, if you are working with the #temp table in dynamic SQL you're going to have the same kind of scope issue working with a #table variable in dynamic SQL.
At the very least, name your outer #temp table something other than #CLIENT to avoid confusion - then at least nobody has to guess which #temp table is not being referenced correctly.
Since the global temp table failed, use a real table, run this when you start your create script and drop the temp table once you are done to make sure.
IF OBJECT_ID('dbo.temptable', 'U') IS NOT NULL
BEGIN
DROP TABLE dbo.temptable
END
CREATE TABLE dbo.temptable
( ... )
You need to run the two queries within the same connection and use a global temp table.
In SQL Server 2008 you can declare User-Defined Table Types which represent the definition of a table structure. Once created you can create table parameters within your procs and pass them a long and be able to access the table in other procs.
I guess the reason for such behavior is that when you call OPENROWSET from another server it firstly and separately requests the information about procedure output structure (METADATA). And the most interesting thing is that this output structure is taken from the first SELECT statement found in the procedure. Moreover, if the SELECT statement follows the IF-condition the METADATA request ignores this IF-condition, because there is no need to run the whole procedure - the first met SELECT statement is enough. (By the way, to switch off that behavior, you can include SET FMTONLY OFF in the beginning of your procedure, but this might increase the procedure execution time).
The conclusions:
— when the METADATA is being requested from a temp table (created in a procedure) it does not actually exists, because the METADATA request does not actually run the procedure and create the temp table.
— if a temp table can be replaced with a table variable it solves the problem
— if it is vital for the business to use temp table, the METADATA request can be fed with fake first SELECT statement, like:
declare #t table(ID int, Name varchar(15));
if (0 = 1) select ID, Name from #t; -- fake SELECT statement
create table #T (ID int, Name varchar(15));
select ID, Name from #T; -- real SELECT statement
— and one more thing is to use a common trick with FMTONLY (that is not my idea) :
declare #fmtonlyOn bit = 0;
if 1 = 0 set #fmtonlyOn = 1;
set fmtonly off;
create table #T (ID int, Name varchar(15));
if #fmtonlyOn = 1 set fmtonly on;
select ID, Name from #T;
The reason you're getting the error is because the temp table #Client was not declared before you ran the procedure to insert into it. If you declare the table, then execute the list proc and use direct insert -
INSERT INTO #Client
EXEC LIST_CLIENTS