I have a table in SQL Server which it contains ids for items, and they're not unique in that table. I am trying to pass it to an openquery to grab additional information for these items from oracle.
The table can have 5000-17000 row at a time. I tried to convert all the rows for that column to a comma separated string and then pass it to the open query
declare #results varchar(max)
set #results = (select stuff([list],1,3,'') as stuff_list
from (select '''''' + ',' + '''''' + cast(itemId as varchar(10)) as [text()]
from ItemTable sub
order by itemId asc
for xml path('')) sub_query([list]) )
then pass it to the openquery
DECLARE #SQL varchar(max)
SET #SQL = #SQL + 'SELECT * FROM OPENQUERY(op, ''SELECT * FROM ITEM_INFO WHERE ITEM_CODE IN('+#results+')'')'
--SET #SQL = #SQL + 'WHERE ITEM_CODE IN(' + #results +')'
EXEC (#SQL)
I get:
The character string that starts with 'SELECT * FROM.....' is too long. Maximum length is 8000.
So I thought I might be able to loop through the table and select 100 rows at a time and then pass it to the open query and so on. What's the best way to accomplish this?
You can leverage EXEC AT [LinkedServer] to overcome the limitation of 8000 characters.
Here, the datatype is MAX datatype, so you dont have 8000 character issue.
EXEC at MSDN
#string_variable Is the name of a local variable. #string_variable can
be any char, varchar, nchar, or nvarchar data type. These include the
(max) data types.
DECLARE #SQL varchar(max)
SET #SQL = 'SELECT * FROM ITEM_INFO WHERE ITEM_CODE IN('+#results+')'
EXEC (#SQL) AT op
REFERENCES
You can also dynamically pass parameters. More info
technet article for EXEC AT
I've a question about my SQL CTE construction. I'm working with Azure Data Factory and a Stored Procedure in my database. What I want to do is:
Return my data from my view to Azure Data Factory in JSON format.
Return the data filtered dynamically in the WHERE clause based on my ObjectCode in my view in JSON format.
-> To test step 2 I tried with a statical declared ObjectCode
But the single quote does not work right in the WHERE clause. I tried some things with REPLACE, CHAR(39) and double the quotes. Like they said here, here and here
Step 1 I've finished succesfull with this code:
BEGIN
DECLARE #TABLE TABLE(RESULT NVARCHAR(MAX))
DECLARE #QUERY NVARCHAR(MAX) = '
;WITH x(Params) as
(
SELECT * FROM [Schema].[View] FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
)
Select * from x
'
Insert #TABLE
EXEC (#QUERY)
Select ',"pipelineParameters": ' + LEFT(RESULT,LEN(RESULT)-1) + '}' as Params from #TABLE;
END
This query above gave me the right result. But, now I need to make a change for Step 2. I need to filter with the WHERE clause in the query.
So, I tried:
DECLARE #TABLE TABLE(RESULT NVARCHAR(MAX))
DECLARE #ObjectCode NVARCHAR(MAX) = 'Objectname'
DECLARE #QUERY NVARCHAR(MAX) = '
;WITH x(Params) as
(
SELECT * FROM [Schema].[View]
WHERE Objectcode = REPLACE(#ObjectCode, '''', '')
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
)
Select * from x
'
Insert #TABLE
EXEC (#QUERY)
Select ',"pipelineParameters": ' + LEFT(RESULT,LEN(RESULT)-1) + '}' as Params from #TABLE;
But when I run these query I get this error:
Does anyone know what I can improve here so I can get it working?
Does this do what you want?
Insert #TABLE
exec sp_executesql #QUERY, N'#ObjectCode nvarchar(max)', #ObjectCode=#ObjectCode;
As written, I would expect a syntax error when the query is run because of this WHERE clause:
WHERE Objectcode = REPLACE(#ObjectCode, '', ')
I think you intend:
DECLARE #QUERY NVARCHAR(MAX) = '
WITH x(Params) as (
SELECT *
FROM [Schema].[View]
WHERE Objectcode = REPLACE(#ObjectCode, '''''''', '''')
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
)
Select *
from x';
Quoting things in dynamic SQL is always verbose. You can print out #Query to see if it is really what you intend.
'
Thanks Gordon, the first one works for me:
Insert #TABLE
exec sp_executesql #QUERY, N'#ObjectCode nvarchar(max)', #ObjectCode=#ObjectCode;
I need a script that generate insert statements but with check for if the data doesn't already exist, this because it should be periodically run on parallell systems where different dtata will be added to the systems but we want them tables to be in sync. I have the basic ides and borrowed parts of code but get a syntax error i have trouble solving.
I'm basing my code on the code Param Yadav showed at Converting Select results into Insert script - SQL Server but I need to check for data already in the table. (I need to add more "bells & whistles later, but take this step-by-step)
My own main addition is the #NOT_EXISTS part which should be in the WHERE clause of the NOT EXISTS check. If I replace that with a plain WHERE 0=1 I get no syntax error so it indicates the error is in my #NOT_EXISTS string.
Edit: Yesterday I thought I had an answer to my own question but when running on "real data" I saw that some lines are too long for QUOTENAME, I have to fix those quotation marks "manually" (concats in script) instead...
SET NOCOUNT ON
DECLARE #CSV_COLUMN VARCHAR(MAX),
#QUOTED_DATA VARCHAR(MAX),
#NOT_EXISTS VARCHAR(MAX),
#SQL_KOD VARCHAR(MAX),
#TABLE_NAME VARCHAR(MAX),
#FILTER_CONDITION VARCHAR(MAX)='',
#FIRST_COL INT,
#LAST_COL INT
/* INPUT DATA */
SELECT #TABLE_NAME = 'WorkflowError'
SELECT #FIRST_COL = 2
SELECT #LAST_COL = 4
/* */
SELECT #CSV_COLUMN=STUFF
(
(
SELECT ',['+ NAME +']' FROM sys.all_columns
WHERE OBJECT_ID=OBJECT_ID(#TABLE_NAME) AND
is_identity!=1 FOR XML PATH('')
),1,1,''
)
--SELECT #CSV_COLUMN
SELECT #QUOTED_DATA=STUFF
(
(
SELECT ' ISNULL(QUOTENAME('+NAME+','+QUOTENAME('''','''''')+'),'+'''NULL'''+')+'','''+'+' FROM sys.all_columns
WHERE OBJECT_ID=OBJECT_ID(#TABLE_NAME) AND
is_identity!=1 FOR XML PATH('')
),1,1,''
)
SELECT #QUOTED_DATA=SUBSTRING(#QUOTED_DATA,1,LEN(#QUOTED_DATA)-5)
SELECT #QUOTED_DATA
SELECT #NOT_EXISTS=STUFF
(
(
SELECT ' ['+ COLUMN_NAME +']=', 'ISNULL(QUOTENAME('+COLUMN_NAME+','+QUOTENAME('''','''''')+'),'+'''NULL'''+') AND '
FROM information_schema.columns
WHERE table_name = #TABLE_NAME AND
ordinal_position BETWEEN #FIRST_COL AND #LAST_COL
FOR XML PATH('')
),1,1,''
)
SELECT #NOT_EXISTS=SUBSTRING(#NOT_EXISTS,1,LEN(#NOT_EXISTS)-4)
SELECT #NOT_EXISTS
--SELECT #NOT_EXISTS=' 0=1 '
SELECT #SQL_KOD='SELECT ''
IF NOT EXISTS(SELECT 1
FROM ' + #TABLE_NAME + ' WHERE ' + #NOT_EXISTS + ')
BEGIN
INSERT INTO '+#TABLE_NAME+'('+#CSV_COLUMN+')
VALUES('''+'+'+#QUOTED_DATA+'+'+''')
END
GO '''+' Insert_Scripts
FROM '+#TABLE_NAME + #FILTER_CONDITION
SELECT #SQL_KOD
EXECUTE (#SQL_KOD)
GO
[stackoverflow won't let me post code unless it's formatted, but then the strings below won't be as they are created in the script...]
When I do SELECT #NOT_EXISTS=' 0=1 ' I get an INSERT line for each row in my table:
IF NOT EXISTS(SELECT 1 FROM WorkflowError WHERE 0=1 )
BEGIN
INSERT INTO WorkflowError([TargetSystem],[ErrorCode],[ErrorText],[RetryMaxCount],[RetryStrategyName],[ErrorDescription])
VALUES('EttLiv','800','Value cannot be null. Parameter name: source','0',NULL,'Value cannot be null. Parameter name: source')
END
GO
With my #NOT_EXISTS code the #SQL_KOD string becomes this:
SELECT 'IF NOT EXISTS(SELECT 1 FROM WorkflowError
WHERE [TargetSystem]=ISNULL(QUOTENAME(TargetSystem,''''),'NULL'))
BEGIN
INSERT INTO WorkflowError([TargetSystem],[ErrorCode],[ErrorText],[RetryMaxCount],[RetryStrategyName],[ErrorDescription])
VALUES('+ISNULL(QUOTENAME(TargetSystem,''''),'NULL')+','
+ ISNULL(QUOTENAME(ErrorCode,''''),'NULL')+','
+ ISNULL(QUOTENAME(ErrorText,''''),'NULL')+','
+ ISNULL(QUOTENAME(RetryMaxCount,''''),'NULL')+','
+ ISNULL(QUOTENAME(RetryStrategyName,''''),'NULL')+','
+ ISNULL(QUOTENAME(ErrorDescription,''''),'NULL')+')
END
GO ' Insert_Scripts FROM WorkflowError
However, trying to execute that #SQL_KOD line just gives:
Msg 156, Level 15, State 1, Line 3
Incorrect syntax near the keyword 'NULL'.
...and I can't find out where I have done wrong, if it's in my thinking or if it's just a misplaced quotation mark...
Where do you expect #SQL_KOD to get its values from? Because if you are retrieving your values for TargetSystem / ErrorCode / ... / ErrorDescription from somewhere outside of your insert statement, I would expect a "from" statement. If you want to input variables, you are missing both the definition of the variables and the #-sign in front of the variable name.
As far as keeping quotes happy: try writing your code with QUOTED_IDENTIFIER OFF - you can create the entire #SQL_KOD variable by writing between double quotes ("), and single quotes would behave like normal quotation marks.
A very basic re-write of your code could be something as follows:
SET QUOTED_IDENTIFIER OFF
DECLARE #SQL_KOD VARCHAR(MAX)
SET #SQL_KOD =
"DECLARE #WorkFlowError TABLE ([TargetSystem] NVARCHAR(200),[ErrorCode] NVARCHAR(200))
IF NOT EXISTS ( SELECT 1 FROM #WorkFlowError )
BEGIN
INSERT INTO #WorkFlowError ([TargetSystem],[ErrorCode])
SELECT ISNULL(QUOTENAME([TargetSystem],''''),'NULL')
, ISNULL(QUOTENAME([ErrorCode],''''),'NULL')
FROM (
SELECT [TargetSystem]='Foo'
, [ErrorCode]='Bar'
) src
END";
I originally used QUOTENAME as in the Param Yadav script I borrowed from but that function can't handle long strings. It doesn't complain, just returns NULL if the string is too long. Now the script is less readable (long lines of quotation marks) but now works.
SET NOCOUNT ON
DECLARE #CSV_COLUMN VARCHAR(MAX),
#QUOTED_DATA VARCHAR(MAX),
#NOT_EXISTS VARCHAR(MAX),
#SQL_KOD VARCHAR(MAX),
#TABLE_NAME VARCHAR(MAX),
#FILTER_CONDITION VARCHAR(MAX),
#FIRST_COL INT,
#LAST_COL INT
/* INPUT DATA */
SELECT #TABLE_NAME = 'WorkflowError'
SELECT #FIRST_COL = 2
SELECT #LAST_COL = 4
SELECT #FILTER_CONDITION = ''
/* */
SELECT #CSV_COLUMN=STUFF
(
(
SELECT ',['+ NAME +']' FROM sys.all_columns
WHERE OBJECT_ID=OBJECT_ID(#TABLE_NAME) AND
is_identity!=1 FOR XML PATH('')
),1,1,''
)
SELECT #QUOTED_DATA=STUFF
(
(
SELECT ' ISNULL('''''''' + REPLACE('+NAME+','''''''','''''''''''') + '''''''','''+'NULL'''+''+')+'',''+'
FROM sys.all_columns
WHERE OBJECT_ID=OBJECT_ID(#TABLE_NAME) AND
is_identity!=1 FOR XML PATH('')
),1,1,''
)
SELECT #QUOTED_DATA=SUBSTRING(#QUOTED_DATA,1,LEN(#QUOTED_DATA)-5)
SELECT #NOT_EXISTS=STUFF
(
(
SELECT ' ['+ COLUMN_NAME +']='' + ', 'ISNULL('''''''' + REPLACE('+COLUMN_NAME+','''''''','''''''''''') + '''''''','''+'NULL'''+''+')+'' AND '
FROM information_schema.columns
WHERE table_name = #TABLE_NAME AND
ordinal_position BETWEEN #FIRST_COL AND #LAST_COL
FOR XML PATH('')
),1,1,''
)
SELECT #NOT_EXISTS=SUBSTRING(#NOT_EXISTS,1,LEN(#NOT_EXISTS)-6)
SELECT #SQL_KOD='SELECT ''IF NOT EXISTS(SELECT 1 FROM ' + #TABLE_NAME + ' WHERE ' + #NOT_EXISTS + ' + ' + ''') BEGIN INSERT INTO '+#TABLE_NAME+'('+#CSV_COLUMN+')VALUES('''+'+'+#QUOTED_DATA+'+'+''') END '''+' Insert_Scripts FROM ' + #TABLE_NAME + ' ' + #FILTER_CONDITION
EXECUTE (#SQL_KOD)
SET NOCOUNT OFF
I'm trying to group my sql table together, but I am receiving an "incorrect syntax"
Declare #pet varchar(max)
SET #pet = (select petsName from pets);
select * from pets
AS basedata
Pivot (
count(PetID)
for petsName in (' + #pet+ ') <-- error here
)
as pivottable
Why am I receiving an incorrect syntax near #pet?
Thanks
You need to use dynamic sql for this
DECLARE #pet VARCHAR(max)
SET #pet = (SELECT ',' + Quotename(petsName) -- Quotename is used to escape illegal characters
FROM pets
FOR xml path('')); -- to concatenate the records
SET #pet = Stuff(#pet, 1, 1, '') -- to remove the leading comma
DECLARE #sql NVARCHAR(8000) = ''
SET #sql = '
select * from pets AS basedata
Pivot ( count(PetID)
for petsName in (' + #pet + ') ) as pivottable'
exec sp_executesql #sql -- to execute the dynamically framed string
Another mistake in your query is, you are trying to assign petsName to #pet variable.
SET #pet = (select petsName from pets);
But a variable can store only one record. So you need to concatenate the records into one single record separated by comma. Then the variable can be used in pivot list
SET #pet = (SELECT ',' + Quotename(petsName)
FROM pets
FOR xml path(''));
I have a tsql dynamic pivot table query that works fine although I´m not happy with the column names in the final output.
To make it work, the user has to select up to 20 fruit names from a list of 200 fruit names. Then I build the pivot table so I end up with different column names everytime the select is run.
For example:
First time the column names are: apple, orange and pear
Second time is: .orange, banana, kiwi and apple
My question is: Is there a posibility to have static names, like for example: name of the first column always "col_1", second column "col_2" etc?
The select statement is as follows:
DECLARE #idList varchar(800)
DECLARE #sql nvarchar(max)
SELECT #idList = coalesce(#idList + ', ', '') + '['+ltrim(rtrim(id_producto)) +']'
from gestor_val_pos
group by id_producto order by id_producto
SELECT #sql = 'select * from #correlaciones pivot (max (correl)
for codigo2 in (' + #IDlist + ')) AS pvt order by codigo1;'
exec sp_executeSQL #sql
sure.. just created a new variable to hold the column aliases and Row_Number to get the column number.
DECLARE #idList varchar(800)
DECLARE #idListAlias varchar(800)
DECLARE #sql nvarchar(max)
SELECT
#idList = coalesce(#idList + ', ', '') + '['+ltrim(rtrim(id_producto)) +']',
#idListAlias = coalesce(#idListAlias + ', ', '') + '['+ltrim(rtrim(id_producto)) +'] as col_' + CONVERT(VARCHAR(10), ROW_NUMBER() OVER(ORDER BY id_producto))
from gestor_val_pos
group by id_producto order by id_producto
SELECT #sql = 'select ' + #idListAlias + ' from #correlaciones pivot (max (correl)
for codigo2 in (' + #IDlist + ')) AS pvt order by codigo1;'
exec sp_executeSQL #sql
Yes, but it'll make your query significantly more complex.
You'll need to return a list of the possible column names generated from #IDList and convert that into a SELECT clause more sophisticated than your current SELECT *.
When you've got that, use some SQL string splitting code to convert the #IDList into a table of items with a position parameter. Append AS <whatever> onto the end of any you want and use the FOR XML PATH trick to flatten it back, and you've got a SELECT clause that'll do what you want. But, as I said, your code is now significantly more complicated.
As an aside - I really hope that #idList is either completely impossible for any user input to ever reach or hugely simplified from your real code for this demonstration. Otherwise you've got a big SQL injection hole right there.