I've always thought that looping over sql files in directory in SSIS is easy... But I've got a problem today:
Execute SQL Task isn't executing statements that are in the sql file.
In the sql file I've got delete statement and then insert statement.
SSIS Execute SQL Task component is done after about 2 sec, while executing the same script manually takes usually about 2 minutes, and of course in SSIS it doesn't insert anything.
I checked variable value that is coming from Foreach loop (with full filemane path) and it is ok.
I've got parametrized (by Expression) File Connection with this variable.
What am I doing wrong? Thanks for help.
Do you have GO after each call within your SQL file?
Example:
// Your insert TSQL code here
GO
// Your delete TSQL code here
GO
// Etc...
It will not continue if you don't have this.
Related
I have simple SQL Query and need to get the input from parameter. I am getting the when i click parameter button. Please let me know how to get the parameter if i have declare statement in my scripts.
Note :OLEDB connection manager used for connection
You can't use parameters in an OLEDB Source if your command contains DECLARE, or anything other than a single SELECT statement.
The way to do what you want is to use build your entire SQL Command into a single SSIS variable, and use the "SQL Command from variable" option in the OLEDB source.
EDIT based on comments:
If you have to handle a SQL Command over 4000 characters, the only way to do it is in a Script Component. The SQL Command in a script component can be any length. Here is an example.
I execute a SQL query (for PostgreSQL) via psql.exe inside a Windows batch. I get an error I can't explain, saying that a FROM clause is missing for a table that is not called within the query (see below). When I search in the batch file for geo_c3_0_3_mo table, the string is not found...
Any idea on this kind of issue?
EDIT :
If I copy-paste the query from the batch file into a pgAdminIII SQL query window, the query runs perfectly and no error message is returned.
When I remove one of the subqueries, the error either disappear or mention another badly written table name (for instance: missing FROM-clause for table "geoc__0_3_mo")... It seems more and more that the issue comes from the length of the line (19,413 characters!). To me, it is not possible to write the query on several lines within a batch file, like inside a pgAdminIII SQL query window. The solution would be to keep the query inside a *.sql file and to call that file from the batch file.
Write the query to a tempfile in your batch, then execute it with psql -f. This will bypass command-line length issues.
Using an SSIS package, is it possible to load sql code that contains variables in it, replace those variables and run the code without saving it(similarly to DTS 2000 packages)?
For example, I want to run Code.sql, which contains the following:
SELECT *
INTO XXDBXX.dbo.Table2
FROM XXREADDBXX.dbo.Table1
I was planning on creating a script to replace the "XX" strings with variables I set up using SSIS.
So, here are the steps I want to do:
Run script to replace sql file variables.
Run SQL code
SQL code finishes running, and code still retains its variables.
Please let me know if anything needs clarification.
create a variable in SSIS #SQL (User::SQL)
Script task to load template sql from file & replace variables with required values. Use C# String.Replace();
Execute task to use #SQL (SQLSourceType = Variable)
Create a variable in SSIS
https://www.red-gate.com/simple-talk/sql/ssis/ssis-basics-introducing-variables/
You can also run SQL code and then replace parameters such as:
EXEC PROC XXDBX ?,?
The question marks being your parameter or variable created above.
I have a pair of linked SQL servers: ServerA and ServerB. I want to write a simple INSERT INTO SELECT statement which will copy a row from ServerA's database to ServerB's database. ServerB's database was copied directly from ServerA's, and so they should have the exact same basic structure (same column names, etc.)
The problem is that when I try to execute the following statement:
INSERT INTO [ServerB].[data_collection].[dbo].[table1]
SELECT * FROM [ServerA].[data_collection].[dbo].[table1]
I get the following error:
Msg 16902, Level 16, State 48, Line 1
sp_cursor: The value of the parameter 'value' is invalid.
On the other hand, if I try to execute the following statement:
INSERT INTO [ServerB].[data_collection].[dbo].[table1] (Time)
SELECT Time FROM [ServerA].[data_collection].[dbo].[table1]
The statement works just fine, and the code is executed as expected. The above statement executes just fine, regardless of which or how many tables I specify to insert.
So my question here is why would my INSERT INTO SELECT statement function properly when I explicitly specify which columns to copy, but not when I tell it to copy everything using "*"? My second question would then be: how do I fix the problem?
Googling around to follow up on my initial hunch, I found a source I consider reliable enough to cite in an answer.
The 'value' parameter specified isn't one of your columns, it is the optional argument to sp_cursor that is called implicitly via your INSERT INTO...SELECT.
From SQL Server Central...
I have an ssis package that needs to populate a sql table with data
from a pipe-delimited text file containing 992 (!) columns per record.
...Initially I'd set up the package to contain a data flow task to use
an ole db destination control where the access mode was set to Table
or view mode. For some reason though, when running the package it
would crash, with an error stating the parameter 'value' was not valid
in the sp_cursor procedure. On setting up a trace in profiler to see
what this control actually does it appears it tries to insert the
records using the sp_cursor procedure. Running the same query in SQL
Server Management Studio gives the same result. After much testing and
pulling of hair out, I've found that by replacing the sp_cursor
statement with an insert statement the record populated fine which
suggests that sp_cursor cannot cope when more than a certain number
of parameters are attempted. Not sure of the figure.
Note the common theme here between your situation and the one cited - a bazillion columns.
That same source offers a workaround as well.
I've managed to get round this problem however by setting the access
mode to be "Table or view - fast load". Viewing the trace again
confirms that SSIS attempts this via a "insert bulk" statement which
loads fine.
SSIS 2008. Very simple task. I want to retrieve a System Variable and use it in an SQL INSERT. I want to retrieve the value of System:MachineName and use it in an insert statement.
Using the statement INSERT INTO MYLOG (COL1) SELECT #[System::MachineName] gives the error Error: ..failed to parse. Must declare the scalar variable "#"
Using the statements SELECT #System::MachineName or SELECT ##[System::MachineName] gives the error 'Error Incorrect systax near '::'
I am not trying to pass a parameter to the query. I have searched for a day already but couldn't find how to do this one simple thing!
Here is one way you can do this. The following sample package was created using SSIS 2008 R2 and uses SQL Server 2008 R2 as backend.
Create a sample table in your SQLServer database named dbo.PackageData
Create an SSIS package.
On the SSIS, add an OLE DB connection manager named SQLServer to connect to your database, say to an SQL Server database.
On the Control flow tab, drag and drop an Execute SQL Task
Double-click on the Execute SQL task to bring the Execute SQL Task Editor.
On the General tab of the editor, set the Connection property to your connection manager named SQLServer.
In the property SQLStatement, enter the insert statement INSERT INTO dbo.PackageData (PackageName) VALUES (?)
On the Parameter Mapping tab, click Add button, select the Package variable that you would like to use. Change the data type accordingly. This example is going to insert the PackageName into a table, so the Data Type would be VARCHAR. Set the Parameter Name to 0, which indicates the index value of the parameter. Click OK button.
Execute the package.
You will notice a new record inserted into the table. I retained the package name as Package. That's why the table
Hope that helps.
Per my comment against #ZERO's answer (repeated here as an answer so it isn't overlooked by SSIS newcomers).
The OP's question is pretty much the use case for SSIS property expressions.
To pass SSIS variables into the query string one would concatenate it into an expression set for the SqlStatementSource property:
"INSERT INTO MYLOG (COL1) SELECT " + #[System::MachineName]
This is not to suggest the accepted answer isn't a good pattern, as in general, the parameterised approach is safer (against SQL injection) and faster (on re-use) than direct query string manipulation. But for a system variable (as opposed to a user-entered string) this solution should be safe from SQL injection, and this will be roughly as fast or faster than a parameterised query if re-used (as the machine name isn't changing).
I never use it before but maybe you can check out the use of expression in Execute SQL task for that.
Or just put the whole query into an expression of a variable with evaluateAsExpression set to true. Then use OLE DB to do you insert
Along with #user756519's answer, Depending on your connection string, your variable names and SQLStatementSource Changes