Paramaterising SQL in SSIS - sql

I'm trying to paramaterize some queries in SSIS. After some reading, it sounds like my best option is to create one variable that contains my base sql, another that contains my criteria and a final variable that is evaluated as an expression that includes both of these. I want to end up with an SQL query that is effectively
UPDATE mytable set something='bar' where something_else='foo'
So my first two variables have the scope of my package and are as follows:
Name: BaseSQL
Data Type: String
Value: UPDATE mytable set something = 'bar' where something_else =
Name: MyVariable
Data Type: String
Value: foo
My third variable has a scope of the data flow task where I want to use this SQL and is as follows:
Name: SQLQuery
Data Type: String
Value: #[User::BaseSQL] + "'" + #[User::MyVariable] + "'"
EvaluateAsExpression: True
In the OLE DB Source, I then choose my connection and 'SQL command from variable' and select User::SQLQuery from the dropdown box. The Variable Value window then displays the following:
#[User::BaseSQL] + "'" + #[User::MyVariable] + "'"
This is as desired, and would provide the output I want from my DB.
The variable name dropdown also contains User::BaseSQL and User::MyVariable so I believe that my namespaces are correct.
However, when I then click preview, I get the following error when configuring an OLE DB Source (using SQL command from variable):
TITLE: Microsoft Visual Studio
Error at Set runtime in DB [Set runtime in myDb DB [1]]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80040E14 Description: "Statement(s) could not be prepared.".
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80040E14 Description: "Must declare the scalar variable "#".".
(Microsoft Visual Studio)
Can anyone advise what I'm missing or how I can resolve this please ?
Thanks in advance!

I had a similar issue.
It seems to be that the designer wants something at design time.
I was getting the error when i tried to click OK on the oleDBSource task after choosing a variable name.
I had a similar situation where the where clause logic was a bit involved and required a script task to assign its value.
This is what things looked like at design time.
#basesql = "select from mytable where "
#where = empty string
#fullsql = #basesql + #where
i was getting the E14 error you mentioned above.
by changing the design time value of #where to something silly, the oleDBSource task got what it wanted and let me use the variable.
#where = " value1 is null and value2 = 'easteregg' "
Related issue, to even get the thing to work, i had to start out by putting a basic select stmt into the sql command so it could go get source columns.
It makes sense, but that step took me and google a while to figure out.

You should set the Expression property of SQLQuery, instead of the Value property. If you do it correctly, the Variable Value window should show you the result of the expression:
UPDATE mytable set something = 'bar' where something_else = 'foo'

I would create a script task and concatenate the strings into one variable and use that as your source:
Dts.Variables["SQLQuery"].Value = Dts.Variables["BaseSQL"].Value.ToString() +
Dts.Variables["MyVariable"].Value.ToString();
Then, in the advanced properties under the component properties tab, you need to change the ValidateExternalMetadad property to False so SSIS will not try to prevalidate your sql query in your variable which would give you a run time error. Just created a simple package and that seemed to work. Hope this helps.
-Ryan

Related

Unable to pass a variable value to a stored procedure in ssis

When executing an SSIS package, the following error appears:
[OLE DB Source [83]] Error: The SQL command requires a parameter named
"#Sales_person", which is not found in the parameter mapping.
[SSIS.Pipeline] Error: OLE DB Source failed the pre-execute phase and
returned error code 0xC0207014.
Below is the screenshot of my OLE DB Source editor
I do see Param direction tab in Set Query parameters, how is that used? In my case will I be using Input or Output or InputOutput
After searching i didn't find a solution for this issue that worked for me. Ther are many suggestions like adding SET NOCOUNT ON before the execute command. Below some related links:
http://geekswithblogs.net/stun/archive/2009/03/05/mapping-stored-procedure-parameters-in-ssis-ole-db-source-editor.aspx
http://www.sqlservercentral.com/articles/Data+Flow+Task+(SSIS)/117370/
http://www.ssistalk.com/2007/10/10/ssis-stored-procedures-and-the-ole-db-source/
You can do a workaround
Declare a SSIS variable (assuming #[User::Query])
Set #[User::Query] property EvaluateAsExpression = True and use the following expression
"EXEC [dbo].[GetDales_Person_data] " + #[User::Sales]
if #[User::Sales] is a string use the following
"EXEC [dbo].[GetDales_Person_data] '" + #[User::Sales] + "'"
Then In OLEDB Source use SQL Command from variable and select #[User::Query]
The problem is with the mapping. See image and source below to make corrections, you should set it to:
Exec [dbo].[GetSales_Person_Data] #Sales_person = ?
Source - geek with blogs
There seems to be a problem with mapping parameters directly into the EXEC statement.
One way you can get around this is to use:
DECLARE #SalesPerson int = ? -- or whatever datatype
EXEC [dbo].[GetSales_Person_Data] #SalesPerson

Dynamically Passing Table Names - using SQL Command from Variable

I read this thread: SSIS : Dynamically passing Table names and this is exactly what I'm trying to achieve. I got to the point of creating the string variable.
However, in the OLEdb source editor, when I chose to use SQL Command from Variable, the preview gave the following error prompt:
Exception from HRESULT: 0xC0202009
Error at Data Flow Task [OLE DB Source [37]]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 11.0" Hresult: 0x80040E14 Description: "Statement(s) could not be prepared.".
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 11.0" Hresult: 0x80040E14 Description: "Must declare the table variable "#".".
Appreciate some help on what could be the issue here.
Below are the components that I have set up.
EXECUTE SQL TASK:
SELECT name from sys.tables WHERE name like'%$Company Information'Order by name
Result Set: Variable Name = User::IndivTableName; Result Name = 0; Data Type=Object
For Each Loop Container:
Enumerator: Foreach ADO Enumerator
ADO Object source variable: User::IndivTableName
Enumeration mode: Rows in the first table
(Is this needed?) Variable mappings: Variable = User::IndivTable; Index=0; Data Type = Object
String Variable:
Variable Name: User::CompanyInformation
Expression:
"select [name],[address],[address 2],[city],[phone no_],[fax no_],
[VAT Registration no_],[registration no_],[business type] from " + "#[User::IndivTableName] "
Evaluated Value:
select [name],[address],[address 2],[city],[phone no_],[fax no_],
[VAT Registration no_],[registration no_],[business type] from #[User::IndivTableName]
Data Source Editor:
Data Access Mode: SQL command from variable
Variable Name: User::CompanyInformation
Variable Value:
select [name],[address],[address 2],[city],[phone no_],[fax no_],
[VAT Registration no_],[registration no_],[business type] from #[User::IndivTableName]
Variable mappings at ForEach ADO Enumerator are mandatory, your mappings are ok. Please change datatype of User::IndivTable variable to string.
Change Variable 'User::CompanyInformation' expression to
"select [name],[address],[address 2],[city],[phone no_],[fax no_],
[VAT Registration no_],[registration no_],[business type] from " + [User::IndivTable]
Variable User::IndivTableName contains DataTable, so it cannot be used in expression. You iterate over it and store the first column into User::IndivTable variable.

Invalid operation result set is closed errorcode 4470 sqlstate null - DB2 data extract

I am running a very simple query and trying to extract the results to a text file. The entire query is essentially what is below, I am selecting everything from one single table with one piece of where criteria which is limiting the data to one month's worth. After it has extracted around 1.2 gig this error shows up. Is there any way that I can work around this other than extracting smaller date ranges? I am trying to pull a couple of years worth of data so if I can only get it a few days at a time it will take a lot of manual work.
I am currently using the free trial of a DB2 query tool - Razor SQL if that makes a difference, I can probably purchase different software if it would help. I am trying to get IBM's tool but for some reason it freezes during the download so I am still working on that. I have searched about this error but everything I see seems much more complex than what I am doing and I can't tell if it applies or not. Thanks in advance.
select *
from MyTable
where date_col between date '2014-01-01' and date '2014-01-31'
I stumbled at this error too, found out it is related to db2jcc.jar (type 4) driver.
Excerpt: If there are no items in the result set left (or to begin with), the Result set is closed automatically and therefore the Exception. Suggestion is to handle it in the application, perhaps in my case, I started checking if(rs.next()) but otherwise, there is a work around. Check out the source link below for how you can set some properties to Data source and avoid exception.
Source :
"Invalid operation: result set is closed" error with Data Server Driver for JDBC
In my case, i missed some properties in WAS, after add allowNextOnExhaustedResultSet the issue is fixed.
1.Log in to the WebSphere Application Server administration console.
2.Select Resources > JDBC > Data sources > Application Center DataSource name > Custom properties and click New.
3.In the Name field, enter allowNextOnExhaustedResultSet.
4.In the Value field, type 1.
5.Change the type to java.lang.Integer.
6.Click OK.
Sometimes you need also check whether resultSetHoldability properties exists. Details refer to here.
I encountered this failure also when ugrading from JDBC Type 2 driver (db2java.zip) JDBC type 4 driver (db2jcc4.jar)
Statement statement = results.getStatement();
if (statement != null)
{
connection = statement.getConnection(); // ** failed here
statement.close();
}
Solution was to check if the statement is closed or not as follows.
Changed to:
Statement statement = results.getStatement();
if (statement != null && !statement.isClosed()) {
{
connection = statement.getConnection();
statement.close();
}
Creating property bellow with type Integer it's worked for me:
allowNextOnExhaustedResultSet:
I had the same issue on WAS 7 so i had to add and change few this on Admin Console.
This TeamWorksRuntimeException exception should be fixed by applying APAR JR50863 which is available on top of BPM V8.5.5 or included on BPM V8.5 refresh pack 6.
For the case that the APAR does not solve the problem, try following workaround:
Log in to the WebSphere Application Server admin console
Select Resources > JDBC > Data sources > DataSource name (TeamWorksDB) > Custom properties and click New
In the Name field, enter downgradeHoldCursorsUnderXa
In the Value field, type true
Change the type to java.lang.Boolean
Click OK to save your changes
Select custom property resultSetHoldability
In the Value field, type 1
Click OK to save your changes
Source of the Answer : https://developer.ibm.com/answers/questions/194821/invalid-operation-result-set-is-closed-errorcode-4/
Restarting the app may fix the problem if connection pool lost session to Db2. If using Tomcat then connection pool property of 'testonBorrow' may reestablish the connection to Db2.

SSIS Execute SQL Task Error using Parameters

SSIS 2012....I want to post a success message to a SQL table in the same DB where I'm working. I created a variable called "InputRowCount" as Int32, and set it in the RowCount transform. Then, in a SQL Task, I set it as an input parameter, incidentally giving it a name. My DB connection is OLEDB, so the book says I just use a "?" to reference the lone Parameter (I expect to insert more parameters later....) It gets a "Parameter name is unrecognized" error. Here's the SQL Statement:
[Edit...now I get that the parameter type cannot be deduced...]
[Execute SQL Task] Error: Executing the query "INSERT INTO dbo.tblpendingmessages
([Recipi..." failed with the following error: "The parameter type for '#P1' cannot be uniquely deduced; two possibilities are 'float' and 'datetimeoffset(7)'.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
INSERT INTO dbo.tblpendingmessages
([Recipient]
,[MessageSubject]
,[MessageBody]
,[MessageGenerated]
,[MessageUser]
,[MessageFormat])
VALUES
(N'CPAS_Administrator#xxxxxx.COM' -- Recipient
,N'All Projects Import' -- Subject Line
,N'ALL Projects Import has replaced ' + FORMAT(?,'N','en-us') + N' rows of the AllProjects Table in CPAS.' -- MessageBody
,GETDATE() -- MessageGenerated
,N'SSIS'
,N'TEXT')

SQL Server OLEDB error with no details

I have a SQL Server Agent job running, which uses a stored procedure to do several operations, then exports some data to an xls spreadsheet and emails that spreadsheet. Most of the time it works, but several times a month the job fails with the error:
OLE DB provider 'Microsoft.Jet.OLEDB.4.0' reported an error. The provider did not give any information about the error. [SQLSTATE 42000] (Error 7399). The step failed.
Thanks, Microsoft, for the detailed error message. Anyway, the short term fix is usually to simply re-run the job. Usually this works, but in rarer cases it does not, and I must restart the SQL Server instance.
Here is how my code interacts with OLEDB:
Insert into OPENROWSET('Microsoft.Jet.OLEDB.4.0', 'Excel 5.0;Database=\\Excel\POStatus\POStatus.xls;',
'SELECT * FROM [POStatus$]')
Select --Tons of columns with tons of math and functions
FROM --5 tables joined together (left joins)
WHERE -- Tons of where conditions
Order by --Case statement for custom sorting
Set #vCommand = 'copy \\Excel\POStatus\POStatus.xls \\Excel\POStatus\POStatus_' + #vDate + '.xls'
EXEC master..xp_cmdshell #vCommand , NO_OUTPUT
... omitted for brevity...
Set #nvSubject = ' POStatus ' + #vDate
Set #nvMessage = ' This is an automated message, please respond to the IS department, thank you '
Set #nvMessage = #nvMessage + char(13) + char(10)
Set #nvAttachments = '\\Excel\POStatus\POStatus_' + #vDate + '.xls'
Exec master..xp_sendmail
#recipients = #nvRecipients , #copy_recipients = #nvCopy_recipients ,
#subject = #nvSubject , #message = #nvMessage ,
#query = #nvQuery , #width = #iWidth , #attachments = #nvAttachments
So, what is the cause of this, and how can I prevent it?
When you call OPENROWSET, it load the DLL for OLEDB provider for Excel. These operations happens inside the SQL Server Stack memory. When a lot of call to older DLL (ActiveX/COM) are made, it's not impossible that your stack might get full. You can resolve this in 2 ways:
1) You make these operations outside a TSQL code. You can use a SSIS package for example but you have to change your code not to use OPENROWSET. You can use the DTS Wizard to do most of the job. This is my personnal recommandation !
2) You can also set SQL Server to have a bigger memory Stack by using the -g command parameter. I think 256 MB is the default and you can set it to 512. To do that you need to open the SQL Server Configuration Manager. Depending on your version, you should have a place to change the "Startup Parameters"