I'm running into an issue trying to pass data into a child package.
My parent package runs through a db, checks it against another in order to run or not. Currently i'm using a ForEach Loop in order to do so. I then can identify the row i need to run by one unique id. Then i pass it to the appropriate child package to process it.
My question is, what's the easiest method of writing that identifier to a 4 byte variable so that it is available on a solution level, instead of package level. I can't pass anything through to the child package as far as i have seen either.
Thanks for assist!
-D
Parameters are the thing you will be looking for in an SSIS 2012, Project Deployment Model, to allow you to pass data to the child package. In the child package, you will need to add a Parameter.
I created 5 packages: MasterPackage and ChildPackages_0 ... ChildPackage4
MasterPackage
MasterPackage has a 2 variables defined: "Variable" (Int32) and "ChildPackage". ChildPackage (String) has an expression applied "ChildPackage_" + (dt_wstr,1) #[User::Variable] + ".dtsx" As the value of Variable changes, so does this string.
Add a Foreach Loop (Item enumerator, goes from 0 to 4) and I map the current value to my poorly named variable "Variable."
I have an Execute Package Task defined that runs a package. In the Parameter Bindings tab, I mapped the child parameter named "Parameter" to the variable "User::Variable" I also applied an expression on the PackageName property to be the variable #[User::ChildPackage]. This results in the name of the package changing to reflect the value of the variable.
ChildPackage_X
In my child packages, I created a single parameter named "Parameter", assigned a default value of -1 and checked the Required attribute. The child package simply has a Script task that emits the value of the parameter. I added System::PackageName, $Package::Parameter to the list of Read Only parameters for that script. That code follows
public void Main()
{
// System::PackageName, $Package::Parameter
bool fireAgain = true;
string myVariableName = "$Package::Parameter";
int myVariableValue = -1;
string packageName = Dts.Variables["System::PackageName"].Value.ToString();
myVariableValue = (int)Dts.Variables[myVariableName].Value;
Dts.Events.FireInformation(0, string.Format("{0} Script task", packageName), string.Format("{0}:{1}", myVariableName, myVariableValue), string.Empty, 0, ref fireAgain);
Dts.TaskResult = (int)ScriptResults.Success;
}
Output
After running the master package, I received the following expected output. Each child package fired when it was their turn and the script task showed that the value was passed to them via their parameter as expected.
SSIS package "C:\sandbox\SO_SSIS\SO_SSIS\MasterPackage.dtsx" starting.
Executing ExecutePackageTask: C:\sandbox\SO_SSIS\SO_SSIS\ChildPackage_0.dtsx
Information: 0x0 at SCR Emit parameter, ChildPackage_0 Script task: $Package::Parameter:0
Executing ExecutePackageTask: C:\sandbox\SO_SSIS\SO_SSIS\ChildPackage_1.dtsx
Information: 0x0 at SCR Emit parameter, ChildPackage_1 Script task: $Package::Parameter:1
Executing ExecutePackageTask: C:\sandbox\SO_SSIS\SO_SSIS\ChildPackage_2.dtsx
Information: 0x0 at SCR Emit parameter, ChildPackage_2 Script task: $Package::Parameter:2
Executing ExecutePackageTask: C:\sandbox\SO_SSIS\SO_SSIS\ChildPackage_3.dtsx
Information: 0x0 at SCR Emit parameter, ChildPackage_3 Script task: $Package::Parameter:3
Executing ExecutePackageTask: C:\sandbox\SO_SSIS\SO_SSIS\ChildPackage_4.dtsx
Information: 0x0 at SCR Emit parameter, ChildPackage_4 Script task: $Package::Parameter:4
SSIS package "C:\sandbox\SO_SSIS\SO_SSIS\MasterPackage.dtsx" finished: Success.
Related
I've managed to get a job parameter context on a Tasklet but I didn't figure out how to change this value so the next Tasklet can access a modified value
val params = JobParametersBuilder()
.addString("transaction", UUID.randomUUID().toString())
.addDouble("amount", 0.0)
jobLauncher.run(
paymentPipelineJob,
params.toJobParameters()
)
First task:
override fun beforeStep(stepExecution: StepExecution) {
logger.info("[$javaClass] - task initialized.")
this.amount = stepExecution.jobParameters.getDouble("amount")
// Prints 0.0
logger.info("before step: ${this.amount}")
}
override fun afterStep(stepExecution: StepExecution): ExitStatus? {
// Change the execution content to pass it to the next step
stepExecution.jobExecution.executionContext.put("amount", this.amount!! + 3)
// Still prints 0.0
logger.info("after step: ${stepExecution.jobParameters.getDouble("amount")}")
logger.info("[$javaClass] - task ended.")
return ExitStatus.COMPLETED
}
How can I modify a job parameter so all the steps can access it?
While the execution context is a mutable object, job parameters are immutable. Therefore, it is not possible to modify them once the execution is launched.
That said, according to the code you shared, you are putting an attribute amount in the job execution context and expecting to see the modified value from the job parameters instance. This is a wrong expectation. The execution context and the job parameters are two distinct objects and are not "inter-connected".
Edit: Add suggestion about how to address the use case
You can use the execution context to share data between steps. You are already doing this in your example:
stepExecution.jobExecution.executionContext.put("amount", this.amount!! + 3)
Once that is done in a listener after step 1, you can get the value from the EC in a listener before step 2:
double amount = stepExecution.jobExecution.executionContext.get("amount");
Please check Passing Data to Future Steps from the reference documentation.
I have an SSIS package with for each loop > sequence container. The sequence container is trying to read file from For each loop and process its data. The requirement was to not fail the entire package when any exception happened in processing a file but to continue processing the next file until all the files were processed from the for each loop. For this, I have set the Propagate variable for the sequence container to False. I have also added email step on On Error event of Sequence container. The package is running as expected and able to process all files even when any exception happened with any file. But I would like the status of my SSIS package to be failed finally since one of the files got failed. How can I achieve that ?
Did you try this options?
(SSIS version in russian on the left side but it's sequence container)
View -> Properties window -> Then click on your sequence container and it will show you ther properties of sequence container.
If i were you first of all i would try property "FailPackageOnFailture" - it should cover your question if i get it right.
P.S. Also you can see the whole properties of your project when you click on a free place in your project
UPDATED (after comments and more clear understanding task):
The idea is - set this param Maximum ErrorCount for SQ as max as you want - in this case it wont stop the package because 1 of the files was failed in SQ and next file will process, but it should stop package after SQ will finish his work because you don't change MaximumErrorCount for package.
Important - a value of zero sets the error count threshold to infinity and package or task never get's Failure
I am trying to pass Package variable: packno from Execute SQL Task to Data flow.
Variable is filled with needed value on OnProgress event. The value is:20717.
But on OnPostExecute event this value is cleared and assigned: -1.
So SQL command is executed with -1 value.
Any idea why this is happening?
There is a problem with the server. Everything worked fine on different one.
I have a SSIS package which calls a number of execute package tasks which perform ETL operations
Is there a way to configure the Execute package tasks so that they retry a defined number of times (currently, on the failure of one of the tasks in the child package, the execute package task fails. When this happens, I would like the task to be retried before giving up and failing the parent package)
One solution I know of is to set a flag for each package in the database, set it to a defined value on success and call each package in a for loop container till the flag is successful or the count exceeds a predefined retry count.
Is there a cleaner or more generic way to do this?
Yes, put Execute Package Task in a For Loop Container. Define a variable, which will do the count, one for a successrun indicator and a MAX_COUNT Constance. In properties of the Package Task - Expressions, define
FailPackageOnFailure - False
After Execute Task put a Script Task Read/Write Vars: SuccessfulRun, script:
Dts.Variables["SuccessfulRun"].Value = 1
In properties of the For Loop:
InitExpression - #Val_Counter = 0
EvalExpression - #Counter < #MAX_COUNT && #SuccessfulRun == 0
AssingExpression - #Val_Counter = #Val_Counter + 1
Connect PackageTask with ScriptTask using Success line.
OR
In For Loop Container define expression
MaximumErrorCount - Const_MAX_COUNT
But this one hasn't been tested by me yet...
If I use OnError event handler in my SSIS package, there are variables System::ErrorCode and System::ErrorDescription from which I can get the error information if any things fails while execution.
But I cant the find the same for OnTaskFailed event handler, i.e. How to get the ErrorCode and ErrorDescription from the OnTaskFailed event handler when any things fails while execution in case we want to only implement OnTaskFailed event handler for our package?
This might be helpful, it's a list of all system variables and when they are available.
http://msdn.microsoft.com/en-us/library/ms141788.aspx
I've just run into the same issue, and I've worked around it by :
Creating a #[ErrorCache] variable
In my case the task was being retried multiple times, so I needed an Expression task to reset the #[ErrorCache] variable at the beginning of each retry
Create an OnError event handler, which contains an Expression task purely to append the #[ErrorMessage] to the #[ErrorCache]
Create an OnTaskFailed event handler which then utilises the #[ErrorCache]
Go to the event handler of the task you want to monitor for errors and click on the link to create a new handler.Then create a task like Send Mail and create 2 variables: mail_header and mail_body.
IMPORTANT: Move the variables from the current scope to the OnError scope otherwise the values won't be available when processing the package.
Define the mail_subject variable as string and set the expression as: "Error " + #[System::TaskName] + " when executing " + #[System::PackageName] + " package."
Define the mail_body variable as string and set the expression as: REPLACENULL( #[System::ErrorDescription],"" ) + "\nNotify your system administrator."
On the task editor, create an expression assigning Subject to the #mail_subject variable. Define the MessageSourceType as a variable and set MessageSource to the #mail_body.
In the task that you put on the error event handler you can select parameters that are only available in an error handler such as system:Errordescription or system:Sourcename (which provideds the task that it failed on). We use these as input variables to a stored proc which inserts into an error table (and to send an email for a failed process) that stores other information beyond just the logging table. We also use the logging table to log our steps and in clude on error in that so general error information goes there.