I am designing an ETL with spoon from Pentaho.
My purpose is to execute 'Calculate Value' after 'Load To Database' and 'Update Database'.
What step should be used to replace '??? Step wait' ?
replace ??? step wait with a dummy step.
then after that have a "Block rows on step"
You could move the "Calculate Value" step into a new transformation and create a job to execute this after the first transformation is completed.
Related
I'm running a macro that needs the status date. I am using ActiveProject.CommandBars.ExecuteMso ("StatusDate") and the macro just keeps executing before I set the status date on Project 2016. How can I code to wait for the date to be entered?
I can see how practical it would be to use that command, but there are the consequences you have found.
If you are ready to spend another click on setting the Status Date you can just add a dummy MsgBox dialog after your command:
ActiveProject.CommandBars.ExecuteMso ("StatusDate")
MsgBox "Status Date set"
...or have you found another solution meanwhile?
I have a ssis package which exports data from excel files and dump it into a SQL Table. For processing files, I am using a foreach loop and a dataflow opens an excel source and dumps data into ole db destination. If any file is not containing the required tab, I want the ssis package to log error and move to next iteration. I have tried following things but the package fails:
Propagation = false
ForceExecutionResult = Success
How can I handle this?
Attached images are the screen shots of control flow, data flow and progress.
Try on the For Each loop set maximum error count to 0 to ignore all errors.
In your for each loop have an on error sql script to write out if error so they are not lost.
This question already has answers here:
Wait for Shell to finish, then format cells - synchronously execute a command
(8 answers)
Closed 6 years ago.
I have two batch files. The first one is to download files from SFTP. The second is to manipulate some local files. Since I have to wait until the first one is finished, how to hold the program in VBA so the batch files can be executed one by one?
Right now I am using
Call Shell("Bat1.bat")
Call Shell("Bat2.bat")
However, two batch files will be executed at the same time. Bat2.bat will not wait Bat1.bat.
Thanks
First off, drop the Call statement, it's not needed.
Shell "Bat1.bat"
Make Bat1.bat create an empty "marker" file when it's done, e.g. bat1.ready. Then make your VBA code loop until it can find that .ready file:
While Dir("bat1.ready") = vbNullString
DoEvents
Wend
This should keep your UI/host app responsive, while essentially doing nothing until the marker file is up.
Next you delete the marker file and proceed with bat2:
Kill "bat1.ready"
Shell "Bat2.bat"
Ta-da!
The Shell function returns immediately, and returns the created TaskID. If you can't modify the batch file in any way (or ask whoever can modify it if they could do that), then you'll probably need to use some Win32 calls to use that TaskID to determine whether the task has completed, as in the ShellAndWait link that Tim Williams provided in a comment
The below script is written in 'Winwrap basic' which is almost identical to VBA.
I would like this script to work on SPSS 20, the script works fine on SPSS15 (by changing the file extension from STT to TLO as that is what the tablelook file was back then).
However, whenever I run this script in SPSS 20 the wwb processor crashes with a generic error message 'WWBProcessor has encountered a problem and needs to close. We are sorry for the inconvenience.'
The script is well commented, but the purpose of the script is to change the tablelook of every table in the output viewer window, by activating each table in turn and setting the table look to one specified by the user, rotating the inner column labels, closing the table and activating the next table.
The loop continues until every table has been set to the new tablelook and rotation.
Manually setting the rotation of a few hundred tables is arduous and very time consuming not to mention numbingly boring. This script used to perform this task in seconds back in version 15, but ever evolving needs and lack of support for the older version has meant that I've been forced to use the newer version.
I'd be grateful for any assistance.
Mav
Option Explicit
Sub Main
'BEGIN DESCRIPTION
'This script changes all tabs to the same 'Tablelook' style. You will be prompted to choose the tablelook file.
'END DESCRIPTION
'******************
'Old description
'This script assumes that objSpssApp ist the currently running
'SPSS-Application and assigns every existing Pivot Table
'in the Output Navigator a new TableLook which can be selected
'from a Dialog box. Hidden tables will also be affected.
'Originally Created by SPSS Germany. Author: Arnd Winter.
'******************
'This script is written in the BASIC revision 'WinWrap Basic' code copied from VB or other basic languages may have to be modified to function properly.
On Error GoTo Bye
' Variable Declaration
' For an undertermined reason scripts cannot be executed throught the Utilites -> Run scripts menu,
' Instead they must be opened like a syntax file and ran from the SPSS 19 Scripting page.
' Functionality on SPSS 20 is now completely gone, error message only reads 'WWB processor has encountered a problem and needs to close'.
Dim objOutputDoc As ISpssOutputDoc 'Declares the Output variable
Set objOutputDoc = objSpssApp.GetDesignatedOutputDoc 'Assigns currently active output to Output variable
Dim strAppPath As String
Dim objOutputItems As ISpssItems 'variable defining every item in the current output window
Dim objOutputItem As ISpssItem 'variable defining the current item
Dim objPivotTable As PivotTable
Dim intCount As Integer 'declare the variable that will store the number of instances
Dim varStrLook As String
Set objOutputItems=objOutputDoc.Items
Dim i As Integer 'for loops we need an INT variable that will be counted against the number of instances 'i' is standard notation
' Find out SPSS Directory
strAppPath = objSpssApp.GetSPSSPath
' Select TableLook
'The Parametres you must enter into the GetFilePath() function are as follows:
'(Optional)Firstly you enter the initial file name (if none is required use an asterisk * and the file extention, or *.*)
'(Optional)The second part is the file extention expected, you can choose multiple filetypes if you seperate them with a semi-colon ;
'(Optional)The third parametre is the directory where the file should be opened.(default - Current path)
'The fourth parametre is the Title of the prompt, which should be enclosed in speech marks.
'The Final parametre is the 'Option'
'0 Only allow the user to select a file that exists.
'1 Confirm creation when the user selects a file that does not exist.
'2 Allow the user to select any file whether it exists or not.
'3 Confirm overwrite when the user selects a file that exists.
'+4 Selecting a different directory changes the application's current directory.
'For more detailed information visit the WWB website.
' http://www.winwrap.com/web/basic/language/?p=doc_getfilepath__func.htm
varStrLook = GetFilePath$("*.stt","stt",strAppPath,"Select Tablelook and confirm with Save.",4)
' Tested re-applying the dollar sign, cofusingly removing or adding the Dollar sign ($)
' seems to have no effect.
' If user presses Cancel or selected a file with the wrong file type then exit script
If (Len(varStrLook)= 0) Or (Right(varStrLook,3)<>"stt") Then
Exit Sub
End If
' Loop which assigns a new TableLook to all existing Tables.
intCount = objOutputItems.Count 'Assigns the total number of output items to the count-marker
For i = 0 To intCount-1 'Start loop
Set objOutputItem=objOutputItems.GetItem(i) 'Get current item
If objOutputItem.SPSSType=SPSSPivot Then 'If the item is a pivot table then...
Set objPivotTable=objOutputItem.ActivateTable 'Activate the table for editing
objPivotTable.TableLook = varStrLook 'Apply the earlier selected table look.
objPivotTable.RotateColumnLabels=True 'Rotate collumn lables
objOutputItem.Deactivate 'Confirm changes and deactivate the table
End If
Next 'End loop
'********************************************************
'Updated script from Version 15 ->
'Script now includes inner column label rotation
'Script has been modified and adapted to improve performance
'and to help people who wish to use/adapt the script
'in future endeavours.
'********************************************************
Bye:
End Sub
The first thing to try is to replace the activate/deactivate calls with
GetTableOLEObject
This is much more efficient and does not require the pivot table editor, but you can do all the things that you could do on an activated table.
If you don't have the current fixpack for V20, fixpack2, installing that would be a good idea, too.
Being fairly new to SSIS and the ETL process, I was wondering if there is anyway to loop though a record set within a DataFlowTask and pass each row (deriving parameters from the row) into a Stored Procedure (the next step in the ETL phase). Once i have passed the row into the stored procedure, I want the results from each iteration to be written to a Table.
Does anyone know how to do this?
Thanks.
Any OLEDB command transformation (which is used to execute a database command) in an SSIS dataflow is executed once per input row - which I think is the behaviour you want. More details here.
In your scenario, the minimum you would need would be:
Data Source -> OLEDB Command -> Data Target
Note that this isn't going to give great performance - it might be better to try and refactor your stored procedure to operate on the whole input set in one go (in which case you'd use an Execute SQL task in the control flow).
The following structure would work:
Create an object variable. (recordset_object)
Create an string variable. (record_string)
Create an "Execute SQL Command" in the control flow. The command should return the record set that you want to loop through.
In the "Execute SQL Command", in the General tab set the Result Set = Full result set.
In the "Execute SQL Command", in the Result Set tab set the Result Name = 0 and Variable Name = (recordset_object).
Create a "Foreach Loop Container" and create a precedence constraint between the "Execute SQL Command" and the "Foreach Loop Container".
In the "Foreach Loop Container", in the Collection tab set Enumerator = Foreach ADO Enumerator.
In the "Foreach Loop Container", in the Collection tab set the ADO object source variable = User::recordset_object.
In the "Foreach Loop Container", in the Collection tab set the Enumeration mode = Rows in the first table.
In the "Foreach Loop Container", in the Variable Mappings tab set teh Variable = User::record_string and the Index = 0.
In the "Foreach Loop Container" in the design surface of the Control Flow, add an "Execute SQL Command".
For the child "Execute SQL Command", you can (13) set the SQLStatement to either use a variable that generates the code you want to execute, or (14) map in a parameter, or (15) make the record_string a SQL command that is executed by the code.
If you use a variable, then it could be something like User::sql_code_string and its value could be something like "EXEC schema.some_stored_procedure '" + #[record_string] + "';". You would then set the SQLSourceType in the General tab of the child "Execute SQL Command" = Variable and set the SQLStatement to User::sql_code_string.
If you use a parameter, in the child "Execute SQL Command" in the Parameter Mapping
tab set Variable Name = User::record_string, Direction = Input, Data Type = VARCHAR, Parameter Name = 0, Parameter Size = -1. In the General tab of the child "Execute SQL Command", set the SQLStatement to "EXEC schema.some_stored_procedure ?".
Similar to 13, but instead of creating a separate variable, you can execute User::record_string. This could work if the content of record_string that was returned by your data set is the query you want to execute.
I generally prefer this approach over #Ed's solution you can include additional steps for each record. For instance, I often add in additional objects in my Control Flow like Script Tasks, Data Flows, and Execute SQL Commands. It's a more flexible, easy to understand approach from my perspective, but #Ed's solution definately meets the criteria of your question.
Good luck and let me know if you need clarification on the instructions.