I have an SSIS solution where depending on a parameter, it launches the extraction of different databases, each in a different file, and the name must contain the date of the extraction so we can't hard code the name in the Destination wizard, I must use a variable where I concatenate the date to a file name, and I have to use that variable as the file name. But the destination wizard only seems to let me write plain text.
Any idea how I can do this ?
Select the connection manager in the Connection managers bar. Go to the Properties Pane and find the "expressions" option. Click the ellipsis in the adjacent blank window to open a new window.
Find "File Name" or "Connection String" in the drop down, and then in the expression, put the name of your variable.
If the file may not exist before the package is run, ensure you enable delay validation on the connection manager too.
Related
I want to ask from users to load an excel file as an input data. this process must be done by opening a browsing window to select excel file.
what can I do?
You can use open_dialog_native from Gtk.jl.
julia> open_dialog_native("Choose the input Excel file", GtkNullContainer(), ("*.xlsx",))
"/path/to/myfile.xlsx"
It opens the file chooser interface appropriate to the user's OS, and once the file is chosen, returns the chosen file's full path as a string.
The ("*.xlsx",) is a tuple that constrains what type of files are shown by default in the file chooser. (The GtkNullContainer() argument just specifies that you're not running this as part of an existing GTK app.)
Documentation here
I tried to create a SSIS package to input excel file into database.
The excel file has dynamic name, and also i need to add filename into the database as well.
What I'm trying to do:
use for each file loop to set a variable called FileFound, with the value of
the filename (with full path)
set connection for the input file with above variable
use data load to load the excel to db, with above connection.
I got error saying the connection is invalid.
enter image description here
enter image description here
#Admin,
I tried to upload 2 images, and I got 2 image urls but they are not working.
can you please check?
https://i.stack.imgur.com/THtMd.jpg
https://i.stack.imgur.com/zApQf.jpg
There is step you missed when using dynamic file names in a foreach loop. You need to set delay validation property of the dynamic source (in your case the Excel Conn) to TRUE.
The package tries to validate in pre-execute phase unless you delay it.
I am setting up a For Each File Loop to upload all the text files in a folder and create a DB item with the same name. This however is causing me to get the above error. As you can see from the two screenshots below, I have (I think) set the Connection Manager for the OLE DB Destination object up correctly, but the error remains. Does anyone know why I am getting this and where I am supposed to specify a table name if a declared variable is being used for the looping?
In your variable definition (User::FileFound), you have to specify the sheet name in the Excel workbook you used to set up your Excel Connection Manager (e.g. 'Sheet1$').
You're presumably changing the value of the variable at some point in your For Each File loop process, which makes sense, but at design time, you have to specify a real sheet name.
In your Control Flow, click on your data Flow Component
On properties, change the 'DelayValidation' to 'True'
I am new to Pentaho kettle and have a requirement where we want to unzip a set of files and the paths to those files are there in a table. I am wondering how to go about it.
This should be your main Job:
First transformation connects to your database and extract the paths, after this another Job is called (Unzip) which extracts those files. I'll be more clear, the next is the transformation called "Table input":
Use the "Table input" step to connect to your database. When you open it you have to create a new connecion and then put your query in the canvas. (make a query which extract values just from the interested column, not every columns). The step "Copy rows to result" gives the values form the database to the next job.
The following is the job "Unzip":
This job receives the values from the previous transformation and pass tose to the "Unzip file" job entry.
Things to know:
1) In the main job double click on the Unzip job icon, go to "advanced" and specify "Copy previous result to parameters" and "Execute for every input row". Of course in the Job specification you have to specify the path of this job.
2) Also double click on the Unzip job icon, go to parameters and put a parameter named as the value which you extract from the database:
3) Enter in the sub-Job (Unzip in my case) and right click, then go to "Job settings" and then to "parameters". Now put the same parameter name as before:
4) Remember to set the destination folder of the files and the receving parameters in the "Unzip files" job entry:
I am getting the output in flat file as shown below.
How do I avoid column names from being written to the flat file output?
The best way that I've found to accomplish this is the following:
I create a test output flat file. I populate the .txt file with the result set from the SQL query that will be used in the package. Make sure that the first row of the .txt file contains the column header names.
Create a Flat File Connection. Point it to the text output flat file. Check the "Column names in the first data row" box. This will make sure that the actual column header names will be used/shown, instead of "Column 0", "Column 1", etc. Click "OK" to close the Flat File Connection Manager.
Edit the Flat File Destination item to make sure that the actual column header names are picked up and that the fields are mapped properly between your query and the output columns of your Flat File Connection. Click "OK".
Highlight the Flat File Connection and press "F4" to open the "Properties" window. There, change the "ColumnNamesInFirstDataRow" to "False".
Now your output flat file will contain data only...no column header row. Yet you can still go to the Flat File Destination item and see the actual column header names being used there.
Avoiding column names in the flat file output:
Double-click the Flat File Connection Manager to open the Flat File Connection Manager Editor
On the Flat File Connection Manager Editor, uncheck the box Column names in the first data row.
This will prevent flat file connection manager from writing the column names to the flat file output.
If you already have header in the file - you can simply skip the first row. See screenshot from #user756519 answer.
Set Header rows to skip to 1