I have seen multiple topics about related issues, but no solution yet.
I am using SSIS to load data from a query into a CSV file. When I configure the package and click preview, everything looks fine, but when opening the CSV file, the four separate values are in one column and not in 4 separate columns with headers.
This is my preview in the Flat File destination (which looks fine):
[preview destination]
This is the configuration of the flat file connection:
[configuration flat file connection]
This is the preview from the connection manager:
[connection manager]
But this is how the results looks like:
[result all in 1 column]
Is there anyone who knows how to fix this?
Thanks in advance. Best regards, Mike
If you want to open CSV using excel, use ; for value/column delimiter instead of , for separate value in each column.
And yes, CSV stands for Comma Separated Value, but excel read it as a single value.
I had same issues yesterday...
Related
I am new to Pentaho. I am trying to build a transformation that can convert a bunch of .xlsx files to .csv (utf-8).
I tried Get file Names and Text File Output, but it saves a single file as csv and the content of that file is the file properties.
I also tried Microsoft Excel Input and Microsoft Excel Output and that did not work either.
Any help will be appreciated. TIA!
I have prepare a SOLUTION for you. I have made my solution full dynamic. For that reason solution is combination of 6 (transformation & job). You only need to define following 2 things:-
Source folder location
Destination folder location
Others will work dynamically.
Also, I have learn a lot with this solution.
Would you like to generate a separate CSV for each Excel file?
It is better to do it like this:
Using the Get File Names component, read the list of Excel files from the folder.
Then call Execute Transformation, and pass the name of the file.
Then a separate Transformation will be performed for each file, and a separate CSV will be generated in it for each Excel file.
I’m making an SSIS package to create a CSV file using OLE DB Source and Flat File Destination.
When I get the file it doesn't contain the headers but they are clearly defined in the destination.
I've tried all the options related to this:
headers rows to skip -1,
column names in the first data row,
column delimiter,
data rows to skip
and even resetting the columns.
Please check the option in the Flat file connection manager as
Just in case this doesn't work then the setting are not being carried over to your config files. In such cases there are two workarounds:
Editing the .dtconfig to edit "firstcolumnhasnames" to 1 adds the column names without needing to delete the connection from the package.
delete the destination connection, and recreate from scratch.
I want to use a format file to help import a comma delimited file using bulk insert. I want to know how you generate format files from a flat file source. The microsoft guidance on this subjects makes it seem as though you can only generate a format file from a SQL table. But I want it to look at text file and tell me what the delimiters are in that file.
Surely this is possible.
Thanks
The format file can, and usually does include more than just delimiters. It also frequently includes column data types, which is why it can only be automatically generated from the Table or view the data is being retrieved from.
If you need to find the delimiters in a flat file, I'm sure there are a number of ways to create a script that could accomplish that, as well as creating a format file.
I know that by doing:
COPY test FROM '/path/to/csv/example.txt' DELIMITER ',' CSV;
I can import csv data to postgresql.
However, I do not have a static csv file. My csv file gets downloaded several times a day and it includes data which has previously been imported to the database. So, to get a consistent database I would have to leave out this old data.
My bestcase idea to realize this would be something like above. However, worstcase would be a java program manually checks each entry of the db with the csv file. Any recommendations for the implementation?
I really appreciate your answer!
You can dump latest data to the temp table using COPY command and MERGE temp table with the live table.
If you are using JAVA program for execute COPY command, then try CopyManager API.
I have an SSIS package written in Visual Studio 2008 that runs a few stored procedures and places the results in files. These files are then sent off to a vendor. All of the files were created with header rows. One of these files should not be sent with a header row.
I'm not sure how to accomplish this.
I was writing my question while I was testing out an answer. I decided, since I hadn't seen the question on StackOverflow, I would just ask it anyway and then answer it with with what I found.
Here are the steps I took:
I removed the header row from the base file.
In the Flat File Connection Editor, I unchecked the Column names in the first data row box.
I previewed the file and it looked good.
In the Flat File destination editor, I remapped the columns.
I re-ran the package and the csv file was created without a header row.
Yes the trick is not to check the "Column names in the first data row" at the Flat File Connection Manager Editor