Initial situation:
I'm stuck in a simple task (in my opinion it should be simple...)
I have a collection of data which should be exported weekly to Excel.
This export contains 104 columns, from which 57 are nvarchar(max) and contains item descriptions and other information in different languages for our sales guys.
The report will have something around 2000 to 8000 rows.
I use a SQL Server 2017 CU 16
My intention:
I intended to do an SSI Job with an Excel template where the columns are predefined (width, data type and so on).
This job would have something like those steps:
Delete existing Excel file
copy Excel template as a new Excel file
DataFlowTask using SQL Server as the source and Excel destination as the target
What I already tried:
If I use the excel template with only headers, I get the following error for each of the nvarchar(max) columns:
[Excel Destination [2]] Error: An error occurred while setting up a
binding for the "ColumnName" column. The binding status was
"DT_NTEXT".
When I prepare the template having it prefilled with one row. This row has a long text (more than 255 characters) for the columns where nvarchar(max) is in the source, everything runs fine but, this dummy line is still existing.
Another try I did was dropping the sheet using an "Execute SQL Task" to the Excel File Connection and recreating the sheet using a create table statement in another "Execute SQL Task" to the same Excel file connection, I get the same error as above. Although I'm using NTEXT as the datatype for the relevant columns.
Question:
How can I export data seamlessly into a preformatted excel file which contains NTEXT?
Thank you very much in advance for any assistance.
Related
I need to export a subset of db data tables into a single file. I tried using SSMS to export to an excel workbook. Each table would have its own worksheet. Sounds ideal !! However, many of our table names are > 31 characters in length, thus table names are being truncated by excel (which apparently has a nonconfigurable worksheet naming convention) as the worksheets are created. This won't work for us, and its far too late to change the table names. Therefore, seems excel workbook is out.
I am thinking there must be an existing tool that has a wizard that would allow us to select a handful of tables and store it in hierarchical way, in a single file, such that the table names are properly preserved in the file. Perhaps an export to json ??
Any suggestions ??
Could not find a product, or open source tool, that would properly save Azure SQL Tables whose table name length could be > 31 characters. Wrote a python script and saved tables in json format (snippet).
{
"artifacts": [{"tablename": "[dbo][Config_Entity_Metadata_Sample_Table]",
"contents": "{\"schema\"{\"fields[{\"name\":\"index\",\"type\":\"integer\"},}]
}
I've got an SSIS package that loads data from a flat file source into a table within a SQL Server Database i then have a SQL task to get specific data and load into an excel destination, i want to add another SQL task to load different data into the same excel workbook but on a different tab is this possible?
Yes it's possible. You can use same excel file to save diffrent data sets in different worksheets. In Excel destination editor for Excel Destination component press [New] button and change name of the table in CREATE TABLE statment to name of the worksheet you want to use. After that select new it as a worksheet name.
Can I execute an Excel package worksheets with different table structure from SSIS to different tables in SSMS dynamically ?
I have an excel file with 3 worksheets. I want to process those worksheets into one database in SSMS but each to their own table. what will be the best practice processing this file? i'm new to SSIS . Thank you in advance
You could, suppose you have target set already in database. there is one properties called openrowset in the advanced properties for excel source. you could specify which sheet to be loaded as well as the columns.
For example, Sheet1$A1:Z, which will load data from sheet1 A column to Z, start from row #1 if you did not check header row as row #1
I have SSIS Package which Exports Data from table to Excel file
Control Flow :-
Data Flow :-
This is My Step :-
Drop Excel Table
Create Excel Table with format as of my Select Query which i used to retrieve data from database
Insert Data from Database to Excel file
I Used Query Like Select * From Table Where --Some Condition
I retrieve 3000 rows out of 10000 rows and put that 3000 rows in my excel sheet.
But when open my excel sheet i saw scrollbar which goes till 10000th row and ends hence my excel sheet size also increses . how can i reduce my excel sheet size ? my excel sheet contains only 3000 rows then why blank cells which goes till 10000th row ?
SQL Server 2008 &
Visual Studio 2008 with BIDS
I believe your issue is around the method in which you are using to create the file. You have two alternatives and both should fix your issue:
Solution #1:
You can create an Excel file with those predefined columns, essentially your empty output file - this would act as your 'Template File'. Your flow would then be this:
File System Task - Copy template file to output or working directory (rename if necessary)
OLEDB Source Task - Query your source for the data (3000)
Data Conversion Task
Excel Destination Task - Put data into new Excel file
Note: You already have steps 2 thru 3 complete, you just need to make sure you are connecting to the new Excel file. Also, to clarify, step 1 is outside the Control Flow Task.
This way is helpful because you always have a blank and consistently formatted Excel file to copy and work with.
Solution #2:
The other option is to use a Script Task and create the Excel file - you could also load the data into the file in this task. This requires some basic understanding of VB.NET or C#. Basically you would need to get a XLS library (like NPOI). This is more complicated, but gives you the best functionality.
I recommend you try solution #1 and see how that works for you.
Drop table SheetName doesn't delete the sheet instead it just deletes the row . If for the 1st time you have loaded 10K rows and then again executed the package by restricting the number of rows to 3K ,the excel file will still contain those 10K empty rows as it retains the sheet along with the empty spaces .
You can use script task to delete the sheet using COM obects .But for that you need to place the Excel PIA(Primary Interop Assemply) to make it visible for VSA or else create a new excel file every time the package runs
Else as suggested by Nicarus use File System Task to delete the existing file and create a new Excel file on every execution .
Diagram :
File System Task :
Use the same components and the query for Create Table using Execute SQL task and your DFT
So I have a relatively simple SSIS job that does the following:
Execute SQL Task - Drop Sheet from Excel File
drop table `Table`
Execute SQL Task - Create Sheet in Excel File
CREATE TABLE `Table` (
`Col1` VarChar (255) ,
`Col2` Long ,
`Col3` VarChar (84) ,
`Col4` VarChar (60) ,
`Col5` VarChar (255) ,
`Col6` VarChar (20) ,
`Col7` VarChar (255) ,
`Col8` VarChar (255) ,
`Col9` VarChar(255))
Data Flow Task - Export Data from SQL to Excel
This just runs an SQL query [OLE DB Source], coverts everything to unicode strings, and exports the data to an Excel Destination.
NOTE: This job executes perfectly with no errors in BIDS 2005. However, when I initially tried running it in BIDS 2008 (32 bit mode), I got the following error on both the Drop Sheet and Create Sheet Execute SQL Tasks mentioned above:
Warning: Multiple-step OLE DB operation generated errors. Check each
OLE DB status value, if available. No work was done.
I found I could fix this by changing the ConnectionType property of my Execute SQL Tasks to ADO, and using the following connection string:
Data Source=\\<filelocation>\<ExcelFileName>.xls;Provider=Microsoft.ACE.OLEDB.12.0;Extended Properties=Excel 8.0;
At this point, I'm back to the package executing in BIDS 2008 with no errors. Everything runs great!
Then I tried to update the job from exporting to an Excel 97-2003 .xls file to exporting to an Excel 2007 .xlsx file.
So, per Microsoft, I had to change my Execute SQL Tasks to use the following connection string:
Data Source=\\<filelocation>\<ExcelFileName>.xlsx;Provider=Microsoft.ACE.OLEDB.12.0;Extended Properties=Excel 12.0 XML;
I also had to update the connection manager for my Excel Destination step (which DOES support the Excel 2007 format in BIDS 2008 per Microsoft) to the following:
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=\\<file location>\<Excel filename.xlsx;Extended Properties="Excel 12.0 XML;HDR=YES";
So at this point, all the connections test good, and everything APPEARS like it should work. However, when I execute the package I have the following problem.
The Drop Sheet Execute SQL Task completes successfully according to BIDS. However, it doesn't actually drop the sheet in the Excel file anymore. It DOES delete all the data contained in the sheet though.
Because of #1, the Create Sheet Execute SQL Task fails because the sheet was never actually dropped.
Any ideas why this isn't working anymore? Honestly I've looked all over the internet and SO, and I have yet to see someone explain how to do this. Is there some new command to drop a sheet in Excel 2007?
For anyone else who may find this:
I fixed the issue with exporting data to XLSX files by replacing the two Execute SQL tasks with two File System Tasks. One that deletes the existing file, and another that copies a "template" Excel file (basically just an empty spreadsheet with column headers) to the reporting directory.
Then I export the data to the new template file using the data flow task.
Not my ideal solution, I much prefer the old method of having one file and just re-creating the table. However, apparently that's no longer an option in Excel 2007, and the File System task method using a template file will work.
The two most helpful resources I found when working on this job are:
SSIS - Excel 2007 Connection Guide
Copy/Rename Files Using SSIS File System Task