I am passing different excel files to SSIS package. I am picking up from folder by using for each loop, mapped variable in that. I am passing that variable to Excel connection manager, but it is not taking. We are getting below error, but if we hardcode the excel file path it is working fine, but I want it should take from for each loop, because we are getting different Excel files, for example: abc.xls or abc(1).xls or abc 03-03-2016. How to handle this in SSIS?
Error:
TITLE: Package Validation Error
------------------------------
Package Validation Error
------------------------------
ADDITIONAL INFORMATION:
Error at Load Attendance data [Excel Source [1]]: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "Excel Connection Manager 2" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
Error at Load Attendance data [SSIS.Pipeline]: component "Excel Source" (1) failed validation and returned error code 0xC020801C.
Error at Load Attendance data [SSIS.Pipeline]: One or more component failed validation.
Error at Load Attendance data: There were errors during task validation.
Error at Tardy Load [Connection manager "Excel Connection Manager 2"]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft JET Database Engine" Hresult: 0x80004005 Description: "Invalid argument.".
(Microsoft.DataTransformationServices.VsIntegration)
------------------------------
BUTTONS:
OK
------------------------------
This is the approach we took to handle a similar problem:
The For Loop assigns the full path of the Excel file to a variable
In the Excel Connection Manager, set the ExcelFilePath element of the Expressions property to that variable
Inside the loop, we have a Script task that will extract the first sheet name assigning it to a ReadWriteVariable in the Script task:
public void Main()
{
string ConnectionString;
OleDbConnection ExcelConnection;
DataTable TablesInFile;
string FirstSheet = "";
ConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0; Data Source=" + Dts.Variables["WorkingFolderAndFile"].Value.ToString() + ";Extended Properties=Excel 8.0";
ExcelConnection = new OleDbConnection(ConnectionString);
ExcelConnection.Open();
TablesInFile = ExcelConnection.GetSchema("Tables");
foreach (DataRow SheetInFile in TablesInFile.Rows)
{
FirstSheet = SheetInFile["TABLE_NAME"].ToString();
break;
}
Dts.Variables["WorkingSheetName"].Value = FirstSheet;
ExcelConnection.Close();
ExcelConnection.Dispose();
Dts.TaskResult = (int)ScriptResults.Success;
}
Then in the Excel Source Editor, for Data Access Mode, select: Table name or view name variable. And for the variable name, use the variable that contains the first sheet name that was assigned in the Script task.
Also for the Excel Source, you may also have to set these properties:
ValidateExternalMetadata = False
AccessMode = OpenRowset From Variable
OpenRowsetVariable = the sheet variable
Excel doesn't have the capability to regenerate every time, The same package with CSV will work fine.
But, iF you want to use the excel file for sure then Use an Execute SQL task and use an excel connection manager and add the script.
Example:
CREATE TABLE
`Excel Destination` (
`PromotionKey` INTEGER,
`PromotionAlternateKey` INTEGER,
`EnglishPromotionName` NVARCHAR(255),
`SpanishPromotionName` NVARCHAR(255),
`FrenchPromotionName` NVARCHAR(255),
`DiscountPct` DOUBLE PRECISION,
`EnglishPromotionType` NVARCHAR(50),
`SpanishPromotionType` NVARCHAR(50),
`FrenchPromotionType` NVARCHAR(50),
`EnglishPromotionCategory` NVARCHAR(50),
`SpanishPromotionCategory` NVARCHAR(50),
`FrenchPromotionCategory` NVARCHAR(50),
`StartDate` DATETIME,
`EndDate` DATETIME,
`MinQty` INTEGER,
`MaxQty` INTEGER
)
The query will generate a new excel file everytime you run the package and will solve your issue.
Hope it solves your issue.
Create a variable Excel File source as String and add the location of excel folder ,
create another variable excel file Name as a string and also add one file name to it .
Go back to your connection manager , on one of your excel file ...right click to to properties , under expressions select Excel file Path then #[$Package::ExcelFileSource]+"\"+ #[$Package::ExcelFileName]. This will let you not hard code your excel file names .
Related
I'm trying to load multiple excel format files (.xlsx) into sql. I have set up my package as followed. My excel files name and sheet name will change daily
File name: PROD File Tracking 02-10.xlsx - Month and Day change daily
Sheet name: 2-10$ -- month and day change daily
Package Structure
For each Loop Container -> Data Flow Task -> Excel Source -> OLE DB Destination
Variables Values
FileName: Z:\Users\darsftp\BDS\GBRTest\PROD File Tracking 02-10.xlsx
FolderPath: Z:\Users\darsftp\BDS\GBRTest
ExtProperties: "Excel 12.0;HDR=Yes"
Need Help
To dynamically pick up file daily with a dynamic sheet name.
I know how to pick up files with a dynamic file name but not with a dynamic sheet name. That's where I'm having the issue.
Create a new variable Sheetname and Set it's property Evaluate as expression to True and use the following expression:
REPLACE(SUBSTRING(#[User::Filename],FINDSTRING(#[User::Filename],"PROD File Tracking", 1 ) + LEN("PROD File Tracking"),100),".xlsx","") + "$"
So if your Filename variable value its Z:\Users\darsftp\BDS\GBRTest\PROD File Tracking 02-10.xlsx so Sheetname variable value will be 02-10$ wich is the sheetname.
And in your Excel Source read the sheetname from variable Sheetname
Side note: Excel sheetname always end with $ sign (it doesn't appear in excel)
Read more about variable and expressions in this article
EDIT 1
You have to add an expression the the Excel connection manager connection string property:
"Provider=Microsoft.ACE.OLEDB.12.0;Data Source="+ #[User::Filename] + ";Extended Properties=\"Excel 12.0;HDR=YES\";"
Set DelayValidation property of Data Flow task to True.
Usefull Links
SSIS with variable excel connection manager
Dynamically assign filename to excel connection string
https://www.google.com.lb/amp/s/sqlserverrider.wordpress.com/2013/01/07/dynamic-file-name-for-excel-connection-manager-ssis/amp/
I am trying to create a project that will produce a separate excel file for every account in a specific table. The first steps to pull the data into a temp file and load an array variable with the accounts for the loop, this is working fine.
I am getting the 2 errors below. And when I execute the package it fails on the export to excel data flow task.
Error 1 Error loading RAW_DATA_EXPORT.dtsx: The connection string
format is not valid. It must consist of one or more components of the
form X=Y, separated by semicolons. This error occurs when a connection
string with zero components is set on database connection manager.
Error 2 Error loading RAW_DATA_EXPORT.dtsx: The result of the
expression ""\\server\DATA\Status\Testing\Filename" +
#[User::Accts] + ".xls"" on property "ConnectionString" cannot be
written to the property. The expression was evaluated, but cannot be
set on the property.
This is what my dynamic connection string looks like evaluated:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\\server\DATA\Status\Testing\Filename03500.xls;Extended Properties="EXCEL 8.0;HDR=YES";
I think the issue is since the excel file are non-existent, the validation fails when it gets to the step. From what I found online I need to either create the sheets through a a script task (not fond of) or use a variable value to create the sheet in the file. I have not been successful on either.
(...) since the excel file are non-existent, the validation fails when it gets to the step.
Have you tried delaying validation in the Data Flow task that exports to Excel? Check Data Flow's properties window.
I'm working on a data export package where I move few data from sql to a excel sheet.
This package is automated to execute every weekend .
Unless i delete the previous excel file , I'm not able to run the package again as it throws the destination table is already present error.
Suggest me a way to retain the old excel file and create a new one everytime.
For example , if the earlier excel sheet was named export_1 , the next sheet on the subsequent week should be named export_2
Create 2 parameteres
File_Path
Connection_String
2.Use a Script task that creates an Excel File and Assign the values to
parameters "File_Path" and "Connection_String" based on the newly
created Excel file
Parameterize the 'Excel Connection'
ConnectionString --> Connection_String
ExcelFilePath --> File_Path
You can dynamically pass the file name through the expressions based on the Datetime.
Ex: Filepath+Date1_mmddyy_hhmmss
Every time the file generates, It will create a file with new file name and Excel file should be passed as an expression.
I'm trying to use DataReader object retrieving data from a text file and use it as a source for table valued parameter of a stored procedure (SQL Server 2012).
The procedure works fine when I try it within SQL Server (SSMS), text file data is retrieved into DataReader without problems but command cannot be executed.
Here's a piece of my code:
Dim oInsertCommand As New SqlClient.SqlCommand("dbo.proc_load_file", oSqlConn)
oInsertCommand.CommandType = CommandType.StoredProcedure
Dim oTVPar As SqlClient.SqlParameter = oInsertCommand.Parameters.AddWithValue("#LoadRM", oReader)
oTVPar.SqlDbType = SqlDbType.Structured
oInsertCommand.ExecuteNonQuery()
The error message is: "String or binary data would be truncated.
The data for table-valued parameter "#LoadRM" doesn't conform to the table type of the parameter."
The statement has been terminated." The error occures in thw last code line posted here.
Thanks for any suggestions.
I have created SSIS to generate a excel file from sql
and giving file name dynamically depending upon todays date.
I have not yet deployed package because of testing.
When I changed System date to test wether it is able to create file for that date or not
but it is giving me an following error
Error at Data Flow Task [Excel Destination [34]]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E37.
Error at Data Flow Task [Excel Destination [34]]: Opening a rowset failed. Check that the object exists in the database.
Error at Data Flow Task [SSIS.Pipeline]: "component "Excel Destination" (34)" failed validation and returned validation status "VS_ISBROKEN".
Error at Data Flow Task [SSIS.Pipeline]: One or more component failed validation.
Error at Data Flow Task : There were errors during task validation.
(Microsoft.DataTransformationServices.VsIntegration)
In excel connection manager i have set property excelfilepath in expression
code is #[User::ExcelFileName] + (DT_WSTR, 20) (DT_DBDATE) GETDATE()+".xls"
which is giving evaluated value like C:\2013-05-24.xls
How do I resolved it.
To fix this issue. In the Excel connection manager properties you have to set the "DelayValidation" property to TRUE.