I am getting the following error when I try and run my package. I am new to ssis. Any suggestions. Tahnks
===================================
Package Validation Error (Package Validation Error)
===================================
Error at Data Flow Task [SSIS.Pipeline]: "OLE DB Source" failed validation and returned validation status "VS_NEEDSNEWMETADATA".
Error at Data Flow Task [SSIS.Pipeline]: One or more component failed validation.
Error at Data Flow Task: There were errors during task validation.
(Microsoft.DataTransformationServices.VsIntegration)
Program Location:
at Microsoft.DataTransformationServices.Project.DataTransformationsPackageDebugger.ValidateAndRunDebugger(Int32 flags, IOutputWindow outputWindow, DataTransformationsProjectConfigurationOptions options)
at Microsoft.DataTransformationServices.Project.DataTransformationsProjectDebugger.LaunchDtsPackage(Int32 launchOptions, ProjectItem startupProjItem, DataTransformationsProjectConfigurationOptions options)
at Microsoft.DataTransformationServices.Project.DataTransformationsProjectDebugger.LaunchActivePackage(Int32 launchOptions)
at Microsoft.DataTransformationServices.Project.DataTransformationsProjectDebugger.LaunchDtsPackage(Int32 launchOptions, DataTransformationsProjectConfigurationOptions options)
at Microsoft.DataTransformationServices.Project.DataTransformationsProjectDebugger.Launch(Int32 launchOptions, DataTransformationsProjectConfigurationOptions options)
VS_NEEDSNEWMETADATA shows up when the underlying data behind one of the tasks changes. The fastest solution will probably be to just delete and re-create each element which is throwing an error.
How about disabling validation checks?
Like if you right click on source or destination component and select properties then you will have the property named validateExternalMetadata put that as false and try.
This Solution is working for me.
This normally occurs if there has been a change to your schema, not to stress, just double click on your input and output and it should resolve itself
Make sure your connection is valid. If you are using dynamic connections, then try to set the option "delay validation" = true on the package or dataflow.
In my case destination table structure was not matching with matadata in OLEDB component. I added the missing column which i forgot to add and after that it was fixed.
After researching a bit (check to extract your own conclusions: this and this one), I think I've found a nice workaround for when the problem with the metadata comes from a Ole DB object, but only for a very specific case.
The thing is that when you change your columns names / remove columns / add columns, you can't do anything but update the metadata.
However, if you use a SQL query to retrieve the data from the object, in the case that you don't need to update the query itself, you won't need to update the metadata if the query still can ask for what it wants. Basically, if the query is still valid.
I've tried it within my own ETL, and changed an Ole DB object which was reading the data from an Excel file, targeting one sheet and then I had all the columns selected in the tab.
Changing it for an SQL query to retrieve the full sheet like:
SELECT * FROM ['Sheet_Name$']
Solved completely the case for me, even introducing files with different metadata in headers.
Related
I have an issue with a very simple task in SSIS 2012.
I have a for-each container that runs in FOR-EACH-FILE Enumerator mode. I want to read a target folder with XML files. The path to the folder is correctly configured. The files field is set to *.xml
The variable mapping is defined with the follwing Variable: User::FileVar , Index 0.
Now I add a simple data flow task inside the container. The dataflow task only has a XML-Data Source task, that's it. For the XML Data source task, the XSD location is set. When I click choose columns, I can see the columns from the XSD schema.
BUT: When I save the XML task , I always get the error message: The Property XMLDataVariable is empty. I tried both data Access modes, XML file from variable and XML data from variable. The error message remains, I cannot run the package.
I don't use any expressions, neither at the foreach loop container nor at the data flow task.
I dont know what's wrong here, I did the steps exactly as shown in some tutorials for older versions of SSIS.
Do you have any ideas?
The issue is that the XML Source is trying to validate the existence of the given file during the design time. However, you will know the file name only during runtime when the Foreach loop container executes and loops through every XML file available in a given folder.
I recreated an SSIS 2012 package using my answer to one of other SO questions.
SSIS reading multiple xml files from folder
I was able to reproduce the error The property "XMLDataVariable" on the XML Source was empty
On the XML source, I set the property ValidateExternalMetadata to False. Setting this to false will force the package not to verify the existence of the xml file path during design time.
I was successfully able to execute the package.
Hope that helps.
This the scenario have. There's an SSRS server that has a report where the datasource is of type XML and it uses windows authentication, nothing else to it. Inside the report there's a dataset that uses that data source with this for a query "=Code.ReportUser.GetListOfItems()". This calls a method in the custom assembly that connects to the database and returns data in a string formated like this (angle brackets changed to curly for read
"<Query><XmlData><Items>" + xmlDocument.DocumentElement.InnerXml + "</Items></XmlData></Query>"
where sql datatable is written into the xmlDocument. If I go to the reporting server through an IE and run the report everything works as it should. The Problem occures when add this link to Zangle application which runs it as "IEHOST.app 'report's url'". I believe its a foxpro app that opens a browser object and report fails on the dataset that uses my xml data source. There's no question about url or reports correctness because it functions in a different environment.
This is the server error i get:
library!ReportServer_0-8!ca0!02/24/2012-10:18:45::
Call to GetPermissionsAction(path to my report).
library!ReportServer_0-8!e68!02/24/2012-10:18:45::
Call to GetSystemPropertiesAction().
library!ReportServer_0-8!e68!02/24/2012-10:18:45::
Call to GetPropertiesAction(path to my report, PathBased).
library!ReportServer_0-8!10bc!02/24/2012-10:18:45::
Call to GetSystemPermissionsAction().
library!ReportServer_0-8!e68!02/24/2012-10:18:45::
Call to GetSystemPropertiesAction().
processing!ReportServer_0-8!10bc!02/24/2012-10:18:45::
e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: ,
Microsoft.ReportingServices.ReportProcessing.ReportProcessingException:
Error during processing of the CommandText expression of dataset ‘Items’.;
processing!ReportServer_0-8!10bc!02/24/2012-10:18:45::
e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: ,
Microsoft.ReportingServices.ReportProcessing.ReportProcessingException:
Cannot set the command text for dataset 'Items'. ---> Microsoft.ReportingServices.ReportProcessing.ReportProcessingException:
Error during processing of the CommandText expression of dataset ‘Items’.
at Microsoft.ReportingServices.OnDemandProcessing.RuntimeDataSet.RunEmbeddedQuery(Boolean& readerExtensionsSupported, Boolean& readerFieldProperties, List`1 queryParams, Object[] paramValues)
--- End of inner exception stack trace ---;
I'm not sure what this means exactly. Any thoughts? it looks like a permission issue but i don't understand where. I did try to change xml data source authentication from windows to none and then add my AD account as an execution user on the reporting server. This didnt' change anything still working from browser but from Zangle browser object. One more thing, server requires authentication and all the report regardless of the environment promt for login and successfully authenticate so my credentials seems to pass to the server and only my xml data source is throwing an error when launched from with Zangle. Please share your ideas. I'm a programmer but I'm not too server stuff savy and I'm new to SSRS. Thank you.
Resolved. Turns out that foxpro app lowercases everything and the name of the sql server passed to the assembly doesn't match the name stored in the assembly. Silly :) So we'll be lowercase comparing everything from now on.
I'm seeing an issue with an SSIS (SQL Server 2005) job where I'm getting the following error:
The file name "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\UNC\FOLDERS\filename.xls;Extended Properties="EXCEL 8.0;HDR=YES";" specified in the connection was not valid.
My searching around this site and others indicates that the most common cause of this is a permissions error but I don't believe that's the case in this situation since any number of files have successfully been processed through this implementation.
Here's an overview of the setup:
Vendors FTP files to us on a daily basis that a Windows service picks up, copies to a temporary directory and then calls SSIS jobs on those files. There are two SSIS jobs for each vendor one for a snapshot data feed and one for a transaction listing.
There are currently over 50 different SSIS jobs in the overall process. All of them work except for one specific transaction job which fails with the above error in a script task step. Files come in at least daily with unique file names so I grab the job, determine the vendor based off the source directory and then the file type based off indicators in the file name to determine which SSIS job to call. Since file names change every day, when the service calls the SSIS job, I pass in a series of parameters including the vendor file name so it can properly connect to the file.
Each job begins with a script task that sets necessary variable values for the rest of the job. For example, since the vendor file name changes with each run, I pass in the vendor file name through the SSIS variables collection then set the connection string of a datasource using that file name as the DataSource in the string. It is at that point of the script task that the above error occurs. Here's the task script code where the error occurs:
Dts.Connections("Transactions File").ConnectionString = _
Dts.Variables("ConnectionString").Value.ToString().Replace("##FILE_PATH##", sourceFilePath)
The ConnectionString value is: Provider=Microsoft.Jet.OLEDB.4.0;Data Source=##FILE_PATH##;Extended Properties="EXCEL 8.0;HDR=YES";
The sourceFilePath is the full UNC path to the vendor file in the processing directory
I don't believe it's a permissions error since all the other files going through this process (using the same holding directory for processing) are working. It shouldn't be an issue of the file not existing since again it follows the same process as every other file and I have verified the file properly ends up in the correct directory. I also considered that the connection string might be too long, but the filepath ends up at 109 characters and even with a shorter (<90) full path, the same error occurs.
Is there anything else you can you think of for me to look at? Thanks for any help.
Based on the information presented, you are doing everything correct. If you're new to SSIS, one thing I'd suggest, is that you get a copy of the excellent add-in BIDSHelper. It has great features that can really save you time especially with regard to configurations and expressions.
I created a reference package that had an Excel Connection Manager pointing to C:\ssisdata\so_paulsmithjr.xls and wired everything up.
At this point, I know things are working so it was time to make the package move. I created the following variables and their values
CurrentFile - C:\ssisdata\so_paulsmithjr.xls
PlaceHolder - ##FILE_PATH##
TemplateConnection - Provider=Microsoft.Jet.OLEDB.4.0;Data Source=##FILE_PATH##;Extended Properties="Excel 8.0;HDR=YES";
A fourth variable is set to be an expression (Right click on variable, properties window. Set Evaluate as Expression = True & Expression is below)
CurrentConnection - REPLACE(#[User::TemplateConnection], #[User::PlaceHolder], #[User::CurrentFile])
I compared the CurrentConnection value to the ReferenceConnection (which is the original value of the Excel Connection Manager's connection string) and things were a match. At this point, if I were to change the value of CurrentFile to C:\ssisdata\so_paulsmithjr - Copy.xls, that would automatically be reflected in the value of CurrentConnection.
The final trick would be to use an Expression on the Excel Connection Manager. Again, right click on the CM and under Properties, there will be Expressions. It won't expand as there is nothing under it. Instead click the ellipses and then select ConnectionString property and select the ellipses again and this time drag down the #[User::CurrentFile] variable. Click OK x2 and now your connection manager is set to use wherever the CurrentConnection variable specifies.
Does that work any better?
we've got a real confusing problem. We're trying to test an SQL Bulk Load using a little app we've written that passes in the datafile XML, the schema, and the SQL database connection string.
It's a very straight-forward app, here's the main part of the code:
SQLXMLBULKLOADLib.SQLXMLBulkLoad4Class objBL = new SQLXMLBULKLOADLib.SQLXMLBulkLoad4Class();
objBL.ConnectionString = "provider=sqloledb;Data Source=SERVER\\SERVER; Database=Main;User Id=Username;Password=password;";
objBL.BulkLoad = true;
objBL.CheckConstraints = true;
objBL.ErrorLogFile = "error.xml";
objBL.KeepIdentity = false;
objBL.Execute("schema.xml", "data.xml");
As you can see, it's very simple but we're getting the following error from the library we're passing this stuff to: Interop.SQLXMLBULKLOADLib.dll.
The message reads:
Failure: Attempted to read or write protected memory. This is often an indication that other memory has been corrupted
We have no idea what's causing it or what it even means.
Before this we first had an error because SQLXML4.0 wasn't installed, so that was easy to fix. Then there was an error because it couldn't connect to the database (wrong connection string) - fixed. Now there's this and we are just baffled.
Thanks for any help. We're really scratching our heads!
I am not familiar with this particular utility (Interop.SQLXMLBULKLOADLib.dll), but have you checked that your XML validates to its schema .xsd file? Perhaps the dll could have issues with loading the xml data file into memory structures if it is invalid?
I try to understand your problem ,but i have more doubt in that,
If u have time try access the below link ,i think it will definitely useful for you
link text
I know I did something that raised this error message once, but (as often happens) the problem ended up having nothing to do with the error message. Not much help, alas.
Some troubleshooting ideas: try to determine the actual SQL command being generated and submitted by the application to SQL Server (SQL Profiler should help here), and run it as "close" to the database as possible--from within SSMS, using SQLCMD, direct BCP call, whatever is appropriate. Detailing all tests you make and the results you get may help.
I have a SSIS 2008 package that calls about 25 other SSIS packages.
Each of those child packages loads a specific file into a table. But sometimes one or more of these input files will be missing.
How can I let a child package fail (because a file is missing) but let the rest of the parent package keep on running?
I've tried increasing the maximum error count on the parent package, the tasks in the parent package that call each child, and in the child package itself. None of that seemed to make any difference. I still get this error when I run it with a file missing:
SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The
Execution method succeeded, but the
number of errors raised (2) reached
the maximum allowed (1); resulting in
failure. This occurs when the number
of errors reaches the number specified
in MaximumErrorCount. Change the
MaximumErrorCount or fix the errors.
Edit:
failpackageonfailure and faulparentonfailure are already all set to false everywhere.
I haven't tried this, but this is how I would approach it.
Create a variable for the file name and the child package name.
Use a For Each Loop container. Have it go through the location of the files and pull the file names one at a time. Use the file name to change the child package name variable. In the container have the task to run the child package and have the name dynamically set based on the values of the child package name variable.
Then it should only try to run the child packages which have appropriate files.
in the properties of the execute package task, you can set the failpackageonfailure and faulparentonfailure. i haven't worked with these, but you can probably play with them to get your desired results.
Side note: for simplicity, I'd set these settings on the parent SSIS package.
There is a MaximumErrorCount values at the Sequence Containers & package level. If you're using this be sure your values are in-sync because the package level settings take precedence.
Another option is the ForcedExecutionValue.
To set this up, load the properties tab for each of container and:
1) ForceExecutionValue to TRUE
This will cause the container to return whatever value you put in the variable (see step #2), despite the outcome of the task(s).
2) ForcedExecutionValue to 0
This acts a return value for that task, and sets it to 0 (true, think "return 0" as in C++).
I hope that helps.
This will cause the package to
Load the properties using "ForcedExecutionValue" to 0, then Then set the Force
I have done this kind of scenario development, first plan the package execution method as whenever you will get a file we need to process the package if not either fail or leave the package ultimately our target is to process all the package of files existing. take a variable for all the packages. set the variable to "Y" or "N" on the existing of the file using script component or connection string in the parent package. the existing condition to execute the package on the value of the variable.
This method gave us desired results of process multiple files with different occurences of source files.
thanks
prav