I am working on implementing an SSIS package in SQL Server 2005 which deals with XML files.
Multiple XML files will be read-in from a directory on the file system and data from each XML file will be imported into a database table. The XML to database row works fine and I can iterate over all *.xml files in a specific directory.
I have run into an instance during development when the XML file does not validate against the XSD provided. When this happens the package halts execution. I would like to log the invalid XML filename (and maybe the XML file contents as well) to an error log table in SQL Server and then CONTINUE processing the remaining XML files in the directory.
Has anyone ever done this before? I will also be implementing error logging for the XML data itself so if it can be incorporated into this, I would be open to that idea as well.
Thanks!
I haven't dealt with xml like this in particular, but I can suggest some places to start.
You could place an action in the Event Handlers Tab to handle the error. If you have the the filename and contents in a variable, you can insert that into a table.
I was thinking you could use the error route out of your datasource in a dataflow, but the error probably happens before that.
Related
I have several packages/projects deployed in the SSISDB catalog. I want to find which packages have a specific expression attached to a variable value. Is there a TSQL way of doing this? I know the package data is encrypted in [internal].[packages]. But with the proper credentials, is it possible to decrypt it? Or what are the other options I have? Can I use some C#/Powershell script to search?
There is no tables that stores this kind of information. You should search for it by reading the package XML. Using SSISDB, it is not possible to read the package XML using T-SQL since SSISDB encrypts the whole project as binary.
To read the package's XML, you should extract the project binary using the SSISDB.cataloag.get_project stored procedure. Change the extract file extension to .zip and extract its content. Then, loop over packages to check if the variable is used in each package. This can be done using C# or PowerShell:
Extracting dtsx from Integration Service Catalog from C#
Get Package XML from SSIS Catalog with PowerShell
In case you are storing the database within SQL Server (Msdb) you can read the package XML data from the msdb.dbo.sysssispackages table.
I am new to Database and SSIS. Can anyone please let me know is there a way to look or view SQL code generated by SSIS transformations.
I know in BI reporting tools such as Business Objects, when we pull fields or columns into the reporting panel, we can view its corresponding SQL.
Similarly in SSIS, is there any option to view the SQL for SSIS Transformations.
Thanks in Advance
Raj
SSIS unlike other tools does not generate SQL per se, although you can include your own SQL inside tasks and components, but I guess you are not interested in the SQL that you write yourself but rather what SSIS is doing behind the scenes.
An SSIS package is essentially an XML-structured file with a collection of properties marking up the flow and process of its components. You can access to this xml file by right clicking on the package and selecting View Code:
The example above is an empty package so it's a very small XML file. In a complex package, this file can be very large as you will see all the tasks, components, parameters, variables, etc. as well as your own SQL code and C#/VB scripts if any.
When the project is built it generates a .ispac file which is no other thing that a zip file containing the package(s) in project plus a manifest, a content type and any other file required for the package to be deployed and executed.
You can see what is inside a .ispac by renaming to .zip and opening it. In this example I've built the above empty package and renamed the ispac to zip, then opened it :
In summary, unlike other tools that are purely SQL generators, in SSIS there is not much you can see about the generated code, all you can see is its structure as shown above.
Also, as mentioned by Marko Ivkovic in comments it might be possible to get some more info about what is happening at run time by using tools like SQL profiler.
I looking for how to export and import generated from table xml file to a selected path. Path to the file is must be as parameter becouse user in C# program will send path to save it on disk. I do little research and found this but it's don't saying very much to me. Here is link: https://www.mssqltips.com/sqlservertip/4963/simple-image-import-and-export-using-tsql-for-sql-server/
I don't use sql server too much and that's why im here
I've been doing some porting of old SSIS packages from a legacy system into a new system. I was running some tests only to see some kind of error output related to the ODBC connection with Code: 0xC0202009.
The package's two connection managers are both built with SQLNCLI11.1 as the provider.
I believe I can fix the error if I switch that to SQLOLEDB.1. Is there a simple way to do that without having to rebuild the entire package from scratch? Is there an XML file somewhere I can just replace the old value with the new one?
The only way is to open the package (.dtsx) file with a text editor (notepad, notepad++). And search for this property and replace it manually. (.Dtsx file is an xml file)
But replacing this property may cause other errors if each provider has different properties. So Take a backup of these packages before editing.
Take a look at this question it may help you (check my answer and the others. It will give you an idea on how a dtsx file can be readed outside of visual studio):
Automate Version number Retrieval from .Dtsx files
I have an SSIS package which when I run in VS it runs fine. I have changed the protection level to Dont Save Sensitive. I have also given any files related permission to be overwritten by the sql agent job account.
The error I am getting is:
Parsing XML with internal subset DTDs not allowed. Use CONVERT with style option 2 to enable limited internal subset DTD support.
So far I have got the impression that I need to change the code but if the code was really a problem my package wouldnt run in the first place which clearly isnt the case.
How can I reslve this?