My current client stores all of their configuration information for the enterprise applications in a single table that holds XML. They then use a custom built front end to maintain the configuration values.
I'm writing a fairly straight-forward import process for them using SSIS. I need to make the connection strings and some other information configurable and they want me to use their table. It seems like SSIS expects a file though. Is there any way that I can point SSIS to retrieve its configuration values from an XML stream instead of a path to a file?
The configuration table that they use does not match the structure of a standard SSIS configuration table that you would get using SQL Server as a configuration source with the standard wizard.
Thanks for any advice!
You can retrieve values from the table, put it in variables, and using a script, transfer the varaibale values into the SSIS parameters.
Having the XML formatted just like the SSIS XML file is a huge bonus, though.
Is there a away to put a trigger on thier table to update the SQL Server config table or create a new XML document anytime an SSIS configuration is inserted or updated? Then you could use what you need and they could do what they need and all would be happy.
Related
We want some of our customers to be able to export some data into a file and then we have a job that imports that into a blank copy of a database at our location. Note: a DBA would not be involved. This would be a function within our application.
We can ignore table schema differences - they will match. We have different tables to deal with.
So on the customer side the function would ran somethiug like:
insert into myspecialstoragetable select * from source_table
insert into myspecialstoragetable select * from source_table_2
insert into myspecialstoragetable select * from source_table_3
I then run a select * from myspecialstoragetable and get a .sql file they can then ship to me which we can then use some job/sql script to import into our copy of the db.
I'm thinking we can use XML somehow, but I'm a little lost.
Thanks
Have you looked at the bulk copy utility bcp? You can wrap it with your own program to make it easier for less sophisticated users.
Since it is a function within your application, in what language is the application front-end written ? If it is .NET, you can use Data Transformation Services in SQL Server to do a sample export. In the last step, you could save the steps into a VB/.NET module. If necessary, modify this file to change table names etc. Integrate this DTS module into your application. While doing the sample export, export it to a suitable format such as .CSV, .Excel etc, whichever format from which you will be able to import into a blank database.
Every time the user wants do an export, he will have to click on a button that would invoke the DTS module integrated into your application, that will dump the data to the desired format. He can mail such file to you.
If your application is not written in .NET, in whichever language it is written, it will have options to read data from SQL Server and dump them to a .CSV or text file with delimiters. If it is a primitive language, you may have to do it by concatenating the fields of every record, by looping through the records and writing to a file.
XML would be too far-fetched for this, though it's not impossible. At your end, you should have the ability to parse the XML file and import it into your location. Also, XML is not really suited if the no. of records are too large.
You probably think of a .sql file, as in MySql. In SQL Server, .sql files, that are generated by the 'Generate Scripts' function of SQL Server's interface, are used for table structures/DDL rather than the generation of the insert statements for each of the record's hard values.
I have a bunch of different xml souces (over 100), and I am trying to pipe them all into an OLE DB via SSIS. I started out trying to use the Union All function, but since not all of the data that goes into each row of the database is of the same type, some of the values are entered in as "ignore" by SSIS, and come out as null. Each xml source has multiple outputs as well, so I am trying to avoid manually pointing each node of each xml source at a different db source, and then combining them. I know that you can change the input type of the data in the xml source under the advanced options, but there doesn't appear to be a way to default everything to one data type (string would work for me). Is there a way to use the union all feature to work for me, or a better way to do this? Thanks in advance.
Does your server have access to folder that contains xml files?
If it does, using SSIS for that is not nesessary. Just use OPENROWSET
Also you can make stored procedure that selects data from all xml sources and union all results. This SP you can use in your SSIS package.
I need to export the data from a particular table in my database to Excel files (.xls/.xlsx) that will be located into a shared folder into my network. Now the situation is like this -
I need to use SQL SERVER Agent Jobs.
2.I need to generate a new excel file in every 2 minutes that will contain the refreshed data.
I am using sql server 2008 that doesn't include BI development studio. I'm clueless how to solve this situation. First, I'm not sure how to export the data using jobs because every possible ways I tried had some issues with the OLEDB connection. The 'sp_makewebtask' is also not available in SQL 2008. And I'm also confused how to dynamically generate the names of the files.
Any reference or solution will be helpful.
Follow the steps given below :
1) Make a stored procedure that creates a temporary table and insert records to it.
2) Make a stored procedure that read records from that temporary table and writes to file. You can use this link : clickhere
3) Create an SQL-job that execute step 1 and step 2 sequentially.
I found a better way out. I have created a SSIS(SQL Server Integration Services) package to automate the whole Export to Excel task. Then I deployed that package using SQL Server Agent Jobs. This is a more neat and clean solution as I found.
How do I create a package that extracts multiple databases(and all tables in each database) from another server based on a lookup table (also found on the other server) that contains a column where all the databases I need to extract is listed ?
I need to use the lookup table because new databases is created from time to time on the source, so I cannot just create a job that extracts a "static set" of databases to a destination. It needs to be a bit dynamic...
Furthermore I also need to extract the databases incremental where I can use a timestamp that exists in all databases/tables.
Im new to SSIS, so an "easy" guide would be appriciated.
Thanks
As a rough idea, you could work with SSIS Package Configurations and executing packages from within packages, and then use the Transfer SQL Server Objects Task:
Make a "Main package" that iterates over the column in your lookup table.
For each entry, it should UPDATE the Package Config entry of your second SSIS package accordingly. Use the "SQL Server" configuration for that second package.
The Main package should then execute the second package - there is a also a task for this.
The second package looks at its config to find out which server to get databases from and uses the Transfer SQL Server Objects Task to do so.
then the Main package continues with the next entry from your lookup table.
Ideally you would want to have your "second SSIS package" inside SQL Server's MSDB rather than the file system. Its easier to execute.
here is my newbie to SQL Server Problem.
I created a table with a flat file (.txt) using the Import and Export Data Wizard.
In order to use this table in Arc SDE, I had to create another field named ObjectID.
Need to do the following:
Use the daily generated flat file and erase data from table and replace with new data.
ObjectID field is derived and not in flat file, but need it to stay in table and autopopulate.
Develop script or sql statement
Set up daily process
provide error or completed reports.
Generally speaking, I would use an Integration Services package executed as part of a Sql Server Scheduled Task. Those are the two primary technologies involved, and that should get you started.
you can use ssis technologie (Business Intelligence Development Studio) BIDS, it allows you
to integrate your text file in your sql server table with tranformation you want (adding Object ID field )
a sample :http://www.databasejournal.com/features/mssql/article.php/3832386/Flat-File-Imports-with-SQL-Server-Integration-Services-2008.htm
make a schedule task using a sql agent
a sample here :http://decipherinfosys.wordpress.com/2008/09/17/scheduling-ssis-packages-with-sql-server-agent/