How do I create a package that extracts multiple databases(and all tables in each database) from another server based on a lookup table (also found on the other server) that contains a column where all the databases I need to extract is listed ?
I need to use the lookup table because new databases is created from time to time on the source, so I cannot just create a job that extracts a "static set" of databases to a destination. It needs to be a bit dynamic...
Furthermore I also need to extract the databases incremental where I can use a timestamp that exists in all databases/tables.
Im new to SSIS, so an "easy" guide would be appriciated.
Thanks
As a rough idea, you could work with SSIS Package Configurations and executing packages from within packages, and then use the Transfer SQL Server Objects Task:
Make a "Main package" that iterates over the column in your lookup table.
For each entry, it should UPDATE the Package Config entry of your second SSIS package accordingly. Use the "SQL Server" configuration for that second package.
The Main package should then execute the second package - there is a also a task for this.
The second package looks at its config to find out which server to get databases from and uses the Transfer SQL Server Objects Task to do so.
then the Main package continues with the next entry from your lookup table.
Ideally you would want to have your "second SSIS package" inside SQL Server's MSDB rather than the file system. Its easier to execute.
Related
I was wondering if someone could help me with creating a while loop to iterate through multiple databases(100 databases) and drop/Truncate the tables(Around 60 tables in each database) within those databases.Thank you.
My task is to create an SSIS package to move the data from source database to target database.Data in the destination needs to be truncated as part of the process whenever I run that package the old data need to be truncated and the new data needs to be inserted.
Kindly help.
Thank you.
I am just explain how i would go about this problem.
Hoping the destination and source database,tables are of similar schema.
1) Try to create a table with all the database (Source and destination) details all 60 in a separate database.
2) You need get the database details using execute SQL task from the table and use For Each container for looping the logic of truncate the destination table and then move the data from source database to destination (The data is moved from source database one after other).
3) You will need to use Dynamic SQL and stored procedures for moving the data from source database to destination database (That you must be knowing). or You can use Data flow task too if you don't want to use Stored procedure or Dynamic Queries.
Hope it helps you! :)
I want to do the following two things:
Create a table "Lite AC" with two columns (id, lite_ac) in a Postgresql database via Microsoft SSIS package
Update that newly created table via another SSIS package
I want to use that table in order to integrate it into a WHERE SUBSTRING IN (Lite AC) statement.
I am using pgAdmin to connect with that Postgresql database. I know how I can export data from Postgreql via an SSIS package so all the drivers should be installed, but no idea how to do that the other way round.
Is that possible?
I do not have Postgres installed.
You need two connection managers. One for source and the other for destination.
In your case destination is Postgres DB. In order to test out, you may use a Flat File Connection to insert set of sample data from a .txt file. Following tutorial emphasizes on Login credentials to Postgres DB.
Install the relevant drivers (32bit and 62bit accordingly)
Follow this tutorial
Once you are able to connect to Postgres DB via SSIS and rows are inserted into the DB using .txt file - try to add a SQL Scripting Task to be executed after Data Flow Task, so to update the records in the new table.
I need to export the data from a particular table in my database to Excel files (.xls/.xlsx) that will be located into a shared folder into my network. Now the situation is like this -
I need to use SQL SERVER Agent Jobs.
2.I need to generate a new excel file in every 2 minutes that will contain the refreshed data.
I am using sql server 2008 that doesn't include BI development studio. I'm clueless how to solve this situation. First, I'm not sure how to export the data using jobs because every possible ways I tried had some issues with the OLEDB connection. The 'sp_makewebtask' is also not available in SQL 2008. And I'm also confused how to dynamically generate the names of the files.
Any reference or solution will be helpful.
Follow the steps given below :
1) Make a stored procedure that creates a temporary table and insert records to it.
2) Make a stored procedure that read records from that temporary table and writes to file. You can use this link : clickhere
3) Create an SQL-job that execute step 1 and step 2 sequentially.
I found a better way out. I have created a SSIS(SQL Server Integration Services) package to automate the whole Export to Excel task. Then I deployed that package using SQL Server Agent Jobs. This is a more neat and clean solution as I found.
here is my newbie to SQL Server Problem.
I created a table with a flat file (.txt) using the Import and Export Data Wizard.
In order to use this table in Arc SDE, I had to create another field named ObjectID.
Need to do the following:
Use the daily generated flat file and erase data from table and replace with new data.
ObjectID field is derived and not in flat file, but need it to stay in table and autopopulate.
Develop script or sql statement
Set up daily process
provide error or completed reports.
Generally speaking, I would use an Integration Services package executed as part of a Sql Server Scheduled Task. Those are the two primary technologies involved, and that should get you started.
you can use ssis technologie (Business Intelligence Development Studio) BIDS, it allows you
to integrate your text file in your sql server table with tranformation you want (adding Object ID field )
a sample :http://www.databasejournal.com/features/mssql/article.php/3832386/Flat-File-Imports-with-SQL-Server-Integration-Services-2008.htm
make a schedule task using a sql agent
a sample here :http://decipherinfosys.wordpress.com/2008/09/17/scheduling-ssis-packages-with-sql-server-agent/
I have two database named: olddatabase and newdatabase. I want to create a new database named newdatabase2 then copy database structure from newdatabase to newdatabase2 and then convert data from olddatabase to newdatabase2.
To copy database structure from newdatabase to newdatabase2 I use Transfer SQL Server Object Task component in SSIS, and then execute SSIS Package to convert data. But in step 1 I dont now how to create new database named newdatabase2. Currently I create the database manually.
There is a "Transfer Database Task". You can use it to make a copy of newdatabase to newdatabase2:
EDIT:
or you can use the "Copy Database Wizard" once. It will generate an SSIS package and a SQL Server job, then you can re-use it as many times as you want.
EDIT2:
it would't make much sense to open on BIDS because you will only see something like this:
I don't think these packages are meant to be editable.
You can access it, though: