Thanks for answering...
I am new to SSIS and I have below requirement:
We are getting bulk records from Oracle based on some criteria.
I need to insert all these bulk records first to an audit table in SQL server The tables has only few columns. the reason for this is we will get to know how many records in each cycle we have got from oracle and if they are pilling up what is the exact reason.
After inserting in audit table, I need to process them one by one in SQL server and at the end of processing I would be either accepting the record and inserting the entire record in SQL serever or rejecting the record in SQL server based on business condition.
Currently to implement this I have first added a Data flow task in SSIS package which selects few columns(which are required by Audit table) from the Oracle and inserts them to SQL server after conversion. Than I am again getting the records from oracle using SQL execute task and processing them one by one using for each container in sql server.
As mentioned above I am making twice the call to Oracle, I am not able to reduce this to one?
can someone please guide me how can I achieve this?
Thanks in advance.
If you have the data already in SQL (you did a bulk copy in your audit table) you can take the data from there and do the insert in the table you want.
Instead of reading the data from Oracle twice, you can use your SQL Execute Task to manipule the data in SQL. You can make the insert statement from here to the final table.
Related
How to save a query in SQL Server using a trigger?
I have a database that supplies different applications. I have detected that one of them is updating a column to null. I want to save the query that does it, so I can know which application is having this error.
Is there a way to save the query that executes a trigger?
I got a big file contains with 10000 lines table rows data, and I had to insert it with a SQL method in my project, so I generated a very long SQL after parsing it. But it doesn't allow me to execute this at one time.
I think it should be placed in a transaction, and I got the way to use transaction with one short SQL in this doc. What method should I use in its document?
Hope you could process at SQL Server side using Bulk insert or OLEDB
Is there any tool or concept that work For the below case,
I have Table named Test in PSQL, SQL, Oracle server.IF I insert or Update or Delete the Data in a Table Test in PSQL, The result should update in the remaining SQL and Oracle. The same case when we do in SQL or Oracle.
Whatever change that happens in any DB Engine that reflects in the remaining Engine also.
Writing my first SSIS package in VS 2012 and have managed to get it to connect to he Paradox tables without any problems.
What I need to do is go though each table and import the data into a corresponding table on a SQL Server database. There is no transformation of the data, as the table structures are the same. All that needs to be done is the data in the SQL Server database must first be deleted and then the data from the Paradox tables inserted.
I can connect one table in Paradox to one table in SQL Server but I want to do them all, please tell me I don't need a separate data task for each
Thanks
Ken
Try using a ForEach Loop Container to enumerate all the tables in your database.
I have a PostgreSQL database that stores real-time data from sensors in a specific table (every 30sec).
What I want to do, is to get periodically the data from the remote PostgreSQL database (for instance every 30sec) and store them in SQL Server 2005 to manipulate them locally. I don't care about having the two databases with duplicate tables. Actually this is what I want to achieve!
So far, I have as Linked Server the PostgreSQL to SQL Server and I can query and retrieve the sensor data. However, I prefer to store them in my SQL Server for performance reasons.
Solution so far:
Make select openquery statements with the linked PostgreSQL and insert the results to my table in SQL Server. Repeat this periodically and store fresh data only (e.g. with a larger timestamp).
I assume that my proposed solution is not ideal. I want to know what are the best practices to achieve this synchronization between the two databases.
Thank you in advance!
If you don't want to write your own code(implementations) to do that you can use SymmetricDS to synch the table from postgreSQL to MSSQL .