I have source and target table containing 100 columns and 100 million of records each. Now my source table got updated with extra 5 columns i.e. now my source table is having 105 columns and 100 millions of records.
I want to update my target table with the extra 5 columns and update the data as well added in the source table.how can i do it with the best optimized approach ?(may be something like update and insert)
The source and target table is having one to one mapping of columns. there is no transformation happening from source to target table.The source table is truncate load so there might be case of new records added or some records present in target but not in source.
enter image description here
Related
I have a staging table called u_xyzand a target table called sn_abc.
The target table contains other records too from different staging table.
I need to compare whether all the records from staging table(u_abc) is getting reflected in target table(sn_abc) and the records which are missing out in the target table needs to be highlighted in a column(missedout records) in staging table?
Please tell me how to pass 250+ import sets in the below code ?
I have two SQL Server source tables. Each table that contains two columns, one is phoneno1 & ID and the other columns are phoneno2 & ID.
First source table has 30k rows and second source table has 1.8 million rows including 20k records out of 30k records from source table1.
I have 100 target tables and each table contains only one column - phoneno.
I need to compare these above two source tables with 100 target tables one by one and if any target tables phoneno values is matched with source tables phoneno1&2 values then I have to overwrite that source table ID column value in to target tables phoneno values.
If not matching then I need to insert that not matching values into one new table that contains two columns phoneno & ID.
How to do achieve in SQL Server?
I have two existing tables and many records already added.
formula(formulaId,formulaName,formulaType)
and formula_detail(detailId,formulaId,fieldType,value)
Now there is change in formula table and new column is being added , branchId as
formula(formulaId,formulaName,formulaType,branchId),
and branch table is branch(branchId,branchName)
I want to copy and paste every existing record in formula table for every branch.
e.g if there are 3 existing records in formula table (with ID 1,2,3) and 2 branches. Then copy paste operation should produce total new (3*2)=6 records in formula table and also replicate records in formula_detail table for every newly created formula as follows
for formulaId 1 , If there were 5 records in formula_detail table, then copy paste in formual_detail table will have (2*5) new records added in formual_detail table.
I tried some solutions but number of are records huge and script is taking time. Please help. If need any test code I can add.
First of all replicating same column in detail table is against the Normalized Form used in your Data model.
Still If you want to add the column anyway,
Add the column using ALTER statement in formula_detail table
Try using this Merge statement
.
MERGE INTO formula_detail fd
USING (SELECT formulaId, branchId from Formula) temp
ON (fd.formulaId=temp.formulaId)
WHEN MATCHED THEN
UPDATE SET fd.branchId=temp.branchId;
I have a scenario as described below need to create a SSIS Package for that.
I have 3 COLUMNS in source table which needs to be entered in destination table.
But all these columns has to be looked up in the look up table of destination database and then enter their ID's in the destination column.
For example
Source table has 3 columns with values
idnum static type timedimension geography modified date
1 price daydate france 8/12/2015
2 RetailpRICE WEEK ITALY 9/12/2014
I want a package which looks up the column values with the matchin ID and populates in the destination table...
I know we can use the LOOKUP transform to update the data for one single column in destination table what about the other columns which I need to insert along with the lookup insertion.
How can I achieve this ? Also is there a way to pull only the recent data from the source table using modified date column values
Use a different lookup for each lookup table that you need to reference to get the Ids. So if each of your columns that you want IDs for gets its ID from a different table, then you need to use three lookups, one after the other, until you have all three IDs.
part of a longer SSIS task is to read in 10 Mio. rows of data into a table "trunk" and sort this by two different ids in order to join another column.
It turned out that sorting 10 Mio. rows is too much for our server adn thus I need a solution that automatically splits the 10 Mio. rows in table "trunk" into multiple parts.
Currently I did this manually by selecting the TOP 5 Mio via an SQL statement executing the package and then selecting the BOTTOM 5 Mio.
Is there any way to do this either automatically or completely different?
While reading your data from source to the destination table (trunk), you can split it.
You Row Counter between the source and destination. If row count > 5 M, divert to data flow to the next table. You can even create the table dynamically.