I have a new requirement with me in which I have to merge data of two identical DB2 databases(around 200 tables in each) keeping all the unique rows from the both. these two databases are identical as they are from two different environment of same application and now clients wants them to merge together.
These tables are business tables, so they always have single records on the basis of Primary keys and foreign keys. which makes that there will be good chances of having the 2 records with same primary key when we will try to merge the databases. We only need to keep only single record out of these two records.
I am not getting a way to start doing this or how to proceed any Idea or approach will help, thanks in advance.
I have to prepare some set of JCLs which can use DB2 utilities, SQL or COBOL program to achieve this, but I am not getting a way to start doing this or how to proceed .. any Idea or approach will help, thanks in advance.
Both Syncsort and DFSORT have match merging capabilities, look for the JOIN control statement in the documentation. Use your shop's DB2 unload utility to extract the data from each table into a flat file, then use your shop's SORT utility to match merge the two versions of each table, outputting matched records to one file, unmatched records from the first version to another file, and unmatched records from the second version to a third file.
This isn't difficult, just tedious.
Related
I've been trying to find a way of comparing huge amount of data in two different tables but I am not sure if this is the correct approach.
So, this is why I am asking it here to understand a problem more and getting some clarity in order to solve it.
As title says I have two tables with less than 2 million rows of data and I need to a data comparison on them. So basically I only need to check if data in one table matches the data from other tables. Each table comes from separate database and I've managed to create views in order to have the same column names.
Here is my approach which gives me differences from two tables.
SELECT db1.ADDRESS_ID, db1.address
FROM UAT_CUSTOMERS_DB1 db1
EXCEPT
SELECT db2.ADDRESS_ID, db2.address
FROM UAT_CUSTOMERS_DB2 db2;
I have two questions, so basically this works:
It seems pretty straightforward but can someone explain to me in a bit more depth how this query works with such speed ? Yes I know - read docs but I would really appreciate alternative answer.
How can I include all the columns from tables without specifying manually each column name ?
I have 30 databases from a survey application that all have a table of results with approximately 100 columns in each. Most of the columns are identical but each survey seems to have a unique column or two added in with no real pattern (these are the added questions and results of the survey). As I am working on the statement to join all of the tables into one large master table the code is getting quite complex. Is there a more efficient way to merge these tables from multiple databases and just select all rows and columns so it will merge if the column exists and create if it encounters a new column?
No, there isn't an automatic way to merge a bunch of similar, but not quite the same, tables into one. At least, not in any database system that I know of.
You could possibly automate something like that with a fairly simple script that relies on your database's information schema (or equivalent).
However, with only 30 tables and only a column or two different in each, I'm not sure it's worth it. A manual approach, with copying and pasting and making minor changes, would probably be faster.
Also, consider whether the "extra" columns that are unique to individual tables need to go into the combined table. The point of making a big single table is to process/analyze all the data together. If something only applies to a single source, this isn't possible.
I have two databases, which are basically identical running on the same machine.
I would like to compare the records in a table on database A vs the same table in database B
I would like to know which records exist in the table on database A that do not exist in the same table on database B.
Database A = "RICSTOREV341"
Database B = "RICHOSTV341"
The table is "Price_Tab"
The columns I would like to pull are F01, F26, F27, F19, F38
Can this be accomplished?
Yes, this can be done.
You could use something like a three part identifier to identify the tables in different database. Kinda like this :
RICSTOREV341.dbo.Price_Tab
Then you can perform a join on the primary key and fetch the result.
Besides the solutions above - I can also suggest using some 3rd party tools for performing data comparison, and most of them have a fully functional free trial (like SQL data comparison tools from ApexSQL or Redgate).
These tools can help you save lots of time, as they can do data comparison and synchronization in just a couple of clicks.
Hope I helped.
I need to copy tables from one teradata server to another one pretty much tables. In order to solve this problem, I have been advised to use arcmain. So table can be transfered this way:
logon ZZZZ/YYYY,XXXX;
COPY DATA TABLES
(DATABASENAME.TABLENAME11) (FROM(DATABASENAME.TABLENAME1)),
(DATABASENAME.TABLENAME12) (FROM(DATABASENAME.TABLENAME2)),
(DATABASENAME.TABLENAME13) (FROM(DATABASENAME.TABLENAME3)),
RELEASE LOCK,
FILE=NVDSID1;
However I have some tables with same names among different databases, in addition this table need to transfered just their structure and some rows (let it be WHERE service_quality ='epic'). Is there solution how to just partionally copy tables?
Initially, I have been figured out quite different way:
1) I copy all these tables `s structure to temp database
2) Insert just required rows to them
3) Copy these tables to required DB on another table
But, one more time, tables with same names rune this solution, they just cant be simply pasted to same DB. Is that possibly to do these 3 steps in a loop, adding one more step - drop table to avoid conflicts?
Creating 100 temp DB really bad solution, and there are already tables with 30 symbols long name.
Any ideas?
What we have done in our environment is to build a set of databases where we populate tables containing slices of data that are destined for another environment. This is very similar to your approach except we use multiple databases and don't have redundant object names. We then use ARCMAIN to ARCHIVE these tables and COPY them to the destination environment.
If you have multiple tables that share the same name across databases I would suggest you create multiple databases to seed the slices of data unless the table structures are the same and the intent is to merge the slices on the target environment. Then you can merge the data in these seed tables for your archive process.
Other solutions include using FastExport and FastLoad or Teradata's Data Mover. The latter is likely going to require additional licensing from Teradata if you are not already using it. The former being script driven can be more flexible than ARCMAIN to accommodate the needs of your particular environment.
I have two databases for two different stores. They have the same tables and structure.
When I created the second one, I made a copy of the first one and I'm pretty sure that I forgot to delete some data from the first one before I activated the second one.
Now I need to find this data and delete it from db two because I have for example articles from store 1 in store 2's database.
Is there some easy way to identify the data that is in both databases?
So i find one way but this would cost me a lot of time.
In heidisql that i use i made a copy of the article table in db 1 and put it in db 2 with the Name article_db1
Then i made a SELECT * FROM article a INNER JOIN article_db1 ON a.artnr = b.artnr and like this i got the matches that is in both databases.
My Problem is that im afraid i have to do this on all 40 tables in the databases and becuase of that i was thinking that it's maybe some faster way to solve this.