I'm new to using IBM IIDR and I am considering using IIDR to do data replication between DB2 - kafka - Postgresql but I can't find an easy way to test this software, I know that the management console and access server can be obtained from IBM central fix, but how can I get the CDC to test on my local machine?
Any help i will appreciate it a lot
You can find the replication engines for Db2, Kafka and PostgreSQL on FixCentral as well.
For example, the IBM InfoSphere Data Replication CDC for all Linux agents 11.4.0.2 Build x installer has all the Linux x64 engines.
The installer will ask you which database type you would like to use. If you will be replicating from PostgreSQL, please select "PostgreSQL source". If you will replicate to PostgreSQL, select "FlexRep". For Kafka and Db2, simply select the matching entry.
To get started with CDC for Kafka, I recommend starting with this CDC Kafka Installation and Configuration guide. More resources are available on the IBM Data Replication wiki.
To get started with CDC for PostgreSQL as a target, see the JDBC configuration information in Knowledge Center. For PostgresSQL as a source, check here for required database user privileges and settings.
CDC for Db2 has a number of deployment options to choose from, described here.
If you can't find the info you need, reach out to the IBM Data Replication support team.
Hope that helps,
Sarah
IBM Data Replication development
Related
We have a situation where IBM Data Replication CDC for Db2 on i ver 6.1 (AS400) is installed as the source and Oracle 10.2.x being the target. A hardware upgrade is being scheduled on the source with a new AS400 box coming in, the existing database would be migrated/transferred to the new box. The Oracle box will stay the same and only an upgrade of the CDC agent is to be done. Management Console and Access Server are going to be upgraded too as part of this exercise.
In this scenario, we want to install CDC ver 11.4 on the new AS400 box to take advantage of the latest version and fixes. My queries are, (some might be silly, my apologies)
Would this be an install or upgrade on the new AS400 box as the database contains CDC related information?
What would happen to the subscriptions, would they have to be created again or an export/import of subscriptions work?
Will a refresh have to be triggered?
Given the target is the same and source is a new server, what are the implications and how to resolve them?
You should be aware that CDC v6 and CDC v 10.2.x are no longer supported (except for the zSeries 10.2.1 version) so if you have any issues during your migration process you may find it difficult to get support.
In general terms, you should be able to install something like CDC 11.3.3 on the current iSeries in advance of the migration as this version of CDC supports OS/400 6.1 and 7.1 7.2 and 7.3. Then you can save/restore the CDC product library along with all the other libraries. Do not forget to add the TCP/IP listener port service as well. If there is a change of hostname or IP address, you should follow the documented procedure to update the datastore properties in the Access Server https://www.ibm.com/support/knowledgecenter/SSTRGZ_11.4.0/com.ibm.cdcdoc.mcadminguide.doc/tasks/handlinghostportforsource.html
Before you restart operations on the new iSeries, you will either need to mark table capture point for all source tables or use SETJRNPOS to manually specify the restart position for replication as the target bookmarks are based on the old journal receiver chain and on the new system this will different
I prefer to upgrade pre-install mainly because it avoids making two changes at the same time. If you had an issue with CDC after migration and then upgrading you would have to determine if it was the new version or the migration that was the cause. However, upgrading the migrated CDC instance immediately after the migration should be OK.
If the IP address or hostname used in the Access Server datastore configuration is changed, you will need to change this following this procedure
https://www.ibm.com/support/knowledgecenter/SSTRGZ_11.4.0/com.ibm.cdcdoc.mcadminguide.doc/tasks/handlinghostportforsource.html
Please help me regarding sync of data between two sql servers automatically using replication and publication.
Please help me which edition provide these features and how to configure these services to automatically sync data between local sql server and remote sql server.
Thanks
Replication has been part of SQL Server at least since the version 2005.
It's not a question of version number, but of type of edition (express, standard...)
If you need replication, you will need at least a standard version on the publisher server. Any version will work on the subscriber.
Note that if you just want to get a backup (for development purposes or a preprod server), you're probably better off writing a small script (in Powershell for instance) that generates a backup of the source server and downloads/uploads it to the target server every night or so.
I'm using a macbook and because I don't want to install a virtual machine to have an Oracle XE running (I don't have enough space in my HD), I subscribed in the APEX service. So I have two questions:
1. How can I load a dump in the server? I just found a SQL place, but nothing about dumps...
2. How can I use the SQL Developer cloud service to manage my cloud database? I'm not seeing where can I find the information requested in the screen:
I'm using Infosphere Datastage & Quality Stage 8.5.
I need to know how to backup the whole datastage environment including DB2 files, configurations, etc. to prevent crash event on servers.
Please provide with the documentation as well.
Detailed instructions for backing up InformationServer 8.5 can be found here: 8.5 Backup
Later versions of Information Server do have a backup and recovery tool. This tool works against multiple versions of the DataStage product (back to 8.5). See the following link: InformationServer Backup/Recovery. Contact IBM support to obtain this tool.
I wanted to broach the issue of SQL Server's Hadoop distribution called HDInsight.
Given that there is a connection provided to Hadoop, does anyone have experience with HDInsight and particularly a comparison between the Hadoop / SQL Server connector and HDIinsight / SQL Server from a real life DTP scenario or personal 1 node installation?
http://sqlmag.com/blog/use-ssis-etl-hadoop
http://www.microsoft.com/en-us/download/details.aspx?id=27584
http://www.microsoft.com/en-us/sqlserver/solutions-technologies/business-intelligence/big-data.aspx
HDInsight is the distribution of Hadoop that Microsoft maintains for use in Azure. You could roughly compare this to Amazon Elastic MapReduce. They both serve the purpose of being a hosted Hadoop service that has almost no management overhead.
The Hortonworks Data Platform for Windows contains the open source changes that Hortonworks and Microsoft have collaborated on to make Hadoop run well on Windows. HDP isn't HDInsight.
In short - you don't need to use HDInsight if you want to run Hadoop in a Windows environment.
While I can't speak directly to using HDInsight and moving data back and forth between SQL Server, I've done implemented a data processing solution using SQL Server, Hadoop, and Elastic MapReduce. Barring some data quality issues and BULK INSERT weirdness, the process was painless.
Finally, you ask "do we really want to run Hadoop size datasets on Windows servers?" - Windows performs well and has solid tooling around it. I've been somewhat skeptical about running Hadoop and other Java platform software on Windows because of legacy Java I/O issues and a lack of community support, not because of any performance issues.
The largest issues that Windows companies will find moving to Hadoop is there will be limited support in community forums and channels when the problem becomes a Hadoop + Windows issue. It's very easy for people to throw their hands up and say "Nope, not helping out, don't have Windows." With time and adoption, this problem goes away. Besides, nothing says you have to finish on the same platform you start with. You could easily deploy with HDP on Windows and move to HDP on Linux at a later date.
I have put together some SQL Server and Hadoop basics for DBAs that should be helpful.