Leverage capabilities of Machine Learning to CDC to enhance the product and make it smart

There are different features in CDC that are usually configured manually based on customer's environment. For example: Refresh (Standard, Differential, External etc), Fast Apply. There are also other environment related configurations like size of transaction queue, and staging store, JVM memory, CPU, disk I/O and machine hardware. Additionally there are scenarios where upgrade would fail depending on the upgrade procedure customer followed, any specific considerations not followed during upgrades etc.

All the above features/use cases require manual intervention either carefully reading the CDC guides/documents, and taking help from CDC support. Inspite of being cautious, customers tend to encounter issues in any of the above areas.

The proposal is to leverage the capabilities of Machine Learning to CDC to enhance the product to make it smart, to determine for each customer environment what would be the best way the product needs to be configured and proactively alert of potential issues.

So using ML, we would continuously train the model to determine what and how any of the above features should be used. For example: when CDC is running with fast apply off but the performance stats indicate possibility of improvements enabling fast apply, then ML will recommend and automatically enable fast apply and also selects specific fast apply algorithm based on the env details it gathered.

ML can also help with real time analysis of data replication related to the following and provide recommendations.

- Size of database transactions
- Database delay in writing changes to the log
- Relative size/performance of the source and target databases
- Complexity of transformations – homogeneous versus heterogeneous replication
- Available communication bandwidth

ML could be tightly coupled with CDC or a separate component that would make the CDC smart to tune its features automatically for better customer experiences.