Data Replication in Daton
Documentation and management of data replication in Daton integrations.
Data Replication Approach
Select Data
The first step in data replication in Daton is to select the data.
This involves identifying the sources, selecting the tables or files from which data needs to be extracted, and determining the filters and criteria to be used for data selection.
This involves understanding the data source and selecting the tables and columns you want Daton to replicate from your integration.
Replication Scheduling
This creates a replication schedule and determines how frequently Daton will run replication jobs in an integration.
The frequency of replication scheduling depends on the volume and volatility of data in the source system, and the availability of resources.
After selecting the data and determining the replication schedule, next Daton determines the replication method to be used.
There are several replication methods available for data replication, including full table replication, incremental replication, and change data capture (CDC).
The replication method chosen depends on the nature of the data, the size of the data, the performance requirements, and the available resources.
Replication Keys
The next step is to determine the replication keys.
Replication keys are columns in the source table that are used to track changes and identify new or modified data since the last job run.
The choice of replication keys depends on the nature of the method and the requirements with respect to the particular integration.
Replication keys may form on the basis of:
Primary Keys
Timestamps
Version Numbers
Hash Values
Sequences
Composite Keys
Replication Progress
Finally, it is essential to monitor the replication progress during an integration job run.
Replication progress monitoring involves tracking the status of data replication, identifying any errors or issues during replication, and taking corrective action to resolve them.
This step ensures that the data is replicated correctly and efficiently and that Daton can transform and load the data accurately.
Last updated