Handling Changes to Schema
A schema for an integration is determined in one of two ways.
A user selects the tables and columns to replicate for a particular source
A predetermined schema is enabled for the integration which cannot be modified by a user.
A detailed list of the schema selection flexibility offered by Daton is documented in the table below.
Schema Change in Source
Handling new data columns in source
When source schemas as modified by the provider of that source, those fields are made available for selection to the user. In order to incorporate those fields into the data warehouse tables, users will have to use the Edit Integration functionality. Once the integration is edited, the user can select the newly available fields in the source.
When new columns are added to an existing integration, users are given a couple of choices
Add columns without reloading data
When users select this option, newly selected columns are added to the table in the data warehouse and any replication job after this change will extract data for the newly added column(s) in addition to the existing columns.
Add columns and reload data
When users select this option, newly selected columns are added to the table in the data warehouse and a full reload of the entire table is performed. When user selects this option, all of the rows in the table will be re-processed and therefore will be counted towards the billing quota.
Handling deprecated data columns in source
When a source provider deprecates a column from their APIs, the data for those columns can no longer be populated in the data warehouse. The column, however, will continue to exist in the data warehouse. Deprecated columns are automatically handled by Daton and no intervention is needed from the user.
Schema Changed by User
Use can use the Edit Integration functionality to modify the schema in the destination. There are a few scenarios that are possible.
Adding New Columns
Users can modify an integration and add additional columns to table. This may be required for various reasons and Daton effortless handles these type of scenarios. When new columns are added to an existing integration, users are given a couple of choices
Add columns without reloading data
When users select this option, newly selected columns are added to the table in the data warehouse and any replication job after this change will extract data for the newly added column(s) in addition to the existing columns.
Add columns and reload data
When users select this option, newly selected columns are added to the table in the data warehouse and a full reload of the entire table is performed. When user selects this option, all of the rows in the table will be re-processed and therefore will be counted towards the billing quota.
Removing Selected Column(s)
Users can modify an integration and de-select previously selected column(s). This may be required for various reasons and Daton effortlessly handles these type of scenarios. When column(s) are de-selected, the column will continue to exist in the data warehouse but no new data will be populated in the table. This action has no impact on billing.
Last updated