Inserts, Updates, and Upserts

Daton supports a variety of data destinations and has to ensure that the data loading happens in accordance to the capabilities and limitations of the data destination. This page gives you a high level overview of how data loading behaves for different destinations.

Oracle Autonomous Data Warehouse

Autonomous Data Warehouse is Oracle's flagship data warehousing product. It is based on the tried and tested Oracle database with additional capabilities being added to make the database fully autonomous. Daton leverages the native data loader provided by Oracle ADW to load data extracted from source systems. When these loads happen, the native loader also generates log tables in the warehouse. <>

Google BigQuery

Google BigQuery does not support Upserts i.e updates to existing records based on a primary key while doing bulk loads. The tables in BigQuery are append only and as a result users may see duplicate records if data in the source system changes and you have set that data to replicate to the data warehouse. So users of BigQuery will have to use the metadata created by Daton to extract the most recent record for a particular primary key and use that record for any reporting or analysis. <>

AWS Redshift

AWS Redshift is Amazon's flagship data warehousing product. It was one of the first generation cloud data warehouses that quickly garnered a lot of popularity. Redshift is based on an underlying PostgreSQL DB and supports a lot of functionality that comes out of the box with PostgreSQL. <>