LogoLogo
Daton
Daton
  • Daton Defined: From Web App Essentials to Advanced Integrations
  • Platform
    • Essentials
      • Billing
      • Integrate Facebook Ads on Daton
      • Integrate Shopify on Daton
      • Integrate Google Analytics on Daton
      • Access permissions for Daton
      • How to get started with Daton setup
      • Terminology in Daton
        • Source and Source_Table statuses
      • Various pages in Daton
    • Features
      • Navigating the pages in Daton
        • Notifications
        • Logs
        • Dashboard
        • My Account
        • Connectors
        • My Integrations
        • Integration Details
      • Templates
      • Account Management
        • My Account Page
      • Notifications
      • Monitoring
        • Logging and Log Retention
      • Manage Integrations
        • Handling Changes to Schema
        • Modifying an integration
      • Data
        • Handling Nested Data
        • Daton Metadata
        • Inserts, Updates, and Upserts
        • Data Type Mapping
        • Data Replication in Daton
          • Replication Fields
          • Replication Methods
            • Key-based Incremental Replication
      • Knowledge Base
        • What is ELT and how is it different from traditional ETL tools
          • Billing
      • Pipelines
        • Pipeline Jobs
        • Loading Behavior
      • Security
      • Feature Updates
        • Amazon Vendor Central
  • Connectors
    • Data Sources
      • Advertising
        • Adjust v1
        • Adjust v2
          • About Adjust v2 Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Adjust v2 with Daton
          • Schema Information
        • Amazon Ads
          • Release Notes
        • Amazon Attribution
          • Release Notes
        • Amazon Brand Metrics
          • About Amazon Brand Metrics
          • Essential Attributes and Supported Tables
          • How to Integrate Amazon Brand Metrics with Daton
          • Schema Infomation
        • Amazon DSP V1
          • Release Notes
        • Amazon DSP V2
        • Amazon Marketing Cloud (AMC)
        • Amazon Marketing Streams
          • About Amazon Marketing Streams Connector
          • Essential Attributes
          • How to Integrate Amazon Marketing Streams with Daton
          • Release Notes
        • Amazon Sponsored Brand Ads
          • Release Notes
        • Amazon Sponsored Display Ads
          • Release Notes
        • Amazon Sponsored Product Ads
          • Release Notes
        • Amazon Sponsored TV Ads
        • Bing Ads
          • Release Notes
        • Bol Ads
          • Release Notes
        • Facebook Ads
          • Release Notes
        • Google Ads
          • Release Notes
        • Google My Business
        • Google My Business v2
          • About Google My Business v2 Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Google Business Profile Account with Daton
          • Schema Information
        • Gorgias
        • Gorgias V2
        • HelpScout
        • HelpScout v2
          • About HelpScout Platform
          • Essential Attributes and Supported Tables
          • How to Integrate HelpScout with Daton
          • Schema Information
          • Release Notes
        • Outbrain
          • Release Notes
        • Outbrain v2
          • About Outbrain Platform
          • Essential Attributes & Supported Tables
          • How to Integrate Outbrain v2 with Daton
          • Schema Information
        • Pinterest v2
          • About Pinterest v2 Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Pinterest v2 with Daton
          • Schema Information
        • Rakuten
          • About Rakuten Connector
          • Essential Attributes
          • How to Integrate Rakuten with Daton
        • StackAdapt
          • About StackAdapt Platform
          • Essential Attributes and Supported Tables
          • How to Integrate StackAdapt with Daton
          • Schema Information
        • Typeform
          • About Typeform Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Typeform with Daton
          • Schema Information
        • Upscribe
          • Release Notes
        • Upscribe V2
          • About Upscribe Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Upscribe v2 with Daton
          • Schema Information
      • Analytics Platforms
        • Appsflyer
        • GA4
          • About Google Analytics Platform
          • Essential Attributes and Table Selection
          • How to Integrate GA4 with Daton
          • Schema Information
        • Google Analytics
          • Release Notes
        • Jungle Scout
        • Northbeam
          • About Northbeam Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Northbeam with Daton
          • Schema Information
        • Prodco Analytics
          • About Prodco Analytics Platform
          • Essential Attributes and Table Selection
          • How to Integrate Prodco Analytics Account with Daton
          • Schema Information
        • Rainforest
          • About Rainforest Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Rainforest Account with Daton
          • Schema Information
        • Sprout Social
          • About Sprout Social Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Sprout Social with Daton
          • Schema Information
        • Taboola
        • Taboola v2
          • About Taboola v2
          • Essential Attributes and Supported Tables
          • How to Integrate Taboola v2 with Daton
          • Schema Information
        • Edge by Ascential
      • Business
        • ClickUp
          • About ClickUp Connector
          • Essential Attributes and Supported Tables
          • How to Integrate ClickUp with Daton
          • Schema Information
        • Bubble
        • Pipedrive
          • About Pipedrive Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Pipedrive with Daton
          • Schema Information
        • Teamwork
          • About Teamwork Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Teamwork with Daton
          • Schema Information
        • Teamwork v2
          • About Teamwork Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Teamwork v2 with Daton
          • Schema Information
        • TikTok Business
          • About TikTok Business Connector
          • Essential Attributes and Supported Tables
          • How to Integrate TikTok Business with Daton
          • Schema Information
        • Unbounce v2
          • About Unbounce v2 Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Unbounce v2 with Daton
          • Schema Information
      • Call Center
        • Aircall
        • Callpage
        • JustCall
          • About JustCall Connector
          • Essential Attributes and Supported Tables
          • How to Integrate JustCall with Daton
          • Schema Information
      • Databases
        • Amazon Aurora MySQL
          • About Amazon Aurora Platform
          • Essential Attributes
          • How to Integrate Amazon Aurora with Daton
        • Amazon Redshift
          • About Amazon Redshift Platform
          • Essential Attributes
          • How to Integrate Amazon Redshift with Daton
        • GCP MySQL
          • About GCP MySQL Platform
          • Essential Attributes
          • How to Integrate GCP MySQL with Daton
        • GCP PostgreSQL
          • Setup Manual
        • Magento (DB)
          • About Magento Connector
          • Essential Attributes
          • How to Integrate Magento with Daton
        • MySQL
          • Release Notes
        • NetSuite
        • PostgreSQL
          • Release Notes
        • SQLServer
        • WooCommerce SQL
          • About WooCommerce SQL Platform
          • Essential Attributes
          • How to Integrate WooCommerce SQL with Daton
      • Daton Connector
        • Release Notes
      • Ecommerce Platforms
        • Amazon MWS
          • Release Notes
        • Amazon Selling Partner
          • All Report Types and Tables
          • Release Notes
        • Amazon Vendor Central
          • All Report Types and Tables
          • Real Time Reports
          • Daily Reports
          • Weekly Reports
          • Snapshot Reports
          • Monthly Reports
          • Raw Tables
          • Release Notes
        • Bigcommerce
        • BigCommerce v2
          • About BigCommerce Platform
          • Essential Attributes and Supported Tables
          • How to Integrate BigCommerce Account with Daton
          • Schema Information
          • Release Notes
        • Bol Retail v2
          • Release Notes
        • Capillary Martjack
          • Release Notes
        • Kaufland
          • About Kaufland Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Kaufland with Daton
          • Schema Information
        • Keepa
          • About Keepa Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Keepa with Daton
          • Schema Information
        • Lazada
        • Loopreturns
          • Release Notes
        • Magento
          • Release Notes
        • Magento 2 (APIs)
          • Release Notes
        • Mercado Libre
          • About Mercado Libre Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Mercado Libre with Daton
          • Schema Information
        • Shopify
          • Access tokens for Shopify
          • Release Notes
        • Shopify V2
        • Walmart
          • Release Notes
        • Walmart_v2
          • Release Notes
        • WooCommerce
      • Files
        • Amazon S3
          • Release Notes
        • Dropbox
          • Release Notes
        • Email Attachments
          • About Email Attachments Connector
          • Essential Attributes and Connector Guidelines
          • How to Integrate Email Attachments with Daton
        • Google Cloud Storage
          • Release Notes
        • Google Drive
        • Google Sheets
          • Release Notes
        • SFTP (SSH File Transfer Protocol)
      • OMS/WMS
        • Cin7
        • Goflow
          • About Goflow Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Goflow with Daton
          • Schema Information
        • Rippling
          • About Rippling Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Rippling with Daton
          • Schema Information
      • Payments
        • PayPal
          • About PayPal Connector
          • Essential Attributes and Supported Tables
          • How to Integrate PayPal with Daton
          • Schema Information
        • Razorpay
          • About Razorpay Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Razorpay with Daton
          • Schema Information
        • Razorpay v1
        • Stripe
      • Productivity Trackers
        • GitLab
          • Release Notes
        • Time Doctor
          • Release Notes
      • SaaS Integrations
        • Aftership
          • Release Notes
        • Aftership V2
        • Anvyl
        • Ascend
        • Awtomic
          • Release Notes
        • Braintree
          • About Braintree Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Braintree Account with Daton
          • Schema Information Braintree
        • Calendly
          • Release Notes
        • Calendly V2
        • Campaign Monitor
        • Capsule CRM
        • Chargebee
        • Freshworks CRM
        • Loop Subscription
        • Commission Junction
        • Constant Contact
        • Copper CRM
          • About Copper
          • Essential Attributes and Supported Tables
          • How to Integrate Copper Account with Daton
          • Schema Information
          • Release Notes
        • Criteo
        • Criteo Retail Media
        • Customer.io
        • DotDigital
        • dotdigital V2
        • Easyecom
        • EasyEcom V2
          • Release Notes
        • ePallet ERP
        • Etsy
        • Exchange Rates
          • About Exchange Rate Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Exchange Rates with Daton
          • Schema Information
          • Release Notes
        • Fairing (previously EnquireLabs)
          • Release Notes
        • FreshBooks
          • Release Notes
        • Freshdesk
          • Release Notes
        • Freshsales
          • Release Notes
        • Fulfil
        • Gladly
          • Release Notes
        • Google Search Console
          • Release Notes
        • HubSpot
          • Release Notes
        • HubSpot v2
          • About HubSpot v2 Connector
          • Essential Attributes and Supported Tables
          • How to Integrate HubSpot v2 with Daton
          • Schema Information
        • Impact
          • Release Notes
        • Infusionsoft (now Keap Max Classic)
          • About Infusionsoft Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Infusionsoft with Daton
          • Schema Information
        • Insightly
        • Intercom
          • Release Notes
        • Intercom V2
          • Release Notes
        • Outbrain V2 (Old)
          • Release Notes
        • Inventory Planner
        • Jira
        • JudgeMe
          • Release Notes
        • Kibo Commerce
        • Klaviyo
          • Release Notes
        • Klaviyo V2
          • Release Notes
        • Kustomer
        • LeadSquared CRM
        • LeadSquared V2
        • LinkedIn Ads
        • LiveChat
        • LoyaltyLion
        • MailChimp
          • Release Notes
        • MNTN
        • Monday
        • NewRelic
        • Okendo Reviews
          • About Okendo Reviews Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Okendo Reviews with Daton
          • Schema Information
        • Optimove
          • Release Notes
        • Pingdom
        • Pinterest
        • Pipedrive
        • PushEngage
        • Quickbase
        • QuickBooks
        • Quickbooks v2
          • About Quickbooks Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Quickbooks v2 with Daton
          • Schema Information
        • Re:amaze
        • ReCharge Payments
          • Release Notes
        • ReCharge Payments v1
          • Setup Manual
          • Schema Information
        • Recharge Payments V2
          • Setup Manual
          • Schema Information
        • Reviewmonitoring
          • Release Notes
        • Salsify
          • Release Notes
        • SendGrid
          • Release Notes
        • Shiphero
        • Shippo
        • Shiprocket
          • Release Notes
        • Shiprocket v2
          • About Shiprocket Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Shiprocket with Daton
          • Schema Information
        • Shipstation
          • Release Notes
        • Skubana
        • SkuVault
          • About SkuVault Connector
          • Essential Attributes and Supported Tables
          • How to Integrate SkuVault with Daton
          • Schema Information
        • Smartrr
        • Smartrr v2
          • About Smartrr Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Smartrr v2 with Daton
          • Schema Information
        • Stamped.io
          • About Stamped.io Platform
          • Essential Attributes and Supported Tables
          • How to Integrate Stamped.io with Daton
          • Schema Information
        • Stamped.io v1
        • Stay AI
        • Sticky.io
        • SugarCRM
          • Release Notes
        • Survey Monkey
        • Tune
        • Unbounce
        • Unicommerce
        • Unicommerce
          • About Unicommerce Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Unicommerce with Daton
          • Schema Information
          • Release Notes
        • When I Work
        • Xero
        • Xero V2
          • About Xero Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Xero V2 with Daton
          • Schema Information
        • Yahoo Gemini
        • Yotpo
        • Yotpo v2
          • About Yotpo Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Yotpo v2 with Daton
          • Schema Information
        • Zendesk Chat
        • Zendesk ETL
          • Release Notes
        • Zoho CRM
          • Release Notes
        • ZohoDesk
          • Release Notes
      • Surveys
        • Alchemer
          • About Alchemer Platform
          • Essential Attributes and Supported Tables Alchemer
          • How to Integrate Alchemer Account with Daton
          • Schema Information Alchemer
        • Fairing
          • About Fairing Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Fairing with Daton
          • Schema Information
        • Survicate
          • About Survicate Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Survicate with Daton
          • Schema Information
      • Shipping Platforms
        • 3PL Central
        • Byrd
        • Cin7
        • ClickPost
        • Dear
        • Easypost
        • Ohi
          • About Ohi Connector
          • Essential Attributes and Supported Tables
          • How to Integrate Ohi with Daton
          • Schema Information
        • ShipBob
          • About ShipBob Connector
          • Essential Attributes and Supported Tables
          • How to Integrate ShipBob with Daton
          • Schema Information
      • Webhooks
    • Destinations
      • Amazon RDS PostgreSQL
        • Amazon RDS for PostgreSQL Integration with Daton
        • How to Set up RDS PostgreSQL Destination in Daton
      • Amazon RDS MySQL
        • About Integrating Amazon RDS for MySQL database with Daton
        • How to Set up RDS MySQL Destination in Daton
      • Amazon Redshift
        • Querying Daton created tables on RedShift
        • Release Notes
      • Amazon S3
        • Release Notes
      • Azure One Lake
        • About Azure One Lake as a connector in Daton
        • How to Set up an Azure One Lake Database for Integrating with Daton
        • How to Integrate your Azure One Lake Database as a Destination in Daton
      • Azure Data Lake Storage
        • About Azure Data Lake Storage as a connector in Daton
        • How to Set up an Azure Data Lake Storage for Integrating with Daton
        • How to Integrate Your Azure Data Lake Storage Database as a Destination in Daton
      • GCP PostgreSQL
        • About Google Cloud SQL for PostgreSQL Database with Daton
        • How to Set up GCP PostgreSQL Destination in Daton
      • Google BigQuery
        • Release Notes
      • GCS
      • MySQL
        • About Integrating MySQL with Daton
        • How to Set up a MySQL Database for Integrating with Daton
        • How to Set up Your MySQL Database as a Destination in Daton
      • Oracle Autonomous Data Warehouse
      • Snowflake
        • Release Notes
  • FAQs and Troubleshooting
    • Connector permissions and API Keys
      • Access tokens for Shopify
      • Amazon Ads Permissions
      • Where to find my Amazon MWS Merchant Token and Auth Token
      • How to create BigCommerce API Account and generate Client Id, Client Secret and Access tokens
      • How to create Client credentials for Bol Retail
      • How to create Developer application for Mercado Libre
      • How to create API Keys for SendGrid
      • Where to find API Keys for Freshsales
      • Where to find API Keys for Freshdesk
      • How to create REST API Keys for NewRelic
      • How to create API Keys for Chargebee
      • How to generate Access Token for Calendly
      • How to obtain Klaviyo API Key
      • Where to find API Keys for RazorPay
      • How to create API Keys for PushEngage
      • How to create Criteo API User and generate Client Id and Client Secret
      • How to find site-id/domain and create WooCommerce Consumer Key and Secret
    • Authentication Failure while connecting Database using SSH tunnel
    • Common Errors and Resolution
    • FAQs
      • How to get access to FB Ads Business Manager?
      • How to get access to Google Analytics?
      • Querying Daton created child tables for nested data
      • Google Analytics - FAQs
      • How does Lite Plan Work?
      • How do I upgrade my subscription?
      • How does loading behaviour impact billing?
      • What are the tables highlighted with the letter "F"?
      • Amazon Advertising FAQs
      • Amazon Business and Inventory Reports FAQs
      • Why don't I see my data?
      • How exactly do Business Reports work in Daton?
      • Steps to upload your own ASIN and SKU file in Amazon SP connectors
      • What are 'FATAL' and 'CANCELLED' report statuses in Amazon Selling Partner?
      • How to setup Google BigQuery Project
      • Amazon Selling Partner, Vendor Central - Access to resource denied error.
      • VendorSalesReportByManufacturing showing a FATAL report status?
      • How to Enable SSH Tunneling during a Database Source Authentication?
  • New Subscription Plans and Billing
    • Subscription Plans Overview
    • Find Your Perfect Data Integration Fit - Growth and Enterprise Plans Explored
    • Daton's Subscriptions Dashboard: Harness the Power of Your Data
    • How to Select and Subscribe to a Daton Subscription Plan?
    • Subscription Plan Upgrade and Downgrade
    • Changing Billing Cycle
  • Product Insights: Report, Roadmap, and Release Notes
    • Introduction to Product Insights
    • 2023 Recap and 2024 Roadmap
    • Release Notes
Powered by GitBook
On this page
  • What is this connector?
  • What data is made available?
  • USER_V
  • SOURCE_V
  • WAREHOUSE_V
  • SOURCE_TABLES_V
  • SOURCE_TRANSACTION_V
  • SOURCE_LOG_RECENT_V
  • LAST_JOB_STAT_V
  • How can you use this data?
  • Tracking Jobs
  • Visibility into Last processed value for tables
  • Understanding Usage
  • Impact on Billing
  • Useful Queries
  • FAQ
  • How do I convert timestamp columns into a readable format?
  • Why is last record pointing to 1/1/1970 for some tables?
  • What is the “state” column in SOURCE_LOG_RECENT_V table?
  • What is the parallelism column in SOURCE_V table?
  • Why is one of my tables not in SOURCE_LOG_RECENT_V?

Was this helpful?

  1. Connectors
  2. Data Sources

Daton Connector

What is this connector?

Daton connector exposes the data that Daton generates once you start using it. In addition to the information we show in the Daton UI, users can configure the Daton connector to sync essential data to your warehouse so that users can write their own queries and build alerts on the data being published. Configuring Daton connector is like configuring any other connector in Daton.

What data is made available?

The following tables will be sync’ed to your data warehouse.

USER_V

This table provides a list of all users belonging to a company.

Schema

Column Name

Data Type

Description

userid (Primary Key)

numeric

Daton generated id of user

companyid

numeric

Daton generated id of user company

role

integer

0, 1 - admin. 2 - user

status

integer

0 – Created, 1 – Active, 2 - Inactive

created

numeric

Timestamp when user was created

updated

numeric

Timestamp when user info was updated

createdby

numeric

Created by

User_type

string

User or Admin

email

string

User email

First_name

string

User first name

Last_name

string

User last name

Company_name

string

User company name

_daton_user_id

numeric

Daton generated id of user. Same as userid column in user_v table

_daton_batch_runtime

numeric

Timestamp when a particular entry was processed

_daton_batch_id

numeric

Batch id of the processed entry

SOURCE_V

This table provides a list of all sources configured in Daton and their status.

Column Name

Data Type

Description

sourceid (Primary Key)

numeric

Daton generated id of the integration

userid

numeric

Daton generated id of the user

companyid

numeric

Daton generated id of user company

created

numeric

Timestamp when user was created

updated

numeric

Timestamp when user info was updated

warehouseid

numeric

Daton generated id of destination warehouse of source

parallelism

integer

1 if jobs are processed simultaneously, 0 if jobs are processed sequentially

name

string

Name of integration

frequency

string

Frequency of data replication of the integration – once in the number of hours mentioned

history

string

Number of years of historical data from when the integration was configured

status

string

0 – Created, 1 – Disabled, 2 – Active, 3 – Updated, 4 – Configured, 5 – Deleted, 6 - Paused

type

string

Type of source

destinationtype

string

Type of destination (warehouse)

_daton_user_id

numeric

Daton generated id of user

_daton_batch_runtime

numeric

Timestamp when a particular entry was processed

_daton_batch_id

numeric

Batch id of the processed entry

WAREHOUSE_V

This table provides a list of all warehouses configured in Daton and their status.

Column Name

Data Type

Description

warehouseid (Primary Key)

numeric

Daton generated id of destination warehouse

userid

numeric

Daton generated id of the user

companyid

numeric

Daton generated id of user company

updated

numeric

Timestamp when warehouse info was updated

created

numeric

Timestamp when warehouse was created

type

numeric

Type of warehouse

name

string

User generated name of warehouse

status

string

0 – Created, 1 – Disabled, 2 – Enabled, 3 – Updated, 4 - Configuring

_daton_user_id

numeric

Daton generated id of user.

_daton_batch_runtime

numeric

Timestamp when a particular entry was processed

_daton_batch_id

numeric

Batch id of the processed entry

SOURCE_TABLES_V

This table provides a list of all the tables belonging to the sources configured in Daton and their status.

Column Name

Data Type

Description

maxdepthofnesting

integer

Maximum level of nesting in the table

fullload

boolean

If a job is processed every time along with all of the history it is a full load, else it is an incremental load when only new rows are added

updated

numeric

Timestamp when table was updated

created

numeric

Timestamp when table was created

sourceid (Primary Key)

numeric

Daton generated id of the integration

userid

numeric

Daton generated id of the user

companyid

numeric

Daton generated id of user company

tablename (Primary Key)

string

Name of table

tableschema

string

Schema of table

frequency

string

Frequency of data replication of the table – once in the number of hours mentioned

history

string

Number of years of historical data from when the integration was configured

status

string

0 – Created, 1 – Disabled, 2 – Active, 3 – Updated, 4 – Configured, 5 – Deleted, 6 - Paused

modificationtype

string

None, created, updated, dropped or drop_create

updatemode

string

Inherit, upsert, insert, write_truncate

_daton_user_id

numeric

Daton generated id of user

_daton_batch_runtime

numeric

Timestamp when a particular entry was processed

_daton_batch_id

numeric

Batch id of the processed entry

SOURCE_TRANSACTION_V

This table provides a history of all the jobs executed for tables configured in Daton starting from 1-Jan-2022.

Column Name

Data Type

Description

transactionid (Primary Key)

numeric

Daton generated id of job

sourceid

numeric

Daton generated id of integration

userid

numeric

Daton generated id of user

companyid

numeric

Daton generated id of user company

starttime

numeric

Timestamp when job started

endtime

numeric

Timestamp when job ended

records

numeric

Number of records processed

updated

numeric

Latest primary key from source used to upsert/append data (usually a timestamp)

created

numeric

Timestamp when job was created

errorcode

integer

Error code

transactionlogid

numeric

Id of the number of jobs replicated for that table

errortype

integer

Error type

callstatus

integer

-1 for when job failed and 0 for when job succeeded.

lastrecord

numeric

Last updated primary key of the table if job failed. (Usually timestamp)

scheduledat

numeric

Timestamp of when the job was scheduled

error

string

Error

tablename

string

Name of the table in the integration

_daton_user_id

numeric

Daton generated id of user

_daton_batch_runtime

numeric

Timestamp when a particular entry was processed

_daton_batch_id

numeric

Batch id of the processed entry

SOURCE_LOG_RECENT_V

This table provides the value of the last record replicated by Daton for each of the table configured for replication.

Column Name

Data Type

Description

sourceid (Primary Key)

numeric

Daton generated id of integration

lastrecord

numeric

Timestamp of when the last record was replicated in a table

created

numeric

Timestamp when job was created

updated

numeric

Timestamp when table was updated

status

integer

0 for inactive, 1 for active

sourcelogrecentid

numeric

Id of job in source_log_recent

sourcelogid

numeric

Id of job in source_log

userid

numeric

Daton generated id of user

companyid

numeric

Daton generated id of user company

tablename (Primary Key)

string

Name of table in the integration

state

string

Json describing additional information of job and job state

_daton_user_id

numeric

Daton generated id of user

_daton_batch_runtime

numeric

Timestamp when a particular entry was processed

_daton_batch_id

numeric

Batch id of the processed entry

LAST_JOB_STAT_V

This table provides the data related to the last job replicated by Daton for each of the table configured for replication.

Column Name

Data Type

Description

statid

numeric

stat id

userid

numeric

Daton generated id of user

companyid

numeric

Daton generated id of user company

sourceid (Primary Key)

numeric

Daton generated id of integration

lasterrorcode

integer

Error code if any in the last job. 0 if there was no error.

laststarttime

numeric

Timestamp of the start of last job

lastendtime

numeric

Timestamp of the end of last job

lastrecordcount

numeric

Count of records processed in the last job

totalrecordcount

numeric

Total number of records processed/in the table

sourceloglastrecord

numeric

Timestamp up to which data has been replicated in the last job.

lastscheduledat

numeric

Last scheduled timestamp of job

nextfiretime

numeric

Timestamp of when next job will be fired

created

numeric

Timestamp when this particular row was created

updated

numeric

Timestamp when this particular row was updated

tablename (Primary Key)

string

Name of table in the integration

lasterror

string

Error in the last job if any. Empty if no error.

_daton_user_id

numeric

Daton generated id of user

_daton_batch_runtime

numeric

Timestamp when this particular row was processed

_daton_batch_id

numeric

Batch id of this row

How can you use this data?

Tracking Jobs

This connector exposes data related to jobs being run by Daton. Users can analyze this data to understand what jobs are being run, at what time, and become more self-sufficient in troubleshooting in case they notice an issue.

Visibility into Last processed value for tables

Users can get visibility into the last replicated value for each table by going into the integration details page and checking the replicated up to value for any table. However, using the UI can be difficult when you are trying to analyze replicated up to values for tables across multiple integrations. Data from this connector can be synced to your warehouse and SQL queries can be written to analyze the data.

Understanding Usage

Users on usage-based billing plans can analyze which tables and integrations are contributing to their monthly bill. This data can be leverage to fine tune the setups to bring overages down.

Impact on Billing

Adding Daton connect will be counted towards your quota of available connector in your plan. Data replicated by this connector counts towards your monthly quota for usage-based billing plans.

Useful Queries

/*Get a list of sources by user*/ select * from SOURCE_V s , USER_V u where s.userid = u.userid ; /*Last job status of a table.*/ select s.name , ljs.* from SOURCE_V s , LAST_JOB_STAT_V ljs where s.sourceid = ljs.sourceid and s.name = 'INTEGRATION_NAME' and ljs.tablename = 'TABLE_NAME' ; /*History of jobs for a table*/ select * from SOURCE_V s , SOURCE_TRANSACTION_V st where s.sourceid = st.sourceid and s.name = 'INTEGRATION_NAME' and st.tablename = 'TABLE_NAME' ;

FAQ

How do I convert timestamp columns into a readable format?

BigQuery

  • SELECT TIMESTAMP_MILLIS(CAST(COLUMN_NAME AS INT64)) FROM TABLE NAME

Redshift

  • SELECT  timestamp 'epoch' + CAST(COLUMN_NAME  AS BIGINT)/1000 * interval '1 second' AS date FROM TABLE NAME

Snowflake

  • select to_timestamp(daton_batch_runtime ) FROM table_name

Why is last record pointing to 1/1/1970 for some tables?

This happens in two cases.

  1. When the source is a full load table

  2. When the first job hasn’t run yet for a table

What is the “state” column in SOURCE_LOG_RECENT_V table?

Daton uses this column to store additional job-related data for some sources. If most cases, you don’t need to use this data as it is primarily for internal user. Reach out to support if you have any questions.

What is the parallelism column in SOURCE_V table?

Daton uses this column to determine how many jobs run in parallel for the source.

Why is one of my tables not in SOURCE_LOG_RECENT_V?

Although this scenario is very rate, this can happen when the table never managed even a single successful run.

PreviousHow to Integrate WooCommerce SQL with DatonNextRelease Notes

Last updated 2 years ago

Was this helpful?