Amazon S3

Amazon S3 ETL connector for data replication

Features	Details
Release Status	Released
Table Selection	No
Column Selection	No
Edit Integration	Yes
Replication Type Selection	No
Replication Key	File Modified Date
Suggested Replication Frequency	1 hr

Integrate Amazon S3 with Daton

1. Signing to Daton

2. Select Amazon S3 from the list of Integrations

3. Provide Integration Name, Replication Frequency. Integration name would be used in creating tables for the integration and cannot be changed later

4. Provide AccessKey, SecretKey, and the AWS region where files are located

5. Post successful authentication, you will be prompted to enter a folder path

6. Select the file type - Daton supports CSV and XLS formats

7. Enter the row number where the columns names (headers) are present.

8. Then select required fields for each table.

9. Overwrite the column names and update any data types as required

10. Submit the integration

Daton processed files based on modified date and appends data to the tables in the warehouse. Customers have to de-dup data based on applicable business logic.

Workflow

1. Integrations would be in Pending state initially and will be moved to Active state as soon as the first job loads data successfully on to the configured warehouse

2. Users would be able to edit/pause/re-activate/delete integration anytime

3. Users can view job status and process logs from the integration details page by clicking on the integration name from the active list

Integration Parameters for S3

Step 1 - Navigate to IAM and click on the user who has Read access to the S3 buckets where files to be replicated are located.

Step 2 - Create an Access Key and Secret Key

Enter the access key and secret key in the Daton UI

Step 3 - Select the File Path

Navigate to your S3 buckets where files for replication reside

• Click on the Copy S3 URI button

• S3 URI copied will have the following format - "s3://amazon-rpa/OutputFiles/developer/Sales and Traffic by ASIN/"

• Remove the s3:// from the URL and enter the rest of the string the Daton UI.

• Select the row number in the file that has the column names.

Daton expects a Header row to be present in the file.

You will get an error if the parameters entered are incorrect. Reach out to us on chat or email us at [e-mail] if you have any issues.

Daton expects that the files in the folder have the same format. So, pplease make sure of this before creating the integration

Step 4 - Select the schema

Daton auto-detects the schema from the folder and extracts the column names and infers the data types.

Override the Daton detected values if necessary to control the schema in the warehouse.

Step 5 - Click submit to create the integration.

Daton will create the tables in the warehouse shortly and start processing data shortly after.