GitLab

GitLab ETL connector for data replication

Snapshot

Features
Details

Release Status

Released

Source API Version

V4

Table Selection

Yes

Column Selection

Yes

Edit Integration

Yes

Replication Type Selection

No

Authentication Parameters

GitLab Company Name GitLab API Key

Replication Type

Full Replication Key Based Incremental

Replication Key

Date

Suggested Replication Frequency

1 hour

Tables/APIs supported

Projects

Groups

Epics

Project Milestones

Group Milestones

Releases

Tags

Todos

Issues

Comments

Commits

Merge Requests

Branches

Users

Integrate GitLab with Daton

  1. Sign in to Daton

  2. Select GitLab from the list of Integrations

  3. Provide Integration Name, Replication Frequency, and History. Integration name would be used in creating tables for the integration and cannot be changed later

  4. Provide your GitLab company name and API Key

  5. Post successful authentication, you will be prompted to choose from the list of available tables

  6. Then select all required fields for each table

  7. Submit the integration

Workflow

  1. Integrations would be in Pending state initially and will be moved to Active state as soon as the first job loads data successfully on to the configured warehouse

  2. Users would be able to edit/pause/re-activate/delete integration anytime

  3. Users can view job status and process logs from the integration details page by clicking on the integration name from the active list

GitLab Data

Projects

Purpose: This API is used to fetch all the projects in your GitLab Account

Source API Documentation: https://docs.gitlab.com/ee/api/projects.html

Replication: Key Based Incremental

Replication Key: last_activity_after, last_activity_before

Fields

Name
Data Type

id

NUMERIC

description

STRING

name

STRING

name_with_namespace

STRING

path

STRING

path_with_namespace

STRING

created_at

TIMESTAMP

default_branch

STRING

tag_list

STRING

topics

STRING

ssh_url_to_repo

STRING

http_url_to_repo

STRING

web_url

STRING

readme_url

STRING

avatar_url

STRING

forks_count

NUMERIC

star_count

NUMERIC

last_activity_at

TIMESTAMP

namespace

RECORD

Groups

Purpose: This API is used to fetch all the groups

Source API Documentation: https://docs.gitlab.com/ee/api/groups.html

Replication: Full Replication

Replication Key: NA

Fields

Name
Data Type

id

NUMERIC

name

STRING

path

STRING

description

STRING

visibility

STRING

share_with_group_lock

BOOLEAN

require_two_factor_authentication

BOOLEAN

two_factor_grace_period

NUMERIC

project_creation_level

STRING

subgroup_creation_level

STRING

lfs_enabled

BOOLEAN

default_branch_protection

NUMERIC

avatar_url

STRING

web_url

STRING

request_access_enabled

BOOLEAN

full_name

STRING

full_path

STRING

file_template_project_id

NUMERIC

created_at

TIMESTAMP

Epics (GitLab Premium Users Only)

Purpose: This API is used to fetch all the Epics under the groups. This is available only to GitLab Premium Self-managed, GitLab Premium SaaS and higher tiers

Source API Documentation: https://docs.gitlab.com/ee/api/epics.html

Replication: Key Based Incremental

Replication Key: updated_after, updated_before

Fields

Name
Data Type

id

NUMERIC

iid

NUMERIC

group_id

NUMERIC

parent_id

NUMERIC

parent_iid

NUMERIC

title

STRING

description

STRING

state

STRING

confidential

STRING

web_url

STRING

reference

STRING

references

RECORD

author

RECORD

start_date_is_fixed

BOOLEAN

end_date

DATE

due_date

DATE

due_date_is_fixed

BOOLEAN

due_date_from_milestones

DATE

due_date_from_inherited_source

DATE

created_at

TIMESTAMP

updated_at

TIMESTAMP

closed_at

TIMESTAMP

upvotes

NUMERIC

downvotes

NUMERIC

color

STRING

_links

RECORD

Project Milestones

Purpose: This API is used to fetch the milestones of all the projects

Source API Documentation: https://docs.gitlab.com/ee/api/milestones.html

Replication: Key Based Incremental

Replication Key: last_activity_after, last_activity_before

Fields

Name
Data Type

id

NUMERIC

iid

NUMERIC

project_id

NUMERIC

title

STRING

description

STRING

due_date

DATE

start_date

DATE

state

STRING

updated_at

TIMESTAMP

created_at

TIMESTAMP

expired

BOOLEAN

Group Milestones

Purpose: This API is used to fetch milestones of all the groups

Source API Documentation: https://docs.gitlab.com/ee/api/group_milestones.html

Replication: Full Replication

Replication Key: NA

Fields

Name
Data Type

id

NUMERIC

iid

NUMERIC

group_id

NUMERIC

title

STRING

description

STRING

due_date

DATE

start_date

DATE

state

STRING

updated_at

TIMESTAMP

created_at

TIMESTAMP

expired

BOOLEAN

web_url

STRING

Releases

Purpose: This API is used to fetch the releases project wise

Source API Documentation: https://docs.gitlab.com/ee/api/releases/

Replication: Key Based Incremental

Replication Key: last_activity_after, last_activity_before

Fields

Name
Data Type

tag_name

STRING

description

STRING

name

STRING

created_at

TIMESTAMP

released_at

TIMESTAMP

author

RECORD

commit

RECORD

milestones

RECORD

commit_path

STRING

tag_path

STRING

assets

RECORD

evidences

RECORD

Tags

Purpose: This API is used to get a list of repository tags from a project

Source API Documentation: https://docs.gitlab.com/ee/api/tags.html

Replication: Key Based Incremental

Replication Key: last_activity_after, last_activity_before

Fields

Name
Data Type

commit

RECORD

release

RECORD

name

STRING

target

STRING

protected

BOOLEAN

project_id

STRING

Todos

Purpose: This API is used to fetch all to-dos that are done and pending

Source API Documentation: https://docs.gitlab.com/ee/api/todos.html

Replication: Full Replication

Replication Key: NA

Fields

Name
Data Type

id

NUMERIC

project

RECORD

author

RECORD

action_name

STRING

target_type

STRING

target

RECORD

target_url

STRING

body

STRING

state

STRING

created_at

TIMESTAMP

updated_at

TIMESTAMP

Issues

Purpose: This API is used to fetch all the issues in the account

Source API Documentation: https://docs.gitlab.com/ee/api/issues.html

Replication: Key Based Incremental

Replication Key: updated_after, updated_before

Fields

Name
Data Type

state

STRING

description

STRING

author

RECORD

milestone

RECORD

project_id

NUMERIC

assignees

RECORD

assignee

RECORD

type

STRING

updated_at

TIMESTAMP

id

NUMERIC

title

STRING

created_at

TIMESTAMP

iid

NUMERIC

labels

STRING

upvotes

NUMERIC

downvotes

NUMERIC

merge_requests_count

NUMERIC

user_notes_count

NUMERIC

due_date

DATE

web_url

STRING

references

RECORD

time_stats

RECORD

has_tasks

BOOLEAN

task_status

STRING

confidential

BOOLEAN

discussion_locked

BOOLEAN

issue_type

STRING

severity

STRING

_links

RECORD

task_completion_status

RECORD

Comments

Purpose: This API is used to fetch all comments in issues, epics, snippets and merge requests

Source API Documentation: https://docs.gitlab.com/ee/api/notes.html

Replication: Key Based Incremental

Replication Key: updated_after, updated_before

Fields

Name
Data Type

id

NUMERIC

body

STRING

author

RECORD

created_at

TIMESTAMP

updated_at

TIMESTAMP

system

BOOLEAN

noteable_id

NUMERIC

noteable_type

STRING

noteable_iid

NUMERIC

resolvable

BOOLEAN

confidential

BOOLEAN

internal

BOOLEAN

Commits

Purpose: This API is used to fetch the commits made in all the projects. In commit responses, created_at and commited_date are identical.

Source API Documentation: https://docs.gitlab.com/ee/api/commits.html

Replication: Key Based Incremental

Replication Key: since, until

Fields

Name
Data Type

id

STRING

short_id

STRING

title

STRING

author_name

STRING

author_email

STRING

authored_date

TIMESTAMP

committer_name

STRING

committer_email

STRING

committed_date

TIMESTAMP

created_at

TIMESTAMP

message

STRING

parent_ids

STRING

web_url

STRING

project_id

STRING

Merge Requests

Purpose: This API is used to fetch all the merge requests made by all users

Source API Documentation: https://docs.gitlab.com/ee/api/merge_requests.html

Replication: Key Based Incremental

Replication Key: updated_after, updated_before

Fields

Name
Data Type

id

NUMERIC

iid

NUMERIC

project_id

NUMERIC

title

STRING

description

STRING

state

STRING

merged_by

RECORD

merge_user

RECORD

merged_at

TIMESTAMP

created_at

TIMESTAMP

updated_at

TIMESTAMP

target_branch

STRING

source_branch

STRING

upvotes

NUMERIC

downvotes

NUMERIC

author

RECORD

assignee

RECORD

assignees

RECORD

reviewers

RECORD

source_project_id

NUMERIC

target_project_id

NUMERIC

labels

STRING

draft

BOOLEAN

work_in_progress

BOOLEAN

milestone

RECORD

merge_when_pipeline_succeeds

BOOLEAN

merge_status

STRING

detailed_merge_status

STRING

sha

STRING

user_notes_count

NUMERIC

should_remove_source_branch

BOOLEAN

force_remove_source_branch

BOOLEAN

allow_collaboration

BOOLEAN

allow_maintainer_to_push

BOOLEAN

web_url

STRING

references

RECORD

time_stats

RECORD

squash

BOOLEAN

task_completion_status

RECORD

Branches

Purpose: This API is used to fetch branches under all projects

Source API Documentation: https://docs.gitlab.com/ee/api/branches.html

Replication: Full Replication

Replication Key: NA

Fields

Name
Data Type

name

STRING

merged

BOOLEAN

protected

BOOLEAN

default

BOOLEAN

developers_can_push

BOOLEAN

developers_can_merge

BOOLEAN

can_push

BOOLEAN

web_url

STRING

commit

RECORD

project_id

STRING

Users

Purpose: This API is used to fetch all users

Source API Documentation: https://docs.gitlab.com/ee/api/users.html

Replication: Full Replication

Replication Key: NA

Fields

Name
Data Type

id

NUMERIC

username

STRING

name

STRING

state

STRING

avatar_url

STRING

web_url

STRING

Last updated