GitLab
GitLab ETL connector for data replication
Snapshot
| Features | Details |
|---|---|
Release Status | Released |
Source API Version | V4 |
Table Selection | Yes |
Column Selection | Yes |
Edit Integration | Yes |
Replication Type Selection | No |
Authentication Parameters | GitLab Company Name GitLab API Key |
Replication Type | Full Replication Key Based Incremental |
Replication Key | Date |
Suggested Replication Frequency | 1 hour |
Tables/APIs supported
Projects | Groups |
Epics | Project Milestones |
Group Milestones | Releases |
Tags | Todos |
Issues | Comments |
Commits | Merge Requests |
Branches | Users |
Integrate GitLab with Daton
Sign in to Daton
Select GitLab from the list of Integrations
Provide Integration Name, Replication Frequency, and History. Integration name would be used in creating tables for the integration and cannot be changed later
Provide your GitLab company name and API Key
Post successful authentication, you will be prompted to choose from the list of available tables
Then select all required fields for each table
Submit the integration
Workflow
Integrations would be in Pending state initially and will be moved to Active state as soon as the first job loads data successfully on to the configured warehouse
Users would be able to edit/pause/re-activate/delete integration anytime
Users can view job status and process logs from the integration details page by clicking on the integration name from the active list
GitLab Data
Projects
Purpose: This API is used to fetch all the projects in your GitLab Account
Source API Documentation: https://docs.gitlab.com/ee/api/projects.html
Replication: Key Based Incremental
Replication Key: last_activity_after, last_activity_before
Fields
| Name | Data Type |
|---|---|
id | NUMERIC |
description | STRING |
name | STRING |
name_with_namespace | STRING |
path | STRING |
path_with_namespace | STRING |
created_at | TIMESTAMP |
default_branch | STRING |
tag_list | STRING |
topics | STRING |
ssh_url_to_repo | STRING |
http_url_to_repo | STRING |
web_url | STRING |
readme_url | STRING |
avatar_url | STRING |
forks_count | NUMERIC |
star_count | NUMERIC |
last_activity_at | TIMESTAMP |
namespace | RECORD |
Groups
Purpose: This API is used to fetch all the groups
Source API Documentation: https://docs.gitlab.com/ee/api/groups.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
|---|---|
id | NUMERIC |
name | STRING |
path | STRING |
description | STRING |
visibility | STRING |
share_with_group_lock | BOOLEAN |
require_two_factor_authentication | BOOLEAN |
two_factor_grace_period | NUMERIC |
project_creation_level | STRING |
subgroup_creation_level | STRING |
lfs_enabled | BOOLEAN |
default_branch_protection | NUMERIC |
avatar_url | STRING |
web_url | STRING |
request_access_enabled | BOOLEAN |
full_name | STRING |
full_path | STRING |
file_template_project_id | NUMERIC |
created_at | TIMESTAMP |
Epics (GitLab Premium Users Only)
Purpose: This API is used to fetch all the Epics under the groups. This is available only to GitLab Premium Self-managed, GitLab Premium SaaS and higher tiers
Source API Documentation: https://docs.gitlab.com/ee/api/epics.html
Replication: Key Based Incremental
Replication Key: updated_after, updated_before
Fields
| Name | Data Type |
|---|---|
id | NUMERIC |
iid | NUMERIC |
group_id | NUMERIC |
parent_id | NUMERIC |
parent_iid | NUMERIC |
title | STRING |
description | STRING |
state | STRING |
confidential | STRING |
web_url | STRING |
reference | STRING |
references | RECORD |
author | RECORD |
start_date_is_fixed | BOOLEAN |
end_date | DATE |
due_date | DATE |
due_date_is_fixed | BOOLEAN |
due_date_from_milestones | DATE |
due_date_from_inherited_source | DATE |
created_at | TIMESTAMP |
updated_at | TIMESTAMP |
closed_at | TIMESTAMP |
upvotes | NUMERIC |
downvotes | NUMERIC |
color | STRING |
_links | RECORD |
Project Milestones
Purpose: This API is used to fetch the milestones of all the projects
Source API Documentation: https://docs.gitlab.com/ee/api/milestones.html
Replication: Key Based Incremental
Replication Key: last_activity_after, last_activity_before
Fields
| Name | Data Type |
|---|---|
id | NUMERIC |
iid | NUMERIC |
project_id | NUMERIC |
title | STRING |
description | STRING |
due_date | DATE |
start_date | DATE |
state | STRING |
updated_at | TIMESTAMP |
created_at | TIMESTAMP |
expired | BOOLEAN |
Group Milestones
Purpose: This API is used to fetch milestones of all the groups
Source API Documentation: https://docs.gitlab.com/ee/api/group_milestones.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
|---|---|
id | NUMERIC |
iid | NUMERIC |
group_id | NUMERIC |
title | STRING |
description | STRING |
due_date | DATE |
start_date | DATE |
state | STRING |
updated_at | TIMESTAMP |
created_at | TIMESTAMP |
expired | BOOLEAN |
web_url | STRING |
Releases
Purpose: This API is used to fetch the releases project wise
Source API Documentation: https://docs.gitlab.com/ee/api/releases/
Replication: Key Based Incremental
Replication Key: last_activity_after, last_activity_before
Fields
| Name | Data Type |
|---|---|
tag_name | STRING |
description | STRING |
name | STRING |
created_at | TIMESTAMP |
released_at | TIMESTAMP |
author | RECORD |
commit | RECORD |
milestones | RECORD |
commit_path | STRING |
tag_path | STRING |
assets | RECORD |
evidences | RECORD |
Tags
Purpose: This API is used to get a list of repository tags from a project
Source API Documentation: https://docs.gitlab.com/ee/api/tags.html
Replication: Key Based Incremental
Replication Key: last_activity_after, last_activity_before
Fields
| Name | Data Type |
|---|---|
commit | RECORD |
release | RECORD |
name | STRING |
target | STRING |
protected | BOOLEAN |
project_id | STRING |
Todos
Purpose: This API is used to fetch all to-dos that are done and pending
Source API Documentation: https://docs.gitlab.com/ee/api/todos.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
|---|---|
id | NUMERIC |
project | RECORD |
author | RECORD |
action_name | STRING |
target_type | STRING |
target | RECORD |
target_url | STRING |
body | STRING |
state | STRING |
created_at | TIMESTAMP |
updated_at | TIMESTAMP |
Issues
Purpose: This API is used to fetch all the issues in the account
Source API Documentation: https://docs.gitlab.com/ee/api/issues.html
Replication: Key Based Incremental
Replication Key: updated_after, updated_before
Fields
| Name | Data Type |
|---|---|
state | STRING |
description | STRING |
author | RECORD |
milestone | RECORD |
project_id | NUMERIC |
assignees | RECORD |
assignee | RECORD |
type | STRING |
updated_at | TIMESTAMP |
id | NUMERIC |
title | STRING |
created_at | TIMESTAMP |
iid | NUMERIC |
labels | STRING |
upvotes | NUMERIC |
downvotes | NUMERIC |
merge_requests_count | NUMERIC |
user_notes_count | NUMERIC |
due_date | DATE |
web_url | STRING |
references | RECORD |
time_stats | RECORD |
has_tasks | BOOLEAN |
task_status | STRING |
confidential | BOOLEAN |
discussion_locked | BOOLEAN |
issue_type | STRING |
severity | STRING |
_links | RECORD |
task_completion_status | RECORD |
Comments
Purpose: This API is used to fetch all comments in issues, epics, snippets and merge requests
Source API Documentation: https://docs.gitlab.com/ee/api/notes.html
Replication: Key Based Incremental
Replication Key: updated_after, updated_before
Fields
| Name | Data Type |
|---|---|
id | NUMERIC |
body | STRING |
author | RECORD |
created_at | TIMESTAMP |
updated_at | TIMESTAMP |
system | BOOLEAN |
noteable_id | NUMERIC |
noteable_type | STRING |
noteable_iid | NUMERIC |
resolvable | BOOLEAN |
confidential | BOOLEAN |
internal | BOOLEAN |
Commits
Purpose: This API is used to fetch the commits made in all the projects. In commit responses, created_at and commited_date are identical.
Source API Documentation: https://docs.gitlab.com/ee/api/commits.html
Replication: Key Based Incremental
Replication Key: since, until
Fields
| Name | Data Type |
|---|---|
id | STRING |
short_id | STRING |
title | STRING |
author_name | STRING |
author_email | STRING |
authored_date | TIMESTAMP |
committer_name | STRING |
committer_email | STRING |
committed_date | TIMESTAMP |
created_at | TIMESTAMP |
message | STRING |
parent_ids | STRING |
web_url | STRING |
project_id | STRING |
Merge Requests
Purpose: This API is used to fetch all the merge requests made by all users
Source API Documentation: https://docs.gitlab.com/ee/api/merge_requests.html
Replication: Key Based Incremental
Replication Key: updated_after, updated_before
Fields
| Name | Data Type |
|---|---|
id | NUMERIC |
iid | NUMERIC |
project_id | NUMERIC |
title | STRING |
description | STRING |
state | STRING |
merged_by | RECORD |
merge_user | RECORD |
merged_at | TIMESTAMP |
created_at | TIMESTAMP |
updated_at | TIMESTAMP |
target_branch | STRING |
source_branch | STRING |
upvotes | NUMERIC |
downvotes | NUMERIC |
author | RECORD |
assignee | RECORD |
assignees | RECORD |
reviewers | RECORD |
source_project_id | NUMERIC |
target_project_id | NUMERIC |
labels | STRING |
draft | BOOLEAN |
work_in_progress | BOOLEAN |
milestone | RECORD |
merge_when_pipeline_succeeds | BOOLEAN |
merge_status | STRING |
detailed_merge_status | STRING |
sha | STRING |
user_notes_count | NUMERIC |
should_remove_source_branch | BOOLEAN |
force_remove_source_branch | BOOLEAN |
allow_collaboration | BOOLEAN |
allow_maintainer_to_push | BOOLEAN |
web_url | STRING |
references | RECORD |
time_stats | RECORD |
squash | BOOLEAN |
task_completion_status | RECORD |
Branches
Purpose: This API is used to fetch branches under all projects
Source API Documentation: https://docs.gitlab.com/ee/api/branches.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
|---|---|
name | STRING |
merged | BOOLEAN |
protected | BOOLEAN |
default | BOOLEAN |
developers_can_push | BOOLEAN |
developers_can_merge | BOOLEAN |
can_push | BOOLEAN |
web_url | STRING |
commit | RECORD |
project_id | STRING |
Users
Purpose: This API is used to fetch all users
Source API Documentation: https://docs.gitlab.com/ee/api/users.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
|---|---|
id | NUMERIC |
username | STRING |
name | STRING |
state | STRING |
avatar_url | STRING |
web_url | STRING |
Last updated