12 min to read
GitLab CI/CD Template Guide
Optimizing Pipeline Implementation Using GitLab CI Templates
Overview
Learn how to optimize your GitLab CI implementation using templates. Templates help reduce code duplication and maintain consistent pipeline configurations across projects.
Using GitLab CI Templates
Built-in Templates
# Reference GitLab's built-in templates
include:
- template: Terraform/Base.gitlab-ci.yml
- template: Jobs/SAST-IaC.gitlab-ci.yml
Project Templates
If the template is defined in your gitlab’s project, you can write it as follows.
# Reference templates from your project
include:
- project: 'somaz94/server'
ref: master
file: '/template/my_template_file.yml'
- It is irrelevant to define it in
.gitlab-ci.ymlwithout reference.
🔧 Creating Custom Templates
Build Template Example
Using Build Template
And it can be used as below.
build_manual_image:
<<: *common_job_config
stage: build
image:
name: gcr.io/kaniko-project/executor:v1.22.0-debug
before_script: *common_build_before_script
script: *common_build_script
artifacts:
paths:
- build_status.txt
- service_status.txt
rules:
- if: '($CI_PIPELINE_SOURCE == "web")'
tags:
- build-image
And it can be used for other jobs.
.change_files_game: &change_files_game
changes:
- apps/game/src/**/*
build_auto_image_game:
<<: *common_job_config
stage: build
image:
name: gcr.io/kaniko-project/executor:v1.22.0-debug
before_script: *common_build_before_script
script: *common_build_script
variables:
SERVICE: $GAME_SERVICE
IMAGE_URL: $IMAGE_URL_GAME
IMAGE_URL_LATEST: $IMAGE_URL_LATEST_GAME
artifacts:
paths:
- build_status.txt
- service_status.txt
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
<<: *change_files_game
tags:
- build-image
Slack Notification Template
The template for sending a message to SLACK can also be written as follows. You can receive the result as an artifact for each job and send a message.
.common_update_after_script: &common_update_after_script
- >
if [ "$CI_JOB_STATUS" = "success" ]; then
echo "✅ success" > /builds/somaz94/server/update_status.txt
elif [ "$CI_JOB_STATUS" = "failed" ]; then
echo "❌ failed" > /builds/somaz94/server/update_status.txt
elif [ "$CI_JOB_STATUS" = "canceled" ]; then
echo "⚠️ canceled" > /builds/somaz94/server/update_status.txt
else
echo "🔍 unknown" > /builds/somaz94/server/update_status.txt
...
update_manual_image:
stage: update
image: alpine:latest
interruptible: true
retry:
max: 2 # Maximum of 2 retries
when:
- runner_system_failure
- unknown_failure
before_script: *common_update_before_script
script: *common_update_script
after_script: *common_update_after_script
artifacts:
paths:
- update_status.txt
rules:
- if: $CI_PIPELINE_SOURCE == "web"
tags:
- deploy-image
dependencies:
- build_manual_image
I used the SLACK webhook, and I used the workflow builder to write so that I could only get the result value.
Notification Script Template
Create a Slack notify template.
Using Notification Template
And write the yml file as below.
notify_slack:
stage: notify
image: curlimages/curl:latest
script: *common_notify_slack_script
rules:
- if: $CI_PIPELINE_SOURCE == "web"
- if: $CI_PIPELINE_SOURCE == "trigger"
tags:
- deploy-image
when: always
...
notify_slack_auto_game:
stage: notify
image: curlimages/curl:latest
script: *common_notify_slack_script
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
<<: *change_files_game
tags:
- deploy-image
when: always
dependencies:
- build_auto_image_game
- update_auto_image_game
If used in this way, a template can be created and applied to various jobs as follows.
More Template Examples
Docker Build Template with Multiple Stages
This template handles Docker builds with dedicated stages for building, testing, and pushing:
.docker_build_template: &docker_build_template
image: docker:20.10.16
services:
- docker:20.10.16-dind
variables:
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_BUILDKIT: 1
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- export IMAGE_TAG=${CI_COMMIT_REF_SLUG}-${CI_COMMIT_SHORT_SHA}
- export FULL_IMAGE_NAME=$CI_REGISTRY_IMAGE:$IMAGE_TAG
script:
- echo "Building image $FULL_IMAGE_NAME"
- >
docker build
--build-arg BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
--build-arg VCS_REF=$CI_COMMIT_SHORT_SHA
--build-arg VERSION=$CI_COMMIT_REF_NAME
-t $FULL_IMAGE_NAME
-t $CI_REGISTRY_IMAGE:latest
.
- docker push $FULL_IMAGE_NAME
- docker push $CI_REGISTRY_IMAGE:latest
Multi-Environment Deployment Template
Template for deploying to different environments based on conditions:
.deploy_template: &deploy_template
image:
name: bitnami/kubectl:latest
entrypoint: [""]
variables:
GIT_STRATEGY: none
before_script:
- kubectl config use-context $KUBE_CONTEXT
- |
if [[ -n "$NAMESPACE" ]]; then
# Ensure namespace exists
kubectl get namespace $NAMESPACE || kubectl create namespace $NAMESPACE
fi
# Development deployment
.deploy_dev: &deploy_dev
<<: *deploy_template
variables:
NAMESPACE: development
REPLICAS: 1
RESOURCES_REQUESTS_CPU: 100m
RESOURCES_REQUESTS_MEMORY: 128Mi
RESOURCES_LIMITS_CPU: 200m
RESOURCES_LIMITS_MEMORY: 256Mi
script:
- echo "Deploying to development environment"
- envsubst < k8s/dev/deployment.yml | kubectl apply -f -
- kubectl rollout status deployment/$CI_PROJECT_NAME -n $NAMESPACE
environment:
name: development
url: https://dev.example.com
only:
- develop
# Production deployment
.deploy_prod: &deploy_prod
<<: *deploy_template
variables:
NAMESPACE: production
REPLICAS: 3
RESOURCES_REQUESTS_CPU: 200m
RESOURCES_REQUESTS_MEMORY: 256Mi
RESOURCES_LIMITS_CPU: 400m
RESOURCES_LIMITS_MEMORY: 512Mi
script:
- echo "Deploying to production environment"
- envsubst < k8s/prod/deployment.yml | kubectl apply -f -
- kubectl rollout status deployment/$CI_PROJECT_NAME -n $NAMESPACE
environment:
name: production
url: https://example.com
only:
- main
when: manual
Test Templates for Different Languages
Templates for running tests in different languages:
# Node.js testing template
.node_test_template: &node_test_template
image: node:16-alpine
cache:
key: ${CI_COMMIT_REF_SLUG}-node
paths:
- node_modules/
before_script:
- npm ci
script:
- npm run lint
- npm run test:ci
artifacts:
reports:
junit: junit-reports/*.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura-coverage.xml
# Python testing template
.python_test_template: &python_test_template
image: python:3.9-slim
cache:
key: ${CI_COMMIT_REF_SLUG}-python
paths:
- .pip-cache/
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.pip-cache"
PYTHONPATH: "$CI_PROJECT_DIR"
before_script:
- python -m pip install --upgrade pip
- pip install -r requirements.txt
- pip install pytest pytest-cov
script:
- python -m pytest --cov=./ --cov-report=xml --junitxml=junit.xml
artifacts:
reports:
junit: junit.xml
coverage_report:
coverage_format: cobertura
path: coverage.xml
Advanced Template Organization
Splitting Templates into Files
For larger projects, you can organize templates in separate files:
project-root/
├── .gitlab-ci.yml
└── .gitlab/
├── ci/
│ ├── templates/
│ │ ├── build.yml
│ │ ├── test.yml
│ │ ├── deploy.yml
│ │ └── notify.yml
│ └── jobs/
│ ├── frontend.yml
│ └── backend.yml
Main .gitlab-ci.yml:
include:
- local: '.gitlab/ci/templates/build.yml'
- local: '.gitlab/ci/templates/test.yml'
- local: '.gitlab/ci/templates/deploy.yml'
- local: '.gitlab/ci/templates/notify.yml'
- local: '.gitlab/ci/jobs/frontend.yml'
- local: '.gitlab/ci/jobs/backend.yml'
stages:
- build
- test
- deploy
- notify
variables:
GLOBAL_VAR: "value"
Creating Pipeline Template Library
For shared templates across multiple projects, create a dedicated template repository:
# In your template repository
# templates/docker-build.yml
.docker_build:
image: docker:latest
services:
- docker:dind
variables:
DOCKER_TLS_CERTDIR: "/certs"
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG .
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
Usage in your project:
include:
- project: 'your-group/gitlab-ci-templates'
ref: main
file: '/templates/docker-build.yml'
build_image:
extends: .docker_build
# Add custom configuration here
Generating Dynamic Templates
Template with Environment Variables
Create templates with variables that can be substituted:
.deploy_template: &deploy_template
script:
- echo "Deploying to ${ENVIRONMENT} environment"
- |
cat > kubernetes/${ENVIRONMENT}/kustomization.yaml << EOF
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: ${NAMESPACE}
resources:
- deployment.yaml
- service.yaml
images:
- name: app-image
newName: ${CI_REGISTRY_IMAGE}
newTag: ${CI_COMMIT_SHORT_SHA}
EOF
- kubectl apply -k kubernetes/${ENVIRONMENT}/
Matrix Jobs with Templates
Use GitLab’s matrix feature to generate multiple jobs from a template:
.test_template: &test_template
stage: test
script:
- echo "Running tests for ${PYTHON_VERSION} on ${OS}"
- pip install -r requirements.txt
- pytest
python-tests:
parallel:
matrix:
- PYTHON_VERSION: ["3.8", "3.9", "3.10"]
OS: ["linux", "windows"]
image: python:${PYTHON_VERSION}-${OS}
<<: *test_template
Template Best Practices
Template Design Principles:
1. Reusability: - Make templates generic enough to be used in different contexts
- Use variables for customizable parts
- Document required variables and their formats
- Avoid hardcoding project-specific values
2. Maintainability: - Keep templates focused on specific responsibilities
- Add comments to explain complex logic
- Version templates with semantic versioning
- Use descriptive names for templates and anchors
3. Performance: - Keep templates lightweight
- Use caching where appropriate
- Consider pipeline execution time when designing templates
- Avoid unnecessary logic or commands
4. Fault Tolerance: - Include proper error handling
- Add retry mechanisms for network-dependent operations
- Provide informative error messages
- Validate inputs and prerequisites
Troubleshooting Templates
Common Template Issues:
1. Reference Errors: - Verify anchor names match their references
- Check for typos in variable names
- Ensure templates are included before they're referenced
- Look for YAML indentation issues
2. Variable Scope Problems: - Remember that job-level variables override global ones
- Use YAML anchors to share variable definitions
- Consider how variables interact with included templates
- Watch for CI_* variable name collisions
3. Include Issues: - Verify project paths and file names in includes
- Check access permissions for included projects
- Ensure referenced branches or tags exist
- Look for circular includes
# Debug job for CI variables
debug_variables:
stage: .pre
image: alpine:latest
script:
- echo "== CI Variables =="
- env | grep "^CI_" | sort
- echo "== Custom Variables =="
- env | grep -v "^CI_" | sort
rules:
- if: '$CI_PIPELINE_SOURCE == "web"'
when: manual
tags:
- linux
Comments