January 23, 2025 13 min to read

Setting up GitHub Action Self-Hosted Runner on Kubernetes

A comprehensive guide to implementing GitHub Action Hosted Runner with Helm

Overview

This guide explores how to create and manage GitHub Action Hosted Runners. We’ll focus on implementing self-hosted runners using Kubernetes and Helm.

Understanding GitHub Action Hosted Runners

Components Overview

1. actions-runner-controller

Manages GitHub Actions runners in Kubernetes clusters
Handles automatic provisioning and lifecycle management
Supports repository, organization, and enterprise-level deployment
Uses Kubernetes HPA for auto-scaling

2. gha-runner-scale-set-controller

Manages large-scale runner deployments
Utilizes Scale Set Runners feature
Provides efficient runner management through single endpoint
Integrates with GitHub's Scale Set Runners API

3. gha-runner-scale-set

Configures and deploys runner scale sets
Controls scaling and lifecycle through GitHub Actions API
Optimizes communication between workflows and runners

Self-Hosted vs. GitHub-Hosted Runners Comparison

Self-Hosted Runners:

Advantages: - Custom hardware specifications
- Access to internal network resources
- No minute limitations
- Better performance for specific workloads
- More control over the environment

Disadvantages: - Requires maintenance and updates
- Security considerations for public repositories
- Infrastructure costs
- Setup and configuration overhead

GitHub-Hosted Runners:

Advantages: - Zero maintenance
- Automatic updates
- Clean environment for each job
- Simple setup

Disadvantages: - Limited job execution time (6 hours)
- Limited storage and CPU
- Minute limitations on free/paid plans
- No access to internal network resources

Prerequisites

Authentication Setup

Choose one of the following authentication methods:

Option 1: GitHub PAT Token

Github Organization/Account settings → Settings → Developer settings → GitHub Personal access tokens

PAT Token is required for the runner to authenticate with GitHub.
Below is an example of PAT Token permissions.

Required scopes:

Repository Level: repo, workflow
Organization Level: read:org, admin:org, workflow
Enterprise Level: admin:enterprise, workflow

Option 2: GitHub App

If you have an existing app

Click the app
Check “App ID” in “About” section

If you don’t have an existing app

Create new GitHub App
Input the required Information
- GitHub App name
- Homepage URL
- Webhook URL (Optional)
- Repository permissions Settings:
  - Actions: Read and Write
  - Administration: Read and Write
  - Metadata: Read-only
Click”Create GitHub App”

Get Installation ID

Click “Install App”
Select the repository or organization
Click “Install”
Check “Installation ID” in “About” section
- https://github.com/organizations/YOUR-ORG/settings/installations/INSTALLATION_ID

Generate private key

Move App Settings Page
Click “Generate a private key”
Download the private key file

1.Required permissions:

Actions: Read and Write
Administration: Read and Write
Metadata: Read-only

2. Get App ID and Installation ID 3. Generate private key

Installation

1. Add Helm Repository

helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller

2. Create Namespace

kubectl create namespace actions-runner-system

3. Install Controller

4. Deploy Runner

kubectl apply -f runner-cr.yaml

Configuration Files

`runner.yaml`

authSecret:
  enabled: true
  create: true
  name: "controller-manager"
  github_token: "ghp_xxxxxxxxxxxxxxxxxxxxxxxx"

image:
  repository: "summerwind/actions-runner-controller"
  actionsRunnerRepositoryAndTag: "summerwind/actions-runner:latest"
  dindSidecarRepositoryAndTag: "docker:dind"
  pullPolicy: IfNotPresent

rbac:
  allowGrantingKubernetesContainerModePermissions: true

serviceAccount:
  create: true

`runner-cr.yaml`

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: example-runner
spec:
  replicas: 1
  template:
    spec:
      repository: "your-github-username/your-repository"
      labels:
        - self-hosted
        - linux
        - x64

Advanced Runner Configurations

1. Auto-Scaling Runners

Create a Horizontal Pod Autoscaler (HPA) for runners:

apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: runner-autoscaler
spec:
  scaleTargetRef:
    name: example-runner
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: TotalNumberOfQueuedAndInProgressWorkflowRuns
    repositoryNames:
    - your-github-username/your-repository

2. Organization-Level Runners

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: org-runner
spec:
  replicas: 2
  template:
    spec:
      organization: "your-organization"
      labels:
        - org-runner
        - linux
        - x64

3. Runner with Docker-in-Docker (DinD) Support

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: docker-runner
spec:
  replicas: 1
  template:
    spec:
      repository: "your-github-username/your-repository"
      image: summerwind/actions-runner:latest
      dockerEnabled: true
      labels:
        - docker
        - linux

4. Ephemeral Runners (Run once and terminate)

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: ephemeral-runner
spec:
  replicas: 1
  template:
    spec:
      repository: "your-github-username/your-repository"
      ephemeral: true
      labels:
        - ephemeral
        - linux

5. GitHub App Authentication Configuration

# GitHub App Secret
apiVersion: v1
kind: Secret
metadata:
  name: github-app-secret
  namespace: actions-runner-system
type: Opaque
data:
  app-id: "APP_ID_BASE64"
  installation-id: "INSTALLATION_ID_BASE64"
  private-key: "PRIVATE_KEY_BASE64"

# Controller configuration
authSecret:
  enabled: true
  create: false
  name: "github-app-secret"

Resource Management and Optimization

1. Runner Resource Configuration

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: resource-optimized-runner
spec:
  replicas: 1
  template:
    spec:
      repository: "your-github-username/your-repository"
      resources:
        limits:
          cpu: "2.0"
          memory: "4Gi"
        requests:
          cpu: "1.0"
          memory: "2Gi"
      labels:
        - resource-optimized

2. Runner with Node Selectors and Tolerations

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: dedicated-node-runner
spec:
  replicas: 1
  template:
    spec:
      repository: "your-github-username/your-repository"
      nodeSelector:
        node-type: actions-runner
      tolerations:
      - key: "dedicated"
        operator: "Equal"
        value: "actions-runner"
        effect: "NoSchedule"
      labels:
        - dedicated-node

Verification

Check Installation

kubectl get po -n actions-runner-system
kubectl get runnerdeployments.actions.summerwind.dev

Verify Runner Connection

Check runners in your repository: Settings → Actions → Runners

Example Workflow

name: Example Workflow
on:
  workflow_dispatch:

jobs:
  test-job:
    runs-on: self-hosted
    # or runs-on: [self-hosted, linux, x64]
    steps:
      - name: Test Step
        run: echo "Running on self-hosted runner"

Advanced Workflow Examples

Using Docker in Self-Hosted Runner

name: Docker Build and Test
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: [self-hosted, docker]
    steps:
      - uses: actions/checkout@v3
      
      - name: Build Docker image
        run: |
          docker build -t my-app:$ .
          
      - name: Test Docker image
        run: |
          docker run --rm my-app:$ npm test

Workflow with Multiple Runner Types

name: Multi-Runner Workflow
on:
  pull_request:
    branches: [main]

jobs:
  build:
    runs-on: [self-hosted, linux, build]
    steps:
      - uses: actions/checkout@v3
      - name: Build App
        run: npm ci && npm run build
      - name: Upload Artifact
        uses: actions/upload-artifact@v3
        with:
          name: build
          path: build/
          
  test:
    needs: build
    runs-on: [self-hosted, linux, test]
    steps:
      - uses: actions/checkout@v3
      - name: Download Artifact
        uses: actions/download-artifact@v3
        with:
          name: build
          path: build/
      - name: Run Tests
        run: npm ci && npm test

Troubleshooting

Common Issues and Solutions:

1. Runner Registration Failures: - Check PAT token or GitHub App authentication
- Verify network connectivity to GitHub
- Ensure proper permissions for tokens
- Check logs: `kubectl logs -n actions-runner-system deployment/actions-runner-controller-controller-manager`

2. Runner Pod Crashes: - Check resource limits
- Verify container image compatibility
- Inspect pod logs: `kubectl logs -n actions-runner-system pod/runner-pod-name`
- Check pod events: `kubectl describe pod -n actions-runner-system runner-pod-name`

3. Runners Not Picking Up Jobs: - Verify runner labels match workflow configuration
- Check if runners are idle or busy
- Ensure proper repository/organization configuration
- Check runner status in GitHub UI

4. Docker-in-Docker Issues: - Verify `dockerEnabled: true` is set
- Check if runner has proper privileges
- Inspect DinD sidecar logs
- Consider volume configuration for Docker socket

Useful Debugging Commands

# Check controller status
kubectl logs -n actions-runner-system deployment/actions-runner-controller-controller-manager

# View runner deployment status
kubectl describe runnerdeployments.actions.summerwind.dev example-runner

# Check runner pod logs
kubectl logs -n actions-runner-system pod/example-runner-pod-name

# Verify runner set status
kubectl get runnersets.actions.summerwind.dev -n actions-runner-system

# Check runner autoscaler status
kubectl describe horizontalrunnerautoscaler.actions.summerwind.dev runner-autoscaler

Security Best Practices

Security Recommendations:

1. Runner Isolation: - Use ephemeral runners for untrusted code
- Isolate runners in dedicated namespaces
- Consider using runner groups for sensitive projects
- Implement network policies to restrict communication

2. Authentication: - Use GitHub Apps instead of PAT tokens
- Rotate private keys regularly
- Store secrets securely using Kubernetes Secrets
- Implement least privilege principle for tokens

3. Runner Protection: - Don't grant sudo/admin privileges unless necessary
- Limit runner capabilities
- Consider using gVisor or similar sandbox technologies
- Scan runner images for vulnerabilities

4. Docker Security: - Use rootless Docker when possible
- Implement proper resource limits
- Consider alternatives like Kaniko for building without Docker daemon
- Scan container images during build

Sample Secure Runner Configuration

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: secure-runner
spec:
  replicas: 1
  template:
    spec:
      repository: "your-github-username/your-repository"
      ephemeral: true
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        fsGroup: 1000
      volumeMounts:
        - name: tmp
          mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir: {}
      labels:
        - secure-runner

Upgrade and Maintenance

Upgrading Actions Runner Controller

Scaling for Production Workloads

For production environments, consider the following scaling configurations:

apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: production-runner-autoscaler
spec:
  scaleTargetRef:
    name: production-runner
  minReplicas: 2
  maxReplicas: 20
  scaleDownDelaySecondsAfterScaleOut: 300
  metrics:
  - type: PercentageRunnersBusy
    scaleUpThreshold: '0.75'
    scaleDownThreshold: '0.25'
    scaleUpFactor: '2'
    scaleDownFactor: '0.5'

Key Considerations

1. Authentication:

You must choose between GitHub App or PAT authentication
GitHub App is recommended (better security and permissions management)

2. Permissions:

Set the proper permissions and resources required for Runner

3. Network Policy:

Check network policy and security settings