AWS IRSA (IAM Roles for Service Accounts)

A comprehensive guide to AWS IRSA implementation and workflow

Featured image



Overview

Today, we’ll explore AWS IRSA (IAM Roles for Service Accounts) and understand how it enables secure AWS service access for pods running in EKS clusters.


What is AWS IRSA?

AWS IAM Roles for Service Accounts (IRSA) is a method of granting access to AWS services using ServiceAccount on a pad running on a Kubernetes cluster managed by AWS EKS (Elastic Kubernetes Service).

So how can I allocate an IAM Role when ServiceAccount is not a resource for AWS?

OpenID Connect, called OIDC, and the Security Token Service, called STS, perform the function.

Key Components
  • Kubernetes ServiceAccounts
  • AWS IAM Roles
  • OpenID Connect (OIDC)
  • Security Token Service (STS)


What is OIDC(OpenID Connect)?

OpenID Connect (OIDC) is an authentication protocol built on OAuth 2.0 that allows users to authenticate with a third-party service using their existing accounts.

It extends OAuth 2.0 authorization to provide user information in addition to tokens.

Create and Verify EKS OIDC Provider



What is ServiceAccount?

ServiceAccount is a Kubernetes resource that represents a service account, which is an identity for an application or a user.

It is used to manage access to resources within a Kubernetes cluster.

Create IAM Role and ServiceAccount



What is STS(Security Token Service)?

STS is a service that provides temporary security credentials for AWS resources.

It is used to manage access to AWS resources securely.


🔄 IRSA Workflow

Example Workflow (Pod accessing S3)
  1. Pod sends JWT and IAM Role ARN to AWS STS
  2. STS requests verification from IAM
  3. IAM verifies with OIDC Provider
  4. IAM confirms authorization to STS
  5. STS provides temporary credentials to Pod
  6. Pod accesses S3 using temporary credentials

IRSA Workflow


Key Components Explained

OIDC (OpenID Connect)

AWS STS (Security Token Service)

JWT (JSON Web Token)


Implementation Steps

1. Create an OIDC Provider for your EKS Cluster

# Get your EKS cluster OIDC issuer URL
OIDC_PROVIDER=$(aws eks describe-cluster --name my-cluster --query "cluster.identity.oidc.issuer" --output text | sed -e "s/^https:\/\///")

# Create the OIDC Identity Provider in IAM
aws iam create-open-id-connect-provider \
  --url https://$OIDC_PROVIDER \
  --client-id-list sts.amazonaws.com \
  --thumbprint-list <thumbprint>

2. Create an IAM Role with Trust Policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/<OIDC_PROVIDER>"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "<OIDC_PROVIDER>:sub": "system:serviceaccount:default:my-service-account",
          "<OIDC_PROVIDER>:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}

3. Create a Kubernetes ServiceAccount

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-service-account
  namespace: default
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<ACCOUNT_ID>:role/my-iam-role

4. Reference the ServiceAccount in Your Pod

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      serviceAccountName: my-service-account
      containers:
      - name: aws-cli
        image: amazon/aws-cli
        command: ['sh', '-c', 'aws s3 ls && sleep 3600']


Common Use Cases

Popular IRSA Implementations
  • S3 Access: Applications reading/writing to S3 buckets
  • DynamoDB: Microservices with their own database tables
  • SQS/SNS: Event-driven applications processing messages
  • Secrets Manager: Securely retrieving configuration secrets
  • CloudWatch: Custom logging and monitoring

Example: Application Reading from SQS Queue

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sqs-consumer
spec:
  replicas: 2
  selector:
    matchLabels:
      app: sqs-consumer
  template:
    metadata:
      labels:
        app: sqs-consumer
    spec:
      serviceAccountName: sqs-consumer-sa
      containers:
      - name: app
        image: my-sqs-consumer:latest
        env:
        - name: AWS_REGION
          value: us-west-2
        - name: SQS_QUEUE_URL
          value: https://sqs.us-west-2.amazonaws.com/123456789012/my-queue


Best Practices


1. Security: - Apply the principle of least privilege to all IAM roles
- Use namespaced service accounts to isolate permissions
- Regularly audit and rotate IAM roles
- Limit service account usage to specific pods

2. Implementation: - Use infrastructure as code (IaC) to manage IRSA setup
- Keep trust policies specific to namespaces and service accounts
- Consider using tools like ExternalDNS, Cluster Autoscaler with IRSA
- Implement proper error handling for credential failures

3. Monitoring: - Enable CloudTrail for IRSA-related actions
- Set up alerts for suspicious or high-volume credential requests
- Monitor for unauthorized AssumeRoleWithWebIdentity calls
- Use AWS CloudWatch to track API calls made with IRSA credentials


Troubleshooting IRSA


1. Authentication Failures: - Verify OIDC provider is correctly configured
- Check IAM role trust policy for proper namespace/service account
- Ensure AWS SDK is using the EKS Pod Identity webhook credentials
- Verify service account annotation has the correct IAM role ARN

2. Permission Issues: - Check that IAM role has required permission policies
- Verify request is coming from the correct service account
- Use AWS STS get-caller-identity to debug credential chain
- Review CloudTrail for specific permission denials

3. Common Errors: - "An error occurred (AccessDenied) when calling the AssumeRoleWithWebIdentity operation"
- "Error validating provider information"
- "Not authorized to perform sts:AssumeRoleWithWebIdentity"

Debugging Commands

# Check if service account has the correct annotation
kubectl describe serviceaccount my-service-account

# Test AWS API access from a pod
kubectl run aws-cli --image=amazon/aws-cli --command -- sleep infinity
kubectl exec -it aws-cli -- aws sts get-caller-identity

# Check for recent authentication events
aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=AssumeRoleWithWebIdentity



References