7 min to read

AWS Secrets Manager - Secure Secret Management Service

A comprehensive guide to AWS Secrets Manager and its features

Featured image

Image Reference


Overview

AWS Secrets Manager is a secrets management service that helps you protect access to your applications, services, and IT resources.

Without the upfront investment and ongoing maintenance costs of operating your own infrastructure, you can manage and retrieve database credentials, API keys, and other secrets throughout their lifecycle.


Key Features of AWS Secrets Manager

Secure Secret Storage and Management

Fine-grained Access Control

Automated Secret Rotation

Audit and Monitoring


AWS Secrets Manager CLI Commands

# Available Commands
aws secretsmanager help

# Common Commands
aws secretsmanager create-secret
aws secretsmanager get-secret-value
aws secretsmanager list-secrets
aws secretsmanager rotate-secret
aws secretsmanager delete-secret


Secret Types and Examples

1. Database Credentials

{
  "username": "myDatabaseUser",
  "password": "mySecurePassword",
  "host": "myDatabaseHost",
  "port": 3306
}

2. API Keys

{
  "api_key": "sk_test_1234567890abcdef",
  "secret_key": "sh_test_1234567890abcdef"
}

3. OAuth Tokens

{
  "access_token": "ya29.1234567890abcdef",
  "refresh_token": "1/1234567890abcdef",
  "client_id": "myClientID.apps.googleusercontent.com",
  "client_secret": "myClientSecret"
}

4. SSH Keys

-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAz0FzLw1bmK5OzYDhLTVlLQzPgh6T9T...
... (rest of the private key) ...
-----END RSA PRIVATE KEY-----


Rotation Configuration

Rotation Settings

Required Permissions

  1. Secrets Manager permissions
  2. Lambda function permissions
  3. Application permissions


Understanding OAuth and SSO


OAuth Token Types
  1. Access Token
    • Short-term credentials
    • Used for resource access
    • Has expiration time
  2. Refresh Token
    • Long-term credentials
    • Used to obtain new access tokens
    • Reduces need for frequent re-authentication


OAuth Token Flow

sequenceDiagram participant Resource_Owner as Resource Owner (Financial Customer) participant Client as Client (Fintech Application) participant Auth_Server as Authorization Server (Financial Institution Server) participant Resource_Server as Resource Server (Financial Open API Provider) Resource_Owner->>Client: 1. Request for Permissions (Direct/Indirect) Client->>Resource_Owner: 2. Authorization Grant Client->>Auth_Server: 3. Access Token Request Auth_Server->>Client: 4. Access Token Issuance (Optional: Refresh Token Issuance) Client->>Resource_Server: 5. Protected Resource Request (Certified by Access Token) Resource_Server->>Client: 6. Service Response


SSO Process
  1. User initiates login
  2. Application redirects to OAuth provider
  3. User grants permissions
  4. OAuth provider issues authentication code
  5. Application exchanges code for tokens


SSO Flow

sequenceDiagram participant User as User participant Application as Application participant OAuth_Provider as OAuth Provider User->>Application: 1. Initiates Login Application->>OAuth_Provider: 2. Redirects to OAuth Provider OAuth_Provider->>User: 3. Prompts for Permissions User->>OAuth_Provider: 4. Grants Permissions OAuth_Provider->>Application: 5. Issues Authentication Code Application->>OAuth_Provider: 6. Exchanges Code for Tokens OAuth_Provider->>Application: 7. Issues Access & Refresh Tokens Application->>User: 8. Grants Access


Implementation Best Practices


1. Security: - Use customer managed KMS keys for additional control
- Implement least privilege IAM policies
- Enable automatic rotation for all secrets
- Consider multi-region replication for disaster recovery

2. Performance: - Cache secrets in application memory when appropriate
- Use VPC endpoints to access Secrets Manager within your VPC
- Implement proper error handling for retrieval failures
- Consider using Parameter Store for non-sensitive configuration

3. Cost Optimization: - Consolidate related credentials into a single secret when possible
- Clean up unused secrets to avoid unnecessary charges
- Use tagging for cost allocation and tracking
- Evaluate Parameter Store for less sensitive configs (free tier available)


Integration Examples

1. AWS CLI Example



2. AWS SDK (Python) Example

import boto3
import json
import base64
from botocore.exceptions import ClientError

def get_secret(secret_name, region_name="us-west-2"):
    # Create a Secrets Manager client
    session = boto3.session.Session()
    client = session.client(
        service_name='secretsmanager',
        region_name=region_name
    )

    try:
        get_secret_value_response = client.get_secret_value(
            SecretId=secret_name
        )
    except ClientError as e:
        # Handle exceptions
        if e.response['Error']['Code'] == 'DecryptionFailureException':
            raise e
        elif e.response['Error']['Code'] == 'InternalServiceErrorException':
            raise e
        elif e.response['Error']['Code'] == 'InvalidParameterException':
            raise e
        elif e.response['Error']['Code'] == 'InvalidRequestException':
            raise e
        elif e.response['Error']['Code'] == 'ResourceNotFoundException':
            raise e
    else:
        # Decrypts secret using the associated KMS key
        if 'SecretString' in get_secret_value_response:
            secret = get_secret_value_response['SecretString']
            return json.loads(secret)
        else:
            decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])
            return decoded_binary_secret

# Example usage
def connect_to_database():
    secret = get_secret("production/app/database")
    
    # Connect to the database using the secret
    connection = mysql.connector.connect(
        host=secret['host'],
        user=secret['username'],
        password=secret['password'],
        database=secret['dbname']
    )
    return connection

3. AWS CDK Example



Common Secrets Manager Use Cases

Industry-Specific Use Cases
  • Finance: API keys for payment gateways, banking credentials
  • Healthcare: Credentials for accessing patient record systems
  • E-commerce: Database credentials, payment processor API keys
  • DevOps: CI/CD pipeline credentials, deployment tokens
  • SaaS: Multi-tenant database credentials, third-party API tokens

Specific Implementation: Automatic RDS Credential Rotation



Troubleshooting


1. Access Denied Errors: - Check IAM permissions for both the user and resources
- Verify KMS key permissions if using a customer managed key
- Check for resource policies restricting access
- Ensure your VPC endpoint policy allows the action

2. Rotation Failures: - Review Lambda execution logs for detailed error messages
- Ensure Lambda has proper permissions to access the secret and database
- Check network connectivity if your database is in a VPC
- Verify that the rotation Lambda matches your database type

3. Performance Issues: - Implement caching to reduce frequent API calls
- Use VPC endpoints to avoid latency when accessing from a VPC
- Check for throttling or rate limiting issues
- Consider regional replicas for global applications


References