11 min to read

Understanding HashiCorp Vault - Core Concepts and Architecture

A comprehensive guide to HashiCorp Vault's fundamentals and working principles

Featured image

Image Reference link


Introduction to HashiCorp Vault

HashiCorp Vault is an identity-based secrets and encryption management system designed to help organizations secure, store, and control access to tokens, passwords, certificates, API keys, and other sensitive data. With its robust security architecture and extensive feature set, Vault enables organizations to implement a centralized solution for managing secrets across diverse infrastructure.


What is Vault?

The Secret Management Platform

HashiCorp Vault addresses the critical challenge of secrets management in modern infrastructure by providing:

  • Secure Storage: Encrypted repository for sensitive credentials and data
  • Dynamic Secrets: On-demand generation of short-lived credentials
  • Data Protection: Encryption as a service for applications
  • Access Control: Fine-grained permissions and authentication methods
  • Audit Trail: Detailed logging of all secret access and operations

Vault serves as a single source of truth for secrets, eliminating insecure practices like hardcoded credentials or configuration files with plaintext secrets.

graph TD A[HashiCorp Vault] --> B[Secret Storage] A --> C[Identity Management] A --> D[Encryption Services] A --> E[Access Control] B --> B1[Static Secrets] B --> B2[Dynamic Secrets] B --> B3[Encryption Keys] C --> C1[Authentication Methods] C --> C2[Identity Federation] D --> D1[Transit Engine] D --> D2[Key Management] E --> E1[Policies] E --> E2[ACLs] E --> E3[Token Management] style A stroke:#333,stroke-width:1px,fill:#f5f5f5 style B stroke:#333,stroke-width:1px,fill:#a5d6a7 style C stroke:#333,stroke-width:1px,fill:#64b5f6 style D stroke:#333,stroke-width:1px,fill:#ffcc80 style E stroke:#333,stroke-width:1px,fill:#ce93d8


Core Security Model

Security Principle Implementation in Vault
Secure by Default Vault starts in a sealed state requiring explicit unsealing, and uses TLS for all communications
Least Privilege Policy-based access control grants only the minimum required permissions to each client
Defense in Depth Multiple layers of security including authentication, authorization, and encryption
No Single Point of Failure Support for high availability configurations and multiple storage backends
Auditability Comprehensive audit logging of all operations and access attempts


Vault Workflow and Components

Understanding Vault's authentication and authorization flow is essential for implementing it effectively. Vault follows a structured process to verify identity, apply policies, and grant appropriate access to secrets.

Authentication and Authorization Flow

Core Workflow Steps:
  1. Authentication: Clients provide credentials to prove their identity
  2. Validation: Vault validates clients against trusted external sources (GitHub, LDAP, Kubernetes, etc.)
  3. Authorization: Vault checks client permissions against defined security policies
  4. Access: Vault issues tokens with appropriate policies attached for subsequent requests
  5. Audit: All operations are logged to configured audit devices

This workflow ensures that only authenticated clients with proper authorization can access specific secrets.

sequenceDiagram participant Client participant Auth as Auth Method participant Policy as Policy Engine participant Secret as Secret Engine Client->>Auth: Authenticate (credentials) Auth-->>Client: Client Token Client->>Policy: Request Access (token + path) Policy->>Policy: Evaluate Policies Policy-->>Client: Allow/Deny Client->>Secret: Request Secret (token) Secret->>Policy: Check Permission Policy-->>Secret: Approved Secret-->>Client: Secret Data


Key System Components

Component Description Function
Storage Backend Physical storage for encrypted data
  • Stores encrypted secrets and system configuration
  • Supports multiple backend options (Consul, etcd, DynamoDB)
  • No access to plaintext data (zero-knowledge architecture)
Barrier Encryption layer
  • Encrypts/decrypts all data entering/leaving storage
  • Uses a master key derived from unseal keys
  • Implements authenticated encryption (AES-GCM)
Auth Methods Identity verification systems
  • Multiple verification options (LDAP, GitHub, AWS, etc.)
  • Maps external identities to internal policies
  • Pluggable architecture for custom methods
Secret Engines Secret storage and generation
  • Modular backend systems for different types of secrets
  • Support for static and dynamic secrets
  • Specialized engines for different systems (databases, cloud providers)
Policies Access control rules
  • Define permissions using HashiCorp Configuration Language (HCL)
  • Path-based access control model
  • Capability-based permissions (create, read, update, delete, list)


Key Features and Capabilities

Vault provides a comprehensive set of features for secrets management and data protection. These capabilities address different aspects of security challenges in modern infrastructure.

Secret Storage and Management

1. Secure Secret Storage
  • Centralized Repository: Single source of truth for all secrets
  • Encrypted Storage: All data encrypted before persistence
  • Version Control: Secret versioning and history tracking (in KV v2)
  • Structured Organization: Path-based hierarchical storage
2. Dynamic Secrets
  • Just-In-Time Credentials: Generated on-demand when requested
  • Automatic Lifecycle: Credentials automatically rotated and revoked
  • Reduced Attack Surface: Short-lived credentials minimize exposure
  • Integration Support: Works with databases, cloud platforms, and more
3. Data Encryption
  • Transit Engine: Encryption as a service for applications
  • Key Rotation: Non-disruptive rotation of encryption keys
  • Multiple Algorithms: Support for various encryption standards
  • Secure Offloading: Applications delegate cryptographic operations
4. Lease Management
  • Time-Bound Access: All dynamic secrets have expiration times
  • Renewable Leases: Support for extending secret lifetimes
  • Automatic Cleanup: Expired secrets automatically revoked
  • Lease Tracking: Central visibility into all issued secrets
5. Revocation Capabilities
  • Granular Control: Revoke individual secrets or groups
  • Prefix Revocation: Revoke all secrets under a path prefix
  • Immediate Effect: Instant invalidation of credentials
  • Security Response: Quick mitigation for compromised credentials


Supported Secret Engines

graph LR A[Secret Engines] --> B[Key-Value] A --> C[Database] A --> D[Cloud Providers] A --> E[Cryptographic] A --> F[Identity] B --> B1[KV Version 1] B --> B2[KV Version 2] C --> C1[MySQL/MariaDB] C --> C2[PostgreSQL] C --> C3[MongoDB] C --> C4[MSSQL] D --> D1[AWS] D --> D2[Azure] D --> D3[GCP] E --> E1[Transit] E --> E2[PKI] F --> F1[TOTP] F --> F2[SSH] style A stroke:#333,stroke-width:1px,fill:#f5f5f5 style B stroke:#333,stroke-width:1px,fill:#a5d6a7 style C stroke:#333,stroke-width:1px,fill:#64b5f6 style D stroke:#333,stroke-width:1px,fill:#ffcc80 style E stroke:#333,stroke-width:1px,fill:#ce93d8 style F stroke:#333,stroke-width:1px,fill:#ef9a9a


Vault Architecture and Lifecycle

Vault follows a distinct lifecycle with specific phases for initialization, unsealing, and operation. Understanding this lifecycle is crucial for proper deployment and administration.

Lifecycle Phases

Phase Description Security Considerations
Initialization One-time setup process that generates the master key and initial root token
  • Produces unseal keys that must be securely distributed
  • The number of key shares and threshold are configurable
  • Once initialized, cannot be repeated without data loss
Sealed State Default state after startup; Vault cannot access stored data
  • All data remains encrypted and inaccessible
  • Only unseal and status API endpoints are available
  • Provides protection for data at rest
Unsealing Process of reconstructing the master key to decrypt data
  • Requires threshold number of unseal key shares
  • Must be performed after every Vault restart
  • Can be automated with auto-unseal features
Active Operation Normal operating state where Vault processes requests
  • All API endpoints become available
  • Master key remains in memory only
  • Proper authentication and authorization required for all operations
Resealing Return to sealed state, either manually or due to restart
  • Immediately clears master key from memory
  • Provides protection during maintenance or security events
  • Can be triggered via API or CLI

Vault Operational Flow

sequenceDiagram participant Client participant Vault Server participant Storage Backend Note over Client, Vault Server: Initialization Phase Client ->> Vault Server: Initialize Vault (POST /v1/sys/init) Vault Server ->> Storage Backend: Store master keys and initial token Storage Backend -->> Vault Server: Acknowledge storage Vault Server -->> Client: Return unseal keys and initial root token Note over Client, Vault Server: Unseal Phase loop Unseal Process Client ->> Vault Server: Unseal Vault (POST /v1/sys/unseal) Vault Server -->> Vault Server: Verify unseal key Vault Server -->> Client: Unseal progress update end Note over Client, Vault Server: Operational Phase Client ->> Vault Server: Login (POST /v1/auth/{method}/login) Vault Server ->> Storage Backend: Validate credentials Storage Backend -->> Vault Server: Token generation Vault Server -->> Client: Return token Client ->> Vault Server: Request secret (GET /v1/secret/data/{name}) Vault Server ->> Storage Backend: Retrieve secret Storage Backend -->> Vault Server: Return secret Vault Server -->> Client: Secret data

Initialization Phase

graph LR; A[Client] -->|Init Request| B[Vault Server]; B -->|Store| C[Storage Backend]; B -->|Return Keys| A;
Initialization Security

During initialization, Vault generates two critical security elements:

  • Unseal Keys: Used to reconstruct the master key for unsealing
  • Root Token: Initial super-admin token with full system access

Both must be securely stored and protected, as they provide complete access to the Vault system and its contents.

Operational Phase

graph LR; A[Client] -->|Login Request| B[Vault Server]; B -->|Validate| C[Storage Backend]; B -->|Session Token| A; A -->|Secret Request| B; B -->|Return Secret| A;


Deployment Considerations

Deploying Vault in production requires careful planning for security, high availability, and operational management. Several key factors should be considered in your deployment strategy.

High Availability Setup

Resilient Deployment

For production use, Vault should be deployed in a high-availability configuration:

  • Multiple Servers: Deploy multiple Vault instances behind a load balancer
  • Cluster Mode: Configure servers in a cluster for automatic leader election
  • Consistent Storage: Use a consistent storage backend like Consul
  • Sealed Redundancy: Each node can independently handle unseal operations

This architecture ensures continuous availability even during node failures or maintenance.

Security Best Practices

Security Area Recommendations
Network Security
  • Use TLS for all communications with proper certificate validation
  • Implement network segmentation with Vault in a protected zone
  • Configure restrictive firewall rules allowing only necessary traffic
Authentication
  • Avoid using the root token for regular operations
  • Implement multi-factor authentication for sensitive operations
  • Use centralized identity providers when possible
Operational Security
  • Enable audit logging to persistent, tamper-evident storage
  • Implement secure key sharing for unseal operations
  • Consider automatic unsealing using cloud KMS services
Monitoring
  • Monitor Vault telemetry data for anomalies
  • Set up alerts for unusual access patterns or failed operations
  • Track token usage and revocations


Key Points

💡 Vault Essentials
  • Core Purpose
    - Centralized secrets management system for sensitive credentials
    - Provides both static and dynamic secrets capabilities
    - Offers encryption as a service for applications
  • Security Architecture
    - Multi-layered security model with defense in depth
    - Sealed/unsealed state management for data protection
    - Fine-grained access control through policies
    - Comprehensive auditing capabilities
  • Integration Capabilities
    - Multiple authentication methods for various identity sources
    - Rich API for automation and programmatic access
    - Extensive plugin architecture for extensibility
    - Support for various deployment environments


Next Steps

Continuing Your Vault Journey

In our next post, we'll explore the practical aspects of Vault deployment, including:

  • Installation: Step-by-step guide to installing Vault on Kubernetes
  • Configuration: Setting up authentication methods and secret engines
  • Integration: Connecting applications with Vault for secret access
  • Operations: Day-to-day operational tasks and best practices

Stay tuned for a hands-on approach to implementing Vault in your environment.


References