Deep Dive into OpenStack Manila

Understanding OpenStack's Shared File System as a Service

Featured image



Understanding OpenStack Manila


Manila is OpenStack’s Shared File System as a Service (SFaaS) that provides file sharing capabilities across multiple virtual machines and containers.

It supports various file sharing protocols like NFS and CIFS, enabling flexible and scalable file storage solutions in the cloud.


What is Manila?


The Shared File System Service

Manila serves as OpenStack’s file sharing service, providing essential functionality:

By providing shared file system capabilities, Manila enables efficient data sharing and collaboration in cloud environments while maintaining security and performance.


Why Use Manila?

Manila addresses critical challenges in cloud storage management:


Supported File System Types

Manila supports various file system types and protocols:

Protocol/Type Description Use Cases
NFS Network File System Linux environments, container storage, shared application data
CIFS/SMB Common Internet File System Windows environments, mixed OS deployments, enterprise integration
CephFS Ceph Distributed File System High-performance computing, scalable storage, object storage
HDFS Hadoop Distributed File System Big data analytics, distributed computing, data lakes
GlusterFS Scale-out Network File System Distributed applications, high availability, scale-out storage


Manila Architecture Overview (Diagram Description)

graph LR A[OpenStack Manila] A --> B[Core Features] A --> C[Service Integration] A --> D[Storage] A --> E[Protocols] B --> B1[File Sharing] B --> B2[Multi-tenancy] B --> B3[Access Control] C --> C1[Nova] C --> C2[Neutron] C --> C3[Keystone] D --> D1[Backend Storage] D --> D2[Snapshots] D --> D3[Replication] E --> E1[NFS] E --> E2[CIFS] E --> E3[CephFS] style A stroke:#333,stroke-width:1px,fill:#f5f5f5 style B stroke:#333,stroke-width:1px,fill:#a5d6a7 style C stroke:#333,stroke-width:1px,fill:#64b5f6 style D stroke:#333,stroke-width:1px,fill:#ffcc80 style E stroke:#333,stroke-width:1px,fill:#ce93d8



Manila Architecture and Components

Manila’s architecture is designed to provide flexible and scalable file sharing capabilities.

Each component plays a specific role in managing and providing access to shared file systems.


Detailed Architecture

Manila follows a distributed architecture with clear separation of responsibilities:

graph TB subgraph "Client Layer" CLI[OpenStack CLI] HORIZON[Horizon Dashboard] API_CLIENT[API Clients] VMS[Virtual Machines] end subgraph "Manila Services" API[Manila API] SCHEDULER[Manila Scheduler] SHARE[Manila Share Service] DATA[Manila Data Service] end subgraph "Storage Backends" NAS[Enterprise NAS] CEPH[Ceph Storage] NETAPP[NetApp Filers] GENERIC[Generic Backend] end subgraph "Network Layer" NEUTRON[Neutron Networks] SHARE_NET[Share Networks] end CLI --> API HORIZON --> API API_CLIENT --> API VMS --> SHARE_NET API --> SCHEDULER SCHEDULER --> SHARE SHARE --> DATA SHARE --> NAS SHARE --> CEPH SHARE --> NETAPP SHARE --> GENERIC SHARE_NET --> NEUTRON NEUTRON --> SHARE style API stroke:#333,stroke-width:2px,fill:#4CAF50 style SCHEDULER stroke:#333,stroke-width:2px,fill:#2196F3 style SHARE stroke:#333,stroke-width:2px,fill:#FF9800 style DATA stroke:#333,stroke-width:2px,fill:#9C27B0


Core Components

Component Role Description
Manila API API Service
  • Handles RESTful API requests for share management
  • Provides authentication and authorization via Keystone
  • Manages share lifecycle operations
  • Handles quota management and tenant isolation
  • Supports multiple API versions for backward compatibility
Manila Scheduler Resource Management
  • Selects optimal storage backend based on policies
  • Manages capacity planning and resource allocation
  • Handles share type mapping to backends
  • Optimizes placement for performance and availability
  • Supports custom scheduling filters and weighers
Manila Share Share Service
  • Manages file system operations on storage backends
  • Handles protocol-specific share creation and management
  • Controls access permissions and export rules
  • Manages share exports and mount points
  • Handles share snapshots and replication
Manila Data Data Service
  • Handles data plane operations like copy and migration
  • Manages share data replication between backends
  • Performs data transformation and migration tasks
  • Handles backup and restore operations


Storage Backend Types

Manila supports multiple storage backend architectures:

Backend Type Description Examples
Driver-only Direct integration with storage arrays NetApp, EMC, HPE 3PAR, Pure Storage
Generic Driver Uses Nova instances as share servers Generic NFS/CIFS servers, development environments
Container Driver Container-based share servers Docker-based NFS servers, lightweight deployments
Native Drivers Purpose-built distributed file systems CephFS, GlusterFS, HDFS


Service Integration

Manila integrates seamlessly with several OpenStack services to provide comprehensive file sharing:

Service Integration Type Benefits
Nova Compute integration for share servers
  • Automatic share server provisioning
  • Instance-based file sharing
  • Dynamic scaling of share servers
Neutron Network connectivity and isolation
  • Share network creation and management
  • Network isolation between tenants
  • Security group integration
Keystone Identity and access management
  • User authentication and authorization
  • Project-based access control
  • Role-based permissions
Cinder Block storage backend integration
  • Shared block storage volumes
  • Snapshot and backup integration
  • Volume-based share servers
Glance Share server image management
  • Custom share server images
  • Pre-configured NFS/CIFS images
  • Image lifecycle management
Barbican Encryption and key management
  • Share encryption at rest
  • Key management for encrypted shares
  • Certificate management for secure protocols

This comprehensive integration enables robust file sharing within the complete OpenStack ecosystem.



Key Features and Capabilities

Manila provides comprehensive file sharing capabilities that enable efficient data sharing and collaboration.

These features make it a powerful tool for file system management in OpenStack environments.


Core Features

Feature Description Benefits
Protocol Support Multiple file sharing protocols
  • NFS support
  • CIFS/SMB support
  • CephFS integration
  • Protocol flexibility
Storage Management Flexible storage options
  • Multiple backends
  • Dynamic resizing
  • Snapshot support
  • Replication
Access Control Security features
  • ACL management
  • Export rules
  • Multi-tenancy
  • Security groups


Best Practices

Key considerations for Manila deployment and operation:

Category Best Practice Implementation
Protocol Selection Choose appropriate file sharing protocol
  • NFS for Linux-heavy environments
  • CIFS/SMB for Windows integration
  • CephFS for high-performance needs
Backend Selection Select suitable storage backend
  • Enterprise arrays for production
  • Generic driver for development
  • Distributed filesystems for scale
Network Design Plan network architecture carefully
  • Dedicated share networks
  • Proper VLAN segmentation
  • Network performance optimization
Access Control Implement comprehensive security
  • IP-based access restrictions
  • User/certificate authentication
  • Share-level permissions
Performance Tuning Optimize for workload requirements
  • Share type QoS settings
  • Network bandwidth allocation
  • Backend performance tuning
Monitoring Set up comprehensive monitoring
  • Share availability monitoring
  • Performance metrics tracking
  • Capacity utilization alerts


Share Types and Capabilities

Understanding share types is crucial for optimal Manila deployment:

Capability Description Use Cases
Snapshot Support Point-in-time share snapshots Data protection, testing, rollback scenarios
Replication Share data replication across sites Disaster recovery, geographic distribution
Extend Size Dynamic share size expansion Growing datasets, elastic storage needs
Shrink Size Dynamic share size reduction Cost optimization, resource management
QoS Support Quality of Service controls Performance guarantees, multi-tenant isolation

These practices ensure reliable, secure, and maintainable file sharing in production environments.



Implementation and Usage

Effective implementation of Manila requires proper configuration and integration with other OpenStack services.

Here are key considerations and best practices for utilizing Manila effectively.


Configuration and Setup


Basic Configuration

Key configuration files and settings:

# /etc/manila/manila.conf
[DEFAULT]
# API Configuration
osapi_share_listen = 0.0.0.0
osapi_share_listen_port = 8786
api_paste_config = /etc/manila/api-paste.ini

# Database Configuration
sql_connection = mysql+pymysql://manila:password@localhost/manila

# RabbitMQ Configuration
transport_url = rabbit://manila:password@controller:5672/

# Keystone Authentication
auth_strategy = keystone

[keystone_authtoken]
www_authenticate_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = manila
password = manila_password

# Share Backend Configuration
enabled_share_backends = generic,cephfs,netapp
enabled_share_protocols = NFS,CIFS,CEPHFS

[generic]
share_driver = manila.share.drivers.generic.GenericShareDriver
interface_driver = manila.network.linux.interface.OVSInterfaceDriver
service_instance_flavor_id = 100
service_image_name = manila-service-image
service_instance_user = manila
service_instance_password = manila
connect_share_server_to_tenant_network = True

[cephfs]
share_driver = manila.share.drivers.cephfs.driver.CephFSDriver
driver_handles_share_servers = False
cephfs_conf_path = /etc/ceph/ceph.conf
cephfs_auth_id = manila
cephfs_cluster_name = ceph


Advanced Backend Configuration

Example configuration for enterprise storage arrays:

# NetApp ONTAP Backend
[netapp]
share_driver = manila.share.drivers.netapp.dataontap.cluster_mode.drv.NetAppCmodeMultiSVMShareDriver
driver_handles_share_servers = True
netapp_server_hostname = 192.168.1.100
netapp_server_port = 443
netapp_login = admin
netapp_password = password
netapp_vserver = manila_vserver
netapp_aggregate_name_search_pattern = ^aggr.*
netapp_root_volume_aggregate = aggr1
netapp_port_name_search_pattern = ^e.*


Share Network Management


Creating Share Networks

Share networks provide network connectivity for shares:


Security Services

Configure authentication and access control:


Advanced Usage Examples

Share Replication Setup

Configure disaster recovery with share replication:


Share Groups Management

Manage related shares as a group:


Mount Share Examples

Examples of mounting shares on different platforms:


API Usage Examples


Using REST API

Direct API calls for automation and integration:


Python Client Usage

Programmatic access using Python Manila client:

from manilaclient import client
from keystoneauth1.identity import v3
from keystoneauth1 import session

# Authentication
auth = v3.Password(
    auth_url='http://keystone.example.com:5000/v3',
    username='admin',
    password='password',
    project_name='admin',
    user_domain_name='default',
    project_domain_name='default'
)
sess = session.Session(auth=auth)
manila = client.Client('2', session=sess)

# Create share
share = manila.shares.create(
    share_proto='NFS',
    size=100,
    name='python-share',
    share_type='default',
    share_network='<share-network-id>'
)

# Wait for share to be available
manila.shares.wait_for_share_to_become_available(share.id)

# Allow access
manila.shares.allow(
    share.id,
    access_type='ip',
    access='192.168.1.0/24',
    access_level='rw'
)

# Create snapshot
snapshot = manila.share_snapshots.create(
    share.id,
    name='python-snapshot',
    description='Created via Python client'
)

# List shares
shares = manila.shares.list()
for share in shares:
    print(f"Share: {share.name}, Status: {share.status}, Size: {share.size}GB")


Use Cases

Manila is particularly useful for various enterprise and cloud scenarios:

Use Case Description Benefits
Multi-VM Applications Applications requiring shared data access
  • Shared configuration files
  • Log aggregation
  • Distributed application data
Container Storage Persistent storage for containerized apps
  • Kubernetes persistent volumes
  • Docker volume plugins
  • Stateful application data
Content Management Web content and media storage
  • Website assets sharing
  • Media file distribution
  • Content delivery optimization
Development Environments Shared development resources
  • Source code repositories
  • Build artifacts sharing
  • Development tool assets
Enterprise Integration Legacy application modernization
  • Lift-and-shift migrations
  • Hybrid cloud scenarios
  • Enterprise NAS integration
Big Data Analytics Large-scale data processing
  • Hadoop/Spark data lakes
  • Analytics input datasets
  • ML training data storage

These use cases demonstrate Manila’s versatility and enterprise readiness across various deployment scenarios.



Monitoring and Troubleshooting

Effective monitoring and troubleshooting are essential for maintaining reliable Manila file sharing services.


Performance Monitoring


Key Metrics to Monitor

Metric Category Key Metrics Monitoring Tools
Share Performance
  • I/O operations per second (IOPS)
  • Throughput (MB/s)
  • Latency and response time
  • Queue depth
Prometheus, Grafana, Manila metrics
Service Health
  • API response times
  • Share server availability
  • Backend connectivity
  • Service process status
Nagios, Zabbix, OpenStack Health Monitor
Capacity Utilization
  • Share space usage
  • Backend capacity utilization
  • Network bandwidth usage
  • Connection counts
Manila dashboards, Storage array tools


Log Analysis

Key log files and monitoring patterns:

# Manila service logs
tail -f /var/log/manila/manila-api.log
tail -f /var/log/manila/manila-scheduler.log
tail -f /var/log/manila/manila-share.log
tail -f /var/log/manila/manila-data.log

# Key patterns to monitor:
# - Share creation/deletion operations
# - Access rule modifications
# - Backend communication errors
# - Network connectivity issues
# - Authentication failures


Common Issues and Solutions


Troubleshooting Guide

Issue Symptoms Solution
Share Creation Failures
  • Shares stuck in "creating" state
  • Backend communication errors
  • Check backend connectivity
  • Verify driver configuration
  • Review scheduler logs
Mount Issues
  • Unable to mount shares
  • Permission denied errors
  • Verify access rules
  • Check network connectivity
  • Review export paths
Performance Problems
  • Slow file operations
  • High latency
  • Analyze network performance
  • Check backend storage load
  • Review share server resources


Performance Optimization

Strategies for optimizing Manila performance:



Security and Access Control

Comprehensive security guidelines for Manila deployments in production environments.


Access Control Models


Access Types

Manila supports multiple access control mechanisms:

Access Type Description Use Cases
IP-based Control access by IP address or subnet Network-based security, simple deployments
User-based Control access by username Individual user access, fine-grained control
Certificate-based Control access using client certificates High-security environments, PKI integration
Kerberos Kerberos-based authentication Enterprise environments, Active Directory


Security Services Integration

Configure enterprise authentication:


Network Security

Best practices for securing Manila network traffic:


Data Protection

Comprehensive data protection strategies:



Key Points

Manila Essentials
  • Core Functionality
    - Shared file system as a service
    - Multi-protocol support (NFS, CIFS, CephFS)
    - Enterprise storage integration
    - Share lifecycle management
  • Advanced Features
    - Share networks and security services
    - Snapshot and replication capabilities
    - Dynamic resizing and QoS controls
    - Multi-backend support
    - API and CLI management interfaces
  • Enterprise Benefits
    - Seamless legacy integration
    - Active Directory authentication
    - High availability and disaster recovery
    - Performance optimization
    - Comprehensive access control
  • Operational Excellence
    - Automated share server management
    - Comprehensive monitoring and alerting
    - Backup and disaster recovery
    - Security and compliance features
    - Scalable multi-tenant architecture



References