29 min to read

Python Exception Handling and Context Managers: A Complete Guide to Robust Code

Master error handling and resource management for reliable and maintainable Python applications

Python Exception Handling and Context Managers: A Complete Guide to Robust Code


Overview

Exception handling and context managers are essential features for writing robust and safe Python code. They enable you to handle errors gracefully and manage resources safely, preventing resource leaks and ensuring your applications remain stable even when things go wrong.


Understanding how to properly handle exceptions and manage resources is crucial for building production-ready applications. Exception handling allows your code to recover from unexpected situations, while context managers ensure that resources like files, database connections, and network sockets are properly cleaned up, even when errors occur. By mastering these concepts, you’ll write more reliable and maintainable code.


1. Exception Handling Basics

The foundation of error handling in Python is the try-except block, which allows you to catch and handle exceptions.


Basic try-except Structure

try:
    # Code that might raise an exception
    result = 10 / 0
except ZeroDivisionError:
    # Handle the exception
    print("Cannot divide by zero!")


Handling Multiple Exception Types

def safe_divide(a, b):
    try:
        result = a / b
        return result
    except ZeroDivisionError:
        print("Cannot divide by zero!")
        return None
    except TypeError:
        print("Only numbers are allowed!")
        return None

print(safe_divide(10, 2))     # 5.0
print(safe_divide(10, 0))     # Cannot divide by zero! None
print(safe_divide(10, "a"))   # Only numbers are allowed! None


Catching Multiple Exceptions Together

try:
    value = int(input("Enter a number: "))
    result = 100 / value
except (ValueError, ZeroDivisionError) as e:
    print(f"Error occurred: {e}")


Why Exception Handling Matters

Prevents Crashes:

Improves User Experience:

Enables Debugging:


2. Working with Exception Objects

Exception objects contain valuable information about what went wrong and where.


Accessing Exception Information

try:
    with open("nonexistent_file.txt", "r") as f:
        content = f.read()
except FileNotFoundError as e:
    print(f"File not found: {e}")
    print(f"Exception type: {type(e)}")
    print(f"Exception message: {str(e)}")


Catching All Exceptions (Use with Caution)

try:
    # Risky operation
    risky_operation()
except Exception as e:
    print(f"Unexpected error: {e}")
    # However, it's better to catch specific exceptions when possible!
Warning: Catching all exceptions with except Exception can hide bugs and make debugging difficult. Always prefer catching specific exception types when you know what might go wrong.


Exception Attributes

try:
    result = int("not a number")
except ValueError as e:
    print(f"Error: {e}")
    print(f"Args: {e.args}")
    print(f"Traceback: {e.__traceback__}")


3. The else and finally Clauses

The else and finally clauses provide additional control over exception handling flow.


else: Executes When No Exception Occurs

def read_file(filename):
    try:
        f = open(filename, 'r')
    except FileNotFoundError:
        print("File does not exist.")
    else:
        # Only executes if try block succeeded
        content = f.read()
        f.close()
        return content

print(read_file("data.txt"))


finally: Always Executes

def process_file(filename):
    f = None
    try:
        f = open(filename, 'r')
        data = f.read()
        return data
    except FileNotFoundError:
        print("File not found.")
        return None
    finally:
        # Always executes, regardless of exceptions
        if f:
            f.close()
            print("File closed.")

process_file("test.txt")


Complete Structure

try:
    # Code that might raise an exception
    result = dangerous_operation()
except SomeException as e:
    # Handle the exception
    handle_error(e)
else:
    # Executes if no exception occurred
    print("Success!")
finally:
    # Always executes (cleanup operations)
    cleanup()


When to Use Each Clause

Clause When It Runs Use Case
try Always Code that might raise exceptions
except When exception occurs Error handling
else When no exception occurs Success-case code
finally Always Cleanup operations


4. Raising Exceptions

Sometimes you need to raise exceptions yourself to signal error conditions.


Basic Exception Raising

def validate_age(age):
    if age < 0:
        raise ValueError("Age cannot be negative!")
    if age > 150:
        raise ValueError("Age is too high!")
    return True

try:
    validate_age(-5)
except ValueError as e:
    print(f"Validation failed: {e}")


Re-raising Exceptions

def process_data(data):
    try:
        # Process data
        result = complex_operation(data)
    except Exception as e:
        print(f"Log: Error occurred - {e}")
        raise  # Re-raise the exception to propagate it upward

try:
    process_data(None)
except Exception:
    print("Exception handled at higher level")


Raising with Custom Messages

def withdraw_money(balance, amount):
    if amount > balance:
        raise ValueError(
            f"Insufficient funds: balance={balance}, requested={amount}"
        )
    return balance - amount

try:
    withdraw_money(100, 150)
except ValueError as e:
    print(f"Transaction failed: {e}")


5. Creating Custom Exceptions

Custom exceptions help you create domain-specific error handling that makes your code more readable and maintainable.


Basic Custom Exception

class InsufficientFundsError(Exception):
    """Raised when account balance is insufficient"""
    def __init__(self, balance, amount):
        self.balance = balance
        self.amount = amount
        message = f"Insufficient funds: {balance} available, {amount} required"
        super().__init__(message)

class BankAccount:
    def __init__(self, balance):
        self.balance = balance
    
    def withdraw(self, amount):
        if amount > self.balance:
            raise InsufficientFundsError(self.balance, amount)
        self.balance -= amount
        return self.balance

# Usage
account = BankAccount(1000)
try:
    account.withdraw(1500)
except InsufficientFundsError as e:
    print(f"Withdrawal failed: {e}")
    print(f"Current balance: ${e.balance}")
    print(f"Required amount: ${e.amount}")


Exception Hierarchy

class ValidationError(Exception):
    """Base validation error"""
    pass

class EmailValidationError(ValidationError):
    """Email validation error"""
    pass

class PasswordValidationError(ValidationError):
    """Password validation error"""
    pass

def validate_user(email, password):
    if "@" not in email:
        raise EmailValidationError("Invalid email format")
    if len(password) < 8:
        raise PasswordValidationError("Password must be at least 8 characters")

try:
    validate_user("test", "123")
except EmailValidationError as e:
    print(f"Email error: {e}")
except PasswordValidationError as e:
    print(f"Password error: {e}")
except ValidationError as e:
    # Catches all validation errors
    print(f"Validation error: {e}")


Advanced Custom Exceptions

class APIError(Exception):
    """Base API error"""
    def __init__(self, message, status_code=None, response=None):
        super().__init__(message)
        self.status_code = status_code
        self.response = response

class APIConnectionError(APIError):
    """API connection failed"""
    pass

class APITimeoutError(APIError):
    """API request timed out"""
    pass

class APIAuthenticationError(APIError):
    """API authentication failed"""
    pass

def call_api(endpoint, token):
    if not token:
        raise APIAuthenticationError(
            "Authentication required",
            status_code=401
        )
    # API call logic...

try:
    call_api("/users", None)
except APIAuthenticationError as e:
    print(f"Auth failed: {e}")
    print(f"Status code: {e.status_code}")


6. Context Managers

Context managers provide a clean way to manage resources, ensuring proper setup and cleanup.


Basic with Statement

# Bad: File might not be closed if exception occurs
f = open("data.txt", "r")
data = f.read()
f.close()  # Won't execute if exception occurs above!

# Good: File automatically closed
with open("data.txt", "r") as f:
    data = f.read()
# File is automatically closed here


Managing Multiple Resources

with open("input.txt", "r") as infile, open("output.txt", "w") as outfile:
    content = infile.read()
    outfile.write(content.upper())
# Both files are automatically closed


Why Use Context Managers?

Automatic Cleanup:

Cleaner Code:

Safety:


Common Built-in Context Managers

# File handling
with open("file.txt") as f:
    content = f.read()

# Threading locks
import threading
lock = threading.Lock()
with lock:
    # Critical section
    shared_resource.modify()

# Decimal context
from decimal import localcontext, Decimal
with localcontext() as ctx:
    ctx.prec = 2
    result = Decimal('1.00') / Decimal('3.00')


7. Creating Custom Context Managers

You can create your own context managers using classes or decorators.


Class-Based Context Manager

class DatabaseConnection:
    def __init__(self, db_name):
        self.db_name = db_name
        self.connection = None
    
    def __enter__(self):
        print(f"Connecting to {self.db_name}...")
        self.connection = f"Connection to {self.db_name}"
        return self.connection
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        print(f"Closing connection to {self.db_name}")
        if exc_type is not None:
            print(f"Exception occurred: {exc_type.__name__}: {exc_val}")
        # Return False to propagate exceptions
        # Return True to suppress exceptions
        return False

# Usage
with DatabaseConnection("mydb") as conn:
    print(f"Connected: {conn}")
    # Perform operations
# Connection automatically closed


Context Manager with Exception Handling

class FileManager:
    def __init__(self, filename, mode):
        self.filename = filename
        self.mode = mode
        self.file = None
    
    def __enter__(self):
        try:
            self.file = open(self.filename, self.mode)
            return self.file
        except FileNotFoundError:
            print(f"File not found: {self.filename}")
            raise
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        if self.file:
            self.file.close()
            print("File safely closed.")
        
        if exc_type is IOError:
            print("IO error occurred but was handled.")
            return True  # Suppress the exception
        return False

# Usage
try:
    with FileManager("test.txt", "r") as f:
        content = f.read()
except FileNotFoundError:
    print("File processing failed")


Understanding exit Parameters

class DebugContext:
    def __enter__(self):
        print("Entering context")
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        print(f"Exiting context")
        print(f"Exception type: {exc_type}")
        print(f"Exception value: {exc_val}")
        print(f"Traceback: {exc_tb}")
        
        # Return True to suppress the exception
        # Return False (or None) to propagate it
        return False

with DebugContext():
    print("Inside context")
    raise ValueError("Test error")


8. The @contextmanager Decorator

The contextlib module provides a simpler way to create context managers using decorators.


Basic @contextmanager Usage

from contextlib import contextmanager

@contextmanager
def timer(name):
    import time
    print(f"{name} started")
    start = time.time()
    try:
        yield  # This is where the with block executes
    finally:
        end = time.time()
        print(f"{name} completed: {end - start:.4f} seconds")

# Usage
with timer("Data processing"):
    # Work to be timed
    total = sum(range(1000000))
    print(f"Sum: {total}")


Yielding Values

from contextlib import contextmanager

@contextmanager
def temporary_setting(setting_name, new_value):
    import os
    old_value = os.environ.get(setting_name)
    
    # Change setting
    os.environ[setting_name] = new_value
    print(f"{setting_name} changed: {old_value} → {new_value}")
    
    try:
        yield new_value  # Pass value to with statement
    finally:
        # Restore original setting
        if old_value is None:
            os.environ.pop(setting_name, None)
        else:
            os.environ[setting_name] = old_value
        print(f"{setting_name} restored: {new_value} → {old_value}")

# Usage
with temporary_setting("DEBUG", "true") as debug:
    print(f"Debug mode: {debug}")
    # Work in debug environment
# Automatically restored to original setting


Error Handling in @contextmanager

from contextlib import contextmanager

@contextmanager
def error_handler(operation_name):
    print(f"Starting {operation_name}")
    try:
        yield
    except Exception as e:
        print(f"Error in {operation_name}: {e}")
        # Re-raise if needed
        raise
    else:
        print(f"{operation_name} completed successfully")
    finally:
        print(f"Cleaning up after {operation_name}")

with error_handler("database operation"):
    # Risky operation
    connect_to_database()
    perform_query()


Practical Example: Transaction Management

from contextlib import contextmanager

@contextmanager
def transaction(connection):
    """Database transaction context manager"""
    try:
        yield connection
        connection.commit()
        print("Transaction committed")
    except Exception as e:
        connection.rollback()
        print(f"Transaction rolled back: {e}")
        raise
    finally:
        connection.close()
        print("Connection closed")

# Usage
with transaction(get_db_connection()) as conn:
    conn.execute("INSERT INTO users VALUES (...)")
    conn.execute("UPDATE accounts SET ...")
# Auto-commit or rollback


9. Practical Example: File Processing System

A comprehensive example combining exception handling and context managers.

from contextlib import contextmanager
import json
import logging

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

class FileProcessingError(Exception):
    """File processing error"""
    pass

@contextmanager
def safe_file_operation(filename, mode='r'):
    """Context manager for safe file operations"""
    file_obj = None
    try:
        logger.info(f"Opening file: {filename} (mode: {mode})")
        file_obj = open(filename, mode, encoding='utf-8')
        yield file_obj
    except FileNotFoundError:
        logger.error(f"File not found: {filename}")
        raise FileProcessingError(f"File not found: {filename}")
    except PermissionError:
        logger.error(f"Permission denied: {filename}")
        raise FileProcessingError(f"Permission denied: {filename}")
    except Exception as e:
        logger.error(f"Unexpected error: {e}")
        raise
    finally:
        if file_obj:
            file_obj.close()
            logger.info(f"Closed file: {filename}")

class DataProcessor:
    def __init__(self, input_file, output_file):
        self.input_file = input_file
        self.output_file = output_file
    
    def process(self):
        """Main data processing logic"""
        try:
            data = self._read_data()
            processed_data = self._transform_data(data)
            self._write_data(processed_data)
            logger.info("Data processing completed")
            return True
        except FileProcessingError as e:
            logger.error(f"File processing failed: {e}")
            return False
        except json.JSONDecodeError as e:
            logger.error(f"JSON parsing failed: {e}")
            return False
        except Exception as e:
            logger.error(f"Processing error: {e}")
            return False
    
    def _read_data(self):
        """Read data from file"""
        with safe_file_operation(self.input_file, 'r') as f:
            return json.load(f)
    
    def _transform_data(self, data):
        """Transform data"""
        if not isinstance(data, list):
            raise ValueError("Data must be a list")
        
        processed = []
        for item in data:
            if 'name' not in item or 'value' not in item:
                logger.warning(f"Invalid data format: {item}")
                continue
            
            processed.append({
                'name': item['name'].upper(),
                'value': item['value'] * 2,
                'processed': True
            })
        
        return processed
    
    def _write_data(self, data):
        """Write data to file"""
        with safe_file_operation(self.output_file, 'w') as f:
            json.dump(data, f, indent=2, ensure_ascii=False)

# Usage example
if __name__ == "__main__":
    processor = DataProcessor("input.json", "output.json")
    
    # Create input file (for testing)
    with safe_file_operation("input.json", "w") as f:
        json.dump([
            {"name": "apple", "value": 10},
            {"name": "banana", "value": 20},
            {"name": "cherry", "value": 15}
        ], f)
    
    # Process data
    success = processor.process()
    if success:
        print("✅ Processing successful!")
    else:
        print("❌ Processing failed!")


Output Example

INFO:__main__:Opening file: input.json (mode: w)
INFO:__main__:Closed file: input.json
INFO:__main__:Opening file: input.json (mode: r)
INFO:__main__:Closed file: input.json
INFO:__main__:Opening file: output.json (mode: w)
INFO:__main__:Closed file: output.json
INFO:__main__:Data processing completed
✅ Processing successful!


10. Common Built-in Exceptions

Understanding Python’s exception hierarchy helps you catch the right exceptions.

Exception When It Occurs Example
ValueError Invalid value int("abc")
TypeError Wrong type "2" + 2
KeyError Dictionary key not found d["missing_key"]
IndexError Index out of range list[100]
FileNotFoundError File doesn't exist open("none.txt")
ZeroDivisionError Division by zero 10 / 0
AttributeError Attribute doesn't exist obj.missing
ImportError Import fails import nonexistent
RuntimeError General runtime error Various situations
StopIteration Iterator exhausted next(iter([]))


Exception Hierarchy

BaseException
 ├── SystemExit
 ├── KeyboardInterrupt
 ├── Exception
 │   ├── ArithmeticError
 │   │   ├── ZeroDivisionError
 │   │   └── OverflowError
 │   ├── LookupError
 │   │   ├── KeyError
 │   │   └── IndexError
 │   ├── OSError
 │   │   ├── FileNotFoundError
 │   │   └── PermissionError
 │   ├── ValueError
 │   ├── TypeError
 │   └── ...


Best Practices Summary


Exception Handling Best Practices

1. Catch Specific Exceptions:

# Good
try:
    value = int(user_input)
except ValueError:
    print("Invalid number")

# Bad
try:
    value = int(user_input)
except Exception:
    print("Something went wrong")


2. Don’t Ignore Exceptions:

# Bad
try:
    risky_operation()
except:
    pass  # Silent failure!

# Good
try:
    risky_operation()
except SpecificError as e:
    logger.error(f"Operation failed: {e}")
    # Handle appropriately


3. Use finally for Cleanup:

resource = acquire_resource()
try:
    use_resource(resource)
finally:
    release_resource(resource)  # Always executed


4. Create Domain-Specific Exceptions:

class PaymentError(Exception):
    pass

class InsufficientFundsError(PaymentError):
    pass

class PaymentTimeoutError(PaymentError):
    pass


5. Provide Helpful Error Messages:

if amount < 0:
    raise ValueError(
        f"Amount must be positive, got {amount}"
    )


Context Manager Best Practices

1. Use for Resource Management:

2. Use for Temporary State:

@contextmanager
def changed_directory(path):
    old_dir = os.getcwd()
    os.chdir(path)
    try:
        yield
    finally:
        os.chdir(old_dir)


3. Use for Timing and Logging:

@contextmanager
def log_operation(name):
    logger.info(f"Starting {name}")
    start = time.time()
    try:
        yield
    finally:
        duration = time.time() - start
        logger.info(f"{name} took {duration:.2f}s")


4. Prefer @contextmanager for Simple Cases:


Conclusion

Mastering exception handling and context managers is essential for writing robust, production-ready Python code.


Key Takeaways


Exception Handling:

Context Managers:

Production Readiness:


Remember

Robust programs are those that handle exceptions well and manage resources properly. By following these practices:

Exception handling and context managers aren’t just about preventing crashes—they’re about building software that works correctly even when things go wrong!


References