pathlib, Files, Directories & Safe Filesystem Work

Manipulating file paths using raw string operations (os.path.join) leads to path separator bugs, security vulnerabilities (Path Traversal attacks), and OS platform incompatibilities. Python’s pathlib module provides an object-oriented filesystem hierarchy (PurePath, Path, PosixPath, WindowsPath).

This chapter details pathlib class architecture, atomic file writing patterns, Path Traversal security validation, and os.scandir() directory traversal performance.

1. pathlib Class Architecture (PurePath vs. Path)

pathlib separates path calculation logic from actual OS filesystem operations:

pathlib Class Hierarchy:

[ PurePath ] (Pure computational path arithmetic; no OS I/O system calls)
    ├── PurePosixPath  (Linux, macOS, Unix paths)
    └── PureWindowsPath (Windows drive letters & UNC paths)

[ Path ] (Inherits from PurePath; executes live OS system calls: stat, read, write)
    ├── PosixPath     (Executes POSIX stat(2) & open(2) system calls)
    └── WindowsPath   (Executes Win32 API system calls)

• PurePath: Safe for parsing Windows paths on a Linux server (or vice versa) because it performs string manipulation without accessing the local OS filesystem.
• Path: Instantiates a PosixPath or WindowsPath depending on the host operating system, enabling live filesystem I/O (path.read_text(), path.mkdir(), path.stat()).

2. Preventing Path Traversal Security Attacks

Allowing user input to determine file paths creates a severe Path Traversal Vulnerability (allowing attackers to read /etc/passwd using ../../ payloads).

Always validate input paths using path.resolve() and is_relative_to() (Python 3.9+):

from pathlib import Path

BASE_DIR = Path("/var/app/user_uploads").resolve()

def get_user_file(filename: str) -> Path:
    # Resolve symlinks and relative '../' elements to an absolute path
    target_path = (BASE_DIR / filename).resolve()

    # SECURITY GUARD: Ensure target path is strictly contained within BASE_DIR!
    if not target_path.is_relative_to(BASE_DIR):
        raise PermissionError("Path Traversal attack detected!")

    return target_path

3. Atomic File Writes & Symlink Safety

Writing directly to a target file (with open(file, 'w')) creates a corrupt zero-byte file if the system crashes or encounters an out-of-memory error mid-write.

Production Atomic Write Pattern:

1. Write payload to a temporary file in the same directory.
2. Flush and sync temporary file descriptors to disk (os.fsync).
3. Atomically rename the temporary file over the target path (temp_path.replace(target_path)). In POSIX systems, rename(2) is an atomic OS kernel operation.

import os
from pathlib import Path

def atomic_write_text(filepath: Path, content: str):
    temp_file = filepath.with_suffix(".tmp." + str(os.getpid()))
    try:
        with open(temp_file, "w", encoding="utf-8") as f:
            f.write(content)
            f.flush()
            os.fsync(f.fileno()) # Force write to physical disk
        # Atomic rename over target destination
        temp_file.replace(filepath)
    except Exception:
        if temp_file.exists():
            temp_file.unlink()
        raise

4. High-Performance Directory Traversal (os.scandir())

Legacy os.listdir() returns string filenames, requiring subsequent os.stat() system calls for every file to inspect file sizes or modifications. Path.iterdir() uses os.scandir() under the hood:

• os.scandir(): Yields DirEntry objects containing OS stat metadata directly from directory stream reads, eliminating thousands of individual stat(2) C system calls and executing 2x to 20x faster.