pymalloc, Memory Arena Layout & System Deallocation

Directly issuing OS system calls (malloc(3) or mmap(2)) for every small object allocation in Python creates severe OS memory fragmentation and performance overhead. CPython implements its own custom small-object memory allocator: pymalloc.

This chapter details the pymalloc sub-512 byte allocation hierarchy (Arenas, Pools, Blocks), OS memory deallocation boundaries, memory fragmentation, and __slots__ struct inlining.

1. The pymalloc Small-Object Allocator Architecture

pymalloc is specialized for allocating small objects ($\le 512$ bytes), which represent 95%+ of objects in standard Python workloads. Objects $> 512$ bytes bypass pymalloc and use standard system malloc().

pymalloc Memory Allocation Hierarchy:

[ Arena (256 KB Memory Chunk allocated via mmap) ]
├── Pool 0 (4 KB Chunk aligned to 4KB page)
│   ├── Block 0 (16 Bytes)
│   ├── Block 1 (16 Bytes)
│   └── Block N (16 Bytes)
├── Pool 1 (4 KB Chunk for 32-Byte Size Class)
└── Pool M (4 KB Chunk for 512-Byte Size Class)

Hierarchy Breakdown:

1. Arenas (256 KB): Large memory chunks allocated from the OS kernel via mmap(2) or virtualalloc.
2. Pools (4 KB): 4KB memory blocks inside an Arena, aligned to virtual memory pages. Each Pool is dedicated to a single Size Class (e.g., 16B, 32B, 48B … 512B).
3. Blocks (8B - 512B): Individual memory slots allocated to Python objects (PyObject).

2. OS Deallocation Boundaries & Memory Retention

Why doesn’t a Python process’s RSS (Resident Set Size) memory decrease in top or htop after deleting millions of objects?

Memory Return Cascade:

[ Python Object Deleted (del obj) ]
                |
                v
[ Block Freed inside 4KB Pool ]
                |
                +---> Are ALL Blocks in Pool Empty?
                |         ├── NO: Pool remains allocated in RAM!
                |         └── YES: Pool is marked free for the Arena!
                v
[ Are ALL Pools inside 256KB Arena Empty? ]
                |
                +---> NO: Arena REMAINS ALLOCATED in OS RAM! (Process RSS stays HIGH!)
                └── YES: Arenas freed back to OS kernel via munmap(2)!

Because an Arena is returned to the OS kernel only when ALL of its internal 4KB Pools become 100% empty, a single surviving 16-byte object inside a Pool prevents the entire 256KB Arena from being freed back to the OS!

3. Memory Fragmentation: __dict__ vs __slots__

Standard Python objects allocate a PyObject header plus a separate PyDictObject dictionary for attributes:

Standard Object Allocation (High Fragmentation):
 Allocation 1: PyObject Header (16B)      -> pymalloc Pool (Size Class 16B)
 Allocation 2: PyDictObject (200B)         -> pymalloc Pool (Size Class 200B)
 Allocation 3: PyDictKeysArray (128B)      -> pymalloc Pool (Size Class 128B)
 (3 Separate Memory Allocations per Instance!)

__slots__ Struct Inlining (Low Fragmentation):

Declaring __slots__ inlines attribute pointers directly into the instance struct payload. CPython allocates a single contiguous memory block for the object and its slotted attributes, reducing memory allocations by 66% and eliminating dictionary fragmentation!

4. Production Memory Diagnostics

• sys.getsizeof(obj): Returns memory size of obj in bytes (note: does NOT recursively measure nested child objects).
• tracemalloc: Standard library module for tracking memory allocation origins by file line numbers.