Python Memory Not Freed After del? Diagnose the Cause

A suspected Python memory leak needs more evidence than a high process-memory reading. Compare the same workload over repeated batches, check which objects remain reachable, and inspect allocations that keep growing after cleanup.

Distinguish a leak from a high-water mark

Run equal batches with a bounded input size and compare memory after the same cleanup point. Growth followed by a stable plateau suggests reuse; continued growth in retained objects warrants an ownership investigation. Neither pattern alone identifies the cause. If process memory grows while tracemalloc stays flat, investigate native extensions and other allocations outside the traced Python allocator.

Object lifetime starts with references

CPython uses reference counting, although details such as immortal objects and free-threaded builds mean the count is not always a simple count of ordinary references. Assigning an object to another variable, placing it in a container, or capturing it in a closure can add a reference. Removing a name with del removes that reference; it does not necessarily destroy the object.

items = [{"id": 1}, {"id": 2}]
alias = items
del items

# The list is still reachable through alias.
print(alias)

When the reference count reaches zero, CPython can usually deallocate the object immediately. This explains why resources sometimes appear to close promptly, but application correctness should not rely on that implementation detail. Use context managers for files, sockets, locks, and transactions.

with open("events.jsonl", "rb") as source:
    payload = source.read()

Why cyclic garbage collection exists

Reference counting alone cannot reclaim a cycle. If object A refers to B and B refers to A, both counts can remain above zero after the program loses every outside reference. CPython's cyclic garbage collector periodically examines tracked container objects and finds unreachable groups.

class Node:
    def __init__(self, name):
        self.name = name
        self.peer = None

left = Node("left")
right = Node("right")
left.peer = right
right.peer = left

del left, right  # The cycle may be collected by cyclic GC.

The collector uses generations because most objects die young while a smaller group survives for much longer. Collection frequency and generation details can change between Python releases. Use the gc module to observe behavior, but do not tune thresholds without evidence from a representative workload.

Why process memory may not fall

Deallocating Python objects does not guarantee that the process resident set size immediately shrinks. CPython's allocator keeps pools and arenas available for later Python allocations, and the system allocator also has its own retention and fragmentation behavior. The memory can be reusable inside the process even when an operating-system chart stays flat.

This distinction prevents a common false diagnosis. A stable high-water mark after a temporary workload is not automatically a leak. A process whose memory grows with every repeated request, never levels off, and retains more live objects deserves investigation.

Measure allocations with tracemalloc

tracemalloc records where Python memory blocks were allocated. Take a baseline snapshot, run the operation several times, force the same cleanup path the application uses, and compare snapshots.

import tracemalloc

tracemalloc.start(25)
before = tracemalloc.take_snapshot()

for _ in range(20):
    run_one_batch()

after = tracemalloc.take_snapshot()
for stat in after.compare_to(before, "lineno")[:10]:
    print(stat)

Run several iterations because imports, caches, and one-time initialization can dominate the first pass. Group by traceback when the same helper allocates from multiple callers. tracemalloc focuses on Python allocations; memory held by native extensions, GPU libraries, memory-mapped files, or subprocesses may need their own profilers.

Find retention paths, not just large objects

A snapshot tells you where memory was allocated. A leak investigation also asks why the object is still reachable. Frequent causes include unbounded dictionaries, caches without eviction, global registries, completed tasks kept in a list, closures that capture large values, queued messages without backpressure, and logging handlers added repeatedly.

Check long-lived containers first. If a cache is intentional, give it an explicit size or expiry policy and expose hit rate, entry count, and evictions. Weak references can help with observer registries, but they are not a substitute for clear ownership.

from functools import lru_cache

@lru_cache(maxsize=2048)
def parse_schema(schema_id: str):
    return load_and_parse(schema_id)

An unlimited cache is a memory policy, even when it is written as a convenience decorator.

Measure retained size, not only shallow size

sys.getsizeof() reports the shallow size of one object. For a list, that includes the list's own storage but not the objects referenced by its elements. Adding the reported size of a container to every child can also double-count shared objects. Use it to compare like-for-like objects, not as a complete process-memory meter.

For production investigations, combine several views: process RSS for the operating-system footprint, tracemalloc for Python allocation sites, application counters for cache and queue ownership, and library-specific tools for native or accelerator memory. A consistent story across those measurements is stronger evidence than any single number.

Generators reduce peaks only when the pipeline streams

Generators can avoid building a full collection, but the benefit disappears if a later stage immediately calls list(), sorts all rows, or groups the entire input in memory. Trace the full pipeline.

def read_ids(path):
    with open(path, encoding="utf-8") as source:
        for line in source:
            yield line.partition(",")[0]

for record_id in read_ids("large.csv"):
    process(record_id)

Batching is often a better compromise when a downstream API benefits from groups. Choose the batch size from measured throughput and peak memory, not a round number copied from another system.

Use explicit ownership for external memory

Python objects can own memory outside the Python allocator. NumPy arrays may expose views of a much larger base array; a tiny slice can keep the entire allocation alive. Machine-learning tensors can keep accelerator memory until references disappear and the framework's caching allocator reuses or releases blocks. Database and compression libraries can maintain native buffers.

Inspect the owning library's memory tools, copy a small slice only when the ownership tradeoff justifies it, and close resources deterministically. Do not call gc.collect() after every request as a generic fix. Forced collections add latency and do not solve a reachable-object leak.

A reproducible leak investigation

Choose one operation and repeat it in a controlled process.
Record process RSS, Python allocation snapshots, object counts, and library-specific memory.
Warm up imports and caches before taking the baseline.
Compare snapshots after several equal batches.
Inspect the long-lived owners of growing object types.
Make one ownership or bounding change.
Repeat the same workload and include a long soak test.

Memory fixes need regression tests. A useful test repeats the operation enough times to expose growth and asserts on a stable application-level quantity, such as cache entries or queued tasks. Process memory can vary across platforms, so treat a strict RSS threshold cautiously.

Frequently asked questions

Is every increase in RSS a memory leak?

No. The Python and system allocators may retain freed memory for reuse. Look for continued growth under a repeated steady workload and confirm that live allocations or owning containers also grow.

Should I disable garbage collection for performance?

Only after profiling shows cyclic collection is material and the workload is known not to create problematic cycles. Reference counting continues, but unreachable cycles can accumulate while cyclic GC is disabled.

Does del free memory?

del removes a reference. The object is reclaimed only when no owning references remain, and reclaimed memory may stay available to the process instead of returning to the operating system.

Primary sources

A small retention experiment

Create a list, assign it to a second variable, and delete the first variable using the example above. The second reference still owns the list. Repeat your actual workload after removing or bounding the owner you suspect, then compare snapshots after the same number of iterations. Record cache length beside RSS so you can tell whether the application is retaining more entries.

The profiler fragments use application-specific functions such as run_one_batch and load_and_parse. Replace them with your own workload; they are not built-in Python functions.