Memory: Finding What Never Goes Away

Python frees objects nothing refers to. So a “memory leak” in Python is almost always the same thing: something is referring to them, and you forgot.

The library has had one since Day 1. Today you measure it.

tracemalloc: what grew between here and here

tracemalloc records where allocations happen. Take a snapshot, do the work, take another, and compare — which is exactly what the day’s leak.py does:

import tracemalloc

tracemalloc.start()
before = tracemalloc.take_snapshot()

load(connection, ids)                      # the work under suspicion

after = tracemalloc.take_snapshot()
tracemalloc.stop()

grew = sum(stat.size_diff for stat in after.compare_to(before, "lineno"))
for stat in after.compare_to(before, "lineno")[:3]:
    print(stat)

leak.py runs that against 5000 rows, loaded one at a time, under two cache sizes — the shipped default of 128, and 10000 (effectively uncapped for this workload). On the author’s laptop:

cache_size= 10000: cache holds  5000 objects, memory grew     1701 KiB
    models.py:103                                 550 KiB
    models.py:108                                 478 KiB
    models.py:92                                  364 KiB
cache_size=   128: cache holds   128 objects, memory grew       55 KiB
    models.py:103                                  23 KiB
    models.py:92                                   17 KiB
    models.py:108                                  12 KiB

1.7 MB against 55 KB, for the same 5000 loads. The lines named are where the surviving objects were allocated — the model construction in get and the dict insert in _remember — which is the question you usually want answered: not “what is big” but “where did the things that are still alive come from”.

A cache without a policy is a leak

The identity map was a good idea with a missing half. Day 1 introduced it so two get() calls for one row return the same object. Nothing ever removed an entry, so the map holds every row the process has ever touched, for as long as the process lives.

That is textbook: a cache needs a policy — a size, a lifetime, or both. Without one it is an ever-growing dictionary that happens to be useful.

    cache_size: ClassVar[int] = 128

    def _remember(self) -> None:
        model = type(self)
        cache = model._instances
        cache[self.id] = self
        cache.move_to_end(self.id)
        while len(cache) > model.cache_size:
            cache.popitem(last=False)          # drop the least recently used

OrderedDict.move_to_end plus popitem(last=False) is least-recently-used eviction in two lines. get() also calls move_to_end on a hit, which is what makes it LRU rather than “oldest inserted”.

And the cap changes the meaning of the identity map, which is the part to be honest about. It used to guarantee that two get()s return the same object. Now that holds only while the object is among the most recent 128, so code relying on is for a rarely-touched row will break in production and not in tests. That’s a real cost, paid to bound memory, and it belongs in the documentation rather than in a footnote.

What an object costs

import sys


class WithDict:
    def __init__(self, title, views):
        self.title = title
        self.views = views


class WithSlots:
    __slots__ = ("title", "views")

    def __init__(self, title, views):
        self.title = title
        self.views = views


def total_size(obj):
    """The object plus its instance dict, if it has one."""
    size = sys.getsizeof(obj)
    if hasattr(obj, "__dict__"):
        size += sys.getsizeof(obj.__dict__)
        size += sum(sys.getsizeof(key) for key in obj.__dict__)
    return size


plain = WithDict("post 1", 3)
slotted = WithSlots("post 1", 3)

print(f"with __dict__: {total_size(plain):4} bytes")
print(f"with __slots__:{total_size(slotted):4} bytes")
print(f"10,000 of each: {total_size(plain) * 10_000 / 1024:6.0f} KiB "
      f"vs {total_size(slotted) * 10_000 / 1024:6.0f} KiB")

__slots__ replaces the per-instance dictionary with a fixed array of attribute slots. For a class with millions of instances that is a real saving, and it comes with real costs: no new attributes at run time, no weak references unless you add __weakref__, and a subtle interaction with inheritance where one base class without __slots__ gives every instance a __dict__ again.

This library does not use it, and the reason is worth stating: the fields store their values in instance.__dict__ under _title, _views. Adopting __slots__ would mean declaring those private slots on every model — which is exactly the kind of bookkeeping __init_subclass__ removed on Day 2. The memory saved is a fraction of what an unbounded cache was wasting, so the bounded cache is the better fix for the same problem.

That comparison — which change buys more, for what complexity — is what measurement is for.

import unittest
from collections import OrderedDict


class LRUCache:
    """At most `size` entries, least recently used evicted first.

    TODO: get(key) -> value or None, put(key, value), __len__, and the
    hits/misses/evictions counters the tests read.
    """

    def __init__(self, size=3):
        self.size = size
        self.entries = OrderedDict()
        self.hits = 0
        self.misses = 0
        self.evictions = 0

    def get(self, key):
        raise NotImplementedError("get")

    def put(self, key, value):
        raise NotImplementedError("put")

    def __len__(self):
        return len(self.entries)


class Cache(unittest.TestCase):
    def test_stores_and_returns(self):
        cache = LRUCache()
        cache.put("a", 1)
        self.assertEqual(cache.get("a"), 1)

    def test_a_miss_is_none(self):
        self.assertIsNone(LRUCache().get("nothing"))

    def test_it_never_exceeds_its_size(self):
        cache = LRUCache(size=3)
        for index in range(10):
            cache.put(index, index)
        self.assertEqual(len(cache), 3)

    def test_the_least_recently_used_goes_first(self):
        cache = LRUCache(size=2)
        cache.put("a", 1)
        cache.put("b", 2)
        cache.get("a")              # "a" is now the most recently used
        cache.put("c", 3)           # evicts "b"
        self.assertEqual(cache.get("a"), 1)
        self.assertIsNone(cache.get("b"))

    def test_reading_counts_hits_and_misses(self):
        cache = LRUCache()
        cache.put("a", 1)
        cache.get("a")
        cache.get("b")
        self.assertEqual((cache.hits, cache.misses), (1, 1))

    def test_evictions_are_counted(self):
        cache = LRUCache(size=2)
        for index in range(5):
            cache.put(index, index)
        self.assertEqual(cache.evictions, 3)

    def test_overwriting_is_not_an_eviction(self):
        cache = LRUCache(size=2)
        cache.put("a", 1)
        cache.put("a", 2)
        self.assertEqual((len(cache), cache.evictions, cache.get("a")), (1, 0, 2))


unittest.TextTestRunner(verbosity=2).run(
    unittest.defaultTestLoader.loadTestsFromTestCase(Cache)
)

    def get(self, key):
        if key not in self.entries:
            self.misses += 1
            return None
        self.hits += 1
        self.entries.move_to_end(key)
        return self.entries[key]

    def put(self, key, value):
        self.entries[key] = value
        self.entries.move_to_end(key)
        while len(self.entries) > self.size:
            self.entries.popitem(last=False)
            self.evictions += 1

What you learned

• A Python leak is an unwanted reference; tracemalloc snapshots show which lines allocated the objects that survived.
• A cache with no eviction policy is an unbounded dictionary — measured here at 1.7 MB for 5000 rows against 55 KB with a cap.
• Bounding a cache weakens the guarantee it offered; say so in the docs.
• __slots__ saves real memory and constrains your design; weigh it against the alternatives rather than reaching for it.