Descriptors: Attributes With Behaviour

Warm-up: from Day 1

From memory first — that’s the part that does the work.

1. Predict both lines.

Warm-up: predict Python

a = [1, 2]
b = a
c = [1, 2]
b.append(3)
print(a, a == c, a is b, a is c)

t = (a, "x")
a.append(4)
print(t)

Check your prediction

[1, 2, 3] False True False
([1, 2, 3, 4], 'x')

b is another name for a’s list, so appending through it changes a. And a tuple holds references: it can’t be rebound, but the list inside it can still change. From Names, Objects and Identity.

2. Fix. Changing your copy of the settings shouldn’t change the original: this should print ['python'].

Warm-up: fix Python

import copy

config = {"tags": ["python"], "limit": 10}
mine = copy.copy(config)
mine["tags"].append("orm")
print(config["tags"])

Check your fix

import copy

config = {"tags": ["python"], "limit": 10}
mine = copy.deepcopy(config)
mine["tags"].append("orm")
print(config["tags"])

['python']

A shallow copy is a new dictionary holding the same list. deepcopy copies the list too. From Copying, Shallow and Deep.

3. Predict three lines of attribute lookup.

Warm-up: lookup Python

class Base:
    kind = "base"

class Post(Base):
    pass

post = Post()
print(post.kind)
post.kind = "mine"
print(post.kind, Post.kind)
del post.kind
print(post.kind)

Check your prediction

base
mine base
base

Assigning on the instance puts kind in the instance’s __dict__, which shadows the class attribute without changing it. Delete it, and lookup falls through to the class again. From Where Attributes Come From — and today’s descriptors change exactly this order.

Today

Yesterday columns = ("title", "views") was a tuple of names, and nothing stopped post.views = "lots". Today those names become objects that can say no.

The mechanism is the descriptor protocol, and it isn’t an advanced corner of Python — it’s how property, classmethod, staticmethod and ordinary methods already work. Learning it doesn’t add a feature to your library; it explains machinery you have been using since your first class.

The protocol

A descriptor is an object, stored in a class, that defines any of:

__get__(self, instance, owner) — reading the attribute
__set__(self, instance, value) — assigning to it
__delete__(self, instance) — del
__set_name__(self, owner, name) — called once, when the class body finishes, telling the descriptor what name it was given

The first three are the access protocol — what makes attribute lookup call your code. __set_name__ is a separate convenience hook: an object can define it alone (to learn its name) without being a descriptor at all, and a real descriptor needs at least one of __get__/__set__/__delete__ to do anything on access.

A descriptor that narrates Python

class Loud:
    def __set_name__(self, owner, name):
        print(f"__set_name__: I am {owner.__name__}.{name}")
        self.name = name

    def __get__(self, instance, owner=None):
        print(f"__get__: instance={instance!r}")
        return 42

    def __set__(self, instance, value):
        print(f"__set__: {value!r}")


class Post:
    views = Loud()

print("--- class body finished ---")
post = Post()
print("reading:", post.views)
post.views = 7
print("through the class:", Post.views)

Three things to notice. __set_name__ ran while the class was being created, not when a Post was made. __get__ receives the instance — one descriptor object serves every Post that will ever exist. And reading through the class passes instance=None, which is why a field has to decide what Post.views means (usually: the field itself, so tooling can inspect it).

Data and non-data descriptors

This is the distinction that decides who wins against the instance dict:

Data descriptor: defines __set__ or __delete__. It takes priority over the instance dict, both for reading and writing.
Non-data descriptor: only __get__. The instance dict wins.

Who wins Python

class DataDescriptor:
    def __get__(self, instance, owner=None):
        return "from the descriptor"

    def __set__(self, instance, value):
        instance.__dict__["data"] = f"stored {value!r}"


class NonData:
    def __get__(self, instance, owner=None):
        return "from the descriptor"


class Post:
    data = DataDescriptor()
    plain = NonData()


post = Post()
post.__dict__["data"] = "straight into the instance dict"
post.__dict__["plain"] = "straight into the instance dict"

print("data: ", post.data)
print("plain:", post.plain)

The data descriptor is consulted even though the instance dict has an entry with the same name; the non-data one is shadowed by it. That’s step 1 of yesterday’s lookup order, and it’s why property can’t be overwritten by an attribute of the same name, while a cached-property-style object can be.

Methods are non-data descriptors

A plain function in a class body is a descriptor with only __get__, and what it returns is a bound method: the function with self already attached. That single mechanism is all there is to method binding.

A bound method is made on the fly Python

class Post:
    def describe(self):
        return f"post at {id(self):#x}"

post = Post()

print("in the class dict:", type(Post.__dict__["describe"]).__name__)
print("through the instance:", type(post.describe).__name__)
print("same object twice?", post.describe is post.describe)
print("what __get__ does:  ", Post.__dict__["describe"].__get__(post))

post.describe is post.describe is False: each access calls __get__ and builds a new bound method object. Harmless, and a good demonstration that attribute access is a computation, not a lookup in a struct.

Where the value goes

A descriptor is shared by every instance of its class, so it must not keep per-instance data on itself. The usual home is the instance’s own __dict__, under a key derived from the name __set_name__ learned:

    def __set_name__(self, owner, name):
        self.name = name
        self.slot = f"_{name}"

    def __set__(self, instance, value):
        instance.__dict__[self.slot] = value

Writing to instance.__dict__ directly matters: setattr(instance, self.name, value) would call __set__ again, forever.

The alternative — a dictionary on the descriptor keyed by instance — is a trap worth naming, because it looks tidy:

    def __set__(self, instance, value):
        self.values[instance] = value     # don't

It requires every model to be hashable, and it keeps every instance alive forever, since the descriptor lives as long as the class does. If you ever genuinely need it, weakref.WeakKeyDictionary fixes the second problem and not the first.

Exercise

A positive-integer descriptor

Write Positive so the tests pass. It’s the smallest real descriptor: validate on assignment, store per instance, and read back what was stored.

import unittest


class Positive:
    """A number that may not be negative."""

    def __set_name__(self, owner, name):
        pass          # TODO: remember the name, and a per-field slot to store under

    def __get__(self, instance, owner=None):
        raise NotImplementedError("__get__")

    def __set__(self, instance, value):
        raise NotImplementedError("__set__")


class Counter:
    views = Positive()
    likes = Positive()


class PositiveDescriptor(unittest.TestCase):
    def test_stores_and_reads_back(self):
        counter = Counter()
        counter.views = 3
        self.assertEqual(counter.views, 3)

    def test_rejects_a_negative_number(self):
        counter = Counter()
        with self.assertRaises(ValueError):
            counter.views = -1

    def test_each_field_is_separate(self):
        counter = Counter()
        counter.views, counter.likes = 3, 9
        self.assertEqual((counter.views, counter.likes), (3, 9))

    def test_each_instance_is_separate(self):
        first, second = Counter(), Counter()
        first.views, second.views = 1, 2
        self.assertEqual((first.views, second.views), (1, 2))

    def test_through_the_class_you_get_the_descriptor(self):
        self.assertIsInstance(Counter.views, Positive)


unittest.TextTestRunner(verbosity=2).run(
    unittest.defaultTestLoader.loadTestsFromTestCase(PositiveDescriptor)
)

Hint 1: the idea

Three methods, three jobs. __set_name__ runs once, when the class is created, and tells the descriptor its own name: remember it, and pick a per-field key to store under. __set__ checks the value, then stores it in the instance’s __dict__ under that key. __get__ reads it back — or, when accessed through the class (instance is None), returns the descriptor itself.

Hint 2: a first step

In __set_name__, keep name and a slot such as f"_{name}". Write __set__ to store into instance.__dict__[self.slot] and __get__ to read it back: the first test should pass. Then add the validation, and last the instance is None case.

Show one solution

class Positive:
    def __set_name__(self, owner, name):
        self.name = name
        self.slot = f"_{name}"

    def __get__(self, instance, owner=None):
        if instance is None:
            return self
        return instance.__dict__[self.slot]

    def __set__(self, instance, value):
        if value < 0:
            raise ValueError(f"{self.name} may not be negative")
        instance.__dict__[self.slot] = value

Each test maps to one line. __set_name__ gives each descriptor its own slot, which is what keeps views and likes apart. Storing in instance.__dict__[self.slot] is what keeps two counters apart. The instance is None branch is what makes Counter.views the descriptor rather than an error.

Raising before storing matters: a rejected assignment must leave the old value in place, or a caught ValueError leaves the object half-updated.

What you learned

A descriptor is a class attribute implementing __get__, __set__, __delete__ or __set_name__.
Data descriptors beat the instance dict; non-data ones lose to it.
Methods are non-data descriptors, and binding happens at each access.
One descriptor serves every instance, so values belong in the instance’s __dict__ — which is exactly what today’s bug hunt gets wrong.