post.title looks like reading a slot in a struct. It isn’t: it’s a search, and
knowing the order of that search is what lets you predict which value you get,
where an assignment lands, and why a class attribute can look like an instance
attribute right up until someone writes to it.
Two dictionaries
Most objects keep their attributes in a dictionary, and so does their class.
vars() shows either one.
class Post:
table = "post" # lives on the class
def __init__(self, title):
self.title = title # lives on the instance
first = Post("Hello")
second = Post("Goodbye")
print("instance:", vars(first))
print("class: ", {k: v for k, v in vars(Post).items() if not k.startswith("__")})
print(first.table, second.table) # both find it on the class
first.table = "archive" # writes to the *instance*
print(first.table, second.table)
print("instance now:", vars(first))
print("class still: ", Post.table)Reading first.table finds nothing on the instance and falls back to the class.
Assigning to it puts a new entry in the instance dict, which then shadows the
class attribute — for that one object. second never noticed.
This asymmetry is the source of a whole family of bugs: reading works, so the code looks right, and the first write splits one shared value into two independent ones.
The order Python searches
For obj.name, roughly:
type(obj)and its bases (in MRO order) are searched forname. If what is found is a data descriptor — a property, or anything else defining__set__— it wins immediately, before the instance dict is even consulted.- The instance’s own
__dict__. - What step 1 found, if it wasn’t a data descriptor: a plain class attribute, a
method, a
classmethod. - If nothing was found,
type(obj).__getattr__(obj, name)runs, if it exists. - Still nothing:
AttributeError.
Step 1 is why property can’t be shadowed by an instance attribute, and it’s the
subject of tomorrow’s lesson on descriptors. Today, steps 2 to 5 are enough.
The MRO is a single ordered list
With more than one base class, “the class and its bases” means the method resolution order: a single list, computed once per class, that every lookup walks in order.
class Model:
def describe(self):
return "a model"
class Timestamped:
def describe(self):
return "timestamped " + super().describe()
class Post(Timestamped, Model):
pass
print(" -> ".join(cls.__name__ for cls in Post.__mro__))
print(Post().describe())Post.__mro__ is Post -> Timestamped -> Model -> object. super() inside
Timestamped.describe doesn’t mean “my base class” — it means “the next class
after Timestamped in the MRO of the object’s type”, which is why a mixin
written this way composes with classes it has never heard of.
A mixin must cooperate
If Timestamped.describe returned a string without calling super().describe(),
Model.describe would never run. Every class in a chain has to pass the call
along, or the chain stops there. This is the practical reason mixins are hard to
retrofit: cooperation has to be designed in.
__getattr__: the fallback, not the hook
__getattr__ runs only when normal lookup has already failed. That makes it
cheap — no cost for attributes that exist — and it makes it the right place to
put a helpful error, or a computed value that has no stored form.
Its evil twin __getattribute__ runs for every attribute access, including
self.something inside your own methods, which makes infinite recursion the
default outcome. You will very rarely want it.
class Row:
"""A row whose columns are known at run time, not written in the class."""
def __init__(self, table, values):
self.table = table
self.values = values
def __getattr__(self, name):
# Only reached when `name` isn't on the instance or the class.
values = self.__dict__.get("values", {})
if name in values:
return values[name]
known = ", ".join(sorted(values)) or "none"
raise AttributeError(f"{self.table} has no column {name!r}; columns: {known}")
row = Row("post", {"title": "Hello", "views": 3})
print(row.title, row.views)
print("table is a normal attribute:", row.table)
print("hasattr says:", hasattr(row, "author"))
try:
row.author
except AttributeError as error:
print("AttributeError:", error)Two details in there matter more than they look.
self.__dict__.get("values", {})rather thanself.values. During__init__, and during unpickling, an object can exist beforevaluesis set. Writingself.valuesinside__getattr__in that state calls__getattr__again, for"values", forever. Reading the instance dict directly can’t recurse.- Raising
AttributeError, notKeyError.hasattris defined as “does getattr raise AttributeError?”, and the standard library probes objects for optional attributes all the time —copy.deepcopyasks each object for a__deepcopy__,inspectfor a__wrapped__. A__getattr__that raises the wrong type of error breaks them in ways that look unrelated to attributes.
A row wrapper that behaves
Finish Record so it exposes its data as attributes. The tests cover the two
traps above, plus one more: __getattr__ must not answer for dunder names, or a
field called __deepcopy__ will be handed to copy.deepcopy as if it were a
method.
import copy
import unittest
class Record:
def __init__(self, **fields):
self.fields = fields
def __getattr__(self, name):
raise NotImplementedError("write me")
class Records(unittest.TestCase):
def test_fields_are_readable_as_attributes(self):
record = Record(title="Hello", views=3)
self.assertEqual(record.title, "Hello")
self.assertEqual(record.views, 3)
def test_unknown_names_raise_attribute_error(self):
record = Record(title="Hello")
with self.assertRaises(AttributeError):
record.author
self.assertFalse(hasattr(record, "author"))
def test_the_message_names_the_field_and_what_exists(self):
record = Record(title="Hello", views=3)
with self.assertRaises(AttributeError) as caught:
record.author
message = str(caught.exception)
self.assertIn("author", message)
self.assertIn("title", message)
def test_dunder_names_are_left_alone(self):
record = Record(**{"title": "Hello", "__deepcopy__": "not a method"})
clone = copy.deepcopy(record)
self.assertEqual(clone.title, "Hello")
unittest.TextTestRunner(verbosity=2).run(
unittest.defaultTestLoader.loadTestsFromTestCase(Records)
)Hint 1: the idea
__getattr__ only runs when ordinary lookup has already failed, so it’s where
“a field with this name” belongs. Three cases, in this order: a dunder name —
refuse it at once with AttributeError, so protocols like copy.deepcopy
behave; a known field — return it; anything else — AttributeError with a
message naming the field and the fields that do exist.
Hint 2: a first step
Handle the known-field case first, and run: the first test should pass. Read
the fields through self.__dict__, not self.fields — this lesson explains why
self.fields inside __getattr__ can recurse forever at the moment fields
doesn’t exist yet.
Show one solution
def __getattr__(self, name):
if name.startswith("__") and name.endswith("__"):
raise AttributeError(name) # leave the protocol alone
fields = self.__dict__.get("fields", {})
if name in fields:
return fields[name]
known = ", ".join(sorted(fields)) or "none"
raise AttributeError(f"no field {name!r}; fields: {known}")The dunder guard is what makes the last test pass. Most special methods are
looked up on the type, which __getattr__ never sees — but copy.deepcopy is
one of the exceptions: it does getattr(x, "__deepcopy__", None) on the object
itself. Without the guard, a field with that name is returned as if it were a
method, and deepcopy calls a string. With it, deepcopy gets the
AttributeError it expects and falls back to copying the object field by field.
Reading self.__dict__ rather than self.fields is the anti-recursion habit
from the lesson, and deepcopy is exactly where it pays off: it builds the new
object without calling __init__, so for a moment the clone has no fields
at all.
What you learned
- Attributes are searched: data descriptors on the type, then the instance dict,
then the class and its MRO, then
__getattr__. - Assignment always writes to the instance, which is how a shared class attribute quietly becomes a per-instance one.
super()follows the MRO of the object’s type, not the class it’s written in.__getattr__runs only after a failed lookup; it must raiseAttributeError, avoid touching attributes that may not exist yet, and stay out of dunder names.