Time to start the library. By the end of this lesson Post(title="Hello").save(connection)
writes a row, Post.get(connection, 1) reads it back as a Post, and you will
have met the first design decision with real consequences.
What the course supplies
db.py is given to you and stays unchanged all week: a SQLite connection with
two tables already created, and rows that behave like dictionaries.
import sqlite3SCHEMA = """create table if not exists post ( id integer primary key autoincrement, title text not null, views integer not null default 0);create table if not exists user ( id integer primary key autoincrement, email text not null);"""def connect(path=":memory:"): connection = sqlite3.connect(path) connection.row_factory = sqlite3.Row connection.executescript(SCHEMA) return connection
row_factory = sqlite3.Row is what lets dict(row) give you {"id": 1, "title": "Hello", ...}, which is how a row becomes keyword arguments for a model.
One base class, a subclass per table
The shape is the one every ORM uses: a base class with the behaviour, and a
subclass per table carrying only what differs.
class Post(Model): table = "post" columns = ("title", "views")
Those two are class attributes, so self.table finds "post" through the
lookup you learned about in the last lesson, and there is exactly one copy of
each string no matter how many posts exist. They are also immutable — a string
and a tuple — which is what makes sharing them safe. Tomorrow, columns is
replaced by field objects that can validate; today it’s a tuple of names.
__init__ takes the columns as keyword arguments, and refuses anything else:
def __init__(self, id=None, **values): self.id = id for name in self.columns: setattr(self, name, values.pop(name, None)) if values: raise TypeError(f"{type(self).__name__} has no column {sorted(values)[0]!r}")
setattr(self, name, ...) writes to the instance, so each post gets its own
title and views. Popping as it goes leaves values holding only the names that
aren’t columns, and a library that says Post has no column 'author' costs one
line and saves its callers an afternoon.
Values are parameters; names are code
save builds SQL. Two different rules apply to the two kinds of things going
into that string.
Parameters, and what they don't coverPython
import sqlite3connection = sqlite3.connect(":memory:")connection.execute("create table post (id integer primary key, title text)")title = "Robert'); drop table post; --"connection.execute("insert into post (title) values (?)", (title,))print("stored safely:", connection.execute("select title from post").fetchone()[0])print("table still here:", connection.execute("select count(*) from post").fetchone()[0])try: connection.execute("select * from ? where id = 1", ("post",))except sqlite3.OperationalError as error: print("OperationalError:", error)
A ? parameter is never parsed as SQL, so the injection attempt is stored as a
title and nothing else happens. But a parameter can’t stand in for a table or
column name — SQLite has to know what it’s selecting from before it binds
anything.
So identifiers have to be interpolated, and the only safe source for them is
your own code:
names = ", ".join(self.columns) # from the class marks = ", ".join("?" for _ in self.columns) connection.execute(f"insert into {self.table} ({names}) values ({marks})", values)
self.table and self.columns are written in a class body by the developer
using your library, and the values are bound as parameters. That distinction —
identifiers from code, data from parameters — is what keeps this f-string from
being the mistake it looks like. Day 3 makes it a documented rule, and shows
what happens when a library lets a caller supply a column name.
insert or update?
save() has to decide. The rule this library uses: self.id is None means the
row doesn’t exist yet, so insert and record cursor.lastrowid as the new id;
otherwise update the row with that id. One method, two statements, and the
caller never thinks about which.
The checkpoint: write save and get
Everything else is in place. models.py has the class, the tests, and two
methods raising NotImplementedError.
Checkpoint
Day 1 checkpoint: save and get
Make the seven tests pass. Read the docstrings — they say what each method owes
its caller, including the cache, which the next section is about.
models.pyPython"""A model that saves and loads itself.
One base class, and a subclass per table. A subclass says which table it maps
to and which columns it owns; the base class does the work for all of them.
"""
class Model:
"""Rows of one table, as objects.
Subclasses set `table` and `columns`. `id` is the primary key: None until
the row exists in the database.
"""
table = ""
columns = ()
def __init__(self, id=None, **values):
self.id = id
for name in self.columns:
setattr(self, name, values.pop(name, None))
if values:
raise TypeError(f"{type(self).__name__} has no column {sorted(values)[0]!r}")
def save(self, connection):
"""Insert this object, or update the row it came from. Returns self.
TODO: read the column values off self with getattr, then
- self.id is None: insert them, and put the new lastrowid on self.id
- otherwise: update the row with that id
Values go in as ? parameters. Table and column names can't: they come
from the class, which is why an f-string is safe for those alone.
Finish by committing, caching self under (type(self), self.id), and
returning self.
"""
raise NotImplementedError("save")
@classmethod
def get(cls, connection, id):
"""The row with this id, as an instance of this class.
TODO: return the cached object if there is one, otherwise select the
row, raise LookupError when there is none, build cls(**dict(row)),
cache it and return it.
"""
raise NotImplementedError("get")
# One dictionary, shared by every model — which is why the key includes the
# class. See "Bug hunt" in the lesson: keying on the id alone let a User
# come back from a Post.get().
_instances = {}
@classmethod
def cache(cls):
return Model._instances
def __repr__(self):
shown = ", ".join(f"{name}={getattr(self, name)!r}" for name in self.columns)
return f"{type(self).__name__}(id={self.id!r}, {shown})"
class Post(Model):
table = "post"
columns = ("title", "views")
class User(Model):
table = "user"
columns = ("email",)
"""SQLite scaffolding, supplied by the course.
Day 1 is about Python's object model, not about SQL, so this file is given to
you: a connection with the library's two tables already created. The rest of
the week replaces almost everything else, but never this.
"""
import sqlite3
SCHEMA = """
create table if not exists post (
id integer primary key autoincrement,
title text not null,
views integer not null default 0
);
create table if not exists user (
id integer primary key autoincrement,
email text not null
);
"""
def connect(path=":memory:"):
"""A connection whose rows behave like dictionaries."""
connection = sqlite3.connect(path)
connection.row_factory = sqlite3.Row
connection.executescript(SCHEMA)
return connection
import unittest
from db import connect
from models import Post, User
class SaveAndLoad(unittest.TestCase):
def setUp(self):
self.connection = connect()
Post._instances.clear() # a fresh cache per test, like a fresh process
def test_save_gives_the_object_an_id(self):
post = Post(title="Hello", views=3)
self.assertIsNone(post.id)
post.save(self.connection)
self.assertEqual(post.id, 1)
def test_get_reads_the_columns_back(self):
Post(title="Hello", views=3).save(self.connection)
loaded = Post.get(self.connection, 1)
self.assertEqual(loaded.title, "Hello")
self.assertEqual(loaded.views, 3)
def test_saving_twice_updates_instead_of_inserting(self):
post = Post(title="Hello", views=3).save(self.connection)
post.views = 4
post.save(self.connection)
rows = self.connection.execute("select count(*) from post").fetchone()[0]
self.assertEqual(rows, 1)
self.assertEqual(self.connection.execute("select views from post").fetchone()[0], 4)
def test_a_missing_row_is_an_error(self):
with self.assertRaises(LookupError):
Post.get(self.connection, 99)
def test_an_unknown_column_is_refused(self):
with self.assertRaises(TypeError):
Post(title="Hello", author="nobody")
def test_two_models_with_the_same_id_stay_apart(self):
"""The shared cache is keyed by class as well as id."""
Post(title="Hello", views=0).save(self.connection)
User(email="someone@example.com").save(self.connection)
self.assertIsInstance(Post.get(self.connection, 1), Post)
self.assertIsInstance(User.get(self.connection, 1), User)
def test_get_returns_the_same_object_each_time(self):
post = Post(title="Hello", views=0).save(self.connection)
self.assertIs(Post.get(self.connection, post.id), post)
import sys
import unittest
import test_models
suite = unittest.defaultTestLoader.loadTestsFromModule(test_models)
unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)
Hint 1: the idea
save and get are two halves of one idea. Every column name comes from
self.columns and goes into the SQL text — safe, because the class wrote them.
Every value travels as a ? parameter. Build both lists from the same
columns, in the same order, and they can’t drift out of step. The cache key
is (class, id).
Hint 2: a first step
Write the insert branch of save first: the values via getattr, one ? per
column, and self.id = cursor.lastrowid afterwards. Run: the tests about new
posts should pass. Then the update branch, and get last.
Show one solution
def save(self, connection): values = [getattr(self, name) for name in self.columns] names = ", ".join(self.columns) if self.id is None: marks = ", ".join("?" for _ in self.columns) cursor = connection.execute( f"insert into {self.table} ({names}) values ({marks})", values ) self.id = cursor.lastrowid else: assignments = ", ".join(f"{name} = ?" for name in self.columns) connection.execute( f"update {self.table} set {assignments} where id = ?", [*values, self.id] ) connection.commit() self.cache()[(type(self), self.id)] = self return self @classmethod def get(cls, connection, id): cached = cls.cache().get((cls, id)) if cached is not None: return cached row = connection.execute( f"select * from {cls.table} where id = ?", (id,) ).fetchone() if row is None: raise LookupError(f"no {cls.__name__} with id {id}") found = cls(**dict(row)) cls.cache()[(cls, id)] = found return found
Three things worth pointing at:
getattr(self, name) reads the columns in the order self.columns lists them,
and the same list builds the placeholders — so values and ?s can’t drift out
of step.
return self makes Post(...).save(connection) an expression, which is why
the tests can write post = Post(title="Hello").save(connection).
cls(**dict(row)) includes id, which __init__ takes as its first
parameter. A loaded object therefore has an id, and saving it updates instead
of inserting a duplicate.
Bug hunt: two models, one cache
Your get() keeps loaded objects in a dictionary, so asking twice costs one
query. That dictionary is where today’s bug lives.
Run the playground below. Six tests pass and one fails.
ReproducePython"""A model that saves and loads itself.
One base class, and a subclass per table. A subclass says which table it maps
to and which columns it owns; the base class does the work for all of them.
"""
class Model:
"""Rows of one table, as objects.
Subclasses set `table` and `columns`. `id` is the primary key: None until
the row exists in the database.
"""
table = ""
columns = ()
def __init__(self, id=None, **values):
self.id = id
for name in self.columns:
setattr(self, name, values.pop(name, None))
if values:
raise TypeError(f"{type(self).__name__} has no column {sorted(values)[0]!r}")
def save(self, connection):
"""Insert this object, or update the row it came from. Returns self."""
values = [getattr(self, name) for name in self.columns]
names = ", ".join(self.columns)
if self.id is None:
marks = ", ".join("?" for _ in self.columns)
cursor = connection.execute(
f"insert into {self.table} ({names}) values ({marks})", values
)
self.id = cursor.lastrowid
else:
assignments = ", ".join(f"{name} = ?" for name in self.columns)
connection.execute(
f"update {self.table} set {assignments} where id = ?", [*values, self.id]
)
connection.commit()
self.cache()[self.id] = self
return self
@classmethod
def get(cls, connection, id):
"""The row with this id, as an instance of this class."""
cached = cls.cache().get(id)
if cached is not None:
return cached
row = connection.execute(
f"select * from {cls.table} where id = ?", (id,)
).fetchone()
if row is None:
raise LookupError(f"no {cls.__name__} with id {id}")
found = cls(**dict(row))
cls.cache()[id] = found
return found
# Saved objects, so a second get() of the same row costs nothing.
_instances = {}
@classmethod
def cache(cls):
return Model._instances
def __repr__(self):
shown = ", ".join(f"{name}={getattr(self, name)!r}" for name in self.columns)
return f"{type(self).__name__}(id={self.id!r}, {shown})"
class Post(Model):
table = "post"
columns = ("title", "views")
class User(Model):
table = "user"
columns = ("email",)
"""SQLite scaffolding, supplied by the course.
Day 1 is about Python's object model, not about SQL, so this file is given to
you: a connection with the library's two tables already created. The rest of
the week replaces almost everything else, but never this.
"""
import sqlite3
SCHEMA = """
create table if not exists post (
id integer primary key autoincrement,
title text not null,
views integer not null default 0
);
create table if not exists user (
id integer primary key autoincrement,
email text not null
);
"""
def connect(path=":memory:"):
"""A connection whose rows behave like dictionaries."""
connection = sqlite3.connect(path)
connection.row_factory = sqlite3.Row
connection.executescript(SCHEMA)
return connection
import unittest
from db import connect
from models import Post, User
class SaveAndLoad(unittest.TestCase):
def setUp(self):
self.connection = connect()
Post._instances.clear() # a fresh cache per test, like a fresh process
def test_save_gives_the_object_an_id(self):
post = Post(title="Hello", views=3)
self.assertIsNone(post.id)
post.save(self.connection)
self.assertEqual(post.id, 1)
def test_get_reads_the_columns_back(self):
Post(title="Hello", views=3).save(self.connection)
loaded = Post.get(self.connection, 1)
self.assertEqual(loaded.title, "Hello")
self.assertEqual(loaded.views, 3)
def test_saving_twice_updates_instead_of_inserting(self):
post = Post(title="Hello", views=3).save(self.connection)
post.views = 4
post.save(self.connection)
rows = self.connection.execute("select count(*) from post").fetchone()[0]
self.assertEqual(rows, 1)
self.assertEqual(self.connection.execute("select views from post").fetchone()[0], 4)
def test_a_missing_row_is_an_error(self):
with self.assertRaises(LookupError):
Post.get(self.connection, 99)
def test_an_unknown_column_is_refused(self):
with self.assertRaises(TypeError):
Post(title="Hello", author="nobody")
def test_two_models_with_the_same_id_stay_apart(self):
"""Two tables can both have a row 1. They are different objects."""
Post(title="Hello", views=0).save(self.connection)
User(email="someone@example.com").save(self.connection)
self.assertIsInstance(Post.get(self.connection, 1), Post)
self.assertIsInstance(User.get(self.connection, 1), User)
def test_get_returns_the_same_object_each_time(self):
post = Post(title="Hello", views=0).save(self.connection)
self.assertIs(Post.get(self.connection, post.id), post)
import sys
import unittest
import test_models
suite = unittest.defaultTestLoader.loadTestsFromModule(test_models)
unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)
Reproduce. The failing test saves a post and a user, then loads each by id:
Inspect. The message is specific: User(id=1, email='someone@example.com') is not an instance of <class 'models.Post'>. Post.get returned a User. Not a
wrong title — a wrong class.
Narrow. Both rows have id 1; both tables number from 1 independently. Add a
line to the failing test to see the cache itself:
print(Model._instances)
It holds {1: User(id=1, ...)}. One entry, because the user overwrote the post,
and one dictionary, because _instances = {} is written once in Model’s class
body. Every subclass inherits the same dictionary object — exactly the sharing
from the last two lessons, with a twist: there is no shared list in sight at the
call site, and Post._instances is User._instances is True.
Pin. The test that just failed is the regression test. It says what callers
need: Post.get returns a Post, whatever else is in the database.
Fix. The key has to identify the row and the class:
That’s the other fix, and it’s better — but writing _instances = {} in every
subclass puts a requirement on the people using your library, and they will
forget. Tomorrow __init_subclass__ does it for them automatically, which is
how a library enforces its own invariants instead of documenting them. Keying by
class is today’s honest fix: one line, no new machinery.
Use type(self) when saving and cls when loading, and note they’re the same
object: for post.save(...), type(post)isPost.
What the cache commits you to
An identity map is a real design decision, not an optimisation you can slip in:
Two get() calls for one row return the same object, so a change through
one name is visible through the other. Callers rely on this once they notice it.
Nothing ever leaves the cache, so a long-running process keeps every object it
has loaded alive. That’s a memory leak with a pleasant name, and you will
measure it with tracemalloc on Day 6.
A row changed by someone else won’t be re-read while a copy is cached.
Keeping it for now is a deliberate choice: it gives Day 6 something real to
measure, and Day 7 something real to argue about in the design write-up.
Where Day 1 leaves the library
The finished Day 1Python"""A model that saves and loads itself.
One base class, and a subclass per table. A subclass says which table it maps
to and which columns it owns; the base class does the work for all of them.
"""
class Model:
"""Rows of one table, as objects.
Subclasses set `table` and `columns`. `id` is the primary key: None until
the row exists in the database.
"""
table = ""
columns = ()
def __init__(self, id=None, **values):
self.id = id
for name in self.columns:
setattr(self, name, values.pop(name, None))
if values:
raise TypeError(f"{type(self).__name__} has no column {sorted(values)[0]!r}")
def save(self, connection):
"""Insert this object, or update the row it came from. Returns self."""
values = [getattr(self, name) for name in self.columns]
names = ", ".join(self.columns)
if self.id is None:
marks = ", ".join("?" for _ in self.columns)
cursor = connection.execute(
f"insert into {self.table} ({names}) values ({marks})", values
)
self.id = cursor.lastrowid
else:
assignments = ", ".join(f"{name} = ?" for name in self.columns)
connection.execute(
f"update {self.table} set {assignments} where id = ?", [*values, self.id]
)
connection.commit()
self.cache()[(type(self), self.id)] = self
return self
@classmethod
def get(cls, connection, id):
"""The row with this id, as an instance of this class."""
cached = cls.cache().get((cls, id))
if cached is not None:
return cached
row = connection.execute(
f"select * from {cls.table} where id = ?", (id,)
).fetchone()
if row is None:
raise LookupError(f"no {cls.__name__} with id {id}")
found = cls(**dict(row))
cls.cache()[(cls, id)] = found
return found
# One dictionary, shared by every model — which is why the key includes the
# class. See "Bug hunt" in the lesson: keying on the id alone let a User
# come back from a Post.get().
_instances = {}
@classmethod
def cache(cls):
return Model._instances
def __repr__(self):
shown = ", ".join(f"{name}={getattr(self, name)!r}" for name in self.columns)
return f"{type(self).__name__}(id={self.id!r}, {shown})"
class Post(Model):
table = "post"
columns = ("title", "views")
class User(Model):
table = "user"
columns = ("email",)
"""SQLite scaffolding, supplied by the course.
Day 1 is about Python's object model, not about SQL, so this file is given to
you: a connection with the library's two tables already created. The rest of
the week replaces almost everything else, but never this.
"""
import sqlite3
SCHEMA = """
create table if not exists post (
id integer primary key autoincrement,
title text not null,
views integer not null default 0
);
create table if not exists user (
id integer primary key autoincrement,
email text not null
);
"""
def connect(path=":memory:"):
"""A connection whose rows behave like dictionaries."""
connection = sqlite3.connect(path)
connection.row_factory = sqlite3.Row
connection.executescript(SCHEMA)
return connection
import unittest
from db import connect
from models import Post, User
class SaveAndLoad(unittest.TestCase):
def setUp(self):
self.connection = connect()
Post._instances.clear() # a fresh cache per test, like a fresh process
def test_save_gives_the_object_an_id(self):
post = Post(title="Hello", views=3)
self.assertIsNone(post.id)
post.save(self.connection)
self.assertEqual(post.id, 1)
def test_get_reads_the_columns_back(self):
Post(title="Hello", views=3).save(self.connection)
loaded = Post.get(self.connection, 1)
self.assertEqual(loaded.title, "Hello")
self.assertEqual(loaded.views, 3)
def test_saving_twice_updates_instead_of_inserting(self):
post = Post(title="Hello", views=3).save(self.connection)
post.views = 4
post.save(self.connection)
rows = self.connection.execute("select count(*) from post").fetchone()[0]
self.assertEqual(rows, 1)
self.assertEqual(self.connection.execute("select views from post").fetchone()[0], 4)
def test_a_missing_row_is_an_error(self):
with self.assertRaises(LookupError):
Post.get(self.connection, 99)
def test_an_unknown_column_is_refused(self):
with self.assertRaises(TypeError):
Post(title="Hello", author="nobody")
def test_two_models_with_the_same_id_stay_apart(self):
"""The shared cache is keyed by class as well as id."""
Post(title="Hello", views=0).save(self.connection)
User(email="someone@example.com").save(self.connection)
self.assertIsInstance(Post.get(self.connection, 1), Post)
self.assertIsInstance(User.get(self.connection, 1), User)
def test_get_returns_the_same_object_each_time(self):
post = Post(title="Hello", views=0).save(self.connection)
self.assertIs(Post.get(self.connection, post.id), post)
import sys
import unittest
import test_models
suite = unittest.defaultTestLoader.loadTestsFromModule(test_models)
unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)
Working on your own machine? Download the Day 1 files
and run python -m unittest in that directory — no run_tests.py needed
outside the browser.
What you learned
A subclass carries what differs (table, columns); the base class carries
the behaviour.
Values go into SQL as parameters; identifiers are interpolated, and must come
from your code rather than your caller’s input.
A mutable class attribute is one object shared by every subclass — the same
trap as a mutable default, with nothing at the call site to warn you.
Returning the same object for the same row is a promise about identity, with
costs you will measure later this week.
Tomorrow: columns = ("title", "views") becomes real field objects that
validate what you assign, and __init_subclass__ registers each model as it’s
defined.
Your progress is saved in this browser.
Sign in
to keep it on every device.