Copying, Shallow and Deep

“Give me a copy” is ambiguous, and Python answers it three ways. A library that hands out objects has to know which one it is doing, because the caller will assume the most convenient answer.

Shallow: a new container, the same contents

list(x), x[:], dict(x), x.copy() and copy.copy(x) all build one new object whose elements are the same objects as before. That’s enough when the elements are strings or numbers, and not enough as soon as one of them is itself mutable.

import copy

original = {"title": "Hello", "tags": ["python", "orm"]}
shallow = copy.copy(original)

shallow["title"] = "Changed"          # rebinds a key in the new dict only
shallow["tags"].append("sqlite")      # mutates the list both dicts share

print("original:", original)
print("shallow: ", shallow)
print("same dict?", original is shallow, " same list?", original["tags"] is shallow["tags"])

The title stayed put; the tags did not. Nothing here is broken — this is what “shallow” means — but a method called copy() that behaves like this will surprise a caller who nests data one level deeper than you imagined.

Deep: copy everything reachable

copy.deepcopy(x) walks the whole graph of objects and rebuilds it, keeping track of what it has already copied so shared objects stay shared and cycles don’t loop forever.

import copy

shared = ["python"]
original = {"a": shared, "b": shared}
original["self"] = original           # a cycle: deepcopy handles it

deep = copy.deepcopy(original)
deep["a"].append("orm")

print("original a:", original["a"])
print("deep a:    ", deep["a"])
print("still one list inside the copy?", deep["a"] is deep["b"])
print("cycle rebuilt?", deep["self"] is deep)

Notice the second-to-last line: a and b referred to one list, and they still do in the copy. deepcopy preserves the shape of what it copies.

Deep copies fail on things that can’t be copied

An object is only deep-copyable if everything it holds is. Database connections, sockets, locks and open files are not: they refer to something outside the process, and duplicating that is meaningless.

import copy
import sqlite3


class Row:
    """A row that keeps its own connection — which is the mistake here."""

    def __init__(self, connection, title):
        self.connection = connection
        self.title = title


row = Row(sqlite3.connect(":memory:"), "Hello")

try:
    copy.deepcopy(row)
except TypeError as error:
    print("TypeError:", error)

You can teach a class how to copy itself by writing __copy__ and __deepcopy__, and sometimes that’s the right answer. More often, an error like this is telling you something about the design: an object that holds a connection can’t be copied, pickled, or sent to another process.

That’s why Model.save(connection) in this course takes the connection as an argument instead of storing it on the instance. The model stays a plain bundle of data, and the one thing that can’t be copied stays outside it. Design choices like this one are the week’s real subject; the mechanics of copy are just how you notice them.

The default argument that everyone hits once

A function’s default values are evaluated once, when the def runs — not on each call. A mutable default is therefore one object shared by every call that doesn’t supply its own.

def add_tag(tag, tags=[]):          # the trap
    tags.append(tag)
    return tags

print(add_tag("python"))
print(add_tag("orm"))
print(add_tag("sqlite"))
print("the default itself:", add_tag.__defaults__)


def add_tag_safely(tag, tags=None):   # the fix
    tags = [] if tags is None else tags
    tags.append(tag)
    return tags

print(add_tag_safely("python"))
print(add_tag_safely("orm"))

add_tag.__defaults__ shows the shared list growing, which is the clearest proof that the default is one object with the same lifetime as the function.

The same rule applies to class bodies: a mutable class attribute is one object shared by every instance and every subclass. That’s today’s bug hunt, and it’s the harder version of this trap because nothing in the code looks like a default value at all.

import unittest


class Query:
    filters = []                      # bug 1

    def where(self, column, value):
        self.filters.append((column, value))   # bug 2
        return self

    def sql(self):
        clauses = " and ".join(f"{name} = ?" for name, _ in self.filters)
        return f"select * from post where {clauses}" if clauses else "select * from post"

    def values(self):
        return [value for _, value in self.filters]


class QueryBuilder(unittest.TestCase):
    def test_builds_the_sql_and_the_values(self):
        query = Query().where("title", "Hello").where("views", 3)
        self.assertEqual(query.sql(), "select * from post where title = ? and views = ?")
        self.assertEqual(query.values(), ["Hello", 3])

    def test_two_queries_are_independent(self):
        first = Query().where("title", "Hello")
        second = Query().where("views", 3)
        self.assertEqual(first.values(), ["Hello"])
        self.assertEqual(second.values(), [3])

    def test_where_does_not_change_the_query_it_came_from(self):
        base = Query().where("title", "Hello")
        narrowed = base.where("views", 3)
        self.assertEqual(base.values(), ["Hello"])
        self.assertEqual(narrowed.values(), ["Hello", 3])
        self.assertIsNot(base, narrowed)


unittest.TextTestRunner(verbosity=2).run(
    unittest.defaultTestLoader.loadTestsFromTestCase(QueryBuilder)
)

class Query:
    def __init__(self, filters=()):
        self.filters = list(filters)

    def where(self, column, value):
        return Query([*self.filters, (column, value)])

What you learned

• Shallow copies duplicate the container and share the contents; deep copies rebuild the whole graph, preserving sharing and cycles.
• Some objects can’t be copied at all, and that is usually a hint about where they belong in your design.
• Defaults are evaluated once at definition time: never a mutable one.
• A class attribute is shared by every instance, which is the same trap wearing a different hat.