The Silent Miscount

The on-call team asked for a number at the top of their report: how many records need attention, meaning warnings or worse. Someone added it in five minutes, with a test. It’s been in the report for a week.

Bug hunt: four problems, one reported

import sys
import unittest
from dataclasses import dataclass


@dataclass(frozen=True)
class LogRecord:
    level: str
    source: str


def needs_attention(records):
    """Records at WARNING or worse."""
    return [r for r in records if r.level >= "WARNING"]


TODAY = [
    LogRecord("INFO", "auth"),
    LogRecord("WARNING", "payments"),
    LogRecord("ERROR", "payments"),
    LogRecord("CRITICAL", "db"),
    LogRecord("ERROR", "search"),
    LogRecord("DEBUG", "auth"),
]
print(f"{len(needs_attention(TODAY))} record(s) need attention")


class TestNeedsAttention(unittest.TestCase):
    def test_warnings_need_attention(self):
        records = [LogRecord("INFO", "auth"), LogRecord("WARNING", "db")]
        self.assertEqual(needs_attention(records), [records[1]])


suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestNeedsAttention)
unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)

Reproduce. Count by eye: one warning, two errors and a critical. That’s four records needing attention. The report says 1, and the test passes.

Inspect. Look at what the function kept and what it left out:

print([r.level for r in needs_attention(TODAY)])
print([r.level for r in TODAY if r not in needs_attention(TODAY)])

It kept WARNING. It left out ERROR and CRITICAL, the two levels that matter most.

Before you open this, explain why the existing test passes, and write one that fails. Our diagnosis and test are below. The fix is the rest of the lesson.

print(sorted(["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]))

def test_every_level_from_warning_up_needs_attention(self):
    records = [LogRecord(level, "db") for level in ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]]
    self.assertEqual([r.level for r in needs_attention(records)], ["WARNING", "ERROR", "CRITICAL"])

Fix. That’s the interesting part, because there are two quite different cures.

Cure one: compare ranks, not names

The analyzer already has a correct version, the level_at_least filter from Day 4, which compares positions in LEVEL_ORDER:

def needs_attention(records):
    return [r for r in records if LEVEL_ORDER.index(r.level) >= LEVEL_ORDER.index("WARNING")]

It fixes this function. It doesn’t stop the next person writing r.level >= "ERROR" somewhere else, or r.level == "Error", or level_at_least("WARNIGN") in a place that isn’t checked early. Levels are still just strings. Anything can be a level, any comparison is allowed, and most of them are wrong.

Cure two: make levels their own type

The analyzer’s levels are a fixed, known set of values with a meaningful order. That’s exactly what an Enum is for:

from enum import IntEnum, StrEnum


class Level(IntEnum):
    DEBUG = 10
    INFO = 20
    WARNING = 30
    ERROR = 40
    CRITICAL = 50


print(Level.ERROR >= Level.WARNING)                # ordered by severity
print(Level["WARNING"], Level(40), list(Level)[:2])
print(sorted([Level.CRITICAL, Level.DEBUG, Level.ERROR]))
print(Level.ERROR == 40)                           # an IntEnum is also an int

try:
    Level.WARNIGN
except AttributeError as error:
    print("AttributeError:", error)


class Named(StrEnum):
    WARNING = "WARNING"
    ERROR = "ERROR"


print(Named.ERROR >= Named.WARNING)                # a StrEnum compares like text

What you get:

• The order is the severity. Level.ERROR >= Level.WARNING is True, because an IntEnum compares by its numbers. The numbers match Python’s logging module, which will matter on Day 7.
• A typo fails at the line with the typo. Level.WARNIGN raises AttributeError immediately. A string "WARNIGN" is just a string that matches nothing.
• Converting is explicit. Level["WARNING"] looks up by name, Level(40) by value, and both raise for anything that isn’t a level.
• A checker can help again. A function that takes level: Level rejects a plain string, so the conversion from text has to happen, in one place.

And what to watch for:

• The kind of Enum decides the comparison. A plain Enum has no order at all: < raises TypeError. A StrEnum compares as text, which is today’s bug all over again. The last line of the playground prints False. Only IntEnum, or an Enum you give an explicit order, compares by severity.
• IntEnum equals plain ints. Level.ERROR == 40 is True. That’s convenient for interoperating with logging. It also means Level.ERROR == 40 can sneak into code, where a name would be clearer.
• It prints as a number. print(Level["WARNING"]) printed 30. An IntEnum is an int, so it formats like one. A report wants names, so the analyzer’s Level defines __str__ to return self.name.

Converting at the edge

Log files contain text, so text has to become a Level somewhere. As with timestamps, do it once, where the text comes in, and have nothing downstream ever handle a level as a string:

ALIASES = {"WARN": "WARNING", "ERR": "ERROR", "FATAL": "CRITICAL"}


class Level(IntEnum):
    ...

    @classmethod
    def from_text(cls, text: str) -> Self:
        """Level.from_text("warn") is Level.WARNING. Anything unknown raises ValueError."""
        name = text.strip().upper()
        try:
            return cls[ALIASES.get(name, name)]
        except KeyError:
            raise ValueError(f"unknown level {text!r}") from None

The conversion is also the natural place to handle messy input. The billing service logs WARN and ERR. With levels as strings, every comparison in the program would need to know that. With from_text, one dictionary does.

ALIASES lives outside the class on purpose. Inside an Enum class body, every plain assignment becomes a member, so a dictionary there would become Level.ALIASES, a sixth “level”. Declared inside an IntEnum, it doesn’t even get that far: the class fails to build, because a dict isn’t an int.

So, is an Enum the right cure?

This is your decision, and it’s a real one. An Enum costs something: every place that reads or writes levels, like the parser, the database, JSON and tests, has to convert, and some code has to change.

Reach for an Enum when:

• the values are a fixed set you know when you write the code,
• they’re compared or ordered in more than one place,
• a typo should fail loudly, not match nothing,
• they come in from outside as text, so there’s already an edge to convert at.

Keep plain strings when the set is open-ended. Source names like auth and payments change whenever a team adds a service. An Enum of sources would need a code change every time, and it would reject real data.

Levels tick every box, so the Day 5 reference adopts Level. If you decided otherwise, the ranks-not-names cure is a sound fix. What matters is that it’s a decision, with its costs known, and not the default you drifted into.

Your turn

import sys
import unittest
from enum import IntEnum
from typing import Self

ALIASES = {"WARN": "WARNING", "ERR": "ERROR", "FATAL": "CRITICAL"}


class Level(IntEnum):
    """Ordered by severity. The numbers match the standard logging module."""
    # your code here


class TestLevel(unittest.TestCase):
    def test_levels_are_ordered_by_severity(self):
        self.assertLess(Level.WARNING, Level.ERROR)
        self.assertEqual(sorted([Level.ERROR, Level.DEBUG, Level.CRITICAL]),
                         [Level.DEBUG, Level.ERROR, Level.CRITICAL])

    def test_numbers_match_logging(self):
        self.assertEqual([int(level) for level in Level], [10, 20, 30, 40, 50])

    def test_from_text_accepts_other_spellings(self):
        for text, level in {"warn": Level.WARNING, "Error": Level.ERROR,
                            " ERR ": Level.ERROR, "FATAL": Level.CRITICAL}.items():
            with self.subTest(text=text):
                self.assertIs(Level.from_text(text), level)

    def test_from_text_rejects_anything_else(self):
        for text in ["LOUD", "", "40"]:
            with self.subTest(text=text):
                with self.assertRaises(ValueError):
                    Level.from_text(text)

    def test_prints_as_its_name(self):
        self.assertEqual(f"{Level.ERROR} {Level.INFO!s:>6}", "ERROR   INFO")


suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestLevel)
unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)

class Level(IntEnum):
    """Ordered by severity. The numbers match the standard logging module."""

    DEBUG = 10
    INFO = 20
    WARNING = 30
    ERROR = 40
    CRITICAL = 50

    def __str__(self) -> str:
        return self.name

    @classmethod
    def from_text(cls, text: str) -> Self:
        """Level.from_text("warn") is Level.WARNING. Anything unknown raises ValueError."""
        name = text.strip().upper()
        try:
            return cls[ALIASES.get(name, name)]
        except KeyError:
            raise ValueError(f"unknown level {text!r}") from None

What you can now do

• Recognise a comparison that’s well typed, well tested and still wrong.
• Test a small fixed set of inputs exhaustively, rather than a convenient pair.
• Use an IntEnum for a fixed, ordered set, and know how Enum, IntEnum and StrEnum differ when compared and printed.
• Convert text to an enum once, at the edge, with aliases and a clear error.
• Decide when an Enum is worth its cost, and when plain strings are better.

Next: the Day 5 checkpoint, where Level goes through the whole analyzer.