From Lines to Records

Warm-up: from Day 1

Three quick ones before the new material. From memory first — that’s the part that does the work.

1. Refactor or behaviour change? No code. For each change to yesterday’s script, decide which it is.

1. Renaming c to level_counts everywhere.
2. Making source_of remove a trailing colon only when there is one.
3. Moving the counting loop into a function count_levels(lines).
4. Skipping lines that start with #.

2. Predict. Today leans on dataclasses, so a reminder of what they give you:

from dataclasses import dataclass

@dataclass
class Reading:
    sensor: str
    value: float

a = Reading("t1", 20.5)
b = Reading("t1", 20.5)
print(a == b, a)
print([r.sensor for r in [a, Reading("t2", 3.0)] if r.value > 10])

True Reading(sensor='t1', value=20.5)
['t1']

3. Write source_of(line) so it drops the colon only when there is one. This should print db db:timeout:

# def source_of(line): ...

print(source_of("2026-09-01 10:00:00 ERROR db: timeout"),
      source_of("2026-09-01 10:00:05 ERROR db:timeout"))

def source_of(line):
    return line.split()[3].rstrip(":")

print(source_of("2026-09-01 10:00:00 ERROR db: timeout"),
      source_of("2026-09-01 10:00:05 ERROR db:timeout"))

db db:timeout

Where Day 1 ended

Yesterday you split a messy script into small, tested functions. Here is where you left off. Run executes the tests, and they all pass:

It works. But open analyzer.py and look at how it works. level_of knows the level is the third word. source_of knows the source is the fourth, with a colon stuck to it. Every function that needs anything re-slices the raw string itself.

That has three costs, and you will feel all of them today:

• The layout is known in many places. Change the log format and every function changes with it.
• Everything is a string. "2026-09-01" is not a date; you cannot ask it what hour it was.
• Bad lines break things far from where they came in. A junk line crashes level_of deep inside a report, not where it was read.

The fix is an old one: parse each line once, at the edge of the program, into a value with named parts. Everything after that works with the value.

A record for one line

You met dataclasses on Day 6 of the beginner course. They are exactly the tool here: a record is a few named values and nothing else.

from dataclasses import dataclass
from datetime import datetime


@dataclass(frozen=True)
class LogRecord:
    timestamp: datetime
    level: str
    source: str
    message: str


record = LogRecord(datetime(2026, 9, 1, 9, 14, 2), "INFO", "auth", "user=ada")
print(record)
print(record.level, record.timestamp.hour)
record.level = "ERROR"

frozen=True is a deliberate choice. A record describes something that already happened, so nothing downstream should be able to change it by accident, and the last line proves that nothing can. One limit to know: freezing stops a field being reassigned, not a mutable value inside it being changed. If a record holds a dict, as the analyzer’s will by the end of today, record.fields["x"] = 1 still works. Treat such values as read-only by convention.

Parsing the prefix with string methods

Every line starts with the same four things: date, time, level, and source:. Where the parts are separated by whitespace and no part contains whitespace, string methods are all you need.

line = "2026-09-01 09:14:02 INFO    auth: user=ada action=login"

print(line.split())              # every word - the message is chopped up
print(line.split(maxsplit=3))    # stop after 3 splits - the rest stays whole

date, time, level, rest = line.split(maxsplit=3)
source, colon, message = rest.partition(":")
print(repr(source), repr(colon), repr(message))

Two small tools doing the work:

• split(maxsplit=3) splits on runs of whitespace, so the padding after INFO doesn’t matter. It stops after three splits, so the message stays in one piece.
• partition(":") splits at the first colon and always returns three parts. If there’s no colon at all, the middle part is empty, which will be useful later today when bad lines turn up.

And the timestamp becomes a real datetime with datetime.fromisoformat(f"{date} {time}").

Bug hunt: the invisible character

Same routine as yesterday: reproduce it with a small input, inspect the failure and the relevant state, narrow it down to the cause, pin it with a test, then fix it.

Here is a first attempt at parse_record, with a test that fails:

import sys
import unittest
from dataclasses import dataclass
from datetime import datetime


@dataclass(frozen=True)
class LogRecord:
    timestamp: datetime
    level: str
    source: str
    message: str


def parse_record(line):
    date, time, level, rest = line.split(maxsplit=3)
    source, _, message = rest.partition(":")
    return LogRecord(datetime.fromisoformat(f"{date} {time}"), level, source, message)


class TestParseRecord(unittest.TestCase):
    def test_message(self):
        record = parse_record("2026-09-01 09:14:02 INFO    auth: user=ada")
        self.assertEqual(record.message, "user=ada")


suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestParseRecord)
unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)

Reproduce. Already done: one line of input, one assertion. Small inputs are the point. When a failing case is small enough to read in full, the cause often jumps out.

Inspect. Read the failure before the code. ' user=ada' != 'user=ada' and the ? - marker under it point at one character: a leading space. When the difference is whitespace, print(repr(value)) is the tool to reach for, because repr shows spaces, tabs and \r that an ordinary print hides.

Narrow. Where did the space come from? partition(":") splits at the colon, so the message starts with whatever came after it, and in this format that’s a space. It was already visible in the previous playground’s output: ' user=ada action=login'. Easy to miss, which is exactly why a test caught it.

Pin it. The test you just watched fail is the regression test. Keep it, and this bug can never quietly return.

Fix. Add message = message.strip() after the partition line, and run again.

Your turn

def parse_record(line):
    """Turn one log line into a LogRecord."""
    date, time, level, rest = line.split(maxsplit=3)
    source, _, message = rest.partition(":")
    return LogRecord(
        timestamp=datetime.fromisoformat(f"{date} {time}"),
        level=level,
        source=source,
        message=message.strip(),
    )

What you can now do

• Parse a line once, at the edge of the program, into a frozen dataclass.
• Choose split(maxsplit=…) and partition when whitespace and a fixed separator describe the format.
• Run the day’s bug-hunt routine: reproduce small, inspect with repr, narrow, pin with a test, fix.

Next: the message itself holds key=value pairs, and one of them is about to defeat split().