A Command Line with argparse

Everything the analyzer can do is currently reached by editing main.py. A colleague who wants warnings from payments only shouldn’t have to change code. They should type:

python cli.py server.log --level warning --source payments

Turning a list of words into settings is the job of argparse, in the standard library.

A parser in a few lines

import argparse

parser = argparse.ArgumentParser(prog="logan", description="Summarise server logs.")
parser.add_argument("logfile", nargs="+", help="log file(s) to read")
parser.add_argument("--top", type=int, default=5, metavar="N", help="how many sources to list")
parser.add_argument("--source", action="append", default=[], metavar="NAME",
                    help="only this source (repeatable)")
parser.add_argument("-v", "--verbose", action="store_true", help="log progress to stderr")

print(parser.parse_args(["messy.log"]))
print(parser.parse_args(["a.log", "b.log", "--top", "3", "--source", "db", "--source", "auth", "-v"]))

for argv in (["--help"], ["messy.log", "--top", "three"]):
    try:
        parser.parse_args(argv)
    except SystemExit as exit:
        print(f"(argparse exited with code {exit.code})\n")

Each add_argument describes one thing the user can type:

parse_args returns a Namespace, an object with one attribute per argument: args.logfile, args.top. You get --help for free, generated from the descriptions. On a bad command line, argparse prints the usage and an error to stderr, and exits with code 2.

Your own argument types

type= accepts any function that takes the text and returns a value. Raise argparse.ArgumentTypeError for bad input, and argparse turns it into a proper usage error. The analyzer’s --level uses Level.from_text, so --level warn works, and --level loud says what went wrong:

def level(text: str) -> Level:
    """argparse type for --level: a level's name, in any case, or an alias."""
    try:
        return Level.from_text(text)
    except ValueError:
        names = ", ".join(level.name.lower() for level in Level)
        raise argparse.ArgumentTypeError(f"unknown level {text!r} (choose from {names})") from None

$ python cli.py messy.log --level loud
usage: logan [-h] [--level LEVEL] [--source NAME] [--top N] [--json PATH]
             [--strict] [-v]
             logfile [logfile ...]
logan: error: argument --level: unknown level 'loud' (choose from debug, info, warning, error, critical)

Converting at the edge, again. The rest of the program only ever sees a Level.

main returns an exit code

A command-line program reports success or failure with its exit code: 0 for success, anything else for failure. Scripts and schedulers check it, as in logan server.log && send-report. logan’s codes are:

• 0: the report was printed, even if some lines were skipped (each one is logged as a warning).
• 1: a file couldn’t be read, or --strict met a bad line.
• 2: the command line itself was wrong. argparse uses 2 for this.

The shape of cli.py makes that easy to test:

def main(argv: Sequence[str] | None = None) -> int:
    """Run logan with these arguments (default: the command line). Returns the exit code."""
    args = build_parser().parse_args(argv)
    ...
    return 0


if __name__ == "__main__":
    sys.exit(main())

main returns the exit code rather than calling sys.exit itself. Only the last line, which runs when the file is executed as a program, turns it into a real exit. A test can call main([...]) with any arguments and check the number that comes back, without the test runner being shut down.

Bug hunt: the arguments that went missing

Here’s the analyzer’s cli.py with its test suite. Every test that calls main fails:

Reproduce. Every test fails. The ones that check an exit code all got 2, and the others find nothing they expected in the output.

Inspect. Exit code 2 means argparse rejected the command line. Look at what it said. The run helper captures stderr, so print it in a scratch line at the end of test_cli.py:

print(run("messy.log"))

stderr says the following arguments are required: logfile. But the test passed "messy.log".

Before you open this, find the fix, which is one word long. Then explain why the program still works perfectly from a terminal. Compare with ours.

Your turn

import argparse
import sys
import unittest
from datetime import time


def time_of_day(text):
    """argparse type for --since: "09:30" becomes time(9, 30)."""
    raise NotImplementedError("your code here")


def build_parser():
    parser = argparse.ArgumentParser(prog="logan")
    parser.add_argument("logfile", nargs="+")
    parser.add_argument("--since", type=time_of_day, metavar="HH:MM")
    return parser


class TestTimeOfDay(unittest.TestCase):
    def test_good_times(self):
        self.assertEqual(time_of_day("09:30"), time(9, 30))
        self.assertEqual(time_of_day("9:05"), time(9, 5))
        self.assertEqual(time_of_day("23:59"), time(23, 59))

    def test_bad_times_are_argument_errors(self):
        for text in ["24:00", "noon", "09:30:15", ""]:
            with self.subTest(text=text):
                with self.assertRaises(argparse.ArgumentTypeError) as caught:
                    time_of_day(text)
                self.assertEqual(str(caught.exception),
                                 f"expected a time like 09:30, not {text!r}")

    def test_the_parser_uses_it(self):
        args = build_parser().parse_args(["messy.log", "--since", "09:16"])
        self.assertEqual(args.since, time(9, 16))


suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestTimeOfDay)
unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)

from datetime import datetime, time


def time_of_day(text):
    """argparse type for --since: "09:30" becomes time(9, 30)."""
    try:
        return datetime.strptime(text, "%H:%M").time()
    except ValueError:
        raise argparse.ArgumentTypeError(f"expected a time like 09:30, not {text!r}") from None

What you can now do

• Describe a command line with argparse: positionals, options, flags, repeatable options and custom types.
• Report outcomes with exit codes: 0, 1, and argparse’s 2.
• Write main(argv=None) -> int, so a program can be run from the terminal and called from tests alike.
• Recognise parse_args() silently reading the wrong command line.

Next: the last feature, the README, and shipping it.