TIL: Testing CLI Tools With pytest in Python

A CLI tool that nobody tests is a CLI tool that will break. pytest makes testing command line output straightforward.

import subprocess

def test_cli_transcribe():
    result = subprocess.run(
        ['python', 'cli.py', 'transcribe', 'test.wav'],
        capture_output=True,
        text=True
    )
    assert result.returncode == 0
    assert 'transcript' in result.stdout

Run your CLI as a subprocess and check the exit code, stdout, and stderr.

def test_cli_missing_file():
    result = subprocess.run(
        ['python', 'cli.py', 'transcribe'],
        capture_output=True,
        text=True
    )
    assert result.returncode == 2
    assert 'error' in result.stderr.lower()

Test error handling by passing invalid arguments. The exit code tells you whether the CLI failed gracefully.

What about Click based CLIs?

Click has a CliRunner for testing without subprocess. It invokes the CLI in-process and captures output.