Aller au contenu

Testing Approach

Context

Constitution Principle III defines the test-first rule for this project — and its exception. Getting this wrong in either direction has real costs: skipping tests for behavior changes means bugs slip to production; writing tests for documentation or operational tools wastes time with no reliability benefit.

Details

The rule and its exception

IF the change affects application behavior:
    → Test-first is NON-NEGOTIABLE
    → A failing test MUST exist before implementation
    → The change is not complete until the test passes in CI

IF the change does NOT affect application behavior:
    → Tests are NOT required
    → Covers: documentation, operational procedures, workflow tooling,
               administrative scripts, internal process changes

Examples of the exception:
    ✓ Activity report authoring
    ✓ mkdocs.yml nav updates
    ✓ GitHub issue creation scripts
    ✓ This wiki itself
    ✗ A new CSV parsing rule in the import pipeline (application behavior → test required)
    ✗ A new alert condition in SMTP code (application behavior → test required)

Unit test structure

def test_[unit]_[condition]_[expected_outcome]():
    # Arrange — set up the minimum state needed
    input_data = build_test_input(...)

    # Act — call the unit under test
    result = the_function_under_test(input_data)

    # Assert — verify ONE thing per test
    assert result == expected_value

    # Edge cases that MUST have separate tests:
    # - empty input
    # - boundary values (0, max, max+1)
    # - missing required fields
    # - invalid types

One assertion per test is a strong guideline. Multiple assertions in one test make it harder to pinpoint which condition failed.

Integration test structure

Integration tests cover the boundaries between components: database ↔ pipeline, pipeline ↔ API, API ↔ alert system.

def test_pipeline_integration_with_database():
    # Use the real database connection (test database)
    # Use fixture data — never production data
    # Verify the boundary contract: data written to DB matches what was computed

    db = connect_to_test_database()
    input_rows = load_fixture("happy_path.csv")

    run_pipeline(input_rows, db)

    stored_rows = db.query("SELECT * FROM holdings WHERE test_run = TRUE")
    assert len(stored_rows) == len(input_rows)
    assert all(row.price is not None for row in stored_rows)

When to mock vs. when to test real

MOCK these (they are external, unreliable, or cost money):
    - Marketstack API calls
    - SMTP send operations
    - GitHub API calls

DO NOT MOCK these (test them real in integration tests):
    - SQLAlchemy database queries (use a test DB)
    - File I/O (use tmp_path fixture)
    - Config parsing (use a test config.ini)

CI gate rule

Tests run in GitHub Actions on every push to main. A deployment cannot proceed if: - Any test fails - Coverage drops below the established baseline (currently ~67%)

Check .github/workflows/deploy.yml for current gate configuration.

Examples

Adding a new import source: Write a failing test in tests/ that exercises the new import path end-to-end with a fixture CSV. Run python -m pytest tests/test_new_import.py -v and confirm it fails. Then implement the feature until the test passes.