Docs & Testing Audit — SoccerPredictAI¶
Date: 2026-04-28
Auditor: GitHub Copilot (Claude Opus 4.7) — /skill-ml-system-audit full (audit 11/12)
Scope: Documentation, ADRs, README, test coverage, contract tests, doc↔code drift
Baseline: docs/validation/20260424/11_docs_testing_audit.md
Delta vs baseline¶
docs/, tests/, README.md, CONTRIBUTING.md show modifications since baseline (see git status: CONTRIBUTING.md, docs/ml/mlflow.md, docs/ml/training-pipeline.md, docs/serving/deployment.md). However these are unstaged edits that pre-date the 2026-04-26 audit and have not been re-edited since. Source-code under tests/ unchanged since 2026-04-26. Baseline findings remain in force.
Confirmed posture¶
Docs structure: docs/{adr,architecture,cicd,data,guides,ml,monitoring,reference,runbooks,serving,validation} + index.md, quickstart.md, status.md. docs/status.md last updated 2026-04-19 — slightly stale relative to today (2026-04-28).
Test inventory (~294 tests across): tests/unit/ (~120), tests/property/ (~30), tests/service/ (~20), tests/contract/test_pipeline_contracts.py (~40), tests/load/locustfile.py, tests/test_api.py (~20), tests/test_features.py (~20), tests/test_metrics.py (~15), tests/test_params.py (~5), other (~25).
Property tests cover ELO ordering / range, feature leakage / shape, metric calibration, temporal split preservation.
Risk register (re-confirmed)¶
| ID | Severity | Description | Status |
|---|---|---|---|
| DOC-01 | P0 | test_pipeline_contracts.py::EXPECTED_STAGES includes validate_interim, which is absent from dvc.yaml → test fails |
Open (= D-01) |
| DOC-02 | P1 | docs/status.md claims Streamlit UI shows predictions + async polling, but src/ui/app/pages/ is empty |
Open |
| DOC-03 | P2 | Contract tests don't cover validate_finished, validate_future, ablation_study, tune_xgb, final_train |
Open |
| DOC-04 | P2 | No tests for Airflow DAGs | Open |
| DOC-05 | P2 | No tests for src/ui/app/ |
Open |
| DOC-06 | P3 | src/data/storage.py MinIO interaction is mock-only; no integration test |
Open |
Summary¶
| Aspect | Status |
|---|---|
| Documentation structure | ✅ |
status.md exists with explicit Implemented/Partial/Planned |
✅ but slightly stale (2026-04-19) |
| Test count + types | ✅ unit, property, service, contract, load |
Contract test ↔ dvc.yaml alignment |
❌ (DOC-01) |
status.md ↔ UI alignment |
❌ (DOC-02) |
| DAG / UI test coverage | ❌ (DOC-04, DOC-05) |
Recommendation: DOC-01 is a CI-red signal — fix immediately by either removing validate_interim from EXPECTED_STAGES or adding the missing DVC stage. DOC-02 should be resolved by either implementing the UI prediction page or correcting the status entry. Update docs/status.md last-reviewed date as part of this cycle.
See baseline §1–§3 for code-level detail and audit 12_docs_validation for full claims-vs-code validation.