ML System Design Coverage¶
Status: ✅ Current as of May 2026 — coverage map complete. See Implementation Status for per-component readiness.
This page is a reading guide for technical interviewers and ML engineers who want to evaluate the project through the lens of ML System Design. It maps each major design topic to the specific files, modules, and decisions where it is addressed in this project.
The project covers all 15 topics. 11 are fully implemented; 3 are partially implemented (infrastructure gaps on the roadmap, not design gaps).
Coverage Map¶
| # | Topic | Coverage | Key Pointers |
|---|---|---|---|
| 1 | Problem & Solution Space | ✅ | problem.md, requirements.md |
| 2 | Preliminary Research | ✅ | baseline.md, limitations.md |
| 3 | Design Document | ✅ | requirements.md, tradeoffs.md, adr/ |
| 4 | Loss Functions & Metrics | ✅ | baseline.md, tuning.md |
| 5 | Datasets & Data Work | ✅ | data/index.md, data/contracts.md |
| 6 | Validation Schemes | ✅ | validation.md |
| 7 | Baselines | ✅ | baseline.md |
| 8 | Error Analysis | 🚧 | src/pipelines/error_analysis.py, limitations.md |
| 9 | Training Pipelines | ✅ | training-pipeline.md, dvc.yaml |
| 10 | Feature Engineering | ✅ | features.md, src/features/ |
| 11 | Reporting | 🚧 | MLflow UI, reports/qmd/, Results |
| 12 | Integration | ✅ | model-contract.md, serving/index.md |
| 13 | Monitoring & Reliability | 🚧 | monitoring/index.md, src/pipelines/monitor_drift.py |
| 14 | Inference Serving & Optimisation | ✅ | inference-modes.md, performance.md |
| 15 | Model Management & Governance | ✅ | model-registry.md, model-registry.md#promotion-policy |
Open gaps (🚧 topics)¶
| Topic | What is missing | Roadmap item |
|---|---|---|
| 8 — Error Analysis | No formal slice-analysis by league / season / team; error_analysis.py computes ROI only |
Post-v1 |
| 11 — Reporting | Dynamic ROI panel in Streamlit UI shows filtered historical ROI; dedicated model-quality metrics page (log-loss, calibration, champion vs challenger) not yet implemented | Post-v1 |
| 13 — Monitoring | Grafana dashboards and Alertmanager rules not yet deployed | Near-term roadmap |