load_odds_fdco stage scrapes closing odds from football-data.co.uk and stores them in data/raw/odds_fdco.parquet. This dataset is joined to holdout predictions in the analysis stage to compute ROI benchmarks in sections 4–6 below.
Show code
import yamlimport pandas as pdfrom IPython.display import display, HTML, Markdownwithopen(project_root /"params.yaml") as _f: _params = yaml.safe_load(_f)_odds = pd.read_parquet(project_root /"data/raw/odds_fdco.parquet")_leagues = [l["name"] for l in _params["odds_fdco"].get("leagues", [])]_extra = [l["name"] for l in _params["odds_fdco"].get("extra_leagues", [])]_seasons = _params["odds_fdco"].get("seasons", [])display(Markdown(f"""| Property | Value ||---|---|| Rows | {len(_odds):,} || Columns | {_odds.shape[1]} (`season`, `league_code`, `date`, `home_team`, `away_team`, `ftr`, `b365h/d/a`, `vig`, …) || Date range | {str(_odds['date'].min())[:10]} → {str(_odds['date'].max())[:10]} || Seasons ingested | {', '.join(_seasons)} || Top-tier leagues | {len(_leagues)} (Premier League, La Liga, Bundesliga, Serie A, Ligue 1, …) || Extra leagues | {len(_extra)} (Norway, Sweden, Brazil, MLS, Japan, …) || Total leagues | {_odds['league_code'].nunique()} unique codes present |"""))
# Overall logloss / brier from the full holdout (all regions)# Use the weighted mean across all regions as proxylogloss_overall = np.average(df_err_region["logloss"], weights=df_err_region["n"])brier_overall = np.average(df_err_region["brier"], weights=df_err_region["n"])ece_overall = np.average(df_err_region["ece"], weights=df_err_region["n"])display(Markdown(f"""| Metric | Value ||---|---|| Log-loss (weighted avg) | {logloss_overall:.4f} || Brier score (weighted avg) | {brier_overall:.4f} || ECE (weighted avg) | {ece_overall:.4f} |"""))
Metric
Value
Log-loss (weighted avg)
1.0057
Brier score (weighted avg)
0.6011
ECE (weighted avg)
0.0267
2. Model quality by region
Regions sorted by logloss (ascending = better calibration). ★ = regions that also have Bet365/Pinnacle odds coverage (ROI benchmark available).
Show code
# Merge error + ROI region datadf_reg = df_err_region.copy()if"logloss"in df_roi_region.columns: df_roi_reg_slim = df_roi_region[["regionId", "n_bets", "roi_pct", "low_logloss"]].copy() df_reg = df_reg.merge(df_roi_reg_slim, left_on="id", right_on="regionId", how="left")logloss_median = df_reg["logloss"].median()# Top-30 by match count for the chartdf_top = df_reg.nlargest(30, "n").sort_values("logloss")has_bets = df_top["n_bets"].notna() if"n_bets"in df_top.columns else pd.Series(False, index=df_top.index)bar_colors = ["#e67e22"if hb else"#3498db"for hb in has_bets]fig, ax = plt.subplots(figsize=(10, 8))ax.barh(df_top["segment"], df_top["logloss"], color=bar_colors)ax.axvline(logloss_median, color="black", linewidth=0.9, linestyle="--", label=f"Median {logloss_median:.3f}")ax.set_xlabel("Log-loss")ax.set_title("Log-loss by region (top-30 by match count)")ax.invert_yaxis()legend_elements = [ mpatches.Patch(facecolor="#e67e22", label="Has odds coverage (★)"), mpatches.Patch(facecolor="#3498db", label="No odds data"),]ax.legend(handles=legend_elements, fontsize=8)plt.tight_layout()plt.show()
ELO gap = |home ELO − away ELO| before the match. Hypothesis: wider gap → more predictable → lower logloss.
Show code
fig, axes = plt.subplots(1, 2, figsize=(12, 4))# Left: loglossaxes[0].bar(df_err_elo["elo_gap_bin"], df_err_elo["logloss"], color="#3498db")axes[0].set_ylabel("Log-loss")axes[0].set_title("Log-loss by ELO gap bin")axes[0].tick_params(axis="x", rotation=20)# Right: ROIifnot df_roi_elo.empty: colors = ["#2ecc71"if v >=0else"#e74c3c"for v in df_roi_elo["roi_pct"]] axes[1].bar(df_roi_elo["elo_gap_bin"], df_roi_elo["roi_pct"], color=colors) axes[1].axhline(0, color="black", linewidth=0.8, linestyle="--") axes[1].set_ylabel("ROI %") axes[1].set_title("ROI by ELO gap bin (matched only)") axes[1].tick_params(axis="x", rotation=20)plt.tight_layout()plt.show()
Coverage: rows matched to Bet365/Pinnacle closing odds via team name fuzzy join. Flat-stake simulation: €1 per bet, payout = actual odds. Uniform prior fallback for unmatched rows.
Show code
row = df_roi_overall.iloc[0]n_matched =int(row.n_bets)n_total =int(row.n_matches)coverage = n_matched / n_totaldisplay(Markdown(f"""| Metric | Value ||---|---|| Holdout matches | {n_total:,} || Matched to odds | {n_matched:,} ({coverage:.1%}) || Bets placed | {n_matched:,} || Hit rate | {row.hit_rate:.3f} || Net profit | {row.net_profit:+.1f} units || **ROI** | **{row.roi_pct:.2f}%** |"""))
Metric
Value
Holdout matches
135,970
Matched to odds
13,317 (9.8%)
Bets placed
13,317
Hit rate
0.347
Net profit
-1526.9 units
ROI
-11.47%
5. ROI by region
Regions with odds coverage only. ★ = logloss below median across all regions.
Show code
df_rr = df_roi_region.copy()label_col ="region_name"if"region_name"in df_rr.columns else"regionId"has_ll ="low_logloss"in df_rr.columnsdf_plot = df_rr.sort_values("roi_pct", ascending=False).head(25)bar_colors = []y_labels = []for _, r in df_plot.iterrows(): name =str(r[label_col])if r["roi_pct"] >=0: bar_colors.append("#2ecc71")elif has_ll and r.get("low_logloss", False): bar_colors.append("#e67e22")else: bar_colors.append("#e74c3c") y_labels.append(("★ "+ name) if (has_ll and r.get("low_logloss", False)) else name)fig, ax = plt.subplots(figsize=(10, 7))ax.barh(y_labels, df_plot["roi_pct"].tolist(), color=bar_colors)ax.axvline(0, color="black", linewidth=0.8, linestyle="--")ax.set_xlabel("ROI %")ax.invert_yaxis()if has_ll: logloss_thr = df_roi_region.loc[df_roi_region["low_logloss"].notna(), "logloss"].median() ax.set_title(f"ROI by region (top-25 by bets)\n★ = logloss < {logloss_thr:.3f} (median)") legend_elements = [ mpatches.Patch(facecolor="#e67e22", label=f"Low logloss (< {logloss_thr:.3f})"), mpatches.Patch(facecolor="#e74c3c", label="Other"), ] ax.legend(handles=legend_elements, loc="lower right", fontsize=8)else: ax.set_title("ROI by region (top-25 by bets)")plt.tight_layout()plt.show()
ROI by region (top-25 by bet count, matched rows only)
