# Foretec Energy Bench

> An open, live benchmark for day-ahead electricity forecasting. Every day at 11:30 (Europe/Brussels), before the 12:00 day-ahead auction gate, every model forecasts each quarter-hour of the next delivery day for price, wind, solar and load in BE, NL, FR, DE-LU, DK1, DK2, from one frozen set of public inputs, and is scored against the grid operators' actuals.

Updated 2026-10-28T22:12Z. Data licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Code: https://github.com/ujohn33/foretec-energy-bench

## Rules

- Gate-aligned by construction: inputs are fetched at the 11:30 cut-off and frozen; the weather is the numerical weather prediction run actually available then (ICON-EU, GFS, ECMWF IFS), plus the ENTSO-E load forecast and fuel prices.
- The benchmark runs every model itself, so every model sees exactly the same inputs; entries are code, not forecasts.
- The grid operators' own forecasts are included: tso_cutoff (published before the gate, ranked) and tso_final (published after the gate, a reference that is not ranked).
- Backtest days are scored the same way but kept apart from live days.

## Metrics

- Rel. MAE: a model's absolute errors pooled per series (zone and target) and divided by those of naive_weekly (the same quarter-hour one week earlier) on the same days. Below 1 beats it; 0.5 halves its error. Rel. RMSE likewise with squared errors.
- Across series, Rel. MAE is combined by the mean (default), median or geometric mean. The mean is moved by one very good or very bad series, the median rewards winning most series, the geometric mean treats halving and doubling the error alike.
- Elo: one race per issue day and series; every pair of models in a race is a game won by the lower error. A Bradley-Terry fit on all games gives the rating on the Elo scale (1000 = the field's average, +400 = 10:1 odds); a bootstrap over issue days gives a 95% interval.
- Also reported: MAE, RMSE, bias, pinball loss, weighted interval score, interval coverage, and the revenue value of forecasts.

## Live standings (27 issue days, 2026-10-02 to 2026-10-28, all series)

| # | Model | Elo | Rel. MAE (mean) | Rel. MAE (median) | Series | Races |
|---|---|---|---|---|---|---|
| 1 | ens_hybrid | 1261 | 0.344 | 0.306 | 24 | 256 |
| 2 | ens_fm_cov | 1238 | 0.351 | 0.307 | 24 | 256 |
| 3 | t0_gamma_cov | 1209 | 0.363 | 0.328 | 24 | 256 |
| 4 | timestral_cov | 1193 | 0.371 | 0.335 | 24 | 256 |
| 5 | chronos2_cov | 1191 | 0.393 | 0.339 | 24 | 393 |
| 6 | timesfm3_cov | 1190 | 0.382 | 0.318 | 24 | 393 |
| 7 | tso_cutoff | 1154 | 0.955 | 0.471 | 10 | 154 |
| 8 | t0_beta_cov | 1139 | 0.435 | 0.360 | 24 | 379 |
| 9 | tabby_cov | 1130 | 0.406 | 0.374 | 24 | 256 |
| 10 | tabby_pretrain_cov | 1129 | 0.404 | 0.398 | 24 | 256 |
| 11 | lgbm_prod | 1117 | 0.419 | 0.373 | 24 | 256 |
| 12 | tirex2_cov | 1113 | 0.423 | 0.368 | 24 | 354 |
| 13 | t0_alpha_cov | 1084 | 0.454 | 0.431 | 24 | 379 |
| 14 | lgbm_cov | 1069 | 0.526 | 0.373 | 24 | 393 |
| 15 | timesfm25_cov | 1061 | 0.526 | 0.407 | 24 | 393 |
| 16 | lin_cov_2y | 1010 | 0.636 | 0.432 | 24 | 292 |
| 17 | toto2_1b | 980 | 0.803 | 0.675 | 22 | 56 |
| 18 | ens_fm | 980 | 0.623 | 0.575 | 24 | 256 |
| 19 | timestral | 977 | 0.622 | 0.588 | 24 | 256 |
| 20 | t0_gamma | 960 | 0.643 | 0.593 | 24 | 256 |

Lowest live Rel. MAE per target (models forecasting every zone):

- Day-ahead price: ens_hybrid (0.298), ens_fm_cov (0.298), timesfm3_cov (0.300)
- Wind: lgbm_cov (0.220), lgbm_prod (0.227), tirex2_cov (0.233)
- Solar: ens_hybrid (0.296), ens_fm_cov (0.302), timestral_cov (0.307)
- Load: t0_gamma_cov (0.537), timestral (0.542), ens_hybrid (0.542)

Backtest leaders (Elo): ens_hybrid (1275), ens_fm_cov (1253), timestral_cov (1219), timeh2o_cov (1217), t0_gamma_cov (1208)

## Pages

- [Leaderboard](https://energybench.foretec.co/): standings by Elo, Rel. MAE and other metrics, filterable by zone, target and period
- [Forecast Explorer](https://energybench.foretec.co/explorer.html): every model's forecast against the actual, per issue day and series, with the frozen inputs
- [Forecast Value](https://energybench.foretec.co/value.html): revenue of a virtual 100 MW wind or solar asset selling each forecast day-ahead
- [Methodology](https://energybench.foretec.co/methodology.html): task, data sources, scoring and ranking in full

## Data

- [Standings (CSV)](https://energybench.foretec.co/leaderboard.csv): live and backtest, one row per model
- [All scores (JSON)](https://energybench.foretec.co/data/summary.json): every score row (issue day, model, zone, target, metrics), models, config
- Per issue day: https://energybench.foretec.co/data/days/YYYY-MM-DD.json (actuals and every forecast)
- [Full dataset on Hugging Face](https://huggingface.co/datasets/ujohn33/foretec-energy-bench): every forecast, actual and score as parquet; DOI 10.57967/hf/11052