CECadence
Technical diligence note — model validation
A Compounding Energy product
cadence.compoundingenergy.com

The held-out record — 2025 and 2026H1, in full

Pre-registered · published unmodified
We publish the places our model is weakest out of sample — the 2025 scarcity-tail (P95) and the 2026H1 distribution shape (P50/P95) — instead of hiding them.
Two pre-registered held-out windows have now been graded and published unmodified. Here is exactly what fails, why it fails, why the failure is a data limit rather than a modelling error — and how to size a financing decision around it conservatively. A forecaster that hides its weakness is the one you cannot trust.

The numbers — error vs realised GB, by year and held-out window

YearBasisMeanP50P95Read
2022synthetic2.8%3.7%7.0%in-sample — all pass (crisis year)
2023synthetic16.8%1.2%44.5%P95 hot (scarcity knee)
2024synthetic10.4%10.0%55.8%mean/P95 hot
2025synthetic12.0%8.2%59.4%HELD-OUT — FAIL on P95
2025realised1.6%24.0%55.4%HELD-OUT — best mean of any year; FAIL on P50/P95
2026H1synthetic1.4%23.0%55.6%HELD-OUT WINDOW — first held-out mean PASS; FAIL on P50/P95
2026H1realisedNOT OBTAINABLE at generation — NESO's Historic Demand Data 2026 ended at its 2026-06-13 publication frontier: 90.3% per-series coverage, below the frozen 95% gate; the fetch failed closedHELD-OUT — the gap is published, not papered over
Adoption thresholds (pre-registered): mean ≤ 5% · P50 ≤ 10% · P95 ≤ 20%. 2022–25 grade at the frozen 26-week convention; 2026H1 grades at its pre-registered 13-weeks-over-the-half-year horizon (the committed 26-over-52 convention, halved with the window). Disclosure: the 2026H1 mean's 90% bootstrap CI is 0.2–6.0% and spans the 5% target — the verdict grades on the point value (1.4% PASS); the CI is published so the pass is not oversold. Source: docs/evidence/backtest_evidence.json, reproducible live from the in-product backtest form.

Why the P95 runs hot

2025 was a moderate-gas, low-scarcity year: realised prices had a flat tail (P95/P50 ≈ 1.5) where the frozen stylised scarcity knee expects a steeper one (≈ 2.6–3.0). The knee is too convex for a calm year, so the model over-states the top of the distribution. The error is in the tail shape, not the level — the realised-2025 mean is the best of any backtest year (1.6%). The held-out 2026H1 window repeats the pattern: a hot P95 (55.6%, the third regime running) while the window mean transfers (1.4% — the first held-out mean pass in the record) and the P50 runs cold — realised P50 sat above the window mean, a top-heavy gas-cost distribution the frozen single-σ family cannot shape.

Why it's a data limit, not a flaw

The body and the tail are coupled in the single-σ stylised curve family: you cannot flatten 2025's calm tail without regressing the 2022 energy-crisis tail it must also fit. With only one genuine scarcity year (2022) in the public record, there isn't enough tail data to fit a curve that's right for both regimes. This is a one-crisis-year data scarcity ceiling — more knobs would overfit, not fix it. 2026H1 graded exactly this diagnosis out of sample: the level machinery transfers (mean PASS), the shape does not (P50/P95 FAIL) — the fix is more crisis-regime data, not more shape degrees of freedom.

Why you can still trust the process

Pre-registered, not curve-fit

Every adoption rule was frozen in committed code before any held-out input was fetched — the git history proves the gap for both windows (2025; 2026H1 under protocol §14). True virgin hold-outs, graded once, published as-is.

Dual-solver oracle

Every dispatch is cross-checked by two independent solvers (HiGHS and CEMeridian) to a 1e-6 objective tolerance, so results aren't an artefact of one optimiser.

The level transfers

What sets central revenue holds up out of sample: the realised-2025 mean is the best of any year (1.6%) and the held-out 2026H1 window mean passes outright (1.4%). Where the P50 misses, it misses cold — the conservative direction — and the hot gap stays confined to the upside tail you should not be banking anyway.

In-sample is badged, never "passed"

Cells fit to the targets they're graded against are labelled fit (in-sample), never a pass — so you always know which numbers are out-of-sample evidence.

How to size a decision around it

Size downside on the P90 (exceedance, low) band and the 1.10× P90-DSCR covenant — both conservative by construction and unaffected by an over-stated upside P95. Treat the model's central case (mean/P50) as the planning number, the P90 as the bankable floor, and do not lean on a single high percentile for downside protection. The one place the model is optimistic is the one place a prudent financing never relies on. On the held-out 2026H1 window the P50 miss is cold — model £74.0/MWh vs realised £96.1 — an under-statement of the central case, not an over-statement; the over-statement stays confined to the P95 upside tail. The full procedure for credit teams — P50 sizing, P90 covenant test, binding-year discipline — is written up in the underwriting note.

What we're doing about it