Texas wind power: how much room does the official forecast leave?

01First, prove the time axis is right

Three data sets: ERCOT's public generation-by-fuel reports (hourly wind generation, 2018–2024), ERA5 reanalysis winds, and the complete 2023–2024 archive of the official short-term wind-power forecast: 17,736 issues, one per hour, each covering the next 168 hours.

The worst failure in forecast evaluation is a time shift: the numbers look normal, nothing errors, and the conclusion is wrong. Alignment was checked twice, independently:

• Against EIA-930 data: r = 0.99998 for 2024, MAE 28.5 MW (0.22% of the mean).
• Against the actuals inside the official forecast files: after removing 27 bad hours (about 13 times normal), r = 0.99997, and the lagged correlation of first differences peaks at lag 0 (0.999, about 0.70 at ±1 h).
• My first acceptance test, “the correlation must peak at lag 0”, was wrong: hourly generation has autocorrelation near 0.98, so lag 0 and lag 1 differ by 0.004, all noise. First differences give a sharp peak.
• On the autumn daylight-saving day the reports contain a repeated hour. Mis-ordering it leaves plausible numbers and no error; only the checks above catch it.

02The official forecast: about 6% error one day ahead

Error is 3.1% at lead 0, about 6.2% one day ahead, 8.4% at three days and 16.3% at seven. The day-ahead market uses the issue published at 09:55 the day before (15–39 h ahead), with a 2024 MAE of 6.24%.

The dashed line is my best model on ERA5 winds, 6.38%. ERA5 is a reanalysis that has already absorbed observations, equivalent to a perfect weather forecast, so it is a ceiling, not a forecast, and not a fair comparison. Operational use can only do worse, and the official forecast already matches it.

03The high bias is curtailment

Against actual generation the official forecast runs about 2.9% of capacity high, and generation falls below the official median in 64% of hours (nominally 50%). That looks like systematic over-forecasting. But the official target is what the farms could produce, available power; actual generation also subtracts curtailment. From 26 September 2024 the files include available-power actuals, so the two can be separated:

• At lead 0 the error is 0.47% against available power and 3.53% against actual generation. The difference is curtailment.
• Against available power the bias is −0.1% to −0.7%; available power falls below the official median 46–49% of the time (nominal 50%), and below the 20th percentile 16–19% of the time at 2–72 h (nominal 20%). The official probabilistic forecast is well calibrated for its own target.
• Beyond 72 h weather error dominates and the curves converge; past 85 h the available-power error is slightly larger.

So the weather-to-available-power step is already done well; the room left is available power to actual generation, which depends on load, prices and transmission limits, exactly what trading desks care about.

Available-power actuals cover only the last three months of 2024; the first three quarters cannot be checked the same way.

04More seasons beat a better model

One task: estimate system-wide wind generation at the same hour from ERA5 winds. Trained on 2023, tested on all of 2024 (8,784 hours), three seeds per ML model.

The three ML models differ by under 0.2 points, the same size as seed-to-seed noise, and all beat a binned power curve (9.44%) by about a third. What separates results is the seasons in the training data:

Without weather, forecasting 24 h ahead from generation history alone, the best model (a Transformer, 18.36%) is only 12% better than climatology (20.81%). Almost all wind-power skill comes from the weather.

05Real operational forecasts: HRES, Pangu-Weather, GraphCast

ERA5 is a ceiling, not a forecast; a fair comparison needs forecasts that existed at the time. WeatherBench 2 provides archived ECMWF HRES, Pangu-Weather and GraphCast operational forecasts. I took 2022 00 UTC runs at 30–54 h (the day-ahead window), converted each source's winds to power with the same model (XGBoost, trained on ERA5 2019–2020, validated on 2021), and compared with the official day-ahead forecast at the same times: 355 initialization days, 1,774 samples, with 95% intervals of the difference from a 2,000-draw block bootstrap over initialization days.

• All three are significantly worse than official, by 1.8–2.2 points; no interval crosses zero.
• Even the ERA5 ceiling trails official by 0.72: WeatherBench 2 has no 100 m wind, so hub-height wind must be estimated from 10 m and 850 hPa, and the mapping loses some skill (with 100 m wind in 2024 the ceiling was 6.38%). HRES trails ERA5 by a further 1.11, which is the weather-forecast error.
• AI models vs HRES: in raw error GraphCast is 0.39 worse (interval 0.03–0.75, just significant) and Pangu-Weather 0.30 worse (not significant); after removing each source's annual mean bias neither differs significantly from HRES. The extra error comes mainly from 10 m wind speeds about 0.22 m/s too weak over West Texas, not from worse synoptic forecasts.

Forecast fields are 6-hourly, so only four times a day are compared; GraphCast's last WeatherBench 2 run is 2022-12-21, so late December is excluded. The official target is available power while truth is actual generation, a handicap for the official forecast that does not change the direction of the result.

06Placed in China: penalties under the Northwest grid rules

Wind and solar farms in China are penalized for forecast error under regional grid rules. Using the Northwest rules issued in December 2023 (articles 15, 40 and 49), I applied ERCOT's 2024 official forecasts and generation to a hypothetical 100 MW wind farm:

“Error ×k” scales the forecast error. ERCOT is a fleet of about 40 GW whose errors largely average out, so ×1 is an excellent forecast; a single farm is closer to k > 1. At twice the error, day-ahead accuracy is 85.3%, the annual penalty about ¥261k, and the worst month about ¥36k.

• k is not measured for a real farm; it is placed using the national accuracy metric (1 − RMSE / capacity) against the Northwest short-term threshold.
• Texas winds have a different diurnal cycle; time windows use local clock hours as-is.
• Data are hourly and the rules use 15-minute points; each hour counts as four identical points.
• The rules do not state how often ultra-short-term penalties settle; once a day is assumed. Per submission would multiply that part.
• Curtailment flags are missing for most of the year, so actual generation is used. Recomputing with available power after 26 September gives a slightly higher short-term penalty, so the estimate is not inflated.

07Not done yet

• Restore 100 m wind and train a MOS correction on HRES 2016–2021 reforecasts to see how much an operational forecast can recover.
• Curtailment forecasting is a direction, not yet started.
• Dedicated probabilistic (quantile, CRPS) and ramp-event evaluation is planned.

The analysis code is not public yet and is available on request.