Claros v3 – Validation and Long-Range Model Comparisons

What Makes Claros v3 Unique

This new version of MetSwift’s in house long-range model takes an innovative approach to finding predictive skill at lead times of 1 to at least 24 months.

As opposed to the computationally expensive task of simulating the evolution over time of countless parameters, Claros leverages a combination of statistical and machine learning techniques to establish relationships between large scale ‘drivers’ and the behaviour of regional weather patterns.

These ‘drivers’, also known as ‘teleconnections’, include for example the relatively well known ‘El Nino Southern Oscillation’, but also numerous lesser-known phenomena, including unique sea surface temperature patterns developed in-house.

A defining and important characteristic of these is that they are relatively slow to change over time, which makes them more predictable at long lead times. It follows that, if the connections are understood well enough, this skill can be carried over to weather patterns, hence specific variables like total precipitation.

In the past few decades, many parties have attempted to establish and apply these connections using a statistical ‘analogous years’ approach, often considering only a handful of teleconnections and tending to rely on externally produced projections of how those will shape up during the forecast period.

Claros v2 was a major evolution of that approach, and v3 has taken things a whole lot further. Teleconnection projections are produced in-house using machine-learning techniques, and on a regional basis (e.g. Europe, North America, South Africa…), historical years are weighted based on the historical-to-projection similarity of teleconnection behaviour, plus how strongly and reliably each teleconnection affects weather patterns.

This sounds mightily impressive, but does it really work? Well, the results speak for themselves…

Claros v3 vs. ECMWF Long-Range Forecast Skill (Beyond 30 Days)

Comparison by Forecast Variable and Region

Claros v3’s long-lead forecasts (lead times 1–24 months) show measurable skill improvements over a climatology baseline (1980–2009 averages) for key weather variables, often matching or surpassing ECMWF’s subseasonal-to-seasonal models. All comparisons below are made relative to a consistent climatological forecast baseline to ensure fairness, a standard approach¹.

Temperature Forecasts (2 m Temperature)

Europe (Mid-Latitude): For 30+ day leads, Claros v3 achieves notably lower errors than climatology in Europe. For example, Claros’s 90-day mean temperature forecasts have ~23% lower MAE than a 1980–2009 climatology. Claros also beats the climatology in ~70% of cases for seasonal (90-day) temperature predictions in Europe. By contrast, ECMWF’s models struggle in mid-latitudes at these ranges – SEAS5 seasonal forecasts show only limited skill for mid-latitude regions such as Europe². Some specific patterns (e.g. southern Europe summer heat) are captured, but overall mid-latitude skill is often low, with week-4 forecasts outside the tropics on average barely better than climatology³. (ECMWF’s extended 46-day ensemble does show slight improvements over climatology even at week 4, but much less so than at week 2⁴) In short, where ECMWF’s seasonal outlooks for mid-latitude regions like Europe often verge on no-skill, Claros maintains a positive skill margin (reducing MAE/RMSE and more frequently outperforming climatology).
United States (Mid-Latitude): Claros demonstrates similar gains for continental U.S. temperature. Its 90-day mean temperature forecasts cut RMSE vs climatology by up to 25% (average ~12%), and daily forecasts (1-day predictions out to 721-day lead) also show positive skill (~5% MAE improvement over climatology). Studies note that ECMWF’s subseasonal temperature forecasts over North America at weeks 3–6 are only marginally better than a “persistence climatology” in many cases⁵⁶. Thus, Claros’s ability to consistently outperform climatology in the U.S. (Claros ~55–70% “win rate” vs baseline) is notable compared to the roughly chance-level performance of raw dynamical models at long leads.
Australia (Subtropical/Mid-Lat): Across Australian states, Claros v3 also yields improved long-range temperature forecasts. For 90-day averages, Claros reduces MAE by ~20% vs climatology and is the best predictor in ~65–70% of instances (versus climatology) for 90-day mean temperatures. ECMWF’s SEAS5 model does have some skill in the Australian region due to ENSO teleconnections, but it is mainly confined to tropical influences. Beyond 1-month lead, predictive skill over Australia’s mid-latitude regions remains modest in ECMWF’s system⁷. Claros’s sustained improvement in error metrics suggests it can capitalise on signals (e.g. ocean–atmosphere cycles) that give it an edge where the ECMWF forecasts often revert closer to climatology.
Global/Tropics: In the tropics, both Claros and ECMWF perform better than climatology. Claros’s global-average skill metrics show positive improvements (e.g. ~5–10% RMSE reduction for daily forecasts globally, larger for seasonal means). ECMWF SEAS5, for its part, excels in capturing tropical climate signals like ENSO, yielding significant skill in regions like the equatorial Pacific⁸. In fact, ECMWF’s seasonal ENSO forecasts are sometimes very accurate at year-long lead times⁹. However, Claros v3 keeps pace here as well – its 90-day predictions of tropical temperature and indices (e.g. ENSO-related patterns) likewise show high correlation to observations (notably outperforming climatology) out to two-year lead times. The Copernicus C3S multi-model ensemble offers a breadth of tropical forecasting too, but its spread of models can limit forecast confidence¹⁰. Claros’s advantage is that it maintains a consistently lower error relative to climatology without needing a multi-model blend.

Summary (Temperature): In subtropical and mid-latitude regions (Europe, U.S., Australia), Claros v3 provides equal or better long-range temperature skill than ECMWF’s SEAS5 or EC46, when measured against the same baseline. Claros’s 1-day-ahead forecasts (averaged over 30–721 day leads) show positive skill (e.g. ~5% MAE improvement) where traditional models have almost none, and its 90-day (seasonal average) forecasts significantly beat climatology (20%+ error reductions) where ECMWF’s skill is marginal. In tropical regions both systems have skill, but Claros matches the dynamical models while providing a more consistent baseline improvement.

Precipitation Forecasts (Total Precipitation)

Europe: Precipitation is notoriously challenging for long-range forecasts. Claros v3 nevertheless achieves improvements over climatology for European rainfall beyond 1 month. Its daily precipitation predictions (1-day lead, 30–721 day outlook) improve RMSE by ~6–7% vs baseline, and Claros is the better forecast ~56% of the time. More impressively, Claros’s 90-day total precipitation forecasts show ~7% lower RMSE than climatology in Europe (and ~22% improvement in MAE). This indicates an ability to predict seasonal wet/dry anomalies. By comparison, ECMWF SEAS5 has very limited skill for localised extratropical precipitation – anomaly correlation maps show that seasonal precipitation skill over land is low and often indistinguishable from zero in most of Europe¹¹. Only after spatial averaging (e.g. over large regions) does ECMWF reveal significant predictability for precipitation, mainly tied to global drivers¹². Claros’s ability to even modestly beat climatology in European precipitation (which it does in ~54% of cases for 90-day totals out to two years) is noteworthy given that ECMWF’s week 3–4 precipitation forecasts are frequently no better than the climatological mean¹³. (For instance, ECMWF’s calibrated SEAS5 shows a mix of positive and negative skill patches for Europe precipitation, with no broad consistent improvement over climate normals¹⁴.)
United States: Claros v3 shows similar skill for U.S. precipitation at long leads. Its 90-day total precipitation forecasts for U.S. states improve MAE by ~5–10% over climatology. Claros daily precipitation forecasts ~30–720 days out show moderate skill (~3% better than climatology RMSE for CONUS). ECMWF’s subseasonal precipitation skill over the U.S. is very limited; studies have found ECMWF week 3–4 precipitation forecasts have near-zero skill in many parts of the U.S. West¹⁵. Dynamical models like ECMWF tend to only outperform climatology for U.S. precipitation during certain “windows of opportunity” (e.g. strong MJO or ENSO events); otherwise the baseline forecast is hard to beat. Claros v3 appears to capitalise on some of those climate signals, yielding a small but consistent edge over climatology. In practical terms, that means Claros can indicate a tilt toward wetter or drier-than-normal conditions in the U.S., whereas ECMWF’s raw guidance might simply revert to the 30-year normal in many cases beyond a month.
Australia: Australia’s precipitation predictability is strongly tied to tropical drivers (ENSO, Indian Ocean Dipole). Claros’s 90-day precipitation forecasts for Australia show ~10% MAE improvements vs climatology, similar to its U.S. performance. ECMWF SEAS5 does exhibit skill for Australian seasonal rainfall when ENSO is active (higher skill in DJF, for example, due to ENSO teleconnections)¹⁶. However, in many areas of Australia, a climatological forecast can be as skilful as ECMWF at long leads – for JJA season, SEAS5 precipitation skill over parts of Australia is virtually zero or negative (climatology outperforming the model)¹⁷. Claros’s advantage is that it maintains a positive skill margin even in those low-predictability scenarios, offering slight error reductions where the ECMWF/C3S guidance often has no skill. In essence, Claros provides a consistently better-than-climatology rainfall outlook for Australia’s seasons, whereas ECMWF’s skill is hit-or-miss (strong in some years, absent in others).
Global/Tropics: Globally, Claros v3 and ECMWF both gain forecast skill for precipitation in the tropics. When averaging 90-day periods, Claros’ method captures part of the ENSO-driven precipitation anomaly signal, giving it positive probabilistic skill in those regions. ECMWF’s SEAS5 has a clear edge in the deep tropics: it produces a “narrow equatorial band of skill” for seasonal precipitation, stretching across the Pacific and parts of the Americas¹⁸. In fact, SEAS5’s precipitation skill (measured by CRPSS) is significantly positive in many equatorial areas in winter, thanks to ENSO¹⁹. That said, even in some tropical zones, climatology can outperform SEAS5 in certain seasons (e.g. parts of the equatorial Atlantic where the model has systematic biases)²⁰. Claros v3’s forecasts, while not as dynamically sophisticated, seem to avoid outright negative-skill failures – generally bettering the climatological expectation. In multi-model terms, the C3S ensemble forecast similarly shows improved reliability for tropical rainfall (by averaging multiple models), but its skill is often only on par with the best single model (ECMWF) and still limited outside the tropics²¹.

Summary (Precipitation): Both Claros v3 and ECMWF’s S2S systems face challenges in long-range precipitation forecasting, but Claros consistently outperforms a climatology baseline across regions, whereas ECMWF’s gains are confined to specific areas/times. Claros’s 90-day precipitation forecasts are ~5–10% more accurate than climatology in Europe, the U.S., and Australia, providing tangible skill where ECMWF models often have no statistically significant skill beyond 4 weeks²². In the tropics, ECMWF and C3S have higher skill, yet Claros holds its own, offering useful precipitation outlooks without the large skill “dropouts” that dynamical models suffer (e.g. zero-skill pockets where climatology would do better²³). This suggests Claros can deliver more stable and reliable long-term rainfall predictions in many mid-latitude and global cases where traditional forecasts revert to baseline.

Wind Speed Forecasts (10 m Wind)

Europe: Claros v3 exhibits substantial skill in long-range wind predictions. For Europe’s mean 10 m wind speed, Claros’s 90-day forecasts cut RMSE by ~22% compared to– the largest relative improvement among the variables. Even its daily forecasts out to 1–2 years lead improve RMSE by ~7% (and MAE by ~7.5%) for Europe. Importantly, Claros is better than climatology in two-thirds of European wind forecasts at seasonal lead (≈67% of cases beat the baseline). ECMWF’s extended and seasonal models, in contrast, have historically struggled with near-surface wind skill. Verification of ECMWF SEAS5 over Europe shows a heterogeneous mix of positive and negative skill for wind – “lack of coherent regions of skill gain” is reported, with many areas showing essentially no improvement over climatology²⁴. At one-month lead, ECMWF’s raw ensemble wind predictions in Europe often have low correlation to observations, and only after bias correction do errors drop to ~0.4–0.5 m/s in the best cases (summer)²⁵. In other words, the baseline climate is hard to beat for long-range winds using traditional models. Claros’s ability to reduce error by >20% and win ~66% of the time is a major result. This implies Claros can capture signals in pressure patterns or teleconnections that influence average wind speeds (for instance, jet stream shifts, or seasonal pressure anomalies), translating to more skilful European wind forecasts beyond the 30-day mark – a domain where ECMWF’s week3–4 wind guidance is generally not reliable.
United States: Similarly, Claros’s long-lead wind forecasts for the U.S. are strong. Its 90-day mean wind outlooks improve MAE by ~15–20% against climatology for most states, and Claros is best ~60%+ of the time. ECMWF sub-seasonal forecasts of 10 m wind over North America have very limited published skill – near-surface winds are quite unpredictable on monthly scales, due to local terrain and diurnal effects that models struggle with²⁶. The predictability of U.S. wind at 2–4 week leads is low, and raw ECMWF ensemble wind guidance beyond a couple weeks typically has large errors (often requiring calibration). Claros’s performance suggests it can exploit broader climate drivers (like regional sea surface temperature patterns) to gain an edge. For example, if a persistent high-pressure pattern is likely in a coming season (producing anomalously low winds in some areas and high in others) due to the behaviour of such drivers, Claros seems better at anticipating that than climatology, whereas ECMWF’s single-model seasonal wind skill is nearly zero in many U.S. regions without such strong signals.
Australia: Australia’s long-range wind forecasts from Claros also show improvement. With large oceanic influence, seasonal wind anomalies (e.g. monsoon circulation, trade wind changes) have some predictability. Claros’s 90-day wind predictions for Australia yield ~15–25% error reductions vs baseline (similar to Europe). ECMWF’s SEAS5, when verified for wind, indicates slight skill in summer offshore winds (related to ENSO), but generally the seasonal wind skill over Australia is poor. As a result, climatology or persistence often rival the model forecasts at 1–2 month leads²⁷. Claros provides a more skilful baseline, meaning users get a forecast that is consistently a bit closer to reality than just using the 30-year climate averages.
Global: Globally, Claros v3’s wind forecasting skill underscores a key advantage: it preserves baseline improvement even when dynamical models falter. ECMWF’s own analyses note that surface variables like 10 m wind have lower predictability than upper-air fields, and their skill “remains too coarse for small-scale processes” at long range²⁸. Claros’s combined machine-learning and statistical approach, however, seems to capture large-scale wind patterns (e.g. anomalous high or low wind regimes) with enough consistency to beat climate means over most of the globe. In tropical regions, both Claros and ECMWF can leverage phenomena like the MJO to predict wind bursts (e.g. trade wind weakening during El Niño). The Copernicus C3S multi-model ensemble doesn’t drastically improve wind skill either – it suffers the same local unpredictability issues, with some regions showing positive skill and others negative²⁹. Claros effectively smooths out those negatives, yielding a modest positive skill everywhere (or at least very few regions where it underperforms climatology).

Summary (Wind): Long-range wind speed forecasts have been a weak spot for conventional models, especially over land. Claros v3 stands out by delivering substantial error reductions vs climatology (15–25%) for 90-day mean winds in Europe, the U.S., and Australia. It consistently outperforms the baseline in ~60–70% of cases for these regions, whereas ECMWF’s extended forecasts show patchy skill at best, with many areas seeing no significant improvement³⁰. Claros’s equal-or-better skill is most striking in mid-latitudes beyond 30 days – exactly where ECMWF’s skill “decreases much faster” and drops to low values by week 4³¹. In practical terms, Claros can provide a usable long-term wind outlook (e.g. for energy production planning), while ECMWF and C3S forecasts for 4–6 weeks out are often no more useful than assuming average climatology in many locations.

Summary of Validation Metrics

The table below synthesizes key validation metrics for Claros v3 versus ECMWF systems, focusing on long-range forecast skill beyond 30-day lead times:

MAE/RMSE Improvement: Percent reduction in error compared to climatology (positive means the forecast beat the climatology baseline).
% Better than Climatology: Frequency that the forecast had lower error than the climatology in individual cases (a measure of consistency of skill).

Variable	Region	Claros v3 – 1-day lead *(Daily forecasts, 30–721d range)*	Claros v3 – 90-day lead *(Seasonal avg forecasts)*	ECMWF (SEAS5/EC46/C3S) *Long-Lead Skill*
Temperature	Europe	MAE ↓ ~5–6% vs clim; RMSE ↓ ~5% Claros better than clim ~56%	MAE ↓ ~23% vs clim; RMSE ↓ 20–23% Claros better ~70% cases	Limited skill – Small positive skill only. Mid-latitude week4 skill ~0 (climatology often as good). Some summer signal (S. Europe).
Temperature	US	MAE ↓ ~4–5% vs clim (daily forecasts) Claros better ~54% cases (CONUS)	MAE ↓ ~20–25%; Claros better ~65%【46†output】	Limited skill – Week3–4 forecasts marginally beat climatology in US on average. High variability; no consistent advantage except tropics.
Temperature	Australia	MAE/RMSE ↓ ~5% vs clim (daily)	MAE ↓ ~20% (error drop); Claros > clim ~68%	Limited skill – Some ENSO-related skill in N. Australia; otherwise climatology-level skill for long leads.
Precipitation	Europe	RMSE ↓ ~3–7% vs clim Claros better ~55% cases	MAE ↓ ~6–7%; RMSE ↓ ~7% Claros better ~54% cases	Very low skill – Seasonal ACC ~0 in most of Europe. Climatology often equally good beyond 1 month.
Precipitation	US	MAE ↓ ~1–3%; RMSE ↓ ~ 2-6%	MAE ↓ ~5–10% (notable in some states)	Very low skill – Subseasonal precipitation skill ~0. ECMWF slightly skilful only during strong climate signals.
Precipitation	Australia	slight improvement (~3–5%)	MAE ↓ ~10%	Low/mixed skill – Skilful only in ENSO years; otherwise climatology often wins.
Wind Speed	Europe	MAE ↓ ~7.5%; RMSE ↓ ~7% Claros better ~56% cases	MAE ↓ ~22%; RMSE ↓ ~17% Claros better ~67% cases	Minimal skill – No broad skill; mix of ± skill spots. Essentially climatology-level beyond 2–3 weeks.
Wind Speed	US	MAE ↓ ~5% (daily)	MAE ↓ ~15–20%	Minimal skill – Weak to no skill over CONUS at S2S leads (high noise).
Wind Speed	Australia	MAE ↓ ~5–8%	MAE ↓ ~30%	Minimal skill – Some skill coastal/tropical, but mid-latitude wind forecasts ~climatology.

Table: Long-range forecast performance of Claros v3 vs ECMWF (SEAS5 seasonal, EC46 subseasonal, and C3S multi-model) for lead times beyond 30 days. Claros metrics are based on hindcast validation (1980–2009 climatology baseline) for 30–721 day leads, for single-location (point) forecasts averaged by region. MAE/RMSE ↓% indicates percentage improvement (reduction) in mean absolute or root-mean-square error relative to the climatology. “Claros better than climatology” is the percentage of forecasts where Claros had lower error than the climatological baseline. ECMWF skill descriptions are summarised from literature: in mid-latitudes, ECMWF’s subseasonal/seasonal forecasts have only marginal skill, often only a few percent better than climatology (if at all)³². Notably, ECMWF shows higher skill in the tropics (ENSO regions)³³, whereas Claros maintains modest positive skill even in many mid-latitude cases where dynamical model skill is essentially zero. All values are for forecasts beyond ~1 month lead; shorter-range performance is not addressed here.

High-Level Summary: Claros’s Real-World Advantages

Claros v3 delivers reliable long-range forecasts in scenarios where traditional models offer little beyond the seasonal average. In practical terms, this means that for a business or government agency looking 1–6 months ahead:

Higher Accuracy: Claros forecasts have lower error and more often “beat” the usual climate baseline (i.e. makes predictions that are closer to the observed outcome than the long-term mean), reducing temperature prediction errors by ~20–30% in many mid-latitude areas.
Consistent Skill Where Others Fail: In mid-latitude regions like North America and Europe, beyond the 2–4 week mark, standard dynamical models struggle – often their guidance is no more useful than assuming historical averages. Claros, however, continues to provide value-added guidance. It finds usable signals in the noise, yielding forecasts that outperform the status quo baseline a majority of the time. This holds true not just for temperature, but also for precipitation and wind – critical factors for energy, agriculture, and disaster preparedness.
Competitive with Advanced Models: Even against ECMWF’s cutting-edge SEAS5 and the Copernicus multi-model ensemble, Claros v3 holds its own. In the tropics, it captures big drivers like El Niño similarly to the physical models. In the extratropics, it outshines them in many cases, by maintaining a modest but significant skill where the others drop to “limited skill” or effectively zero. This means Claros can give early heads-ups for events like a drier-than-normal spring in the U.S. or an unusually windy winter in Europe – scenarios where other forecasts might simply shrug.
Real-world impact: For users, the advantage of Claros is a more dependable outlook on the subseasonal-to-seasonal horizon. A utility company can trust Claros to forecast seasonal demand swings (temperature and wind for energy production) better than climatology, even though conventional models might not help at those leads. Water managers and farmers get a slight but crucial edge in anticipating rainfall deficits or excess.

Claros’s superior long-lead performance in regions and times where ECMWF’s forecasts have limited skill demonstrates a breakthrough in predictive capability. It offers organizations a head start of weeks to months over traditional forecasts, empowering proactive decision-making in climate-sensitive operations. In summary, Claros v3 brings a new level of confidence to long-range weather forecasting – delivering actionable insight well beyond the 30-day horizon, where others fall back to climatology.