FAQ: Backtest and Calibration

How are you backcasting results?
Why is the calibration factor static over the forecast horizon?

How are you backcasting results?

Covering Jan 2021 - March 2024, this page explores how well our fundamentals model predicts hourly prices, as compared to the Nordpool Day ahead hourly price.

We also explore how well our full model predicts battery revenues in our Model Calibration pages, where the impact of modeling error, site availability, perfect hindsight, and all sorts of other effects are quantified and you can see how well the Modo forecast does against the Modo benchmark.

Why is the calibration factor static over the forecast horizon?

There are a number of factors that could change calibration over the next 25 years. However, a lot of these are unknown and difficult to reason from first principles. Thus, we use a data-driven method for calibration, trained on recent data and updated annually (updated in v3.4).

This calibration factor captures the difference in revenues we observe in real-world data vs the revenues that come out of our modelling. Reasons for a difference include:

Inability to perfectly forecast prices
The success rate of bidding into frequency response auctions
Not all assets able to achieve 'the most optimal' dispatch
That volume is not guaranteed to be taken in the Balancing Mechanism
Real-world availability of battery fleet: eg from downtime of assets, networks, and curtailment impacts

It is important to note other factors are considered in other parts of the forecast, which do change over time:

Ancillary service saturation, and subsequent ability of a single site to win contracts. As the size of the battery fleet grows, the amount of frequency response any single asset can do reduces
Intraday trading revenues are modelled from fundamentals, which means the 'uplift' on Day-Ahead wholesale revenues changes over time with the changing generation stack
The transmission network buildout is modelled within the BM logic, with differing volumes of energy and system actions in different locations over time