Probabilistic price forecasts for short-term trade optimization

Despite significant advancements in data science, forecasts remain inherently imperfect. As a society, we navigate this uncertainty in different ways – making contingency plans, implementing risk management protocols, or fostering resilience and adaptability.

In the field of short-term renewable energy trading, the reliance on forecasting is two-fold. An optimal strategy requires a power generation forecast to minimize imbalance volume, thus reducing the risk of open positions on the balancing market. Secondly, an optimization strategy that considers price forecasts is indispensable to maximize revenues. How to embrace uncertainty in this context?

Trade strategies that incorporate probabilistic price forecasts offer a comprehensive view of uncertainty. In this article, we review the advantages and qualities of probabilistic forecasts, then share ways to apply conformal prediction for well-calibrated forecasts.

Deterministic versus probabilistic forecasting

Our starting point is the common and familiar approach of point estimates. Using a time series of actual prices and various other data sources such as weather forecasts and market data, this technique predicts the most likely future prices expressed as a single numerical value per time unit. Needless to say, the chances that the actual prices will precisely match these predicted prices are extremely low.

Point forecasting is deterministic, with all the outcome likelihood probability mass centered on one value. As such, it does not capture the full range of possible outcomes.

In contrast, a probabilistic forecast provides a range of possible values and the likelihood of each occurring. This can be achieved by employing quantiles, i.e., statistical measures that divide the range of a probability distribution into intervals with specified probabilities. For example, the 25th quantile indicates the value below which 25% of the observations may be found.

The image below places the deterministic and probabilistic approaches side-by-side. Which one offers a more comprehensive view of uncertainty?

The answer is, of course, probabilistic forecasting. Incorporating full estimated price distribution in optimized trade recommendations ensures decision-makers that trade-offs have been considered to provide strategies in line with their risk appetite.

Thus, we must ensure that the forecasts we use to send optimized trade strategies are reliable, i.e., making valid predictions. Let’s review what that entails.

What makes a good probabilistic forecast

There are two main concepts to consider when assessing the quality of a probabilistic forecast: calibration and sharpness.

Calibration measures how well the predicted probabilities align with the actual outcomes. Looking back at the figure above, calibration would be equivalent to checking if, for example, 20% of the observations in the test set are below the 20th quantile.

To understand the concept, we can plot the forecasted probabilities against the observed outcomes. On the x-axis, we place the theoretical quantiles representing the predicted probabilities (e.g., 20%, 30%, etc.). On the y-axis, we have the sampled quantiles. These indicate the number of observations that lie below the theoretical quantile when making predictions on a validation data set.

A perfectly calibrated model would produce a plot that aligns along the diagonal line, meaning that the forecasted probabilities match the observed outcomes. Should the model not be well-calibrated, the plot will show a coverage error.

Yet even if a model is perfectly calibrated, it may still not provide rich predictive information. A second dimension is, thus, relevant: sharpness. It measures how narrow or wide the distribution of predicted probabilities is, reflecting, essentially, how confident we are in our predictions. For instance, a price prediction within the 25 to 35 euros range suggests higher confidence than a model with a 20 to 40 euros range.

The two plots are drawn out below for reference:

A helpful metric, the Continuous Ranked Probability Score (CRPS), measures both calibration and sharpness. We’ll cover this metric in detail in another article. For now, let’s focus on creating a probabilistic forecast.

How to obtain a probabilistic forecast

Building a probabilistic price forecast can be viewed as a time series regression problem — predicting a target variable based on time-related, independent variables. Machine learning is well-equipped to solve such problems. As with every AI model, the foundation is high-quality data.

For the short-term energy market, we input data from three sources into a machine learning model:

• Market and grid data;
• Weather data;
• Data related to the impact of our trading actions.

(Note that the latter, market impact, is one of the most overlooked aspects of price forecasting. Our CPO Tom Lemmens explains its mechanics in this white paper on backtesting.)

The process of obtaining probabilistic models consists of training and testing, just like point prediction models. The difference lies in how the model processes the information to deliver probabilistic instead of deterministic results. Different probabilistic models ‘learn’ the distribution of the forecast in various ways. In general, the forecasted distribution is based on the distribution of the samples in the training data set.

Working with probabilistic models, we investigated methods such as conditional uncertainty estimates through quantile loss, quantile forest, or quantile binning. However, many approaches fall short in ensuring a calibrated distribution, often due to overfitting or noise in the data. This may lead to inaccurately predicted confidence measures and, consequently, flawed trade decisions.

Luckily, a relatively new framework solves these problems in an effective and elegant way: conformal prediction.

The basics of conformal prediction

Conformal prediction (CP) is a machine learning framework that quantifies uncertainty by creating statistically rigorous prediction intervals.

The concept has gained popularity in recent years thanks to several advantages. First, it guarantees that prediction intervals align with the data distribution. Second, it is compatible with any point or probabilistic forecast. Third, it is a flexible method, not requiring predefined assumptions about the distribution. Last but not least, it is straightforward to implement.

To use conformal prediction, you’ll have to take three steps:

1. Training. Begin with a baseline model, typically a point forecasting model

2. Calibration. Here, take these two sub-steps:

2a. Select a subset of the data, referred to as the calibration dataset;

2b. Calculate the residuals by determining the differences between the actuals and the predicted values from the baseline model.

3. Testing. Finally, overlay the distribution of residuals onto the point forecast, with the forecast serving as the mean.

The image below can act as a cheatsheet of these three stages:

This approach is fairly simple, with two positive properties and one notable limitation. On the bright side, it is “distribution-free” as it doesn’t assume a specific shape or form for the underlying data distribution. It further guarantees that the quantiles are correct.

Applying conformal prediction to price forecasting

The two most common ways to apply CP to price forecasting are prediction interval and calibration. This article provides a concise overview of these methods. The detailed steps, computations, and corresponding Python code are in this article and this repository.

Add a prediction interval to a point forecast

As mentioned, CP is model-agnostic; thus, it is possible to add a prediction interval to any point forecast. Visually, this method presents itself as follows:

To achieve this, set aside a calibration set for computation, train your point forecast model on the training set, and calculate conformity scores (residuals) by predicting the calibration set. Then, create a sorted list of absolute conformity scores.

The width of the prediction interval is determined by selecting the quantile from this list that matches your desired confidence level. When predicting the test set using your trained model, incorporate this calculated interval width by adding and subtracting it from the point prediction.

Note that, with this method, the probability for each possible future price will be the same for all new predictions, with only the mean shifting. On the other hand, real-world scenarios are highly dynamic; a short-term trader in intermittent renewables will experience different uncertainty profiles on a sunny versus cloudy day. In statistical terms, this is called heteroscedasticity. The following approach allows you to overcome this challenge.

Calibrate a probabilistic forecast

An alternative way to leverage CP is to calibrate a probabilistic forecast, a method also known as conformalized quantile regression. This approach has two advantages that make it worth considering: it results in a full distribution, and the predictions are specific to the prediction sample (in other words, the predicted distribution adapts to the local variability of this input space).

Imagine the results in this form:

As in the previous section, the process starts with a calibration set, followed by training the quantile regression model. The next steps are more complex: computing the correction for calibration and incorporating this correction when iterating over all predicted intervals.

Going back to the sharpness of a probabilistic forecast, a positive correction indicates that the initial prediction was too confident or narrow. Conversely, if the correction is negative, it is too uncertain or wide. Accordingly, the calibration will result in a wider, respectively narrower distribution.

From theory to practice

Machine learning models, often called “black boxes” due to their opaque internal processes, pose particular challenges when applied to energy price forecasting. These markets are dynamic, fast-paced, and constantly influenced by geopolitical events, weather fluctuations, and regulatory changes, among other factors.

Therefore, at Dexter Energy, we strongly and constantly focus on quality checks and improvements to ensure that the predictions generated by our machine learning models are reliable. The methods presented in the sections above reflect some of the extensive research performed by our data scientists. In practice, we often combine methods to find patterns in our data and evaluate the efficacy of different approaches under varying market scenarios.

What we consider essential when applying scientific methods to price forecasting is to avoid assumptions when possible or be explicit when making them. Extreme market movements can significantly impact the short-term trading cycle; in this context, some errors will be more costly than others. This awareness drives our commitment to optimizing trade results beyond a good forecast.

Going further

In this article, we dove into price forecasting for short-term trade optimization in the field of renewable power, exploring how to quantify and embrace the inherent uncertainty. The main takeaways are:

1. Probabilistic price forecasting offers a more comprehensive view of uncertainty than point estimates;
2. Calibration and sharpness are indicators of a good probabilistic forecast;
3. Conformal prediction is a promising framework for quantifying uncertainty.

For a deeper understanding of the topics discussed here, we recommend watching the recordings of our recent PyData presentations by clicking below:

• “Harnessing uncertainty: The role of probabilistic time series forecasting in the renewable energy transition” with Alexander Backus;
• “Leveraging conformal prediction for calibrated probabilistic time series forecasts” with Inge van den Ende.