- Jump to section
- From data to forecast: The three pillars
- Solar power forecasting: What’s next?
- More than AI
It’s only a matter of time before the first rays of sunshine reach solar panels across Europe, marking the return of solar power to the spotlight. Following last year’s price cannibalization and decline in capture rates, short-term power traders might want to reassess strategies for solar portfolios, starting with one of its fundamental components: production forecasts.
The most advanced solar power forecasts today rely on artificial intelligence (AI) – specifically, machine learning – and for good reason. In the energy transition, AI is indispensable for predicting the production – and prices – of intermittent renewables like solar and wind. Yet, despite its growing importance, AI often gets caught up in hype, leading to misunderstandings about its real-world applications.
This article explores a tangible use case: AI-driven solar forecasting to improve trading decisions in volatile power markets. We’ll take a high-level look at how we build and deliver accurate solar power forecasts today and discuss where solar forecasting is headed.
From data to forecast: The three pillars
AI models may steal the spotlight, but they wouldn’t exist without high-quality data, and they wouldn’t deliver value without robust pipelines, seamless access, and scalability.
At Dexter, we combine these elements into a streamlined three-step process that transforms raw data into accurate solar power forecasts:
Years of training data
For an AI model to predict the future, it needs a solid understanding of the past. The quantity and quality of training data are fundamental to building a well-performing model.
To forecast the power production of a solar park, we typically use several years of the following types of data:
- Asset properties: the number of solar panels, the direction they are facing, and their orientation. These data points define the park’s baseline production potential.
- Actuals: the historical production values recorded for each solar panel at 15-minute intervals. This data will ‘teach’ the model how the solar park behaves under different conditions.
- Historical metadata: information about the park’s availability. Solar panels may experience downtime due to maintenance, upgrades, or unexpected incidents like copper theft, which have been known to happen.
- Weather data: historical and expected irradiance in the park’s location. We collect millions of data points from all relevant weather models and process ~1TB of weather data each day. What’s more, we continuously benchmark models to identify ‘edges’ – specific scenarios and moments where certain inputs deliver the best performance.
The model pipeline
Once we’ve collected high-quality data, the next step is transforming it into accurate solar power forecasts. This happens within our model pipeline, a structured system where we prepare, process, and refine the data. The actions we take at this stage reflect our day-to-day efforts as researchers to constantly improve data quality and model performance.
For starters, we work on data filtering. Solar assets are sometimes scheduled to produce less than possible from a meteorological (i.e., solar irradiance) or operational (i.e., availability) perspective. Examples include outages or down-scheduling due to congestion. The model should not predict this exceptional behavior but instead focus on the asset’s normal production. Hence, one key activity in our model pipeline is to identify the data that the model shouldn’t learn from and disregard it.
While automation handles the bulk of data processing, we also use a human-in-the-loop approach to make sure we remove all the noise. In short, this approach allows us to ensure data quality by picking up asset-level changes, faulty data, and curtailment moments.
Secondly, accurate forecasting depends not just on raw data but on how we represent it within the model. This is where feature engineering comes in – designing the most effective set of inputs for our models that will output the most accurate forecast.
Moreover, as new data becomes available – either from our customers or updated weather models – the pipeline is designed to adapt. Our model retrains automatically at a frequency we’ve found to be the most efficient balance between responsiveness and stability. Additionally, we generate a new forecast every hour or even every 15 minutes.
Finally, to keep our models sharp, we stay up-to-date with the latest techniques and innovations, with our mitigation of data drift in input weather data as a recent example.
A modern API
In the third step, the model’s output is posted to our API, which enables customers to retrieve their latest forecasts easily. Our API is built on a modern RESTful framework, making it easy to use, secure, and capable of handling dynamic and complex renewables portfolios.
The API is also fully self-service, meaning that our customers can, on their own, add or remove assets to/from their portfolios and be sure that the next forecast will already include these changes. This is particularly time-saving and appreciated during the end-of-the-year changes in portfolios.
AI solar forecasting at scale
The entire process – as explained above and summarized in the image below – is continuously executed across a vast and growing portfolio of solar assets.
Providing accurate, timely forecasts at scale isn’t just about having smart models; it takes a solid infrastructure that can handle large amounts of data in real time. Over the years, our machine learning engineers have built this infrastructure from the ground up, making sure our forecasts stay accurate and reliable for expanding renewable portfolios.
Whether forecasting a single solar park or a portfolio with tens of thousands of solar assets, our infrastructure is built for scale.
Solar power forecasting: What’s next?
Solar energy is growing at an unprecedented pace – having just overtaken coal in Europe – while forecasting is maturing as a discipline. At the intersection of these developments, compelling trends are emerging. Here are four of our expectations:
Jointly forecasting and steering assets
To optimize the use of renewable assets, forecasting will no longer operate in isolation. Instead, we expect to see greater integration of forecasting with asset steering.
Instead of just predicting solar output, models will increasingly support decision-making on when and how to dispatch power across multiple markets, ensuring better alignment with grid demand and price signals.
A shift from day-ahead to intraday
While much of the current trading activity in renewables occurs in the day-ahead market, we notice the focus is shifting toward intraday trading. Thus, forecasting small, short-term fluctuations within the day itself will become increasingly important.
If you’d like to hear about our near-time feature and its value for intraday trading, please reach out.
The rise of modal fusion
With the above shift comes a need for more granular data. Traditional numerical weather prediction (NWP) models may give way to modal fusion, combining diverse data sources for higher accuracy. Examples include satellite imagery for real-time cloud tracking, real-time measurements from solar parks, sky cameras, and storm tracking models.
Asset co-location and forecasting complexity
Grid congestion remains a pressing challenge. Co-locating renewable assets (i.e., wind and solar) and batteries behind a single connection point is emerging as a solution.
However, while this strategy maximizes grid utilization, it also complicates forecasting. Wind and solar profiles often counterbalance each other – windy days are usually less sunny, and vice versa. The result is a complex, ‘mixed signal’ to predict.
More than AI
In conclusion, we recognize both the immense capabilities of AI and the limitations of full automation. As we always say, a successful trading cycle isn’t just about predicting power production accurately but about making the right decisions at the right market moments. In line with this mindset, our processes combine intelligent tools and human involvement.
To find out more about our Wind and Solar Power Forecasts – or further explore how you can capitalize on extreme power prices by using the flexibility of your assets – get in touch.