వి హావ్ అ నెక్స్ట్ స్పీకర్ ఇంగే వెన్ ఎండే అండ్ షి ఇస్ గోయింగ్ టు టాక్ అబౌట్ ప్రబబిలిస్టిక్ ఫోర్కాస్టింగ్ ఇంగే ఇస్ ఏటక్లీ at ఎనర్జీ షి వాస్ ద personర్స హబిల్ట్ దఫ్రస్్రచ్ సో ఇనిషally ఇట్ was aboutబౌ 10 peopleల్ అర నౌ are మోర్ దన్ 100 peopleల్ అ it is also సోడ ఎనర్జీ is a compపెనీy which also isన functioning multiple count I would give it to India from here to niet welcome everyone.

I'm a tech lead of our forecasting team at Dexter Energy. And at Dexter we forecast and we optimize renewable energy.

And today I'm gonna talk to you about forecasting and specifically about probabilistic forecasting. So do we even need that? I will give a small example.

So imagine you own your own wind turbine. It's happily spinning in your backyard and you wan to sell the energy that it produces.

So, usually we sell energy a day in advance. So, to be able to sell your energy of your wind turbine, you get a wind forecast.

How much wind is there going to be tomorrow? The forecast model is not spinning very happily in your backyard because there is less wind than was predicted.

Yeah, damn, you already sold it and you're not producing it. These kinds of scenarios can lead to very high costs for renewable asset owners.

Um, and if you would have known beforehand more about the uncertainty, how likely was it that the wind was gonna be lower?

You could have made a wiser decision. Optimize it, think about your risk. That amount of information would be given by a probabilistic forecast.

So, usually we don't do probabilistic forecasting. Usually we work with point forecasts.

So, here you see a time series on the horizontal axis and vertical axis you see the quantity that you predict. So now, energy production. So, until now we have observed the actuals, which are actually points.

For every point in time we observe only one value. So, we build a forecast, which is also one value per point in time.

But actually we already know that it's not gonna be exactly that value.

The value we forecast is what we think is gonna happen. But quite likely it's gonna be a bit different. But is it gonna be very different or will it be very similar?

What do we expect? This line gives us no information about that. So, for that we need probabilistic forecasting.

And I will start with a short refresh of some concepts of probabilistic forecasting to get you all ready to listen to this presentation.

So if I talk about a probabilistic forecast, it means we forecast the full distribution. So a distribution often is shown as a probability density function which basically shows, yeah, the density of the probability where we expect our forecast.

So the higher, the more likely that it's going to be there. So here on the horizontal axis, you see the quantity energy, vertical axis, how likely we expect it.

It's often called PDF, and the area under the curve together is one, so 100%.

So we expect it for 100% to be within the full curve, but most likely it's going to be at the highest point.

And now we're here at PyData. So you work in Python, and then, yeah, this is not something we see in our code, a smooth curve.

So what can you do? You can either do it in a parametric way, that you say I have a Gaussian curve with a sort of standard deviation that describes my curve, or you can do it in a discrete way, which we do a lot, eh, yeah, if you work in Python, that's a very common way to do it.

So then actually, basically, it's just a histogram. You have your quantiles, you say, for certain quantiles, I expect a certain level, and that way you describe the distribution.

Eh, you can do that by setting quantiles, but you could also do that by having a set of samples that together describe a forecast distribution.

We need to evaluate it only one point anymore, so we need different evaluation metrics that we usually use.

And there are lots of evaluation metrics for probabilistic forecasting. I'll talk about only two: calibration and sharpness. So calibration here on the left shows the reliability of your forecast.

Calibration 45 line, all the forecasts exactly are observed in the actual values. So if I look at the distribution of my actual values, it is exactly the same distribution as I'm forecasting.

Usually, that's not the case. Usually, it's not perfectly calibrated.

So here in the example, you see the blue line. So let's say I say for a certain value, I think it's going to be 20% probability that the value is lower, then I look at my actual values and actually I observe that 30% of them is lower.

So then it's not perfectly calibrated as in this example, but we want to strive towards improving this calibration to have a reliable forecast.

But only having a very well-calibrated forecast is not enough. So for example, I take all the data in my train data, I take the average distribution and I forecast that for every moment in time, just give exactly the same distribution would be a very well-calibrated forecast, but it doesn't give me any information because everything is going to be the same.

So if I actually want to make a decision like when is it going to be higher, when is it going to be lower, I have to add more information to the forecast.

Sharpness gives how precise my forecast is. So if I have a very sharp forecast, I'm very certain.

So it's going to be a very narrow distribution. If I don't have a very sharp forecast, it will be a very wide distribution.

Basically saying, I don't really know. Good luck making a decision.

So I want to have a very sharp forecast because then I know, okay, I'm really likely it's going to be there and the combination is essential, then I have a precise and a reliable forecast.

So it can say very precisely it's going to be there and I'm also on average correct.

It matches with the real distribution. So now we heard some things about probabilistic forecasts, about the amount of information that it actually adds.

And I hear you all thinking, this is very cool. I want to build a probabilistic forecast. I want to start on this today and I'm for my project, I will turn the point forecast that I'm working with, I will turn it into a probabilistic one and then you start.

Probabilistic forecasting is a very extensive field. It's quite easy to get swamped up in lots of papers, lots of blogs, also lots of evaluation metrics, many Python packages.

What to pick from? Um, so it's a good kickstart to just try it out, see where it adds value for your project.

And that's my goal for this presentation today. After this presentation, you can start today to convert your project that you already work on into a probabilistic forecast.

Just try it out. See where it adds value without first having to do a whole study on this topic.

So to do that, I will present you two practical models that give you the full distribution without having to do any assumptions beforehand.

So you don't have to know anything about the distribution of your data beforehand, but still they give you the full distribution and you can directly start on it and you can also use it in every project.

So we'll start with the Level Set Forecaster.

The cool thing about this one is that you can combine it with every point forecast that you already have. So if you're working on a project, you have a regression forecast.

It doesn't matter which model you're using. It can be linear regression, can be tree-based models, can be deep learning.

Whatever model you already have, you can use it, but it will be converted into a probabilistic forecast.

So how does that work? This is the workflow you know from the model that you already have. We train it and we predict.

So we train, we fit it on our train set and we predict on our test set. So here on the slide I show an LGBM model, but that can be any model, just whatever you are using in your project.

So the level set forecaster follows this same workflow but it adds a few extra steps to add some more information to your forecast, to add information about this distribution.

So it starts with training your model. And this is the code you already have in your project.

There's nothing new about that. Use your usual model, you train it. And instead of moving to prediction, we start with doing an in-sample prediction.

So we don't predict the test set, but we first predict the train set.

And these predictions give us information for the next step. So we have these predictions, our in-sample prediction, and we sort them. We sort them from low to high.

And with that, we also sort the actual value. So the actual target that we're working with.

And this sorting is used to create the bins. So you have a certain bin size that you select and then you create bins with the same level of prediction.

So a level set is a set with one similar prediction level. And the observations in this set will be your forecast.

It will be a probabilistic forecast. So how would it look like with a decision tree? So let's assume the model that you work with is a decision tree.

Here I did draw the tree. You have your splits based on your features. You end up in one of the leaves.

That leaf will predict a certain value, for example, here 1.3, and then all the observations with these predictions will be grouped together.

So all the observations in this leaf of this decision tree will be grouped into one set.

But it can also happen that two of the leaves give the same prediction. In that case, those will also be grouped together.

So the level set forecasts are really groups based on prediction, not based on features. Groups based on prediction, not based on features.

And then in this example, all the observations of the two leaves are grouped together and that set of samples will be your probabilistic forecast.

So in code, it looks like this. You have already your sample predictions from the previous step. You sort them from low to high.

So that gives you the sorting index, and then you sort your target. So your actual observations, you sort in the same order as the predictions.

You pick your bin size and you make the groups. The size of the bins is a hyperparameter of the level set forecaster.

So this actually defines what you, yeah, where you also want to go with your forecast. If it's very small, the bin size, you will get a very sharp forecast.

If it's very high, the bin size, you group a lot of points together, you will have more reliable, better-calibrated forecasts.

But it also is less precise. So you have to find the balance for you in your data.

So now we have done everything to group those sets together. We have kept them as a group so that we can select from it.

And we do again our point prediction first. So we predict on the test set. This also is just the same as you always do.

Just your usual point model, you predict on your test set. Exactly the same code, but then instead of using that value that we predict, we use it to select one of those sets.

So we see, okay, is this a high prediction? Then I select a group that correlates with this high prediction.

I select a group of observations, and that group of observations is my probabilistic forecast. So you have just made your first probabilistic forecast.

It's given as a set of samples, but from that set, you can also determine quantiles that works easier for you for the rest of your code.

And here we see our first probabilistic forecast. So you see in the light gray, you see 10 to 90% probability, darker gray 30 to 70%, and the darkest is the median and in blue the mean.

So you see there is a lot more information than if this would be only one line, because I can really see at some point one side is very uncertain, while just one line wouldn't have given me that information.

Um, but we also see something else, most likely you already saw it. It can be repetitive because of this grouping and selecting these groups.

It can be that for multiple times in a row we select the same group, which means I get exactly the same project.

So if this is a problem for your project, you can use the next method, the Quantile Regression Forest. So the Quantile Regression Forest will give a unique forecast for every sample.

So that doesn't give this repetitiveness. So if that's important to you, you can use it.

But the downside is it works only with a random forest. So random forest needs to be working for your project.

So the Quantile Regression Forest also follows the same workflow: train and predict. Similar to the level set, again we use our same code.

We fit a random forest, so here you cannot swap it with other models, but just your random forest. But if in your project you already use a random forest, that's perfect, you can just continue with it.

And how we usually use a random forest is that we predict an average value per leaf.

We look at all the trees of the random forest and we take the average prediction of, so per leaf, we give an average prediction, we take the average of all the trees and we have our prediction.

So it gives us a point forecast, just one value. Now what we do in the Quantile Regression Forest is that we don't only look at this one value, we actually keep the information that's in the leaves.

We don't throw away all those samples that are there.

We keep that information that is actually in the leaves already. So again, we have the first tree of a random forest.

We follow the split. So the features are split, follow it into one leaf and we keep the observation in that leaf.

So this is also important to realize where the level set forecaster really makes splits based on predictions only. Here we actually split based on features because it uses the features that are used in the model to make the splits.

Then we do that for every tree. So we have a lot of trees and we have a lot of groups of samples together.

We want to have that every leaf has the same weight.

So for that we have to resample. So we resample all the leaves to the same number of observations which means that a leaf has a weight of one instead of that every observation has a weight of one.

Inge van den Ende - Kickstart Your Probabilistic Forecasting | PyData Amsterdam 2025

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

We aim to be an accessible, community-driven conference, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.

Inge van den Ende - Kickstart Your Probabilistic Forecasting | PyData Amsterdam 2025

Scripts