Series: The Sequentia Lectures: Unlocking the Math of AI
Part 7: The Frontier – Open Problems & Research Directions
Lecture 63: Uncertainty Quantification: Making AI Say “I Don’t Know” with Mathematical Confidence
We’ve learned how a standard classification model, at its final layer, uses a function like Softmax to output a probability distribution. It might look at an image and say, “I’m 95% confident this is a cat.” This sounds great.
But what happens when you show this same model a picture of a car? The network has only been trained on cats and dogs. It has no “car” category. It will still be forced to make a choice, and it might output something like, “I’m 55% confident this is a dog.” The model is making a prediction on something it knows nothing about, but it presents its guess with a misleading and potentially dangerous level of confidence.
This is a critical failure mode for AI in high-stakes situations. In medical diagnosis or autonomous driving, an AI that is silently “confused” but presents a confident answer is a huge liability. What we really want is an AI that can say, “I don’t know,” and can quantify how much it doesn’t know. This is the goal of Uncertainty Quantification (UQ).
Two Flavors of Uncertainty
In AI, uncertainty isn’t a single concept. It’s generally broken down into two types:
- Aleatoric Uncertainty (Data Uncertainty): This is the inherent randomness or “noise” in the data itself. If you’re trying to predict a coin flip, there will always be uncertainty, no matter how good your model is. This type of uncertainty cannot be reduced by adding more data of the same kind. It’s a property of the system being measured.
- Epistemic Uncertainty (Model Uncertainty): This is the uncertainty that comes from the model itself being unsure. It’s the model’s own ignorance about the underlying function it’s trying to learn. This happens when the model is given data that is very different from what it saw during training (like showing a car to a cat/dog classifier). This type of uncertainty can be reduced by adding more diverse training data.
Standard neural networks are good at capturing aleatoric uncertainty in their probabilistic outputs, but they are notoriously bad at expressing epistemic uncertainty. They don’t know what they don’t know.
The Bayesian Approach: A Distribution of Models
So, how can we build a model that understands its own ignorance? One of the most powerful approaches comes from Bayesian statistics.
In a standard neural network, each weight is a single, precise number. The training process (Gradient Descent) is all about finding the single “best” value for each weight.
In a Bayesian Neural Network (BNN), we take a different approach. We don’t learn a single value for each weight. Instead, we learn a probability distribution for each weight.
- Instead of Weight₁ = 2.5, a BNN learns that Weight₁ is likely a Normal distribution centered around 2.5 with a certain standard deviation.
A single BNN, therefore, represents not just one model, but an entire infinite ensemble of possible models.
Making a Prediction with Uncertainty
To make a prediction with a BNN, we don’t just do one forward pass. Instead, we:
- Sample a model: Randomly draw a specific value for every weight from its learned probability distribution. This gives us one complete, plausible neural network.
- Make a prediction: Run our input data through this sampled network to get one prediction.
- Repeat, Repeat, Repeat: Go back to step 1 and repeat the process hundreds or thousands of times, each time with a slightly different set of weights sampled from the distributions.
We now have a distribution of predictions, not just a single one.
- If the input data is familiar (in-distribution): All the sampled models will have learned similar things about this type of data, so their predictions will all be very similar. The final distribution of predictions will be tight and narrow. The model is certain.
- If the input data is unfamiliar (out-of-distribution): The different sampled models, with their slightly different weights, will react to this strange new data in very different ways. Their predictions will be all over the place. The final distribution of predictions will be wide and spread out. The model is uncertain, and it is mathematically expressing that uncertainty.
The Importance of “I Don’t Know”
Building AI systems that can quantify their own uncertainty is a critical frontier for AI safety and reliability.
- In a self-driving car: An uncertainty-aware perception system, when faced with a confusing or novel object on the road, could signal its low confidence, prompting the car to slow down or hand control back to the human driver.
- In medical diagnosis: A model that is uncertain about a particular scan could flag it for review by a human expert, rather than making a silent, potentially incorrect, low-confidence guess.
Uncertainty Quantification aims to make our AI models more honest about the limits of their knowledge. It’s the crucial step in moving from models that just provide answers to models that can be trusted partners in a decision-making process, capable of knowing when it’s wisest to say, “I’m not sure, you should take a look.”
