Example Notebooks

UQLM offers a broad collection of tutorial notebooks to demonstrate usage of the various scorers. These notebooks aim to have versatile coverage of various LLMs and datasets, but you can easily replace them with your LLM and dataset of choice. Below is a list of these tutorials:

🎯 Black-Box Uncertainty Quantification

Black-Box Demo

🎯 White-Box Uncertainty Quantification

White-Box Single-Generation Demo

🎯 White-Box Uncertainty Quantification

White-Box Multi-Generation Demo

🎯 BS Detector: Off-the-Shelf Ensemble for LLM Uncertainty

BS Detector Off-the-Shelf Ensemble Demo

🎯 Tunable Ensemble for LLM Uncertainty (Advanced)

Ensemble Uncertainty Quantification Demo

🎯 LLM-as-a-Judge

LLM-as-a-Judge Demo

🎯 Multimodal Uncertainty Quantification

Multimodal Demo

🎯 Semantic Entropy

Semantic Entropy Demo

🎯 Semantic Density

Semantic Density Demo

🎯 Confidence Score Calibration Demo

Score Calibration Demo