Temperature Sampling — Step-by-Step Visualization

Algorithm Pattern

Logit Rescaling Before Softmax

Key Idea

Temperature T controls randomness: T<1 makes the model more confident (peaky distribution), T>1 makes it more exploratory (flat). T=1 is standard softmax.

Step-by-Step Approach

1. Divide logits by temperature T: z' = z / T.
2. Apply softmax to the scaled logits.
3. T < 1: distribution sharpens — model picks likely tokens more often.
4. T > 1: distribution flattens — more diverse, creative outputs.
5. T → 0: greedy decoding (always pick the highest-probability token).

Common Gotchas

• Temperature does NOT change which token has the highest probability — just the gap between them.
• Top-p (nucleus) sampling combines temperature with a probability mass cutoff.
• High temperature increases diversity but can produce incoherent text.