Diagram
A decision tree branching on need for factual consistency versus creativity to set low or high temperature, top-K and top-P values, showing predictable versus diverse outputs and tradeoffs.
Balancing Creativity Against Factual Consistency
Balancing creativity against factual consistency is the tradeoff controlled by temperature and top-K sampling together. Lower values narrow the model's token choices, which supports expected and repeatable output, while higher values widen those choices, which supports varied and creative output but raises the chance of inaccurate, irrelevant, or nonsensical results:
| Parameter | What it controls | Effect of a lower value | Effect of a higher value |
|---|---|---|---|
| Temperature | Randomness of predictions | Less random, more predictable output | More random, more diverse output |
| Top-K sampling | How the model selects tokens for output | Less random responses | More random responses |
Domain 4 of 4
36 practice questions
Practice this DomainLast updated