greedy: the model will always select the most likely token. This generation mode
is deterministic and only suited for applications in which there is a ground truth the
model is expected to return (e.g. question answering).
nucleus: the model will only consider the most likely tokens with total
probability mass p. We recommend this setting for most applications.
topk: the model will only consider the k most likely tokens.
typical: the model will discard high probability tokens with low expected information content.
Generated using TypeDoc
How the model will decide which token to select at each step.