Top-p / Top-k

Top-p (nucleus sampling) and Top-k are parameters that control which set of tokens the model selects from next. Top-k limits the selection of the k most likely tokens. Top-p selects the minimum number of tokens whose total probability is >= p (for example, 0.95).

In practice, these parameters work in conjunction with temperature. I rarely use them - usually the default API values ​​(top-p=1, top-k disabled) give normal results. But if you need fine tuning: for coding and structured tasks I lower top-p to 0.7–0.8, for creative writing I leave it at 1.

The main thing to understand is: if temperature = 0, then top-p and top-k have no effect - with a deterministic choice, only the most probable token is always taken. These options only work when there is randomness. For most marketing tasks with LLM, there is no need to change them - just write the prompt correctly.