Hyperparameters Definition and Examples for ML

Hyperparameters are settings that control how a model trains or generates output. You choose them. The model does not learn them as weights during training.

A generation request can look deceptively simple: temperature set to 0.7, max_tokens set to 500, and top_p set to 0.9. The API example sets temperature to 0.7. Related analysis The API example sets max_tokens to 500. Related analysis The API example sets top_p to 0.9. Related analysis

Those settings can make the same prompt produce a terse answer, a sprawling draft, a repetitive refusal, or something pleasantly unexpected. The prompt matters, but hyperparameters decide how the model behaves while it responds.

For prompt engineers, the useful hyperparameter meaning is practical: these are the controls around the model. They set the conditions for learning during training or the boundaries for output during inference.

TLDR

Hyperparameters are user-chosen settings that shape model training or generated output. Model parameters are learned weights. For generation, change temperature or top-p separately, then review the result before adjusting max tokens or the prompt.

What Is a Hyperparameter?

A hyperparameter is a setting selected before or around a model run. It influences the process without becoming part of the learned model itself.

That distinction sounds academic until you are debugging output. If a model gives you an answer that wanders, repeats itself, or stops too early, you are usually looking at an inference setting, a prompt problem, or both. You are not editing the model’s internal weights.

Hyperparameters appear in two places:

Where	What the settings control	Common examples
Training	How the model learns from data	learning rate, batch size, epochs
Inference	How the trained model generates a response	temperature, top-p, max tokens

Training settings shape the learning process. Inference settings shape a response from a model that has already been trained. Same word, different job.

That split prevents a common mistake: treating temperature as though it changes what a model knows. It does not. Temperature changes how the model selects among possible next tokens. A better retrieval system, a clearer prompt, or a different model may address a knowledge problem. Temperature will not.

Hyperparameters vs Model Parameters

Model parameters are the values a model learns from training data. They include the weights that help it map an input to a likely output. During training, the system adjusts those values based on its objective and the data it sees.

Hyperparameters sit outside that learned set. A team chooses a learning rate before training. A prompt engineer chooses a temperature before generation. Neither setting becomes a learned weight simply because it affects the result.

Think of model parameters as the model’s accumulated internal configuration. Hyperparameters are the controls you expose around the process.

That makes hyperparameters easier to change and easier to test. You can send the same prompt again with a lower temperature. You can cap the response length. You can widen or narrow the pool of likely next tokens. The model stays the same, but its output behavior can change materially.

It also means a bad result does not always call for hyperparameter tuning. If the prompt is ambiguous, the model may still be ambiguous at a low temperature. If the source material is weak, a more creative setting may simply produce a more polished version of the same weak answer.

Use the right tool for the failure. Hyperparameters govern behavior. Prompts govern instructions. Context governs what information the model has available.

Training Hyperparameters

Training hyperparameters control how a model learns. They are selected before or during the training process, and they affect the path the model takes toward its learned parameters.

Learning rate is the familiar example. It controls how aggressively the training process updates the model’s weights after evaluating an error. Set it poorly and training can become unstable or move too slowly to be useful.

Batch size controls how much training data the model processes together before an update. Epochs describe how often the system works through the training dataset. These settings interact, which is why training work tends to involve careful experimentation rather than a single magic configuration.

Prompt engineers usually do not touch training hyperparameters in ordinary API work. If you are calling a hosted model, the training phase has already happened. Your controls begin at inference.

That boundary is worth keeping clean in documentation and conversation. Calling temperature a training setting confuses the person trying to make an API response more predictable. Calling learning rate an inference setting confuses the person trying to improve a fine-tuning run.

The underlying idea is straightforward. Training hyperparameters influence how a model gets built. Inference hyperparameters influence how it behaves when you use it.

Inference Hyperparameters

Inference hyperparameters control generation after training is complete. These are the settings most prompt engineers encounter in an API request.

Temperature affects variation in the model’s token selection. Lower temperature tends to make outputs more repeatable and conservative. Higher temperature allows more variation, which can help with ideation, creative writing, or generating several distinct angles from the same source material.

In 2026, the page describes temperature 0 as deterministic output. Related analysis That does not mean every task should use temperature 0. It means the model will favor the most likely choices in a way that reduces variation across repeated requests.

Creative work often benefits from more room to explore. The page gives 0.7 to 1.0 as a creative-writing temperature range. Related analysis A creative draft still needs a good brief. Higher temperature is not a substitute for knowing what you want.

Top-p is another sampling control. It limits the model’s choice to a probability-based group of likely next tokens. A lower top-p narrows that group. A higher top-p gives the model a broader set of options.

Temperature and top-p both influence variation. Changing both at once turns a simple test into a muddle. You may get a better result, but you will not know which control caused it.

Max tokens sets a ceiling on response length. It is useful when you need concise outputs, predictable response shapes, or a guardrail against a model producing more than the task requires. A cap does not tell the model how to prioritize. It only sets the amount of room available.

A short max-token setting can cut off an answer before it finishes. A large setting can invite unnecessary length if the prompt lacks boundaries. Tell the model the desired format and scope, then use max tokens as a practical limit.

Hyperparameter Examples in an API Call

Here is a simple request shape using the generation settings from the glossary example:

```json { "temperature": 0.7, "max_tokens": 500, "top_p": 0.9 } ```

The API example sets temperature to 0.7. Related analysis The API example sets max_tokens to 500. Related analysis The API example sets top_p to 0.9. Related analysis

Those settings are not a recommended default for every task. They are a readable example of the three controls working together.

If you are generating a structured extraction, you may want a lower temperature because you care about repeatable formatting and faithful use of the source text. If you are generating headline options, you may accept more variation because sameness is the problem.

The same pattern applies whether you work through the OpenAI API with temperature 0.7 or compare providers through the Anthropic API using max_tokens 500. The parameter names and supported behavior can vary by model, so read the provider documentation before assuming that a familiar setting works identically everywhere.

Top-p deserves the same caution. A top_p value of 0.9 can produce useful variation, but it is not a dial you should twist alongside temperature without a reason. One controlled change beats a pile of guesses.

If you are evaluating prompts at scale, a Promptfoo review can help you compare outputs produced with temperature 0.7, max_tokens 500, and top_p 0.9. The point is not to find a universal winner. It is to decide which output behavior fits the task you are shipping.

How to Tune One Setting at a Time

Start with a clear prompt and a representative task. Save the output. That becomes your baseline.

Change a single hyperparameter, run the same task again, and compare the result against the baseline. Look for the difference that matters to the work: factual discipline, formatting consistency, brevity, variety, or tone.

Keep the prompt, model, source context, and evaluation criteria stable while you test. Otherwise you are changing the experiment halfway through and crediting the wrong setting for the outcome.

Temperature is often the first inference control to test because its effect is easy to observe. Lower it when you need stable answers, extraction, classification, or repeatable phrasing. Raise it when the task needs range, alternate approaches, or creative drafts.

Do not adjust temperature and top-p together. Both affect how the model samples possible output, and changing them at the same time makes the results harder to interpret.

After temperature, test top-p only if you have a specific reason to change the range of likely token choices. Then test max tokens if answer length is the issue. A response that has the wrong facts or the wrong structure needs better instructions or better context before it needs a larger output cap.

Write down what changed and what happened. This can be as simple as a small evaluation note beside each run. The goal is not ceremony. The goal is being able to reproduce the setting that produced the output you want.

A useful tuning workflow has a plain sequence:

• Define the task and what a good output looks like.
• Run the prompt with your current settings.
• Change a single setting.
• Compare the outputs against the same criteria.
• Keep the change only when it improves the task.
• Move to the next setting after you understand the last one.

Hyperparameter tuning is less glamorous than finding a clever prompt trick. It also saves time. You stop treating every odd output as evidence that the model is mysterious and start isolating the variable that changed.

Choosing Settings for the Job

For repeatable workflows, start conservative. Tasks like extraction, routing, rewriting to a fixed format, or turning notes into structured fields usually benefit from lower variation. You want the model to follow the same path when the input is similar.

For creative work, use a setting that gives the model room to explore, then judge the results. The page gives 0.7 to 1.0 as a creative-writing temperature range. Related analysis That range can generate variety, but the best value still depends on the model, the prompt, and how much cleanup your workflow can tolerate.

For concise answers, set a clear format in the prompt and use max tokens as a boundary. The API example sets max_tokens to 500. Related analysis If the answer cuts off, inspect the prompt before assuming the cap is the whole problem. The model may be spending its available output on throat-clearing, repeated caveats, or a format you never specified.

For broad exploration, test temperature first and leave top-p alone. If you later test top-p, compare against the same prompt and evaluation set. You are trying to understand behavior, not win an argument with a settings panel.

Hyperparameters reward restraint. A few deliberate changes tell you more than a request full of knobs set to arbitrary values.

Key Takeaways

• Hyperparameters are chosen settings, while model parameters are learned weights.
• Training settings control how a model learns; inference settings control how it generates.
• Temperature affects output variation, while max tokens limits output length.
• Test temperature and top-p separately so you can identify what changed.
• Tune against a defined task, not against a vague feeling that the output looks better.

Sources

• PE Collective hyperparameters glossary temperature 0.7
• PE Collective hyperparameters glossary max_tokens 500
• PE Collective hyperparameters glossary top_p 0.9
• PE Collective hyperparameters glossary temperature 0
• PE Collective hyperparameters glossary creative-writing range 0.7 to 1.0