Prepare and test your skills
Prepare and test your skills
Probabilistic control through temperature and top-p is the use of sampling settings to tune how random or predictable a generative AI model's word choices are. Generative AI models are probabilistic, so the same prompt can produce different results, including offensive, insensitive, or factually incorrect text. Sampling parameters let a developer trade creative exploration against factual consistency by changing how the model picks each next token.
Temperature is the generation parameter that controls the randomness of a model's predictions on a scale from 0.0 to 1.0. A lower temperature makes token choices less random and more predictable, while a higher temperature makes token choices more random and more varied. Because temperature applies to every next-token choice, it directly shapes whether the final response is repeatable or diverse. In the Gemini Enterprise Agent Platform (opens in a new tab), temperature is configured through the Agent Platform SDK for ABAP (opens in a new tab) as the Randomness temperature field in the generation parameters table /GOOG/AI_CONFIG.
The /GOOG/AI_CONFIG table holds temperature alongside Model Key, Model ID, Google Cloud Key Name, Google Cloud Region Location ID, and Publisher ID of the LLM. The Model Key is the unique name assigned to a configuration, and that key is used when instantiating the generative model class so the chosen settings take effect. This relationship means the model class depends on the Model Key, and the Model Key carries the temperature value to the model.
Top-K sampling is the generation parameter that changes how the model selects tokens for output. Top-K sampling works in the same direction as temperature: a lower value produces less random responses and a higher value produces more random responses. Top-K is configured as a numeric float field in the same /GOOG/AI_CONFIG table that holds temperature, so both settings are applied together to the same model configuration.
Balancing creativity against factual consistency is the tradeoff controlled by temperature and top-K sampling together. Lower values narrow the model's token choices, which supports expected and repeatable output, while higher values widen those choices, which supports varied and creative output but raises the chance of inaccurate, irrelevant, or nonsensical results:
A decision tree branching on need for factual consistency versus creativity to set low or high temperature, top-K and top-P values, showing predictable versus diverse outputs and tradeoffs.
| Parameter | What it controls | Effect of a lower value | Effect of a higher value |
|---|---|---|---|
| Temperature | Randomness of predictions | Less random, more predictable output | More random, more diverse output |
| Top-K sampling | How the model selects tokens for output | Less random responses | More random responses |
Randomness settings alone do not guarantee accuracy. Models need context grounded in real-world information, physical properties, and accurate understanding of specific data to reduce poor output. The quality, accuracy, and bias of the prompt or input data also strongly affect response quality, so an inaccurate prompt can still produce false output even at low randomness.
Safety filters and token limit boundaries are guardrails that operate alongside sampling parameters. Gemini Enterprise Agent Platform generative AI APIs provide safety attribute scoring that lets customers test Google's safety filters and set confidence thresholds suited to their use case, and Vertex AI Studio includes built-in content filtering. Google recommends a four-step practice: first assess the application's security risks, then perform safety testing for the use case, then configure safety filters if required, and finally solicit user feedback and monitor content. Token limits interact with safety because models have a maximum input and output token limit, and performance can also drop when input data has an unusual or complex structure.
Exam tip: Exceeding the maximum input or output token limit means the safety classifiers are not applied — the limit is a safety boundary, not just a size boundary.
Output constraints and resource management is the control of how much text a generative AI model can take in and generate in one request. Every request is measured in tokens, which are small pieces of text, and every model has a maximum input and output token limit. Setting a maximum output length below that built-in cap lets an application limit resource use, reduce waiting time, and keep answers the right size for the task. In the Google Cloud ecosystem, models hosted on Gemini Enterprise Agent Platform (opens in a new tab) use these limits together with safety checks to keep performance reliable.
Fixed maximum token limits are built-in caps that define the largest prompt a model can accept and the largest response it can produce. Generative AI models cannot process text beyond these caps without losing reliability. If input or output exceeds the limit, safety classifiers are not applied, which can lead to poor performance and unexpected results. Performance can also drop when input data has an unusual or complex structure, even if the token count is within the limit, because models handle common text formats most reliably.
Setting maximum output length is choosing an application-level cap on response size to match a specific need. A short cap fits tasks like summaries, labels, or chatbot replies, while a longer cap fits tasks like detailed explanations or creative drafts. Shorter outputs consume fewer tokens and resources and generally return faster, which lowers latency for users. Matching the limit to the task keeps responses aligned with application requirements and avoids wasted compute on text the application will cut off or ignore.
Exam tip: If input or output exceeds the maximum token limit, safety classifiers are not applied — keep prompts and max output settings within limits to keep safety checks and quality intact.
Safety configurations and ethical guardrails are the controls that limit who can reach data, how that access is recorded, and what happens when a rule is broken. In Google Cloud (opens in a new tab), these guardrails combine services that gate or log access to data with a detection-and-response process that finds and corrects violations. The access services prevent or record access as it occurs, while the response process handles policy violations that slip through, so systems stay within responsible AI and compliance standards.
Services that gate, log, and protect data give visibility and control over Google's access to customer data. Each of the five services protects a different boundary: some gate access before it happens, some record it afterward, and some protect the data itself.
| Service | What it protects | How it works |
|---|---|---|
| Access Approval (opens in a new tab) | Google administrator access to customer data | Requires customer approval before an authorized Google administrator can access data for support; approved requests are logged with Access Transparency logs linked to the approval, and access must still be properly privileged within Google |
| Access Transparency | Visibility into Google administrative access | Logs administrative access by authorized Google personnel when they support the organization or maintain service availability |
| Assured Workloads | Regulatory and regional compliance | Provides a workflow to create and monitor control packages for dedicated regional support, certified programs such as FedRAMP or ITAR, or programs like Sovereign Controls for EU |
| Cloud KMS with Cloud EKM | Encryption keys for data at rest | Encrypts data with keys stored and managed in a third-party key system outside Google's infrastructure, keeping data separate from its keys |
| Confidential Computing | Data in use | Encrypts data while it is being processed |
These services work together rather than in isolation. Access Approval gates a Google administrator's access until the customer approves it, and Access Transparency records that access in logs tied to the approval request. Cloud KMS with Cloud EKM and Confidential Computing protect the data itself — one at rest and the other in use — so sensitive information stays protected even during processing.
Four-stage incident detection and response is the ordered process Google uses to find and act on suspected access violations. The process relies on expert internal investigation teams plus manual and automated controls that combine machine learning, advanced data processing pipelines, and threat intelligence:
Exam tip: Access Transparency records Google administrative access after it happens, while Access Approval blocks that access until you approve it.
Temperature controls the randomness of a model's predictions on a scale from 0.0 to 1.0. A lower temperature makes token choices less random and more predictable, while a higher temperature makes token choices more random and more varied. Because temperature applies to every next-token choice, it directly shapes whether the final response is repeatable or diverse.
Top-K sampling controls how the model selects tokens for output and works in the same direction as temperature. A lower Top-K value produces less random responses, while a higher Top-K value produces more random responses. Top-K is configured as a numeric float field in the same /GOOG/AI_CONFIG table that holds temperature, so both settings are applied together to the same model configuration.
If input or output exceeds the maximum token limit, safety classifiers are not applied, which can lead to poor performance and unexpected results. Every request is measured in tokens, which are small pieces of text, and every model has a maximum input and output token limit. Keeping prompts and maximum output settings within limits keeps safety checks and quality intact.
Access Approval requires customer approval before an authorized Google administrator can access customer data for support, while Access Transparency logs that administrative access after it happens. Approved Access Approval requests are logged with Access Transparency logs linked to the approval, and access must still be properly privileged within Google.
An enterprise development team is deploying generative AI models on Google Cloud for two distinct internal use cases:
Which combination of sampling parameter adjustments should the team apply to achieve the required behavior for both applications?
Configure high temperature and low top-p for both applications, because top-p overrides temperature completely during inference.
Configure low temperature and low top-p values for Application A, and configure high temperature and high top-p values for Application B.
Configure high temperature and high top-p values for Application A, and configure low temperature and low top-p values for Application B.
Configure temperature to 0.0 for both applications and regulate response diversity solely using the max output token limit.
Last updated