Prepare and test your skills
Prepare and test your skills
Validation and refinement of AI-generated logic is the human review that turns an AI draft test into a trustworthy test. An AI coding assistant can quickly create unit tests for cloud-native applications, but the developer must check every assertion for accuracy and completeness and manually add the edge cases, error handling, and boundary conditions that automated suggestions miss.
AI-assisted test generation starts from a prompt that contains the function under test and a clear instruction. In one example, the prompt prefix Write a unit test for this function: is followed by a Python is_leap_year(year) function and sent with temperature 0.5 and maxOutputTokens 256 to the code-bison@001 model through the Vertex AI prediction service. Cloud Code (opens in a new tab) provides this capability inside the integrated development environment (IDE) by integrating with Gemini Code Assist Standard and Enterprise, an AI-powered collaborator that helps solve problems, generate code, and receive inline suggestions while coding. Cloud Code with Gemini Code Assist is not auto-enabled by default and requires connecting to Google Cloud, selecting a Google Cloud project, and enabling the Identity and Access Management (IAM) permissions before use.
The generated output is only a starting proposal because generative models have intrinsic randomness and variability, even for identical inputs. That variability creates two challenges for testing: difficulty generating a comprehensive set of test cases for open-ended outputs, and reproducibility issues when the same input does not always produce the same result. Human validation therefore treats every AI suggestion as untrusted until it is executed, reviewed, and refined against the real behavior of the code.
Evaluating a generated test means checking that every assertion matches the actual branching logic and that coverage includes both happy paths and failure paths. For the is_leap_year example, a complete test must verify the full decision chain where a year divisible by 4 is a leap year, except when divisible by 100, except when also divisible by 400. Code coverage is tracked at the project level to ensure expected behavior is validated, and any critical user journey must also have integration tests that validate the functionality of all necessary components and dependencies. An AI draft that only checks one or two years is incomplete until the developer adds the missing branches and verifies that the test fails when the function is wrong.
This evaluation follows the shift-left principle, which moves testing earlier into development because fixing a bug during development has no customer impact and no context-switching overhead. In practice, the development flow moves from writing code and corresponding tests, to running the written tests plus presubmit checks, to requesting iterative human review, and only after author and reviewer reach consensus does the code flow from the development environment into the shared repository. Because coding standards block submission when tests fail, early validation prevents incomplete AI tests from reaching later qualification and rollout phases.
Refinement is the manual step where the developer adds the complex inputs that predetermined AI examples usually miss. Fuzzing complements unit tests by bombarding an application with random inputs to expose hidden flaws, security vulnerabilities, or crashes that users might trigger in unexpected ways. For generative AI behavior, the same gap is addressed with adversarial testing to identify vulnerabilities and failure modes, plus techniques such as subgroup analysis and counterfactual generation to identify and mitigate bias. To assess generated outputs for quality, relevance, safety, and adherence to guidelines, teams incorporate human-in-the-loop evaluation and continuous monitoring with automatic metrics for response quality, safety, instruction adherence, grounding, writing style, and verbosity.
Boundary refinement also requires explicit error-path tests around invalid types, null values, out-of-range numbers, and failed dependencies. Static analysis finds erroneous, inefficient, or deprecated code patterns without running the code, while dynamic analysis compiles and runs code to uncover issues that only surface during execution like memory violations and race conditions. The dynamic tools for this runtime validation are:
Exam tip: Unit tests validate expected behavior with predetermined inputs and outputs, while fuzzing validates unexpected behavior with random inputs — use both when an AI draft only covers the happy path.
Presubmit validation confirms refined tests automatically before human review and before merge. Presubmit tests are checks that run before changes are submitted, covering unit tests, fuzz tests, hermetic integration tests, and static and dynamic analysis in parallel while the engineer is proposing changes. Every presubmit test runs automatically for each changelist prior to human code review, and for changes to core libraries or widely used code, a global presubmit tests the change against the entire codebase to minimize impact on other projects. This continuous feedback loop enforces coding standards directly in the development environment.
The same tests then flow through continuous integration and continuous delivery (CI/CD) pipelines that validate changes before deployment to testing, staging, and production. Cloud Build is a fully managed build service that defines and executes build steps and can be triggered by code pushes and pull requests, while Cloud Deploy automates deployment across environments with features like blue-green deployments, traffic splitting, and rollback. The three test levels used in those pipelines differ by scope and purpose:
| Test type | Scope | Choose when |
|---|---|---|
| Unit testing | Individual units such as functions or methods | The goal is to verify that isolated logic works as expected |
| Integration testing | Interactions between components or modules | The goal is to verify that connected parts work properly together |
| End-to-end testing | Application as a whole in real-world scenarios | The goal is to verify that the system meets end-user requirements |
Validating AI-generated unit tests through presubmit checks and then through Cloud Build and Cloud Deploy pipelines ensures that refined logic, edge cases, and error handling are proven before production traffic ever reaches the change.
Mocking Google Cloud services with AI assistance is the practice of replacing live cloud dependencies with locally controlled doubles so unit tests run fast and safely. An AI coding assistant drafts the mock or stub code inside the development environment, and the test then runs against that double instead of against production infrastructure. This approach keeps application logic isolated while still simulating realistic service behavior for Cloud Storage, Pub/Sub, and Firestore.
Isolated unit tests are tests that validate application code without sending traffic to live Google Cloud projects. The test depends on a mock or local emulator, not on the production service, so failures point to code defects rather than network issues, permissions, or shared test data. Isolation also avoids impacting production services, saves costs during repeated test runs, and lets developers iterate faster during development.
This isolation supports shift-left testing, where validation moves earlier into the development loop instead of waiting for deployment. Automated checks run continuously while code is being written and again before changes merge, which catches bugs with no customer impact and no context-switching overhead. Incorporating testing into pipelines built with products like Cloud Build and Cloud Deploy validates changes before deployment and reduces regressions:
AI-assisted mock generation is the use of Gemini Code Assist inside Cloud Code (opens in a new tab) to draft test doubles for Google Cloud client libraries. The developer describes the desired behavior in the IDE, and the assistant generates mock code, proposes fixes, and offers inline suggestions while code is written. Because Cloud Code brings Google Cloud services, Cloud Client Libraries, and run-ready samples directly into the IDE, the generated mock can match the idiomatic patterns for Go, Java, Node.js, Python, and .NET Core without context switching.
Gemini Code Assist Standard and Enterprise are not auto-enabled by default, so access must be configured before generation can begin:
Only after these three steps can the assistant reliably generate and explain code. Generated mocks and stubs must still be reviewed because the assistant can invent methods, omit error paths, or simulate success when the real service would return permission, quota, or not-found errors. Validation means running the unit test locally, checking that the mock receives the expected calls with the expected arguments, and confirming that retry and error handling match the real client library contract.
Google Cloud SDK emulators are local processes that simulate the behavior of Google Cloud services for development and testing. The application connects to the emulator endpoint instead of the cloud endpoint, so requests flow from the test to the local emulator and responses flow back from the emulator to the test without crossing a production trust boundary. This provides a high-fidelity environment for debugging issues and verifying behavior before deployment, while avoiding cost and production impact.
The available emulators differ by the service behavior they reproduce locally:
| Emulator | What it simulates locally | Use when testing |
|---|---|---|
| Firestore | Document database for mobile, web, and server applications | Document reads, writes, and queries without touching live data |
| Pub/Sub | Asynchronous messaging workflows | Publish and subscribe flows, including ordering and delivery handling |
| Spanner | Globally distributed relational database | Relational queries and transactions that need Spanner semantics |
| Bigtable | NoSQL wide-column database | High-throughput row-key access patterns |
| Datastore | NoSQL document database in Datastore mode | Legacy Datastore API behavior |
Choose lightweight mocks or stubs when the goal is pure unit isolation and speed, such as verifying that storage upload code handles an error return. Choose an emulator when the goal is behavioral fidelity, such as verifying that Firestore queries or Pub/Sub message handling work as the service would perform them. In both cases the test must depend on the double, never on the live service, so the same test produces the same result on any workstation or build agent.
Exam tip: Point unit tests at a mock, stub, or local emulator endpoint, not at a live Google Cloud project, to save costs and avoid impacting production services.
Prompt engineering for contextual test generation is the practice of giving an AI coding assistant enough surrounding detail that it can produce a relevant and runnable unit test suite. A vague request such as “write a test” often produces generic code that does not compile or misses edge cases. A contextual prompt instead supplies the function source, expected inputs and outputs, and any cloud dependencies so the model can generate assertions that match real behavior.
In Google Cloud development, this work typically happens through Cloud Code (opens in a new tab) extended with Gemini Code Assist Standard and Enterprise. Cloud Code is an IDE extension that brings Google Cloud services such as Google Kubernetes Engine (GKE) and Cloud Run directly into the editor. Gemini Code Assist is an AI-powered collaborator inside that extension that solves problems with code, generates new code, and offers inline suggestions while coding.
Essential context is the set of details that lets the model move from guessing to generating tests that run against the actual function. Without this context, the assistant cannot know parameter types, valid ranges, or which external service must be mocked. Providing the context up front reduces editing and makes the output executable on the first pass.
Include the following peer pieces of context in the prompt when they apply to the code under test:
When the function depends on a cloud service, state the dependency boundary explicitly. Application code sits on one side of the boundary and the managed service sits on the other side. The generated test should run against the application code and use fakes or mocks for calls that would otherwise flow from the test to the live service.
A unit-test generation prompt (opens in a new tab) for a publisher chat or code model combines an instruction with the code to test and with generation settings. The instruction tells the model what to do, the code instance provides the context, and the parameters control the shape of the response. Data flows from the developer prompt to the model endpoint, and generated test code flows back from the model to the developer for review.
One example uses a prefix field that contains the instruction plus the complete function. The prefix holds Write a unit test for this function: followed by the full is_leap_year(year) Python function with its nested if year % 4, if year % 100, and if year % 400 branches. Because the prompt includes every branch, the model can generate cases for divisible-by-4 years, century years, divisible-by-400 years, and common years. Sending only the function name would leave the model to invent the logic and would produce unreliable assertions.
Generation parameters tune the response length and creativity. The example sets temperature to 0.5 and maxOutputTokens to 256, which balances focused output with some variation in test phrasing while capping the response size. A programmatic call packages this prompt as an instance value and a parameters value, builds an endpoint name from project, location, publisher google, and model code-bison@001, and then calls predictionServiceClient.predict(endpointName, instances, parameterValue) against the regional endpoint us-central1-aiplatform.googleapis.com:443.
Exam tip: Include the full function body in the prompt, not just the function name — the is_leap_year example works because the model sees all three modulo branches when it creates test cases.
Test generation inside Cloud Code happens after the assistant is connected to Google Cloud and enabled with the correct permissions. Gemini Code Assist Standard and Enterprise are not auto-enabled by default, so the developer must complete setup before prompting. Once enabled, prompts can reference open files, selected functions, and project libraries without leaving the IDE.
Complete setup and generation in order:
Cloud Code supports this workflow in VS Code, IntelliJ and other supported JetBrains IDEs, and Cloud Shell Editor, which includes Cloud Code by default. Supported languages for development include Go, Java, Node.js, Python, and .NET Core, so prompts can target test frameworks idiomatic to each language, such as JUnit for Java or pytest for Python.
Validation of generated tests means checking that AI-produced code actually compiles, passes, and asserts correct behavior before it enters the codebase. Generated tests must be treated as proposals, not as trusted output, because the model can invent APIs, miss mocks, or assert wrong edge-case results. The developer runs the suite locally, fixes imports and mocks, and confirms coverage of branches before submitting.
Automated testing in continuous integration and continuous delivery (CI/CD) pipelines enforces this validation on every change. Cloud Build is a fully managed build service that defines and executes build steps and can trigger on code pushes and pull requests. Cloud Deploy is a serverless deployment service that promotes releases through testing, staging, and production with controls such as traffic splitting and rollback. Test results flow from the pipeline back to the developer, and only changes that pass the required checks flow forward to deployment.
The three common test levels divide validation work by scope:
| Test type | Scope | Choose when |
|---|---|---|
| Unit testing | Single function or method in isolation | The goal is to verify logic, branches, and edge cases for AI-generated assertions |
| Integration testing | Interaction between components or modules | The goal is to verify that mocked cloud clients and real application code work together |
| End-to-end testing | Full application in a realistic scenario | The goal is to simulate user journeys after unit and integration tests pass |
Presubmit checks extend this discipline by running unit tests, hermetic integration tests, and static and dynamic analysis continuously while code is proposed and before human review. A global presubmit can test a widely used change against a large codebase to catch cross-project breakage. This shift-left approach lets the engineer fix a failing AI-generated test immediately with no customer impact.
is_leap_year, and manually adding edge cases and error handling.is_leap_year prompt sent with temperature 0.5 to model code-bison@001.Include the full function signature and body with branching logic, data schemas with valid and invalid values and edge cases, and the named Google Cloud service dependencies so the model can mock them instead of calling live services. The is_leap_year example works because the prompt contains the complete function with its divisible-by-4, by-100, and by-400 branches and uses temperature 0.5 and maxOutputTokens 256 with model code-bison@001.
AI-generated tests are only starting proposals because generative models have intrinsic randomness and variability and often miss edge cases, error handling, and boundary conditions. Validation means executing the test, checking every assertion against the real branching logic such as the divisible-by-4, by-100, and by-400 rules, and manually adding missing branches, invalid types, null values, and failure paths.
Choose a lightweight mock or stub when the goal is pure unit isolation and speed, such as verifying that storage upload code handles an error return. Choose a Google Cloud SDK emulator for Firestore, Pub/Sub, Spanner, Bigtable, or Datastore when the goal is behavioral fidelity, such as verifying Firestore queries or Pub/Sub publish and subscribe flows without touching live data or impacting production services.
Last updated