Prepare and test your skills
Prepare and test your skills
Integration of emulators into automated testing pipelines is the practice of starting a local simulation of a Google Cloud service inside a continuous integration (CI) workflow and running tests against it. The application connects to the Google Cloud SDK (opens in a new tab) emulator instead of the real cloud service, so unit, integration, and end-to-end tests validate logic before deployment. This approach saves costs, avoids impacting production services, and lets teams iterate faster during development.
Local emulators for cost-effective validation are local servers that simulate the behavior of Google Cloud services with high fidelity for debugging and verifying application behavior before deployment. The emulator runs on the local machine or CI runner, so tests do not create cloud resources, touch production data, or depend on a live service being reachable. The Google Cloud SDK provides emulators for several services:
Because the same Cloud Client Libraries work against both the emulator and the real service, application code does not need separate logic for testing and production. Traffic flows from the application through the client libraries to either the local emulator endpoint or the production API endpoint, depending on environment configuration.
Automated testing in CI/CD pipelines is the validation of code changes before those changes are promoted to target environments. Incorporating automated testing into the pipeline helps ensure the quality and reliability of cloud applications, significantly reduces the risk of errors and regressions, and builds a more stable software system. Tests run first against the emulator, then only passing changes move forward to deployment.
Cloud Build executes builds on Google Cloud and can import source from Cloud Storage, Cloud Source Repositories, GitHub, or Bitbucket and produce artifacts such as Docker containers or Java archives. Cloud Deploy automates delivery of those artifacts to a series of target environments in a defined promotion sequence. Artifact Registry provides a single location for managing packages and Docker container images across the full artifact lifecycle.
Choosing testing tools for pipeline integration is the decision about which tools can start an emulator, run tests, and report results inside the same automated workflow. Teams need to test for functional and non-functional scenarios to measure and mitigate impact on functionality, integration, security, performance, and availability. Selection factors include:
When the tooling supports emulator startup, test execution, and result reporting in the same workflow, unit, integration, and end-to-end testing can all run automatically on every change.
Starting a local emulator for repeatable tests creates a clean, in-memory stand-in for the cloud service on the CI runner or developer machine. The Bigtable emulator (opens in a new tab) is a local, in-memory emulator for Bigtable that is intended for local development and testing, not for production deployments. The emulator stores data only in memory, so it will not persist data across runs and every test run starts clean.
The emulator does not provide administrative APIs to create or manage instances and clusters, does not support a secure connection, and cannot be used in the HBase shell. After the emulator starts up, any project and instance name can be used to connect, create tables, and read or write data. The two most common ways to run the emulator are the gcloud CLI and Docker. To start it with the gcloud CLI:
gcloud components update beta to get the latest features.gcloud beta emulators bigtable start, which by default binds to localhost:8086 and prints the host and port where it is running.--host-port=HOST:PORT to bind to a specific host and port when running multiple emulators, and type Control-C to stop the emulator.To start it with Docker, run the SDK container and map ports from the container to the local host, for example docker run -p 127.0.0.1:8086:8086 --rm -ti google/cloud-sdk gcloud beta emulators bigtable start --host-port=0.0.0.0:8086. The same container approach can be added to an existing Docker Compose configuration for CI runners.
Exam tip: The Bigtable emulator keeps data only in memory with no persistence across runs, and it accepts any project and instance name because it has no instance-management APIs.
Connecting application code to the emulator depends on the BIGTABLE_EMULATOR_HOST environment variable. When the application starts, the Cloud Client Library automatically checks for BIGTABLE_EMULATOR_HOST and connects to the emulator if it is running. Set the variable to the emulator address, such as localhost:8086, before running tests, then unset it after the emulator is stopped so later runs connect to the real service.
This redirection lets all Cloud Bigtable client libraries use the same code path for local and cloud runs. A Java wrapper is also available that bundles the built-in Bigtable emulator and provides a Java interface for writing tests.
Evaluation of emulator limitations and feature parity is the assessment of where local simulation matches production and where it does not. Google Cloud SDK (opens in a new tab) emulators simulate the behavior of Google Cloud services on a local machine so applications can be developed and tested without using production resources. The simulation provides a high-fidelity environment for debugging issues and verifying application behavior before deployment, but high fidelity does not mean full parity because local emulators remove production controls for identity, durability, security, and scale.
The scope of local emulation covers five database and messaging services that are accessed through Cloud Client Libraries and the gcloud CLI. Each emulator reproduces the service API locally so development can iterate faster, save costs, and avoid impacting production services:
All five emulators run inside the local development boundary, while live services run inside the Google Cloud production boundary with real projects, networks, and billing. Data flows from the application through the Cloud Client Libraries to either the local emulator endpoint or the production API endpoint, depending on environment configuration.
Authentication and access control differences mean that a successful local connection does not prove that Identity and Access Management (IAM) permissions are correct in production. The Bigtable emulator (opens in a new tab) accepts any project and instance name after startup, and the client library connects to the emulator automatically when the BIGTABLE_EMULATOR_HOST environment variable is set. The Bigtable emulator does not support a secure connection, so transport security and credential checks enforced by production are absent locally.
Production Firestore and Datastore mode databases enforce access through a default service account that grants full read and write access within the same project, and changing that service account requires an IAM role with the required permissions. The local Datastore mode and Firestore emulators exist specifically to bypass that production permission boundary during development and testing. Test IAM roles, service-account changes, and cross-project access only against a live project, not against an emulator.
Persistence and performance boundaries separate volatile local state from durable and scalable production state. The Bigtable emulator is a local, in-memory emulator that stores data only in memory and does not persist data across runs. The emulator is intended for local development and testing, not for production deployments.
Because the emulator runs on one machine instead of on distributed Google Cloud infrastructure, local results validate application logic without validating production durability, availability, or throughput. Restarting the emulator returns it to an empty state, and stopping the emulator requires unsetting BIGTABLE_EMULATOR_HOST so later client-library calls return to the production endpoint. Use emulators to avoid cost and production impact during iteration, then verify stateful behavior, failover, and load against live services.
The Bigtable emulator illustrates the pattern of service-specific feature gaps that limit parity with live services. The emulator works with all Cloud Bigtable client libraries for table creation and for reading and writing data, but it omits production management and access paths:
These gaps mean the workflow depends on the emulator only for data-plane logic, while control-plane work remains untested until deployment. Create tables and read or write rows locally to verify code paths, then test instance provisioning, cluster management, secure connectivity, and HBase shell tooling against live Bigtable. The same principle applies to the Spanner, Pub/Sub, and Firestore emulators: local validation confirms application behavior, not global distribution, production messaging guarantees, or IAM enforcement.
Exam tip: The Bigtable emulator accepts any project and instance name and has no secure connection, so a working local test does not prove that production IAM permissions, instance names, or network security are configured correctly.
Local environment configuration and service redirection is the setup that lets an application talk to a local emulator instead of a production Google Cloud service. The Google Cloud CLI (gcloud CLI) (opens in a new tab) installs and starts the emulator on localhost, and mandatory environment variables redirect requests from Cloud Client Libraries to that local endpoint. This pattern saves costs, avoids impacting production data, and allows faster iteration because topics, tables, and messages are created and handled locally.
Installing emulators with the gcloud CLI adds the service-specific emulator binaries to the development machine. Update the CLI first so the emulator components are current, then install only the emulators needed for the application. The application and emulator should run with compatible Cloud Client Libraries for the language in use.
gcloud components update beta; the first gcloud beta emulators bigtable start prompts to download the emulator binary if it is not installed.gcloud components install pubsub-emulator, then run gcloud components update.gcloud components install cloud-spanner-emulator, then run gcloud components update.An alternative install path is Docker, using the google/cloud-sdk image for Bigtable or gcr.io/cloud-spanner-emulator/emulator for Spanner and mapping the container ports to the same localhost ports.
Starting emulators and binding local endpoints creates a localhost server that stands in for the production service endpoint. Each emulator listens on its own default host and port, and the --host-port flag binds it to a specific address when running multiple emulators or when IPv6 is disabled. Resources created in the emulator, such as Bigtable tables or Pub/Sub topics and subscriptions, exist only for the lifetime of the emulator session.
The Bigtable emulator (opens in a new tab) defaults to localhost:8086 and stores data only in memory, so data does not persist across runs. The Pub/Sub emulator (opens in a new tab) defaults to port 8085 on IPv6 localhost (::1), with --host-port=0.0.0.0:8085 for all interfaces or --host-port=127.0.0.1:8085 for IPv4. The Spanner emulator uses two local endpoints: localhost:9010 for gRPC requests and localhost:9020 for REST requests. The Bigtable emulator does not provide administrative APIs for instances and clusters and does not support a secure connection, so any project and instance name can be used to create tables after startup.
Redirecting application traffic with environment variables tells Cloud Client Libraries to connect to the local emulator instead of production Google Cloud endpoints. The variables must be set on the same machine that runs the application, and the client library checks for the variable automatically when the application starts. For Pub/Sub, the variables must be set each time the emulator starts because the assigned port can change on restart.
| Emulator | Local endpoint | Environment variable to set |
|---|---|---|
| Bigtable emulator | localhost:8086 | BIGTABLE_EMULATOR_HOST=localhost:8086 |
| Pub/Sub emulator | Port 8085, for example [::1]:8085 | PUBSUB_EMULATOR_HOST and optional PUBSUB_PROJECT_ID |
| Spanner emulator | localhost:9010 for gRPC, localhost:9020 for REST | SPANNER_EMULATOR_HOST=localhost:9010 |
On Linux and macOS, export the variable such as export BIGTABLE_EMULATOR_HOST=localhost:8086 or export SPANNER_EMULATOR_HOST=localhost:9010, and on Windows use set. For Pub/Sub and Spanner, automatic setup is available with $(gcloud beta emulators pubsub env-init) or $(gcloud emulators spanner env-init) on Linux and macOS. Manual setup runs env-init to display the values, then sets PUBSUB_EMULATOR_HOST on the application machine, which is required when the application and emulator run on different machines. The gcloud CLI itself can also be redirected for Spanner by creating a separate configuration that disables credentials and overrides the endpoint to http://localhost:9020/.
Exam tip: The Bigtable emulator keeps data only in memory with no persistence across runs, while Pub/Sub emulator variables must be reset on every restart because the port assignment can change.
Google Cloud SDK emulators start as local servers inside a continuous integration (CI) workflow so unit, integration, and end-to-end tests run against the emulator instead of the real service. The same Cloud Client Libraries work against both the emulator and production, so tests validate logic before deployment without creating cloud resources, touching production data, or depending on a live service being reachable.
The Bigtable emulator validates table and data operations but does not match production for management, security, durability, or scale. It stores data only in memory with no persistence across runs, accepts any project and instance name, provides no administrative APIs to create or manage instances and clusters, supports no secure connection, and cannot be used in the HBase shell. A successful local test therefore validates application logic but does not validate production Identity and Access Management (IAM) permissions, secure connections, durability, availability, or throughput.
Cloud Client Libraries automatically connect to a local emulator instead of production when the corresponding environment variable is set on the application machine. The Bigtable emulator defaults to localhost:8086 with BIGTABLE_EMULATOR_HOST, the Pub/Sub emulator defaults to port 8085 with PUBSUB_EMULATOR_HOST, and the Spanner emulator uses localhost:9010 for gRPC requests and localhost:9020 for REST requests with SPANNER_EMULATOR_HOST set to localhost:9010.
A software engineering team is building an event-driven microservices application that reads from and writes to Google Cloud Pub/Sub and Cloud Firestore. During continuous integration (CI) pipeline executions on self-hosted runners, the team wants to run automated integration tests to validate end-to-end message flows and data persistence logic.
The CI environment has strict operational constraints:
Which strategy should the team implement in their CI pipeline?
Last updated