Infrastructure Manager, Terraform and Config Connector

GitOps for Google Cloud infrastructure stores declarative Terraform and pipeline definitions in a version-controlled repository as the desired state and uses an automated pipeline to synchronize that state to Google Cloud. A commit or merge to a branch such as main triggers validation and deployment, while Git history plus Google Cloud audit logs provide auditability and git revert provides rollback. Drift between the repository and the live environment is detected out of band and remediated by re-applying the declared configuration.

1.1Declarative Source of Truth and Automated Synchronization

The GitOps source of truth is a repository that holds Terraform files, orchestration pipelines, and related assets such as scripts. Developer workflows create or modify the Terraform configuration in the repository, and a continuous integration and continuous delivery (CI/CD) trigger in Cloud Build (opens in a new tab), GitHub Actions, Jenkins, GitLab Runners, or Terraform Enterprise synchronizes the change to the target project, folder, or organization. Promotion between environments uses separate repositories, branches, or projects per environment, so the same declarative definition moves from development to production without manual recreation. Rollback returns the environment to a prior good state by reverting the repository commit and letting the pipeline re-synchronize, which keeps every change traceable and repeatable.

Promotion across projects depends on parameterizing environment-specific values instead of hardcoding them. In Application Integration, config variables let the same integration definition call different APIs or publish to different Pub/Sub topics per environment, and the per-environment repository holds the correct values. Config variables cannot configure entity operations and actions for connectors, so connector operations must still be managed explicitly per environment. When a private repository is hosted on-premises, the pipeline depends on a private network path from that repository to the Google Cloud environment.

1.2Shift-Left IaC Validation with Cloud Build and Security Command Center

Shift-left Infrastructure-as-Code (IaC) validation checks a Terraform plan against organization policies and Security Health Analytics detectors before resources are created. The validation is performed by Security Command Center (SCC) (opens in a new tab) Premium tier or Enterprise tier, which must be activated at the organization level and enables the securityposture.googleapis.com and securitycentermanagement.googleapis.com application programming interfaces (APIs). The Cloud Build API (cloudbuild.googleapis.com) must also be enabled, and the security posture that defines organization policies and detectors must already be created and deployed.

The validator identity needs organization-level permissions to read the plan, evaluate posture, and write results:

• Security Posture Shift-Left Validator
• Log Writer
• Storage Writer
• Storage Reader

Terraform code must be written so the validator can evaluate it correctly:

• Populate the parent attribute, the project, folder, or organization, on every resource or asset.
• Modify assets and policies separately, because validation does not support a plan that modifies both at the same time.
• Use only supported asset types and policies for IaC validation.
• Review IaC validation limitations and quotas before running at scale.
• Do not include passwords or other sensitive information, because fields marked as sensitive in resource changes are removed before evaluation.

A build config instructs Cloud Build to run validation as part of the GitOps pipeline, and the same validation can also run from the Google Cloud command-line interface (CLI), Jenkins, or GitHub Actions. Creating a report from a Terraform plan follows an ordered workflow:

1. Initialize: Run terraform init with Terraform provider version v5.5 or later, using terraform init -upgrade when an upgrade is required.
2. Render JSON plan: Run terraform plan -out TF_PLAN_FILENAME and then convert it with terraform show -json TF_PLAN_FILENAME > TF_PLAN_JSON_FILENAME.json.
3. Create validation report: Run gcloud scc iac-validation-reports create PARENT --tf-plan-file=TF_PLAN_JSON_FILENAME.json, where PARENT is organizations/ORGANIZATION_ID/locations/LOCATION.

The resulting IaC validation report flows from Security Command Center back to the pipeline and repository owner. A violating plan blocks promotion until the Terraform definition is fixed in version control, which enforces compliance before drift can occur.

1.3Repository and Pipeline Tooling Choices

Repository and pipeline tooling choices determine where GitOps definitions live and what automation executes them. Cloud Source Repositories provides a managed private Git repository, while GitHub, GitLab, or Bitbucket can serve as the version-control system when teams standardize on those platforms. Cloud Build runs the automation pipeline by default in Google Cloud blueprints, while Terraform Enterprise, GitLab Runners, GitHub Actions, or Jenkins can replace it. For data orchestration, a GitHub repository added to the integrated development environment (IDE) workspace stores pipelines and assets, and merging changes to the main branch through a GitHub Action deploys the pipeline to the Managed Service for Apache Airflow environment.

1.4Drift Detection and Background Reconciliation with Cloud Asset Inventory

Cloud Asset Inventory (opens in a new tab) provides organization-wide visibility across Google Cloud resources, Identity and Access Management (IAM) policies, and metadata history for compliance tracking, scheduled audits, and analytics. Cloud Asset Inventory uses periodic synchronization with eventual consistency, so newly provisioned or modified resources may not be immediately queryable and state changes take time to propagate across search queries, feeds, and exports. Delivery from source services to Cloud Asset Inventory can also experience temporary drops, lag, or out-of-order events, so Cloud Asset Inventory is not a synchronous control plane and is not suitable for real-time, mission-critical dependencies where missing or stale data would break availability or security enforcement.

GitOps drift workflows therefore treat Cloud Asset Inventory Pub/Sub feeds as informative signals for background reconciliation loops, dashboards, and delay-tolerant updates. Consumers that reconcile desired state from Git against inventoried state must be resilient and idempotent and handle delayed or out-of-sequence events gracefully. When a critical step requires instant validation, such as verifying a firewall rule before traffic cutover, the workflow calls the corresponding service API directly instead of Cloud Asset Inventory.

Evaluating third-party infrastructure as code (IaC) tools on Google Cloud means choosing how Terraform configurations are written, stored, executed, and governed across environments. The central pattern uses the Terraform provider for Google Cloud (opens in a new tab) to define desired state in human-readable, versioned configuration files that provision and manage resources through a consistent lifecycle workflow. Infrastructure Manager (Infra Manager), Terraform Cloud, and Config Connector then extend that core workflow with managed execution, remote state, and Kubernetes-native management.

2.1Integration Patterns for Terraform on Google Cloud

Terraform integration patterns on Google Cloud separate authoring infrastructure from executing it. Teams author configurations with the Terraform provider for Google Cloud, then select an execution plane that fits their operational needs. Infrastructure Manager automates deployment of Terraform configurations programmatically on Google Cloud so teams do not maintain a separate toolchain, while Terraform Cloud runs Terraform in a stable, remote software as a service (SaaS) environment. Config Connector provides a Kubernetes-centric pattern where Google Cloud resources such as Cloud SQL instances, Pub/Sub topics, Cloud Storage buckets, datasets, models, and endpoints are defined as YAML manifests and deployed to a Google Kubernetes Engine (GKE) cluster.

Terraform Cloud connects to common VCS systems like GitHub, GitLab, and Bitbucket, so new commits to a connected workspace can automatically trigger Terraform plans. Terraform Cloud also offers an API for integration into existing workflows, and Terraform Enterprise provides a self-hosted distribution of Terraform Cloud for organizations that must keep that control plane inside their own boundary.

2.2State Management and Remote Execution

State management for Terraform on Google Cloud is the practice of storing, locking, versioning, and auditing the state file that maps configuration to real resources. Terraform Cloud securely stores state and secrets and runs plans and applies in a stable remote environment, which isolates credentials and execution from developer workstations. This remote backend pattern lets the Terraform command-line interface (CLI) and VCS repositories send changes to a central workspace, where the workspace depends on the connected repository for triggers and on the remote backend for consistent state.

Teams that need reproducible development, test, and production environments combine remote execution with continuous integration and continuous deployment (CI/CD) using Cloud Build and Infrastructure Manager. A complete lifecycle moves code from commit to production in order:

1. Trigger on commit: A commit to version control fires a Cloud Build trigger or Terraform Cloud VCS-triggered plan.
2. Validate and plan: The pipeline automatically runs Terraform validate and plan commands to test syntax and preview changes.
3. Review and store: The reviewed configuration is stored as a versioned artifact with a history that can be audited and reverted.
4. Promote across environments: Distinct dev, staging, and prod configurations in version control are promoted automatically after validation.

Because configurations are versioned artifacts with separate environment definitions, the same desired state can be deployed multiple times to create consistent environments and recover reliably from failure.

2.3Modules and Templates for Complex Deployments

Modules and templates for Terraform on Google Cloud package opinionated resource sets into reusable building blocks that compose into complete environments. A module defines resources such as a BigQuery dataset, Cloud Storage bucket, or Cloud SQL database once, and modules combine into end-to-end solutions that deploy complete Google Cloud projects. Google-curated collections include Terraform blueprints for Google Cloud and Cluster Toolkit example blueprints, which provide ready-to-use Terraform templates for high-performance computing (HPC), artificial intelligence (AI), and machine learning (ML) clusters in Slurm or GKE.

Complex deployments use these templates to standardize specialized infrastructure without copying raw resource blocks. For AI and ML work, Terraform provisions GKE clusters with node pools optimized by workload (opens in a new tab), such as A100, H100, H200, or B200 graphics processing units (GPUs) for training and L4 GPUs for inference, plus serving endpoints and Cloud Storage buckets for data and artifacts. Teams customize the Cluster Toolkit code in version control and integrate it into CI/CD pipelines with Cloud Build to automate provisioning and updates. For data mesh use cases, a BigQuery authorization module grants roles such as roles/bigquery.dataEditor and defines authorized_views and authorized_datasets without granting direct reader access to the underlying datasets, and an authorized dataset is preferred when many views must share access because it automatically authorizes views created inside it.

To make modules consumable by domain teams with minimal Terraform experience, organizations publish them through Service Catalog while still allowing advanced users to compose custom solutions from the same modules. Templating handles environment differences by keeping shared module logic central and varying inputs, manifests, and Identity and Access Management (IAM) policies per environment through CI/CD automation.

2.4Secrets, Identity, and Compliance as Code

Secrets, identity, and compliance as code keep sensitive values and guardrails inside the same versioned workflow as infrastructure. Terraform Cloud securely stores secrets alongside remote state so applies do not depend on local credential files, and the secret-manager Terraform module creates one or more Secret Manager secrets and manages basic permissions for them. IAM policies and service accounts are also implemented with IaC, including BigQuery authorized views, service account creation, and role bindings that flow from the module to the target dataset, view, or project.

Compliance is enforced by making every module satisfy organizational guardrails by default and expressing those guardrails as code. The Google Cloud policy validation tool provides automated compliance verification of Terraform plans, and organizations continuously test platform-provided modules with the same guardrails used to promote changes into production. This dependency means the deployment pipeline depends on both the module registry and the policy checks, so insecure or noncompliant infrastructure is blocked before it reaches Google Cloud.

GCP-native Infrastructure as Code (IaC) management defines the desired state of infrastructure in versioned configuration files and then reconciles Google Cloud to that state in a safe, repeatable way. The same configuration can be reused to create reproducible development, test, and production environments, stored in source control, and promoted through continuous integration and continuous deployment (CI/CD) pipelines. Infrastructure Manager, Config Connector, and Terraform-based templates divide this work by operational model: managed Terraform orchestration, Kubernetes-native reconciliation, and reusable module authoring.

3.1Infrastructure Manager for Managed Terraform Orchestration

Infrastructure Manager (opens in a new tab) (Infra Manager) is a managed Google Cloud service that automates the deployment and management of infrastructure defined in Terraform. Terraform configuration flows from a source such as a Cloud Storage bucket, public Git repository, or local machine to Infra Manager, and Infra Manager provisions the Google Cloud resources, so teams deploy programmatically without maintaining a separate Terraform toolchain. Infra Manager depends on the Infrastructure Manager API config.googleapis.com and a service account with the roles/config.agent role plus resource-specific roles, such as roles/compute.networkAdmin to create a VPC network.

Infra Manager separates orchestration from authoring through previews and deployments. A preview verifies the resources planned to be provisioned before any change is applied, and a deployment then creates and tracks those resources in a specified location such as us-central1. Infra Manager integrates with CI/CD through tools like Cloud Build, where a commit trigger runs automated validation with terraform validate and terraform plan, stores configurations as versioned artifacts, and promotes distinct dev, staging, and prod configurations. Choose Infra Manager when Terraform is the source of truth but execution, state tracking, and deployment history should live as a managed Google Cloud control plane.

3.2Config Connector for Kubernetes-Native Management

Config Connector is a Kubernetes-native operational model that manages Google Cloud resources as Kubernetes objects. Desired state flows from YAML manifests to a Google Kubernetes Engine (GKE) cluster, where Config Connector reconciles Cloud SQL instances, Pub/Sub topics, Cloud Storage buckets, Agent Platform datasets, models and endpoints, Identity and Access Management (IAM) policies, and service accounts. Because application workloads and infrastructure objects share the same cluster API, application and infrastructure configuration are integrated in one declarative workflow.

Config Connector handles ongoing lifecycle management through continuous reconciliation and pipeline-driven updates. After manifests are applied, the controller keeps Google Cloud aligned with the declared state, while CI/CD pipelines automate updates and templating handles environment differences. Choose Config Connector when teams operate primarily through Kubernetes, want kubectl-style lifecycle operations for cloud resources, and need infrastructure changes to move with application releases rather than through a separate Terraform deployment service.

3.3Terraform Foundations and Reusable Templates

HashiCorp Terraform with the Terraform provider for Google Cloud (opens in a new tab) is the general foundation for defining Google Cloud and on-premises resources in human-readable configuration files that can be versioned, reused, and shared. Terraform provides a consistent workflow to provision and manage infrastructure throughout its lifecycle, including specialized resources such as GKE clusters with GPU-optimized node pools, Agent Platform endpoints with machine types and scaling policies, and Cloud Storage buckets for data and artifacts. Terraform CLI execution stores state near the configuration, while managed and SaaS options move execution and state into controlled environments.

Reusable templates accelerate that foundation for complex and AI/ML estates:

• Cluster Toolkit provides Google-curated Terraform blueprints and example blueprints for ready-to-use high-performance computing (HPC), AI, and ML clusters on Slurm or GKE, which teams customize and manage in version control.
• Cloud Build integrates Terraform code into CI/CD pipelines to automate provisioning and update workflows for those templates.
• Terraform Cloud runs Terraform in a stable, remote software as a service (SaaS) environment, securely stores state and secrets, connects to version control systems such as GitHub, GitLab, and Bitbucket to automatically trigger plans on commits, and exposes an API for existing workflows.
• Terraform Enterprise provides a self-hosted distribution of Terraform Cloud for organizations requiring full change management with Terraform inside their own boundary.

Choose direct Terraform and Cluster Toolkit blueprints when the problem is authoring modular, portable infrastructure, and choose Terraform Cloud or Terraform Enterprise when the requirement is organization-wide change control, remote execution, and centralized state and secrets.

3.4Choosing the Operational Model for Native IaC

Choosing the operational model means matching where configuration lives and what corrects drift to how the team operates. Infrastructure Manager centralizes Terraform execution in Google Cloud, Config Connector reconciles cloud resources from inside a GKE cluster, and Terraform with Cluster Toolkit keeps authoring portable while execution varies by environment.

A single estate can compose them: define a GPU cluster pattern once as a Cluster Toolkit Terraform module, orchestrate its environment promotions with Infrastructure Manager deployments and previews, and manage day-to-day dataset, bucket, topic, and IAM changes declaratively through Config Connector manifests in the workload cluster.

Key Facts

• GitOps stores declarative Terraform configurations in version control as the desired state, synchronizes changes through an automated pipeline, and rolls back by reverting the commit and re-synchronizing.
• Shift-left IaC validation by Security Command Center Premium or Enterprise tier checks a Terraform JSON plan against organization policies before resources are created and blocks promotion until violations are fixed.
• Cloud Asset Inventory provides eventual-consistent visibility for scheduled audits and delay-tolerant drift reporting, while immediate state checks before cutover must call the service API directly.
• Infrastructure Manager automates Terraform deployment as a managed Google Cloud service using previews and deployments, while Config Connector manages Google Cloud resources as Kubernetes objects reconciled by a GKE cluster controller.