Prepare and test your skills
Prepare and test your skills
Designing collaborative and reproducible notebook workflows means a team can prototype together in notebooks and promote the same code, data, and environment to production without drift. On Google Cloud, Vertex AI Workbench (opens in a new tab) provides per-user development instances, Colab Enterprise (opens in a new tab) provides a managed collaborative option, and Vertex ML Metadata (opens in a new tab), Git-based version control, container images, Cloud Build (opens in a new tab), and Vertex AI Pipelines (opens in a new tab) connect prototyping to tracked, automated deployment. Traffic flows from the notebook environment to BigQuery, Cloud Storage, Vertex AI Feature Store, and Vertex AI training and serving services.
Collaborative notebook environment options divide by control versus managed collaboration. Vertex AI Workbench instances are user-managed notebook environments for writing code, starting jobs, running queries, and checking status with direct access to Google Cloud data and AI services. Colab Enterprise is a collaborative managed notebook environment that inherits Google Cloud security and compliance controls.
| Environment | Control model | Choose when |
|---|---|---|
| Vertex AI Workbench | Per-user instance customizable for VPC, IAM, and container image | The team needs isolated virtual workspaces with custom dependencies and network controls |
| Colab Enterprise | Managed collaborative environment with Google Cloud security and compliance | The team needs shared, low-operations collaboration without managing instances |
The notebook identity depends on its attached service account and VPC configuration, so cross-project data access is governed by Identity and Access Management (IAM) rather than by copying data into the notebook. Both options provide a secure default for software and access patterns, while Workbench instances are commonly customized for network, IAM, and software through a container.
Team workspace isolation starts with creating a Vertex AI Workbench instance for each data scientist. When one person works on multiple projects with different dependencies, each project gets its own instance treated as a virtual workspace, and instances are stopped when not in use to control cost and reduce attack surface. The simplest access boundary stores raw data and Vertex AI datasets and models in the same Google Cloud project, while multi-project designs use IAM for corporate cross-project access control.
Shared development standards keep those isolated workspaces compatible. Teams use the Vertex AI SDK for Python for end-to-end workflows across PyTorch, TensorFlow, XGBoost, and scikit-learn, so notebook code calls the same dataset, training, and endpoint objects that pipelines will call later. For analysis, Vertex AI TensorBoard tracks and compares experiment metrics such as loss and accuracy over time, visualizes model graphs, and projects embeddings, while Vertex AI Experiments links parameters, metrics, and dataset and model artifacts through Vertex ML Metadata. Small datasets can train inside the Workbench instance, but scheduled training or training on new data arrivals moves to Vertex AI managed training even for small data to make the run repeatable.
Version control and lineage make notebook work auditable from data source to deployed model. Vertex ML Metadata records lineage for models, datasets, and artifacts, logging data sources, transformations, model parameters, and experiment results with encryption at rest. Git provides version control for notebook code and pipeline definitions, and integration with robust continuous integration and continuous delivery (CI/CD) pipelines automates change tracking from commit to deployment.
Lineage operates at two scopes that compose into full provenance:
Exam tip: Vertex ML Metadata tracks what produced an artifact — data source, transformation, parameters, and pipeline lineage — while Cloud Audit Logs track who called which Vertex AI API and when.
Reproducible notebook environments depend on containerized kernels and images rather than manually installed libraries. A Vertex AI Workbench instance associates software through a container, so network and IAM customize the Google Cloud properties while the container pins the Python packages, system libraries, and framework versions. Validated prebuilt containers cover common training and serving cases, and custom containers cover cases where dependencies or serving signatures differ.
Isolation by task prevents a single dependency set from becoming a point of failure. Feature engineering, training or fine-tuning, and inference each use a separate container when they require different dependency management, which also lets engineers control and customize each stage independently. Custom ML packages and vetted dependencies are stored in a private Artifact Registry repository, and Artifact Analysis scans those dependencies and images for known vulnerabilities. Only approved library versions move forward, so the image that ran in the notebook is the same logical image that builds and deploys.
CI/CD promotion moves tested notebook code into Vertex AI Pipelines through Cloud Build with security and versioning enforced at each handoff. Vertex AI Pipelines acts as the central orchestrator for modular components that process data, train, evaluate, and deploy, with orchestration to Dataflow, Vertex AI custom training, Model Registry, and Vertex AI endpoints. Pipeline definitions are written programmatically with the Kubeflow Pipelines (KFP) SDK or TensorFlow Extended (TFX) SDK, which keeps the pipeline as versioned code alongside the notebook.
An automated promotion flow preserves order and traceability:
The pipeline depends on least-privilege execution identities and network boundaries. Dedicated service accounts grant only minimum IAM permissions, such as read from one Cloud Storage bucket and write to Model Registry, and IAM Conditions add attribute-based control so a service account can trigger a pipeline only when the request originates from a trusted Cloud Build trigger. Supply-chain maturity follows the Supply-chain Levels for Software Artifacts (SLSA) progression from source control and automated builds through provenance generation, artifact signing, vulnerability management, and hardened deployment.
Identity and Access Management (IAM) (opens in a new tab) for notebook resources is the system that decides which human users and service accounts can create, open, and use Vertex AI Workbench (opens in a new tab) instances and Colab Enterprise (opens in a new tab) runtimes, and what data those identities can reach once inside. A secure notebook design separates human access from workload access, grants only least-privilege roles, adds conditions and boundaries to narrow those roles, and then enforces network perimeters around the APIs and data. Traffic flows from the user or notebook service account to Google Cloud APIs and to data planes such as Cloud Storage and BigQuery, and each hop is checked against IAM, context, and perimeter policy.
Least-privilege notebook access means every user has only the minimum permissions needed for their role, such as no-access, read-only, or write, to prevent data theft, preserve model integrity, and limit the surface for accidents or attacks. The simplest isolation is to store raw data and Vertex AI resources and artifacts such as datasets and models in the same Google Cloud project, while cross-project work uses corporate cross-project access control with IAM. For collaboration, teams create a Vertex AI Workbench instance for each member and treat each instance as a virtual workspace, creating multiple instances when one person works on projects with different dependencies and stopping instances when they are not used.
Colab Enterprise is the alternative collaborative managed notebook environment that inherits Google Cloud security and compliance controls, while Vertex AI Workbench instances provide a secure set of software and access patterns that teams commonly customize for network, IAM, and container image. In both cases, the notebook control plane manages the instance lifecycle, but data access depends on the attached identities and the VPC and perimeter configuration around the instance. Choose per-member isolated instances when experiments need different packages, data boundaries, or cost control, and choose shared governance controls when teams must audit and reproduce work across members.
A notebook service account is the workload identity that code running in Vertex AI Workbench or Colab Enterprise uses to call Vertex AI, read training data, and write artifacts, and it must be separate from the human user identity. Teams create dedicated, least-privilege service accounts for each workload, so a training pipeline service account can read data from only a specific Cloud Storage bucket and write model artifacts to Model Registry, with no broader project rights. Service accounts used for Vertex AI custom jobs, pipelines, and deployed models must carry only the minimum required IAM permissions, and the same principle applies to the notebook execution identity that launches jobs and queries.
Mapping broad administrative roles to a notebook service account violates least privilege even when tutorials use them for setup, so production notebooks narrow permissions to specific resources and actions. Teams restrict the use of downloaded service account keys because key material persists in gcloud CLI configuration files, local filesystems, and code repositories where an attacker can copy it, and they enforce the constraints/iam.disableServiceAccountKeyCreation organization policy while using more secure authentication alternatives. Production controls block service account key creation and upload, require audit logging for Vertex AI at the organization level, and review Privileged Access Manager audit logs when temporary elevation was used.
User permissions for notebooks combine Vertex AI access control with IAM predefined or custom roles to create repeatable access patterns for different user groups, assigned to users, groups, or service accounts based on project needs, access patterns, or resource tags. IAM Conditions add attribute-based rules to those bindings to manage granular access to data, models, and model configurations such as code, resource settings, and hyperparameters, for example allowing a service account to trigger a Vertex AI pipeline only if the request originates from a trusted Cloud Build trigger. Application-level controls then narrow data-plane access further without changing the notebook IAM binding itself.
Application-level granularity differs by data service:
| Data plane | Control point | Granular capability |
|---|---|---|
| Cloud Storage | Bucket and managed folder IAM policies | Isolate notebook reads and writes to specific buckets or folders |
| BigQuery | Dataset and resource IAM plus row-level and column-level security | Restrict which datasets, rows, and columns a notebook identity can query |
Beyond role bindings, layered limits apply even when a role would otherwise allow access:
The dependency order matters: first the principal must have an IAM role on the notebook and target resource, then IAM Conditions, PAB policies, and IAP or access-level checks must also allow the request, and a deny at any layer blocks the flow from notebook to data.
Network controls for notebooks place a trust boundary around Vertex AI and data APIs so compromised credentials outside the boundary cannot exfiltrate data. VPC Service Controls defines a service perimeter around AI and ML resources and sensitive data, includes Vertex AI and storage APIs in that perimeter, and controls data flow with configured ingress and egress rules for the perimeter. Traffic that originates within the defined perimeter can reach supported resources, while requests to restricted services from attacker-controlled endpoints outside the environment are blocked, which limits the usefulness of stolen tokens.
Private connectivity and hardening remove public exposure from the notebook data path and execution host. Teams use private IP addresses for training jobs and notebook-adjacent compute for increased network security and lower latency than public IP addresses, and they set up a Private Service Connect interface so custom jobs communicate privately with other authorized Google Cloud services without public traffic. Training and prototyping are isolated further with dedicated projects or VPCs, deny-all firewall policies that explicitly allow only required traffic, and hardened hosts using Shielded VM or Confidential Computing, secure boot, and disabled public IPs on Workbench notebooks and instances.
Exam tip: VPC Service Controls is resource-centric and applies to user and service account identities, while Chrome Enterprise Premium is user-centric and applies only to user identities in the organization; use them together to block compromised credentials from outside machines.
Data governance and security controls in notebook environments combine encryption, loss prevention, classification, and auditing to protect data that notebooks read, transform, and write. Data flows from ingestion sources such as Cloud Storage (opens in a new tab) or Pub/Sub through a Dataflow (opens in a new tab) de-identification pipeline into BigQuery (opens in a new tab), while Knowledge Catalog (opens in a new tab) governs lineage and quality and Cloud Logging (opens in a new tab) and Cloud Audit Logs record access. VPC Service Controls perimeters, firewall rules, and customer-managed keys enclose that flow so notebooks can explore trusted data without exfiltrating sensitive values.
Encryption policy in notebook environments starts with default Google encryption and adds customer control with customer-managed encryption keys (CMEKs) in Cloud Key Management Service (Cloud KMS). When CMEKs are used, CMEK-integrated services encrypt data at rest on behalf of the notebook workload, while HTTPS protects API calls to AI and ML services in transit. The CMEK must be in the same location as the protected resource, rotates by default every 30 days, and can be enforced with the gcp.restrictNonCmekServices organization policy.
The documentation identifies these CMEK-integrated services for notebook-related data:
For stronger control, keys can be HSM-protected with Cloud HSM or held outside Google Cloud with Cloud External Key Manager (Cloud EKM). Customer-supplied encryption keys (CSEKs) are not recommended because Cloud EKM supports more services with higher availability; if external keys are used, the customer owns rotation and availability. In transit, traffic flows from notebook clients and pipelines to Google APIs over HTTPS, enforced with HTTPS load balancers, while traffic between an on-premises network and Google Cloud flows over Cloud VPN or Dedicated Interconnect with allowlisted IPs in access-level rules.
Sensitive Data Protection, formerly Cloud DLP, provides discovery, classification, and de-identification that prevents notebooks and logs from exposing raw sensitive data. Data profiler scans BigQuery tables across the organization, folder, or project and writes data profiles to BigQuery with predicted information types (infoTypes), data risk, and sensitivity levels. Sensitive data discovery for Agent Platform scans Agent Platform datasets at the same scopes, and inspection results can be sent from BigQuery into Knowledge Catalog as aspects.
De-identification runs during ingestion before notebooks read the data. Data flows from Cloud Storage through a Dataflow pipeline that calls Sensitive Data Protection to tokenize confidential fields and then loads the result into the BigQuery warehouse. Structured data uses field-based record transformations, such as detecting credit card numbers and PINs, while unstructured data uses infoTypes detectors. Ingestion uses distinct keys managed by Cloud HSM:
Loss prevention also applies to operations. Logging combined with Sensitive Data Protection controls what pipeline components log and hides sensitive values, and notebooks must not log personally identifiable information (PII) in experiment logs or alternate log servers. Sensitive Data Protection with Model Armor adds discovery and protection for intellectual property in large language model (LLM) interactions, while Cloud Storage lifecycle controls limit prolonged retention of staged data.
BigQuery column-level security lets notebook queries read a table while sensitive columns remain hidden unless explicitly granted. Access to a tagged column depends on both Identity and Access Management (IAM) on the table and Data Catalog policy tags on the column, and a data analyst must hold the Fine-Grained Reader role to read protected columns. Anything without a tag remains available to anyone with warehouse access, so classification must be complete before notebooks are shared across teams.
The secured-warehouse taxonomy creates a hierarchy where higher access implies lower access:
3_Confidential for very sensitive data such as credit card numbers; holders can also read 2_Private and 1_Sensitive columns2_Private for sensitive PII such as first name; holders can also read 1_Sensitive columns but not 3_Confidential columns1_Sensitive for non-public data such as credit limit; holders cannot read 2_Private or 3_Confidential columnsTwo related controls extend that model without replacing it. Dynamic data masking applies a masking rule at query runtime so existing queries automatically mask the column unless the caller has Fine-Grained Reader, which supports sharing at scale. Column-level encryption with authenticated encryption with associated data (AEAD) encrypts only selected values inside BigQuery using keysets that support rotation; it creates dual-access control because the user needs permission to both the table and the encryption key to read cleartext. For external sources, client-side encryption with the Tink library wraps a locally generated data encryption key (DEK) with a symmetric CMEK key encryption key (KEK) in Cloud HSM, stores the encrypted data and the wrapped DEK in separate BigQuery tables, and decrypts with AEAD functions only when analysis requires re-identification.
Exam tip: Policy-tag security hides a column until Fine-Grained Reader is granted, while AEAD column-level encryption hides values until both table access and key access are granted.
Knowledge Catalog provides the organization-wide governance layer that lets notebook teams discover, monitor, and reuse trusted data and AI artifacts. The platform centrally manages data lineage, including BigQuery column-level lineage, data quality checks and data profiles, discovery across data marts, feature metadata and model artifacts, and a business glossary with aspects and aspect types for added context. Governance extends across BigLake and open-format tables such as Iceberg and Delta Lake, integrates lakes and warehouses through Dataproc Metastore and Lakehouse runtime catalog, and supports a data mesh where ownership is decentralized to domain teams without losing central policy.
Auditing and perimeters bind notebooks to that governed data. Data flows from governed tables into Gemini Enterprise Agent Platform Workbench only inside a VPC Service Controls perimeter with explicit ingress and egress rules, deny-by-default firewall rules, and context-aware evaluation on each authentication through Access Context Manager and Chrome Enterprise Premium. Cloud Logging streams live pipeline logs, Cloud Monitoring collects metrics and alerts on pipeline and feature drift, Agent Platform Feature Store schedules monitoring of features and feature groups, and Cloud Audit Logs retain API audit trails including Privileged Access Manager elevation. Least-privilege IAM, IAM Conditions, principal access boundary (PAB) policies, and BigQuery row-level security, sharing through Analytics Hub, and data clean rooms then restrict which notebook identity can move or re-share each governed slice.
constraints/iam.disableServiceAccountKeyCreation organization policy.3_Confidential, 2_Private, and 1_Sensitive taxonomy controlling hierarchical read access.Use Vertex AI Workbench when the team needs isolated virtual workspaces with custom dependencies and network controls through per-user instances customizable for VPC, Identity and Access Management (IAM), and container image. Use Colab Enterprise when the team needs shared, low-operations collaboration without managing instances in a managed collaborative environment that inherits Google Cloud security and compliance controls.
Vertex ML Metadata tracks what produced an artifact, including data sources, transformations, model parameters, and pipeline lineage across data preparation, training, evaluation, and deployment. Cloud Audit Logs for Vertex AI track who called which Vertex AI API and when, including API calls and administrative actions showing who accessed or modified data and models.
BigQuery column-level security requires both Identity and Access Management (IAM) permission on the table and Data Catalog policy tags on the column, and a user must hold the Fine-Grained Reader role to read protected columns. The secured-warehouse taxonomy uses 3_Confidential for very sensitive data such as credit card numbers, 2_Private for sensitive personally identifiable information such as first name, and 1_Sensitive for non-public data such as credit limit, where higher access implies lower access.
Service accounts for notebook and pipeline workloads must be dedicated, least-privilege identities separate from human user identities, carrying only minimum required permissions such as reading from a specific Cloud Storage bucket and writing model artifacts to Model Registry. Teams must restrict the use of downloaded service account keys and enforce the constraints/iam.disableServiceAccountKeyCreation organization policy while using more secure authentication alternatives.
Last updated