Collaborative and reproducible notebook workflows on Google Cloud ensure that multiple team members can work together securely while maintaining consistent, version-controlled, and automated machine learning development processes. The primary services supporting these workflows are Vertex AI Workbench and Colab Enterprise, which provide notebook-based development environments with Google Cloud security and compliance capabilities.
Vertex AI Workbench is the recommended notebook environment for experimentation and development, providing access to all of Google's data and AI services in a reproducible way. Each data science team member should have their own Vertex AI Workbench instance, treating each instance as a virtual workspace. When team members work on multiple projects with different dependencies, multiple instances should be created per person. These instances can be stopped when not in use to manage costs. Vertex AI Workbench instances come with secure software and access patterns configured out of the box, and they support customization of network and Identity and Access Management (IAM) properties, as well as software customization through containers. Alternatively, Colab Enterprise offers a collaborative managed notebook environment that uses the security and compliance capabilities of Google Cloud.
Version controlling notebook artifacts requires storing different types of artifacts in appropriate locations. Source control repositories should contain Vertex AI Workbench instances, pipeline source code, preprocessing functions, model source code, model training packages, and serving functions. Experiments and ML metadata—including parameters, hyperparameters, metaparameters, metrics, dataset artifacts, model artifacts, and pipeline metadata—should be stored in Vertex ML Metadata. Trained models belong in the Model Registry, while pipeline containers, custom training environments, and custom prediction environments should reside in Artifact Registry. Deployed models live in Vertex AI Inference. Using a source control repository for pipeline definitions and training code enables version control of ML pipelines and custom components, while Artifact Registry stores, manages, and secures Docker container images without making them publicly visible.
Containerized kernels and custom images ensure environment reproducibility across development and production. Vertex AI Workbench supports custom containers, allowing teams to define consistent runtime environments with specific library versions and dependencies. The Deep Learning Containers provide Docker containers with pre-installed data science frameworks, libraries, and tools to deliver performance-optimized, consistent environments. When using custom training, teams can leverage pre-built containers for TensorFlow, Scikit-learn, XGBoost, and PyTorch, or use custom containers for any framework. This containerization approach decouples the execution environment from code runtime, provides reproducibility between development and production, and isolates each pipeline component so that each can have its own version of runtime, different languages, and different libraries.
Implementing CI/CD pipelines for notebook testing and deployment uses Cloud Build and Vertex AI Pipelines together. Cloud Build can import source from Artifact Registry, GitHub, or Bitbucket, execute builds according to specifications, and produce artifacts such as Docker containers or Python tar files. The build process executes a series of build steps defined in a cloudbuild.yaml configuration file, where each build step runs in a Docker container. Build triggers can execute configured build steps whenever changes are pushed to the source repository, with different routines triggered for development versus main branch commits. Configuration variable substitutions define environment variables at build time, capturing values like $COMMIT_SHA, $REPO_NAME, $BRANCH_NAME, $PROJECT_ID, and $BUILD_ID. The typical CI/CD workflow involves copying the source code repository to the Cloud Build runtime environment, running unit and integration tests, optionally running static code analysis, building Docker container images for each pipeline component and tagging them with $COMMIT_SHA, uploading images to Artifact Registry, updating component specification files with the created image URLs, compiling the pipeline workflow to produce the pipeline.json file, and uploading the pipeline file to Artifact Registry. Cloud Build can be configured to skip triggers when only documentation files or experimentation notebooks are modified.
Vertex AI Pipelines orchestrates the ML workflow by connecting notebook-based experimentation to automated production pipelines. Each pipeline component runs as a containerized task, taking input arguments and producing artifacts. The pipeline definition uses a Python domain-specific language where the topology is implicitly defined by connecting outputs of upstream steps to inputs of downstream steps. Pipeline components can execute multiple times in loops or conditionally based on evaluation thresholds. Vertex ML Metadata automatically stores pipeline parameters and artifacts generated during execution, enabling lineage and execution tracking. For notebook integration, the Vertex AI Workbench executor can run notebook files on a schedule or as one-time execution, with parameters specified for each run to make specific changes such as using a different dataset, changing learning rates, or specifying model versions.
Data governance and security controls protect sensitive machine learning data and notebook infrastructure across ingestion, storage, prototyping, and analysis. Google Cloud establishes this security framework by combining cryptographic key management, automated data discovery, policy-based column access controls, and comprehensive audit logging. Machine learning teams configure these controls across Vertex AI Workbench, BigQuery, Cloud Storage, and Dataplex to enforce organizational compliance and mitigate data exposure risks.
Data encryption protects information at rest and in transit across machine learning notebook workflows and underlying storage resources. Google Cloud encrypts customer content at rest by default using 256-bit Advanced Encryption Standard (AES-256) keys managed by Google. Organizations requiring direct authority over encryption keys can implement customer-managed encryption keys (CMEK) through Cloud Key Management Service (Cloud KMS) or Cloud Hardware Security Module (Cloud HSM), rotating keys by default every 30 days. When external key control is mandated, organizations configure Cloud External Key Manager (Cloud EKM) to store key material outside Google Cloud.
CMEK integration protects notebook disks, compute infrastructure, and managed data stores from unauthorized decryption. Enabling CMEK on Vertex AI Workbench instances, Compute Engine persistent disks, and BigQuery datasets requires specifying the Cloud KMS key at resource creation because CMEK cannot be enabled on existing disks or datasets retroactively. For client-side data protection, teams use the open-source Tink encryption library to encrypt data locally before transit using customer-supplied encryption keys (CSEK) or client-side keys. These client-encrypted payloads are compatible with BigQuery Authenticated Encryption with Associated Data (AEAD) functions, allowing authorized users to unwrap data encryption keys using key encryption keys stored securely in Cloud HSM.
Sensitive data protection strategies discover, classify, and redact sensitive personal information before it enters notebook workflows or model training pipelines. The Sensitive Data Protection service (formerly Cloud DLP) inspects structured and unstructured data using predefined infoTypes such as credit card numbers and personal identification details. Machine learning teams integrate Sensitive Data Protection with automated Dataflow pipelines to tokenize or mask confidential data flowing from Cloud Storage into BigQuery.
Data profiling automates continuous inspection across organizational data assets. The Data profiler scans BigQuery tables across organizations, folders, and projects to evaluate data risk and calculate sensitivity levels without manual data tagging. Identified sensitive attributes can undergo record transformations to obscure values before analysts query the tables in notebook environments. This automated profiling informs data governance teams about where sensitive assets reside so appropriate access boundaries and encryption policies are applied.
Data classification enforces granular read permissions on specific table columns using centralized taxonomies and policy tags. Administrators configure taxonomies within Dataplex (integrating Data Catalog capabilities) and assign policy tags—such as 1_Sensitive, 2_Private, or 3_Confidential—directly to individual BigQuery schema columns. To query plaintext values from tagged columns, a user must be explicitly granted the Fine-Grained Reader (roles/datacatalog.categoryFineGrainedReader) role on the corresponding policy tag.
Granular security mechanisms allow non-privileged notebook users to analyze datasets without accessing raw sensitive values. Dynamic data masking hides sensitive fields at query execution time based on the user's IAM role while still allowing queries over non-sensitive attributes in the same table. For field-level cryptographic isolation, BigQuery column-level encryption uses AEAD SQL functions to create keysets that encrypt individual cell values. Users querying column-level encrypted data require dual access: read permissions on the BigQuery table and read access to the decryption keys managed in Cloud KMS.
Network boundaries and audit logs safeguard notebook environments against data exfiltration and track all interactions with sensitive data. VPC Service Controls establishes security perimeters around Vertex AI Workbench instances, Cloud Storage buckets, and BigQuery datasets to restrict ingress and egress network communications. To access data securely from external environments, traffic travels over dedicated Cloud Interconnect or Cloud VPN tunnels with defined access levels. User access to sensitive resources is managed by granting the Service Account Token Creator (roles/iam.serviceAccountTokenCreator) role to authorized users, allowing short-term service account impersonation without downloading permanent service account keys.
Access auditing provides operational visibility and compliance tracking across all notebook activities. Cloud Logging automatically records Admin Activity audit logs for resource configuration changes across all services, while Data Access audit logs must be enabled for services to record user read and write operations on sensitive data assets. Audit log entries capture caller identity, timestamps, and target resource metadata, which can be visualized in Security Command Center or exported to BigQuery for analysis. Administrators enforce organizational policies to restrict public IP addresses on notebook instances, require automated upgrades, and alert on unauthorized access attempts.
Implementing Identity and Access Management (IAM) for notebook resources involves configuring precise access controls for Vertex AI Workbench and Colab Enterprise instances to secure the machine learning development environment. This process applies the principle of least privilege by defining who can access what resources and under which conditions, ensuring that data scientists and engineers can collaborate safely without exposing sensitive data or systems.
A service account is a non-human identity used by your notebook instance to authenticate and access other Google Cloud services. You must create a user-managed service account for each Vertex AI Workbench instance instead of using the broad permissions of the default Compute Engine service account. Assign this service account only the IAM roles it strictly needs to function, such as Vertex AI User (roles/aiplatform.user) for accessing Vertex AI services and Storage Admin (roles/storage.admin) for managing Cloud Storage buckets. This practice limits the potential impact if the service account's credentials are compromised and ensures the notebook can only perform its intended tasks.
Human users, such as data scientists, access notebook instances through their own Google Cloud identities. You control their access by granting IAM roles at the project, folder, or organization level. For example, granting the Vertex AI User role allows a user to open and run notebooks. To create more granular, context-aware rules, you use conditional IAM policies. These policies can restrict access based on attributes like the user's IP address, the time of day, or the specific resource being accessed, adding an extra layer of security beyond basic role assignments.
VPC Service Controls establish a security perimeter around Google Cloud services, including Vertex AI and Cloud Storage, to prevent data exfiltration. You define a service perimeter at the organization level to restrict resource access to only those requests originating from within your Virtual Private Cloud (VPC) networks. To enforce this, you configure Vertex AI Workbench instances with private IP addresses only, disabling their public internet interfaces. This setup ensures all traffic to and from the notebook, including calls to Vertex AI APIs, flows through your private VPC network and stays within the trusted boundary of your service perimeter.
StudyCerts
StudyCerts$COMMIT_SHA, and uploading pipeline files to Artifact Registry.Vertex AI Workbench is recommended for experimentation and development when each team member needs their own instance with customizable network and IAM properties. Colab Enterprise is a collaborative managed notebook environment that uses Google Cloud security and compliance capabilities, suitable for teams that want a more managed, shared experience without managing individual instances.
CMEK (customer-managed encryption keys) are keys managed by the organization through Cloud KMS that Google Cloud uses to encrypt data at rest; they protect notebook disks, managed data stores, and BigQuery datasets. CSEK (customer-supplied encryption keys) are keys provided by the user for client-side encryption using the Tink library before data is sent to Google Cloud, and they are compatible with BigQuery AEAD functions for decryption.
Use Dataplex to create a taxonomy with policy tags (e.g., 3_Confidential) and assign the tag to the specific column. Then grant the user the Fine-Grained Reader role (roles/datacatalog.categoryFineGrainedReader) on that policy tag. Without that role, the user will see masked values or no data in that column, depending on whether dynamic data masking is also configured.
Professional Machine Learning Engineer
Prepare and test your skills
Prepare and test your skills