Professional Data Engineer: Google Cloud Data Processing and Analytics
Build and operate the data processing systems behind analytics and ML on Google Cloud, including BigQuery, Dataflow, and Pub/Sub, and track your readiness for the Professional Data Engineer exam.
Advanced
Practice Tests
Individual Plan Includes:
- Access to all Practice Tests
- Access to 1,353 practice questions
- Detailed explanations with cited sources
- Performance tracking
- Individual progress tracking
- AI-powered study recommendations
- Follow-up AI-powered conversations
- Time-based practice
- Unlimited attempts while subscribed
30-Day Money-Back Guarantee
Not sure where you stand?
Try a few exam-style questions free and see your result at the end.
Start the free trialAdvanced
Practice Tests
Individual Plan Includes:
- Access to all Practice Tests
- Access to 1,353 practice questions
- Detailed explanations with cited sources
- Performance tracking
- Individual progress tracking
- AI-powered study recommendations
- Follow-up AI-powered conversations
- Time-based practice
- Unlimited attempts while subscribed
30-Day Money-Back Guarantee
About the Professional Data Engineer exam
The Google Cloud Professional Data Engineer certification validates that you can design, build, secure, and monitor data processing systems that make data usable for analysis and machine learning. The exam covers five Domains: designing data processing systems; ingesting and processing data; storing data; preparing and using data for analysis; and maintaining and automating data workloads. Expect questions on choosing and combining services such as BigQuery, Dataflow, Pub/Sub, Dataproc, Cloud Storage, and Bigtable.
Professional Data Engineer at a glance
- Exam domains
- 5
- Skill level
- Advanced
- Practice questions
- 1,353
Professional Data Engineer exam domains
Exam Study Guide
Designing for reliability and fidelity
Preparing and cleaning data on Google Cloud is the work of turning raw source data into trusted tables and features with the right transformation service and validated quality. Dataform runs SQL transformations inside BigQuery, Dataflow runs Apache Beam code for batch and streaming, and Cloud Data Fusion runs managed integration with reusable plugins. Large language models (LLMs) accelerate this work by drafting SQL, Python, and quality rules, while BigQuery pipelines, validation checks, and human review keep generated code safe to deploy.
Continue with this study guide: Designing for reliability and fidelity
Designing data migrations
Analyzing current needs and creating a migration plan is the assessment that connects what stakeholders want to what the current warehouse can do. The work inventories data flows, systems, users, and compatibility, then defines a phased path to a desired state where upstream processes write directly to BigQuery and downstream processes read from BigQuery. Every use case moves through offload before full migration, with validation before legacy tables are deprecated.
Designing for security and compliance
Identity and Access Management (IAM) for data workloads is the system that binds every principal to only the minimum role needed on a specific resource. A principal is a user, group, or service account, a role is a collection of permissions, and access flows from the principal through the role to the dataset, bucket, or project. Organization and project roles control the ability to run jobs or manage projects, while dataset and bucket roles control the ability to read or modify data, and permissions inherit downward unless a lower-level grant narrows them.
Continue with this study guide: Designing for security and compliance
Designing for flexibility and portability
Mapping current and future business requirements to the architecture is the practice of turning business goals, growth forecasts, and limits into a data design that can adopt new services without rework or downtime. The assessment weighs scalability, cost, and operational impact on Google Cloud, while the plan keeps storage and compute separable, adds new sources alongside old pipelines, and keeps reports working during change. A flexible design stores data and runs computation on separate layers that are colocated in the same zone and region for performance.
Continue with this study guide: Designing for flexibility and portability
Exam Study Guide
Building the pipelines
Data cleansing is the process of inspecting, validating, and correcting datasets in Google Cloud before analytical use. Knowledge Catalog centralizes discovery, monitoring, and governance of data and AI artifacts, while Cloud Dataflow filters records in flight and BigQuery standardizes data after load. Data flows from collection through storage and transformation into analytical consumption, with governance monitoring flows, transformations, and access alongside that path.
Deploying and operationalizing the pipelines
Job automation and orchestration (e.g., Cloud Composer and Workflows) coordinates data pipelines across Google Cloud services without manual intervention. Google Cloud Workflows handles serverless orchestration of HTTP-based services with low latency, while Managed Service for Apache Airflow (Managed Airflow), formerly Cloud Composer, handles batch extract, transform, and load (ETL) and extract, load, and transform (ELT) pipelines defined as Python Directed Acyclic Graphs (DAGs). Security for both depends on user-managed service accounts with narrow Identity and Access Management (IAM) roles, and reliability depends on managed infrastructure with Cloud Logging and Cloud Monitoring.
Continue with this study guide: Deploying and operationalizing the pipelines
Planning the data pipelines
Defining data sources and sinks is the work of choosing where pipeline data comes from, where results land, and how both ends stay secure, fast, and correctly typed. A source can be an internal bucket, a streaming topic, or an external on-premises database, while a sink can be a lake, warehouse, or messaging topic that feeds the next stage. The design must control identity, network path, encryption, throughput, and schema together.
Exam Study Guide
Selecting storage systems
Analyzing data access patterns means matching how data is written, read, and retained to the storage layout that scans the least data and spreads load evenly. A workload is described by data type, current and future capacity, whether capacity scales automatically, whether data is accessed from outside Google Cloud, and whether multiple clients read or write at the same time. Read-write behavior typically falls into frequent writes and reads, frequent writes with occasional reads, occasional writes and reads, or occasional writes with frequent reads.
Designing for a data platform
Building a data platform on Google Cloud means selecting storage, processing, and governance tools to match cost, performance, freshness, and compliance requirements. BigQuery serves as the governed analytical warehouse, Cloud Storage serves as low-cost raw and archival object storage, and data mesh with Dataplex Knowledge Catalog provides federated governance across domains. Data flows from source systems through ingestion and processing into serving tables, while catalog, security, and lifecycle controls apply at every stage.
Continue with this study guide: Designing for a data platform
Using a data lake
Managing the lake means enforcing unified discovery, access, and cost controls across Cloud Storage and BigQuery from one governance plane. Identity and Access Management (IAM) sets the coarse boundary for who can act, while query-time controls, encryption, catalog tagging, and network perimeters decide what data is visible. Cloud Storage holds raw objects and open-format files, BigQuery provides SQL compute and fine-grained enforcement, and the Lakehouse runtime catalog centralizes metadata so engines share one source of truth.
Planning for using a data warehouse
Designing the data model in BigQuery is the combined choice of logical schema shape, history strategy, and physical storage layout. Storage is separated from compute: data is stored compressed in columnar format in Colossus, execution is performed by Dremel workers, and Capacitor is the compressed columnar format that lets queries read only needed columns. The optimizer gathers column statistics during load, persists them during Capacitor encoding, and uses them to prune data and pick efficient runtime algorithms. Compute is allocated as slots, units of CPU and RAM shared by fair scheduling, so reducing bytes scanned lowers both on-demand cost and slot contention.
Continue with this study guide: Planning for using a data warehouse
Exam Study Guide
Preparing data for AI and ML
Preparing data for feature engineering, training, and serving is the process of turning raw operational data into clean, reusable features that train models and serve predictions consistently. Raw data flows from operational sources into ingestion tools, then through transformation logic, and finally into training tables or a central feature store. On Google Cloud, BigQuery handles tabular SQL transformation, Dataflow handles batch and streaming transformation, and BigQuery ML applies automatic preprocessing during training.
Continue with this study guide: Preparing data for AI and ML
Preparing data for visualization
Connecting visualization tools to Google Cloud data means establishing secure, fast, and scalable paths from dashboards to BigQuery. Traffic flows from the business intelligence (BI) tool to the BigQuery API, then to in-memory acceleration when data is cached, while uncached stages fall back to standard slots without failing the query. Identity flows from OAuth 2.0 tokens, service accounts, or Application Default Credentials to BigQuery, and federated connections extend queries outward to data that remains in external sources.
Continue with this study guide: Preparing data for visualization
Sharing data
Defining rules to share data is the practice of letting authorized collaborators query data without copying it into uncontrolled environments while recording every access. BigQuery sharing (Analytics Hub) and BigQuery data clean rooms provide the controlled framework for cross-organization collaboration, while Cloud Storage and Looker Action Hub extend sharing to objects and dashboards. Cloud Audit Logs, Cloud Logging, and Cloud Monitoring provide the audit trail that proves who accessed data and whether sharing rules were followed.
Exam Study Guide
Maintaining awareness of failures and mitigating impact
Designing for fault tolerance and managed restarts means keeping progress and data outside ephemeral compute so a failure resumes from the last good point. Cloud Storage holds backups, exports, and transaction logs as replayable sources, Compute Engine persistent disks and snapshots hold VM and filesystem state, and managed databases hold replicated table state. Together, checkpoints, durable copies, and replay steps let a pipeline restart without starting from the beginning.
Continue with this study guide: Maintaining awareness of failures and mitigating impact
Designing automation and repeatability
A directed acyclic graph (DAG) is a Python workflow file that defines tasks and the dependencies between them in an order that never loops back. The Managed Service for Apache Airflow (formerly Cloud Composer) evaluates the whole file to build the execution graph, so every dependency must move forward from start to end without cycles. A DAG definition is the container that binds workflow identity, owner, start date, and schedule together, for example with
with DAG(dag_id="controller_dag_to_trigger_other_dags", default_args={"owner": "airflow"}, start_date=days_ago(1), schedule_interval="@once") as dag:. Tasks such as DummyOperator are created inside that context with a unique task_id, and the bitshift operator >> connects them so control flows from the upstream task on the left to the downstream task on the right.Continue with this study guide: Designing automation and repeatability
Optimizing resources
Minimizing costs per required business need for data means sizing compute and storage to actual demand and letting capacity expand and contract automatically. Elastic compute comes from Compute Engine, Cloud Run, Dataflow, Managed Service for Apache Spark, and BigQuery, while minimize costs guardrails come from Cloud Storage, Persistent Disk, and Cloud Monitoring with billing alerts. The approach optimize continuously by right-sizing, autoscaling, tiering storage, colocating compute with data, and pairing technical controls with budgets and discounted pricing.
Organizing workloads based on business requirements
Capacity management in BigQuery is the practice of buying and assigning dedicated query compute to meet performance, cost, and business requirements. Compute is measured in slots, which are virtual compute units that execute SQL queries and other jobs such as load, export, query, and copy jobs. BigQuery separates compute, memory, and storage into independently managed services, so choosing slot capacity is separate from storing data.
Continue with this study guide: Organizing workloads based on business requirements
Monitoring and troubleshooting processes
Observability of data processes is the practice of using metrics, logs, traces, and admin views to explain performance, cost, and security behavior. BigQuery slot usage, Dataflow stage health, and audit activity flow into different tools that answer different questions. Together they let operators separate slow query design from contention for shared capacity and separate code failures from quota or permission blocks.
Continue with this study guide: Monitoring and troubleshooting processes
Prerequisites
- Associate Data PractitionerRecommended
Google Cloud · Fundamental