professional-cloud-data-engineer
Prepare and test your skills
Prepare and test your skills
Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.
A data engineering team is building a scalable feature preprocessing pipeline using Apache Beam on Google Cloud Dataflow. The pipeline must preprocess numerical features by scaling them to a fixed range [min_value, max_value] and convert categorical text columns into integer indices for model training.
To prevent training-serving skew, the exact transformation parameters (such as the computed feature boundaries and vocabulary mappings) generated during the initial training run must be deterministically preserved and reused during subsequent model evaluation and batch inference pipelines.
Which approach should the team implement in Apache Beam to meet these requirements?
Apache Beam's MLTransform is a specialized framework designed to preprocess and transform datasets for machine learning workflows at scale. It natively integrates feature engineering operations—such as numerical scaling, text tokenization, vocabulary generation, and embedding creation—directly into Google Cloud Dataflow pipelines.
StudyCerts
StudyCertsScaleByMinMax(columns=[...], min_value=min_value, max_value=max_value), MLTransform computes full-dataset minimum and maximum statistics across distributed workers and scales feature values accordingly.MLTransform computes unique token vocabularies across the training dataset and maps them to distinct integer indices.write_artifact_location saves the computed statistics, vocabularies, and transformation metadata to Cloud Storage. Subsequent evaluation or inference pipelines can simply reference read_artifact_location, applying the exact same transformation parameters without re-computing statistics on new batches.ScaleByMinMax, ScaleToZScore), categorical encoding (ComputeAndApplyVocabulary), and text embeddings in a unified interface.MLTransform abstracts the complex multi-pass distributed aggregations required for feature engineering. Reusing artifacts via read_artifact_location ensures that incoming test and inference data are normalized against the exact distribution parameters established during model training, perfectly preventing data leakage and training-serving skew.
Keep the momentum going with these hand-picked practice scenarios
Want more questions like this?
Get a free certification question every week.