Scalable Feature Transformation Pipeline Mltransform

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

A data engineering team is building a scalable feature preprocessing pipeline using Apache Beam on Google Cloud Dataflow. The pipeline must preprocess numerical features by scaling them to a fixed range [min_value, max_value] and convert categorical text columns into integer indices for model training.

To prevent training-serving skew, the exact transformation parameters (such as the computed feature boundaries and vocabulary mappings) generated during the initial training run must be deterministically preserved and reused during subsequent model evaluation and batch inference pipelines.

Which approach should the team implement in Apache Beam to meet these requirements?

Apply Apache Beam's MLTransform with ScaleByMinMax and vocabulary transforms, configuring write_artifact_location during the training data pipeline and read_artifact_location during evaluation and inference pipelines.

Deploy a Google-provided Dataflow template and use a JavaScript User-Defined Function (UDF) to compute rolling min-max boundaries and string hashes prior to BigQuery storage.

Use RunInference with a custom ModelHandler and execute min-max scaling dynamically inside the with_preprocess_fn closure based on each incoming mini-batch.

Use CombineGlobally transforms to calculate minimum, maximum, and distinct vocabulary values, export these values to Cloud Storage, and load them as side inputs inside a standard ParDo transform.

Professional Cloud Data Engineer

Scalable Feature Transformation Pipeline Mltransform

Related questions