nerdexam
Google

PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #244

You are building a custom image classification model and plan to use Vertex AI Pipelines to implement the end-to-end training. Your dataset consists of images that need to be preprocessed before…

The correct answer is B. DataflowPythonJobOp, WaitGcpResourcesOp, and CustomTrainingJobOp. To preprocess images with existing Python functions and then train a custom model in a Vertex AI Pipeline, use DataflowPythonJobOp for scalable preprocessing and CustomTrainingJobOp for model training.

Submitted by jordan8· Apr 18, 2026ML pipeline operationalization

Question

You are building a custom image classification model and plan to use Vertex AI Pipelines to implement the end-to-end training. Your dataset consists of images that need to be preprocessed before they can be used to train the model. The preprocessing steps include resizing the images, converting them to grayscale, and extracting features. You have already implemented some Python functions for the preprocessing tasks. Which components should you use in your pipeline?

Options

  • ADataprocSparkBatchOp and CustomTrainingJobOp
  • BDataflowPythonJobOp, WaitGcpResourcesOp, and CustomTrainingJobOp
  • Cdsl.ParallelFor, dsl.component, and CustomTrainingJobOp
  • DImageDatasetImportDataOp, dsl.component, and AutoMLImageTrainingJobRunOp

How the community answered

(18 responses)
  • A
    11% (2)
  • B
    83% (15)
  • C
    6% (1)

Why each option

To preprocess images with existing Python functions and then train a custom model in a Vertex AI Pipeline, use `DataflowPythonJobOp` for scalable preprocessing and `CustomTrainingJobOp` for model training.

ADataprocSparkBatchOp and CustomTrainingJobOp

`DataprocSparkBatchOp` would be for Spark-based processing, not directly leveraging existing Python functions in a general Dataflow context for image preprocessing.

BDataflowPythonJobOp, WaitGcpResourcesOp, and CustomTrainingJobOpCorrect

`DataflowPythonJobOp` is suitable for executing custom Python preprocessing functions at scale using Dataflow, providing a robust and managed service for data transformation. `CustomTrainingJobOp` is then used to orchestrate the custom model training job after preprocessing, and `WaitGcpResourcesOp` ensures the Dataflow job completes before training starts.

Cdsl.ParallelFor, dsl.component, and CustomTrainingJobOp

`dsl.ParallelFor` and `dsl.component` are general Kubeflow Pipelines constructs for defining pipeline steps and parallelism, but they don't specify the underlying managed service for large-scale data processing like Dataflow.

DImageDatasetImportDataOp, dsl.component, and AutoMLImageTrainingJobRunOp

`ImageDatasetImportDataOp` is for importing data, not for custom preprocessing, and `AutoMLImageTrainingJobRunOp` is for AutoML training, not custom model training as specified in the question.

Concept tested: Vertex AI Pipelines, custom preprocessing with Dataflow

Source: https://cloud.google.com/vertex-ai/docs/pipelines/run-python-code-on-dataflow-python-job-op

Topics

#Vertex AI Pipelines#Dataflow#Custom Training#Data Preprocessing

Community Discussion

No community discussion yet for this question.

Full PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice