MLA-C01 · Question #46
A company has AWS Glue data processing jobs that are orchestrated by an AWS Glue workflow. The AWS Glue jobs can run on a schedule or can be launched manually. The company is developing pipelines in A
The correct answer is C. Use Callback steps in SageMaker Pipelines to start the AWS Glue workflow and to stop the. Callback steps are a native SageMaker Pipelines feature built precisely for this use case: the pipeline pauses execution, sends a token to an SQS queue, triggers the AWS Glue workflow, and then resumes only after the Glue job calls back with a success/failure signal - all within
Question
A company has AWS Glue data processing jobs that are orchestrated by an AWS Glue workflow. The AWS Glue jobs can run on a schedule or can be launched manually. The company is developing pipelines in Amazon SageMaker Pipelines for ML model development. The pipelines will use the output of the AWS Glue jobs during the data processing phase of model development. An ML engineer needs to implement a solution that integrates the AWS Glue jobs with the pipelines. Which solution will meet these requirements with the LEAST operational overhead?
Options
- AUse AWS Step Functions for orchestration of the pipelines and the AWS Glue jobs.
- BUse processing steps in SageMaker Pipelines. Configure inputs that point to the Amazon
- CUse Callback steps in SageMaker Pipelines to start the AWS Glue workflow and to stop the
- DUse Amazon EventBridge to invoke the pipelines and the AWS Glue jobs in the desired order.
How the community answered
(19 responses)- A5% (1)
- B11% (2)
- C79% (15)
- D5% (1)
Explanation
Callback steps are a native SageMaker Pipelines feature built precisely for this use case: the pipeline pauses execution, sends a token to an SQS queue, triggers the AWS Glue workflow, and then resumes only after the Glue job calls back with a success/failure signal - all within a single, unified pipeline definition and with no additional orchestration infrastructure to manage.
Why the distractors fail:
- A (Step Functions): Technically capable, but introduces a third orchestration layer on top of both SageMaker Pipelines and Glue workflows, increasing setup, IAM, and maintenance overhead.
- B (Processing steps pointing to S3 output): Reads Glue output passively - it doesn't trigger or wait for the Glue workflow, so you'd have no guarantee the data is fresh or that the jobs ran at all within the pipeline's execution context.
- D (EventBridge): An event-driven pattern useful for loosely coupled decoupling, but it doesn't provide tight in-pipeline synchronization - you'd need additional logic to ensure ordering and wait for completion, adding complexity.
Memory tip: Think of Callback steps as a "hold-my-place" mechanism - the pipeline hands off a token, says "call me when you're done," and resumes exactly where it left off. Any time an exam question asks you to integrate an external workflow into a SageMaker Pipeline with minimal overhead, Callback steps are the answer.
Topics
Community Discussion
No community discussion yet for this question.