PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #56
You developed an ML model with AI Platform, and you want to move it to production. You serve a few thousand queries per second and are experiencing latency issues. Incoming requests are served by a…
The correct answer is D. Recompile TensorFlow Serving using the source to support CPU-specific optimizations. To improve the serving latency of an ML model on AI Platform, you can recompile TensorFlow Serving using the source to support CPU-specific optimizations and instruct GKE to choose an appropriate baseline minimum CPU platform for serving nodes, this way GKE will schedule the…
Question
Options
- ASignificantly increase the max_batch_size TensorFlow Serving parameter.
- BSwitch to the tensorflow-model-server-universal version of TensorFlow Serving.
- CSignificantly increase the max_enqueued_batches TensorFlow Serving parameter.
- DRecompile TensorFlow Serving using the source to support CPU-specific optimizations.
How the community answered
(40 responses)- A5% (2)
- B10% (4)
- C3% (1)
- D83% (33)
Explanation
To improve the serving latency of an ML model on AI Platform, you can recompile TensorFlow Serving using the source to support CPU-specific optimizations and instruct GKE to choose an appropriate baseline minimum CPU platform for serving nodes, this way GKE will schedule the pods on nodes with at least that CPU platform.
Topics
Community Discussion
No community discussion yet for this question.