A Generative AI Engineer has a provisioned throughput model serving endpoint as part of a RAG application and would like to monitor the serving endpoint's incoming requests and outgoing responses…

The correct answer is C. Inference Tables. Inference Tables is a Databricks Model Serving feature that automatically logs every request payload and every response payload from a serving endpoint into a Delta table. This gives the engineer a queryable, persistent record of all traffic to the endpoint, enabling monitoring…

MLOps

Question

A Generative AI Engineer has a provisioned throughput model serving endpoint as part of a RAG application and would like to monitor the serving endpoint’s incoming requests and outgoing responses. Which Databricks feature should they use?

Options

AAutoML
BVector Search
CInference Tables
DFeature Serving

How the community answered

(54 responses)

A
4% (2)
B
2% (1)
C
87% (47)
D
7% (4)

Explanation

Inference Tables is a Databricks Model Serving feature that automatically logs every request payload and every response payload from a serving endpoint into a Delta table. This gives the engineer a queryable, persistent record of all traffic to the endpoint, enabling monitoring for data drift, quality issues, latency trends, and debugging. AutoML (A) automates model training, not monitoring. Vector Search (B) handles similarity lookups for RAG retrieval. Feature Serving (D) serves pre-computed features to models. None of those three capture raw request/response data from a live serving endpoint-only Inference Tables does.

Topics

#Model Serving#Inference Logging#Monitoring#Databricks

Community Discussion

No community discussion yet for this question.

Full GENERATIVE-AI-ENGINEER-ASSOCIATE Practice