You have deployed a scikit-team model to a Vertex AI endpoint using a custom model server. You enabled autoscaling: however, the deployed model fails to scale beyond one replica, which led to…

Attaching a GPU is for models that benefit from GPU acceleration, but the problem states CPU utilization is low, suggesting the bottleneck is not computational throughput but concurrency within the existing CPU capacity.

If CPU utilization is low but requests are dropped and scaling is stuck at one replica, the bottleneck is likely within the single model server process, meaning it can't handle more concurrent requests effectively. Increasing the number of workers *within* your custom model server allows the single replica to process more requests concurrently, effectively utilizing the available CPU and potentially resolving the bottleneck without needing more replicas.

Scheduling scaling might address predictable load, but it doesn't solve the underlying issue of why autoscaling *failed* to scale beyond one replica despite high load and low CPU utilization, which points to a bottleneck within the single replica's processing.

Increasing `minReplicaCount` forces more replicas, which might provide more capacity, but it doesn't address the root cause of why the single replica isn't fully utilized or why autoscaling isn't working as expected.