You have built a custom model that performs several memory-intensive preprocessing tasks before it makes a prediction. You deployed the model to a Vertex AI endpoint, and validated that results were…

Using a machine type with more memory addresses individual instance capacity but does not directly fix an autoscaling issue if the trigger isn't firing correctly or early enough to provision more instances.

Decreasing the number of workers per machine would likely reduce overall capacity and worsen the autoscaling problem.

Increasing the CPU utilization target would make the autoscaler less sensitive, requiring a higher CPU load before scaling out, which would exacerbate the problem if the bottleneck occurs at lower CPU utilization.

Vertex AI Endpoints autoscale based on configured metrics, typically CPU utilization. If the model is memory-intensive and experiences bottlenecks before CPU utilization becomes high, decreasing the CPU utilization target will cause the autoscaler to provision new instances sooner, even at lower CPU loads, thus improving autoscaling behavior.