Add deployment lifecycle controls and autoscaling for custom foundation models on watsonx.ai SaaS

Who would benefit from it?

Straker plans to run its own Gemma 3-based model, Tiri, as a custom foundation model on watsonx.ai SaaS for SwiftBridge AI v2, Straker's IR translation solution.

This capability would also benefit other partners and enterprise customers running custom foundation models with bursty, intermittent, or latency-sensitive inference workloads. These use cases require capacity to be available when demand increases, while avoiding continuous GPU costs during periods of low or no traffic.

Why is it useful?

A custom foundation model deployment runs at full capacity and full cost from the moment it is created until it is deleted. On the API, there is no stop, pause, suspend, scale-down, or scale-to-zero operation for these deployments. The only state-changing operations are create and delete. Billing is continuous while the deployment exists, independent of traffic.

To manage spend on intermittent workloads, Straker deletes deployments and recreates them when needed. The average time from creation to serving-ready is around 20 minutes, which makes this workaround unusable for latency-sensitive workloads and turns capacity management into a manual scheduling exercise.

In practice, this pushes bursty and intermittent workloads, which represent a large share of real enterprise inference, off the platform.

How should it work?

Custom foundation model deployments should support deployment lifecycle controls.

At minimum, the service should provide a suspend and resume state that stops GPU billing while preserving the deployment record.

Ideally, the service should support scale-to-zero with automatic wake on request, together with replica autoscaling for scale-up. Even coarse-grained scheduled on and off control would change the platform economics for partner models.