Design review: rolling out a new model version

You are reviewing a launch plan written by the serving team. Critique it: rank the risks, propose changes, and say what must be true before you would approve it. You have 45 minutes.

Launch plan: model v2 to all API traffic (draft)

Why. v2 beats v1 on our internal eval suite by 3 points on average. Customers have asked for it.

Plan. On Friday at 10:00 we change the latest alias from v1 to v2 for all customers in all three regions with one config push. Customers who pinned v1 are moved to v2 as well, because v1 will be deprecated.

Capacity. v2 has 1.6 times as many parameters as v1. We keep the same GPU fleet: v2's batching is more efficient, so throughput should be similar. To make room, v1 replicas are scaled to zero right after the switch. The fleet peaks at about 70% utilization today.

Caching. Prompt-cache keys are a hash of the prompt prefix, so cached prefixes keep working after the switch.

Rollback. Push the old config. v1 replicas take about 40 minutes to load weights and warm up.

Monitoring. p50 latency and the 5xx error rate, on the existing dashboard.

Safety. The safety evaluation for v2 is scheduled for the week after launch.