Design a model inference pipeline
Billed as ML design but really ML infrastructure: design a pipeline that serves a trained model's predictions. Cover how requests or data arrive, batching, GPU use, model versions and rollout, latency and throughput, and monitoring.
NuroTechnical phone screenNo answer yet. Write the first one; the best answer gets endorsed.