Debug a model that does worse online than offline

0
Remix

A model scored well in offline evaluation, but after launch its online metrics are worse. How do you find out why? Walk through the likely causes (training-serving skew in features, data drift, leakage in the offline set, a mismatch between offline and online metrics, feedback loops), how you would check each one, and what you would change so it does not happen again.

AmazonML designLast asked Jan 2026
No answer yet. Write the first one; the best answer gets endorsed.