Design a workflow orchestration system
Users define a workflow as steps with dependencies, then start runs of it. Design the system that runs each step once its dependencies finish, records the state of every run and step, survives worker and scheduler crashes, retries failures, and shows users progress and results. One report framed it around machine-learning data pipelines.
MercorSystem designNo answer yet. Write the first one; the best answer gets endorsed.