Design an asynchronous RL post-training system

Design a system for post-training a large language model with reinforcement learning where generation (rollouts) and training run asynchronously. Cover how rollout workers and trainers are split across GPUs, moving fresh weights to the rollout workers, how stale the policy may get and how to correct for it, reward computation, and throughput and failure handling.