Detect and prevent reward hacking

Explain reward hacking when a language model is trained with reinforcement learning: what it looks like, how you would detect it, and what you would change in the reward, the data or the training setup to prevent it.