GRPO in NumPy: group-relative advantages, process and shaped rewards

In a notebook, implement the core math of Group Relative Policy Optimization with NumPy. For each prompt the policy samples a group of completions, each completion gets a score, and every completion is judged against the others in its group.

1. Compute the group-relative advantages from the scores.
2. Extend it to process rewards, where intermediate steps of a completion are scored, and turn those into per-token advantages.
3. Add shaped rewards and show how they change the advantages.

Explain why GRPO needs no learned value function and what can go wrong when a group's scores are all the same.