Dr.GRPO (Group Relative Policy Optimization Done Right)

Identifies two mathematical biases in vanilla GRPO:

1. Dividing by sequence length $\frac{1}{|y_i|}$ makes the model prefer shorter correct responses and longer incorrect responses (because negative advantages are diluted over more tokens when $|y_i|$ is large!).
2. Dividing by $\text{std}(\{r_1,\dots,r_G\})$ over-weights nearly trivial or nearly impossible prompts where $\text{std}(r) \approx 0$. Dr.GRPO removes both normalizations, replacing $\frac{1}{|y_i|}$ with a constant generation budget $L_{\max}$ and using centered advantages $\hat{A}_{i,t} = r_i - \text{mean}(\{r_1,\dots,r_G\})$.

Immediately stops "overlong wrong answer" bloat during RLVR without requiring complex length penalties.