Mechanism
Scales automated process supervision efficiently using a Divide-and-Conquer Monte Carlo Tree Search (OmegaPRM) that performs binary search over a failing trajectory to locate the exact first error step in rollouts instead of linear rollouts. Qwen2.5-Math-PRM further發現 that pure MC estimation has high false-positive rates (a wrong intermediate step can still reach the right answer by luck) and intersects MC labels with an LLM-as-a-Judge critique filter (Consensus Filtering).
References
- Improve Mathematical Reasoning in Language Models by Automated Process Supervision ↗
All 12 authors
Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, Abhinav Rastogi
- The Lessons of Developing Process Reward Models in Mathematical Reasoning ↗
All 9 authors
Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin