Mechanism

Trains an LLM via GRPO/PPO to autonomously decide when and what to query against a live search engine or vector corpus during multi-hop reasoning by emitting <search>query</search> tokens, receiving retrieved passages inside <information>...</information> tags (with retrieved tokens masked out of the policy gradient and KL loss so the model is not penalized for external text), and optimizing purely for final-answer exact match / F1 or citation-grounded rubric rewards.

References

  1. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning ↗

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang et al. · 2025

    All 8 authors

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, Jiawei Han

  2. R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning ↗

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao et al. · 2025

    All 8 authors

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, Ji-Rong Wen

  3. ZeroSearch: Incentivize the Search Capability of LLMs without Searching ↗

    Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang et al. · 2025

    All 10 authors

    Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Yan Zhang, Fei Huang, Jingren Zhou