What it measures
Chinese web-ecosystem counterpart to BrowseComp testing deep multi-hop search across Chinese sources.
Reading the measurement
The useful outcome is a verified change or deliverable. Inspect the tool interface, initial state, user simulator and policy constraints. An agent can claim completion without reaching the required state, or reach it through an action the protocol forbids.
Published evaluation
Answer accuracy; BrowseComp-ZH · original comparison. The table retains the source’s reported model and evaluation configuration.
Where this fits in an evaluation suite
A transcript that says “done” is weaker evidence than the verified final state. Track policy violations, recoveries, and invalid attempts alongside success.
Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.
A reproducible comparison
Answer accuracy; BrowseComp-ZH · original comparison. The table retains the source’s reported model and evaluation configuration.
Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.
PRIMARY EVIDENCE & FURTHER READING
- BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese — published result table ↗
All 16 authors
Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, Yuxin Gu, Sixin Hong, Jing Ren, Jian Chen, Chao Liu, Yining Hua
Reviewed 2026-10-26. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.