Mechanism

Iteratively rewrites seed instructions using an LLM teacher along two axes: In-Depth Evolution (adding constraints, deepening reasoning steps, concretizing, increasing complexity) and In-Breadth Evolution (mutating topic/domain while preserving structural difficulty), followed by elimination filtering of failed evolutions.

Practical Notes

Essential for pushing synthetic instruction datasets past simple single-constraint tasks into multi-constraint regimes (IFEval, IFBench, ComplexBench).

References

  1. WizardLM: Empowering large pre-trained language models to follow complex instructions ↗

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng et al. · 2023

    All 9 authors

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, Daxin Jiang

  2. WizardCoder: Empowering Code Large Language Models with Evol-Instruct ↗

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu et al. · 2023

    All 10 authors

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, Daxin Jiang

  3. WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct ↗

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao et al. · 2023

    All 11 authors

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, Yansong Tang, Dongmei Zhang