Mechanism

Based on the Superficial Alignment Hypothesis—that a model's knowledge and capabilities are learned almost entirely during pretraining, while SFT teaches the subdistribution of formats and styles for interacting with users. Uses aggressive quality, complexity, and diversity filtering (e.g., 1,000–6,000 curated examples) instead of millones of noisy pairs.

Practical Notes

Works well for conversational style and general assistant behavior, but breaks down on deep reasoning or formal proof tasks where the base model lacks complete multi-step trajectories.

References

  1. LIMA: Less Is More for Alignment ↗

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao et al. · 2023

    All 15 authors

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, Omer Levy

  2. AlpaGasus: Training A Better Alpaca with Fewer Data ↗

    Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav et al. · 2023

    All 11 authors

    Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, Hongxia Jin

  3. What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning ↗

    Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, Junxian He · 2023