MiniLLM (Reverse-KL Knowledge Distillation)

Replaces forward KL divergence $\mathbb{D}_{\text{KL}}(\pi_{\text{teacher}}\|\pi_\theta)$ (which forces a small-capacity student to spread probability mass across all modes of the teacher, producing hallucinations in low-density voids) with Reverse KL $\mathbb{D}_{\text{KL}}(\pi_\theta\|\pi_{\text{teacher}})$ optimized via policy gradient with single-step decomposition and teacher-mixed rollout stabilization.