ReST (Reinforced Self-Training: Grow & Improve)

Decouples offline data generation (Grow step: sample a large dataset of trajectories from the current policy $\pi_{\theta_g}$) from iterative offline policy optimization (Improve step: filter the grown dataset with an increasing sequence of reward thresholds $\tau_1 < \tau_2 < \dots < \tau_M$ and train for multiple offline loss steps on each thresholded subset).