AWR / AWAC (Advantage-Weighted Regression / Actor-Critic)

Formulates policy improvement as weighted supervised learning where off-policy or replay-buffer trajectories $(x,y)$ are weighted exponentially by their estimated advantage: $\mathcal{L}_{\text{AWR}}(\theta) = -\mathbb{E}_{(x,y)\sim\mathcal{B}}\left[\exp\left(\frac{R(x,y) - V(x)}{\tau}\right) \log \pi_\theta(y\mid x)\right]$

Naturally handles off-policy replay buffers without importance-sampling ratio explosion.